Forecasting pedestrian behavior remains an open problem in robotics, partly due to emergent physical dynamics resembling complex systems. We argue that social dynamics also play a critical role in pedestrian navigation. People interacting (e.g. conversing) on a path impose variable proxemic norms on pedestrians – tacit social rules that regulate the use of physical space. In 4 field experiments with 4,911 participants, we show that pedestrians are acutely aware of others’ social interactions in that they create an invisible wall between interactants that pedestrians try to avoid breaching. Breaching depends on how interactants display involvement in social interactions (body orientation, gaze, and proximity) and whether preceding pedestrians had breached. These results demonstrate how physical mobility in public places depends on pedestrians’ social computations.