Spatially faithful telepresence supporting varying geometries and moving users

Systems and methods described herein are provided for managing user positions in a shared virtual geometry, capturing 360 degrees views of a local user environment in a spatially faithful system, and computing a perspective view of a first user from the eye-point of a second user. Users may move around the shared virtual geometry, and perspective views of users may be updated. Some embodiments use a co-centric geometry for group conferencing with other users. Some embodiments may use a grid-based geometry for exploration with other users. Some embodiments may reduce transferred bitrates to enable systems and methods to be implemented in data-limited environments. Some embodiments support users moving and navigating a shared virtual geometry.