I think you will have the same problem as with 3D printers and accuracy. Like, people don't realize their 3D printers are precision machines that are not accurate. There is a considerable variation in the actual zero point location of the tip of the nozzle on the bed. It simply does not matter that this is the case because every location is relative to the zero point.
This is why 3D printers are cheap as a hobby tool. When you start trying to design a printer where there are two extruder heads that move independently, you need to know absolute position for each print head and that position must be accurate at all times. This is a MUCH harder problem to solve.
In your VR headset, you are floating like how a cheap printer works. Your absolute position is irrelevant. If you want to incorporate external objects, you need to know absolute position. Computationally that is a larger problem, but also it requires a much more expensive sensory system, especially if you want to triangle location based on sensors mounted to the headset where the separation distance is small. I think you would really struggle to incorporate that and keep a half decent battery life to weight balance.