While somewhat niche, this seems like a great idea (for me it needs gyro or it's a deal breaker though).
I already use a steam deck and xreal glasses to play, so it'd be great to remove some dead weight (mostly streaming 120fps 1080p from my desktop computer via moonlight/sunshine)
This doesn't contradict what the OP said. ChatGPT is now an interface to both an LLM and a diffusion-based image generator.