I dont think LLMs will or should replace properly written dialogue.
Where they would shine is just generating inane background chatter. So instead of hearing an NPC say "i took an arrow to the knee", or "jesus christ be praised! Henry's come to see us!" 300 times, an LLM could generate some short one liners that are a bit more dynamic. It would go a long way to making the world feel more alive.
New text to speech models are incredible these days...
Again, we shouldnt replace actual voice actors for main dialog. But for generating thousands of lines of background chatter (which nobody would have time or resources to make anyway) LLM writing paired with text to speech could really help flesh out a living game world