Firstly, I don't think your contributions to society are limited to work. Spending time with hobbies also contributes, e.g. art, volunteering, or open source coding. You can also work a more enjoyable "job" in retirement, without feeling strangled by the paycheck.
We also already have enough production in society for everyone to be happy, and many people's jobs do not actually contribute to society in a meaningful way, e.g. people who work at advertising firms. I think we should target the billionaires sucking up literally all the resources and seeing if "early" retirement is actually a problem at that point. People are working until they're 70 right now, but maybe everyone could retire closer to 50 if we distributed our resources more fairly.