I think you are overcomplicating and undercomplicating things at once.
Proper VPNs will allow split tunneling — only the traffic that needs to go through the VPN will go through the VPN.
So, the solution would be:
- Set up a VPN capable of split tunneling — vanilla Wireguard and tailscale should work
- Set up split horizon DNS so that you get are pointed to the internal/VPN-facing IP address of your server while connected to the VPN *???
- Profit
I am not sure I understand your requirements. What functionality of Filebrowser do you want to see? Read only or or read-write?
Firefox has (had?) FTP support, you could just run an FTP server and connect from your normal browser.
You could use autoindex built into your web server. That'll be the most lightweight option possible.
If these are not what you are looking for, we need details. Lots more details.