Sure, it can serve files up to players that can decode them. You're going to be absolutely unable to do any transcoding at all and if you try to serve up anything with a bitrate higher than the network adapter can handle you're gonna have problems. I bailed on using a Pi4 as a jellyfin server and got a chepo N-100 based box off Amazon (BeeLink something something with 2 NICs) for under $250 and haven't looked back.
You might be fine if you're sticking to small files that are handled natively by their players. It only costs your time to try it out.