There are a few options there.

As someone else mentioned if you're using IPv6 then it doesn't matter, you're already routing internally even if you're using the public DNS name, no extra work required.

All the rest are for IPv4.

If you're not behind CGNAT some routers/gateways are also smart enough with their routing to recognise when they need to route back to their own external IP and will loop back locally instead of making any hops out to the internet. Again, if this is the case for you then no additional work is required other than perhaps running a traceroute to confirm.

Another option is to add a local DNS entry for the name you're using to resolve to a local IP address instead of your public address. The complexity (or even possibility) of this is going to vary considerably with your setup. If you're running your own local DNS e.g. pihole or similar then it's trivial. This is how mine is set up.

If all your clients are going to be on PCs (or devices you have more than the typical manufacturer allowed modicum of control over) then you can do something kind of like the previous, just with all your local hosts files.

If none of the above are options, then you'll unfortunately have to fall back on using a local name/address, which means a slightly different client setup for devices you use exclusively in your home versus ones you might use elsewhere.

If you set up ipv6 on the jellyfin server, you’ll be connected locally to your server because with ipv6 even if you use a domain name, if the server is in your lan it’ll connect locally to it.