This is my jellyfin nginx setup: https://wiki.gardiol.org/doku.php?id=services:jellyfin#reverse-proxy_configuration
currently i don't use any proxy related authentication because i need to find the time to work with the plugins in Jellyfin. I don't have any chromecast, but i do regularly use the Android Jellyfin app just fine.
I expect, using the OIDC plugin in jellyfin, that Jellyfin will still manage the login via Authelia itself, so i do not expect much changes in NGINX config (except, maybe, adding the endpoints).
Well, here is the relevant part then, sorry if it was not clear:
TLDR: proxy auth doesnt work with Jellyfin, OIDC yes and it bypassess proxy, so in both cases proxy will not be involved.