Ok, I went and read some more about it, and you can manage keys with the kernel user session keyring. So it's possible.

It brought me back around to why systemd is so shitty.

Session Keyring (Rejected)

This strategy involves placing all keys for fscrypt in KEY_SPEC_SESSION_KEYRING. Using the current session keyring means that fscrypt will not need elevated privileges to place keys in this keyring, eliminating the need for a setuid binary. It also means that if something like pam_keyinit is used, the keys will be inherited across things like sudo.

However, this strategy has a few significant downsides that led to it not being used. The first issue is that keys unlocked in one session for a user are (sometimes) not accessible to the user in other sessions. This can create confusion for users unable to access certain directories. However, the bigger problem is that systemd is incompatible with use of the system keyring. The systemd maintainers are of the reasonable position that the session keyring just shouldn’t be used.

fscrypt

Emphasis mine. Because the user session keyring is incompatible with systemd, the Poetterites say it shouldn't be used.

The only way to handle keys securely Ok base Linux shouldn't be used because it's incompatible with systemd. What a way to see the world: so convinced in the superiority of your monolithic monster system that you argue against an immediately available way of improving security.

It's incompatible, by the way, because systemd doesn't run user jobs in the user's session, but in parallel sessions. This means that, if you use systemd, you can't use the most secure way of handling secrets with fscrypt, the kernel user session keyring.