Is it normal for borg to nag for my password so many times?

80a8f8fc1129:/# borgmatic list
local: Listing archives
Enter passphrase for key /mnt/borg-repository:
Enter passphrase for key /mnt/borg-repository:
80a8f8fc1129-2023-11-08T14:54:25.535734 Wed, 2023-11-08 14:54:28 [5a6245718dcb7e7dfec023cf0a62f568a9714b0f27d7a422e97b44870e9ecfbd]
80a8f8fc1129-2023-11-08T14:55:06.056089 Wed, 2023-11-08 14:55:08 [bce2ae9ff5b58212993281c938eb6812fe706bb1b62b80e40a32d38179f3b86c]
80a8f8fc1129-2023-11-08T14:55:24.704310 Wed, 2023-11-08 14:55:28 [7793ab62dac221d3b90434c19e0c44f917c9caedbc6250b71c86c78327348539]
80a8f8fc1129-2023-11-08T15:01:25.862051 Wed, 2023-11-08 15:01:28 [b1c83dc0ccd3d04ddfcb7d39358e8978f93a705b792105ef9c0389faf6d6e387]
80a8f8fc1129-2023-11-08T15:02:25.774879 Wed, 2023-11-08 15:02:28 [fe74cdf86dceea5a21d5305e3314733adf31d5cfe3d63e934381cec2176e59c2]
80a8f8fc1129-2023-11-08T15:02:31.879678 Wed, 2023-11-08 15:02:38 [9a2189207aa475ac866a4f7aa62d6d3c83ae53bd008dd4240de2b4277554620e]
local: Listing archives
Enter passphrase for key /mnt/borg-repository:
Enter passphrase for key /mnt/borg-repository:
80a8f8fc1129-2023-11-08T14:54:25.535734 Wed, 2023-11-08 14:54:28 [5a6245718dcb7e7dfec023cf0a62f568a9714b0f27d7a422e97b44870e9ecfbd]
80a8f8fc1129-2023-11-08T14:55:06.056089 Wed, 2023-11-08 14:55:08 [bce2ae9ff5b58212993281c938eb6812fe706bb1b62b80e40a32d38179f3b86c]
80a8f8fc1129-2023-11-08T14:55:24.704310 Wed, 2023-11-08 14:55:28 [7793ab62dac221d3b90434c19e0c44f917c9caedbc6250b71c86c78327348539]
80a8f8fc1129-2023-11-08T15:01:25.862051 Wed, 2023-11-08 15:01:28 [b1c83dc0ccd3d04ddfcb7d39358e8978f93a705b792105ef9c0389faf6d6e387]
80a8f8fc1129-2023-11-08T15:02:25.774879 Wed, 2023-11-08 15:02:28 [fe74cdf86dceea5a21d5305e3314733adf31d5cfe3d63e934381cec2176e59c2]
80a8f8fc1129-2023-11-08T15:02:31.879678 Wed, 2023-11-08 15:02:38 [9a2189207aa475ac866a4f7aa62d6d3c83ae53bd008dd4240de2b4277554620e]
80a8f8fc1129:/#

Context: I have 2 config files, as I would prefer to have each app I want backed up in a separate config if reasonable. I don't have a remote setup yet - it's just using a local repository.

Could you mount a blank directory into docker, then use borg to mount the backups into that?

Makes sense - compromises like you say.

So in the context of unraid, do I just run the borgmatic container on both ends? Or should there be a specific 'server' one?

EDIT: I found borgserver. I'm pretty sure that's the correct one to use.

Why not?