Does Guix fit your criteria, perhaps? If you haven't heard of it, you can think of it as Nix with a Lisp frontend.

I unfortunately am not very experienced with containerizing packages so I can't say much, but I know you can do it; the Nonguix channel employs containers for some proprietary software.

Like Nix, Guix has all that building-from-source stuff you'd want from Gentoo. There's recently been work on making parameterized packages (the Guix equivalent of USE flags) a thing, but it's still work-in-progress.

Ignoring the steep upfront cost of learning it, I'd say Guix makes it incredibly easy to add your own packages. Here's the custom packages I currently have in my dotfiles repository. I can import one to my main config file, add the package, and it gets included in my environment the next time I reconfigure it.

As for patches, I can't make any comparisons since I'm not familiar with Gentoo, so I think a code snippet is probably better for you to judge if you'd like it.

(define-public custom-pkg
  (package
    (inherit pkg)
    (name "custom-pkg")
    (version (package-version pkg))
    (source (origin
              (inherit (package-source pkg))
              (patches
               (list (string-append (dirname (current-filename))
                                    "/fix-some-thing.patch")))))))

(define-public custom-pkg
  (package-with-patches
   pkg
   (list (string-append (dirname (current-filename))
                        "/fix-some-thing.patch"))))

Not sure if this addresses your concern about multi-architecture support, but the Foreign Architectures section of the manual discusses what you can build to.

EDIT: So I was curious after posting this because usually the CLI often has much less verbose options (like --with-input for replacing inputs), and I started wondering if there was any procedure that would make this simpler. Turns out there is :) I've included it under the example. Although, I suppose I should have mentioned you could write your own if you really wanted to.