Signal is a centralized app, run by a company. If they are offered enough money or legal threat they will sell out or close.
I am sure people will make an argument that its FOSS and people will just fork it if it goes bad, but a new fork will have 0 users and Signal will still have all of your old contacts. Why not make a switch now? Before it is even more popular and you have more reasons to stay? Why fork it if there are already decentralized apps that use same encryption, like XMPP apps?
Sure you can find flaws in every app, including XMPP implementations, but if we will have to write code for a new Signal fork, why not just fix whatever is that bugs you in XMPP clients?
If you want to use Matrix, that is fine as well, we can always bridge the two open protocols. But you cant bridge Signal if their company doesn't allow it.
Why would you need to trust their choice? The only data that is sent from your server to theirs is your username (called JID in xmpp terms) and E2E encrypted message. The worst thing their server can do to yours is to send you a message, if your server decides to pass it on.
Wouldn't you have to trust that they're not logging your IP and/or storing your messages?
XMPP clients support end to end encryption, so the servers only get encrypted messages. Also unlike Signal, XMPP clients support use of Tor to hide your IP.