Config files for programs are in hidden folders in ~ (as you discovered) OR in ~/.config OR in ~/.local/share (yeah it's a bit of a mess)
Config files for flatpaks can be found under ~/.var/app (usually, some flatpaks have permissions to write outside that directory).
For backing up my home folder, I just plug in my external drive and use rsync from the terminal, like so (change to your user name; mine is gecko. Double check the paths and edit as necessary)
rsync -av /home/gecko/ /media/gecko/media/emmahomebackup/ --delete --dry-run
(test first, check for errors at the end of output, there shouldn't be any unless you messed up the paths)then,
rsync -av /home/gecko/ /media/gecko/media/emmahomebackup/ --delete
This command will back up your home folder the first time you run it, and on subsequent runs will sync any changes that have occured since the last run.
To restore, reverse the paths.
For backing up my system files (basically, everything outside of my home folder) I just use Timeshift.