There is a setting in Edge to stop it running in the background after it's closed (it shouldn't do that in the first place, but this is at least useful because if you don't turn off the web links in your Start menu search results, Edge can be triggered to open by accident from there and then continue to run in the BG after you close it).

Still on Windows 10 and I haven't noticed Edge running in the background on startup (and obviously I have it set not to do that in Windows). I'm guessing though that it's possible it might always be on if you use Cortana? I always have Cortana off too.