Get yourself a bit holder with a guide sleeve (https://www.dewalt.com/product/dwairdg/ir-drive-guide-w-2-impact-ready-bits). Works for any type of screw since you load it with whatever bit type you need for your screw (in your case, torx). If you've never seen one, you'll love how easy it makes driving screws when they wanna fall off the bit/work.

Only real drawback is you don't have precise control of the angle the screw starts at but it's close enough for most work.