Do you have much crowd control in your party? Can you disable either him or his minions? Pay attention to what he is vulnerable to and what makes him vulnerable when he goes into that other state. Remember that you can inspect any enemy and see their buffs and what type of enemy they are. Perhaps you have special arrows strong against that type. Perhaps you have some spells that do a type of damage that works well against him.