If it's any consolation at all the first priority of every general contractor (the company in charge of the site as a whole) is being able to get more jobs in the future. A bad reputation is the scariest thing imaginable. If you are in the horrid position of being harassed or worse by someone on a marked construction site you will absolutely be able to cause an absolute shit storm for the company if you make a complaint.

I doubt that is any consolation but the culture, at least right now, is to keep all the toxicity behind the walls/fences so nobody gets in trouble