It's not about the complexity of the music, but the logistics of getting lots of people playing in sync. A large orchestra is large enough that there will be fractions of a second difference between the more distant players. In some cases, the players literally cannot synchronize themselves via audio cues alone.

Also, good luck getting dozens of people to come in from silence at the exact same moment without a conductor. They're pretty critical when they're needed, which is very often for orchestras.