We actually have evidence of some of the earliest bridges using wood. It's just that bridge-building, even with just wood, is a massive undertaking. Pile bridges took a while to come about, and even then, it was the use of the arch and corbeled arch which made bridges to pass large rivers practical. Otherwise you're effectively limited to one span from bank-to-bank - ie you can only cross a river as wide as the shortest log used in the construction.