So all our evidence about life is from a sample of 1: 1 planet and it's development. Everything else is extrapolated from that. We don't know what rates of evolution should be or could be.

Also in terms of civilizations leaving junk everywhere, that is potentially true. But we also only have a sample of 1: 1 planet and 1 solar system which we have barely scratched the surface of.

The absence of evidence is not evidence in itself. We will have to go out in to the universe to see what is there.

In terms of travel to other systems - in theory self replicating ships could spread across the galaxy to every system in about 500 thousand years at sublight speed. Space travel is not doable in a humans life time, but it is doable on the scale of a stable civilizations efforts to spread into the galaxy.

There are also theoretical ways to travel faster than life. Whether they are pure fantasy or potential science only time will tell. We still can't even detect much of the universe, let alone begin to manipulate it.

We simply know too little to know what is going on in the galaxy. To say "there is nobody out there" is just a possibility, not a certainty.