Yeah, it's really hard for the human brain to intuitively grasp exponential growth. Anyone who says a galaxy is "too big" hasn't actually run the numbers on that.

FTL is impossible so each species is stuck in their own solar system.

FTL is in no way necessary to allow for interstellar colonization to proceed.