Gandalf arrived in Middle Earth in the year 1000 of the Third Age. So he's been around in that form for about 2000 years since Bilbo's farewell party took place in the 3001st year of the Third Age. The Fellowship was founded in 3018 so Gandalf is about that old if we're only counting his time in Middle Earth like you say

Edit: This is in part why people are upset with the Rings of Power show. If this not-Gandalf character ends up being Gandalf, he'll have arrived in Middle Earth about 3000 years too early. On top of that there are just a lot of other timeline deviations, including the order of the forging of the rings which you would imagine the whole show to be about.