In most of the popular hiking spots in my state, once you go a mile down a trail there's like nobody around.

IMO it's more about duration than mileage. You can't call it a hike if it's under an hour, that's a walk. I'm of the opinion that if you can do it comfortably without drinking water it's probably a walk. That said, three miles can definitely be a hike, there are plenty of trails in the northeast US that will give you >2000 feet of elevation gain in that distance.