trees near buildings are a nightmare, they can often grow weirdly due to lack of sun, and often make maintenance and clean up on the tree itself a nightmare, if it grows to close to your house you need to do something about it. They are also generally liabilities during storms, especially if they hang directly over your house.

Like a previous commenter said, they can be problematic for foundations and driveways and things like that. It seems fairly common that surface level root structures will expose themselves and start to pop through the top layer of dirt, primarily due to soil erosion and compaction i imagine, but that's another problem for grounds keeping as well.

Speaking of grounds keeping, trees make grass grow really inconsistently, and also generally provide "dead spots" where the grass will get almost no sun, and almost certainly die. Also mowing under them is hard. Trees don't really grow at human accessible heights all that often. And when they do, they're not as good for providing shade.