They were, but the context there is that the alliance was a pretty direct result of the abject failure of both the Anglo-French Supreme War Council and the League of Nations to do literally anything meaningful in the face of the Soviet invasion in the Winter War.

Germany actually did quietly supply them with war materiel during the war, until some Swedish journalists broke the story, after which Germany prioritized Soviet relations due to the Molotov-Ribbentrop Pact. That was one of the main factors that led to the eventual reliance: the Soviets could clearly not be trusted (they invaded); the League of Nations was obviously powerless; the Anglo-French alliance - ostensible allies to Finland - evidently couldn’t find their ass with both hands and completely failed to respond for the entire span of the war; meanwhile, the Germans actually shipped equipment, though they geopolitically had their hands tied with treaty obligations (that they weren’t openly violating at the time).

To be clear, this is not Nazi apologia. This is an explanation of the geopolitical context at the time, and what motivated the eventual alliance between Finland and Nazi Germany, in terms of Finnish motivations: pure pragmatism in the face of an obviously expansionist neighbor with a HUGE military.

Also, yes, as u/grissino said, the Nazis did enjoy burning shit when they had to retreat, regardless of whether it was the territory of an allied nation.

That didn't stop the nazis from doing a little scorched earth behind them (while evacuating Finland and on their way to Norway, through Lapland).