Close, but not quite. One cubic meter of air at atmospheric conditions is about 40 moles, or 575 grams. Let's say 600 g to be generous.

Wind moving at 250 km/h (cat. 5 hurricane) contains about 1.5 kJ of kinetic energy per cubic meter.

If all that energy is used to heat the same air, it's temperature is increased by roughly... 3.5 K.

Now consider the fact that kinetic energy scales as the square of velocity: for a normal windy (say 15 m/s wind), we only get a temperature increase of 0.16 K, which is practically immeasurable.

For these calculations I didn't even consider the building, which has a massive specific heat capacity compared to air.

In summary: Because the thermal energy required to increase temperatures appreciably is of a completely different order of magnitude than the amount of kinetic energy in the wind, the fact that a building isn't heating up is not a solid argument that only a small portion of the kinetic energy in the wind is being lost when it hits the building. In fact, even if all the energy was lost, you wouldn't even notice the temperature change, unless you were in a cat. 5 hurricane, in which case the building is probably already gone.