Tiering is a fundamental concept in HPC. We tier everything starting from registers, over L1-L2 Cache, Numa-shared L3, memory, SSD Cache. It makes only sense to add HDD to the list as long as it's cost effective.

There is nothing wrong with tiering in HPC. In fact, it's the best way to make your service cost effective while not compromising on end-user performance.