Tiering is a fundamental concept in HPC. We tier everything starting from registers, over L1-L2 Cache, Numa-shared L3, memory, SSD Cache. It makes only sense to add HDD to the list as long as it's cost effective.

There is nothing wrong with tiering in HPC. In fact, it's the best way to make your service cost effective while not compromising on end-user performance.

I'm looking forward to when SSDs aren't that much more expensive than HDDs for the same capacity. Seems HDDs have been holding their own better than I thought as of late.