Yeah, even adding a default blank GUI, greatly increases memory usage.

I guess that's another reason to prefer CLI programs. They reduce the system resource you need to have free at any point of time to do whatever you need them to do.

In a condition when my computer is heavily swapping, I'd go with htop rather than plasma-systemmonitor