Somehow your comment garnered a singular downvote, presumably from someone who didn't remember that entry level TV's used to not come with composite (red/white/yellow) inputs on them. You'd have antenna screws, or if your TV was "cable ready" it would have a screw-on type F jack and a tuner capable of hitting those higher channels.

If your console could only output composite, or if you lost the RF modulator it came with, a lot of VCR's did indeed come with RCA composite jacks on them, sometimes even on the front! This was actually intended to allow goobers like your dad to connect a camcorder to them, but as you observed it'd just as well allow you to plug in your Nintendo 64 and would dutifully pass along the video to channel 3 or whatever.