If you don't have any actual background in music - I certainly don't - I recommend using the TheoryTab portion of Hooktheory's website. It puts it in a format almost exactly identical to the Bytebeat interface. Some songs don't translate well, but others translate very well.

I can't help you with the Waveform Modulator, though. That one, I just mess around with it until it sounds good... although come to think of it, I know a few actual musicians who take the same approach lol.

I was able to make a really good version of "This is Halloween" from Nightmare Before Christmas. However, I tried making "Happy X-Mas (War is Over)" by John Lennon and it didn't translate very well because the Bytebeat really can't do the jingle-bell noise at all.

The other thing you can use it for is logic. For example, creating a series of scrolling lights (maybe for Christmas lights or an 8bit light display, idk) is much easier when you use a Bytebeat and Bytebeat Switches rather than trying to create logic circuits to accomplish everything.