The memory is generally done by something called a capacitor (though there are more techniques) which just can hold a little electrical charge - roughly having an electrical charge means it's a 1, otherwise it's a 0.

Get 8 of those things and you have a byte.

It's generally easier to think of it as water: electrical lines are tubes moving water, capacitors are little containers that can have water (meaning bit = 1) or be empty (bit = 0), transistors are one-way water valves which are controlled by water (imagine they have a pressure button that opens the gate if there is water running in the tube passimg by that button, putting pressure on it).

From this simple basis you can actually create a lot of complexity by having a LOT of these things combined in weird ways.

Further, there's also a lot of complexity due to the Physics of the real world being less than perfect (for example, the "capacitors" leak water, so not only do you have to say that bit=1 is "water above a certain level" rathe than "full" - since as soon as it's filled it starts losing water - but you even have to check them once in a while and top up the ones which are supposed to be full before so kuch water leaks that the level has fallen below that treated as a "1" - this is actually how DRAM memory works, though with electric charge rather than water).