At this scale, the stick isn't as solid as your intuition would lead you to believe. Instead, you have to start thinking about the force at the atomic scale. The atoms in your hand have an outer shell of electrons which you use to impart a force to the electrons in the outer atoms of the stick on your end. That force needs to be transferred atom to atom inside the stick, much like a Newton's Cradle. Importantly, this transfer is not instantaneous, each "bump" takes time to propagate down the stick and will do so slower than the speed of light in a vacuum. It's basically a shockwave traveling down the length of the stick. The end result is that the light will get to the person on the other end before the sequence of sub-atomic bumps has the chance to get there.