If only one side of the switch/points remain, depending on the type of crossing and condition of the wheels, there's a chance that the trolley's right side wheels can jump over the switch and continue straight forward, even as the switch is set to diverge onto the non-existent siding.

Or it could derail but continue barreling forward anyway. But trolleys don't tend to be going that fast.