I'm not going to claim to be a privacy/security expert here, but I think the question is "safe from who/what?" Privacy is a sliding scale with diminishing returns, so you need to set your expectations according to the level of effort you are willing to put in.

If you just want to not be tracked/profiled by commercial big data or standard governmental operations, then an old phone with no sim, wifi/bluetooth turned off, and using wired headphones removes you from a couple tiers of lowest-hanging fruit.

But if you think someone is specifically and actively tracking you in particular, that is a whole other question.

Airplane mode might be enough to shut off the radios in a phone that old. Also the 3G network is basically shut down now. So it's a matter of just how careful you want to be. You might be able to use some kind of field strength meter to tell if the phone is still sending any radio signals.