In the short term, I'd stay in the cave, and I'd keep it for emergency backup accomodations like in a hurricane or something. In the long term, I'd live outside of the cave, once I got a decent shelter set up.

Why? Well, if the weather is nice, there's no reason not to live outside. And if it's not nice (but not a hurricane), or I'm cooking food, I'm likely to want a fire. And I remember enough from my high school geology classes that having repeated fires inside some kinds of caves can make rocks fall from the ceiling. And I don't remember enough from my high school geology classes to know whether that cave has fire-safe rocks or not.

I acknowledge that fuel for the fire is likely to be an issue, so I'd try to get some aluminum or something from the plane, or flatten out some tin cans if any have washed ashore, or something similar. Then use the metal to make a solar stove to cook with. The stove would need some monitoring, though, to adjust it to the angle of the sun and to do cook-tasks on - occasional stirring or turning over or taking it out before it burns - so even with a solar stove, I'd still be mostly outside.

So in short: if I'm making a fire or cooking, I'm outside. If the weather is pleasant, I'm outside. If there's a hurricane or something, I'll stay in the cave without a fire.