We did had referendum for EU which was massively popular (75% voted "for") but still there was serious fear that the frequention would make it invalid. It didn't ultimately, 58% went to vote, but Poland generally have very low election participations because our political class is just like the USA, various wings of the neoliberal party. Which make this arguments of yours invalid. They didn't asked about NATO because they feared it would fail, while joining the US empire was mandatory.

And has a party fallen from that? I know the legality is dubious, but were there actual consequences from participating?

Of course not, it was uniform across all the wings of our neoliberal party, that's how you can recognize it's effectively just one party and the country is just an undemocratic vassal of USA since it can ignore its own law and nothing happens. Non govt circles pointed that out of course, only to be ignored. But the political opposition (another very telling symptom which happens all the time in Poland!) didn't say anything - the original invaders was the socialdemocratic party, but then both liberals and conservatives supported it after both parties won the election. Which is yet another interesting thing - back then every major party got to rule in very short time one after another - when the voters seeked for any alternative - and never found it since their policies are nearly the same.