Indeed, and the reason why I think the incompleteness theorems are the nail in the coffin, is that otherwise you could at least prove that certain sets of axioms worked together, then you'd have some basis for the math you use, even if it was self-referential

But it turns out that is impossible, and what we choose as our foundation for the math we use is pretty much arbitrary. Although of course we have reasons to use what we use