I get rather irritated with those arguments because they only return to the start. "Here, a world". "Is it how we experience it, though, and why and how; if not, what's behind?". "Bullshit, a world". That's hardly an answer. And, personally, it feels intellectually dishonest because the question was larger than just "is there a world?".

I prefer an answer like saying that doubting the world in any form might be a mistake on its own because [reasons]. I do not agree, but at least there's explanations and communication.

Also, I think they are fighting a straw man. For instance, I doubt many things about the Universe, our knowledge, our minds, etc. Yet, I accept there are phenomena which appear to me. This has been the case since the ancient school of skepticism, and I have yet to meet a person which declares themself a skeptic and does not do this to some degree. For example, I know I'm hungry right now. I don't know if the pain is real in any other deeper level, or if it is like the pain in a dream that goes away when one wakes up, or a delusion that is felt without external stimuli, or whatever. I don't know the nature of it, yet it is an experience I must attend. I can even add that the mechanisms behind, the anatomical knowledge and such is useful, but it might be entirely wrong or be as illusory as the pain itself. The straw man is that skeptics would say: "I don't know if I'm really feeling hungry", "I don't know if I want to eat" or something like that.

Why does it matter, then? Because it changes everything. In my case, it made me go from a realist teenager to an instrumentalist adult in science. From an atheist teenager to an agnostic adult.

The discussion derives in many interesting branches too. The mere "does it matter if the world is different from what we perceive if we cannot perceive it in any other way?" is an example. Many people answer yes or no without justifying it. And, at this point, some people might be wondering why we need to justify every single belief we hold and every single thing we say, like the ones throughout my comment, and that in itself is a new good question that emerges. The possibility of having any of these conversations is also a good question, and so on...

So philosophy is not going too far, in my opinion. Some philosophers might go too far, but I really think they are rare (or misunderstood).