Look at a piece of paper. Mark two points on it that are a good distance apart. Travel can only happen along the surface of the paper. When it's flat, your time of travel depends on the distance between the two points on that paper.

Now dap a spot of glue next to one of the points and fold the paper in such a way that the other point comes to rest close to that glue. Wait for it to bind and then spread the paper a little without breaking the glue. That glued point is a wormhole, a place where two points of that flat 2D universe touch despite not being next to each other. Travel from point A to point B is now a shorter distance thanks to the wormhole. But there is no way in which the paper universe can be described as flat anymore.

Or think of a papermache ball, that's also made from paper but if you travel long enough in one direction, you'll end up where you started. Because it isn't flat.

Now our universe is 3D not 2D, but from a higher dimensional perspective it has the same prooerties of flatness as that 2D paper has for us.