Glad you're so appreciative and worked through it! I gladly share, discuss, and respond.

Thank you for being you!

I'll have to read up on palette filters. :) I do semi-regularly use ffmpeg, but palette filters are not something I have heard or used before.

Please allow me to point you towards the relevant parts within its documentation; palettegen and paletteuse.

Together, they constitute -from what I can gather- the absolute minimal required to create a .gif with desirable qualities. As such, they will make their appearances within the following two commands that closely mirror the examples found in the documentation:

ffmpeg -i input.mp4 -vf palettegen palette.png

This generates a representative palette with 255 colors maximum from the video. Note that AFAIU the set of colors this can draw from is the same as the one used for gifs. Which will likely come into play when we try to understand why this works in the first place.

ffmpeg -i input.mp4 -i palette.png -lavfi paletteuse output.gif

This starts with converting the colors found in the original .mp4 to their closest counterparts found within the palette. Then, with converted colors, it's turned into a .gif. Note that AFAIU we've effectively eliminated the algorithm that would otherwise kick in to convert the .mp4's wide arrange of colors into the ones compatible with gif.

To be clear, I don't claim to understand why this actually works 😅. But, combined, the above two commands do yield desirable gifs. Like, for example, the one found below.

Note that we can achieve the same with just a single command. For that, consider the command found below.

ffmpeg -i input.mp4 -vf "split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" output.gif

I assume in this case it's a downsampling into fewer colors, evading the issues of almost-same-colors?

That would also be my conjecture.

Especially given the last square/check pattern makes me thing of codecs splitting into square blocks and then encoding those. It could make sense that this division leads to different results for one reason or another, which then produces a check pattern without it being there before.

Makes sense.