The problem you run into there is that the hypothesis is non-falsifiable - it sounds reasonable, but there's no way to test it, so you get what evolutionary biologists call a "just so story."

The other issue, and what makes it so appealing to "scientific racists," is that it becomes an exercise in justifying social stereotypes, and you get papers like "the reason the women are worse at math is because they didn't need to count mammoths like the men did."

The thing about learned behaviors is that they can be heritable and subject to evolutionary forces in epigenetic ways; if I avoid eating pork because there's a cultural taboo and I'm in an area where trichinosis is prevalent, I have a better chance to survive and produce lots of offspring than my pork-eating neighbors. No need to involve any hypothetical baked-in genetic tendencies to avoid pork. Dawkins talked about it a lot before he went off the Cultural Christianity deep end.