Oh, totally. We legitimately know tons of stuff about brains, including what most parts are typically used for. We just can't tell much based on eyeball lookin', we've gotta get in there with FMRI and EEGs.

I'm purely referring to head/brain size or gross physical properties and how they relate to intelligence.

Show a neurologist a photograph of a random brain and ask them to assess the owners intelligence.
The only thing they really have to go on is that if you can photograph someone's brain, signs aren't typically looking great for them.
They'll be making guesses based on vague correlations, and also getting fidgety about what you even mean by intelligence, since that's also not a simple measure.