So, I'm not alone... Thank you!

It's worth noting that a number of languages comparable C in use case and performance (including its predecessors COBOL, Fortran and ALGOL) start indexing at 1 just fine because they have proper array types and don't make heavy use of pointers poking and peeking wherever they like (eg by using references instead).

Decoupling indices from memory offsets doesn't get in the way of performance and actually often allows better optimization because the compiler knows you aren't sharing pointers between arrays or some other shenanigans; see Fortran, the GOAT of fast array processing. This also allows improved type safety and thus memory safety; see index types in Ada/SPARK and the fact that it's the only 'legacy' language that's gotten comprehensive compile-time memory safety analysis (though the latter is moreso from the less willy-nilly pointers in general)

Almost every programming language uses 0.

When you say the first element of a matrix, first implies one and not zero. This is how linear algebra was invented (on paper, by a human mathematician), taught, and passed down to fellow humans.

Starting indexes at zero stem from the lineage of C programming and binary nature of computer. For example,

Computer memory addresses have 2^N cells addressed by N bits. Now if we start counting at 1, 2^N cells would need N+1 address lines. The extra-bit is needed to access exactly 1 address. (1000 in the above case.). Another way to solve it would be to leave the last address inaccessible, and use N address lines.

This is why, math and physics people who learn linear algebra and matrix calculus learn to index at 1 (on a piece of paper) while computer science programmers index at 0.

It's from when arrays were just a block of memory and the index was the offset. So you'd start at pointer x and read memory from there. x + i was your memory location. So you'd start at x + 0 to read your first data element. x + 1 would be the location in memory of the second element.