Yep, there's an alternative, i.e. equivalent mathematical formulation that does everything complex numbers do: the geometric algebra introduced in the article I posted earlier.

The fundamental object of GA is the "multivector", which is essentially a sum of scalars, vectors, bivectors and higher grade elements. For instance, you could take the unit x-vector and add it onto some number, say 2, to get the multivector M = 2 + e_x. (To be precise, the space of multivectors is the direct sum over the n-th wedge of the base vector space, n = 0 to dim V).
Another important concept is k-vectors, which are essentially k-dimensional volume elements. For instance, a bivector is an area with a direction, and a trivector is a volume with a direction (in 3D there is only one possible "direction" for the volume, but in 4D spacetime volumes itself can be oriented like surfaces can be in 3D).

Then, you introduce the "geometric product" for two vectors a and b:
ab = a·b + a ∧ b
where a · b is the normal scalar product between the two vectors, and a ∧ b is the wedge product between them. The wedge product essentially is the plane spanned by the two vectors, and is antisymmetric (a ∧ b = - b ∧ a, because the orientation of the plane is reversed when exchanging the vector). For instance, the unit bivector in the x-y plane is given by
B_xy = e_x e_y = e_x ∧ e_y
Notice how the scalar product part of the geometric product is zero, and only the wedge (i.e. bivector part) remains

In 3D, there are four types ("grades") of objects: scalars, vectors, bivectors (also known as 3D pseudovectors) and trivectors (or also known as 3D pseudoscalars). It's already a very rich subject and has many advantages over classical vector calculus, but for replacing complex numbers, we're mainly concerned with the 2D case.

In the 2D case, there are three types of objects: Scalars, 2D vectors, and bivectors/2D pseudoscalars. There is only one possible orientation for a 2D plane in 2D, so we just denote a bivector with area A as B = A I, where I = e_x ∧ e_y is the only unit bivector/2D pseudoscalar.

A nice thing we notice about the I is that it squares to -1 with the geometric product:
I^2 = (e_x ∧ e_y)^2 = (e_x e_y)^2 = e_x e_y e_x e_y = - e_x e_y e_y e_x = -e_x e_x = -1
The first step works because the scalar product part between e_x and e_y is zero. The second step is just writing out the square. The third step is e_y e_x = e_y ∧ e_x = - e_x ∧ e_y = -e_x e_y, which again works because e_x · e_y = 0. We see that the 2d pseudoscalar I behaves just like the "classic" imaginary unit i.

Because the geometric product is associative, and commutative if only scalars and bivectors are involved, the geometric notion of scalars and 2D pseudoscalars can fully replace the notion of complex numbers by making the substitution a + bi -> a + bI.

If you want to learn more about GA, I can recommend Doran, Lasenby: Geometric Algebra for Physicsists :)