Interesting how it's changed over the years. I was a deputy returning officer a few elections ago and the official line then was that pencil was the accepted implement because it was less likely to smudge and spoil a ballot.
Of course, you'd have to make sure the pens were all the same colour, otherwise it could be used to mark ballots (I. E. I use my green pen to prove I voted for so and so). That could circumvent the secret ballot.