It's not just the sexual aspect that makes people uncomfortable.

Many people view it as childish. Children are really into their stuffed animals and love playing dress up. There is nothing inherently wrong with enjoying activities normally associated with children but other adults tend to look down on it.

Some furries like to talk about their fursona as a spiritual extension of themselves. Many people associate that kind of language with crazy old hippies.