Actually a lot of organic farms rely on blood and bone meal, manure and fish emulsion fertilizers. They're inexpensive as they're byproducts of other industries and are very good for plants.
When I worked in an organic greenhouse I often wondered about how vegans would feel about farmers using animal based fertilizers. We definitely told people what we used, as we sold those products, but no one ever said anything about it. I guess vegans can't control that so maybe it's a nonissue unless they grow their own food and use seaweed based fertilizer(more expensive) instead?