Silver has been considered effective against all kinds of supernatural threats by different cultures in different ways. I would imagine the Witcher's lore was influenced by one of these traditions. As the link indicates, it's a pretty common trope.