I think the pool can be quite large. The main issue is that it is almost always more convenient to drive than bike right now.
That isn't because cars are inherently more convenient, its because we've been making them more convenient for a decade. Some examples of this.
-
Parking: look at the massive land use that we dedicate to storing vehicles, and then give away for free. Obviously it's not free to whoever builds and maintains them, so that cost is just passed on to everyone to increase the convince of cars.
-
Traffic lights: cars and trains are the only vehicles that require lighted signals. Everyone else at an intersection gets slowed down so that cars can drive faster in between them. In my city of Kingston, pretty much every major road is 60kph+, but my average driving travel speed is always <35kph. Lights even inconvenience drivers, they're just trained to accept it as normal, and are rewarded with faster travel between lights.
-
Lanes: cars get a travel lane everywhere. Pedestrians usually get one. Bikes get fucked.
You can see this in action where cities that increase bicycle infrastructure and make it more convenient, get an increase in mode share.
I think there is a balance here. There is risk, so a discussion is required. But, the risk is medium, so a pharmacist can walk you through options.