As someone working in embedded machine learning, I can say two things:
- Deep learning models, as powerful as they are, are the epitome of black box. We know they work, but we're still struggling to understand how they work. Further, they fail in oftentimes really weird and unexpected ways. So if we want to build safety-critical systems that depend on deep learning, that's a very tall order. And I don't think we're there yet with the tech.
- I've never worked with automotive embedded software, but my understanding is it's very hard and very complicated, and there's a real shortage of embedded engineers going into automotive. Trying to build incredibly complex, fault-tolerant embedded software systems for a widespread network of driverless cars sounds like an absolute nightmare and an absolutely gargantuan task.
Theoretically, I don't think it's impossible to build this vision of the future, but I think it would be stupendously difficult, take a lot more time than tech bros would like to lead you to believe, and ultimately just be a worse transit system than automated trains.
Not to mention being a pedestrian crossing the street in such a world would be a nightmare.