I don't have kids (yet?) but I was raised vegetarian (Indian upper caste family). A vegan diet can be easily adapted for a child's needs. I think it's good that the (uncalled-for) remark by your colleague made you think about this, it means that you care about this issue more deeply than most people do, who just feed kids what they are cooking for themselves. I think the point that another commenter made about getting kids exposed, in a safe way, to allergens is very important. I'd even go far as letting them eat meat very very occasionally - who knows, there may be a time in the future where your need to survive will be against your need to be vegan.
I actually started eating meat after I moved out of India, and I'm hoping to shift to a vegan diet when things are more stable in my life. But personally, even if I don't ever eat any other kind of meat, I want to be able to eat beef, in some symbolic and meaningless way at least. Hindu vegetarianism is a toxic thing, it is absolutely not about not harming animals but about upper castes adopting a "pure" diet, and even then they make exceptions because that is how this bullshit works. I loathe the way Indian vegetarianism is mentioned in a positive way without these aspects being known. The same vegetarians will never give up milk, even though they know about the hell a cow has to go through. In India, you are judged by your dietary habits, you may not be able to rent if you eat meat. I've seen my friends in school getting bullied because they got some meat for lunch. Sorry for this off topic rant, but had to vent.
It's not about owning Hindus though. Let me give you some context. In Karnataka, an important South Indian state that the BJP (the far-right Hindutva party) won some time ago, there was a quite popular free lunch program, provided by the previous government. Their idea was to make poor kids feel like there is some point in going to school, and at the same time fix nutritional deficiencies. One of the items in the lunch would be a boiled egg, which you could refuse if you wished. Eggs being a cheap and great source of protein, made it an easy fix for the carbs heavy Indian diet. BJP comes in, and removes this option, largely due to pressure from their upper caste electorate, who mostly wouldn't even make use of the free lunch, but can't cope with children eating eggs, which are considered "non-veg" in India. In some sense, BJP could be seen as lessening the suffering of chickens, but I would never accept this kind of reasoning.
Beef is effectively banned from being sold and even eaten in like >80% of India. It's not just enough that Hindu upper castes, who have a religious obligation to not eat beef (even this is debatable), but they will not let Muslims and Christians, who have no such obligation, eat beef. People get routinely lynched to death because they are suspected of "trafficking" beef. If you get the cops to bother to look into this, they send the meat to a lab to know if it is indeed beef. This is the level of insanity that Indians have to deal with. And some BJP supporters have now started talking about how this is good for the planet and cuts down on animal suffering, as if keeping cows in factory farms and getting them raped repeatedly so they can be milked efficiently is somehow better.
If animal suffering is the only thing you are considering, then it shouldn't matter if you are being forced into not eating meat or not. Personally, I want to eliminate meat from my diet but I will stand for a person's right to eat meat. I realize this is an odd stand, and it is possibly even inconsistent, but I hope I have explained where it is coming from.