You're not wrong for wanting to avoid a situation because of the potential of screaming children, but it's important to remember that, with parents, it's a package deal. To avoid overstimulation, try and go to a park for the first meeting. Like get coffee and go to a place with a playground where the toddler can run wild, and the 6 month can be in a stroller or rocker. Being outside vs bring in a building where screams can echo, makes a big difference.
Something that helps me hang-outs with my friends who have kids, is remembering that screaming children are inevitable... but most of the time, I'm not enjoying the company of my friends alongside the random, misbehaving ones. This time, you can make a friend.
My neighbor HATES me because I've been converting my backyard into clover. We have fireflies, Butterflies, bees, bunnies, all sorts of wildlife. It smells beautiful, but we are an oasis amongst upper-middle class lawn zombies... Mowing, edging, pesticide spraying, weed killing zombies.
Meanwhile, I have milkweed, clover, chive, snapdragons, black eyed susans, grapes, raspberries, lilac, echinacea, chamomile, lavender, hydrangea, coreopsis, and salvia. I welcome wasps that eat pests, I buy bags of ladybugs, I compost... I'm really trying. It's only 1/4 an acre, but I'm trying.