Road tires are much wider than they used to be. Professional road racers are running 28mm or even 30mm tires for lower suspension losses and reduced fatigue. The days of high pressure, thin tires are over. Absolutely go with the largest size tire that fits in your frame. You'll be more comfortable and have better traction on dirt surfaces.

You'll probably not be able to fit them, but IMHO, ~40mm tires are the sweet spot for commuting. They're wide enough to tackle just about any dirt/gravel path but not so bulky as to make it feel like you're spinning a tractor. 584-700 x 40 widely available with many models aiming for that commute / touring crowd with features like puncture resistance and aggressive tread only along the edge of the tire.

Try it with your tires and see what you think would be my suggestion then, 6 miles of dirt or gravel on a road bike might be rough but it could be doable depending on conditions and how they made the trail.