Based on what I know of Imposter Syndrome and the Dunning-Kruger effect, it seems you're at your most competent when you feel like you're at your least.
So if you're feeling badly because you feel like you don't know enough to do your job, take some time to remind yourself that other people who appear to be confident have no idea what they're doing.
It's fake-it-till-you-make-it all the way down.