A lot of people already know the absolutely excellent history of Rome podcast by Mike Duncan. However there are a lot of other history podcasts out there which also do "the start to finish format", inspired by Mike Duncan. Some good ones I have listened to include:

Pax Britannica: A great podcast with good story telling ability about British history, with a focus on the British empire. It begins with the Henry VII and ends with Queen Elisabeth.

Russians rulers podcast: A great podcast that starts with the very first tsar of Russia and follows through the history of the whole country by focusing on the ruler of the time. It begins with Rurik and ends in Putin. However he already finished the leaders years ago so now he does slapshot episodes about various other Russian history topics which is also very interesting.

Fall of civilisations podcast: this is a great one with some of the best story telling in podcast form available. For each episode he chooses a civilization which collapsed in some form or another. He then tells their history from start to finish and but focuses on the decline and how it was to live in those last years. It's really dramatic sometimes but it's really informative, well researched and I highly recommend it.

Okay and now this is not a history podcast but it's still a dear one to me: Sunday school dropouts. It's a podcast by a wholesome married couple composed of a former presbyterian Christian (now atheist) and a "non beliving sort of Jew" (his words) that read through the whole bible for the first time. They begin with a episode on the book of genesis and continue to the book of revelation. Best way to follow along is to read the bible at the same time and after every bible book (most can be read in under an hour) you listen to the episode afterwards. But you can also listen to it blindly because they do summarize everything. Okay so why do I like this one? The bible is a truly interesting book but the discussion in our media about it is horrible. It's either the most anti religion people or the "capital A atheists" discussing it or it's religious people themselves, both of course approach it with very preconceived notions. But this is just a calm podcast where two non Christians seriously read it through, do their research and they discover that some is total garbage and some of the stories are so beautiful they couldn't stop from crying during the show. Also the hosts are very entertaining and easy to like. I believe everyone should read the bible at least once to simply know what's it about. It's the most important book published in world history after all. They finished already and then did two seasons of just random pieces of interesting bible lore which was also fun to listen to.