Many moons ago, when I switched from POP to IMAP, I had to have my laptop and desktop upload all of the locally saved email so I wouldn’t lose anything. This was before I had infinite storage, and the duplicates were eating valuable space on my 4G server drive. Back then, I wrote a perl script that cleaned up my mbox files; but it only deleted duplicates within a given mbox file. These days my email usage is larger, and my sloppage is huge; especially when I consolidate or shift around accounts. I’ve been lazy and just living with it. No longer…
I recently discovered “Remove Duplicate Messages” for Thunderbird. I created a temp account, copied over a few thousand messages, across several folders, twice; then I did a few test runs. As far as I can tell it runs perfectly and doesn’t delete what it shouldn’t.
It isn’t a utility that you’ll be running every day, but when you do run it you will be happy with the results.
-Chris Knight