[MlMt] Local email archiving status/options?

[freron at aetm.us]

I found this message and the related 
[ticket](https://freron.lighthouseapp.com/projects/58672/tickets/81-archiving-to-local-folders) 
searching on options to import my 65 GB of offline archived email into 
MailMate.

I’m loving the power and customizability of MailMate by the way. 
Switched from a PC would have switched a decade ago had I known. The 
lack of features in MacOS Mail.app compared to Outlook on Windows made 
me give up on switching twice before.

I checked and my IMAP servers actually support 50 GB each and so it is 
feasible for me to ditch local email completely and upload everything to 
the servers the MM way.

A few questions:

1) Which is better for importing via MM - eml or mbox? Like Bill, I also 
just purchased Emailchemy to covert the pst files. It offers both mbox 
and eml as options. I presume mbox is better if I want to import 
everything?

2) I might end up with duplicates. Or does MM automatically no import 
duplicates? Any recommendations on de-duplication?

3) When I connected my Office 365 account, there were hundred of error 
messages as MM tried to download the Contacts, Calendar, Notes, and 
Tasks folders. I presume these should be unsubscribed?

4) Just want to confirm: does MM get its address book from MacOS? I see 
that it has Contacts access in System Preferences-Privacy, so that’s 
expected. Just curious because some other mail apps (e.g., Thunderbird) 
actually access the Office 365 contacts directly. I wanted to confirm 
that MM never reads contacts from a mail server?

5) Is there a way to open a message in a separate window? Within MM, the 
only option appears to be to create a duplicate window of the whole 
application, folders and all. Some messages are poorly formatted and 
need to be opened in a wider window to be read. I also tried “Open in 
Apple Mail” but get the following error: “The operation couldn’t 
be completed. (MCMailErrorDomain error 1030.) // Mail was unable to open 
the URL 
“message://%3C20210124214048.D35C4121FC9@mail.netagw.net%3E”.”

Thank you!

Christian



On 25 Aug 2015, at 17:59:48 EDT, Bill Cole wrote:

> On 25 Aug 2015, at 7:03, Brian Scholl wrote:
>
>> Also, a more theoretical postscript: It seems to me that Benny's 
>> reluctance to pursue any sort of .mbox export […]
>
> I can’t speak to the IMAP-*v*-POP3 debate, but I would really love 
> the ability to export a series of messages as a .mbox file in the same 
> format that Apple Mail does.

Isn't mbox great: one always has to specify a particular tool's
flavor...

If you want a reasonably portable mbox on your Desktop containing the
messages in a MM IMAP (not "Smart") folder, it's easily done with a
simple bit of shell calling the crufty old "formail" tool:

cd "~/Library/Application
Support/MailMate/Messages/IMAP/[account at 
server]/[path-to-folder]/Messages/"
for fname in *.eml ;  do formail < $fname ; done >
~/Desktop/Exported.mbox

If you want the delimiter lines' timestamps derived from Date headers
instead of now, the conversion gets more arcane and will not like
improper Date headers:

for fname in *.eml; do formail -a Date: < $fname ; done |
sed '/^From /s/, \(..\) \(...\) \(....\) \(..:..:..\) .*/ \2 \1 \4 \3/'
> ~/Desktop/Exported.mbox

Unfortunately, there's no sound way to get the proper delivery date into
the delimiter lines unless you are an IMAP client, so Mail.app does one
thing unequivocally better than formail. Also, unlike Mail.app, formail
uses (CORRECTLY) the Return-Path header (if present) to derive the
delimiter line address instead of Mail.app's simple use of the From
message header and only adds blank lines ahead of delimiter lines on an
as-needed basis.

So, while MailMate won't duplicate what Apple Mail does, one can get
pretty close using the MailMate message store without MailMate doing the
actual work.

> In addition to using SpamSieve on my Mac (which is quite good), I 
> maintain my own mail server (Mac OS X 10.6.8, until I am forced to 
> “upgrade”)

It is mildly amusing to learn that, as it means I'm not alone and in
respectable company. In my case the excuse is that the original Core Duo
can't do 64-bit mode and so can't run 10.7 or later, and I can't bear to
junk that old machine...

(Tangent: Senior mail admins are insanely over-represented on this
mailing list.)

> and use SpamAssassin there to try to intercept as much crud as 
> possible *before* it gets to SpamSieve. Apple Mail produces the 
> perfect .mbox files for feeding to SpamAssassin’s spam-learning 
> routines, so periodically I have to haul Mail out, select all the junk 
> that SpamAssassin needs to learn how to intercept, and File > Save 
> As… (raw source) to a file that I can then drag to the server and 
> run learnspam against.

I'm surprised that no one in this thread has yet mentioned the simple
ability to select messages in a MM message list and drag them to a
Finder window: a very fast way to create a file-per-message offline
local archive.

As someone who has retained email for 20+ years including a substantial
spam corpus (it's a professional focus) I share the desire for a local,
integrated, purely private, and reliable mail archive. Even though I
have my own Dovecot server running on a machine in the same room as my
main desktop, I don't keep my biggest archive there. Instead, after
years of working with various tools on a semi-converted pile of Eudora
almost-mbox files, I finally bought a license for Emailchemy
(http://www.weirdkid.com/products/emailchemy/index.html) a tool that can
actually recover and convert Eudora's quirky Classic Mac format
(CR-delimited mboxes with resource fork indexing and split attachments
pointed to by HFS CNIDs) as well as just about any other mail format one
might have into a variety of file-per-message and standard mbox formats
INCLUDING a maildir-like tree with a trivial read-only IMAP interface
meant to be used as a "Import Server." Since my "old archive" is just
that, I had Emailchemy do the conversion and run the Import Server for
long enough to have MailMate import the whole tree as a new IMAP
account. That account has been in offline mode in MM for a couple of
years now, happily challenging the robustness of MailMate's indexing &
searching capabilities as a huge collection of very weird mail, much of
it intentionally and maliciously in violation of any known standard.
Works marvelously, except that MM eats a lot of RAM (unavoidable with
400k+ messages) and is a bit sluggish for some searches.

Another approach I've played with but not really exercised hard is a
zombie "IMAP Account" permanently offline as a local archive. The
connection settings don't work and never have, but I can create folders
and subfolders and MM creates the directories just as it would for a
normal account. Benny has vaguely warned that this might not be a
reliable approach, but I think that's only because it has no backend and
ultimately would be lost if MM decided its local cache of messages was
not to be trusted. Tags may not work also...

I think to some degree the objection to MM not having an explicit
mechanism for local archive is a phantom issue perpetuated by Benny's
politely overt hostility to the concept. Every IMAP folder you can see
in MM has a complete local cache in the form of a folder full of plain
text files with .eml extensions in pristine RFC822 format under
~/Library/Application Support/MailMate/Messages/. Unless you exclude
that MM Messages tree from backups or you don't do backups, you have
backups of that cache. If you want a snapshot of what's in a particular
folder right now to save for later outside of MM because you're about to
whack it off your IMAP account, you can copy a whole folder elsewhere in
the Finder. MM even has a command to show you any selected message in
the Finder so you don't have to unhide ~/Library or type a long odd path
into Terminal. MM even has a Spotlight importer now so that if you were
to copy a folder out of the MM "cache" and then wipe it from the server,
you could still use Spotlight to find those messages, which open in MM
when double-clicked. If you wiped a folder last week from your IMAP
account and now really want it back, fire up Time Machine, find it in
the cache, restore it to someplace OUTSIDE of the cache, and you've got
your local archive independent of MM. Import it into a zombie account,
and it's a local archive directly accessible by MM.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freron.com/pipermail/mailmate/attachments/20210125/4c80df49/attachment.htm>