[MlMt] Local email archiving status/options?

[Brian Scholl]

Please pardon the length of this plea for help, and please pardon my
naivete.  (I am switching not only from another client to MailMate, but
from POP to IMAP at the same time -- with all the changes in workflow and
mindset that entails -- and I am probably under the grip of some false
assumptions.)

I have already convinced myself that I must switch to MM for all sorts of
reasons (most of them involving customizability), so that is now a foregone
conclusion.  (I will also say that the degree of transparency available
about the status of MM's development -- in Benny's many quick replies in
this list, plus the issue tracker -- seems like a *huge* plus.)

I am facing a single roadblock, though, that is the lack of support for any
local message archiving.  Having read the list archives extensively, I have
the following two impressions:

1. A good number of people need/want this, in some form or another.  (It
keeps coming up on the list semi-regularly, and by my count it has
occasioned at least 6 different tickets over years, etc.)  For some people,
this is a need driven by server/account size constraints.  For others, it
is simply a workflow-related preference.  And this has led at least some
users to cook up fairly obscure, complicated, and/or kluge-y solutions
(such as Fredrik Jonsson's method of setting up and running a local IMAP
server to implement local archiving).

2. But the probability of Benny adding this option anytime in the
foreseeable future seems to be somewhere between very low and zero.  (He
has generally flagged these sorts of tickets as 'bluesky'.)  I have the
impression that this is not because it's especially technically
challenging, but because he's philosophically opposed to it.  (Most of his
replies in the relevant list threads involve trying to get people to stop
wanting this in the first place, and/or to ask for IMAP quota increases
instead, etc.)  I don't resent this perspective at all, but I nevertheless
need some workaround since this is nonnegotiable for me -- and the purpose
of this message is to try to clarify my options.

What I want, in brief, is this: I want to use MM to sort my incoming mail
into various IMAP folders (probably about 20 of them, and probably using
some combination of filters/rules and manual message-moving via custom
keybindings).  Then I'll interact with my mail on a daily basis via those
folders along with many additional MM Smart folders.  Then, every so often
(e.g. once per year) -- when some of those IMAP folders get huge and/or
old, I want to archive them to some form of local-only folder/storage, such
that (1) I can then mass-delete all of the messages from those IMAP folders
(so that there are no longer any copies stored online), at which point
those particular folders will just start filling up again from scratch with
new incoming mail; but (2) I can still occasionally browse the local-only
archived messages somehow (even if this is done in some other program such
as DevonThink or EagleFiler -- though obviously it would be nice to be able
to do this in MM itself).

I should also stress that my desire to work this way with local-only
archives isn't driven by any external constraints.  (For example, my IMAP
account quota limits are infinite, for all practical purposes.)  Rather, it
is simply how I want to work -- and this isn't going to change.

So my plea to the list members is just this: what are my best options for
doing this?

1. LOCAL EXPORT?
Can I somehow export an IMAP folder from MM to any form of
locally-accessible file/archive (such as an .mbox file)?  It appears that
this isn't possible, and this is what Benny isn't too keen on adding.  (But
I'm not sure about this either.  In an older -- March 2011 -- reply to a
ticket along these lines, Benny wrote: "An export function (probably to the
.mbox format) is also planned since it is frequently requested by users."
-- but I take it that this never materialized.)  And while he also noted in
that same ticket thread that "you can drag messages out of MailMate to get
standard .eml files which should be importable by most other email
applications" -- but it sounds that like that's not really feasible for
exporting/archiving a mailbox with thousands of messages.

2. SNEAKY OFFLINE ACCESS TO MM'S LOCAL IMAP FILES?
MailMate stores its own local copies of your IMAP email so that you can
work offline.  Okay.  So is there any sneaky way to use these very files as
local-only archives?  (Here I'm pretty ignorant; e.g., I don't know if
different IMAP mailboxes [and the messages they contain] are locally stored
by MM as separate folders/files or not.  But clearly MM is storing them
locally in *some* format.)  For example, if there were some way to move
those files around, then would it be possible even in principle to (1) copy
one of those files/folders to some other part of my Mac's file system; then
(2) manually delete the contents of the relevant IMAP folder; but (3) later
be able to manually move the copied files/folders back into the proper
location for MM to access them again (perhaps while forced offline, to keep
MM from automatically synching it all back onto the server again -- as long
as I remove the file again before taking MM back online once I'm done
perusing the archive)?  Is anything like this possible?  It seems from
hints on this list like I can access these files, but that Benny doesn't
recommend it?  (He wrote in 2/15 that "it's not always safe to access the
files stored by MailMate directly. It's a moving target if MailMate is
active."  But I'd be happy to do this only while MM was forced offline.)

3. RUNNING MY OWN MAIL SERVER?
Fredrik Jonsson, Kai Grossjohann, and a few others have talked about how
they have set up their own dovecot mail servers as a workaround.  I take it
that this essentially allows them to use that server as a local archive
(since the information isn't synched online anywhere), while MM can
interact with it as if it was just another (online) IMAP account.  I am
loathe to pursue this, since I am quite sure that it is beyond my technical
abilities and that it will make my head explode.  At least, I have never
done anything even remotely like this in the past.  (And I also don't fully
understand how this works.  For example, if I wanted to manually move
archives in that format to another computer, how would that work?)

4. DIRECT EXPORTING TO EXTERNAL DATABASES/PROGRAMS?
During some of the previous discussions of such things, some list users
have obliquely referred to using MailMate but also archiving their email
using an external app such as EagleFiler or DevonThink.  I would be happy
to do this, but I still don't quite understand how that works.  I don't
want to archive single messages as they arrive and are processed (as I
think you can do with some of the bundles that Benny has made available?);
rather, I want to archive entire folders (with, say, 1000+ messages) in one
fell swoop -- while keeping those folders distinct once they make it into
the other program.  Is this possible?  When others on the list have talked
about using EagleFiler to archive their mail, for example, how do you get
the messages from MM into EagleFiler?  (Back in January, Matthew Cawood
said: "I wonder if the coding you've done will now give Devonthink the
hooks to import whole mailboxes from MailMate", but I don't think that was
ever clarified.)  I would be happy to browse my local-only archived mail in
another program of this sort, if this will work.  (And for others who do
this: am I right that this means that you can't ever access that mail again
in MM, once you've manually deleted the messages from the IMAP folders?)

5. LOCAL ARCHIVING TO MBOX FILES VIA A DIFFERENT EMAIL CLIENT?
Back in January on this list, Scott McIntyre mentioned in passing that it
is possible to use MM for everyday email, but then occasionally run
Mail.app to access the same IMAP account, use that to save/translate the
folder to a local mbox file, and then import that mbox file into DevonThink
or EagleFiler for long term access.  Will that work as a solution to my
needs?  I've never used Mail.app either, but I would be happy to do this if
it will work (especially keeping in mind that I'd probably do the archiving
only once per year or so, and only on a few very large mailboxes -- so it's
fine if the process of using Mail.app is slow or clunky, as long as it
works.)  Does this require lots of prep/setup in Mail.app, or will this
just work out of the box?  Does Mail.app end up storing that file in some
place that is easy to find?  (And an even more naive question: does
Mail.app, like MM, also store local copies of everything -- so that will
this lead/require Mail.app to locally download my entire IMAP account?  Or
will Mail.app keep everything online only, and only locally download those
few folders I designate to be saved as a local mbox file?)  Or is there yet
another IMAP email client that has .mbox export that I should use or this
instead?

Do you think any of these options will work for my needs?  Am I just too
confused about some of these things?  What do you recommend?  Is there some
other option I'm missing?  (FWIW, I'd prefer not to use MailSteward.)

I really am keen to start taking advantage of all MM has to offer, but
having some form of local archive is really non-negotiable for me.

Thank you for any help (and even just for reading this long message!),

-Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freron.com/pipermail/mailmate/attachments/20150825/335de2e0/attachment-0001.html>