[MlMt] Local email archiving status/options?

[Brian Scholl]

Also, a more theoretical postscript: It seems to me that Benny's reluctance
to pursue any sort of .mbox export or local folder option stems from the
notion that such things are intrinsically in opposition to the entire
notion/purpose of IMAP.  For example, in one of the ticket replies about
such things once upon a time, he wrote: "MailMate really is IMAP only and I
don't have any current plans to change that. Essentially (if you think
about it), 'On My Mac' support is the same as supporting POP3. Messages are
only temporarily stored on the server."

That doesn't seem right to me.  Embracing the IMAP worldview but then also
limiting the size and complexity of online accounts by periodically
allowing them to be culled to local-only long-term-storage archives doesn't
feel the same to me as just using POP3 in a different way.  Conversely, it
seems like allowing for a local .mbox export (or something like that) in MM
wouldn't be some sort of betrayal of its underlying nature.  Rather, it
would just be a way for users to adapt true day-to-day IMAP usage to a
longer-term archiving practice.

(A concrete example: I teach classes at a university, and during each
semester I will accrete huge online IMAP folders with emails relating to
those classes.  But then once the relevant semester has finished, I just
hate the idea of keeping those folders online forever.  Even just the sheer
*number* of such folders would become awkward and distracting over time,
not to mention the messages they contain.  But I also don't just want to
delete those messages forever -- since maybe once per year I'll have a need
to dive into a previous semester's folders to find something.  So this is
practice some sort of betrayal of how I'm supposed to use IMAP?)

Do others feel similarly?  Am I just thinking about this the wrong way?

-Brian

Brian Scholl wrote:
> Please pardon the length of this plea for help, and please pardon
> my naivete.  (I am switching not only from another client to
> MailMate, but from POP to IMAP at the same time -- with all the
> changes in workflow and mindset that entails -- and I am probably
> under the grip of some false assumptions.)
>
> I have already convinced myself that I must switch to MM for all
> sorts of reasons (most of them involving customizability), so that
> is now a foregone conclusion.  (I will also say that the degree of
> transparency available about the status of MM's development -- in
> Benny's many quick replies in this list, plus the issue tracker --
> seems like a *huge* plus.)
>
> I am facing a single roadblock, though, that is the lack of support
> for any local message archiving.  Having read the list archives
> extensively, I have the following two impressions:
>
> 1. A good number of people need/want this, in some form or another.
> (It keeps coming up on the list semi-regularly, and by my count it
> has occasioned at least 6 different tickets over years, etc.)  For
> some people, this is a need driven by server/account size
> constraints.  For others, it is simply a workflow-related
> preference.  And this has led at least some users to cook up fairly
> obscure, complicated, and/or kluge-y solutions (such as Fredrik
> Jonsson's method of setting up and running a local IMAP server to
> implement local archiving).
>
> 2. But the probability of Benny adding this option anytime in the
> foreseeable future seems to be somewhere between very low and zero.
> (He has generally flagged these sorts of tickets as 'bluesky'.)  I
> have the impression that this is not because it's especially
> technically challenging, but because he's philosophically opposed
> to it.  (Most of his replies in the relevant list threads involve
> trying to get people to stop wanting this in the first place,
> and/or to ask for IMAP quota increases instead, etc.)  I don't
> resent this perspective at all, but I nevertheless need some
> workaround since this is nonnegotiable for me -- and the purpose of
> this message is to try to clarify my options.
>
> What I want, in brief, is this: I want to use MM to sort my
> incoming mail into various IMAP folders (probably about 20 of them,
> and probably using some combination of filters/rules and manual
> message-moving via custom keybindings).  Then I'll interact with my
> mail on a daily basis via those folders along with many additional
> MM Smart folders.  Then, every so often (e.g. once per year) --
> when some of those IMAP folders get huge and/or old, I want to
> archive them to some form of local-only folder/storage, such that
> (1) I can then mass-delete all of the messages from those IMAP
> folders (so that there are no longer any copies stored online), at
> which point those particular folders will just start filling up
> again from scratch with new incoming mail; but (2) I can still
> occasionally browse the local-only archived messages somehow (even
> if this is done in some other program such as DevonThink or
> EagleFiler -- though obviously it would be nice to be able to do
> this in MM itself).
>
> I should also stress that my desire to work this way with
> local-only archives isn't driven by any external constraints.  (For
> example, my IMAP account quota limits are infinite, for all
> practical purposes.)  Rather, it is simply how I want to work --
> and this isn't going to change.
>
> So my plea to the list members is just this: what are my best
> options for doing this?
>
> 1. LOCAL EXPORT?
> Can I somehow export an IMAP folder from MM to any form of
> locally-accessible file/archive (such as an .mbox file)?  It
> appears that this isn't possible, and this is what Benny isn't too
> keen on adding.  (But I'm not sure about this either.  In an older
> -- March 2011 -- reply to a ticket along these lines, Benny wrote:
> "An export function (probably to the .mbox format) is also planned
> since it is frequently requested by users." -- but I take it that
> this never materialized.)  And while he also noted in that same
> ticket thread that "you can drag messages out of MailMate to get
> standard .eml files which should be importable by most other email
> applications" -- but it sounds that like that's not really feasible
> for exporting/archiving a mailbox with thousands of messages.
>
> 2. SNEAKY OFFLINE ACCESS TO MM'S LOCAL IMAP FILES?
> MailMate stores its own local copies of your IMAP email so that you
> can work offline.  Okay.  So is there any sneaky way to use these
> very files as local-only archives?  (Here I'm pretty ignorant;
> e.g., I don't know if different IMAP mailboxes [and the messages
> they contain] are locally stored by MM as separate folders/files or
> not.  But clearly MM is storing them locally in *some* format.)
> For example, if there were some way to move those files around,
> then would it be possible even in principle to (1) copy one of
> those files/folders to some other part of my Mac's file system;
> then (2) manually delete the contents of the relevant IMAP folder;
> but (3) later be able to manually move the copied files/folders
> back into the proper location for MM to access them again (perhaps
> while forced offline, to keep MM from automatically synching it all
> back onto the server again -- as long as I remove the file again
> before taking MM back online once I'm done perusing the archive)?
> Is anything like this possible?  It seems from hints on this list
> like I can access these files, but that Benny doesn't recommend it?
> (He wrote in 2/15 that "it's not always safe to access the files
> stored by MailMate directly. It's a moving target if MailMate is
> active."  But I'd be happy to do this only while MM was forced
> offline.)
>
> 3. RUNNING MY OWN MAIL SERVER?
> Fredrik Jonsson, Kai Grossjohann, and a few others have talked
> about how they have set up their own dovecot mail servers as a
> workaround.  I take it that this essentially allows them to use
> that server as a local archive (since the information isn't synched
> online anywhere), while MM can interact with it as if it was just
> another (online) IMAP account.  I am loathe to pursue this, since I
> am quite sure that it is beyond my technical abilities and that it
> will make my head explode.  At least, I have never done anything
> even remotely like this in the past.  (And I also don't fully
> understand how this works.  For example, if I wanted to manually
> move archives in that format to another computer, how would that
> work?)
>
> 4. DIRECT EXPORTING TO EXTERNAL DATABASES/PROGRAMS?
> During some of the previous discussions of such things, some list
> users have obliquely referred to using MailMate but also archiving
> their email using an external app such as EagleFiler or DevonThink.
> I would be happy to do this, but I still don't quite understand how
> that works.  I don't want to archive single messages as they arrive
> and are processed (as I think you can do with some of the bundles
> that Benny has made available?); rather, I want to archive entire
> folders (with, say, 1000+ messages) in one fell swoop -- while
> keeping those folders distinct once they make it into the other
> program.  Is this possible?  When others on the list have talked
> about using EagleFiler to archive their mail, for example, how do
> you get the messages from MM into EagleFiler?  (Back in January,
> Matthew Cawood said: "I wonder if the coding you've done will now
> give Devonthink the hooks to import whole mailboxes from MailMate",
> but I don't think that was ever clarified.)  I would be happy to
> browse my local-only archived mail in another program of this sort,
> if this will work.  (And for others who do this: am I right that
> this means that you can't ever access that mail again in MM, once
> you've manually deleted the messages from the IMAP folders?)
>
> 5. LOCAL ARCHIVING TO MBOX FILES VIA A DIFFERENT EMAIL CLIENT?
> Back in January on this list, Scott McIntyre mentioned in passing
> that it is possible to use MM for everyday email, but then
> occasionally run Mail.app to access the same IMAP account, use that
> to save/translate the folder to a local mbox file, and then import
> that mbox file into DevonThink or EagleFiler for long term access.
> Will that work as a solution to my needs?  I've never used Mail.app
> either, but I would be happy to do this if it will work (especially
> keeping in mind that I'd probably do the archiving only once per
> year or so, and only on a few very large mailboxes -- so it's fine
> if the process of using Mail.app is slow or clunky, as long as it
> works.)  Does this require lots of prep/setup in Mail.app, or will
> this just work out of the box?  Does Mail.app end up storing that
> file in some place that is easy to find?  (And an even more naive
> question: does Mail.app, like MM, also store local copies of
> everything -- so that will this lead/require Mail.app to locally
> download my entire IMAP account?  Or will Mail.app keep everything
> online only, and only locally download those few folders I
> designate to be saved as a local mbox file?)  Or is there yet
> another IMAP email client that has .mbox export that I should use
> or this instead?
>
> Do you think any of these options will work for my needs?  Am I
> just too confused about some of these things?  What do you
> recommend?  Is there some other option I'm missing?  (FWIW, I'd
> prefer not to use MailSteward.)
>
> I really am keen to start taking advantage of all MM has to offer,
> but having some form of local archive is really non-negotiable for
> me.
>
> Thank you for any help (and even just for reading this long message!),
>
> -Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freron.com/pipermail/mailmate/attachments/20150825/94962f47/attachment-0001.html>