[MlMt] Memory Usage (Mail Steward)

[Tracy Valleau]

OK: more info on Mail Steward. (I'm just a happy user, and not affiliated, etc etc.)

Mail Steward takes the emails and attachments, and stores them on your local hard drive (or wherever your mySQL database is stored.)

Because it's mySQL, you can search it both faster and more completely than you can with everything residing on your server, and/or using MailMate.

I have my copy update itself automatically from my MailMate mailboxes, twice a day, 8AM and 8PM. I also have it set up so that any message marked as "junk" (actually any tag I choose in MailMate) are skipped during archiving, so I  don't have 250,000 Viagra ads.

Searching is just a matter of launching Mail Steward, typing in the search criteria, and hitting a button. I don't have 3 million messages, but I do have several hundred thousand, and a simple text search takes less than 20 seconds. (Like any database search, the time will change depending on your search critera.)

I can do a simple "search everything for the word "duh" or I can say "search everything for the word "duh" in emails from someone at gmail, and including attachments, between January 2015 and January 2017."  That's nothing special to Mail Steward; it's just standard mySQL searching.

I do not wait more than 2 seconds for the program to load, because that, again, is how mySQL works.

I would suspect that you'd find mySQL to be -significantly- faster and easier than using MM.

And instead of a "virtual folder" for searching the last 6 months, you'd have all 3 million of them immediately available. I have been archiving mine, continuously, since 1998.

I just ran a test: from the time I clicked on MS to launch it, to the time is searched through over 300,000 records for the word "Janice" and display all the results, was 13 seconds.

Out of curiosity, how long would it take from the first launch of Mail Mate to do the same thing?

HTH

T



On 11 May 2021, at 12:33, mailmate-request at lists.freron.com wrote:

> Send mailmate mailing list submissions to
> 	mailmate at lists.freron.com
>
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://lists.freron.com/listinfo/mailmate
> or, via email, send a message with subject or body 'help' to
> 	mailmate-request at lists.freron.com
>
> You can reach the person managing the list at
> 	mailmate-owner at lists.freron.com
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of mailmate digest..."
>
>
> Today's Topics:
>
>    1. Migrating to Big Sur (davecc0000)
>    2. Re: Default format=flowed change in 5800? (Eric Sharakan)
>    3. Re: Migrating to Big Sur (Bill Cole)
>    4. Re: Migrating to Big Sur (davecc0000)
>    5. Memory usage (Tracy Valleau)
>    6. Re: Memory usage (Patrik Fältström)
>    7. Re: Memory usage (Randall Meadows)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 11 May 2021 09:27:29 -0700
> From: davecc0000 <davec2468 at gmail.com>
> To: MailMate Users <mailmate at lists.freron.com>
> Subject: [MlMt] Migrating to Big Sur
> Message-ID: <BA267626-BF72-40C4-9C6F-3AE816C0CDD6 at gmail.com>
> Content-Type: text/plain; charset=utf-8
>
> MM now launches and doesn’t quit immediately.
>
> When I am asked for accounts passwords, immediately after entering one I get an alert:
>
> “MailMate failed to save a file to disk. (/Users/macmini//Library/Application Support/MailMate/Messages/IMAP/[myemail address%40gmail.com at imap.gmail.con/State.plist_backup). Error: no such file or directory.”
>
> I copied the files (see below) from my previous daily-functioning MM install on my Sierra boot disk to the new Big Sur drive (the entire MM folder, not by opening the folder and selecting all files and dragging those).
>
> Any idea why I’m getting this error?
>
> Thanks,
> Dave
> macOS 11.3.1 Big Sur
> MailMate 1.14
>
> - - -
>
>> To migrate MailMate from Sierra to Big Sur, I moved these folders to identical locations on the new hard boot drive by dragging and dropping in Finder:
>>
>> ~/Library/Application Support/MailMate (folder)
>>
>> ~/Library/Preferences/com.freron.MailMate.plist
>>
>> All copied. I then downloaded the latest daily MM beta and copied it to the Applications folder.
>>
>> After booting into Big Sur and launching MailMate, MM quit immediately and Terminal opened and spit out some errors errors. (Terminal purges previous output on Quit so those are gone for now. I’ll post what I see when I next boot BS and run MM.)
>>
>> Is my process correct?
>>
>> Thanks,
>> Dave
>
>
>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 11 May 2021 12:28:33 -0400
> From: Eric Sharakan <esharakan at gmail.com>
> To: Benny Kjær Nielsen <mm-feedback at freron.com>
> Cc: MailMate Users <mailmate at lists.freron.com>
> Subject: Re: [MlMt] Default format=flowed change in 5800?
> Message-ID: <D1422080-ADAA-4632-8A9F-21B44E364919 at gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Benny, can you please address this question, as it has many of us puzzled?
>
> Thanks.
>
> -Eric
>
> On 7 May 2021, at 7:16, Philip Paeps wrote:
>
>> On 2021-05-03 21:43:18 (+0800), Pete Resnick wrote:
>>> Saw this in the release notes for Test Build 5800:
>>>
>>> - Changed: By default, MailMate no longer uses the format=flowed standard. For now, it can be enabled using MmFormatFlowedEnabled if needed/preferred.
>>>
>>> Why the change? There should be more format=flowed in the world, not less, IMO.
>>
>> I've been wondering about this change too.  I know there are awkward interactions between format=flowed and inline PGP (and possibly S/MIME) but `MmFormatFlowedEnabled` seems a lot more useful to be default *on* rather than default *off*.
>>
>> I did a `defaults write com.freron.MailMate MmFormatFlowedEnabled -bool true` immediately after installing r5800.
>>
>> Philip
>>
>> -- 
>> Philip Paeps
>> Senior Reality Engineer
>> Alternative Enterprises
>> _______________________________________________
>> mailmate mailing list
>> mailmate at lists.freron.com
>> https://lists.freron.com/listinfo/mailmate
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.freron.com/pipermail/mailmate/attachments/20210511/9ad890d9/attachment-0001.htm>
>
> ------------------------------
>
> Message: 3
> Date: Tue, 11 May 2021 13:03:19 -0400
> From: "Bill Cole" <mmlist-20120120 at billmail.scconsult.com>
> To: "MailMate Users" <mailmate at lists.freron.com>
> Subject: Re: [MlMt] Migrating to Big Sur
> Message-ID:
> 	<1FDA7A02-4A3C-4930-A6C0-1C7F89E1BFA4 at billmail.scconsult.com>
> Content-Type: text/plain; charset="UTF-8"; format=flowed
>
> On 2021-05-11 at 12:27:29 UTC-0400 (Tue, 11 May 2021 09:27:29 -0700)
> davecc0000 <mailmate at lists.freron.com>
> is rumored to have said:
>
>> MM now launches and doesn’t quit immediately.
>>
>> When I am asked for accounts passwords, immediately after entering one
>> I get an alert:
>>
>> “MailMate failed to save a file to disk.
>> (/Users/macmini//Library/Application
>> Support/MailMate/Messages/IMAP/[myemail
>> address%40gmail.com at imap.gmail.con/State.plist_backup). Error: no such
>> file or directory.”
>>
>> I copied the files (see below) from my previous daily-functioning MM
>> install on my Sierra boot disk to the new Big Sur drive (the entire MM
>> folder, not by opening the folder and selecting all files and dragging
>> those).
>>
>> Any idea why I’m getting this error?
>
> Most likely because ~/Library/Application Support/MailMate/Messages is
> (usually?) a symbolic link which points to the absolute full pathname of
> your actual message store and that path does not exist on the new
> system. If you had a "Custom Location" set for the Messages folder (in
> Preferences->General) you may be able to fix the problem by switching it
> off, restarting MM, and switching it back on. If you didn't have that
> set, you can recreate the default symlink with:
>
> ln -sfn ~/Library/Application\ Support/MailMate/Messages.noindex
> ~/Library/Application\ Support/MailMate/Messages
>
>
>
>
>>
>> Thanks,
>> Dave
>> macOS 11.3.1 Big Sur
>> MailMate 1.14
>>
>> - - -
>>
>>> To migrate MailMate from Sierra to Big Sur, I moved these folders
>>> to identical locations on the new hard boot drive by dragging and
>>> dropping in Finder:
>>>
>>> ~/Library/Application Support/MailMate (folder)
>>>
>>> ~/Library/Preferences/com.freron.MailMate.plist
>>>
>>> All copied. I then downloaded the latest daily MM beta and copied it
>>> to the Applications folder.
>>>
>>> After booting into Big Sur and launching MailMate, MM quit
>>> immediately and Terminal opened and spit out some errors errors.
>>> (Terminal purges previous output on Quit so those are gone for now.
>>> I’ll post what I see when I next boot BS and run MM.)
>>>
>>> Is my process correct?
>
> If you had a "Custom Location" set for the Messages folder on the old
> system, that copy missed all of your messages.
>
>
>
> -- 
> Bill Cole
> bill at scconsult.com or billcole at apache.org
> (AKA @grumpybozo and many *@billmail.scconsult.com addresses)
> Not Currently Available For Hire
>
>
> ------------------------------
>
> Message: 4
> Date: Tue, 11 May 2021 10:42:37 -0700
> From: davecc0000 <davec2468 at gmail.com>
> To: MailMate Users <mailmate at lists.freron.com>
> Subject: Re: [MlMt] Migrating to Big Sur
> Message-ID: <9B8441E8-42D2-442F-A6DA-A7B93545F5B5 at gmail.com>
> Content-Type: text/plain; charset=utf-8
>
> Thanks Bill for your reply. Indeed that was the problem. I had checked the alias and suspecting it, “fixed” it myself (GetInfo>Original>let-monkey-loose-on-keyboard). I should have known better.
>
> All set up on Big Sur now and running smoothly. Many thanks.
>
> Dave
>
> - - -
>
>> Most likely because ~/Library/Application Support/MailMate/Messages is (usually?) a symbolic link which points to the absolute full pathname of your actual message store and that path does not exist on the new system. If you had a "Custom Location" set for the Messages folder (in Preferences->General) you may be able to fix the problem by switching it off, restarting MM, and switching it back on. If you didn't have that set, you can recreate the default symlink with:
>>
>> ln -sfn ~/Library/Application\ Support/MailMate/Messages.noindex ~/Library/Application\ Support/MailMate/Messages
>
>
> ------------------------------
>
> Message: 5
> Date: Tue, 11 May 2021 10:49:06 -0700
> From: Tracy Valleau <tracy at dlsi.biz>
> To: mailmate at lists.freron.com
> Subject: [MlMt] Memory usage
> Message-ID: <A42696D0-72BA-4CF7-9C42-A7FF95B02740 at dlsi.biz>
> Content-Type: text/plain; charset=UTF-8
>
> 3 million messages !  Do you really need all of them actively online?
>
> If not, and you are just archiving them, with only the need for infrequent access, you might want to consider something like MailSteward.
>
> https://mailsteward.com/
>
> I've been using it for over a decade, and it has never lost an email. That allows me to remove the email from my server, and keep significantly less than 3 million "on hand."
>
> (For three million messages, you'd want the full mySQL version.)
>
> T
>
> On 11 May 2021, at 9:00, mailmate-request at lists.freron.com wrote:
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Tue, 11 May 2021 09:10:32 +0200
>> From: Patrik Fältström <paf at frobbit.se>
>> To: MailMate Users <mailmate at lists.freron.com>
>> Subject: [MlMt] Memory usage
>> Message-ID: <D0A13C49-0620-4205-8430-9FAEE8BF858E at frobbit.se>
>> Content-Type: text/plain; charset="utf-8"
>>
>> I have been looking at the memory usage of mailmate throughout the years. I have concluded it "uses lots of memory".
>>
>> I have four imap accounts where specifically one have lots of messages. In total I have 2.988.939 messages, and MailMate eats 4.34GByte of memory about half an hour after starting.
>>
>
>
> ------------------------------
>
> Message: 6
> Date: Tue, 11 May 2021 20:09:43 +0200
> From: Patrik Fältström <paf at frobbit.se>
> To: MailMate Users <mailmate at lists.freron.com>
> Subject: Re: [MlMt] Memory usage
> Message-ID: <5B5E70E5-9AFA-4870-A376-788E72F72C82 at frobbit.se>
> Content-Type: text/plain; charset="utf-8"
>
> On 11 May 2021, at 19:49, Tracy Valleau wrote:
>
>> 3 million messages !  Do you really need all of them actively online?
>
> It is VERY convenient to be able to search over all of them :-)
>
> Btw, I do have a virtual folder which is "all messages last six months" and default search folder is that one. So I have by the use of virtual folders hidden most email from daily work.
>
> Very convenient.
>
>> If not, and you are just archiving them, with only the need for infrequent access, you might want to consider something like MailSteward.
>>
>> https://mailsteward.com/
>>
>> I've been using it for over a decade, and it has never lost an email. That allows me to remove the email from my server, and keep significantly less than 3 million "on hand."
>>
>> (For three million messages, you'd want the full mySQL version.)
>
> Thanks!
>
>   Patrik
>
>> T
>>
>> On 11 May 2021, at 9:00, mailmate-request at lists.freron.com wrote:
>>>
>>> ----------------------------------------------------------------------
>>>
>>> Message: 1
>>> Date: Tue, 11 May 2021 09:10:32 +0200
>>> From: Patrik Fältström <paf at frobbit.se>
>>> To: MailMate Users <mailmate at lists.freron.com>
>>> Subject: [MlMt] Memory usage
>>> Message-ID: <D0A13C49-0620-4205-8430-9FAEE8BF858E at frobbit.se>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> I have been looking at the memory usage of mailmate throughout the years. I have concluded it "uses lots of memory".
>>>
>>> I have four imap accounts where specifically one have lots of messages. In total I have 2.988.939 messages, and MailMate eats 4.34GByte of memory about half an hour after starting.
>>>
>> _______________________________________________
>> mailmate mailing list
>> mailmate at lists.freron.com
>> https://lists.freron.com/listinfo/mailmate
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: signature.asc
> Type: application/pgp-signature
> Size: 256 bytes
> Desc: OpenPGP digital signature
> URL: <http://lists.freron.com/pipermail/mailmate/attachments/20210511/605723e6/attachment-0001.sig>
>
> ------------------------------
>
> Message: 7
> Date: Tue, 11 May 2021 13:23:56 -0600
> From: "Randall Meadows" <randy.meadows at not-pc.com>
> To: "MailMate Users" <mailmate at lists.freron.com>
> Subject: Re: [MlMt] Memory usage
> Message-ID: <D19AABCA-F302-4250-92D8-3212BF8DDD6C at not-pc.com>
> Content-Type: text/plain; charset="UTF-8"; format=flowed
>
> On 11 May 2021, at 11:49, Tracy Valleau wrote:
>
>> If not, and you are just archiving them, with only the need for
>> infrequent access, you might want to consider something like
>> MailSteward.
>>
>> https://mailsteward.com/
>
> Interesting...I don't have that many messages, but I do have lots of
> stuff I like to keep archived, and right now that's in IMAP mailboxes.
>
> Going the MailSteward route—does that create a completely separate
> silo for archived messages, or is there some mechanism for being able to
> include those archived messages in a search from the mail client itself?
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> mailmate mailing list
> mailmate at lists.freron.com
> https://lists.freron.com/listinfo/mailmate
>
>
> ------------------------------
>
> End of mailmate Digest, Vol 122, Issue 16
> *****************************************