[MlMt] MailMate so slow it locks up

[Benny Kjær Nielsen]

On 15 Jun 2016, at 17:10, Alex Bligh wrote:

I'm afraid I cannot provide you with a solution, but at least I can 
provide a few comments/explanations. It may not be clear below, but in 
general I'm always embarrassed when performance clearly could be better.

> ... Thunderbird accesses both accounts directly in the normal way.

But maybe not in a fully offline mode?

MailMate is a so-called offline IMAP email client and there are no 
options to change that. This means that MailMate fetches and stores a 
local copy of every email in the account(s). In addition to that, every 
header of every email is indexed. Although optimizations mean that 
MailMate can handle fairly large email accounts (some users have more 
than a million), MailMate was not really designed to handle “huge” 
accounts. There is likely more that I could do to handle large accounts 
more efficiently, but few of them are likely to be quick fixes. In 
short, MailMate is extremely flexible, but some parts do not scale well.

> On installing MailMate, I set up the smaller account first (300,000 
> messages). It correctly imported the settings, but after that there 
> was a world of pain. It sync'd all the mail, but only by running 2 CPU 
> cores at 100% continuously for over 24 hours. Neither Mail.app nor 
> Thunderbird do this; they import in the background, and (for that mail 
> account) take only a few hours. Saying that, after 24 hours (perhaps a 
> bit more), it had pulled all the mail in and appeared to work.

That's more time than I would have expected if you are on a fast 
connection and it isn't Gmail (Google throttles traffic), but since it 
did finish then some kind of looping behavior is probably unlikely.

> I then enabled the larger account. [...] MailMate was fantastically 
> unresponsive during this time - clearly importing is a real CPU and 
> memory hog.

Yes, it can be pretty bad, and this is probably beyond the number of 
messages MailMate can handle.

> The mail server is running cyrus imapd 2.4, has many users on it, and 
> not one of them has seen any problems with any other MUA to my 
> knowledge.

Most IMAP email clients are “online” in some sense or they only 
index a few headers of each message. And some email clients are probably 
just better than MailMate.

> I regularly use Mail.app, Thunderbird (on OS-X) an iPhone, and until a 
> couple of years ago Mulberry, with no issues. I know others are 
> running all sorts of stuff (so imap logs tell me). The problem is not 
> the mail server, and it's certainly not the client - Thunderbird is 
> pretty fast, and Mail.app is positively zippy.

If I understood correctly then you don't have millions of messages in 
Mail.app, but I get the point.

> In the brief time I used MailMate on the smaller account, I really 
> liked it. But even if I could get over the import issue, the fact that 
> if I ever lose my mail spool I'd need to last 48 hours+ without a mail 
> client that works is a red flag for me.

Very understandable.

> Any ideas what this might be and/or how to fix it?

MailMate does support subscriptions which means that if you don't really 
need all those messages at your fingertips then you should take 
advantage of that. Note that you can edit subscriptions before adding 
the account.

I did not design MailMate to handle millions of messages (or even 
hundreds of thousands). Internally, all messages are put into one big 
pool and everything else is based on queries within this pool —— 
including all IMAP mailboxes. This only works with large message stores, 
because many parts of MailMate are actually quite fast. But if there is 
one thing I've learned then it's that every time I optimize MailMate to 
handle X number of messages then someone comes around asking for 2*X :-)

-- 
Benny