[MlMt] "error: Message contains NUL characters"

[Bill Cole]

On 18 Aug 2014, at 14:19, Lulie Tanett wrote:

> Hi all,
>
> I'm migrating from Mail.app to MailMate and have some offline messages
> which I'm putting on Fastmail. A few messages fail to upload and it
> gives me this error message:
>
> "The IMAP command “APPEND” (to INBOX.ARR Archives) failed with 
> server
> error: Message contains NUL characters."
>
> It also fails (with the same error) if I try to drag the .emlx file 
> from
> the Finder directly into MailMate.

That's not good. It's a strong indication that the message files are 
corrupted. It is theoretically possible that they are unusual cases of 
raw binary encoding in email, but it is far more likely that at some 
point they have been damaged by something.

> These messages are from 1994-1997 (only a few from these folders 
> failed
> though; most of them transferred through fine).

So, these are messages that predate the existence of Mail.app and 
presumably were imported into it from something else. That's actually a 
hopeful sign, since it is not hard to get an errant NUL into text data 
by way of an "off by one" error copying one extra byte in many 
programming languages. If a buggy importing process tacked an extra NUL 
onto the end of each message body in the .emlx files, Mail.app would 
likely never have a problem with it.

> Is there anything I can do to fix them?

Maybe.

A Mail .emlx file is essentially a text file with 3 parts: a line with 
the bytesize of the message data, the message data itself, and a short 
XML property list with Mail's metadata for the message. If there's a NUL 
tacked onto the end of the message data, Mail would probably show no 
sign of it and MailMate might not notice it, but it would be lethal when 
sending the message to an IMAP server using the standard mechanism which 
forbids raw NUL characters.

The way I'd attack the emlx files would start with confirming that they 
are not entirely garbage and look like something close to a proper emlx. 
I'm a Unix geek so I'd use 'head' and 'tail', but any text editor should 
work. If the first line is a number and the next dozen or so are email 
headers and the last 16 are a plist (see a valid emlx for what that 
looks like) then you can probably salvage them by pulling the message 
data out, cleaning it of NULs and any other control characters, and 
saving it as a .eml file.