[MlMt] MailMate passed the CASA tier 2 security assessment (hurrah?)

[Benny Kjær Nielsen]

Hi MailMate users,

most mailing list readers probably missed this, but a couple of months 
ago I wrote something on the list which deserves a follow-up.

On 3 Apr 2024, at 15:27, Benny Kjær Nielsen wrote:

> [...] When I get back, the main priority will be to look into a 
> potential future Gmail issue. The “kill switch” I mentioned in an 
> old blog post might be looming around the corner: 
> https://blog.freron.com/2015/is-oauth2-support-a-good-thing/

Almost half a year ago I got a message from Google informing me that 
they needed me to do some additional verification steps in order for 
MailMate users to be able to continue to use Gmail. They told me that I 
had 90 days to initiate the process -- this naturally meant I ignored it 
for almost 60 days. When I finally initiated the process, I realized 
that it would likely require more work than I had anticipated. It also 
included a deadline:

> “You are required to complete a CASA Tier 2 security assessment for 
> your application [...] by the following date: 2024-06-12. This 
> assessment is required annually; to learn more, please visit the CASA 
> website.”

Before going into what this means, I should tell you that I've completed 
the assessment and MailMate went into the “verified” state a week 
ago. This should make the MailMate/Gmail combination fairly safe for the 
next year. To be more precise, the verification expires May 7th 2025.

Going forward, I will have to do this every year. This means that every 
year there is a potential risk of MailMate no longer being able to 
access Gmail accounts. Google might decide that some 
feature/language/library used by MailMate is no longer considered safe, 
or they might come up with some other requirement that I cannot easily 
satisfy. This is a risk that I somehow need to make sure that users are 
aware of before buying a license key. I'm still considering how to best 
solve this problem.

The same, by the way, is true for any other desktop email client, but I 
think it's fair to assume that many of them are “too big to fail”.

The rest of this email is for anyone wondering what the assessment is 
all about and then some random thoughts on the subject. It's a brain 
dump and probably a bit of a mess :)

---

So what is a CASA Tier 2 security assessment? It's mostly questions 
which are irrelevant to MailMate (and probably most desktop email 
clients) which means I've been clicking a lot of checkboxes on a 
website. Most often the options are Yes, No, and Not Applicable. 
Choosing the last option would trigger a discussion with an assessor 
which would end with me either changing it to Yes or a note being added 
to the item in the final “letter of validation”.

The most problematic part was a so-called static analysis of the source 
code. This can be done in a number of ways, but the only one really 
available to me was a “self scan” using an open source tool. I'll 
skip the details, but let's just say that this doesn't really work well 
for an application having code in C, C++, Objective C, Objective C++, 
Javascript, Ruby, Python, Swift, Bash, and possibly more. Yes, it's a 
complex app.

Scanning code for vulnerabilities is good and I could likely do more to 
integrate that in my build process. I'm not questioning that. 
Apple/Xcode/clang warns about insecure/deprecated APIs/functions in the 
build process and I wouldn't mind if this was extended to also check for 
known/likely vulnerabilities. Every release of MailMate is also scanned 
by Apple (after compilation) as part of the so-called notarization 
process although I think this is mostly about detecting intentionally 
included malware. Nevertheless, this process is more “real” since I 
cannot release an update which does not pass these checks. The CASA 2 
assessment is based completely on good faith.

Other random thoughts on this subject:

* In relation to Gmail, the most sensitive part of talking to the server 
is the OAuth2 authentication process. MailMate never sees the user 
password because that is part of the process. The token used to gain 
access is stored in Keychain Access (Apple).
* Ironically, the whole OAuth2 authentication process is handled by a 
third party framework. This framework would need to be part of the code 
scan described above. The framework (AppAuth) is provided by Google. I 
would therefore be scanning Googles own code. What happens if that has 
an issue ;)
* MailMate is an IMAP client and not a Gmail client. IMAP is how 
MailMate talks to a huge number of different email servers with 
different IMAP implementations. In theory, all of them could come up 
with their own sets of requirements and assessment procedures. Will I 
eventually be going through processes like this every year for every 
email provider?
* Providing an app with IMAP access to an email account means that the 
app gets total control of the emails in that account. No code 
verification process is going to change that. The user has to trust 
MailMate, and by extension, me.

That last point is interesting and I would be willing to look into ways 
that I could strengthen this trust. Using “Little Snitch” is a good 
way to monitor the behavior of an application like MailMate, but I can 
think of ways that this would not detect “evil” email client 
behavior.

It's also interesting what would have happened if I had failed to pass 
the required assessment. Google writes:

> “If you do not verify your app, your app will be subject to an 
> ongoing 100 user limit and your app’s consent screen will display 
> the unverified app user interface.”

The “unverified app user interface” would then show you this line:

> “This app hasn't been verified by Google yet. Only proceed if you 
> know and trust the developer.”

I would be fine with that in general (if it wasn't for the 100 user 
limit). With or without Google verification, it requires a lot of trust 
to let an app access your emails and I would strongly suggest that this 
is not done lightly.

---
Final rant: The worst part of this process might have been the 
assessment portal. A system which requires you to log in every time you 
need to write a message and you have to write it in a text field with a 
login timeout. You do then receive the reply by email (with a subject 
line just stating “New message posted in thread”), but you cannot 
reply to that email. You need to log in to the portal again. As an email 
client developer, it's kind of infuriating not being able to use an 
email client :)

-- 
Benny
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freron.com/pipermail/mailmate/attachments/20240608/a3efc00a/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CASA Framework?_Tier 2 Accelerator_Pivot table.csv
Type: text/csv
Size: 4899 bytes
Desc: not available
URL: <http://lists.freron.com/pipermail/mailmate/attachments/20240608/a3efc00a/attachment-0001.csv>