Mailgun and Heartbleed

Heartbleed

Earlier this week, we were alerted to a security vulnerability in OpenSSL [1] [2]. While this vulnerability was not in Mailgun code, Mailgun does use the OpenSSL library to secure HTTPS connections to our servers, so we were susceptible to it as well. In security parlance, Heartbleed was an arbitrary read vulnerability allowing the attacker to read 64kb of memory out of the affected processes memory space. What made this bug particularly treacherous was that it left no trace and allowed the attacker to gain access to private keys effectively defeating the TLS security mechanism.

Mitigation

Once we found out about Heartbleeed we analyzed where our infrastructure was vulnerable and patched the affected servers. The architecture of Mailgun allows us to terminate all SSL/TLS connections when they enter our data center; thus we only had a few servers to patch. This work was completed by Tuesday, April 8th, 2014 at 1:00 PM PDT.

Once the patch was rolled out, we also updated our certificates to ensure that even if our private keys were stolen, they could no longer be used.

If you want to verify yourself, you can build and run a script called Heartbleed or use their web application to check the security of Mailgun or any other site for the Heartbleed vulnerability [3] [4].

Customer Impact

Since this vulnerability leaves no trace and allowed the attacker to arbitrarily read memory of our nginx processes, your API key and SMTP credentials may have been compromised. While the likelihood of such an attack having occurred is low, we recommend all customers regenerate their API key and SMTP credentials (you can do this from the Control Panel) to be on the safe side [5].

If you have any other questions, feel free to drop us an email at support@mailgun.com.

[1] http://heartbleed.com [2] https://news.ycombinator.com/item?id=7548991 [3] https://github.com/FiloSottile/Heartbleed/ [4] http://filippo.io/Heartbleed/ [5] https://mailgun.com/cp