New Events API, detailed email tracking and search
We are excited to announce the release of our latest API to make life easier for developers. Our completely refactored Events API makes it easy and fast to determine exactly what happened with each and every one of your emails. Just like Fedex gives you a tracking number that lets you follow the entire shipping process, our events API lets you keep track of every single event associated with your email, even events that occur after the email was delivered.
We've also updated the Logs tab in our control panel to use the Events API so you'll see more detailed information there, as well. You will now see events that occur after the email is delivered like opens and clicks. You can also perform more detailed searches, like search for all the events that happened to a particular message by searching by message-id.
These detailed logs are available at no additional charge for Mailgun customers. Customers using a free-trial account will receive at least 2 days of event storage, and customers with a paid account, will receive at least 30 days.
Now, onto the details of why we are so excited about the new API:
Mailgun customers regularly send millions of emails per month, so finding the status of a specific email can be like finding a needle in a haystack. To help with this, we've added a bunch of filtering parameters and allow you to combine filters to create complex queries. We have also improved the method for polling the API. Previously, our Events API just had simple skip and limit parameters to specify which portions of the logs you wanted to fetch. This was cumbersome and resulted in poor performance. With the new Events API, we've provided parameters that allow you to specify a time period in addition to the ability to specify ascending or descending order. We also provide URLs in the response for retrieving the next and previous pages, giving you the ability to easily traverse log data, no matter how many records are returned. If the end date of the time range is not specified and the traversal is ascending (forward in time) you will eventually receive an empty page (i.e., end of the data set). You can use the same URL at a later time and the result will contain log entries written since the previous request.
We built the API to be highly performant, even across 10s of millions of log entries. To do this, we built the API on top of ElasticSearch, an open source framework for building real-time search and analytics. We will do a follow up post about the architectural details.
Below are the parameters and some examples of how to use the API.
|begin||An rfc2822 time string or epoch seconds. It defines the beginning of the time range to select log records from. By default it is the time of the request.|
|end||An rfc2822 time string or epoch seconds. It defines the end of the time range and the direction of the log record traversal. If end is less then begin then traversal is performed in the timestamp descending order, otherwise in timestamp ascending order. By default, if ascending is yes then it is a date in the distant future, otherwise a date in the distant past.|
|ascending||Can be either "yes" or "no". It defines a direction of log record traversal.|
|limit||The maximum number of log entries to return. The default and maximum value is 100.|
|pretty||Can be either "yes" or "no". It defines whether results should be returned in human-readable (indented) or compact form. It is "yes" by default.|
|field||The name or the alias of a log record field to filter by. The value of the parameter should be a valid filtering expression. Filter expressions can be found below|
Filter Field Parameters
|event||An event type. For a complete list of all events written to the log see the events table below.|
|list||The email address of a mailing list the message was originally sent to.|
|attachment||A name of an attached file.|
|from||An email address in the "from" MIME header.|
|message-id||A Mailgun message id returned by the messages API|
|subject||The "subject" MIME header.|
|to||An email addressed in the "to" MIME header.|
|size||Message size. Mostly intended to be used with range filtering expressions (see below).|
|recipient||An email address of a particular recipient. Even though a message may be addressed to several recipients, delivery is tracked on per recipient basis and every event pertains to only one recipient.|
|tags||A user defined Mailgun tag added to the email.|
You can create complex queries by combining parameters:
|foo bar||Matches field values that contain both term "foo" and term "bar".|
|foo AND bar||Same as above.|
|foo OR bar||Matches field values that contain either term "foo" or term "bar".|
|"foo bar"||Matches field values that literally contain "foo bar".|
|foo -bar||Matches field values that contain term "foo" but do not contain term "bar".|
|>10000||Matches values that greater then "10000". This filter can be applied to numeric fields only.|
|>10000 <20000||Matches values that are greater then "10000" and less than "20000". This filter can be applied to numeric fields only.|
Here are some examples: Subject is "Only one day left to save!" AND recipient is "email@example.com" OR "firstname.lastname@example.org"
curl -s --user api:key-3ax6xnjp29jd6fds4gc373sgvjxteol0 -G \ https://api.mailgun.net/v2/samples.mailgun.org/events \ --data-urlencode "subject"="Only one day left to save\!" \ --data-urlencode "recipient"="email@example.com OR firstname.lastname@example.org"
Message size is greater than 25MB AND attachment name is "design-final.pdf" (note - message size is in bytes)
curl -s --user api:key-3ax6xnjp29jd6fds4gc373sgvjxteol0 -G \ https://api.mailgun.net/v2/samples.mailgun.org/events \ --data-urlencode "attachment"="design-final.pdf" --data-urlencode "size"=">26214400"
Know all the events!
Previously, we only logged information that happens to your email until it is accepted by the recipient email server or dropped. The Events API allows you to query for all events, including events that happen after the email has been delivered. The full list of events is below:
|accepted||Mailgun accepted the request to send/forward the email and the message was put in your queue.|
|rejected||Mailgun rejected the request to send/forward the email.|
|delivered||Mailgun sent the email and it was accepted by the recipient email server.|
|failed||Mailgun could not deliver the email to the recipient email server.|
|opened||The email recipient opened the email and enabled image viewing. Open tracking must be enabled in the Mailgun control panel and the CNAME record must be pointing to mailgun.org|
|clicked||The email recipient clicked on a link in the email. Click tracking must be enabled in the Mailgun control panel and the CNAME record must be pointing to mailgun.org.|
|unsubscribed||The email recipient clicked on the unsubscribe link. Unsubscribe tracking must be enabled in the Mailgun control panel.|
|complained||The email recipient clicked on the spam complaint button and the recipient's email server provides feedback loops to Mailgun for these complaints.|
When would I use the Events API?
To replace or augment our events webhooks
If your question is some version of "what happened to my email", Mailgun has always provided a simple way to answer that: webhooks. You simply configured a webhook, for the type of event you were interested in (e.g. delivered, open, click, etc), and listened for events. Each time there is an email, we fire a webhook. However, this requires you to store a lot of data on your side. Also, even though we re-try most failed webhooks, long outages in your system could result in lost data. Finally, if you are sending a lot of emails, that means a lot of hits to your endpoints which can overwhelm your server.
Our new logs API let's you answer the question "what happened to my email" using a pull method (polling the Events API) versus the push method of webhooks.
Provide the status for a particular email
Many of our customers use email for time-critical notifications. We've blogged before about how Rackspace's Cloud Monitoring product uses Mailgun to notify customers when there are issues with their website or app. PagerDuty and Exceptional Error tracking are other well known Mailgun customers where email delivery and the ability to track that delivery is mission critical. In these cases, it is not uncommon to hear from end users: "I didn't get the email". Well, now you can just search the Events API by message-ID and get a status for each message. We will clearly tell you if the message was delivered and you can provide this status to your customer on demand, or even build it into your own UI. No more he said, she said. Just hard data.
That's the Events API in a nutshell! Feel free to check out the full documentation for more info!