New Events API, detailed email tracking and search

We are excited to announce the release of our latest API to make life easier for developers.  Our completely refactored Events API makes it easy and fast to determine exactly what happened with each and every one of your emails.  Just like Fedex gives you a tracking number that lets you follow the entire shipping process, our events API lets you keep track of every single event associated with your email, even events that occur after the email was delivered.

New Logs

We've also updated the Logs tab in our control panel to use the Events API so you'll see more detailed information there, as well.  You will now see events that occur after the email is delivered like opens and clicks. You can also perform more detailed searches, like search for all the events that happened to a particular message by searching by message-id.

New Logs

These detailed logs are available at no additional charge for Mailgun customers. Customers using a free-trial account will receive at least 2 days of event storage, and customers with a paid account, will receive at least 30 days.

Now, onto the details of why we are so excited about the new API:

Powerful

Mailgun customers regularly send millions of emails per month, so finding the status of a specific email can be like finding a needle in a haystack. To help with this, we've added a bunch of filtering parameters and allow you to combine filters to create complex queries. We have also improved the method for polling the API. Previously, our Events API just had simple skip and limit parameters to specify which portions of the logs you wanted to fetch. This was cumbersome and resulted in poor performance. With the new Events API, we've provided parameters that allow you to specify a time period in addition to the ability to specify ascending or descending order. We also provide URLs in the response for retrieving the next and previous pages, giving you the ability to easily traverse log data, no matter how many records are returned. If the end date of the time range is not specified and the traversal is ascending (forward in time) you will eventually receive an empty page (i.e., end of the data set). You can use the same URL at a later time and the result will contain log entries written since the previous request.

Performant

We built the API to be highly performant, even across 10s of millions of log entries. To do this, we built the API on top of ElasticSearch, an open source framework for building real-time search and analytics. We will do a follow up post about the architectural details.

Below are the parameters and some examples of how to use the API.

API Parameters

Parameter Description
begin An rfc2822 time string or epoch seconds. It defines the beginning of the time range to select log records from. By default it is the time of the request.
end An rfc2822 time string or epoch seconds. It defines the end of the time range and the direction of the log record traversal. If end is less then begin then traversal is performed in the timestamp descending order, otherwise in timestamp ascending order. By default, if ascending is yes then it is a date in the distant future, otherwise a date in the distant past.
ascending Can be either "yes" or "no". It defines a direction of log record traversal.
limit The maximum number of log entries to return. The default and maximum value is 100.
pretty Can be either "yes" or "no". It defines whether results should be returned in human-readable (indented) or compact form. It is "yes" by default.
field The name or the alias of a log record field to filter by. The value of the parameter should be a valid filtering expression. Filter expressions can be found below

 

Filter Field Parameters

Parameters Description
event An event type. For a complete list of all events written to the log see the events table below.
list The email address of a mailing list the message was originally sent to.
attachment A name of an attached file.
from An email address in the "from" MIME header.
message-id A Mailgun message id returned by the messages API
subject The "subject" MIME header.
to An email addressed in the "to" MIME header.
size Message size. Mostly intended to be used with range filtering expressions (see below).
recipient An email address of a particular recipient. Even though a message may be addressed to several recipients, delivery is tracked on per recipient basis and every event pertains to only one recipient.
tags A user defined Mailgun tag added to the email.

You can create complex queries by combining parameters:

Expression Description
foo bar Matches field values that contain both term "foo" and term "bar".
foo AND bar Same as above.
foo OR bar Matches field values that contain either term "foo" or term "bar".
"foo bar" Matches field values that literally contain "foo bar".
foo -bar Matches field values that contain term "foo" but do not contain term "bar".
>10000 Matches values that greater then "10000". This filter can be applied to numeric fields only.
>10000 <20000 Matches values that are greater then "10000" and less than "20000". This filter can be applied to numeric fields only.

Here are some examples: Subject is "Only one day left to save!" AND recipient is "customer@example.com" OR "paul@example.com"

curl -s --user api:key-3ax6xnjp29jd6fds4gc373sgvjxteol0 -G \  
        https://api.mailgun.net/v2/samples.mailgun.org/events \
        --data-urlencode "subject"="Only one day left to save\!" \
        --data-urlencode "recipient"="customer@example.com OR paul@example.com"

Message size is greater than 25MB AND attachment name is "design-final.pdf" (note - message size is in bytes)

curl -s --user api:key-3ax6xnjp29jd6fds4gc373sgvjxteol0 -G \  
        https://api.mailgun.net/v2/samples.mailgun.org/events \
        --data-urlencode "attachment"="design-final.pdf"
        --data-urlencode "size"=">26214400"

Know all the events!

Previously, we only logged information that happens to your email until it is accepted by the recipient email server or dropped. The Events API allows you to query for all events, including events that happen after the email has been delivered. The full list of events is below:

Event Description
accepted Mailgun accepted the request to send/forward the email and the message was put in your queue.
rejected Mailgun rejected the request to send/forward the email.
delivered Mailgun sent the email and it was accepted by the recipient email server.
failed Mailgun could not deliver the email to the recipient email server.
opened The email recipient opened the email and enabled image viewing. Open tracking must be enabled in the Mailgun control panel and the CNAME record must be pointing to mailgun.org
clicked The email recipient clicked on a link in the email. Click tracking must be enabled in the Mailgun control panel and the CNAME record must be pointing to mailgun.org.
unsubscribed The email recipient clicked on the unsubscribe link. Unsubscribe tracking must be enabled in the Mailgun control panel.
complained The email recipient clicked on the spam complaint button and the recipient's email server provides feedback loops to Mailgun for these complaints.

 

When would I use the Events API?

 

To replace or augment our events webhooks

If your question is some version of "what happened to my email", Mailgun has always provided a simple way to answer that: webhooks. You simply configured a webhook, for the type of event you were interested in (e.g. delivered, open, click, etc), and listened for events. Each time there is an email, we fire a webhook. However, this requires you to store a lot of data on your side. Also, even though we re-try most failed webhooks, long outages in your system could result in lost data. Finally, if you are sending a lot of emails, that means a lot of hits to your endpoints which can overwhelm your server.

Our new logs API let's you answer the question "what happened to my email" using a pull method (polling the Events API) versus the push method of webhooks. 

Provide the status for a particular email

Many of our customers use email for time-critical notifications. We've blogged before about how Rackspace's Cloud Monitoring product uses Mailgun to notify customers when there are issues with their website or app. PagerDuty and Exceptional Error tracking are other well known Mailgun customers where email delivery and the ability to track that delivery is mission critical. In these cases, it is not uncommon to hear from end users: "I didn't get the email". Well, now you can just search the Events API by message-ID and get a status for each message. We will clearly tell you if the message was delivered and you can provide this status to your customer on demand, or even build it into your own UI. No more he said, she said. Just hard data.

That's the Events API in a nutshell! Feel free to check out the full documentation for more info!

Happy emailing!
Mailgunners

comments powered by Disqus

Mailgun Get posts by email

Like what you're reading? Get these posts delivered to your inbox.

No spam, ever.