Debugging Deliverability: A Real Life Example

"Hi! Our users are reporting that our emails are going to spam recently and it seems like the IP has been blacklisted, do you mind having a look?"

That's a message we received on our Discord channel last week from a customer. I thought it would be interesting to share how we tackled this issue and found the root cause. Who knows, it might help you improve your own email deliverability!

The customer shared a screenshot showing that our IP address had been blacklisted by Barracuda:

Ouch!

Now, Barracuda is a big player in the blacklist world, but landing on it isn't necessarily a catastrophe as not everyone uses it. It would particularly impact our customers who had lots of Microsoft Exchange users.

Given I knew this user had a lot of Gmail users, and that's where the complaints were coming from, my first stop in this investigation was Google Postmaster.

Google Postmaster has a handy feature where you can check the reputation of IP addresses; at Bento all of our IP addresses are listed here to monitor.

The reputation of the questioned IP was marked as "high", which usually means it should be able to deliver emails to Gmail inboxes without a hitch. Google doesn't use Barracuda's blacklist, but usually if I get a listing on Barracuda or another backlist, I'd expect to see a lower reputation score in Postmaster as they are somewhat correlated — but not the case here.

Next, I sent a few test emails from the customer's email address to myself. I noticed that SPF and DKIM weren't set up correctly, as evidenced by a glaring yellow "BE CAREFUL WITH THIS MESSAGE" banner at the top of the email. Our sending domain was also visible there (via ....)

I asked the customer to add the necessary records, thinking that would solve their problem. Afterall, Google and Yahoo were rolling out their new restrictions on email sending and I thought maybe the lack of SPF and DKIM records was the culprit.

They then sent out a resend of their last newsletter.

A few days later, they reported back that the open rate was still as low as before.

To my surprise, my test emails were also landing in spam.

Could it actually be the IP?

I checked the email performance of a handful of other customers who shared the same IP pool and had sent emails in the past 48 hours. They all had decent open rates, despite the Barracuda listing.

So, the IP didn't seem to be the culprit.

Then, I went to this users authors page (a list of addresses they were sending from in their account) and found they were sending from different subdomains and domains.

The customer was sending their newsletter from a subdomain (mail.domain.com). For all their other emails, they were sending from their primary domain (domain.com).

In the world of email marketing, each domain/subdomain combo has its own reputation. It's usually a good idea to send marketing emails from a subdomain to safeguard your primary domain. If you mess up the primary domain, you're in for a world of hurt.

I sent a few test emails from their primary domain and, voila, each one landed in the inbox using the same IP address. Great!

So, the issue seems to be isolated to this subdomain.

To quadruple-check, I switched this customer to another IP address and sent a few test emails from their subdomain to myself. Same result: all emails still landed in spam. This confirmed that the subdomain was the problem.

I asked the customer to add this subdomain to Google Postmaster. Unfortunately, Google doesn't aggregate subdomains in their reports, so each one you send from needs to be added separately.

And there it was:

The subdomain was marked as "bad".

That was the root cause of the deliverability issues with Gmail.

So, why did this happen?

My theory is that given Google has been implementing more stringent measures, the customer's email list might have contained a bunch of bad emails (spam traps, dead inboxes, etc) so when they sent out their newsletter, they crossed the daily threshold for negative signals. This dropped the reputation of the subdomain leading to the "bad" marking in Postmaster and thus the emails were all flagged as spam.

The solution? I've asked them to segment out their active users (i.e., users who logged in the last 30 days), and send their future newsletters at a slower pace (maybe over a few days instead of within a few hours). Once Google sees you're a good actor, your reputation will gradually improve from bad to medium to high within 90 days.

This customer could also try spinning up a new subdomain and sending from there but be warned, if you mess up with that one, you risk having your entire domain flagged which is far from ideal.

At the end of the day, debugging deliverability issues is a process of trial and error. It's also fairly black and white, so you'll need to test thoroughly to figure out the real cause to not landing in the inbox.

If you find you're having deliverability issues, regardless if you're a Bento customer or not, please reach out to support!