Skip to content

Sending (duplicate) emails to email aliases can be tricky

Published on in JavaScript and SendGrid

If you send a single email to a bunch of email addresses, and some of the addresses are aliases, i.e. they point to the same email inbox, chances are high that only one copy of the email is delivered to that inbox.

Table of contents

Skip table of contents

Example problem

  1. I had five test users and one email alias per test user; e.g.:
    • Test user 1: foo@example.com
    • Test user 2: foo+test-user-2@example.com
    • Test user 3: foo+test-user-3@example.com
    • Test user 4: custom-alias-1@example.com
    • Test user 5: custom-alias-2@example.com
    • All five email addresses pointed to the same email inbox: foo@example.com
  2. I sent a single email to those five addresses.
  3. I expected to receive five copies of the email to my email inbox, but I received only one copy.
    • In some test runs I sometimes received two or maybe even three emails, but never all five.

Test users vs real users

In my case the users were test users, making testing email delivery difficult because only some test users received the emails.

But the problem would affect real users too. Example:

  • Somebody has registered to a service multiple times, each time using a different email alias.
  • The service sends an email to all its users.
  • That somebody should receive as many emails as they have users, but (if this problem exists in that service) they might get only one email.

Tangent: SendGrid

I used the sendMultiple() method of the SendGrid Node.js API, but that's tangential:

  • This isn't a JavaScript-specific problem.
  • This isn't a SendGrid-specific problem.
  • This is a general problem of delivering multiple copies of an email to multiple email aliases.
    • The same problem would happen when sending a single email to multiple email aliases non-programmatically, e.g. directly from Gmail.

Still, I wanted to mention this because at first I thought that the problem was in SendGrid.

SendGrid's Activity Feed displayed the "Delivered" status for all emails, meaning that the receiving mail servers had successfully received the emails. Indeed, the problem wasn't that SendGrid failed to send all the emails (it didn't fail); the problem was that the receiving mail servers failed to deliver the duplicate emails to the inboxes.

The root cause: identical Message-Ids

  • Every email has a unique Message-Id.

    • I asked Fastmail Support "Is the Message-Id always created/determined by the sender?" and they replied:

      Yes, that's correct. However, if the sender doesn't specify one, due to a bug, we will create one during email delivery.

  • All emails sent in a single batch share the same Message-Id.

  • Most mail servers silently discard duplicate emails (based on Message-Id). (But there's a precondition; keep on reading.)

    • Based on my testing, this is true for at least these mail servers/providers:

      • Exchange / Outlook
      • Fastmail
      • Gmail / Google Workspace (formerly G Suite)

    • For example, Fastmail's Why messages bounce back page states the following:

      Most messages have a unique identifier called the message ID. When a message arrives at an account, the message ID is noted. If a second message with the same ID arrives, it is ignored and silently discarded. This is used to stop infinite mail loops overloading your account or our system. However, it can result in odd behavior when the same message arrives from different sources, as only the first one will actually be delivered.

      Note the last sentence:

      However, it can result in odd behavior when the same message arrives from different sources, as only the first one will actually be delivered.

      Replacing "from different sources" with "to multiple email aliases" would keep the sentence correct and describe my problem better:

      However, it can result in odd behavior when the same message arrives to multiple email aliases, as only the first one will actually be delivered.

Solution

The solution is to send the emails sequentially (one at a time), e.g. at 1-second intervals.

One way would be to make one API request per recipient, but that would be slow and inefficient because there would be as many API requests as recipients. (Then each email would probably have unique Message-Id, which alone would fix the problem.)

Eventually I found a better way (I'm using SendGrid as mentioned above): specifying different sendAt values for each recipient makes SendGrid send the emails at different times, but allows making as few API requests as possible.

Curiously, the Message-Id is still the same in every email, but this still fixes the problem! That's because the emails don't arrive simultaneously. In other words (this is the precondition mentioned above):

Most mail servers seem to not discard duplicate emails (based on Message-Id) if the emails arrive at different times (not simultaneously).

I have successfully tested this with:

  • Exchange / Outlook
  • Fastmail
  • Gmail / Google Workspace (formerly G Suite)

Fastmail Support confirmed this for their part:

So it seems we ignore duplicate emails delivered in a very short period (e.g. within a second). These emails were delivered within a second apart which is why they were not filtered as duplicates. This is expected behavior.

Code

Using SendGrid:

const emails = [
  'foo@example.com',
  'foo+test-user-2@example.com',
  'foo+test-user-3@example.com',
  'custom-alias-1@example.com',
  'custom-alias-2@example.com',
]

// SendGrid expects a UNIX timestamp in seconds
const currentTimeInMilliSeconds = Date.now()
const currentTimeInSeconds = Math.ceil(currentTimeInMilliSeconds / 1000)

sendGrid.sendMultiple({
  personalizations: emails.map((email, index) => {
    // Send the emails at 1-second intervals
    // to ensure that delivery to email aliases works too.
    // Mail servers silently discard duplicate emails
    // that arrive to the same inbox at the same time,
    // so multiple emails going to the same email inbox (via email aliases)
    // need to be sent at different times.
    // Also add a few seconds just to be safe,
    // i.e. to avoid sending the first few emails simultaneously.
    // More info: https://mtsknn.fi/blog/tricky-email-aliases/
    const sendAt = currentTimeInSeconds + index + 5

    return {
      to: email,
      sendAt,
    }
  }),
  subject: 'Test email',
  html: '<p>Email body</p>',
})

Caveat

This fix/hack will make some recipients receive the email after a small delay.

If you have 1,000 recipients and send the emails at 1-second intervals, the last recipient will receive the email after about 1,000 seconds ≈ 17 minutes. In my case that's totally OK.

If that's too long, you could send the emails in smaller batches.

The maximum is anyway 1,000 recipients per one SendGrid API request. From SendGrid's Personalizations page:

You may not include more than 1000 personalizations per API request. If you need to include more than 1000 personalizations, please divide these across multiple API requests.

Tangent: −24 hours

I stated two things above:

  • Most mail servers silently discard duplicate emails (based on Message-Id).
  • Most mail servers seem to not discard duplicate emails (based on Message-Id) if the emails arrive at different times (not simultaneously).

As a test, I scheduled the emails to be sent at 1-second intervals like before, but also 24 hours in the past. Like this:

sendGrid.sendMultiple({
  personalizations: emails.map((email, index) => {
    // ⚠️ Example, don't actually do this
    const twentyFourHoursInSeconds = 60 * 60 * 24
    const sendAt = currentTimeInSeconds + index - twentyFourHoursInSeconds

    return {
      sendAt,
      to: email,
    }
  }),
  // ...
})

I successfully received all emails – confusing!

I asked about this from Fastmail Support:

The timestamps in the Date headers are a second apart like expected; but the timestamps in the Received headers are exactly the same.

Doesn't this mean that SendGrid sent the five emails at the same time? I would expect SendGrid to do that because the Date headers were 24 hours in the past, but I can't know for sure.

And don't the identical Received timestamps also mean that you received all emails at the same time?

Their reply:

It's based on the received time at our end. Looking at the logs I can see that these emails came in almost within a second.

  • 2022-11-06T23:30:33.991668-05:00 mx2 (info) postfix-mx/cleanup[630908]: F1DBE6A017A: message-id=<a7ZrAF9NQbeVa12gUVIbRd@geopod-ismtpd-3-1>
  • 2022-11-06T23:30:33.992920-05:00 mx3 (info) postfix-mx/cleanup[725605]: F22DB1960153: message-id=<a7ZrAF9NQbeVa12gUVIbRd@geopod-ismtpd-3-1>
  • 2022-11-06T23:30:34.039558-05:00 mx1 (info) postfix-mx/cleanup[774312]: 097E023C00C6: message-id=<a7ZrAF9NQbeVa12gUVIbRd@geopod-ismtpd-3-1>
  • 2022-11-06T23:30:34.066426-05:00 mx3 (info) postfix-mx/cleanup[725548]: 100301960158: message-id=<a7ZrAF9NQbeVa12gUVIbRd@geopod-ismtpd-3-1>
  • 2022-11-06T23:30:34.291125-05:00 mx1 (info) postfix-mx/cleanup[774312]: 46EA223C0074: message-id=<a7ZrAF9NQbeVa12gUVIbRd@geopod-ismtpd-3-1>

So, they were not discarded as duplicates.

So, SendGrid sent the emails at slightly different times even though they were scheduled to be sent 24 hours ago. Or at least Fastmail received them at slightly different times.

I wouldn't rely on this behavior, so I don't recommend this approach (scheduling the emails to be sent in the past), but this is interesting to know nonetheless.

Non-solutions

I tried these two things that didn't work:

  • Custom headers:

    sendgrid.sendMultiple({
  personalizations: emails.map((email, index) => ({
    to: email,
    headers: { 'X-EmailBatchId': `${index + 1}` },
  })),
})

  • Custom subject per recipient:

    sendgrid.sendMultiple({
  personalizations: emails.map((email, index) => ({
    to: email,

    subject: `Email ${index + 1}`,
    // or
    dynamicTemplateData: { subject: `Email ${index + 1}` },
  })),
})

Rant: SendGrid Support was meh

SendGrid Support couldn't help me solve this problem.

That's kind of okay because this is quite an obscure thing and not specific to SendGrid; on the other hand, I'd expect an email delivery service provider to have great knowledge about even the most obscure things related to delivering emails.

What's worse is that one of their "Senior Technical Support Engineers" even said something factually incorrect:

I believe the root of the issue here is that multiple recipients require multiple requests, unless you're including them as CC or BCC. At some point in 2021, our system began no longer accepting multiple "TO" recipients within one request. Whereas previously, you could have 1 request with multiple recipients listed in the TO portion. Now, to send 1 email to 10 recipients individually, this requires 10 individual requests.

I suggest that you and your team look into adjusting your system's workflow so that such situations result in multiple requests, rather than using "sendmultiple" within 1 request.

I don't understand that reply as it's simply false that you can't send an email to multiple recipients with a single request.

The SendGrid docs have a section Sending the same email to multiple recipients. At the top of the page, it's mentioned that you can send an email to max 1,000 recipients with a single request. That's much better and faster than doing 1,000 individual requests.

Plus I had already succeeded at sending an email to multiple recipients with a single request; I was just having problems with email aliases / duplicate emails.

Praise: Fastmail Support is great

Fastmail Support was very helpful in helping me solve and understand the problem.

They have been helpful in the past too, when I have had email problems.