Design Email
Topics
How to send email?
How to send email in real time?
How to store attachments?
How to scale websocket service?
Check spams?
Search emails?
Functional Requirements:
User can send and receive emails.
User can receive email in real time?
User can attach files in an email.
Non-functional requirements:
Low latency
Highly available
API
v1/send POST
json {
uid
email title
recipients
email body
}
response: 201 OK
v1/emails GET
json {
uid
page_number,
page_size
}
Response:
[
Email1: {},
Email2: {},
...
]
websocket API for getting push notification and auto refresh of email list.
/connect
/email
data: Data Schema
User {
uid: uuid,
name: string,
email: string,
profile_url: string,
}
Email {
id: uuid,
uid: uuid,
recipients: []string,
subject: string,
attachments: []string, // files stored in file storage
body: TEXT,
}High Level Design
A user write email on webmail and press send button. The request is sent to load balancer.
The load balancer makes sure it doesn't exceed rate limit and routes traffic to web servers.
Web servers responsible for:
Basic email validation, like email size limit.
Check if domain of recipient email is the same as sender. If it is the same, the web server ensures the email data is spam and virus free. If so, email is stored in sender's sent folder and inbox folder directly without sending to outgoing queue.
Message queues
If basic email validation succeeds, the email data is passed to the outgoing queue.
If basic email validation fails, the email is put in the error queue.
SMTP outgoing workers pull messages from the outgoing queue and make sure emails are spam and virus free.
The outgoing email is stored in the sent folder of storage.
SMTP outgoing workers send the email to recipient mail server.
Email deliverability & Spam
Dedicated IPs:
Recommended to have dedicated IP addresses for sending emails. Email providers are less likely to accept email from new IP addresses that have no history.
Classify emails
Send different categories of emails from different IP addresses. For example, you may want to avoid sending marketing and other important emails from the same servers because it might make ISPs mark all emails as promotional.
Email sender reputation
Warm up new email server IP addresses slowly to build a good reputation. so big providers like Office365, Gmail and Yahoo Mail are less likely to put our emails in the spam folder. takes 2-6 weeks to warm up new IP address.
Ban spammers quickly
Spammers should be banned quickly before they have a significant impact on server's reputation.
Feedback processing
It's very important to set up feedback loops with ISPs so we can keep complaint rate low and ban spam quickly.
Hard bounce: an email is rejected by ISP because the recipient's email is invalid.
Soft bounce: A soft bounce indicates an email failed to deliver due to temporary conditions, such as ISPs being too busy.
Complaint: A recipient clicks the report spam button.
Search
Google search
The whole internet
Sort by relevance
Indexing generally takes time, so some items may not show in the search result immediately.
Email search
User's own email box
Sort by attributes such as time, has attachment, date within, is unread, etc.
Indexing should be near real-time, and the result has to be accurate.
Option 1: Elastic search
Option 2: Custom search solution using LSM
Email 101
SMTP:
Simple Mail Transfer Protocol (SMTP) standard protocol for sending emails from one email server to another.
POP:
standard mail protocol to receive and download emails from a remote mail server to a local email client. Once emails are downloaded to your computer/phone, they are deleted from email server.
IMAP:
standard mail protocol for receiving emails for a local email client. When you read an email, you are connected to an external mail server, and data is transferred to your local device. IMAP only downloads message when you click it, and emails are not deleted from mail servers.
Last updated