Tejas Headshot

Tejas Kumthekar

PostgreSQL [announced](https://www.postgresql.org/about/news/postgresql-14-released-2318/) their latest version (PostgreSQL 14) on September 30th, which includes a bunch of features like pipeline API, gathering statistics on replication slots, query parallelism improvements and so on. While the [origin](https://www.postgresql.org/about/) of PostgreSQL can be traced back to 1986, it has been in active development for the past 30 years. Tons of companies, agnostic of the the type and size, have trusted Postgres over the years and their tagline “The world's most advanced open source relational database” is hardly an overstatement.

In a typical, non-trivial system, there is more than one database server, and the data is often copied across multiple servers. A fairly common notion in a distributed system, this copying of data across multiple server nodes is known as replication. Replication processes are often abstracted from the consumers of the database, so most of the application layers are transparent to which node serves the data and if you’re an app developer like me, we expect things to “just work”. However, what goes behind the scenes of some of the most sophisticated databases of the world is often a fascinating reading and learning experience. 

In this post, we will start diving into the internals of Postgres to understand how replication works and data integrity is ensured using WAL (Write-Ahead Logging). We will then steer towards interesting concepts like logical decoding and output plugins. Finally, we will start hacking some code to write our own plugin that can send push notifications! I know this sounds a bit contrarian to the common approach of sending notifications from application servers, but hey what's the fun in doing common boring things! Let's dive right in :)

Replication at a high level is a process of transferring data from a primary server to a replica server. In Postgres lingo, servers can either be primary or stand-by. Primary servers are the ones that send data, while standbys are the receivers of replicated data. In certain settings, standbys can also act as senders. Replica or standby servers can take over if the primary fails which is the crux of how database systems manage fault tolerance.

![hacking-postgres-1](//images.ctfassets.net/z7iqk1q8njt4/umbekjjItC7BvTPRbw7up/4428b903f180b404208738310f9cb949/hacking-postgres-1.png)
[https://www.enterprisedb.com/postgres-tutorials/postgresql-replication-and-automatic-failover-tutorial#replication](https://www.enterprisedb.com/postgres-tutorials/postgresql-replication-and-automatic-failover-tutorial#replication)

WAL (Write-Ahead Logging) is the standard method to ensure data integrity in Postgres. Systems are bound to fail and database servers are not excused from that. At the application layer, it's comparatively easier to retry and manage failures without typically resulting in data loss, however, when we go deep in the stack, especially at the data layer, persistence is doubtlessly super critical. When consumers make a write operation to the database, before the database is acted-upon, the changes are written to the server’s file system. This by design ensures data persistence and recovery in the cases of operating system crashes or hardware failures. Consumers are (obviously) abstracted from these internal mechanisms and applications connected to the Database expect things to work out of the box. 

The log entry we talked about above is known as the Write-Ahead Log record while the process is called Write-Ahead Logging. Each record has a Sequence number which is used for checkpointing periodically after logs are synchronized to the database. In cases of system crashes, this checkpoint is used to re-read and synchronize. WAL, being a reliable method for synchronizing or recreating the ordered state of the database, is used for replication across multiple servers. WAL can replicate the data using either a File based approach or a Streaming approach where both approaches have their own pros and cons. Streaming replication, although is typically in an asynchronous mode, can be tuned to be synchronous.

![hacking-postgres-2](//images.ctfassets.net/z7iqk1q8njt4/1lsDwMGtrp3KcJK48HkdNV/e17d361ef1604ad97cdbd07ea6cb8a17/hacking-postgres-2.png)
[https://hevodata.com/learn/postgres-wal-replication/](https://hevodata.com/learn/postgres-wal-replication/)

WAL records being representative of the internal state of the Database system, are not easy to be fed into or understood by an external system/consumer. Logical Decoding to the rescue! “[Logical decoding](https://www.postgresql.org/docs/9.4/logicaldecoding-explanation.html) is the process of extracting all persistent changes to a database's tables into a coherent, easy to understand format which can be interpreted without detailed knowledge of the database's internal state.” Using logical decoding, replication solutions and auditing can be achieved much easily.

The following diagram depicts the logical decoding process.

![hacking-postgres-3](//images.ctfassets.net/z7iqk1q8njt4/4LhsYtcGoveivrzRFEmmwd/195ea94dfa863f882d6ff23957f09a12/hacking-postgres-3.png)
Source: [https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/change-data-capture-in-postgres-how-to-use-logical-decoding-and/ba-p/1396421](https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/change-data-capture-in-postgres-how-to-use-logical-decoding-and/ba-p/1396421)

In order to enable the logical decoding, you need to make some configuration changes to the Postgres instance -

```
wal_level = logical # default value is `replica`
max_replication_slots = 1 # good enough for a sample project
max_wal_senders = 1 # default is 10
```

Ref: [https://www.postgresql.org/docs/current/runtime-config-replication.html](https://www.postgresql.org/docs/current/runtime-config-replication.html) and [https://www.postgresql.org/docs/current/runtime-config-wal.html](https://www.postgresql.org/docs/current/runtime-config-wal.html) for details on tuning the configuration

Once we have the configuration up and running, record changes are passed to the Output Plugin which does the key step of transformation from the WAL format to the format specified in the plugin (eg. JSON). These changes are made available on the replication slot(s) and consumer applications can receive the stream of updates as and when those occur.

Some output plugins and consumer apps out there -

[wal2json](https://github.com/eulerto/wal2json) Output Plugin that converts WAL output to JSON objects [Open Source]
[pg_recvlogical](https://www.postgresql.org/docs/current/app-pgrecvlogical.html)  Postgres app that can consume update stream [Out-of-the-box with Postgres]
decoderbufs Output Plugin that delivers data as protobuf [Open Source, Used in [Debezium](https://github.com/debezium/debezium)]

We can write a consumer from scratch or use battle-tested tools built to achieve this at scale. Debezium is one of the most widely used solutions out there. Netflix open sourced their in-house tool for CDC called DBLog - [https://netflixtechblog.com/dblog-a-generic-change-data-capture-framework-69351fb9099b](https://netflixtechblog.com/dblog-a-generic-change-data-capture-framework-69351fb9099b) is a fantastic read on it.

What are we building? Lets keep this simple - whenever a new user entry is made to Postgres table, we will consume replication logs and send a welcome email to the user.

Let’s hack some code! We will be using a Go package [https://github.com/jackc/pglogrepl](https://github.com/jackc/pglogrepl) which is a Postgres logical replication library.

### Step 1: Follow Logical replication instructions
[https://github.com/jackc/pglogrepl](https://github.com/jackc/pglogrepl) README has step by step instructions to configure logical replication in your local Postgres Instance. 

### Step 2: Try out the demo in the repo
Give [https://github.com/jackc/pglogrepl/tree/master/example/pglogrepl_demo](https://github.com/jackc/pglogrepl/tree/master/example/pglogrepl_demo) a try - it demos how the library works under the hood

### Step 3: Update the demo code to log new email inserts
To keep it simple for this blog post, let's add some code that prints out specific email which was inserted on the new row.

#### A.  Look for inserts and print the WAL data

```
if logicalMsg.Type() == 'I' {
   // `I` stands for Insert
   log.Println(string(xld.WALData))
   // this logs the complete entry
   // however it would require a little more cleanup
}
```

For example, a new entry with ID 5 and email tejas@courier.com shows up as I@Nt5ttejas@courier.com

#### B. Clean up the internal WAL representation and derive email

```
// find the second column
  walData := xld.WALData[5:]
  pos := bytes.Index(walData, []byte("t"))

  email := walData[pos+1:]
  pos = bytes.Index(email, []byte("t"))

  email = email[pos+1:]
  emailStr := string(email)
```

#### C. Send a welcome email to the user

We will be creating a new template by configuring an email integration supported by Courier.

```
messageID, err := client.Send(context.Background(), "VDPE8SWN1K4BWMP8RJ101YRZTF3J", "user-id", 
courier.SendBody{
   	Profile: profile{
   	   Email: emailStr,
   	},
   	Data: data{
   	   Foo: "bar",
   	},
})

if err != nil {
  log.Fatalln(err) 
}

log.Println(messageID)
```

Find the modified code at [https://github.com/tk26/pglogrepl](https://github.com/tk26/pglogrepl) Here’s a quick [loom video](https://www.loom.com/share/f96c687c4aa648b7822464c1601e17ee) of how things work.

Now that we have configured Postgres to send emails by listening to replication logs, changing it to send a push notification would be as simple as changing a configuration in the Courier Studio.

![hacking-postgres-4](//images.ctfassets.net/z7iqk1q8njt4/3VHoF3ofVDXTXHwXtWKAUg/3f59344b1a44e1c12cbc5500d7c13f9e/hacking-postgres-4.png)

Thanks for reading! Reach out to us [@trycourier](https://twitter.com/trycourier) if you have any questions or comments or you know, just say Hi!

placeholder-header

This post dives into the internals of Postgres replication, WAL, and logical decoding. We will also write our own plugin to send push notifications.

placeholder-thumbnail

Hacking PostgreSQL Internals to Deliver Push Notifications

Riley Profile Picture

Riley Napier

An internal hackathon is an effective way to engage your employees in short-term collaborative events. It [spurs innovation](https://blog.google/inside-google/life-at-google/unlock-your-teams-creativity-running-great-hackathons/#:~:text=A%20hackathon%20shifts,exploring%20that%20occurs.), increases employee engagement, explores new feature possibilities, imparts corporate culture, and builds teamwork and collaboration skills. By organizing internal hackathons in your company, your employees can challenge and compete to solve internal or customer problems—which can lead to real-life features in your product.

[Courier’s](https://www.courier.com/) internal hackathons have not only led to a better product and smoother customer experiences, but they have also engaged employees and promoted teamwork. Some of our hackathon projects that made it to production demonstrate the value of giving teams the time and space to innovate this way.

We do our best to support these hackathons and create the time and space for them, so these events don’t distract employees from their day-to-day work. Our employees, in turn, appreciate the internal hackathons because they get to explore all their ideas and opinions that they don’t have time to work on in their day-to-day jobs. We reflect our culture in our internal hackathons, and, in return, some of the projects make it to production as regular Courier features.

## Internal Hackathons Represent Our Values at Courier
Our hackathons highlight our [core values](https://www.courier.com/about)—“have an opinion,” “just f-cking do it,” and “talk to customers.” During these hackathons, team members turn their opinions into experiments, challenge each other, and solve problems. We even have friendly competitions and push each other to win, promote teamwork by sharing Loom videos and presentations, celebrate innovation, etc.

We also inform the product, sales, and customer service teams about these projects and talk to our existing customers. In the case when a customer shares similar ideas, they quickly chip in: “Oh, we’ve done this in a hackathon before. Let me sync with the engineer and see what it takes to get it production-ready.”

We host our hackathons twice a year and dedicate a full day each time. Employees are split into two to three technical and non-technical members on a team. Teams record their project demos using Loom, and on the hackathon day, teams give their presentations while the judges watch and vote on the best hackathon project.

## Two Examples of a Hackathon Project Becoming a Live Feature
As an engineer at Courier, you feel the pain of some customers. You understand the features they would benefit from but have not yet scoped out technically, or you have no time to build them. Hackathons give you the time and space to experiment.

Sometimes, this experiment hits home with a common pain point, and we ship it or put it into production. Other times, it is just a fun project. Let’s look at two hackathon projects at Courier that made it to production.

### Embedded Components Project Turned “Toast and Inbox” Feature
For our very first hackathon, my team experimented with the idea of embedding Courier functionalities in our clients’ websites to give them a better user experience. These components included features to provide an inbox, notification preferences, and a list of recent messages received for our customers.

For the hackathon, we built an embedded preferences component to allow end-users (our clients’ customers) to control notification preferences. This allowed our clients to let their customers control and manage their preferences. We presented our project and eventually won the first hackathon.

<div style="position: relative; padding-bottom: 62.5%; height: 0;"><iframe src="https://www.loom.com/embed/bc61ef77561148b381debd4b7d64ada5" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>

Building on top of this, for the second hackathon, my team pushed the idea further to another embedded component to improve the customer experience. For this second hackathon project, we noticed there were a lot of clients with multiple brands asking, “Hey, I love your notification designer, but I need to let my other brands design and change emails, logos, fonts, etc. We want to do that easily without managing every one of them.” So, we built a solution to embed Courier brand designers in an arbitrary website that will allow our clients to update their branding and edit their content seamlessly.

<div style="position: relative; padding-bottom: 62.5%; height: 0;"><iframe src="https://www.loom.com/embed/bb48ca0bace944e3b0f9d492523d2caa" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>

After the first two hackathons, we started getting questions from customers wanting to create an inbox to deliver messages to their end-users. So, we looked at the infrastructure of these embedding components’ projects and built [react components for Toast and Inbox](https://www.courier.com/blog/react-toast-inbox-notifications).

Because hackathon projects are not production-ready code, we use the codebase as a blueprint to build out the feature. The timeline of taking a hackathon project to a feature depends on the functionality. Some are relatively easy, while some are difficult depending on the project. Specifically, “Toast and Inbox” was built on top of the technology used for the hackathon but had so many more functionalities, so it took some time to get it out.

### Column Block Project Turned “Column Element” Feature
At Courier, I am focused on the design and UI side of things—I built a lot of it from scratch. For the third hackathon, my team got creative with our notification block. We wanted to experiment with a new block that would benefit our customers in a different way.

With the version of the notification designer at that time, you could only make a list of blocks (up and down) but could not put blocks side by side. So, taking some inspiration from [SendGrid](https://sendgrid.com/) Designer, we created an idea to make two blocks together in our notification designer. Although it was pretty buggy, we got the project out and presented it at the hackathon.

<div style="position: relative; padding-bottom: 62.5%; height: 0;"><iframe src="https://www.loom.com/embed/4c6c7ffc69e244bcba13817bd2167ae0" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>

Six months later, [UserVoice](https://uservoice.com/) wanted to sign in with Courier and mentioned we lacked some functionalities they wanted. Specifically, they wanted to put two blocks next to each other in the notification designer. So, the sales and product team reached out to me, and I went, “Oh yeah, I hacked that together; it was cool but with a few bugs.”

We had to go back, resurface that code, cut out the buggy scope, clean it up, and build it. But we were able to put it into production in a week and signed the contract with UserVoice.

## Moving to Developer-Focused Hackathons in the Future
Hackathon projects have helped us stay centered around a better product for our clients’ customers—which is great. In the future, we are shifting to focus these hackathons on improving the developer experience at Courier.

If our hackathon and processes inspire you, we have some open roles on our [careers page](https://jobs.lever.co/trycourier). Come work with us!


A boy builds toy wooden planes that take to the sky and become real airplanes

internal-hackathon-features-header

With internal company hackathons, employees can solve internal or customer problems, which can lead to real-life features in your product.

internal-hackathon-features-thumbnail

How We Use Internal Hackathons to Create New Product Features

nick-headshot

Nick Gottlieb

As modern applications have become increasingly feature-rich and performant, user expectations are at an all-time high. Failure to communicate key information or displaying out-of-date information frustrates users and causes a loss of trust. Think about the last time you ordered something online: if you didn’t receive your order confirmation within seconds, you probably began to worry that something went wrong. Users require product transparency for consistent use.

Luckily for developers, a proliferation of new APIs have been developed to help provide features such as real-time order update emails, push notifications based on in-app activity, or live chat with sales and support. Communications APIs are becoming increasingly specialized, with each type being more or less suited to certain tasks.

If you’re an engineer who’s been tasked with planning out your application’s communication strategy, this post will help you map out the landscape. You’ll come away understanding the three core types of user communication APIs and in which circumstances you should use them to create the best possible end-user experience. We’ll also show you how using the right APIs in the right places will streamline your own day-to-day work.

Let’s get into it! 

## Planning Your Communications API Strategy

The best outcome any engineer can hope for is that your communications strategy is so smooth, it isn’t even noticed. For the end user, it just works exactly when it’s supposed to work, the way it’s supposed to work. For your own internal teams, everyone has the ability to craft the messaging they need and send it when they need, without unnecessary bottlenecks that can often impact your own engineering time.

In a marketplace application, for instance, users need to get an automated push notification or email every time someone comments on an item they’re selling. The marketing department also wants to send email campaigns to everyone who has purchased an item in the past six months to let them know there’s a sale coming up. When a buyer’s checkout fails, they might need to message your sales or support team to get help and complete the transaction. 

All of these are very different examples that will need to use a different type of communications API to get the job done.

## Exploring The Three Types

Communications APIs are segmented not only by the functionality they provide and how immediately they need to respond to user actions but also often by which teams at the company will need to use them to send messaging. 

App-to-user APIs are highly customizable, instant communications sent by engineering as integral parts of the application experience, such as push notifications about likes or comments. Brand-to-user APIs collect data and give the company, particularly marketing and product teams, the ability to send users targeted messages individually or en masse. User-to-user APIs facilitate live chat either among app users or with someone at your company, such as sales or support.

Let’s discuss the strengths and weaknesses of the three types of APIs, and see some commonly-used third-party providers for each.

## App-to-user APIs

App-to-user APIs are transactional messages that are embedded into the application code. These APIs are often the most mission-critical of the three, as they form a core part of the application experience. For instance: your rideshare has arrived, your order was received, you have just been added to this chat room, here is your one-time password for login. These types of communications tend to be time-sensitive and are designed to send immediately when the trigger events occur.

A robust App-to-user API will provide multi-channel communication (email, SMS, and push notification), complex routing, and easy opt-in and opt-out for users. Most can be heavily automated as well. For instance, when a new user is created via Auth0, a welcome push notification can be automatically sent.

App-to-user APIs are typically instrumented and used by engineering teams. Companies who have the resources may choose to build their own App-to-user API infrastructure, which we have [a guide on how to do here](https://www.courier.com/blog/the-developers-guide-user-requirements). Courier also provides a ready-to-use App-to-user API infrastructure. 

## Brand-to-user APIs

Brand-to-user communication happens any time your company has a message for a specific group of users. Maybe the product team noticed that a user got partway through the sign-up flow but didn’t hit the “complete” button. Maybe marketing wants to tell everyone in New York about an upcoming in-person event.

Braze and Customer.io are two examples of out-of-the-box Brand-to-user APIs. Engineering implements API calls in the application code that log information about user behavior, which then gets sent to a separate dashboard other teams at the company can access. Anyone on those teams can then use that data to craft and send messages to specific subsets of users. For instance, users who didn’t complete the sign-up flow, or who live in New York. Brand-to-user APIs save engineering time because, once the initial API calls are instrumented, any team who needs to send campaigns or needs to access that data can do so easily and independently.

While Brand-to-user APIs make it easy for anyone at the company to send messaging to users, they typically aren’t configurable enough to power core parts of the application experience. Messages can go out minutes to hours after user criteria for the message have been met. There can also be implementation delays any time you want to send messages based on a different set of criteria than you were using before—new API calls in the app mean the data needs time to populate before they can be used. 

While this trade-off is often worthwhile for marketing and product communications, App-to-user APIs (discussed above) are a much better fit when you need fine control over messaging content or timing within engineering.

## User-to-user APIs

User-to-user communication APIs facilitate active conversations among two or more individuals—either between a user and someone on your own internal staff or from one app user to another. 

Support tickets, for instance, are a type of User-to-user API, as are live chat boxes that let customers have direct conversations with your support or sales teams. Likewise, two users in a marketplace might need to message each other to offer bids or send along shipping information. While these are possible services to build out in-house, most companies opt to save time by going with a ready-to-use SaaS solution, such as Intercom, Zendesk, or Sendgrid.

## Conclusion

Most applications will use more than one type of user communications API for different parts of the user experience. Your application may very well use all three! 

App-to-user APIs provide part of the application’s core functionality. These APIs are highly customizable and provide instant user communication that is tightly coupled with your application code. Marketing and product teams will often want a Brand-to-user API implementation that makes it easier to see user data and send updates to users. User-to-user APIs give customers access to back-and-forth communication either among each other, or with company support or sales teams.

When you’re thoughtful about your communications strategy, you can create a seamless experience for users while also enabling everyone on your team to save time and do their best work. 

Read on for more information on building user communication API infrastructure:

* [The Developer's Guide to Building Notification Systems: User Requirements](https://www.courier.com/blog/the-developers-guide-user-requirements)

* [How to Set Up Multi-Channel Notifications in Your AWS Stack](https://www.courier.com/blog/multi-channel-notifications-aws)

*Illustration by [Rebekka Dunlap](https://rrebekkaa.com/)*


Three different people using a computer, tablet, and smartphone look up as magic surrounds them indicating the ease of communication through apis.

user-communication-apis-header

This post describes three core types of user communication APIs and in which circumstances you should use them to create the best possible end-user experience.

user-communication-apis-thumbnail

3 Types of User Communication APIs and When to Use Them

shreya-headshot

Shreya Gupta

At your CTO’s request, you recently started researching how to go about revamping or building your product’s notification system. You realized the complexity of this project around the same time as you discovered that there’s not a lot of information online on how to do it. Companies like LinkedIn, Uber, and Slack have large teams working just on notifications, but smaller companies like yours don’t have that luxury. So how can you meet the same level of quality with a team of one? This is the fourth and final post in our series on how you, the developer, can build or improve your company’s notification system. It follows the first post about identifying [user requirements](https://www.courier.com/blog/the-developers-guide-user-requirements/), the second about designing with [scalability and reliability](https://www.courier.com/blog/scalability-and-reliability/ "scalability and reliability") in mind, and the third about setting up [routing and preferences](https://www.courier.com/blog/routing-and-preferences/). In this piece, we will learn about using observability and analytics to set your system and company up for success.

Developing an application can often feel like you're building in the dark. Even after development, gathering and organizing performance data is invaluable for ongoing maintenance. This is where observability comes in—it’s the ability to monitor your application’s operation and *understand* what it’s doing. With close monitoring, observability is a superpower that allows developers to use various data points to foresee potential errors or outages and make informed decisions to prevent these from occurring.
As you build your product, consider the implications of having complete observability built into your notification system. As a developer, you’ll need to identify and quickly resolve issues by understanding how your product is performing. In the bigger picture, observability ties your technological infrastructure to your overarching product and business objectives. These key insights will also help to scale the product and manage data as your business grows. 

## Observability Use Cases
You’re here because you want to build an application with a powerful notification system that can rival those of existing products. In this guide, you’ll learn why observability and building strong monitoring mechanisms are crucial. Here are four core observability use cases.

1. Track and use logs for debugging
2. Improve customer service and experience
3. Holistic view of your product
4. Analytics for business development

### Logs for debugging
Telemetry logs are the backbone of an observability system. The more infrastructure you have, the more data there will be from each instrument and service. You need to be able to understand this data. Logs can provide additional context that allows developers to determine where or why certain issues might be occurring, and effectively, how to fix them. For example, if every API request to your notification system results in a specifically formatted log line, it becomes possible to scan those log lines for anomalies. Event logs of particular actions occurring in the system, like privileged access or settings changes, can sometimes shed light on unpredicted behaviors in the system. At the very least, you should have some kind of safety net or global catch to notify you when errors creep up. 

When users are unable to receive notifications, you can use various logs to determine what factor(s) prevented that notification from going through. If, for example, your app is unable to deliver messages to 20% of your users, logs would reveal that those same 20% of users are using your application on a specific device type. You’ll know right away that your application has a bug that prevents it from functioning properly on that device type and act accordingly. You can also update your system to prevent this issue from occurring for future users.

### Customer service
Let’s say a user has contacted you to report that they have unsubscribed from emails but is still receiving them. Your customer care team should be able to note relevant logs or errors, communicate with the user to ensure a good relationship, and also feed that information to the developer team for potential resolution. 

Observability can also help you improve overall user experience, present and future. As you consider important data points to observe within your system, think about what might impact your product, *specifically* through the lens of your customers. If you can see some metric that might impact user experience, work to improve it before it’s an issue. For example, do your time-sensitive notifications get delivered as quickly as they should? Are all messages being delivered only once? If you’re using multiple channels, are messages being routed correctly?

Additionally, if you’re using a third-party provider like SendGrid or AWS SES, you should absolutely observe connection health. If there’s a provider issue, you could notify your customers, such as through a status page, that notifications might not be working optimally. You might not be able to control the operational status of your providers, but you can still take action to maintain your customers’ trust.

### Holistic view of the product
A proper observability environment should provide you with a holistic perspective of your application’s state. Based on usage and performance, you can clearly reason about how certain factors might affect your product. You can gather a real understanding of the rhythm of your notification system. You might see that your users are receiving fewer notifications at specific times in the day or year. How do you know if this is due to your application sending fewer notifications, or because they’re not getting delivered at all? With notifications, data can fluctuate drastically. Peaks in error rates can be cause for concern and require immediate engineering support, while fluctuations in send volume can be observed with caution to ensure that the application is sending the right amount of notifications.

As you set up your observability, try to expose the data through user-friendly dashboards and interfaces, such as those that Datadog and Honeycomb offer. All of the collected data should be organized to provide clear insights on the application’s behaviors. Proper data visualizations are invaluable and should be tailored to more than just developer teams. For the customer service team, it is helpful to understand a specific user’s experience when something goes wrong. Likewise, commercial or marketing teams can glean insights for business development.

### Analytics and business development
If your users trust that you can monitor your application efficiently and resolve issues quickly or even use data to prevent their incidence, you’ll build a strong foundation and customer base. How you organize your observability system should ideally connect to your service-level metrics, which should be recorded in your SLAs (service-level agreements) that you have with your users. The observability data will be more relevant to your business if it is connected to your SLIs (service-level indicators) and SLOs (service-level objectives). An observability system that monitors all varieties of resource consumption without this connection to user experience might not foster the type of growth you want. 

Tracking and analyzing data on how your users are interacting with your notifications can help drive business development opportunities. Link tracking, for one, is a core observability component within a notification system. Did the user click on your notification? When and how many times?

Analyzing your observability data, and especially what you do with the resulting insights, is vital to further business growth. Your observability metrics will allow you to determine if the product is meeting business expectations. For instance, you can use observability data to make scaling decisions. If you want to scale, how might increases in volume affect your system? As you aim to understand the signals from the noise in all of your observable data, every signal you find needs to drive meaningful change. This is where you will find opportunities for further development. 

## Making Your Notification System Observable
Once you know how to use data to effectively monitor your application and make informed decisions, where do you start? The ultimate goal is to design your observability environment so that it is able to understand data, compare the data between various channels and infrastructure, and make it actionable.

There’s a way to structure data to make it more useful to engineers, customer service, and business development teams. Making sense of this data requires two key measures: the *correlating* and *normalizing events data*.
Correlation illustrates how different events are connected to each other and how they are connected to different users. In a notification system, this means that every outbound message, the receipt and opening of a message, and every click on the notification are measured in the ways they are statistically related to one another.

Normalization refers to how we understand data points from different sources or channels and record them in a way that makes them comparable. That means re-scaling the data so that it all varies on a *similar scale*. For example, how would data from email notifications from SendGrid compare to SMS notification data from Twilio? These are not only different companies, but also entirely different channels. 

Correlation and normalization of data allow you to have a more complete understanding of how your product’s notifications are performing, both in general and in relation to one another. You would then be able to filter through data and for example, find all events related to one user. This would include all email, SMS, direct message, and push notifications regardless of how and where the user received them. You can also filter through the data to find all emails sent out on a specific date under certain conditions, like for multiple users or in specific regions. 
Ultimately, if you’re working with several providers for your notification system, correlating and normalizing data points will be a vital step to achieving observability.

## Observability tips for serverless notification systems
Your setup for an observability system will depend on your tech stack. For the sake of this example of a serverless notification system, we’ll use AWS DynamoDB and Lambda. Generally, observability boils down to metrics, logs, and traces—and how you manage them.

Since we’re using AWS DynamoDB and Lambda, [AWS CloudWatch](https://aws.amazon.com/cloudwatch/) is a great tool that provides built-in monitoring in connection with many other AWS services. CloudWatch collects both custom logs and those of other AWS services, as well as infrastructure metrics. 

At Courier we import these metrics and logs into [Datadog](https://www.datadoghq.com/), which aggregates everything on a dashboard. Datadog can be integrated with both [DynamoDB](https://docs.datadoghq.com/integrations/amazon_dynamodb/) and [Lambda](https://docs.datadoghq.com/serverless/installation/nodejs/?tab=datadogcli), as well as hundreds of other services.

In a notification system using Lambda and DynamoDB, you should monitor all default performance metrics such as the number of functions getting called, the number of rows being modified, and so forth. Some of the notification-specific metrics not already mentioned here include latency, error rates, and request rates. 

For DynamoDB in particular, it is valuable to monitor access patterns, such as inputs and outputs, in order to [avoid hot keys](https://www.courier.com/blog/scalability-and-reliability/). A tool like [CloudWatch Contributor Insights](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/contributorinsights.html) can help identify and analyze these access patterns. 

AWS Lambda can automatically capture logs and then connect them to AWS CloudWatch with the [Lambda Logs API](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-logs-api.html). Through this API, extensions can subscribe to function logs, extension logs, and the Lambda platform logs for events and errors. Datadog can import these as well. 

For logging DynamoDB activity, AWS offers [CloudTrail](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/logging-using-cloudtrail.html). All API calls for DynamoDB are captured as events. You can search through the event history, or you can [create a trail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html) for ongoing event delivery to an AWS S3 bucket. This allows for an extended record of events and you can integrate these logs to CloudWatch.

If you’re using a tool like Datadog, you might forward your CloudWatch logs and metrics to Datadog using a [Forwarder Lambda function](https://docs.datadoghq.com/logs/guide/send-aws-services-logs-with-the-datadog-lambda-function/?tab=awsconsole). If you’re also using [Kinesis](https://aws.amazon.com/kinesis/) in your tech stack to quickly process streaming data, you can use their [Firehose delivery stream](https://www.datadoghq.com/blog/stream-logs-with-kinesis-firehose-and-datadog/) to forward logs to Datadog as well. 

### Important logs to track
There are many vital metrics and logs to observe in a notification system. Some, already mentioned in this article, include the sending of notifications, receipt and opening of notifications, and any clicks on notifications. Other important ones include deliverability, open rate, and conversion rate. You’ll also want to note the channel and provider for each notification, such as Twilio for SMS. You are inherently looking for two different things. The first is how your notification system is operating so that you can resolve potential bugs and strive for improvement. The second is how successful your notifications are in engaging with your users. Any log or metric that might help you define those two components will be useful.

Finally, traces can greatly help understand the context of logs or metrics. Traces track requests from beginning to end through all of the components in your tech stack. Tracing is especially key when your tech stack involves several intertwined systems, as it can help identify bottlenecks between those systems. Traces are normally integrated through logs, such as a unique ID for each request. One idea would be to attribute each operation in DynamoDB and Lambda to a specific notification that is sent by a specific user. 

## Conclusion
If you’re building your own notification system, observability is vital for both maintaining and scaling your product. It is a preventative measure that can improve performance and health of your system. Whatever the relevant observability metrics may be for you, the data should be measurable, actionable, and meaningful. Remember that it’s what you do with your data that impacts your business the most.

This piece taught us about the necessity of observability and analytics to monitor the functioning and performance of your in-house notifications system, as well as the advantages it provides for its future ability to scale. This is the last post in this series about how to build your own notifications. Soon, we will release an eBook so that you can access all of this information together and use it as a reference as you get building. To stay in the loop about the upcoming content, subscribe below or follow us [@trycourier](https://twitter.com/trycourier?lang=en)!


A man looks up at a large screen showing visual data as charts and graphs

observability-and-analytics-header

By building observability and analytics into your notification system, you can identify and quickly resolve issues by monitoring how your product is performing.

observability-and-analytics-thumbnail

The Developer's Guide to Building Notification Systems: Observability and Analytics

## TLDR
Easily connect your existing Segment events to trigger and send product notifications.

## Background
We’re excited to announce that we now easily [integrate with Segment](https://segment.com/docs/connections/destinations/catalog/courier), the leading customer data platform.

After 2.5 years and hundreds of customers, our community has convinced us that having additional event sources to act as notification triggers would help expand use cases and reduce the effort required to get up and running with Courier.

Now, Segment and Courier users can:

* Configure Courier as a Segment destination
* Trigger product notifications without writing any additional code
* Take advantage of Courier’s complete product notification platform, including omni-channel routing, dynamic templates, preference management, automations, and more

Don’t use Segment? Remember that you can easily [integrate the Courier Send API into your codebase](https://docs.courier.com/reference/sendmessage) — a single API that can notify your users over any channel. See all of our channel providers in our [Integrations section](https://app.courier.com/integrations).

## Product notification infrastructure without additional code 
While Courier is still a developer API first and foremost, Segment users can now take advantage of the complete Courier platform, including omni-channel routing, dynamic templates, preference management, automations and more by simply adding Courier as a Segment destination. The only requirement is that you are currently sending any events you would want to use to trigger a Courier notification to Segment.

## Courier + Segment use cases
While the ways in which you can use Courier and Segment to automate your product notifications are endless, below are a few highlighted examples of how customers are using Courier and Segment today. Also check out Courier's [case study with Hospitable.com](https://www.courier.com/customers/hospitable-automates-communication-using-courier/) to learn how they use our Segment integration.

### Transactional notification
Using the Segment ‘account created’ or ‘signup’ event, you could send a welcome message to all new users. While the most common channel for welcome messages is email, you could also send a welcome SMS, Slack message, in-app notification, or one of many other channels supported by Courier. You could even set up an automated sequence to nurture new signups as they progress through your product funnel.

![segment-courier-integration-1](//images.ctfassets.net/z7iqk1q8njt4/3PATYalrMWc9Xw128voWkj/207459968e89aa5b23c1117f290bead1/segment-courier-integration-1.png)
*Segment integration in Courier: Segment events, once triggered on your application, can be located in Courier and linked to automations. In this example, our events are linked to the “Account Created/Sign Up” automation that sends a Welcome email to the user when triggered.*

### User activity notification 
Using any segment events you have configured that track user activity that other users may need to know about, for example a product being purchased in a two-sided marketplace, you can easily set up notification automations. One common use case is to send the seller a notification whenever one of their items has been purchased, and then to send the buyer a notification when the seller has shipped their item.

![segment-courier-integration-2](//images.ctfassets.net/z7iqk1q8njt4/JZM5fTEIQ71ahdJzgUJKM/711eda949ab43cb0e271f3ffb0de8825/segment-courier-integration-2.png)
*Segment API sample call: Segment can make calls to Courier that pass along information about how your user is interacting with the application.*

![segment-courier-integration-3](//images.ctfassets.net/z7iqk1q8njt4/6k1EjXyjqfwqZjxOUzq05K/a36a40fd1754bd9e5363ea0d22682ff1/segment-courier-integration-3.png)
*Marketplace automations triggered by Segment API call*

### Action requested notification
You can use any Segment event that requires action from a user to build notification automation in Courier. For example, if your app facilitates employee performance reviews, you may want to begin sending managers both email and Slack reminders to complete their reviews two weeks before they are due. You may then want to send a notification to the manager’s manager, letting them know there are incomplete performance reviews, two days after the due date. All of this is easily achieved, without writing any code, using Courier and Segment together.

## How to get started
The first step is to sign up for a free Courier account and then to configure Courier as a Segment destination. Once that is complete, you can start triggering notifications from Segment events by triggering notification automations from Segment events. A complete getting started guide can be found [here](https://www.courier.com/docs/guides/tutorials/how-to-send-notifications-with-segment).


The Courier logo on the left connects via dotted line to the Segment logo on the right

Courier Segment Integration

We are excited to announce that Courier now integrates easily with Segment, the leading customer data platform.

With Courier's Segment Integration, Developers Create Product Notifications Fast

The Courier and Segment logos connect to each other by dotted line diagonally

Courier Segment Integration - Thumbnail

Courier + Segment Integration: Product Notifications In Minutes

Troy's Avatar

Troy Goode

CEO Troy Good shrugs while explaining how to build a notification system while surrounded by cartoon images of notifications.

Build-A-Modern-Notification-System-Header

Start thinking about improvements to make to a notification infrastructure, send better messages, and maintain these systems moving forward.

Screenshot of CEO Troy Goode from the video.

Build-A-Modern-Notification-System-Thumbnail

How to Improve or Rebuild a Modern Product Notification System: Video

Your CTO handed you a project to revamp or build your product’s notification system recently. You realized the complexity of this project around the same time as you discovered that there’s not a lot of information online on how to do it. Companies like LinkedIn, Uber, and Slack have large teams of over 25 employees working just on notifications, but smaller companies like yours don’t have that luxury. So how can you meet the same level of quality with a team of one? This is the third post in our series on how you, a developer, can build or improve the best notification system for your company. It follows the first post about identifying [user requirements](https://www.courier.com/blog/the-developers-guide-user-requirements) and designing with [scalability and reliability](https://www.courier.com/blog/scalability-and-reliability) in mind. In this piece, we will learn about setting up routing and preferences.

Notifications serve a range of purposes, from delivering news to providing crucial security alerts that require immediate attention. A reliable notification system both enables valuable interactions between an organization and its customers and prospects and also drives user engagement. These systems combine software engineering with the art of marketing to the right people at the right time.

Building a service capable of dynamically routing notifications and managing preferences is vital to any notification system. But if you’ve never built a system like this, it might be difficult to figure out what the requirements are and where the edge cases lie. 

In this article, you’ll learn invaluable points to consider when building your own routing service.  You’ll understand the requirements for multi-channel support and in choosing the right API providers. You’ll also learn how to design user preferences so that you can make the most out of each message. 

## Multi-channel support: a necessity
Let’s say that you have just built a web-based application. The first channel that you’ll use to connect with your users is likely email because of how ubiquitous it is. However, with the diversification of channels and depending on your use case, email might not be the most efficient notification channel for you. Compared to other channels, emails typically have a low delivery rate, a low open rate, and a high time to open rate. It’s not uncommon for people to take a full day to even notice your email. If your email gets to the user, it might take awhile before they open it, if at all. 

To engage with your users more effectively, you’ll want to support channels across a broad range of systems not limited to any one application or device. It’s vital to understand not only which channels are most relevant for you but also for your users. If you opt to use Telegram and your users don’t have it, it won’t be a very useful channel to interact with them. Multi-channel support is also vital because while you might pick appropriate channels today, you won’t know which channels you will need to support in the future. Typically, the more appropriate channels you support, the higher the chances of intersecting with applications your users actually use now and in the future.

## Choosing notification channels and providers

You’ll have to select relevant channels and appropriate providers for each channel. For example, two core providers for mobile push notifications are [Apple Push Notification Service (APNs)](https://developer.apple.com/library/archive/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/APNSOverview.html#//apple_ref/doc/uid/TP40008194-CH8-SW1) and [Firebase Cloud Messaging (FCM)](https://firebase.google.com/docs/cloud-messaging). APNs only supports Apple devices while Firebase supports both Android and iOS as well as Chrome web apps. 

In the world of email providers, SendGrid, Mailgun, and Postmark are all popular but there are hundreds more. All email APIs differ in what they offer, both in supported functionality and API flexibility. Some providers, like Mailgun, only support transactional emails triggered by user activity. Other providers, like SendGrid and Sendinblue, offer both transactional and marketing emails. If your company opts for a provider that can handle both, you’ll still want to separate the traffic sources, by using different email addresses or domains, to aid email deliverability. If you only have one domain for sending both types of emails and the domain gets flagged as spam, your critical transactional emails will also be affected. Whichever provider you choose, you’ll still want to meticulously verify your DKIM, SPF, and DMARC checks, and domain and IP blacklisting using your own tools or a site like [Mail-Tester](https://www.mail-tester.com/).

Making requests and receiving responses also differs with each email API provider. Some providers, like Amazon SES, require the developer to [handle sending attachments](https://www.courier.com/blog/send-email-attachments-aws-s3), while others, like Mailgun, [provide fields in the API schema](https://documentation.mailgun.com/en/latest/api-sending.html#sending) for including attachment files directly. 

There are minute variances in formatting HTTPS requests. The maximum payload sizes range from 10MB with Amazon SES API and up to 50MB with Postmark. There are also differences between the rate limits for requests.

In terms of API responses, Amazon SES [provides](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_SendEmail.html) a message identifier when an email is sent successfully through the API, but, for example, SendGrid [returns an empty response](https://docs.sendgrid.com/api-reference/mail-send/mail-send) in that situation. The HTTP response codes also differ slightly depending on the provider. For example, AWS SES uses the response code [200](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_SendEmail.html#API_SendEmail_Errors) for successful email send operations, while Sendinblue uses [201](https://developers.sendinblue.com/reference/sendtransacemail), and SendGrid uses [202](https://docs.sendgrid.com/api-reference/mail-send/mail-send#responses).

No matter which provider you end up choosing, don’t build your application solely to fit *their* logic and specifications. If you do so, it will be much more difficult to change providers in the future as you’ll have to overhaul your backend. It’s crucial to invest in a layer of abstraction based on your own paradigm. 

## Dynamically routing notifications between channels
How do you determine which channels to use and when? Just because you’re able to use email, SMS and mobile push doesn’t mean that you should use all of them simultaneously, since doing so carries a high risk of annoying your users. This is where you begin to formulate an algorithm to route messages between the different channels and the different providers within each channel. The algorithm needs to be robust to handle delivery failures and other errors. For example, if the user hasn’t engaged with a push notification after a day, do you resend it or use email instead? 

You can begin constructing the algorithm using basic criteria. For example, if there is no phone number, eliminate SMS as an option for that user. If email is the primary channel, opting to send at 10 a.m. or 1 p.m. local time typically improves read rates. If the user is present or active in the app, consider sending an in-app push notification instead of an email. Finally, and especially important, get your user’s preferences for how and when they want to be contacted and integrate these preferences into your routing service. 

## Adding user preferences to your system
Once you’ve got your channels, providers, and routing algorithm figured out, you need to think about providing users with granular control over notification preferences instead of just a binary opt-in/opt-out switch.

Consider this: if you only allow opting in to or out of all notifications at once, your users might unsubscribe from all your communications because they find one specific notification annoying. As a result, you will lose out on valuable user engagement.

With granular control over preferences, a user identifies exactly how and when they hear from you. If a user doesn’t like email but wants SMS messages (not common, but possible!), they can adjust their preferences and keep the SMS line of communication open. Every enabled notification channel is another opportunity to engage the user in a way that’s productive for them. From the end user’s perspective, it’s empowering to control how and when they are contacted.

Note that for some channels, the user’s preferences should be ignored. For instance, two-factor authentication should go to SMS or mobile push regardless of the user’s preference for email. The possibility to override the default logic should be incorporated into your algorithm while you are designing your routing engine. 

If you want to take user engagement further, allow users to opt-in/opt-out of specific channels, frequency, timing and topics. You can allow them to set up their preferences based on time of day, frequency per period, or to specify more than one email address. You can give them the option to receive transactional, digest emails, daily newsletters, or only the critical ones. You can also allow them to redirect their notifications to another address, for example if the user is out of office. 

Granular preferences also extend past the dominion of developers and the user’s experience. Granularity of consent is becoming part of privacy [compliance laws in Europe](https://edpb.europa.eu/sites/default/files/files/file1/edpb_guidelines_202005_consent_en.pdf) and in the [state of California](https://src.bna.com/MVJ?utm_source=ANT&utm_medium=ANP) and might follow elsewhere in the future. Separately, granular preferences are an extremely advantageous analytical tool for the marketing team to improve brand strategy and personalization efforts. Is there a particular channel or topic that seems to be more popular? That information can be highly helpful to pivot in line with your users and grow your company. 

## Tips for future-proof maintenance
When you’re starting with notifications for a new product, there is nothing wrong with sticking to one channel and one provider. The most important principle to keep in mind is to design your notification system so that you can expand it in the future. You should leave the door open to include more providers when you need them. 

Don’t assume that **API paradigms** are the same for each provider or notification type. For example, you want to send an email, and if delivery fails to send a push notification instead. But you won’t get a 400 HTTP response from the email provider in case of failure. The provider will retry your email over a couple of days. Instead, you’ll want to include [webhooks](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks) or queues to notify you of the failure, and you’ll need to track the state of the message here. If you make blanket assumptions about how API calls work or how errors are returned, you’ll have trouble adapting to a different paradigm in the future. Instead, you can add a layer of abstraction on top of the API.

It’s also invaluable to **centralize the way you call the provider APIs**. If you spread out calls to an API throughout your code base, it will be more difficult to integrate other channels or API providers in the future. Let’s say you’re starting with email and [AWS SES](https://docs.courier.com/docs/setup-email-using-aws-ses) as the provider. In two years’ time, you might decide to integrate mobile push notifications as well. What might that look like? The incurred technical debt will include scouring the code base for all instances of calls to the AWS SES API before you can integrate mobile push as an additional channel. But with centralized calls, you’ll have more consistent, cleaner, and reusable code as you grow.

## How many notification channels should you have?
Typically, having three or four channels that are relevant to your product is an ideal scenario for a mature product. When you intersect channels with the preferences and availability of users, you create higher levels of complexity for your algorithm. Offering many channels for notifications might become too complex to maintain. But offering too few channels might harm your chances of interacting with users since some channels might not be viable for all users. For instance, you might decide to offer email and push notifications. But if a user didn’t download your product, your interaction with them is limited only to email. 

## Best technologies for routing and preferences engines
It ultimately pays to choose technologies that will be a good fit for your routing and preferences needs. There will be a great deal of [asynchronous programming](https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Asynchronous/Concepts), as the routing service will often be waiting to receive responses for each function. You’ll want to pick a language or a framework that allows you to respond to async events at scale. 

The routing service also involves considerable state tracking, as most of the routing will depend on waiting on a response for each notification before changing state. The routing service will also need to be re-activated every time it receives a response from a provider and will need to determine if the notification was sent successfully or if it has to pursue next steps. See the example below of how a notification function’s state might be tracked.

![an example of notification state tracking](//images.contentful.com/z7iqk1q8njt4/4EkKmXPxwCkStyd9Y55yWk/88a70a31edd6c13e5b882c06af6d8fd6/image1.jpg)

At Courier, we use [AWS Lambda](https://aws.amazon.com/lambda/). Since our usage tends to come in bursts, serverless technology allows us to adjust and scale for changes in demand throughout each day as well as handle asynchronous operations efficiently. 

## Don’t forget: compliance in notification routing
When creating your own routing and preferences service, you will need to ensure that whichever channels you implement are fully compliant with applicable laws. For example, there are legal mandates on how users may be contacted or how they can unsubscribe from contact.

For commercial email messages, the [CAN-SPAM Act](https://www.ftc.gov/sites/default/files/documents/cases/2007/11/canspam.pdf) of 2003 is a federal United States law that spells out distinct rules and gives recipients a way to stop all contact. Penalties can cost as much as $16,000 per email in violation. This law also outlines requirements such as not using misleading header information or subject lines, identifying ads, and telling recipients how they can opt out of all future email from you. The opt-out process itself is strictly regulated.

For SMS, the United States [Telephone Consumer Protection Act (TCPA)](https://www.fdic.gov/resources/supervision-and-examinations/consumer-compliance-examination-manual/documents/8/viii-5-1.pdf) of 1991 sets forth rules against telemarketing and SMS marketing. Under this law, businesses cannot send messages to a recipient without their consent. This consent needs to be explicit and documented. The consent is also twofold: recipients need to consent to receiving SMS marketing messages and they need to consent to receiving them on their mobile device. Recipients need to be provided a description of what they are subscribing to, how many messages they should expect, a link to the terms and conditions of the privacy policy, and instructions on how to opt-out. 

In California especially, the [California Consumer Privacy Act (CCPA)](https://leginfo.legislature.ca.gov/faces/codes_displayText.xhtml?division=3.&part=4.&lawCode=CIV&title=1.81.5) of 2018 provides additional rights for California residents only. These rights include the right to know which information a company has collected about them and how it’s used as well as the right to delete it or to opt-out of the sale of this information. Information that qualifies under the consumers’ right-to-know includes names, email addresses, products purchased, browsing history, geolocation information, fingerprints, and anything else that can be used to infer preferences. Should a consumer request this information, the company has to share the preceding 12 months of records, and also include sources of this information and with whom it was shared and why. In 2020, [California Privacy Rights Act (CPRA)](https://src.bna.com/MVJ?utm_source=ANT&utm_medium=ANP) of 2020 amended the CCPA. The CRPA provides further consumer rights to limit the use and disclosure of their personal information.

Other countries have their own compliance laws for businesses reaching out to leads and customers. Canada has its [Anti-Spam Legislation (CASL)](https://laws-lois.justice.gc.ca/eng/acts/E-1.6/index.html). The European Union has the [General Data Protection Regulation (GDPR)](https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0679) which now also covers granularity of consent. The United Kingdom has its own regulations along with the GDPR, the [Privacy and Electronic Communications Regulations (PECR)](https://www.legislation.gov.uk/uksi/2003/2426/pdfs/uksi_20032426_en.pdf) and [Data Protection Act](https://www.legislation.gov.uk/ukpga/2018/12/contents/enacted). 

Compliance itself needs to be integrated at the developer level. Providers, like SendGrid, don’t know what you’re sending. It’s up to the developer to ensure that all applicable compliance laws are followed for their choice of channels. 

## Conclusion 
Building a notification system into a product is not for everyone. The process is time-consuming, complex, and expensive. The level of notification customizability and routing options you decide to implement will ultimately dictate a preference for either maximizing user engagement or optimizing cost. A startup with a product that hasn’t yet found its product-market fit has to focus on finding early customers and getting their feedback. But established companies with a proven customer base will have concerns related to more complex routing logic, future-proofing and compliance. This would require more functionality and higher maintenance costs.

This piece taught us about the necessity of sending data for notifications to the right people, at the right frequency, at the right time and how this can be done through routing and customized preferences. Tune in for the next post in this series to learn about [observability and analytics](https://www.courier.com/blog/observability-and-analytics/) to monitor the functioning and performance of your in-house notifications system. To stay in the loop about upcoming content, subscribe below or follow us [@trycourier](https://twitter.com/trycourier?lang=en)!


Workers build pathways and routes in a chaotic factory

routing-and-preferences-header

In this article, you’ll learn how to set up routing for your notifications data and design user preferences so that you can make the most out of each message. 

Building a Routing and Preferences Service for Notifications

Workers build pathways and routes in a chaotic factory to transport data

routing-and-preferences-thumbnail

The Developer's Guide to Building Notification Systems: Routing and Preferences

Seth Profile Pic

Seth Carney

A female engineer sits in front of a computer screen, looking at and adding user details.

scalability-and-reliability-header

In this guide, we cover building a scalable and reliable notification system to give you an idea of the required effort of building one yourself.

Welcome to the Courier Engineering Blog

Hacking PostgreSQL Internals to Deliver Push Notifications

How We Use Internal Hackathons to Create New Product Features

3 Types of User Communication APIs and When to Use Them

The Developer's Guide to Building Notification Systems: Observability and Analytics

Courier + Segment Integration: Product Notifications In Minutes

How to Improve or Rebuild a Modern Product Notification System: Video

The Developer's Guide to Building Notification Systems: Routing and Preferences

The Developer's Guide to Building Notification Systems: Scalability and Reliability

Build your first notification in minutes

Build your first notification in minutes

Product

Developers

Resources

Company