Working with large, complex applications with many services can cause availability issues. A failure of a single service can cause services that depend on it to fail. This can cause a cascade effect that results in your entire application failing. This is especially egregious when the service that failed is not itself a mission-critical service but it causes mission-critical services to fail.

Service tiers are labels associated with a service that indicate the criticalness of that service to the operation of your business. Service tiers allow you to manage your application complexity and understand the importance of individual application services in a distributed and organized manner.

Application Complexity

As illustrated in Figure 7-1, sometimes the smallest and least significant of services can fail.

Figure 7-1. A single service failure...

This can cause your entire application to go down, as illustrated in Figure 7-2.

Figure 7-2. ...can cause a cascade failure

There are many ways to prevent dependent services from failing, and we discuss many of these in Chapter 5. However, adding resiliency between services also adds complexity and cost, and sometimes it is not needed. Looking at Figure 7-3, what happens if Service D is not critical to the running of Service A? Why should Service A fail simply because Service D has failed?

Figure 7-3. What if Service D is not critical?

How do you know when a service dependency link is critical and when it isn’t? Service tiers are one way to help manage this.

What Are Service Tiers?

A service tier is simply a label associated with a service that indicates how critical the service is to the operation of your business. Service tiers let you distinguish between services that are mission critical and those that are useful and helpful but not essential.

By comparing service tier levels of dependent services, you can determine which service dependencies are most sensitive and which are less important.

Assigning Service Tier Labels to Services

All services in your system, no matter how big or how small, should be assigned a service tier. The following sections outline a scale to get you started (you can make adjustments to these recommendations as necessary to accommodate your particular business needs).

Tier 1

Tier 1 services are the most critical services in your system. A service is considered Tier 1 if a failure of that service will result in a significant impact on customers or on the company’s bottom line.

The following are some examples of Tier 1 services:

Login service

If customers can’t log in to your application, then your application is unusable to them.

Credit card processor

If customers can’t use their credit cards, they can’t complete orders and your business can’t make money.

Permission service

Not all customers have the same access to the same capabilities. If the permission service is down, customers will lose access to their allowed capabilities.

Order accepting service

If customers can’t check out and complete the processing of their orders, then your business can’t make money and customers can’t get and use your product.

A Tier 1 service failure is a serious concern to your company.

Tier 2

A Tier 2 service is one that is important to your business but less critical than a Tier 1. A failure in a Tier 2 service can cause a degraded customer experience in a noticeable and meaningful way but does not completely prevent your customer from interacting with your system.

Tier 2 services are also services that affect your backend business processes in significant ways but might not be directly noticeable to your customers.

The following are some examples of Tier 2 services:

Search service

Customers want to be able to search for products and information on your site. Without it, they can still use your site, but with reduced functionality.

Order fulfillment service

Processing your order in the warehouse is important to being able to ship orders to customers, but customers won’t notice brief outages in the ability to fulfill orders.

The failure of a Tier 2 service will have a negative customer impact but does not represent a complete system failure.

Tier 3

A Tier 3 service is one whose failure can have minor, unnoticeable, or difficult-to-notice customer impact or has limited effect on your business and systems.

The following are some examples of Tier 3 services:

Customer icon service

A service that displays a customer icon or avatar on a website page. If it is not working, most people probably wouldn’t even notice. But if they do, it won’t be a major issue.

Recommendations service

A recommendations service is a great way to cross-sell product on your site, but if it is not working, customers can still make purchases and you can still fulfill those orders.

Message of the day service

Often we want to show all customers a message at the top of the page when they first arrive on our site. If we can’t do that, customers may miss out on a sale, but they may not even know that they are missing anything.

Customers may or may not even notice that a Tier 3 service is failing.

Tier 4

A Tier 4 service is a service that, when it fails, causes no significant effect on the customer experience and does not significantly affect the customer’s business or finances.

The following are some examples of Tier 4 services:

Sales report generator service

A service that generates a weekly sales report. Although the sales report is important, a short-term failure of the generator service will not have a significant impact.

Marketing email sending service

A service that generates emails sent regularly to your customers. If this service is down for a period of time, email generation might be delayed, but that will typically not significantly affect you or your customers.

Example: Online Store

Figure 7-4 is an example application composed of many services. It is designed for operating an online store. Each service has a label indicating the service tier assigned to the service.

Figure 7-4. Example application: an online store

Look at Figure 7-4 and imagine from the descriptions what the responsibility of each service is. Imagine what the customer experience can or should be when a particular service is malfunctioning. The service tier should be in line with this perceived customer experience.

Here are some example services from this application for you to consider:

Website frontend service (Tier 1)

This is the service that generates and displays the website. It generates the HTML and interacts with the user’s browser for the main storefront.

This is a Tier 1 service because without it your entire online store is unavailable to your customers. It passes the Tier 1 test because if it is not available, that has a huge impact on your customers.

Catalog view service (Tier 1)

This service reads the catalog database and sends the appropriate catalog data to the frontend service. It’s used to generate the detail pages that show the details of individual products in the database.

This is a Tier 1 service because without it your customers can’t view any products online. It passes the Tier 1 test because if it is not available, that has a huge impact on your customers.

Catalog search service (Tier 2)

This service handles search requests from users and returns lists of products that match the search terms.

This is a Tier 2 service because, even though search is an important customer feature to the website, it is possible for customers to browse to products and still use your site without the search bar working. The experience is obviously diminished, but it is still usable.

Catalog database service (Tier 1)

This is the database that stores the catalog itself.

This is a Tier 1 service because without the catalog database, no product can be displayed.

Catalog editing service (Tier 3)

This is the service that your employees use to add new entries to the catalog and update existing entries.

This service is considered a Tier 3 service because it is not mission critical to the ability of customers to successfully complete a purchase. Although not being able to add products to your database will affect your business, it doesn’t immediately or directly affect your customers, and a bit of an outage might be acceptable.

Checkout service (Tier 1)

This is the service that displays the checkout process to your customers. Without this service, your customers can’t buy products from you.

This is a Tier 1 service, because it has a significant impact on both your customers (they can’t buy things) and your business (you can’t make money without customers buying things).

Order shipping service (Tier 3)

This is the service that manages the process of boxing and shipping your customers’ orders (an obviously simplified example). Without this service, your customers can’t receive orders they have placed.

It may seem like this should be a Tier 1 service because shipping orders is a mission-critical aspect of your business, but think of it this way: if you can’t ship orders for an hour, what’s the impact on your customers? What about your business? In most cases, it would have little to no impact on your customers—a one-hour shipping delay wouldn’t affect when customers receive their orders. It would have some effect on your business, because the employees that pack orders might not be able to do their jobs for a while. Because it does not have a significant effect on your business or a significant impact on your customers, a Tier 3 label is appropriate.

Weekly order report (Tier 4)

This is the service that gathers your ordering data and generates weekly business reports to finance and management.

This is a Tier 4 service because it has no impact on customer experience at all. Having a report delayed for a short period of time might affect your business, but likely not significantly.

This example should give you an idea of how you can generate appropriate service tier labels for all your services.

Now that you understand the various tier levels, you should be able to apply appropriate service tier labels to all of the services in your application. Now that our services are labeled, how do we use the labels, and what value do they bring?

Using Service Tiers

After you have assigned service tiers to all of your services, how do you take advantage of these labels in the operation of your services? There are a few ways:

Expectations

What is the expected uptime for the service? What is its reliability? How many problems does it have? How often is it allowed to fail?

Responsiveness

How quickly should you respond to a problem, and what courses of action are available to you in resolving the issue?

Dependencies

What are the service tiers of your dependencies and those who depend on you, and how do these affect your service interactions?

Let’s look at each of these.

Expectations

Your service’s expectations are an important part of your service to your customers. Service-level agreements (SLAs) are one way to manage these expectations. This is so important that Chapter 8 is entirely dedicated to this topic.

Responsiveness

When a problem occurs in your system, your responsiveness to the issue depends on these two factors:

· The severity of the issue

· The tier of the service that is having the issue

A high-severity problem on a Tier 1 service should be treated as more important than a high-severity problem on a Tier 3 service. That is clear. But if a Tier 1 service has a medium-severity problem, this might need a higher level of responsiveness than a high-severity problem on a Tier 3 service. Figure 7-5 demonstrates this.

Figure 7-5. Responsiveness for service tier versus problem severity

The higher the severity of the problem, or the higher the importance of the service (lower service tier number), the more critical a quick response to the problem becomes. The parallel lines in Figure 7-5 show lines of similar response importance. A low- to medium-severity Tier 1 problem would require a similar response to an extremely high-severity Tier 3 problem. A Tier 4 problem almost never requires a critical response.

Furthermore, a low-severity Tier 2 problem would require a similar response to a high-severity Tier 4 problem.

You can use this information to adjust many aspects of your responsiveness. For example, you can use the responsiveness level to determine the following:

· Which types of problems for which services require an immediate notification to be sent

· The expected resolution SLAs

· The escalation path for slow response or slow resolution

· A schedule for when a response should be provided (24 × 7 or business hours only)

· Whether emergency deployment or production changes are warranted

· The SLAs in which your service should perform around availability and responsiveness

Dependencies

If you are building a service, the relationship between the service tier you assign to your service and the service tier of your dependencies is important. Figure 7-6 shows the relationship between your service tier level and that of a service dependency.

Figure 7-6. Service dependency criticality

If your service is a higher tier (lower number) than your dependent service, your dependency is a critical dependency. If your service is a lower tier (higher number) than your dependent service, your dependency is a noncritical dependency.

Critical dependency

If you’ve determined that your dependency is critical, it is important that you, as a service developer, deal with failures of your dependency in a way that does not significantly affect your service.

Your service is responsible for performing as much of its capabilities as is possible if a critical dependency fails. This is because the dependency is a lower tier (higher number), which means it likely will not have the same level of availability and reliability as your service requires.

As an example, look at the application shown in Figure 7-4 and focus on the website frontend service, which is a Tier 1 service. When this service tries to display a specific product detail page to a customer, it needs to determine the current price of the product. To do this, it makes calls to the price & shipping cost calculator (PSCC) service to determine the price.

What if the PSCC service (a Tier 2 service) is down? The website frontend service (a Tier 1 service) still must function as best as it can. So what does it need to do?

It needs to gracefully handle failure messages (or lack of response) from the PSCC service. As soon as it determines that the PSCC service is down, it needs to figure out what to do in displaying the product detail page. There are a couple options:

· It could show a cached copy of the price on the page (if it had that available).

· It could show the product detail page but without the current price. Instead, it could show a message such as “Not available,” or “Price not currently available,” or even “Add to cart to see current price.”

The customer can still see pictures of the product, customer reviews, and other product details. Although the experience is degraded, the customer can still complete some very important tasks on your site.

We call this graceful degradation (dealing with service failures was covered in greater detail in Chapter 5).

Noncritical dependency

If you’ve determined that your dependency is noncritical, you can mostly ignore service failures of the dependency.

This is because your dependent service, having a higher tier (lower number), will have higher levels of availability and responsiveness than your service requires.

As an example, consider the online store application illustrated in Figure 7-4, but this time focus on the weekly order report service, which is a Tier 4 service. For it to get the information it needs to generate its report, it makes calls to the order management service, which is a Tier 1 service.

What happens if the order management service is down? What should the weekly order report service do? Well, it’s probably reasonable for the weekly order report service to simply fail as well. Given that the order management service is a Tier 1 service, any problems it has will be dealt with very quickly, with a high responsiveness and a high sense of urgency—much higher than would be needed to deal with the failure of the weekly order report service.

As such, the weekly order report service does not need to do anything special to deal with an outage of the order management service, because it is OK for the weekly order report service to simply not operate if the order management service is not available.

Summary

Service tiers provide a convenient way of expressing the criticality of a service to the service’s owners, dependencies, and consumers. They provide a way of understanding expectations between services in a manner that is simple to understand and communicate. Simplicity reduces the chance of mistakes occurring, and service tiers provide a simple model for communicating expectations in a manner designed to be easy to understand and easy to utilize.