Istio Service Mesh

What Is a Service Mesh, and Why Do You Need?

Service mesh is a new form of architecture for software applications. This includes internal applications used to run a business, such as analytics reporting or dashboards, and public-facing applications that deliver functionality to customers, such as a ride-sharing app on your phone.

Part of the reason service mesh is becoming popular is because it supports and extends other important trends in application design and delivery: containerizing software to make it more reliable and more flexible; using Kubernetes to manage the containers; and microservices architecture, which splits an application into small, containerized components (services). Service mesh sits on top of these pre-existing approaches and extends them to help make software easier to develop, easier to deliver, more reliable, and more secure.

The service mesh is made up of a network, or “mesh,” of communications between software services. While software services are, of course, different from each other, each service is attached to a fixed piece of software, called a sidecar proxy. When a service wants to send a message to another service, or to the control plane that’s part of the service mesh, the message goes through the proxy software. The proxy authenticates messages, authorizes them, and encrypts/decrypts them using TLS, among other functions. We go into more depth on this below.

Core development of service mesh software took place at Google, IBM, and Lyft, to solve problems that are hard to manage in any other way at the scale that Google (and a few other companies) run at. The service approach was quickly adopted by other leading, tech-forward companies, such as Twitter. As companies move to the cloud, and take on larger and more complex software projects – or want to develop smaller projects faster, with greater consistency – they are increasingly turning to the service mesh architectural pattern to help.

Note: Tetrate’s team includes core founders and maintainers of several crucial open source software projects, including Istio service mesh software and the Envoy sidecar proxy software. Tetrate makes Istio distributions available for free and sells support in the form of Tetrate Istio Subscription. The Tetrate team also provides training, including certification as an option, against specific Istio-plus-Envoy distributions as the open source project advances. (With significant input from Tetrate engineers.)

Why Istio Service Mesh?

Istio is the first and most widely used open source project to make service mesh accessible to a wider audience. There are alternatives, which we will discuss in a later blog post. While the different approaches have most of their features and functionality in common, this page only discusses Istio service mesh.

Istio service mesh serves as a networking layer, automating and securing communications between applications. Istio service mesh is independent of any specific programming language. Istio is used by architects and engineers building cloud-native applications or following microservice architecture approaches.

Istio service mesh provides a control plane to define and implement the way microservices communicate with each other. Istio is based on a foundation layer of lightweight network proxy instances derived from the Envoy proxy. Envoy is responsible for all service interaction in Kubernetes or virtual machines (VMs). One outstanding feature of the Istio service mesh is that services do not know they are working alongside Envoy instances or using a sidecar proxy.

Using Istio service mesh, platform teams can address needs for traffic management, service security, and application monitoring. Istio service mesh enables developers to develop business logic for loosely coupled microservices without worrying about communication logic and security. Istio is designed to run in various environments like on-premises, multi-cloud, Kubernetes containers, and virtual machines (VMs), so Istio helps platform teams manage and monitor all the service traffic across clusters and data centers.

Features of Istio Service Mesh

Istio provides numerous features that make software development and delivery faster, easier, and more secure. Istio offers authentication, authorization, load balancing, circuit breaker, time outs, retries, and deployment strategies, service discovery, and observability. Following is a brief description of key capabilities that you can expect Istio + Envoy software to provide:

  • Security

    Security

    Istio helps application teams to achieve zero trust security with the ability to define and implement authentication, authorization, and access control policies. All your data communicated among the services, in and outside of the cluster or data center, will be encrypted based on mTLS protocols provided by Istio resources. You can also ensure authentication of apps from internal and external users using JSON Web Tokens (JWT) provided by Istio.

  • Service Discovery

    Service Discovery

    One of the primary needs of an application running in a production environment is to be highly available. This requires one to scale up the number of service instances with increasing load and scale down when needed to save costs. Istio’s service discovery capability keeps track of all the available nodes ready to pick up new tasks. In case of node unavailability, service discovery removes a node from the list of available nodes and stops sending new requests to the node.

  • Traffic Management

    Traffic Management

    Using Envoy proxies, Istio provides flexibility to finely control the traffic among the available services. Istio provides features like load balancing, health checks, and deployment strategies. Istio allows load balancing based on algorithms that include round robin, random selection, weighted algorithms, etc. Istio performs constant health checks of service instances to ensure they are available before routing the traffic request. And based on the deployment type used in the configuration, Istio drives traffic to new nodes in a weighted pattern.

  • Resilience

    Resilience

    Istio removes the need for coding circuit breakers within an application. Istio helps platform architects to define mechanisms such as timeouts to a service, number of retries to be made and planned automatic failover of high availability (HA) systems, without the application knowing about them.

  • Observability

    Observability

    Istio keeps track of network requests and traces each call across multiple services. Istio provides the telemetry (such as latency, saturation, traffic health, and errors) that helps SREs to understand service behavior and troubleshoot, maintain, and optimize their applications.

  • Advance Deployment

    Advance Deployment

    Istio provides visibility and fine-grained network controls for traditional and modern workloads, including containers and virtual machines. Istio helps to achieve canary and blue-green deployment by providing the capability to route specific user groups to newly deployed applications.

How does Istio Service Mesh work?

Istio has two components which together make up a service mesh.

Data plane

The data plane is responsible for translating, forwarding, and monitoring every network packet flowing to and from an instance. It owns key features such as health checking, routing, service discovery, load balancing, security, and telemetry. The Istio features can be implemented in any application using proxy instances deployed in a sidecar pattern. Istio uses Envoy proxy alongside each service in a cluster to observe, validate and manage all inbound and outbound requests. All incoming traffic from APIs is directed to an instance of Envoy (called, in this case, an ingress proxy). The proxy then follows rules or policies, specified in the Istio control plane, to decide how, when, or what service to route the traffic. Using these rules, Istio also enables techniques such as fault injections, circuit breaking, and canary deployments without any change to services.

Control plane

The Istio control plane provides rules and logic to carry out the communication between proxies in the service mesh. Say, when a new service is discovered, the control plane populates the existing list of services, which is then used by other proxies in the mesh to direct traffic. All the logic and configurations related to circuit breaking, retries, timeouts, load balancing, and security information are stored in the control plane. Whenever a new deployment happens, the data plane becomes aware of the new node but the mechanism of routing and shifting traffic will be defined by the control plane. The control plane provides policy, rules and configuration to the data plane without modifying any data in the mesh. Refer to the image below:

Istio Mesh Architecture

Most of the features stated in the data plane such as routing, health checking, service discovery, load balancing, security, etc, are available in projects such as HAProxy, Nginx, and Envoy. However, the configuration has to be written, deployed, and maintained manually using multiple other tools. Istio combines all these and provides an integrated platform, removing boilerplate configurations and offering durability in the solution.

Under the hood , Istio uses five major tools:

  1. Envoy Proxy: Istio uses Envoy proxy as a sidecar proxy in the data plane. Envoy in the data plane is responsible for functionalities like failure handling, health checks, service discovery, and load balancing. Envoy proxies provide detailed information about each service request.
  2. Mixer: Mixer in the control plane acts as Istio’s telemetry hub, gathering attributes about service requests from Envoy proxies in the mesh. Mixer provides an API to fetch those attributes for monitoring and logging purposes.
  3. Pilot: Istio uses Pilot in the control plane to provide traffic controls and load balancing based on the service mesh. All the traffic rules can be specified in Istio, and Pilot under the hood can communicate with Kubernetes infrastructure about deployment changes affecting traffic. Istio also uses Pilot to distribute security policies (such as authentication and authorization policies) to all Envoy proxies.
  4. Citadel: Istio uses Citadel to provide policy-driven and secured communication between Envoy proxies. All the authentication and key-based credential management between sidecar proxies is managed by Citadel.
  5. Galley: Istio control plane also includes Galley, which is responsible for interpreting user-defined Kubernetes YAML files into a format that Istio understands. Galley stores the user configuration, validates it, and then sends it to Pilot for further action.

How Istio is used in Kubernetes

Kubernetes may create multiple instances of a particular service hosted in different nodes. This is usually done to distribute the workloads among various instances. Kubernetes tags all the instances of the same service through Service objects. As per Kubernetes architecture, every node will have a Kube proxy attached to forward traffic and perform simple load balancing functions. Kubernetes handles client traffic to its cluster, having the service, through another proxy called Ingress proxy.

The Kube proxy and Ingress proxy offer rudimentary traffic management functionalities. They cannot handle splitting the traffic effectively in the runtime or can guarantee compliance and security in the network.

Thus organizations use Istio to handle complicated network functions, including security for any number of Kubernetes workloads and clusters across various data centers. Another essential point is that the Kube proxy is just a Layer 4 proxy, and Istio works both as a Layer4 and Layer 7 proxy in the OSI layer.

The architectural diagram represents how Istio handles client traffic and communication between services in a cluster using a control plane called Istiod. Istio manages traffic between Kubernetes workloads and VMs too.

Istio is used in Kubernetes

Benefits of using Istio service mesh

Organizations having large-scale applications based on microservices architecture will benefit the most from Istio. With growing traffic between various microservices, the need for sophisticated routing capabilities and secured flow of data will increase exponentially. Here are the benefits of using Istio service mesh:

Increased developer focus

Increased developer focus

As Istio service mesh can manage the communication layer, it abstracts out the network infrastructure for the code. Developers can now focus on adding business value with each service they build, rather than worrying about how each service communicates with other services.

Implement secured communication

Easy-to-implement secured communication

With the abstraction of the network layer, security operators can easily implement service-to-service security including authentication, authorization, and encryption using mTLS based connections.

Compliant to industry standards

100% compliant to industry standards

Istio service mesh helps engineering and platform teams to define security and compliance policies and ensure their application and infrastructure are compliant with industry standards like PCI, FedRAMP, and GDPR compliance.

Improved business agility

Improved business agility

With Istio service mesh, DevOps teams can easily implement deployment strategies like canary and blue-green deployment by splitting the traffic in run-time. You can get deep visibility on how your applications are consumed by a certain targeted audience so you can focus your efforts to improve performance.

Success Stories of Istio

Learn how FICO migrated their infrastructure from monolithic to microservice architecture while complying with PCI standards using Istio

Read more ›

Read how a DecSecOps team in Department of Defence (DOD) ensured zero-trust security for containerised infrastructure with the help of Istio

Read more ›

Getting Started with Istio Service Mesh

  1. Understand Istio: Guide to learning and understanding the value of Istio and Envoy
  2. Learn Istio Fundamentals: Watch 5 hours of video and learn quickly about Istio for free
  3. Install Tetrate Istio Distro: Get started with trusted, FIPS-compliant, and supported versions of Istio service mesh
  4. Monitoring Istio: Learn how to get metrics from Istio service mesh using Prometheus