Kubernetes is the de facto standard for orchestrating modern cloud-native workloads. But, it doesn’t provide for secure communication out of the box. This means everyone who needs to implement encryption-in-transit to adopt a zero trust security posture for their Kubernetes deployments needs to figure it out for themselves.
Luckily, there are well-understood ways to get there and, in this article, we’ll cover the top four best practices for implementing mutual TLS (mTLS) in Kubernetes. Let’s discuss.
Tetrate offers enterprise-ready, 100% upstream distributions of Istio and Envoy Gateway, the easiest way to implement mTLS for cloud-native applications. Get access now ›
What Is mTLS and Why Is It Important for Security?
Transport layer security (TLS)—the successor to SSL—is the most widely deployed standard for secure communication, most visibly in HTTPS. TLS works great for establishing secure communication that is both confidential (resistant to eavesdropping) and authentic (resistant to tampering) between a server that needs to prove its identity to its clients. But, in situations where both parties need to prove their identity to each other—such as between microservices in a Kubernetes application—TLS isn’t sufficient.
That’s where mutual TLS (mTLS) comes in. mTLS is TLS, but where both parties prove their identities to each other before establishing a secure communication channel. This is a necessary part of what’s needed for secure communication in Kubernetes. mTLS provides:
- Encryption on the wire for confidentiality and tamper-resistance
- Mutual, cryptographically secure proof of identity for authenticity
For a deep dive on how mTLS works, see our article on mTLS by the Book.
The Hard Part of mTLS: Proving Identity
If mTLS is so great, what’s the catch? The hard part is establishing a secure mechanism for services to prove their identity to each other.
For regular TLS, it used to be hard to manage the certificates that prove the identity of a server to its clients. With the advent of Let’s Encrypt and the ACME protocol, that’s now much easier. However, managing service identity and certificates in a dynamic (and mostly private) environment like Kubernetes is harder because there are many, often ephemeral services that need strong, provable identities, but can’t practically use a public ACME service.
Rolling your own automated certificate management system is impractical and risky. Getting mTLS certificate management right is hard and the consequences of getting it wrong are bad. You need a trusted, proven way to do it; that’s where service mesh comes in.
Use a Service Mesh, the NIST Standard for Microservices Security
In its standards for microservices security, the National Institute of Standards and Technology (NIST) recommends using a service mesh as a dedicated infrastructure layer to provide core network security features. One of those core features is strong service identity and certificate management to support mTLS. And, Istio—the most widely used service mesh—gives you mTLS support out of the box. Istio transparently provides the infrastructure—including secure naming, strong service identity, and certificate management—for secure communications between your Kubernetes workloads as well as for connections to and from the outside world.
If you want the details of NIST’s standards for microservices security and how Tetrate helps meet them, check out Tetrate’s Guide to Federal Security Requirements for Microservices.
Best Practice One: Don’t Use Self-Signed Certificates
While Istio will implement mTLS for you, it uses self-signed certificates by default so you can see the mesh working right away, with minimal configuration. This makes the initial user experience easy, but it’s not not suitable for production environments. NIST’s guidance (NIST SP 800-204A, SM-DR12) is to disable the ability to generate self-signed certificates entirely.
Best Practice Two: Root Istio’s Trust in Your Existing PKI
If you’re not supposed to use Istio’s default self-signed certificates, what’s the alternative? The short answer is that you should root Istio’s trust in your existing public key infrastructure (PKI). This will enable communication across your Istio deployments in other clusters by ensuring that they all have the same root of trust. Watch our video on external CA with Istio for more information.
Best Practice Three: Use an Intermediate Certificate
How, exactly, do you root Istio’s trust in your existing PKI? Tetrate founding engineer and co-author of NIST’s security standards for microservices, Zack Butcher, has all the details here. But, in short our recommendation is to use an intermediate certificate from your organization’s root certificate authority. This will:
- Allow for fine-grained cert revocation without forcing new certificates across your entire infrastructure at the same time.
- Enable easy rotation of signing certificates.
For step-by-step instructions on how to automate Istio certificate authority (CA) rotation, see our article on automating Istio CA rotation in production at scale.
What’s Next
If you’re new to service mesh and Kubernetes security, we have a bunch of free online courses available at Tetrate Academy that will quickly get you up to speed with Istio and Envoy.
If you’re looking for a fast way to get to production with Istio, check out Tetrate Istio Distribution (TID). TID is Tetrate’s hardened, fully upstream Istio distribution, with FIPS-verified builds and support available. It’s a great way to get started with Istio knowing you have a trusted distribution to begin with, have an expert team supporting you, and also have the option to get to FIPS compliance quickly if you need to.
Once you have Istio up and running, you will probably need simpler ways to manage and secure your services beyond what’s available in Istio, that’s where Tetrate Service Bridge comes in. You can learn more about how Tetrate Service Bridge makes service mesh more secure, manageable, and resilient here, or contact us for a quick demo.
Further Resources
Watch our videos on: