Discovery selectors in Istio 1.10

Discovery selectors were one of the new features introduced in Istio 1.10. Discovery selectors allow us to control which namespaces Istio control plane watches and sends configuration updates for. By default, the Istio control plane watches and processes updates for all Kubernetes resources in a cluster. Istio configures Envoy proxies in the mesh to reach every workload in the mesh and accept traffic on all ports associated with the workloads.

For example, we have deployed two workloads in separate namespaces — foo and bar. Even though we know that foo will never have to talk to bar and the other way around, the endpoints of one service include the list of discovered endpoints of the other service.

You can watch the explanation and the demo of discovery selectors in this video:

If we run istioctl proxy-config and list all endpoints that the foo workload from the foo namespace can see, you’ll notice an entry for the service called bar:

If Istio keeps updating the proxies with information about every service in the cluster, even though they’re unrelated, we can imagine how this can slow things down.

If this sounds familiar, you probably know that there’s a solution for this already– it’s called the Sidecar resource.

What is the Sidecar resource?

Sidecar resource describes the configuration of sidecar proxies. By default, all proxies in the mesh have the configuration required to reach every workload in the mesh and accept traffic on all ports.

In addition to configuring the set of ports/protocols proxy accepts when forwarding the traffic, we can also restrict the set of services the proxy can reach when forwarding outbound traffic. 

Note that this restriction here is in the configuration only. It’s not a security boundary– you can still reach the services, but the configuration for that service is not propagated to the proxy.

Below is an example of a sidecar proxy resource in the foo namespace that configures all workloads in that namespace to only see the workloads in the same namespace as well as workloads in the istio-system namespace.

If we re-run the proxy-config command and list the endpoints service foo can see, we’ll notice the service bar is not in the list anymore.

What are discovery selectors?

We can configure discovery selectors at the mesh level in MeshConfig. Discovery selectors are a list of Kubernetes selectors that specify the set of namespaces that Istio watches and updates when pushing configuration to the sidecars.

Like with the Sidecar resource, the discoverySelectors can limit the number of items that are watched and processed by Istio.

We can update the IstioOperator to include the discoverySelectors field as shown below:

The above example sets the env=test as a match label. That means workloads in namespaces labeled with env=test will be included in the list of namespaces Istio watches and updates.

If we label the foo namespace with env=test label and then list the endpoints, we’ll notice there are not as many endpoints listed in the configuration now. That’s because the only namespace we labeled is the foo namespace and that’s the only namespace the Istio control plane watches and sends updates for.

$ istioctl proxy-config endpoints deploy/foo.foo
ENDPOINT                         STATUS      OUTLIER CHECK     CLUSTER
10.4.1.5:80                      HEALTHY     OK                outbound|80||foo.foo.svc.cluster.local
127.0.0.1:15000                  HEALTHY     OK                prometheus_stats
127.0.0.1:15020                  HEALTHY     OK                agent
unix://./etc/istio/proxy/SDS     HEALTHY     OK                sds-grpc
unix://./etc/istio/proxy/XDS     HEALTHY     OK                xds-grpc

If we label the namespace bar as well and then re-run the istioctl proxy-config command, we’ll notice the bar endpoint shows up as part of the configuration for the foo service.

$ istioctl proxy-config endpoints deploy/foo.foo
ENDPOINT                         STATUS      OUTLIER CHECK     CLUSTER
10.4.1.5:80                      HEALTHY     OK                outbound|80||foo.foo.svc.cluster.local
10.4.4.4:80                      HEALTHY     OK                outbound|80||bar.bar.svc.cluster.local
127.0.0.1:15000                  HEALTHY     OK                prometheus_stats
127.0.0.1:15020                  HEALTHY     OK                agent
unix://./etc/istio/proxy/SDS     HEALTHY     OK                sds-grpc
unix://./etc/istio/proxy/XDS     HEALTHY     OK                xds-grpcd

How are discovery selectors different from sidecar resources?

One of the key differences between the discovery selectors and a sidecar resource is that with the discoverySelectors you can include namespaces that the Istio control plane will watch and respond to.

For example, you’ve noticed above how the number of endpoints drastically decreased when we used a discoverySelector. That’s because we only included a single namespace. So there are no kube-system namespaces nor other system or non-system namespaces. Without the discoverySelectors, all namespaces are automatically included and the Istio control plane will watch them and push configuration to all workloads.

Both the discoverySelectors and sidecar resources can be applied globally. However, discoverySelectors will typically be configured by the mesh administrators, while the product teams or service owners will use sidecar resources to configure their own namespaces and workloads.

You can also combine the two. You can imagine mesh administrators configuring which namespaces Istio watches using the discoverySelectors. Similarly, service owners are creating sidecar resources either at the global, namespace, or workload level to fine-tune the configuration.

Resources

• Istio 1.10 release announcement
• Istio sidecar resource
• Discovery selectors YouTube video