Learn to use the Pinniped Supervisor without the Concierge
This tutorial shows how to use the Pinniped Supervisor and Pinniped command-line tool to provide federated identity with a single sign-on user experience on many Kubernetes clusters, without using the Pinniped Concierge. If you would like to learn how to use the Pinniped Supervisor and Concierge together, please instead see this other tutorial:
The Kubernetes API server can be configured to trust an OIDC identity provider to provide authentication
for the cluster. This is done by setting the
--oidc-* command-line flags of the
kube-apiserver command-line tool inside
the Pod spec of the Kubernetes API server Pods. The details of these command-line flags are described in the
Kubernetes kube-apiserver documentation
and in the
Kubernetes authentication documentation.
These flags can be used to configure the Pinniped Supervisor as the OIDC provider for your clusters.
If your cluster’s Kubernetes distribution does not allow you to adjust these command-line flags, then this approach will not work for those clusters. For example, most cloud providers will not allow these flags to be adjusted. In that case, you can use the Pinniped Concierge to provide the equivalent functionality on those clusters.
Additionally, if you would like to be able to easily change this configuration at any time on your cluster, then
you can use the Pinniped Concierge instead of these
kube-apiserver command-line flags, even on a cluster where you
have control over these flags. The Pinniped Concierge offers a custom resource called JWTAuthenticator which can
be dynamically configured at any time, which is roughly equivalent to using these
kube-apiserver command-line flags.
One Pinniped Supervisor can provide authentication for many Kubernetes clusters. Each cluster can either use
the Pinniped Concierge or use the
kube-apiserver command-line flags, and both approaches can be mixed and matched on different
clusters all with a single Pinniped Supervisor.
A Kubernetes cluster of a type which allows you to adjust the command-line flags of
Don’t have a cluster handy? Consider using kind on your local machine. See below for an example of using kind.
A kubeconfig where the current context points to the cluster and has administrator-like privileges on that cluster.
A Pinniped Supervisor already installed and running on another cluster, and already configured with a working FederationDomain, TLS certificates, and an external identity provider (e.g. an OIDCIdentityProvider, LDAPIdentityProvider, or ActiveDirectoryIdentityProvider).
Don’t have a Pinniped Supervisor ready? Please refer to the other documents on this site to help you get one up and running and sufficiently configured. This tutorial does not show to install and configure the Pinniped Supervisor. Those steps are shown in the Concierge with Supervisor tutorial, so if you would like you could follow the steps of that tutorial to install and configure a Pinniped Supervisor before returning to this tutorial.
How to configure the kube-apiserver flags
kube-apiserver command-line flags may be configured on each cluster to trust your Pinniped Supervisor to provide
user authentication to that cluster. Add the following flags to the existing list of flags for your cluster:
# Make this exactly match the spec.issuer of your Supervisor's FederationDomain. # Note: It seems like the kube-apiserver pod cannot resolve `cluster.local` # DNS names, so don't use one of those DNS names as the issuer. --oidc-issuer-url="https://my-supervisor.example.com/my-issuer" # This is only required if the kube-apiserver pod is not going to trust your # Supervisor's FederationDomain's TLS certificates, e.g. if you used a # self-signed CA. Make this match where you mounted the CA PEM file into # your control plane node's filesystem, which must be under a directory # that the kube-apiserver container is going to volume mount. --oidc-ca-file="/etc/ca-certificates/supervisor/root-ca.pem" # Choose a unique value for each cluster here. By making this unique, the # Supervisor will be able to issue ID tokens for this cluster that cannot # be used on any other cluster, which improves security. Do not use the # special value "pinniped-cli" or any value that contains the substring # ".pinniped.dev", because these special values are reserved for other # purposes. --oidc-client-id="my-cluster-342klb7h" # Use these exact values. These are based on how the Supervisor issues ID # tokens. Do not change these values. --oidc-signing-algs="ES256" --oidc-username-claim="username" --oidc-groups-claim="groups" # These are optional, use any value you prefer here, or do not set these flags. # These strings will be prepended to the username and group name strings # that were determined by the Supervisor during user authentication to decide # the final username and group names, but only on this cluster. Refer to the # Kubernetes kube-apiserver docs for more information about these flags. --oidc-username-prefix="pinniped:" --oidc-groups-prefix="pinniped:"
--oidc-client-id to choose a string that is unique for each cluster. This could be a GUID or some other random
letters and numbers, and can be combined with a human-readable portion if desired. When a user first authenticates
to the Pinniped Supervisor, it will issue an ID token with the
aud (audience) claim set to the name of the client,
which will be either
pinniped-cli (for the kubectl use case) or will start with
(for a web app client using the OIDCClient CR). Avoid using these names for the
--oidc-client-id value to ensure
that these initial ID tokens cannot be used to authenticate to your cluster. Next, the client will make another call
to the Pinniped Supervisor to obtain a new ID token which is scoped to one specific cluster. This new token will have the
claim’s value changed to the cluster’s unique value. This is the only token that will be sent to that cluster.
--oidc-client-id flag of
kube-apiserver tells it to validate the
aud claim on the incoming ID tokens.
This cluster-scoped ID token will not be accepted by any other cluster, because no other cluster should use the
same unique value for this flag. This improves the security of your clusters by making this token only valuable on
a single cluster.
The procedure to add these command-line flags to the
kube-apiserver’s list of command-line flags depends on
the distribution of Kubernetes that you are using. Please refer to the documentation for your distribution.
Note that you can configure these flags even if the Pinniped Supervisor is not running yet. The Kube API server will continuously try to find the Pinniped Supervisor at the configured URL until it works.
How to create a kubeconfig for the cluster
You can use the Pinniped command-line tool to create a kubeconfig that will work with your Pinniped Supervisor and your cluster.
When using the Pinniped Concierge on the cluster, the Pinniped command-line tool will auto-discover many settings for the kubeconfig.
However, when configuring the
kube-apiserver flags instead of using the Pinniped Concierge, then you must give
more hints to the Pinniped command-line tool to help it create the kubeconfig.
Here is how you would create a kubeconfig for the example configuration of the
kube-apiserver flags shown above:
pinniped get kubeconfig \ --no-concierge \ --oidc-issuer "https://my-supervisor.example.com/my-issuer" \ --oidc-ca-bundle "supervisor_root_ca_cert.pem" \ --oidc-request-audience "my-cluster-342klb7h" \ --kubeconfig "my-admin-kubeconfig-for-this-cluster.yaml" \ > pinniped-kubeconfig.yaml
--no-conciergeto indicate that you are not using the Pinniped Concierge on this cluster.
--oidc-issuervalue should exactly match the issuer URL configured in the
--oidc-issuer-urlflag and the
spec.issuerof your Supervisor’s FederationDomain.
--oidc-ca-bundleflag is only required when the machine on which you are running this command is not going to trust your Supervisor’s FederationDomain’s TLS certificates, e.g. if you used a self-signed CA. This file would have the same content as the file that you provided to
--oidc-request-audiencevalue should exactly match the value that you chose for the
--kubeconfigvalue is the admin kubeconfig of the cluster for which you would like to generate a Pinniped-compatible kubeconfig. This is not needed when your current context is already set to the cluster.
- The command will output the new Pinniped-compatible kubeconfig to stdout. Optionally redirect this to a file.
Example of configuring these kube-apiserver flags on kind
kind is a tool for creating and managing Kubernetes clusters on your local machine which uses Docker containers as the cluster’s nodes. This is a convenient way to try out this feature on a local non-production cluster.
The following steps deploy the latest release of Pinniped on kind using the local-user-authenticator component as the authenticator.
Install the tools required for the following steps.
Install kind, if not already installed. For example,
brew install kindon macOS.
kind depends on Docker. If not already installed, install Docker, for example
brew cask install dockeron macOS.
This demo requires
kubectl, which comes with Docker, or can be installed separately.
Create a kind configuration yaml file to ask kind to configure the
kube-apiserverflags. Note that some of these values will need to be adjusted as described in the comments below before using this file in the next step.
kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 nodes: - role: control-plane extraMounts: # Adjust this path to your CA PEM file. Use an absolute path. - hostPath: /Users/ryan/supervisor_root_ca_cert.pem # This is under /etc/ca-certificates because the kube-apiserver # pod already mounts the /etc/ca-certificates host path on kind. containerPath: /etc/ca-certificates/supervisor/root-ca.pem readOnly: true kubeadmConfigPatches: - | apiVersion: kubeadm.k8s.io/v1beta3 kind: ClusterConfiguration apiServer: extraArgs: # Adjust the values for all these flags as described in the # sections above. oidc-issuer-url: "https://my-supervisor.example.com/my-issuer" oidc-client-id: "my-cluster-342klb7h" # choose a unique value oidc-signing-algs: "ES256" oidc-username-claim: "username" oidc-groups-claim: "groups" oidc-username-prefix: "pinniped:" oidc-groups-prefix: "pinniped:" oidc-ca-file: "/etc/ca-certificates/supervisor/root-ca.pem"
Save this as a new yaml file, for example
Create a new Kubernetes cluster using
kind create cluster --config cluster-config.yaml. Optionally provide a cluster name using the
--nameflag. kind automatically updates your kubeconfig to point to the new cluster as a user with administrator-like permissions. Wait for this command to successfully complete before moving on.
Create a Pinniped-compatible kubeconfig for this new cluster. Note that the previous
kind create clustercommand automatically changed your current kubeconfig context to point at the new cluster.
pinniped get kubeconfig \ --no-concierge \ --oidc-issuer "https://my-supervisor.example.com/my-issuer" \ --oidc-ca-bundle "supervisor_root_ca_cert.pem" \ --oidc-request-audience "my-cluster-342klb7h" \ > /tmp/pinniped-kubeconfig.yaml
Try using the generated kubeconfig to issue arbitrary
kubectlcommands. The first time you run a kubectl command, you will be automatically prompted to authenticate using the external identity provider that is configured in the Pinniped Supervisor.
kubectl --kubeconfig /tmp/pinniped-kubeconfig.yaml get pods -A
Because this user has no RBAC permissions on this cluster, the previous command results in the error
Error from server (Forbidden): pods is forbidden: User "your-username-will-show-here" cannot list resource "pods" in API group "" at the cluster scope, where
your-username-will-show-herewill be your actual username from the Pinniped Supervisor. However, this error does prove that you are authenticated and acting as that identity from the Pinniped Supervisor on this kind cluster.
If desired, you can use the administrator kubeconfig to create RBAC RoleBindings and ClusterRoleBindings for that user or for the groups to which that user belongs.
Carry on issuing as many
kubectlcommands as you’d like as that user. You will not be prompted to log in again for 9 hours for this cluster or for any other similarly configured cluster which uses the same Pinniped Supervisor FederationDomain issuer. Each time a few minutes have passed, the next kubectl command will use the Pinniped command-line tool to securely refresh your identity from the external identity provider that is configured in the Pinniped Supervisor without user interaction. During this refresh, your group memberships may be updated from the external identity provider, or you may be prompted to log in again if the Pinniped Supervisor determines that external identity provider does not want your session to continue.
You may find it convenient to set the
KUBECONFIGenvironment variable rather than passing
--kubeconfigto each invocation.
export KUBECONFIG=/tmp/pinniped-kubeconfig.yaml kubectl get namespaces kubectl get pods -A
Alternatively, you could use the
kubectl config viewcommand to merge this kubeconfig into another kubeconfig.
Take a look at the contents of the
/tmp/pinniped-kubeconfig.yamlfile. It does not contain any particular user’s identity, nor does it contain any credentials. It only contains a recipe for how any user can authenticate. You can safely distribute this file to all users of this cluster. Anyone who uses this kubeconfig will be prompted to authenticate using the external identity provider that is configured in the Pinniped Supervisor. Each user will need to install the Pinniped command-line tool on any machine where they would like to use the kubeconfig.
A brief note about the future
There is an outstanding Kubernetes enhancement proposal to make this Kubernetes API server’s OIDC authentication
settings configurable using a new Kubernetes API resource. At the time of writing this Pinniped document, the
proposal is still under review by the Kubernetes maintainers. If implemented in a future release of Kubernetes,
this would remove the need to edit the command-line flags of the
kube-apiserver binary, making it easier
to configure an OIDC provider for your cluster in a standard way. The details and status of this proposal
may be found in KEP-3331.