Pattern 6: API-M Deployment with IS as Key Manager¶

This deployment consists of a single API-M node with a single API-M runtime with IS configured as a third party key manager.

Contents¶

Pattern 6: API-M Deployment with IS as Key Manager
Contents
Prerequisites
Minimal Configuration
Further IS Customizations
Configuration

Prerequisites¶

Before you begin, ensure you have the following prerequisites in place:

Set Up Basic Configurations¶

Info

The following tools and configurations are necessary for deploying WSO2 API-M in a Kubernetes environment.

Install the required tools:
Git
Helm
Kubernetes client
Ensure you have a running Kubernetes cluster.
Install a routing controller. Choose either:
Envoy Gateway (disabled by default in 4.6.x) - RECOMMENDED
NGINX Ingress Controller (enabled by default in 4.6.x for backward compatibility) - LEGACY

Add the WSO2 Helm chart repository:

helm repo add wso2 https://helm.wso2.com && helm repo update

Build WSO2 Identity Server Docker Image¶

This deployment pattern uses WSO2 Identity Server 7.x as a third-party Key Manager.
Download the WSO2 Identity Server Docker image from DockerHub or use the WSO2 Private Docker Registry if you have an active WSO2 subscription.
Since WSO2 IS 7.x needs to be configured as a Key Manager for WSO2 API Manager, you need to create a custom Docker image with the necessary configurations and extensions.
Below is a sample Dockerfile to build a custom WSO2 IS image for use as a Key Manager:

FROM registry.wso2.com/wso2-is/is:7.2.0.0

ARG USER=wso2carbon
ARG USER_HOME=/home/${USER}
ARG WSO2_SERVER_NAME=wso2is
ARG WSO2_SERVER_VERSION=7.0.0
ARG WSO2_SERVER=${WSO2_SERVER_NAME}-${WSO2_SERVER_VERSION}
ARG WSO2_SERVER_HOME=${USER_HOME}/${WSO2_SERVER}

# Add notification event handler JAR for API Manager integration
ADD --chown=wso2carbon:wso2 https://maven.wso2.org/nexus/content/repositories/releases/org/wso2/km/ext/wso2is/wso2is.notification.event.handlers/2.0.8/wso2is.notification.event.handlers-2.0.8.jar ${WSO2_SERVER_HOME}/repository/components/dropins

After building your custom Docker image, push it to your container registry so it can be accessed by your Kubernetes cluster:
```
docker build -t CONTAINER_REGISTRY/wso2is-km:7.2.0.0 .
docker push CONTAINER_REGISTRY/wso2is-km:7.2.0.0
```

Configure WSO2 Identity Server as Key Manager¶

This section explains how to configure WSO2 Identity Server 7.x as a Key Manager for WSO2 API Manager. In this deployment pattern, both API Manager and Identity Server run as separate containerized applications in the Kubernetes cluster.

Info

Before you begin: You need to import the public certificate of the WSO2 Identity Server 7.x to the truststore of the WSO2 API Manager, and vice-versa. For information on importing the certificates, see the Importing certificates to the truststore guide.

To configure WSO2 Identity Server 7.x to work as a Key Manager with WSO2 API Manager, you need to apply the following configurations:

Configure WSO2 Identity Server using the Helm chart's values.yaml file:

    deploymentToml:
      extraConfigs: |
        [oauth]
        authorize_all_scopes = true

        [[resource.access_control]]
        context="(.*)/scim2/Me"
        secure=true
        http_method="GET"
        cross_tenant=true
        permissions=[]
        scopes=[]

        [[event_listener]]
        id = "token_revocation"
        type = "org.wso2.carbon.identity.core.handler.AbstractIdentityHandler"
        name = "org.wso2.is.notification.ApimOauthEventInterceptor"
        order = 1
        [event_listener.properties]
        notification_endpoint = "https://<APIM_HOST>:<APIM_PORT>/internal/data/v1/notify"
        username = "${admin.username}"
        password = "${admin.password}"
        'header.X-WSO2-KEY-MANAGER' = "WSO2-IS"

Minimal Configuration¶

If you want to quickly try out WSO2 API Manager with WSO2 Identity Server 7.x as a Key Manager on Kubernetes with minimal configuration, you can use the default values provided in the default_values.yaml file.

Quick Start Configuration

This minimal configuration includes:

H2 database (embedded)
Default keystore and truststore
Basic settings for testing purposes

Note: This configuration is ideal for development environments or quick evaluation but is not recommended for production use.

Before you begin

You need to import the public certificate of the WSO2 Identity Server 7.x to the truststore of the WSO2 API Manager, and vice-versa. For information on importing the certificates, see the Importing certificates to the truststore guide.

Follow the steps in the 1.2 Mount Keystore and Truststore section to create a Kubernetes secret containing the keystore and truststore files. Here you will need two keystores: one for the API Manager and one for the Identity Server. The truststore should contain the public certificate of the Identity Server.

To add external keystores and truststores to IS, you can enable externalJKS and define the secretName
To add external keystores and truststores to API Manager, you can specify jksSecretName

First download the IS values.yaml

helm show values wso2/identity-server --version next > default_values.yaml

Update the IS default_values.yaml file with the above configurations.
Deploy IS with minimal configuration using the following command:

helm install is wso2/identity-server --version next \
-f default_values.yaml

Download the APIM values.yaml

curl -o apim_default_values.yaml https://raw.githubusercontent.com/wso2/helm-apim/4.6.x/docs/am-pattern-0-all-in-one/default_values.yaml

Update the apim_default_values.yaml file as needed for your environment (e.g., keystore/truststore, database, etc.).
Deploy API Manager with minimal configuration using the following command:

helm install apim wso2/wso2am-all-in-one --version 4.6.0-1 -f apim_default_values.yaml

In 4.6.x, the default values keep NGINX Ingress enabled for backward compatibility. If you want to use Gateway API instead, follow the steps in 1.1 Add Gateway API controller or Ingress controller, install Envoy Gateway, apply the sample Gateway manifest, and enable Gateway API in your Helm values before deploying.

For this pattern, you will need to deploy both API Manager and Identity Server in your Kubernetes cluster. Configure the values files for both API Manager and Identity Server with the necessary settings and deploy them using Helm.

Further IS Customizations¶

For advanced deployment scenarios and further customizations of WSO2 Identity Server on Kubernetes, refer to the official WSO2 Identity Server Kubernetes deployment documentation. This guide covers topics such as:

Customizing Helm chart values for production
Enabling persistence and external databases
Integrating with external identity providers
Configuring monitoring and logging
Scaling and high availability options

Review these resources to tailor your deployment to your specific requirements and production standards.

Configuration¶

1. General Configuration of Helm Charts¶

The Helm charts for the API Manager deployment are available in the WSO2 Helm Chart Repository. You can either use the charts from the repository or clone the repository and use the charts from the local copy.

Resource Naming Convention

The helm naming convention for APIM follows a simple pattern:

<RELEASE_NAME>-<CHART_NAME>-<RESOURCE_NAME>

1.1 Add Gateway API controller or Ingress controller¶

You can use either Envoy Gateway (Gateway API-based) or NGINX Ingress Controller (Ingress-based) for routing traffic to API Manager. In 4.6.x, charts keep NGINX Ingress enabled by default for backward compatibility. If you want to use Gateway API instead, disable Ingress and enable Gateway API in your Helm values.

Note: It is recommended to use Gateway API with Envoy Gateway instead of NGINX Ingress Controller (Deprecated).

TLS Certificate Configuration (Required for both options)

Create a Kubernetes secret containing the TLS certificate and private key. This secret is used for TLS termination at the load balancer level.

kubectl create secret tls my-tls-secret --key <private key filename> --cert <certificate filename> -n <namespace>

If you use Gateway API, reference this secret in the TLS listeners of your Gateway manifest. If you use NGINX Ingress Controller, set tlsSecret to this secret name in Helm values.

Option 1: Envoy Gateway (Gateway API-based approach) - RECOMMENDED

It is recommended to use Gateway API with Envoy Gateway instead of NGINX Ingress Controller. Gateway API provides a more expressive, extensible, and role-oriented API for configuring traffic routing in Kubernetes.

Install Envoy Gateway.

helm install envoy-gateway oci://docker.io/envoyproxy/gateway-helm \
  --version v1.7.0 -n envoy-gateway-system \
  --set config.envoyGateway.extensionApis.enableBackend=true \
  --set envoyGateway.gateway.experimentalFeatures.enabled=true \
  --create-namespace

Create and apply Gateway and GatewayClass resources.

kubectl apply -f https://raw.githubusercontent.com/wso2/helm-apim/4.6.x/docs/assets/sample-gateway.yaml -n <namespace>

Ensure that the hostnames and Gateway name in your Gateway manifest match the values configured in your Helm chart. Also ensure that the TLS secret created above is referenced in the Gateway listeners.

Create a ConfigMap containing the CA certificate for backend TLS verification and reference it under backendTLSPolicy.caCertificateConfigMap in the Helm chart values if backend TLS verification is enabled.

Note: A default ConfigMap with the name wso2-ca-cert is created when the defaultConfigMapCreation option is enabled in the values.yaml. This default ConfigMap uses the default certificates provided in the APIM truststore. However, for production deployments, it is recommended to create and manage the ConfigMap with the CA certificate yourself, and set defaultConfigMapCreation to false

kubectl create configmap wso2-ca-cert --from-file=ca.crt=/path/to/your/certificate.pem -n <namespace>

Configure the backend traffic policy in the Gateway configuration to enable session affinity if required.
Update values.yaml to enable Gateway API and configure backend TLS policy.

kubernetes:
  gatewayAPI:
    enabled: true
    gatewayName: "wso2-apim-gateway"
    defaultConfigMapCreation: false
    management:
      enabled: true
      hostname: "am.wso2.com"
    gateway:
      enabled: true
      hostname: "gw.wso2.com"
    websocket:
      enabled: true
      hostname: "websocket.wso2.com"
    websub:
      enabled: true
      hostname: "websub.wso2.com"
    backendTLSPolicy:
      enabled: true
      caCertificateConfigMap: "wso2-ca-cert"
      hostname: "<hostname used in the TLS certificate>"
    backendTrafficPolicy:
      enabled: true
      cookie:
        name: "WSO2_CP_STICKY_SESSION"
        ttl: "0s"

Option 2: NGINX Ingress Controller (Ingress-based approach) - DEFAULT IN 4.6.x

You can install the NGINX Ingress Controller using the official Helm chart

Some sample annotations that can be used with Ingress resources are as follows:

The ingress class should be nginx if you are using NGINX Ingress Controller.
The following annotations can be included in Helm values for Ingress resources depending on requirements. Refer to NGINX annotation documentation for details.

ingressClass: "nginx"
ingress:
  tlsSecret: ""
  ratelimit:
    enabled: false
    zoneName: ""
    burstLimit: ""
  controlPlane:
    hostname: "am.wso2.com"
    annotations:
      nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
      nginx.ingress.kubernetes.io/affinity: "cookie"
      nginx.ingress.kubernetes.io/session-cookie-name: "route"
      nginx.ingress.kubernetes.io/session-cookie-hash: "sha1"

1.2 Mount Keystore and Truststore¶

If you are not including the keystore and truststore into the docker image, you can mount them using a Kubernetes secret. Following steps shows how to mount the keystore and truststore using a Kubernetes secret.
Create a Kubernetes secret with the keystore and truststore files. The secret should contain the primary keystore file, secondary keystore file, internal keystore file, and the truststore file. Note that the secret should be created in the same namespace in which you will be setting up the deployment.
Make sure to use the same secret name when creating the secret and when configuring the helm chart.
If you are using a different keystore file name and alias, make sure to update the helm chart configurations accordingly. In addition to the primary, internal keystores and truststore files, you can also include the keystores for HTTPS transport as well.
Refer the following sample command to create the secret and use it in the APIM.

kubectl create secret generic apim-keystore-secret --from-file=wso2carbon.jks --from-file=client-truststore.jks --from-file=wso2internal.jks -n <namespace>

By default, this deployment uses the default keystores and truststores provided by the relevant WSO2 product. For advanced details with regards to managing custom Java keystores and truststores in a container based WSO2 product deployment please refer to the official WSO2 container guide.

1.3 Encrypting Secrets¶

If you need to use cipher tool to encrypt the passwords in the secret, first you need to encrypt the passwords using the cipher tool. The cipher tool can be found in the bin directory of the product pack. The following command can be used to encrypt the password.
```
sh cipher-tool.sh -Dconfigure
```
Also the apictl can be used to encrypt password as well. Reference can be found in following.
Then the encrypted values should be filled in the relevant fields of values.yaml.
Since internal keystore password is required to resolve the encrypted value in runtime, we need to store the value in the cloud provider's secret manager. You can use the cloud provider's secret store to store the password of the internal keystore. The following section can be used to add the cloud provider's credentials to fetch the internal keystore password. Configuration for aws can be at as below.
```
internalKeystorePassword:
  # -- AWS Secrets Manager secret name
  secretName: ""
  # -- AWS Secrets Manager secret key
  secretKey: ""
```
Please note that currently AWS, Azure and GCP Secrets Managers are only supported for this.

1.4 Configure Docker Image and Databases¶

Add the following configurations to reflect the docker image created previously in the helm chart.
```
wso2:
  deployment:
    image:
      imagePullSecrets:
        enabled: false
        username: ""
        password: ""        
      registry: ""
      repository: ""
      digest: ""
```
If you are using a private Docker registry, you must enable imagePullSecrets.enabled and provide the username and password. - Provide the database configurations under the following section.
```
wso2:
  apim:
    configurations:
      databases:
        apim_db:
          url: ""
          username: ""
          password: ""
        shared_db:
          url: ""
          username: ""
          password: ""
```
- If you need to change the hostnames, update them under the Kubernetes ingress section. - Update the keystore passwords in the security section of the values.yaml file. - Review the descriptions of other configurations and modify them as needed to meet your requirements. A simple deployment can be achieved using the basic configurations provided in the values.yaml file. All configuration options for each Helm chart are documented in their respective component guides: - All-in-one - Universal Gateway - Update the admin credentials in the configuration directory.
```
  # -- Super admin username
  adminUsername: ""
  # -- Super admin password
  adminPassword: ""
```

1.5 Configure SSL in Service Exposure¶

SSL Configuration Best Practices

For WSO2 recommended best practices in configuring SSL when exposing internal services to outside of the Kubernetes cluster, refer to the official WSO2 container guide.

Proper SSL configuration is critical for securing API traffic and maintaining compliance with security standards.

2. Add WSO2 Identity Server as Key Manager¶

After setting up WSO2 Identity Server 7.x, you need to configure API Manager to use it as a Key Manager:

Access the API Manager Admin Portal: https://<API-M-HOSTNAME>:9443/admin
Navigate to Key Managers and click Add Key Manager.
Configure the Key Manager with the following settings:

Field	Value
Name	WSO2IS7
Display Name	WSO2 Identity Server 7
Key Manager Type	WSO2 Identity Server 7
Well-known URL	https://wso2is.km:9443/oauth2/token/.well-known/openid-configuration
Issuer	https://wso2is.km:9443/oauth2/token
Client Registration Endpoint	https://wso2is.km:9443/api/identity/oauth2/dcr/v1.1/register
Introspection Endpoint	https://wso2is.km:9443/oauth2/introspect
Token Endpoint	https://wso2is.km:9443/oauth2/token
Display Token Endpoint	https://wso2is.km:9443/oauth2/token
Revoke Endpoint	https://wso2is.km:9443/oauth2/revoke
Display Revoke Endpoint	https://wso2is.km:9443/oauth2/revoke
UserInfo Endpoint	https://wso2is.km:9443/scim2/Me
Authorize Endpoint	https://wso2is.km:9443/oauth2/authorize
Scope Management Endpoint	https://wso2is.km:9443/api/identity/oauth2/v1.0/scopes
Certificate Type	JWKS
JWKS URL	https://wso2is.km:9443/oauth2/jwks
Username (connector config)	admin
Password (connector config)	admin
WSO2 Identity Server 7 API Resource Management Endpoint	https://wso2is.km:9443/api/server/v1/api-resources
WSO2 Identity Server 7 Roles Endpoint	https://wso2is.km:9443/scim2/v2/Roles
Create roles in WSO2 Identity Server 7	Enable if needed

For all these configurations to work correctly in Kubernetes, you must ensure proper service discovery between API Manager and Identity Server pods. Configure the necessary Kubernetes services and ingresses to enable communication between these components.
Update your Helm chart values to include the Identity Server deployment and services along with API Manager.

Note

When using WSO2 IS 7.x as a Key Manager, note the following limitations:

WSO2 IS 7.x cannot be set up as a Resident Key Manager. It can only be set up as a Third-party Key Manager.
Role creation in WSO2 Identity Server 7.x is supported from WSO2 API Manager 4.4.0.5 update level onwards.

3. Add a DNS record mapping the hostnames and the external IP¶

If you are using Gateway API, obtain the external address by listing Gateway resources.

kubectl get gateway -n <NAMESPACE>

If you are using Ingress instead, obtain the external IP from the Ingress resources.

kubectl get ing -n <NAMESPACE>

Use the value from ADDRESS or EXTERNAL-IP as the external IP.

If the defined hostnames are backed by a DNS service, add DNS records that map the hostnames to the external IP in the relevant DNS service.

If the defined hostnames are not backed by a DNS service, for evaluation purposes you may add entries to the /etc/hosts file on the client side.

For Gateway API:

<EXTERNAL-IP> <kubernetes.gatewayAPI.management.hostname> <kubernetes.gatewayAPI.gateway.hostname> <kubernetes.gatewayAPI.websub.hostname> <kubernetes.gatewayAPI.websocket.hostname>

For Ingress:

<EXTERNAL-IP> <kubernetes.ingress.management.hostname> <kubernetes.ingress.gateway.hostname> <kubernetes.ingress.websub.hostname> <kubernetes.ingress.websocket.hostname>

4. Access Management Consoles¶

If you enabled Gateway API:

API Manager Publisher: https://<kubernetes.gatewayAPI.management.hostname>/publisher
API Manager DevPortal: https://<kubernetes.gatewayAPI.management.hostname>/devportal
API Manager Carbon Console: https://<kubernetes.gatewayAPI.management.hostname>/carbon
Universal Gateway: https://<kubernetes.gatewayAPI.gateway.hostname>
Identity Server Management Console: https://<kubernetes.gatewayAPI.is.hostname>/carbon

If you are using Ingress:

API Manager Publisher: https://<kubernetes.ingress.management.hostname>/publisher
API Manager DevPortal: https://<kubernetes.ingress.management.hostname>/devportal
API Manager Carbon Console: https://<kubernetes.ingress.management.hostname>/carbon
Universal Gateway: https://<kubernetes.ingress.gateway.hostname>