Communication Protocol of API Microgateway Components

WSO2 API Microgateway uses an implementation of Envoy's xDS protocol to communicate between its components, especially the Adapter -> Enforcer and Adapter -> Router communication. Here, the Adapter -> Router communication is already implemented by Envoy and its control plane. WSO2 implements the same protocol for Adapter -> Enforcer communication.

Envoy xDS protocol is implemented on top of gRPC. This allows both the server and the client to stream data between each other. Therefore client's can request for required data from the server and server can push the requested data back to client when new data is available.

WSO2 xDS Implementation

WSO2 xDS implementation is mainly used for communication between Adapter and Enforcer. Using this communication link, Enforcer receives all latest updates of resources required during the startup and runtime. These resources can be APIs, Configurations, Subscriptions, Revoked Tokens etc. Enforcer then uses above data to populate in memory data structures and validate requests based on provided configurations.

Following is the request/response flow of Adapter -> Enforcer xDS communication. 1. During startup, the Enforcer sends the initial DiscoveryRequest to the Adapter.

This request mainly specifies the expected type of the resource (e.g.,: API, Config, Application, Subscription) by xDS client (Enforcer).
  1. The Adapter checks if new resources are available in its cache, for the requested resource type.
    • If available, the Adapter sends a DiscoveryResponse.
    • If the resource is unavailable the Adapter doesn't respond to the client immediately. It waits until a new resource update happens for the requested resource type.
    • As soon as the new resource is added to the Adapter xDS cache, it responds to the initial client request with with a DiscoveryResponse
  2. When the Enforcer receives a new DiscoveryResponse it extracts the resources from the response and populates in the memory data structures for request validation.
  3. Then the Enforcer would Ack/Nack DiscoveryRequest to the Adapter.
    • If the Enforcer is able to process the DiscoveryResponse successfully, it sends a new DiscoveryRequest as Ack to the last received version of the resource.
    • If the Enforcer is unable to process the DiscoveryResponse successfully, it sends a new DiscoveryRequest as Nack to the last received version of the resource. Version information of this request contains the version of last successfully processed resource version.
  4. The Adapter keeps track of the last Acked version of resource by an Enforcer node and uses that information to decide when and what to send in the next DiscoveryResponse to the Enforcer.
    • When a new resource cache update happens in the Adapter, it notifies this change to all subscribed Enforcer nodes. If an Enforcer node Acked the response, the Adapter will send another response to that Enforcer node only if a new resource version update happens in Adapter resource cache.
Top