

Distributed Rate Limiting¶

You can enable Distributed Rate Limiting in Choreo Connect using Choreo Connect with WSO2 API Manager as a Control Plane.

Tip

To learn about the concepts of Rate Limiting (Throttling) in Choreo Connect, see API-M as a Control Plane - Rate Limiting.

Enabling Distributed Rate Limiting¶

Rate Limiting in Choreo Connect works with API Manager (Traffic Manager).

Follow the instructions below to enable Distributed Rate Limiting:

Open the Choreo Connect configuration file according to the deployment type you are using.

Click here to see the configuration file location for your Choreo Connect deployment.

Navigate to the correct folder path and open the config.toml or config-toml-configmap.yaml file based on your Choreo Connect deployment.

Deployment	Mode	File name	Directory
Docker Compose	Choreo Connect as a Standalone Gateway	`config.toml`	`<CHOREO-CONNECT_HOME>/docker-compose/choreo-connect/conf/`
Docker Compose	Choreo Connect with WSO2 API Manager as a Control Plane	`config.toml`	`<CHOREO-CONNECT_HOME>/docker-compose/choreo-connect-with-apim/conf/`
Kubernetes	Choreo Connect as a Standalone Gateway	`config-toml-configmap.yaml`	`<CHOREO-CONNECT_HOME>/k8s-artifacts/choreo-connect/`
Kubernetes	Choreo Connect with WSO2 API Manager as a Control Plane	`config-toml-configmap.yaml`	`<CHOREO-CONNECT_HOME>/k8s-artifacts/choreo-connect-with-apim/`

Use the following configurations to enable Distributed Rate Limiting.

 # Throttling configurations
 [enforcer.throttling]
   # Connect with the central traffic manager
   enableGlobalEventPublishing = false
   # Enable global advanced throttling based on request header conditions
   enableHeaderConditions = false
   # Enable global advanced throttling based on request query parameter conditions
   enableQueryParamConditions = false
   # Enable global advanced throttling based on jwt claim conditions
   enableJwtClaimConditions = false
   # The message broker context factory
   jmsConnectionInitialContextFactory = "org.wso2.andes.jndi.PropertiesFileInitialContextFactory"
   # The message broker connection URL
   jmsConnectionProviderURL = "amqp://admin:$env{tm_admin_pwd}@carbon/carbon?brokerlist='tcp://{API-M_HOST/TM_HOST}:5672'"
   # Throttling configurations related to event publishing using a binary connection
   [enforcer.throttling.publisher]
     # Credentials required to establish connection between Traffic Manager
     username = "admin"
     password = "$env{tm_admin_pwd}"
     # Receiver URL and the authentication URL of the Traffic manager node/nodes
     [[enforcer.throttling.publisher.URLGroup]]
       receiverURLs = ["tcp://{API-M_HOST/TM_HOST}:9611"]
       authURLs = ["ssl://{API-M_HOST/TM_HOST}:9711"]

     # Data publisher object pool configurations
     [enforcer.throttling.publisher.pool]
       # Maximum idle number of connections
       maxIdleDataPublishingAgents = 1000
       # Minimum idle number of connections
       initIdleObjectDataPublishingAgents = 200
       # Thread pool core size
       publisherThreadPoolCoreSize = 200
       # The maximum size of the thread pool
       publisherThreadPoolMaximumSize = 1000
       # The timeframe after which the publisher thread pool is terminated in seconds
       publisherThreadPoolKeepAliveTime = 200

     # Data publisher agent configurations
     [enforcer.throttling.publisher.agent]
       # SSL Protocols
       sslEnabledProtocols = "TLSv1.2"
       # Ciphers
       ciphers = "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256,TLS_DHE_RSA_WITH_AES_128_CBC_SHA256,TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256  ,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_DSS_WITH_AES_128_GCM_SHA256  ,TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA,SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA,TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
       # The size of the queue event disruptor which handles events before they are published
       # The value specified should always be the result of an exponent with 2 as the base
       queueSize = 32768
       # The maximum number of events in a batch sent to the queue event disruptor at a given time
       batchSize = 200
       # The number of threads that will be reserved to handle events at the time you start
       corePoolSize = 1
       # Socket timeout
       socketTimeoutMS = 30000
       # The maximum number of threads that should be reserved at any given time to handle events
       maxPoolSize = 1
       # The amount of time which threads in excess of the core pool size may remain idle before being terminated.
       keepAliveTimeInPool = 20
       # The time interval between reconnection
       reconnectionInterval = 30
       # TCP connection pool configurations (for data publishing)
       maxTransportPoolSize = 250
       maxIdleConnections = 250
       evictionTimePeriod = 5500
       minIdleTimeInPool = 5000
       # SSL connection pool configurations (for authentication)
       secureMaxTransportPoolSize = 250
       secureMaxIdleConnections = 250
       secureEvictionTimePeriod = 5500
       secureMinIdleTimeInPool = 5000

Tip

When using multiple traffic manager nodes for high availability, you can use a configuration as given below.

[enforcer.throttling]
  enableGlobalEventPublishing = true
  jmsConnectionProviderURL = "amqp://admin:$env{tm_admin_pwd}@carbon/carbon?failover='roundrobin'%26cyclecount='2'%26brokerlist='tcp://<Traffic-Manager-1-host>:5672?retries='5'%26connectdelay='50';tcp://<Traffic-Manager-2-host>:5672?retries='5'%26connectdelay='50''"

  [enforcer.throttling.publisher]
    username = "admin"
    password = "$env{tm_admin_pwd}"

    [[enforcer.throttling.publisher.URLGroup]]
      receiverURLs = ["tcp://<Traffic-Manager-1-host>:9611"]
      authURLs = ["ssl://<Traffic-Manager-1-host>:9711"]

    [[enforcer.throttling.publisher.URLGroup]]
      receiverURLs = ["tcp://<Traffic-Manager-2-host>:9611"]
      authURLs = ["ssl://<Traffic-Manager-2-host>:9711"]

Conditional Rate Limiting¶

There can be situations where certain APIs require more granular level of Rate Limiting. Assume you want to provide limited access to a certain IP range or a type of client application (identified by User-Agent header). For these scenarios, a simple throttle policy with API/resource level limits is not sufficient. To address complex throttling requirements as above, Choreo Connect is capable of throttling requests based on several conditions. The following types of conditions are supported.

Specific IP or IP range conditions.

This condition can be used to provide specific limits to a certain IP address or a range of IP addresses.
Header conditions.

This condition can be used to set specific limits to a certain header value.
Query parameter conditions.

Same as the header conditions, this allows applying a specific limit to a certain query parameter value.
JWT claim conditions.

This type of condition will evaluate the backend JWT and check if it has a specific claim value in it to set the throttle limit.

Configuring and enabling conditional Rate Limiting¶

Conditional Rate Limiting is done via the Advanced Rate Limiting policies in API Manager.