Performance Test Results¶
The performance of Choreo Connect was measured using an API that invokes a simple "Netty Echo Service". The task covered by the Netty service is to echo back any request posted to the service. The API used for the test cases was a secured API, which directly invokes the backend through Choreo Connect using JWT tokens.
Tests were performed considering 10, 50, 100, 200, 500, 1000 concurrent users. Concurrent users mean that it consists of multiple users accessing Choreo Connect at the same time. Different Message Sizes (Payloads) were used for the tests. The message sizes used are 50B, 1KiB, 10KiB, and 100KiB. The back-end delay was 0ms. The test client was Apache JMeter. Test scenarios were executed for 20 minutes including a warmup period of 5 minutes. When analyzing test results, test outcomes relevant to that period were removed.
Two key performance metrics were used to measure the performance of each test.
- Throughput: The number of requests that the Choreo Connect processes during a specific time interval (e.g. per second).
- Response Time: The end-to-end latency for an operation of invoking a service in Choreo Connect. The complete distribution of response times was recorded.
Deployment used for the test¶
Given below are the details of the deployment that was used for the performance test that is recorded in this documentation.
| Name | EC2 Instance Type | vCPU | Mem(GiB) |
|---|---|---|---|
| Apache JMeter Client | c5.large | 2 | 4 |
| Apache JMeter Server 01 | c5.xlarge | 4 | 8 |
| Apache JMeter Server 02 | c5.xlarge | 4 | 8 |
| AWS EKS cluster (three nodes) | c5.xlarge | 4 | 8 |
- The operating system is Amazon Linux 2018.03
- Java version is JDK 8
Performance test scripts¶
All scripts used to run the performance tests and analyze results are in the following repositories.
Results¶
Choreo Connect is powered by the open source project called Envoy Proxy. Envoy allows to change number of worker threads used for the computations. More details regarding the envoy --concurrency configuration are available in here. Additionally, an explanation for the Envoy Threading Model includes here.
Since we used the c5.xlarge instances (CPU count = 4) for the performance tests and to do a test result comparison, we executed the above-mentioned user counts and payload sizes considering two concurrency levels. We assigned below describing CPU amount and concurrency count to the Router when applying Choreo Connect test deployments. In order to match the increased concurrency level, we increased the CPU allocation to the Router also. Below table lists the test scenarios explained in this documentation.
| Test Scenario | CPU allocation for the Router (m) | Router concurrency level | Test results |
|---|---|---|---|
| 1 | 1000 (1CPU) | 2 | Router with one CPU |
| 2 | 2000 (2CPU) | 4 | Router with two CPU |