Load Testing HTTP APIs with ApiZapi
An overview of ApiZapi, a new HTTP API load testing SaaS application to run scenario-based tests at increasing virtual user loads.
Note: I am the creator and developer of the ApiZapi load testing tool and am writing this as a high-level overview to instruct others on the general capabilities of the tool.
Overview
ApiZapi is a cloud-hosted service that allows you to perform scenario-based testing on your HTTP API endpoints and do that at increasing load by simulating many thousands of user interactions. This tool would cover straight load testing, spike and breakpoint testing. With the application, you specify a series of HTTP calls that represent real-world usage for the interactions you want sent to your API endpoint. The goal is to mimic the realistic interactions of your users, and to run these at scale with multiple virtual users (VUs) all at the same time. The result is that you are putting your infrastructure under a realistic load. These interactions are what comprise the workload. There is also an initialization and shutdown sequence that you can configure. As part of the initialization sequence, you can make calls to create accounts and/or sign into accounts that will be used during the workload run.
ApiZapi can be used to uncover single points of failure and bottlenecks. It can best be used in conjunction with your logging infrastructure and other Application Performance Management tools to do further diagnosis of issues from the load test. It is then up to you to alter your architecture and tune your hosting to handle your ideal normal loads and test that you can also handle the likely peak load usage. You want to stay ahead of any customer usage traffic as it increases and know what your limits are. Tests can be run manually, or can be triggered on a daily schedule. Tests can also be kicked off as part of your CI/CD pipeline through an HTTP POST call. A Webhook is used to receive completion status back into your CI/CD environment.
Getting started
Signing up is extremely easy, just enter an email and password (there is a join code required until the Beta period is over) and you are good to go with a free version that has most of the same capabilities, but runs with a few restrictions.
To enjoy the full capabilities, you need to go to the profile page and switch plans. You will not be charged any monthly subscription fee, you only pay for the seconds of usage for all your test runs for the month. For example, if you go a complete month without running any tests, there is no charge for that month.
There is a dashboard view that you can look at that shows you the up-to-date charges for the current month and a view of past months. This is also where you can enter a dollar amount to set a limit for what you can be charged. If you hit this limit, you will no longer be able to run any tests for that month, unless you increase your spending limit.
Validation of site access
Once you have created a test, you can click to edit it and then eventually run it. This is true if you set it up to run with up to twenty virtual users (VUs). For VU counts greater than 20 it is required that you prove you have access to test your site. In order to prevent ApiZapi from being used as a DDoS attack platform, we require you to provide a special DNS TXT entry to prove that you have permission to perform load testing on your API endpoint. Alternatively, you can provide a file for ApiZapi to access. The file will be a JSON file with a specific id in it. Click the Site Verification button for details on how to accomplish this.
Note: It is worth mentioning that you could use ApiZapi to test an API endpoint that is running on your local machine. It is very easy to do. Simple put in the IP address and port to connect to such as “http://42.27.0.33:3000". Use http://checkip.amazonaws.com/ to see what your IP address is. You can then run your service on your local machine, such as Node.js Express services and start interacting with it. You most likely have a firewall or some type of router in front of your actual machine. For example, with a home router, you can set up port forwarding from that actual external facing IP address to your actual internal machine IP address and port. You must also provide the apizapiverify.json file to be accessed if you set VUs to be more than 20. You can test that you can get the file with your browser first.
Viewing your load tests
You can see a list of tests with a few high-level details about each. At the end of each test row are icon actions you can click on. From the list of existing tests, you can click on one to navigate to the run page, or navigate to the edit page.
Load Test editing
You will click Add Test to create a new load test. From here, there are many tabs in the UI that you can fill out for specifying exactly what should happen. Config is for the main setup for the server host you will hit, the time to run with interval and scaling numbers. Trigger is for setting up a recurring run as well as setting the Webhooks in and out, for triggering externally or for receiving notification of the completion of a run. Proxy Fanout is for setting up usage of AWS regions for where calls will route through to be able to simulate traffic from around the world. Runtime Data is where you can set up internal variables for usage either across all VUs, or for each VU. The last two tabs are for adding the actual workload HTTP calls.
HTTP calls
Calls can be placed into several lists as a “Card”. There are initialize and shutdown lists for calls that need to happen prior to and after to the main workload cards list. Each card represents an individual HTTP call. For each card, you specify the call and the expected results.
The Workload Init-Shut cards are run sequentially. As VUs (Virtual Users) are brought into action because of the step count scaling, each will go through their sequence on their own. Workload init cards run as the VU scaling brings them in. For example, there could be 20 VUs initially that all do the first card and then at another step interval, another set of 20 do their init step. These are run sequentially and only run once per VU.
The Workload sequence of cards will run after the initialization sequence has been completed. Workload cards are randomly run and are not run sequentially like the initialization and shutdown sequence cards are. You can still drag the cards around, however, this is merely for your convenience. For a given interval firing, each virtual user will have a card randomly selected for the HTTP call that will be made.
You can use the toggle control on each card to enable and disable individual cards. There is also a set of radio buttons labeled Low, Med, and High. This sets the weighting for each to determine how often each will be called. Workload cards can also be followed by a secondary call that needs to happen in sequence, these are found in the Secondary Calls list.
The Call Details dialog for the workload cards usage is fairly straightforward. You set up the path you want to hit for the chosen host. You can set header values, including cookies, etc. If the verb is POST, then you can also specify a JSON body for that call. There is a special syntax to pull in variables with ${globalContext.???} and ${userContext.???}. These are values that are either set in the UI upfront, or are values that are saved away in the results processing for each call. Each virtual user will have their own context for these, as well as a global shared context for variables. An example of a user context variable as seen here is that of the token. This represents a JWT token that would have been set from a previous card call that make some call to an authorization route that returned the JWT. Cookies that are returned in response are kept and reused on subsequent calls, just as browser interaction would do. There is also a way to extract the value of a cookie and reuse that as well.
If you change the HTTP Verb to POST or PUT, you can add a body to be sent as part of the request. You can send JSON, Text, x-www-form-urlencoded, form-data and others. This means that you can send files through as well. You also collect data from a response and save it in user or global context and send it back out as JSON, Text, HTML, or XML.
In the Response Handling dialog, you can take header or body response values and save them away into the global and user context. You can also perform tests against the body, response status and average time it took.
Results view
The results from a given run can be viewed to understand exactly what the load results were and determine the outcome and success.
You can look at the Card Stats tab for specific details about how individual calls are performed. The status column gives you the overall information for a given card for if it met your expectations. It would be in a Failed state if either the average response time threshold was surpassed, or if there were any errors in processing the return, such as an unexpected status code.
You can look at the Return Codes tab for specific details about failures. These could be failures because a return code came back that was not what you expected. This gives you the time of the error, the virtual user count at that time, and other details that will help you troubleshoot the cause. You can see in the image that this particular error is a server returning “Bad Gateway”, and only happens when the virtual user count reached 20K.
Conclusion
There is a lot more to go over then can fit in this short article. Hopefully, you now have a good understanding of the fundamentals. Go to the user documentation if you have more time and want a more in-depth look at the capabilities and some use cases.
More content at plainenglish.io
