Recently I am observing a rise of API Testing in the Job descriptions. I once did Web Services testing using SOAP UI testing tool, i.e. verifying the XML request-response. Thought it would be a good time to refresh the concepts. Anyways it was on my list since I started STS. When we look at the Software Testing trends, API testing is rising in priority & relevance. Let’s deep dive into the world of API Testing…
What is API?
API (Application Programming Interface) is a set of procedures and functions that allow interaction between two components of a software application. It enables communication and data exchange between two systems. A software system implementing an API contains functions/sub-routines which can be executed by another software system.
Out of the classic three-tier architecture model:
- Data Tier: The database or file system from where data is retrieved/stored.
- Logic Tier: The brain of the application, this processes the data between the layers, coordinating the application, processing commands, and making logical decisions.
- Presentation Tier: The User interface, which translates tasks into something the user understands.
The logic tier is the API. This is where all of the business logic belongs. It is responsible for taking information from the various User interfaces (UIs), performing calculations and transactions on the database layer and then presenting the results back to the User interface. In certain cases, the logic tier of the application has more complexity, and it is not uncommon that multiple technologies (web server, message queue, etc.) will make up this tier.
A real world example: You book an Ola Share or an Uber pool with a destination address, not aware of the route, traffic & weather conditions. These ride sharing applications then need to communicate with the map service, the traffic and weather services and other specialized applications to guide bookings & your perfect ride. In the modern, interconnected world, we take for granted that all these different systems can speak to each other seamlessly, in reality that would not be possible without APIs.
The API provider defines the set of operations, data formats and protocols that it expects, and the consumer of the API (called the client) will use those rules on the understanding that, as long as it follows the rules, the client will always be able to use the API without having to worry about the internals of the API itself. There are situations where the API is the final product – a public API. A simple Internet search will reveal there are thousands of public APIs for anything from GPS and mapping solutions to locating a radio station.
- Amazon Advertising API: Amazon is known for their products and thus their advertising API accesses their product to discover their functionality and thus advertise accordingly.
- Twitter: The API for twitter is usually in two categories, one for accessing data and the other for interacting with twitter search.
- YouTube: This API used for YouTube includes various functionalities including videos, live streaming, player, etc.
What is API Testing?
In simple words – Testing of the Logic tier (the API interface) to determine if it meet expectations for functionality, reliability, performance, and security.
API testing is used to determine,
- Functional: APIs return the correct response (in the expected format) for a broad range of feasible requests, i.e. output from first application/database is correct and well-structured and useful to another application. The response can be a Pass/Fail status, Data or information or a call to another API.
- Load: API can handle a large amount of calls and react properly to edge cases such as failures and unexpected/extreme inputs.
- Performance: Deliver responses in an acceptable amount of time.
- Security: Type of authentication required, permissions, access controls, whether the sensitive data is transmitted securely over the network and respond securely to potential security attacks.
- API documentation testing — also called discovery testing, the API documentation easily guides the user.
API testing involves testing APIs directly (in isolation) and as part of the end-to-end transactions exercised during integration testing. Beyond RESTful APIs, these transactions include multiple types of endpoints such as web services, ESBs, databases, mainframes, web UIs, and ERPs. API testing is performed on APIs that the development team produces as well as APIs that the team consumes within their application (including third-party APIs). Service virtualization is used in conjunction with API testing to isolate the services under test as well as expand test environment access by simulating APIs/services that are not accessible for testing.
Why API Testing?
APIs now serve as the primary interface to application logic. Agile and DevOps teams working with short iterations and fast feedback loops find that GUI tests require considerable rework to keep pace with frequent change. Tests at the API layer are less brittle and easier to maintain. That’s why we see ‘API Testing’ gaining momentum as one of the job descriptions now-a-days.
The Kind of APIs
Over the years, APIs have evolved from simple code libraries that applications could use to run code on the same computer, to remote APIs that can be used to allow code on one computer to call code hosted somewhere else. Here is a quick list of the more common API technologies that exist in approximate chronological order:
- TCP/IP Sockets
- Remote Procedure Call (RPC)
- Common Object Request Broker Architecture (CORBA)
- Java Remote Method Invocation (RMI) and Enterprise Java Beans (EJBs)
- Microsoft Distributed Component Object Model (DCOM) – also known as ActiveX
- Web Services (SOAP then REST)
SOAP web services make use of the Web Service Definition Language (WSDL) and communicate using HTTP POST requests. They are essentially a serialization of RPC object calls into XML that can then be passed to the web service. The XML passed to the SOAP web services needs to match the format specified in the WSDL.
A RESTful web API (also called a RESTful web service) is a web API implemented using HTTP and REST principles. Unlike SOAP-based web services, there is no “official” standard for RESTful web APIs. This is because REST is an architectural style, unlike SOAP, which is a protocol. Typically REST web services expose their operations as a series of unique “resources” which correspond to a specific URL. Each of the standard HTTP methods (POST, GET, PUT and DELETE) then maps into the four basic CRUD (Create, Read, Update and Delete) operations on each resource. REST web services can use different data serialization methods (XML, JSON, RSS, etc.).
How to perform API Testing?
Since APIs lack a GUI, API testing is performed at the message layer. API testing commonly includes testing REST APIs or SOAP web services with JSON or XML message payloads being sent over HTTP, HTTPS, JMS, and MQ. It can also include message formats such as SWIFT, FIX, EDI and similar fixed-length formats, CSV, ISO 8583 and Protocol Buffers being sent over transports/protocols such as TCP/IP, ISO 8583, MQTT, FIX, RMI, SMTP, TIBCO Rendezvous, and FIX.
API testing is different than other testing types as GUI is not available, and yet you are required to setup initial environment that invokes API with required set of parameters and then finally examines the test result. Instead of using standard user inputs (keyboard) and outputs, in API Testing, we use software to send calls to the API, get output, and note down the system’s response. API Testing requires an application to interact with API. In order to test an API, we need to use a Testing Tool to drive the API.
- Document the API Testing Requirements: What is the purpose of API? What is the workflow of the application? What are the integrations supported by the API? What are the features and functions of the API? Documenting all these API testing requirements is the first thing we need to implement. This will help us in planning API tests throughout the testing process.
- Test Environment: Setting up Test environment involves configuring the database and server for the application’s requirements. Once set up, it’s good to make an API call right away to make sure nothing is broken before we go forward to start more thorough testing.
- Functional: The main goal of API testing is to ensure the logic and functionality of API components with regard to the software package as a whole. With that in mind, a significant amount of the testing effort should be made to flag the functional defects that would most likely cause problems in production, and will also provide the whole team with baseline measures of how the API functions under regular conditions.
- Negative Tests: Thorough API testing should also include sufficient boundary testing and higher stress testing to see how the API reacts to non-ideal conditions. Test suites should ideally include at least a few unusual cases, such as non-ASCII characters, improper data types, or very large number. Another area that should not be overlooked is error testing, as it is vital that the API in test does not break or crash in response to poorly formed input data.
- Masked Data: Start combining the application data with the API tests to ensure that the API performs as expected against possible known input configurations.
API Testing Tools
Since API Testing is gaining popularity, we have many tools available for the same. By the way Selenium is just for browser-based testing, as a result we have different tools to use for Rest and Soap web service-based / API testing. Here are some of the top API testing tools that can be used for Rest and Soap Web Service Testing.
- SOAPUI – The most widely popular open-source tool for API testing in the world, SoapUI allows you to test REST and SOAP APIs with ease – as it has been built specifically for API testing. It can automate functional, regression, compliance and load testing of both SOAP and REST web services. It comes with an easy-to-use graphical interface and supports industry-leading technologies and standards to mock and stimulate behavior of web services.
- Postman: Send a post request to your web server and get the response. Run on Mac, Windows, Linux & Chrome Apps, it allows you to set up all the headers and cookies your API expects, and then check the response. Use it for both automated and exploratory testing. Inbuilt integrations like support for Swagger & RAML formats, it supports Run, Test, Document and Monitoring.
- Tricentis Tosca: Supports a wide array of protocols including HTTP(s) JMS, AMQP, Rabbit MQ, TIBCO EMS, SOAP, REST, IBM MQ,NET TCP. It uses model-based test automation that makes script maintenance easy. Enables end-to-end testing as API tests can be used across mobile, cross-browser, packaged apps, etc.
- HttpMaster: A web development and test tool to automate testing of web sites and services – RESTful web services and API applications. HttpMaster also allows you to monitor API responses.
- Rest-Assured: Open-source Java Domain-specific language (DSL) that makes testing REST service simple. It simplifies things by eliminating the need to use boiler-plate code to test and validate complex responses. It also supports XML and JSON Request/Responses.
- Parasoft: Paid tool, Automate API testing with Parasoft using its support for multiple platforms like Java, C, C++, or.NET. It supports end-to-end testing and has a very user-friendly interface.
- vRest: Provides an online solution for automated testing, mocking, automated recording and specification of REST/HTTP APIs/RESTful APIs.. This Tool can be used to test applications hosted locally, intranet or the Internet. Some of its good features include supporting Jira and Jenkins integration and also allows imports from Swagger and Postman. API mocks can be created in vREST with the help of Mock Server Functionality. User can directly start developing frontend using mock HTTP requests
- Apache JMeter: supports performance testing for Web services (SOAP/REST).
- HP QTP/UFT: Provides an extensible framework helpful in executing and building the functionality of headless system that do not have a user interface. It helps to test the headless technologies like Databases and Web services, JMS, etc.
- Rapise: A robust automation tool with powerful and extensible features. It is based on an open and flexible architecture for rapid functional testing of REST/SOAP web services. It uses HTTP standard methods such as POST, GET, PUT, and DELETE. Rapise also provides support for testing web applications built in Java, .NET, Ajax, Silverlight, and Flash.
- WebInject: Free tool for automated functional, acceptance, and regression testing of web and web services. It is a command-line tool and is based on Perl, which simplifies the execution of tests since it doesn’t require one to spend time at the command prompt. Further, it has no IDE like user interface which means, the tests are written outside of the WebInject UI. It can run on platforms that have Perl interpreter.
- Eclipse SDK tool for Automated API testing.
There are many other API Testing tools available in the QA market. When looking at an API testing tool, it is important to understand which API technologies you will be using and how best to test them. Nowadays most APIs you will come across will be of the Web Service variety (either REST or SOAP), but you may come across other technologies such as Java EJBs or Microsoft DCOM/ActiveX DLLs.
Challenges of API testing
- There is no GUI available to test the application which makes difficult to give input values – parameter combination, parameter selection, categorization and call sequencing.
- In-depth knowledge of application internals is required to sufficiently test the API. Some APIs may interact with the OS kernel, other APIs, with other software to offer their functionality.
- Adequate programming skills: API tests are generally in the form of sequences of calls, namely, programs. Each tester must possess expertise in the programming language(s) that are targeted by the API.
- No documentation: The APIs developed will hardly have any proper documentation available. Without the documentation, it is difficult for the Test designer to understand the purpose of calls, the parameter types and possible valid/invalid values, their return values, the calls it makes to other functions, and usage scenarios.
- Typically, project managers are not concerned with assigning time specifically to developing a rich and detailed API, let alone testing it.
- Time constraints: Thorough testing of APIs is time consuming, requires a learning overhead and resources to develop tools and design tests.
- Exception handling needs to be thoroughly tested.
Some Best practices for API Testing
- First & foremost, test for the functionality – the basic request-response is working consistently.
- Group Test cases by Test category.
- For complete Test coverage, create test cases for all possible API input combinations.
- Test for failure and invalid parameters for how it handles unforeseen problems and loads making sure the API fails gracefully.
- Add stress to the system through a series of API load tests.
- Parameters selection should be explicitly mentioned in the Test case itself.
- Prioritize API function calls so that it will be easy for testers to test in a timely fashion.
- Automate API documentation creation with a standard like Swagger, but then run through the tests, making sure the documentation makes sense for all levels of user experience.
- Automate whatever you can.
If any API doesn’t work efficiently and effectively, it will never be adopted, regardless if it is a free and open API. If an API breaks, it could not only break a single application but a chain of business processes hinged to it. Putting more effort into API testing leads to a much healthier final product. Ensuring that all data access (read and write) goes only through the API significantly simplifies security and compliance testing and thereby certification, since there is only one interface. Ensuring that the API offers complete functionality allows for easy future expansion of the application as new business needs arise.
API testing is quite crucial and it is highly required. It is one of the areas where automation testing is highly recommended, particularly in the world of DevOps, agile development and continuous delivery cycles. The way our generation is moving towards Artificial Intelligence, Cloud computing and IoT, there will soon be a higher demand for rigorous API testing. Many of the services that we use every day rely on hundreds of different interconnected APIs, if any one of them fails then the service will not work. API testing is a necessity. If you’re a Software tester, API testing is a must-have skill to have in your arsenal.