You are correct Sridhar, that one major drawback of Postman and Newman is the inability to run parallel collection tests out of the box. By default, Newman runs collections in a single thread, which means that tests are executed sequentially rather than in parallel. However, there are some workarounds that can be used to achieve parallelism in Newman. One approach is to use a test runner like Mocha or Jest that supports parallel execution, and then use Newman as a test reporter to execute the Postman collections. Another approach is to use a third-party tool like k6 or Artillery that provides native support for parallel execution and can integrate with Postman collections. While these workarounds can provide some level of parallelism, they require additional setup and configuration, and may not be as robust or flexible as a dedicated parallel test runner. It's worth noting that there are many other test automation frameworks and tools available that do support parallel execution out of the box, such as Selenium Grid, TestNG, and JUnit.
Postman is very popular, REST Assured is also good and gives you more control over the project, I will suggest that you check the tools agains your requirements, do a POC and then finalize
Hi Dhana Rest Assured and Katalon are both popular tools for API automation testing. They have different strengths and weaknesses, so the best tool for you will depend on your specific needs and preferences Rest Assured is a Java library that is used for testing REST APIs. It is a powerful tool that allows you to create complex test cases. Rest Assured is also a good choice if you need to integrate your API testing with other Java applications. However, Rest Assured can be a bit difficult to learn Katalon is an automation testing platform that supports a variety of different technologies, including REST APIs. It is a good choice if you are looking for a tool that is easy to learn and use. Katalon also has a graphical user interface, which can make it easier to create and run test cases. However, Katalon is not as powerful as Rest Assured, and it may not be a good choice if you need to integrate your API testing with other Java applications Rest Assured is free and Katalon is commercial I hope this helps
I use postman a lot, part of which I've learnt from you. I used the runner option a lot but recent cap in running a collection has made me think again about postman. I know there's option to run the collection via newman but I found it bit tedious as you need to export the collection everytime if you have any change in the collection or you don’t want to run a specific endpoint from the collection. At this stage I'm confused between katalon and jmeter. Which one you recommend?
Katalon will be easier but again it has a paid version and free ver is very limited. JMeter does not have a very good UI but is free and open source, I will suggest to do a POC and then finalize
Hi Dhana Can check the API Testing section here - automationstepbystep.com/ Apart from Postman, some other popular tools for API automation testing in the industry include: * Rest Assured: Rest Assured is a Java library that is used for testing REST APIs. It is a powerful tool that allows you to create complex test cases. Rest Assured is also a good choice if you need to integrate your API testing with other Java applications * SoapUI: SoapUI is a popular tool for testing SOAP APIs. It has a graphical user interface that makes it easy to create and run test cases. SoapUI also supports a variety of different scripting languages, so you can customize your tests as needed * Karate: Karate is a newer tool that is gaining popularity for API automation testing. It is a open-source tool that is written in Ruby. Karate is known for its simplicity and ease of use * JMeter: JMeter is a popular tool for load testing and performance testing. However, it can also be used for API automation testing. JMeter is a powerful tool that allows you to create complex test cases. However, it can be a bit difficult to learn The best tool for you will depend on your specific needs and preferences. If you are looking for a tool with a graphical user interface, then Postman is a good choice. If you are looking for a tool that is easy to learn, then Karate is a good choice. If you need a tool that is powerful and can be integrated with other Java applications, then Rest Assured is a good choice. And if you need a tool that can be used for load testing and performance testing, then JMeter is a good choice. Ultimately, the best way to choose a tool for API automation testing is to try out a few different tools and see which one you like best
In our company, we are using REST APIs. Are we debating between Playwright and RESTAssured tools for API Testing? For UI Automation, we are using Playwright. Please suggest to us, which one is Good?
Radha Here's a comparison of Playwright and REST Assured for API testing, considering your company's use of Playwright for UI automation: Playwright: - Strengths: - Unified experience for UI and API testing: Use the same framework and language for both, reducing context switching and cognitive load. - Modern features for complex scenarios: Handles redirects, cookies, and authentication well, often simplifying test setup. - Good browser support: Tests APIs across multiple browsers with consistent behavior. - Headless API testing: Interacts with APIs without UI rendering, improving speed and resource efficiency. - Considerations: - Relatively new: Still evolving, with potential for breaking changes or limited resources. - Not API-specific: Might require more configuration or setup for API-only testing compared to REST Assured. REST Assured: - Strengths: - API-focused: Built specifically for REST API testing, offering a streamlined experience and comprehensive features. - Mature and stable: Widely adopted, with a large community and extensive documentation. - Powerful assertions: Provides a rich set of assertions for validating API responses effectively. - Flexible integration: Works well with various testing frameworks and tools. - Considerations: - Separate tool for UI testing: You'll need a different framework for UI automation. - Might require more Java expertise: REST Assured is Java-based, so familiarity with Java is essential. Recommendations: - Unified experience and modern features: Choose Playwright if you prioritize a seamless experience between UI and API testing, value its modern features, and are comfortable with its potential newness. - API-specific focus and maturity: Choose REST Assured if you prefer a tool dedicated solely to API testing, value its stability and maturity, and have strong Java skills. Additional factors to consider: - Team expertise: Assess your team's familiarity with JavaScript/TypeScript (Playwright) or Java (REST Assured). - Project complexity: Evaluate the complexity of your API tests and whether Playwright's features are necessary. - Future needs: Consider potential expansion into UI testing or the need for API testing in non-browser environments. Experimentation: - If you're unsure, try both tools in a small-scale proof of concept to see which aligns better with your team's preferences and project needs. Best practice: - Choose the tool that best suits your team's skills, project requirements, and testing goals to ensure effective API testing.
Hi Raghav, Thanks- This is really useful. I have a requirement where we are migrating legacy system which we tested soap apis and now migrating to rest apis. Can i use RestAssured for validating both tests and compare the response attributes of soap apis and rest apis? And, which language is prefferable? I had java knowledge and not sure if python will make it easier?
Lokeshwari While RestAssured is primarily designed for testing RESTful APIs, it can be used to validate both SOAP and REST APIs with some adjustments. However, comparing the response attributes directly might not be straightforward due to the inherent differences in structure between SOAP and REST responses. Here's a breakdown of your options: 1. Testing Both APIs with RestAssured: REST API: RestAssured can fully handle testing your new REST API endpoints. SOAP API: You can use RestAssured to send HTTP requests and receive responses from your SOAP API. However, you'll need additional libraries like SAAJ (SOAP with Attachments API for Java) to parse and manipulate the XML-based SOAP responses. This can be more complex than native RestAssured functionality. 2. Comparing Response Attributes: Direct Comparison: Directly comparing raw SOAP and REST responses is challenging due to their different structures. SOAP responses are XML-based, while REST responses are typically JSON or plain text. You'd need to extract specific data points from each and then compare them, which can involve custom parsing and logic. Data-Driven Comparison: Instead of directly comparing responses, consider a data-driven approach. Define expected data points and values for both APIs, then compare these expected values with the actual values extracted from the responses using appropriate parsing libraries. This approach is more flexible and maintainable. 3. Language Choice: Java vs. Python: Both Java and Python are popular for API testing with their own strengths: Java: Offers mature libraries like RestAssured and SAAJ, along with your existing experience. Python: Has a simpler syntax and potentially faster development due to its dynamic nature. You'd need to learn libraries like `requests` and `zeep` for SOAP testing. Recommendation: Use RestAssured for both APIs: Leverage your existing Java knowledge and RestAssured's capabilities for REST API testing. For SOAP API testing: Use RestAssured with SAAJ for sending requests and receiving responses. Consider a data-driven approach for comparing response attributes, focusing on extracting and comparing specific data points instead of direct responses. Choose Java or Python: Based on your existing experience and project requirements. Java might be quicker to start with, but consider Python's simplicity if you're open to learning a new language. Remember, the best approach depends on your specific needs and preferences. Evaluate the complexity of your testing scenarios and choose the tools and languages that best suit your team's skills and project requirements. ..
Thank you so much Raghav for wonderfully outlining .. we have less time to achieve this May be 2 months for automation of 130 test cases . I am planning to learn python and the libraries you have mentioned request and zeep with Bdd approach . Hope that will help me. I have basic knowledge of python
@@RaghavPal Thank you so much raghav for such a wonderful reply. I have basic python knowledge and the requirement is we have less time to automate 130 test cases [ both soap and rest ] can I use Robot framework ? does it helps if the response has more than 50 attributes and schema validation ?
Lokeshwari Certainly! Robot Framework is a powerful and versatile open-source automation framework that can be a great choice for your test automation needs. Let's break down your requirements: 1. 130 Test Cases: - Robot Framework allows you to create and manage test cases efficiently. With its simple syntax, you can quickly define test steps, expected outcomes, and organize them into test suites. 2. SOAP and REST APIs: - Robot Framework supports both SOAP and REST API testing. You can use built-in libraries like `RequestsLibrary` for REST API testing and `SOAPLibrary` for SOAP API testing. These libraries provide keywords to send requests, validate responses, and handle authentication. 3. Response with More Than 50 Attributes: - Robot Framework can handle complex responses with ease. You can extract specific attributes using keywords like `Get Json Value`, `Get XML Value`, or even custom Python scripts. - For large responses, consider breaking them down into smaller chunks and validating specific attributes separately. 4. Schema Validation: - Schema validation is crucial to ensure that responses adhere to expected formats. Robot Framework allows you to write custom validation logic using Python or utilize existing libraries. - For JSON schema validation, you can use the `JSONSchemaValidator` library. 5. Parallel Execution: - If you need to speed up execution, consider running test cases in parallel. The `Pabot` library allows parallel execution of Robot Framework test suites⁴. 6. Setup and Teardown: - Robot Framework provides setup and teardown functionality. You can define common setup steps (e.g., authentication) and teardown steps (e.g., closing connections) for all test cases. 7. Custom Keywords: - Create reusable custom keywords to encapsulate common actions. For example, you can create a keyword to validate specific attributes in responses. 8. Reporting and Logs: - Robot Framework generates detailed reports and logs, making it easy to analyze test results. Remember that Robot Framework is highly extensible. You can create your own custom libraries or use existing ones to enhance its capabilities. Given your time constraints and the need to automate 130 test cases, Robot Framework can be a productive choice. ..
Hi Naveen Yes, you can use Visual Studio, C#, and BDD to automate your project APIs. Here's an overview of how you can do it: Install Visual Studio: You can download and install Visual Studio from the Microsoft website. Create a new C# project: Open Visual Studio and create a new C# project. Install the necessary NuGet packages: Use the NuGet Package Manager in Visual Studio to install the necessary packages for API automation, such as RestSharp and NUnit. Write your BDD scenarios: Use a BDD framework like SpecFlow to write your scenarios in the Gherkin syntax. For example, you might write a scenario like "Given a valid API endpoint, when I send a GET request, then I should receive a response with a status code of 200." Implement your step definitions: In Visual Studio, implement your step definitions using C# code. The step definitions should map to the steps in your BDD scenarios. Run your tests: Use NUnit or another testing framework to run your API tests and verify that they pass. By following these steps, you can use Visual Studio, C#, and BDD to automate your project APIs
While there are no truly codeless testing tools, there are several Python-based BDD tools that can significantly simplify GraphQL API testing: 1. Cypress: Excellent for API and UI testing, providing a BDD-style syntax and a graphical interface for test creation and execution. Offers GraphQL testing capabilities out-of-the-box with dedicated plugins like @cypress/graphql. Requires basic Javascript knowledge for writing tests, but can be learned alongside Python if needed. 2. Postman: Popular API testing tool with a user-friendly interface and collection management features. Supports BDD through scripting with libraries like Newman and Super Agent. Offers dedicated tools for GraphQL testing, including query builder and execution tools. Not strictly codeless, but scripting requirements are minimal and focused on data preparation and test logic. 3. Robot Framework: Keyword-driven testing framework used for various types of testing, including APIs. Requires writing keywords in Python, but provides libraries like robotframework-graphql for simplified GraphQL interactions. Offers BDD features through embedded Gherkin syntax for describing test scenarios. 4. RestAssured: Powerful Java library for API testing, but can be used with Python through Jython bridge. Supports GraphQL testing through extensions like graphql-java-client and offers BDD capabilities with libraries like Serenity BDD. Less codeless than other options, but provides flexibility and control for complex scenarios. Choosing the best tool depends on your specific needs and preferences: Cypress: Great for combined API and UI testing, good for beginners. Postman: Easy to use for basic API tests, convenient for collection management. Robot Framework: Flexible and extensible, but requires some Python knowledge. RestAssured: Powerful and versatile, but best for experienced testers. Bonus: Consider tools like GraphCool Playground and Insomnia for interactive GraphQL exploration and testing, which can complement your chosen BDD framework. Ultimately, the most important factor is selecting a tool that aligns with your team's existing skills and project requirements. Feel free to compare features and explore trials of these tools to find the best fit for your GraphQL API testing with Python and BDD approach
Thanks@@RaghavPal , Really appreciate you taking out time and looking into my query. Yes Codelss is difficult. I am looking for a tool similar to KARATE for my Graphql API automation. I found karate the most efficient tool as it doesn't have step definitions. The only reason we can't select this tool is that its java based, hence CI/CD integration is not feasible. Could you point out in above listed tools which is similar to Karate? Thanks in advance
A Karate-like GraphQL API automation tool that isn't Java-based for easier CI/CD integration. While Karate's lack of step definitions is certainly attractive, several other tools offer similar capabilities and are better suited for your CI/CD environment. Here are some options to consider: GraphQL-Specific Tools: - GraphqlTester: Written in JavaScript, it focuses on GraphQL API testing specifically. Its syntax resembles plain English, similar to Karate. - Apollo Client Devtools: Integrates with the Apollo GraphQL client library and offers a built-in testing environment. - Postman: Not exclusively focused on GraphQL, but its GraphQL capabilities have improved significantly, including scripting and environment variables. General API Testing Tools: - REST Assured: While mainly for REST APIs, it can be used for GraphQL with some additional libraries. It's Java-based, but might work if CI/CD integration requirements are flexible. - Cypress.io: primarily for web UI testing, it can also be used for GraphQL API testing with the `cypress-graphql` plugin. It's written in JavaScript. - Playwright: Another web UI testing tool with the `@playwright/test` library supporting GraphQL testing. It's also JavaScript-based.
@@RaghavPal Thanks a lot for getting back. Sorry If I missed to mention I am looking for PYTHON based as our CI/CD is Python Based. So I am looking for compatibilities - BDD, Python Based, Opensource, Graphql API, No Step definition What are your thoughts on Robot Framework?
Here are some excellent options along with key considerations: 1. Behave: - Pros: Fully Pythonic, strong BDD support, built-in GraphQL features, flexible and well-maintained. - Cons: Less mature than some alternatives, can require more configuration for complex scenarios. 2. Graphene-Python: - Pros: Seamless integration with Python GraphQL libraries, clean syntax, built-in mutation capabilities. - Cons: Primarily focused on testing GraphQL schemas, requires some understanding of GraphQL internals. 3. Hypothesis (with the `graphql-tester` library): - Pros: Highly versatile testing framework, can be used for both unit and integration testing, strong data-driven capabilities. - Cons: Might require more setup and coding compared to dedicated BDD tools. 4. Graphql-Client (with additional libraries like `pytest-graphql`): - Pros: Well-tested and popular library for making GraphQL requests, can be adapted for BDD-style tests. - Cons: Requires more manual effort to construct BDD-like syntax, relies on external libraries for BDD features. Additional Options: - REST-Assured (with the `rest-assured-graphql` extension): - Pros: Popular Java-based API automation tool, extension adds basic GraphQL support. - Cons: Requires Java environment, potentially less intuitive for Python users. - Gatling: - Pros: Powerful performance testing tool, can be used with GraphQL plugins. - Cons: Primarily focused on performance testing, may require adaptation for BDD-style tests. Robot Framework: - Pros: Mature and popular tool, wide range of libraries and extensions, BDD support. - Cons: Primarily keyword-driven, might not offer the same level of code expressiveness as Python-based options.
Hello Raghav, I need to learn Typescript and playwright framework for QA. Can you please suggest the path or channels. Is learning JavaScript a pre requisite and how much depth is required for Typescript.Thank you.
Hi Hima, Learning JavaScript is a prerequisite for learning Typescript and Playwright, as they are built on top of JavaScript. If you don't know JavaScript, it's recommended to learn JavaScript basics first before diving into Typescript. To learn Typescript and Playwright, you can follow these steps: Learn JavaScript basics: Start by learning JavaScript basics such as variables, data types, functions, and control structures. You can find plenty of resources online, including video tutorials, books, and online courses. Learn Typescript: After learning the basics of JavaScript, you can start learning Typescript. Typescript is a superset of JavaScript that adds features such as static typing, interfaces, and classes. You can find official documentation and tutorials on the Typescript website. Learn Playwright: Once you have a good grasp of Typescript, you can start learning Playwright. Playwright is a Node.js library for automating web browsers, and it has good documentation and tutorials on its website Practice and build projects: The best way to learn Typescript and Playwright is by practicing and building projects. Start with simple projects and gradually move on to more complex ones As for how much depth is required for Typescript, it depends on your needs and the complexity of your projects. However, a good understanding of basic concepts such as interfaces, classes, and types is essential. It's recommended to spend some time learning Typescript before moving on to Playwright
@@RaghavPal thank you so much Raghav for the detailed reply and your valuable advise. 🙏🙏 I learned core Java for selenium automation. I was hoping with knowledge in Java i can directly learn typescript.
Hi, It's important to respect the security measures put in place to protect websites and users from automated attacks and misuse of data. Attempting to bypass security measures can lead to legal and ethical issues, and can also harm the integrity of the testing process. Instead, I recommend looking into ways to automate your tests in a secure and ethical manner. Cypress provides various options to work with reCAPTCHA, such as stubbing the reCAPTCHA API or using a service like AntiCaptcha to handle the reCAPTCHA challenge. However, it's important to ensure that any methods used to handle reCAPTCHA are in compliance with Google's policies and do not compromise the security of the website or its users.
Thank you. This clarified all doubts and helped choosing the tool..!
Most welcome
Superb.. Waiting for this 👍
Hope you like it Saurabh
Very good video, I was searching another for this info.. really clear and objective
Thank you so much
Most welcome Asmaa
I Have used Jmeter for functional testing API automation 👍👍 integrated with CI pipeline too
Great 👍
Thank you for the mention! Awesome video always
Most welcome
Great video Raghav!
Thanks a ton!
Is there any detail course on this?
Sachin
can check here - automationstepbystep.com/
Raghav, one this mayor drawback for postman > newman we cannot run parallel collection test if there one I am not ware
You are correct Sridhar, that one major drawback of Postman and Newman is the inability to run parallel collection tests out of the box. By default, Newman runs collections in a single thread, which means that tests are executed sequentially rather than in parallel.
However, there are some workarounds that can be used to achieve parallelism in Newman. One approach is to use a test runner like Mocha or Jest that supports parallel execution, and then use Newman as a test reporter to execute the Postman collections. Another approach is to use a third-party tool like k6 or Artillery that provides native support for parallel execution and can integrate with Postman collections.
While these workarounds can provide some level of parallelism, they require additional setup and configuration, and may not be as robust or flexible as a dedicated parallel test runner. It's worth noting that there are many other test automation frameworks and tools available that do support parallel execution out of the box, such as Selenium Grid, TestNG, and JUnit.
from an automation tester point...which to learn sir? which tool is highly used in market?
Postman is very popular,
REST Assured is also good and gives you more control over the project, I will suggest that you check the tools agains your requirements, do a POC and then finalize
Hi Raghav, which one I need to choose between restassured and katalon.
Hi Dhana
Rest Assured and Katalon are both popular tools for API automation testing. They have different strengths and weaknesses, so the best tool for you will depend on your specific needs and preferences
Rest Assured is a Java library that is used for testing REST APIs. It is a powerful tool that allows you to create complex test cases. Rest Assured is also a good choice if you need to integrate your API testing with other Java applications. However, Rest Assured can be a bit difficult to learn
Katalon is an automation testing platform that supports a variety of different technologies, including REST APIs. It is a good choice if you are looking for a tool that is easy to learn and use. Katalon also has a graphical user interface, which can make it easier to create and run test cases. However, Katalon is not as powerful as Rest Assured, and it may not be a good choice if you need to integrate your API testing with other Java applications
Rest Assured is free and Katalon is commercial
I hope this helps
Thanks Raghav, I was looking for this differences
Thanks, Raghav.
Most welcome
I would really appreciate if you gave us something on locust too, otherwise thanks for the good work Raghav
Hi Mwine, will add Locust in Performance Testing Tools soon
I use postman a lot, part of which I've learnt from you. I used the runner option a lot but recent cap in running a collection has made me think again about postman. I know there's option to run the collection via newman but I found it bit tedious as you need to export the collection everytime if you have any change in the collection or you don’t want to run a specific endpoint from the collection.
At this stage I'm confused between katalon and jmeter. Which one you recommend?
Katalon will be easier but again it has a paid version and free ver is very limited. JMeter does not have a very good UI but is free and open source, I will suggest to do a POC and then finalize
Apart from postman which is most demanding tool in industry for api automation testing
Hi Dhana
Can check the API Testing section here - automationstepbystep.com/
Apart from Postman, some other popular tools for API automation testing in the industry include:
* Rest Assured: Rest Assured is a Java library that is used for testing REST APIs. It is a powerful tool that allows you to create complex test cases. Rest Assured is also a good choice if you need to integrate your API testing with other Java applications
* SoapUI: SoapUI is a popular tool for testing SOAP APIs. It has a graphical user interface that makes it easy to create and run test cases. SoapUI also supports a variety of different scripting languages, so you can customize your tests as needed
* Karate: Karate is a newer tool that is gaining popularity for API automation testing. It is a open-source tool that is written in Ruby. Karate is known for its simplicity and ease of use
* JMeter: JMeter is a popular tool for load testing and performance testing. However, it can also be used for API automation testing. JMeter is a powerful tool that allows you to create complex test cases. However, it can be a bit difficult to learn
The best tool for you will depend on your specific needs and preferences. If you are looking for a tool with a graphical user interface, then Postman is a good choice. If you are looking for a tool that is easy to learn, then Karate is a good choice. If you need a tool that is powerful and can be integrated with other Java applications, then Rest Assured is a good choice. And if you need a tool that can be used for load testing and performance testing, then JMeter is a good choice.
Ultimately, the best way to choose a tool for API automation testing is to try out a few different tools and see which one you like best
In our company, we are using REST APIs. Are we debating between Playwright and RESTAssured tools for API Testing? For UI Automation, we are using Playwright. Please suggest to us, which one is Good?
Radha
Here's a comparison of Playwright and REST Assured for API testing, considering your company's use of Playwright for UI automation:
Playwright:
- Strengths:
- Unified experience for UI and API testing: Use the same framework and language for both, reducing context switching and cognitive load.
- Modern features for complex scenarios: Handles redirects, cookies, and authentication well, often simplifying test setup.
- Good browser support: Tests APIs across multiple browsers with consistent behavior.
- Headless API testing: Interacts with APIs without UI rendering, improving speed and resource efficiency.
- Considerations:
- Relatively new: Still evolving, with potential for breaking changes or limited resources.
- Not API-specific: Might require more configuration or setup for API-only testing compared to REST Assured.
REST Assured:
- Strengths:
- API-focused: Built specifically for REST API testing, offering a streamlined experience and comprehensive features.
- Mature and stable: Widely adopted, with a large community and extensive documentation.
- Powerful assertions: Provides a rich set of assertions for validating API responses effectively.
- Flexible integration: Works well with various testing frameworks and tools.
- Considerations:
- Separate tool for UI testing: You'll need a different framework for UI automation.
- Might require more Java expertise: REST Assured is Java-based, so familiarity with Java is essential.
Recommendations:
- Unified experience and modern features: Choose Playwright if you prioritize a seamless experience between UI and API testing, value its modern features, and are comfortable with its potential newness.
- API-specific focus and maturity: Choose REST Assured if you prefer a tool dedicated solely to API testing, value its stability and maturity, and have strong Java skills.
Additional factors to consider:
- Team expertise: Assess your team's familiarity with JavaScript/TypeScript (Playwright) or Java (REST Assured).
- Project complexity: Evaluate the complexity of your API tests and whether Playwright's features are necessary.
- Future needs: Consider potential expansion into UI testing or the need for API testing in non-browser environments.
Experimentation:
- If you're unsure, try both tools in a small-scale proof of concept to see which aligns better with your team's preferences and project needs.
Best practice:
- Choose the tool that best suits your team's skills, project requirements, and testing goals to ensure effective API testing.
Hi Raghav, Thanks- This is really useful. I have a requirement where we are migrating legacy system which we tested soap apis and now migrating to rest apis.
Can i use RestAssured for validating both tests and compare the response attributes of soap apis and rest apis?
And, which language is prefferable? I had java knowledge and not sure if python will make it easier?
Lokeshwari
While RestAssured is primarily designed for testing RESTful APIs, it can be used to validate both SOAP and REST APIs with some adjustments. However, comparing the response attributes directly might not be straightforward due to the inherent differences in structure between SOAP and REST responses.
Here's a breakdown of your options:
1. Testing Both APIs with RestAssured:
REST API: RestAssured can fully handle testing your new REST API endpoints.
SOAP API: You can use RestAssured to send HTTP requests and receive responses from your SOAP API. However, you'll need additional libraries like SAAJ (SOAP with Attachments API for Java) to parse and manipulate the XML-based SOAP responses. This can be more complex than native RestAssured functionality.
2. Comparing Response Attributes:
Direct Comparison: Directly comparing raw SOAP and REST responses is challenging due to their different structures. SOAP responses are XML-based, while REST responses are typically JSON or plain text. You'd need to extract specific data points from each and then compare them, which can involve custom parsing and logic.
Data-Driven Comparison: Instead of directly comparing responses, consider a data-driven approach. Define expected data points and values for both APIs, then compare these expected values with the actual values extracted from the responses using appropriate parsing libraries. This approach is more flexible and maintainable.
3. Language Choice:
Java vs. Python: Both Java and Python are popular for API testing with their own strengths:
Java: Offers mature libraries like RestAssured and SAAJ, along with your existing experience.
Python: Has a simpler syntax and potentially faster development due to its dynamic nature. You'd need to learn libraries like `requests` and `zeep` for SOAP testing.
Recommendation:
Use RestAssured for both APIs: Leverage your existing Java knowledge and RestAssured's capabilities for REST API testing.
For SOAP API testing: Use RestAssured with SAAJ for sending requests and receiving responses. Consider a data-driven approach for comparing response attributes, focusing on extracting and comparing specific data points instead of direct responses.
Choose Java or Python: Based on your existing experience and project requirements. Java might be quicker to start with, but consider Python's simplicity if you're open to learning a new language.
Remember, the best approach depends on your specific needs and preferences. Evaluate the complexity of your testing scenarios and choose the tools and languages that best suit your team's skills and project requirements.
..
Thank you so much Raghav for wonderfully outlining .. we have less time to achieve this May be 2 months for automation of 130 test cases . I am planning to learn python and the libraries you have mentioned request and zeep with Bdd approach . Hope that will help me. I have basic knowledge of python
@@RaghavPal Thank you so much raghav for such a wonderful reply. I have basic python knowledge and the requirement is we have less time to automate 130 test cases [ both soap and rest ] can I use Robot framework ? does it helps if the response has more than 50 attributes and schema validation ?
Lokeshwari
Certainly! Robot Framework is a powerful and versatile open-source automation framework that can be a great choice for your test automation needs. Let's break down your requirements:
1. 130 Test Cases:
- Robot Framework allows you to create and manage test cases efficiently. With its simple syntax, you can quickly define test steps, expected outcomes, and organize them into test suites.
2. SOAP and REST APIs:
- Robot Framework supports both SOAP and REST API testing. You can use built-in libraries like `RequestsLibrary` for REST API testing and `SOAPLibrary` for SOAP API testing. These libraries provide keywords to send requests, validate responses, and handle authentication.
3. Response with More Than 50 Attributes:
- Robot Framework can handle complex responses with ease. You can extract specific attributes using keywords like `Get Json Value`, `Get XML Value`, or even custom Python scripts.
- For large responses, consider breaking them down into smaller chunks and validating specific attributes separately.
4. Schema Validation:
- Schema validation is crucial to ensure that responses adhere to expected formats. Robot Framework allows you to write custom validation logic using Python or utilize existing libraries.
- For JSON schema validation, you can use the `JSONSchemaValidator` library.
5. Parallel Execution:
- If you need to speed up execution, consider running test cases in parallel. The `Pabot` library allows parallel execution of Robot Framework test suites⁴.
6. Setup and Teardown:
- Robot Framework provides setup and teardown functionality. You can define common setup steps (e.g., authentication) and teardown steps (e.g., closing connections) for all test cases.
7. Custom Keywords:
- Create reusable custom keywords to encapsulate common actions. For example, you can create a keyword to validate specific attributes in responses.
8. Reporting and Logs:
- Robot Framework generates detailed reports and logs, making it easy to analyze test results.
Remember that Robot Framework is highly extensible. You can create your own custom libraries or use existing ones to enhance its capabilities. Given your time constraints and the need to automate 130 test cases, Robot Framework can be a productive choice.
..
Thanks again raghav. I will stick to this 🙏
Can we use visual studio, c# and BDD to automate our project APIs?
Hi Naveen
Yes, you can use Visual Studio, C#, and BDD to automate your project APIs. Here's an overview of how you can do it:
Install Visual Studio: You can download and install Visual Studio from the Microsoft website.
Create a new C# project: Open Visual Studio and create a new C# project.
Install the necessary NuGet packages: Use the NuGet Package Manager in Visual Studio to install the necessary packages for API automation, such as RestSharp and NUnit.
Write your BDD scenarios: Use a BDD framework like SpecFlow to write your scenarios in the Gherkin syntax. For example, you might write a scenario like "Given a valid API endpoint, when I send a GET request, then I should receive a response with a status code of 200."
Implement your step definitions: In Visual Studio, implement your step definitions using C# code. The step definitions should map to the steps in your BDD scenarios.
Run your tests: Use NUnit or another testing framework to run your API tests and verify that they pass.
By following these steps, you can use Visual Studio, C#, and BDD to automate your project APIs
@@RaghavPal Thanks a lot..
In which of these tools, can we see both request and response payloads in Reports?
Hi Saurabh,
you should be able to see in most tools like Postman, SoapUI, some may need some configuration or scripting
Hi,
Can you suggest CODELESS testing tool for - GRAPHQL API with BDD & python based.
Thanks
While there are no truly codeless testing tools, there are several Python-based BDD tools that can significantly simplify GraphQL API testing:
1. Cypress:
Excellent for API and UI testing, providing a BDD-style syntax and a graphical interface for test creation and execution.
Offers GraphQL testing capabilities out-of-the-box with dedicated plugins like @cypress/graphql.
Requires basic Javascript knowledge for writing tests, but can be learned alongside Python if needed.
2. Postman:
Popular API testing tool with a user-friendly interface and collection management features.
Supports BDD through scripting with libraries like Newman and Super Agent.
Offers dedicated tools for GraphQL testing, including query builder and execution tools.
Not strictly codeless, but scripting requirements are minimal and focused on data preparation and test logic.
3. Robot Framework:
Keyword-driven testing framework used for various types of testing, including APIs.
Requires writing keywords in Python, but provides libraries like robotframework-graphql for simplified GraphQL interactions.
Offers BDD features through embedded Gherkin syntax for describing test scenarios.
4. RestAssured:
Powerful Java library for API testing, but can be used with Python through Jython bridge.
Supports GraphQL testing through extensions like graphql-java-client and offers BDD capabilities with libraries like Serenity BDD.
Less codeless than other options, but provides flexibility and control for complex scenarios.
Choosing the best tool depends on your specific needs and preferences:
Cypress: Great for combined API and UI testing, good for beginners.
Postman: Easy to use for basic API tests, convenient for collection management.
Robot Framework: Flexible and extensible, but requires some Python knowledge.
RestAssured: Powerful and versatile, but best for experienced testers.
Bonus: Consider tools like GraphCool Playground and Insomnia for interactive GraphQL exploration and testing, which can complement your chosen BDD framework.
Ultimately, the most important factor is selecting a tool that aligns with your team's existing skills and project requirements. Feel free to compare features and explore trials of these tools to find the best fit for your GraphQL API testing with Python and BDD approach
Thanks@@RaghavPal , Really appreciate you taking out time and looking into my query.
Yes Codelss is difficult. I am looking for a tool similar to KARATE for my Graphql API automation. I found karate the most efficient tool as it doesn't have step definitions.
The only reason we can't select this tool is that its java based, hence CI/CD integration is not feasible.
Could you point out in above listed tools which is similar to Karate?
Thanks in advance
A Karate-like GraphQL API automation tool that isn't Java-based for easier CI/CD integration.
While Karate's lack of step definitions is certainly attractive, several other tools offer similar capabilities and are better suited for your CI/CD environment. Here are some options to consider:
GraphQL-Specific Tools:
- GraphqlTester: Written in JavaScript, it focuses on GraphQL API testing specifically. Its syntax resembles plain English, similar to Karate.
- Apollo Client Devtools: Integrates with the Apollo GraphQL client library and offers a built-in testing environment.
- Postman: Not exclusively focused on GraphQL, but its GraphQL capabilities have improved significantly, including scripting and environment variables.
General API Testing Tools:
- REST Assured: While mainly for REST APIs, it can be used for GraphQL with some additional libraries. It's Java-based, but might work if CI/CD integration requirements are flexible.
- Cypress.io: primarily for web UI testing, it can also be used for GraphQL API testing with the `cypress-graphql` plugin. It's written in JavaScript.
- Playwright: Another web UI testing tool with the `@playwright/test` library supporting GraphQL testing. It's also JavaScript-based.
@@RaghavPal Thanks a lot for getting back.
Sorry If I missed to mention I am looking for PYTHON based as our CI/CD is Python Based.
So I am looking for compatibilities - BDD, Python Based, Opensource, Graphql API, No Step definition
What are your thoughts on Robot Framework?
Here are some excellent options along with key considerations:
1. Behave:
- Pros: Fully Pythonic, strong BDD support, built-in GraphQL features, flexible and well-maintained.
- Cons: Less mature than some alternatives, can require more configuration for complex scenarios.
2. Graphene-Python:
- Pros: Seamless integration with Python GraphQL libraries, clean syntax, built-in mutation capabilities.
- Cons: Primarily focused on testing GraphQL schemas, requires some understanding of GraphQL internals.
3. Hypothesis (with the `graphql-tester` library):
- Pros: Highly versatile testing framework, can be used for both unit and integration testing, strong data-driven capabilities.
- Cons: Might require more setup and coding compared to dedicated BDD tools.
4. Graphql-Client (with additional libraries like `pytest-graphql`):
- Pros: Well-tested and popular library for making GraphQL requests, can be adapted for BDD-style tests.
- Cons: Requires more manual effort to construct BDD-like syntax, relies on external libraries for BDD features.
Additional Options:
- REST-Assured (with the `rest-assured-graphql` extension):
- Pros: Popular Java-based API automation tool, extension adds basic GraphQL support.
- Cons: Requires Java environment, potentially less intuitive for Python users.
- Gatling:
- Pros: Powerful performance testing tool, can be used with GraphQL plugins.
- Cons: Primarily focused on performance testing, may require adaptation for BDD-style tests.
Robot Framework:
- Pros: Mature and popular tool, wide range of libraries and extensions, BDD support.
- Cons: Primarily keyword-driven, might not offer the same level of code expressiveness as Python-based options.
What about locust ??
will add Locust in Performance Testing Tools soon
Hello Raghav, I need to learn Typescript and playwright framework for QA. Can you please suggest the path or channels. Is learning JavaScript a pre requisite and how much depth is required for Typescript.Thank you.
Hi Hima,
Learning JavaScript is a prerequisite for learning Typescript and Playwright, as they are built on top of JavaScript. If you don't know JavaScript, it's recommended to learn JavaScript basics first before diving into Typescript.
To learn Typescript and Playwright, you can follow these steps:
Learn JavaScript basics: Start by learning JavaScript basics such as variables, data types, functions, and control structures. You can find plenty of resources online, including video tutorials, books, and online courses.
Learn Typescript: After learning the basics of JavaScript, you can start learning Typescript. Typescript is a superset of JavaScript that adds features such as static typing, interfaces, and classes. You can find official documentation and tutorials on the Typescript website.
Learn Playwright: Once you have a good grasp of Typescript, you can start learning Playwright. Playwright is a Node.js library for automating web browsers, and it has good documentation and tutorials on its website
Practice and build projects: The best way to learn Typescript and Playwright is by practicing and building projects. Start with simple projects and gradually move on to more complex ones
As for how much depth is required for Typescript, it depends on your needs and the complexity of your projects. However, a good understanding of basic concepts such as interfaces, classes, and types is essential. It's recommended to spend some time learning Typescript before moving on to Playwright
@@RaghavPal thank you so much Raghav for the detailed reply and your valuable advise. 🙏🙏 I learned core Java for selenium automation. I was hoping with knowledge in Java i can directly learn typescript.
Can you please make a video on INSOMNIA api test tool too
Sure Kalpana, I will plan on this
Which one is the most demanding tool these days?
Kshipra
It depends, i have seen tools like Playwright, Postman, Cypress getting more demands these days
Hello Raghav sir can you tell me how can we bypass "i am not a robot" google captcha in cypress??
Hi,
It's important to respect the security measures put in place to protect websites and users from automated attacks and misuse of data. Attempting to bypass security measures can lead to legal and ethical issues, and can also harm the integrity of the testing process.
Instead, I recommend looking into ways to automate your tests in a secure and ethical manner. Cypress provides various options to work with reCAPTCHA, such as stubbing the reCAPTCHA API or using a service like AntiCaptcha to handle the reCAPTCHA challenge. However, it's important to ensure that any methods used to handle reCAPTCHA are in compliance with Google's policies and do not compromise the security of the website or its users.
Hello @Raghav Can you please make a video on the 90-days notice period
I have it Tanmay - th-cam.com/video/T-mQ2Z9Cwdg/w-d-xo.html