Test Automation Forum

The Evolution of Software Testing: Shaping the Past, Present, and Future of Technology

Functional Testing / By Subrat K. Acharya

Introduction: In the ever-changing world of software development, testing has emerged as the unsung hero that ensures the seamless functioning of the applications we rely on daily. From its humble beginnings as a manual process to the cutting-edge AI-driven techniques of today, software testing has evolved to become a cornerstone of the IT industry. Its impact is profound, not just in enhancing efficiency but also in laying the foundation for the technological marvels of tomorrow. The Early Days: Where It All Began In the early days of software development, testing was a manual and labor-intensive process. Developers and testers would sit for hours running test cases, meticulously recording results, and verifying if the application behaved as expected. This approach, while effective for its time, was prone to human error, time-consuming, and unable to scale with the growing complexity of software. Despite these challenges, early testing practices laid the groundwork for understanding the importance of quality assurance. Bugs discovered late in the development cycle often led to costly delays, and organizations began to realize that testing is not just an afterthought but a necessity. The Automation Revolution: A Game-Changer The advent of automation tools like Selenium, QTP, and LoadRunner marked a pivotal moment in the evolution of testing. Automation brought with it the ability to run repetitive test cases quickly and accurately, saving significant time and resources. Testers transitioned into automation engineers, mastering scripting languages and leveraging tools to improve test efficiency. This era also introduced the concept of regression testing, where automation scripts ensured that new changes didn’t break existing functionality. For the first time, testing became scalable, enabling organizations to tackle larger, more complex projects. The Agile and DevOps Era: Continuous Everything As software development embraced Agile methodologies, the nature of testing underwent a dramatic shift. Agile emphasized frequent, iterative releases, necessitating continuous testing. This led to the rise of test automation frameworks that integrated seamlessly into the development process. The adoption of DevOps took this a step further. In a CI/CD (Continuous Integration/Continuous Deployment) pipeline, automated testing became the gatekeeper for deploying code. Concepts like shift-left testing emerged, encouraging teams to test early and often, catching defects before they snowballed into larger issues. Testing was no longer a siloed activity but an integral part of the software lifecycle, fostering closer collaboration between developers and testers. Emerging Trends: The Future is Now The testing landscape continues to evolve, with emerging technologies pushing the boundaries of what’s possible: AI and Machine Learning in TestingArtificial intelligence is revolutionizing testing by enabling smarter test case generation, defect prediction, and self-healing test scripts. Tools powered by AI can analyze vast amounts of data to identify patterns, making testing faster and more reliable. Cloud-Based TestingCloud platforms offer scalable and cost-effective environments for testing, allowing teams to simulate real-world conditions. This has become especially crucial for distributed teams working in different regions. IoT and Big Data TestingThe rise of IoT devices and data-driven applications has introduced new challenges in testing. Ensuring the security, performance, and functionality of interconnected systems is becoming a top priority. Performance and Security TestingWith increasing cyber threats and user demands for faster applications, performance and security testing are at the forefront of ensuring that systems are both reliable and safe. Impact of Testing on the IT Industry The impact of testing on the IT industry cannot be overstated. Testing ensures that software is reliable, secure, and user-friendly, building trust with users and protecting the reputation of organizations. Key benefits include: Efficiency: Automated testing accelerates delivery timelines, enabling companies to bring products to market faster. Cost Savings: Early defect detection reduces the financial burden of fixing bugs post-deployment. User Satisfaction: By ensuring seamless performance, testing enhances the overall user experience, driving customer loyalty. Testing has also created new career opportunities, with specialized roles such as performance engineers, test architects, and security analysts. The profession is no longer confined to debugging; it’s a dynamic field that demands creativity, problem-solving, and technical expertise. As we look ahead, the future of software testing is brimming with possibilities: Hyperautomation: Testing will become fully automated, with minimal human intervention. From generating test cases to analyzing results, machines will handle most of the workload. Autonomous Testing: Imagine systems capable of testing themselves, adapting to changes in real time. This is no longer a distant dream but a rapidly approaching reality. AI-Driven Testing: AI will become even more sophisticated, enabling tools to predict defects, optimize test coverage, and ensure flawless execution. Collaboration and Integration: The line between developers and testers will blur, with testing deeply embedded in every phase of development. Teams will work together seamlessly to deliver quality products. Focus on Ethical Testing: As AI and automation take center stage, ensuring the ethical use of technology will become a priority, with testing playing a crucial role in validating fairness, transparency, and accountability. Conclusion The evolution of software testing is a testament to the relentless pursuit of excellence and innovation in the IT industry. From manual processes to AI-driven testing, the journey has been transformative, enabling faster, more reliable, and secure software development. Testing is no longer just a phase in the development cycle; it’s a strategic enabler that ensures the success of every project. As technology continues to advance, testing will play an even more critical role in shaping the future, driving innovation, and ensuring that software meets the highest standards of quality. In this rapidly changing digital era, one thing is certain: testing is not just about finding bugs; it’s about building trust, improving lives, and paving the way for a smarter, more connected world. 5+

Unit Testing: The Backbone of Reliable Software in Rapid Development

Functional Testing / By Ajit Kumar Parida

The importance of Unit Testing in modern Software Development: In today’s fast-paced software development industry, the need for rapid delivery of reliable, high-performing applications has driven the adoption of Agile and DevOps methodologies, which prioritize continuous integration and automated testing. Unit testing is essential in this context, as it helps catch bugs early and maintain code quality by ensuring individual components function correctly. Skipping unit testing can lead to costly mistakes and hinder future code improvements. Despite challenges in implementing it, effective strategies ensure unit testing remains a key practice for delivering robust, maintainable software in a constantly evolving environment. Why is Unit Testing necessary? Early bug detection: Unit testing helps find problems early by testing small parts of the code separately, making it easier to spot where issues are. Improved Code Quality: Writing tests helps you consider different scenarios, leading to cleaner, more organized code that’s easier to maintain and less prone to errors. Faster Debugging: Unit tests make it easier to find and fix bugs since failed tests point directly to the part of the code causing the issue. Confidence in Refactoring: Unit tests ensure that when you change or improve the code, it still works as expected, providing safety when making updates. Facilitates Continuous Integration: Unit tests support frequent code changes by automatically checking that new updates don’t break the existing code, ensuring smooth and stable development. Challenges faced during Unit Testing: Test coverage: Covering all important parts of the code with tests can be hard. Developers might miss certain cases, leading to incomplete testing, which can leave bugs unnoticed. Complex dependencies: Real-world apps depend on things like databases or external services, making testing difficult. Developers use tools like mocking to simulate these dependencies, but it can be tricky. Maintenance overhead: Since unit tests are also code, they need updates as the application changes. Keeping tests updated can be time-consuming, especially with frequent changes. Time and effort: Writing good unit tests takes time. Some developers may skip testing to meet deadlines, which can lead to missing bugs and issues later on. False positives and negatives: Sometimes tests pass or fail for the wrong reasons. These incorrect results waste time and reduce confidence in the tests. Difficulty testing legacy code: Testing older systems that weren’t designed for testing is challenging because they might not follow modern practices, and it is often hard to break into smaller parts for testing. Checkpoints and strategies for effective Unit Testing:To overcome the challenges of unit testing, it’s important to adopt best practices and strategies that ensure thorough, maintainable, and meaningful tests. Here are some key strategies to follow: Test early and often: Start writing unit tests as soon as possible, ideally before the code. This catches bugs early and reduces the cost of fixing them later. Maintain proper test coverage: Focus on covering important parts of the code, not just getting high coverage numbers. Quality matters more than quantity when writing tests. Isolate units for Testing: Test one piece of code at a time without relying on databases or external systems. Use mocks to simulate these dependencies for faster, reliable tests. Focus on business logic, not trivial code: When writing unit tests, it’s essential to focus on testing complex business logic and edge cases rather than trivial code like simple getters and setters. Testing these trivial aspects often results in wasted effort with little value added. Instead, prioritize testing the critical areas where the logic is more intricate and prone to issues. This approach ensures that testing efforts are more efficient and effective. Keep tests simple and readable: Write tests that are easy to understand. Avoid adding unnecessary complexity, so test failures give clear feedback on what went wrong. Automate Testing: Integrate tests into your CI pipeline, so every change is automatically checked. Automation saves time and prevents bugs from reaching production. Don’t aim for 100% test coverage: Test coverage tools are helpful, but don’t get obsessed with hitting 100%. Push for perfect coverage led to meaningless tests. Instead, we aim for good coverage in critical areas. Quality matters more than quantity when it comes to tests. Refactor tests regularly: Keep tests up to date as your code changes. Remove outdated tests and improve old ones to keep everything relevant. Measure test performance: Make sure tests run quickly. If they slow down, optimize or refactor them to keep development efficient. Practice code reviews for tests: Review test code just like production code. This ensures proper design, coverage, and helps maintain high testing standards across the team. Conclusion: Unit testing is crucial in modern software development, ensuring individual components work correctly, catching bugs early, and improving code quality. Despite challenges like complex dependencies, time constraints, and maintenance efforts, unit testing provides long-term benefits such as faster debugging, simplified refactoring, and better support for continuous integration. By adopting strategies like starting tests early, isolating units, and automating tests, teams can overcome these challenges. In today’s fast-paced environment, unit testing not only ensures stable, reliable code but also enhances collaboration, enabling developers to deliver high-quality applications that meet user expectations. 1+

Qualities of a good Automation Test Report

Introduction: A good test report is the one which can potentially help the Test Engineer/Automation Engineer at not just finding the failure in the application but would also guide them with number of supporting steps to easily locate the failure which enables the faster communication & quick resolutions especially in the DevOps world. This also helps to standardize your quality assurance process. In this article let us try to learn those qualities of an automation test report that contributes to the overall quality of the software application under test. Here, we are not referring to any particular Automation tool or its respective report format, we are just trying to appreciate the qualities which are already existing (could also be the reports you manually create) & some hypothesis of how they can be improved/utilized in some context in your everyday reporting structure. What info should a good test report capture? NOTE: We are not just thinking about the immediate benefiters of the reports, but about all the stakeholders of the project. 1.Number of Tests Passed/Failed When you are doing your batch runs, this metric is a great demonstration of the overall success rate of the automated test suites & can be used for your daily/weekly management of your quality reports 2. Screenshots Your test failed at a particular step & it’s good that the tool captures the screenshot for that step. But not just for failures, it will be fantastic if even the successfully passed steps displays the screenshot. This also helps the test engineer to understand the application quickly as well as the business process (if the person would like to check them when free) & can hold good knowledge of the test scripts if new to the team & is quick to the regression execution work😊 I am someone who has done that in the past! 3. Smart identification – Warnings We can’t thank Automation tools enough for being great all the times in identifying the exact properties of the field that you are using in your script. The tool will try its best to check all the possible properties & try to pass your test by being so SMART before giving up for failure! These warnings in your test report quickly contributes to your script maintenance & be ready for your script updates as soon as you see them, capture those steps & field names. 4. Incomplete tests in the execution Your script fails which is fine for you, but you are not happy about the pending incomplete tests in your test suite which sometimes can be huge & requires re-running of your whole test suite when you are back at your work next Morning! Try to analyse the dependency in the business process & see any of the tests from the incomplete runs can be run independently & can be positioned somewhere else in your test suite if the probability of the failure at that particular step is high! 5. Colour/themes in your report for different metrics Knowingly/unknowingly a human eye is biased towards some favourite colours & always like to see success/failures in certain colours. How good it is to satisfy this criterion for not just for Pass/Fail but also colour coding other major metrics in the report as well. 6. Export option So, you have your test report & how about exporting the report in a certain format to share it with your stakeholders as well as your team. (And may be able to Read/Edit once exported according to their convenience) Example: Excel, HTML, XML etc. 7. Report Filters Everyone has their own way of looking at the test results. Can we think of having the ability to filter the test reports based on the metric which you are expecting to get the better visibility into the test analysis! I love filters 😉 8. Auto suggestive error messages Unknown errors in the real time execution of an application are very common & we absolutely love the automation tools to come up with the auto suggestive descriptive error messages instead of just telling STEP FAILED or ERROR at the step etc. for some of the application failures like browser issues, window popup etc 9. Application Browser link to the failed steps These days as we mostly work with the cloud applications such as Salesforce & record links works amazing when dealing with the issues. In built variables in the automation tools to capture the browser link for the failure steps works miracles provided even if the Test Engineer don’t necessarily use these variables at certain steps & may not be useful sometimes. Makes sense? 10. Deciding the order of execution of a test suite Sometimes the business requires certain processes to be run for some urgent basis & it’s cool that you can run your test suite in a certain order yet be able to know the results that can be decided based on your comprehensive report 11. Time & Date metrics It’s an important metric for any test automation report & I absolutely like the exact time taken to run the test suite displayed BUT also the time taken for every step! This helps in deciding whether manual work is appropriate for some critical business requirements of the application. 12. Ability to display the test steps in the report Instead of some major metrics like test scenarios, it’s always great to see the step names in some way in the report to quickly catch the issue & be able to replicate it on your own. 13. Ability to tag some test cases as critical in the suite & this being captured in the report In regression testing, it’s sometimes important for certain requirements to be critical & we tag them as such in the script and your report shows those steps as critical & you have a separate view to see the progress of such scripts. When report shared with the stakeholders, they would also look at those which they need & are tagged

Selecting the right Test Automation Tool for your organization

1 Comment / Functional Testing, Performance Testing / By Tri Rath

There are hundreds of test automation tools that are being offered in the market today and many new entrants are also coming to market every now and then which makes the tool selection process a bit cumbersome. Selecting the right automation tool will drive the success of your automation journey as the right tool will bring the desired outcomes like the productivity, cost savings and the speed to the market. Few of you might have already carried out this exercise in the past as well. In the current times of digital transformation, the factors that drive the tools selection process have changed considerably, because of the changes in software development models e.g., Agile, DevOps, shift-left etc. and also due to the emergence of the latest technologies like AI/ML, RPA and so on. In this article, we will discuss around how to choose the right automation tool for your organization. When we think of selecting a tool for our organization, it’s obvious that we have to consider other programs or projects outside our team. The projects/products outside our team might be using different technologies, different interfaces, services and so on and it is not an easy task to identify one tool to cover all the applications/products in the organization. So, we have to consider and analyse all the applications/products carefully and group them logically so that we can initiate our search for the perfect tool for each of the groups. This will ensure effective utilization of the tool in our organization for better and quicker returns on the investment, Prerequisites Before we proceed with initiating the selection process of automation tools, we must finish the detailed analysis of the scope of the automation i.e. applications/products to be automated, budget/cost, team composition/skillset, tools/platforms that are already in place in the eco system, company’s IT strategy etc., Once the automation scope analysis is completed, in order to kick-off the tool selection process we need to work on identifying the selection criteria for the overall evaluation process. The automation tool selection process generally follows a 4-step method as below: Stakeholders, roles and responsibilities Who should be part of the overall evaluation, review and final selection process? Well, for a structured, unbiased and an overall effective tool selection process, we strongly recommend you involve below members as part of the exercise: Test engineers Tech. Architect Test managers Project Managers BU Leader/QA Head (recommended for selection process for enterprise-wide rollout) A representative from your sourcing team (for a better negotiation/deal with the vendor at the later stage of your selection process) For a smooth coordination, a Test Lead or a QA Manager should own and champion this task under the guidance of a senior leader like QA Head or so. The above structure can be customized as needed depending on several factors like centralized/decentralized evaluation and so on. Before this task begins, ensure you have roles and responsibilities identified for each stakeholder in the overall evaluation process. Below is a sample RACI matrix for your reference. You can add/remove other stakeholders to customize it as per your Org structure or for any specific evaluation need. One more crucial point to be considered during the overall process is professional ethics. Senior leaders have a critical responsibility here to set some ground rules around how do we communicate and partner with the representatives of the vendors partners and guide the team as needed. The vendor partners should be dealt with due professionalism and transparency for the best outcomes. Each and every tool have their own advantages as per their designs. Our goal should be primarily focused on identifying our automation need and ultimately selecting the one that fits to most of our requirements and not finding faults with the individual tools. Identifying the selection criteria for the right automation tool This is the critical phase of the tool selection process. In order to make our selection process structured and unbiased, we need to evaluate all the potential tool candidates against the same selection criteria so that we have a common rating method to identify the best tool for our purpose. We will discuss about a template later in the article that captures the common selection criteria and the rating for individual tool candidates to find out the best tool. Below are several aspects or selection criteria of automation tools that you might consider as part of your overall evaluation process. The list might not be an exhaustive one; you have to add/remove/edit criteria based on your company’s automation needs, IT strategy, etc. Test coverage/Versatility: In today’s digital age, modern Apps/Products use complex architecture covering Web, Windows, Linux/Unix, Mobile, Databases, Web Services and so on. You have to ensure the tool you are picking up from the market has the ability to support the required end to end UIs/Layers of your app. We also recommend you to check if the tool can also help in doing other types of testing like Unit, Integration, Code reviews/performance checks, security scans, etc. Learning curve, scripting speed, maintenance effort and reusability: This criterion will help you evaluating the ease of adoption of the tool for your project/company. The skillset/competency of the current team, ease of developing/debugging the script and the script maintenance effort etc. are few important factors that you have to check to determine the right tool. Script reusability inside or outside your project is also a key factor. Script reusability will help driving the productivity of the QA team. You should also study the abilities of the tools around development of frameworks like Key-word driven, Data-driven or script-less models etc. that best fit to your need. Seamless integration with other tools: As we adopt more and more of processes like Agile, DevOps etc. the software development ecosystem becomes more diversified in terms of other tools (open source/commercial) that are used for different other activities like version control, build, testing, deployment, etc. as part of the overall development process. In this scenario, the test automation tool’s capabilities around talking to

Key Benefits of Continuous Testing in Mobile application development

Functional Testing / By Prerna Bhatt

Introduction Undoubtedly we are in an era where digitization is the ruler. It is difficult to spend a day without our smartphones. You can step out of the home without a wallet but not a smartphone. Ordering food, transportation, viewing directions on a map, creating official docs in ms doc , excel to ppt, everything can be done over a mobile phone. So with time, the smartphone users have transformed being tech savvy and their demands are rising. That’s the reason Brands are looking for technical solutions which are faster and qualitative. Continuous testing accelerates the speed of testing and also increases the quality of the application by multiple folds. In this blog we will discuss continuous testing and how it helps in enhancing the quality of the application under development. What is Continuous Testing ? In continuous testing, testing takes place in all the stages of the development cycle. In other words, it is the process of implementing continuous testing throughout the DevOps. As shown in the figure above, continuous testing can be started right from the planning till spanning the complete DevOps loop till the monitoring part. Enables faster delivery of Mobile and Web Apps The combination of DevOps, Agile and Continuous Integration has reduced the timeline for design, development and delivery for software updates. In continuous integration, small chunks of code are committed frequently and hence the deployment rate increases by multiple folds. In one day multiple deployments are made instead of weekly or monthly unlike the traditional sdlc. For productive results in continuous testing, one has to make sure that the high speed automation tools are used. Hence continuous testing accelerates the speed of testing. More Efficient Testing In Continuous testing, testing is done at each and every stage of the pipeline, ensuring the developers the confidence for building a secure and highly flexible framework. Moreover, testing from the earlier stages helps detecting and rectifying bugs easier and also saves huge bucks in rectifying bugs at the later stages unlike the traditional sdlc’s As testing starts from the earlier stages and frequently committing codes stops the testers to wait for a long time for resolving the bugs. Continuous testing helps the developers to conduct the right test at the right time .i.e allowing them to determine whether a shift left or shift right is necessary in their delivery pipeline. End to end testing with automated tools further accelerates the speed of testing. As continuous testing takes place in an integrated environment which is very flexible and agile in nature, which ensures a strong architecture for future expansion of the application under development. Broader Test Coverage Continuous testing usually takes place in continuous integration and DevOps environments. Here, testing takes place at a high speed because of integrated automation tools. It covers a huge range of test cases covering the functional and non- functional test cases. Regression testing , Integrated testing, Performance and Cross Browser testing, it covers a variety of test suites. Moreover continuous testing increases the efficiency of testing by multiple folds which is a cherry on cake. Conclusion Mobile application has evolved rapidly since last decade and sohas mobile application testing . Continuous testing has brought a transformational growth because of its innumerable plus points like increasing the overall efficiency in mobile application testing, providing high performance applications, wider test coverage etc. These are some of the key benefits of continuous testing which makes it the most adopted testing solution of these times. I hope you found this article helpful. Let me know if you would like to discuss further if you have any questions or suggestions around this topic. Thank you for your time reading through this article. 8+

Setting up ROBOT FRAMEWORK using Red Editor

Functional Testing / By Renuka SV

Introduction: In today’s world of Agile/DevOps, the top priority for QA Engineers is to finish the thorough testing of the application under development within the sprint cycle itself. Not all QA professionals have access to commercially available automation tools (due to cost and other constraints) and hence, an open-source platform becomes the best choice to test the application within a very limited time. In this article, I’m going to discuss how can we quickly get started with a popular open-source platform called Robot Framework which is purposefully used for acceptance level testing and acceptance test-driven development (ATDD), using an Eclipse based editor called Red Editor. So, let’s get started. Background: Few months ago, following an internal decision, our team was challenged to switch to an open-source automation tool from a licensed automation tool and develop the automated test suites from scratch. Considering several factors like our team structure, different technologies & platforms used, cross project reusability, and so on, we started exploring few open-source automation tools available in the market. We did few POCs as well & based on the results and efficiency of the tools, we had to finally choose the one that met most of the preidentified success criteria for the POC. So, the automation tool we finally zeroed on is none other than the Robot Framework. There are various tools available for test automation in the market – it could be licensed or open-source. Robot Framework is simple, open-source, easy to use, powerful and is also easily extensible tool which utilizes the keyword driven testing approach. Robot framework helps us to automate the regression test cases in much lesser time and with greater efficiency, thus saving cost, faster go-to market time and enhancing the quality of the software product. Why Robot framework? At the end of the POC, Robot framework was found to satisfy almost all our business requirements. It’s a python based, extensible keyword-driven which can be implemented on all major platforms. So, one of the major challenges in our requirements was to have the multiplatform support fulfilled. Robot Framework is used for TDD, BDD and RPA. Robot Framework is also maintained/upgraded regularly by Nokia. So, now as you know what is Robot framework, you might be having some questions! And let me answer! Where are the Robot Framework test cases stored? – HTML Files Can we create the Reusable Keywords from the library? – Yes Can we create the customized test libraries? – Yes, by using Python or Java Can we store the Regression test suite to any source control system? – Yes How to basically Integrate Robot Framework with CI/CD Systems? – Command Line Interface Can we create Data Driven test cases? – Yes Prevent some test cases from execution? – Use Tagging GUI Editor: After we finalized on the Robot Framework, we also finalized on the test editor to be used, that is RED Editor. This is an Eclipse based editor and I wanted to give a quick overview in the next section on its architecture and how to set it up in order to get started with developing and executing automation scripts. High-level architecture: Installation & setup of RED Editor: Below are the few prerequisites/steps you can follow to install and configure RED Editor: Java 8 must be installed on PC and Environment variables must be set Install Eclipse EE Oxygen 3A Package Go to Eclipse Marketplace and Install RED editor Install Python 2.7.17 from below location https://www.python.org/downloads/release/python-2717/ Add Python 27 and Python 27 Scripts location to Path Environment Variable as shown below Check if Python is installed using python –version command in cmd prompt Use “pip install robotframework” command in cmd prompt to install robot framework Use “pip install robotframework-seleniumlibrary” command in cmd prompt to install Robot Selenium Library Use “pip install robotframework-openpyxllib” command in cmd prompt to install Openpyxllib for Excel Manipulation Go to Eclipse and navigate to File->New->Others->Robot Project and specify project name Once Project is created Navigate to red.xml, go to Library tab and Add Desired Libraries by Clicking on Add New Library File->C:Python27Libsite-packagesSeleniumLibrary and Select __init__ file. Do the same for Openpyxl Library and Save file Test Case creation: RED editor is a text & table-based editor with code colouring, debugging, comment features. It has the keyword completion feature that shows the keywords that are found either from the test suite, resource files or libraries. You can simply type the starting letter of the keyword which in turn displays all the keywords starting with that letter (along with the arguments) which also makes this framework very easy to use, even for a test automation beginner to work with. Also, the arguments passed in the editor are validated automatically and displays the errors or warnings if there are any syntax errors. This feature works for built-in as well as for customised keywords. Keywords and variables can be separately stored into resource file, so it can be then used in number of other test cases or test suites. It’s always a best practice to move the keywords to shared resource to enhance the reusability of the keywords which also avoids the duplication of works and saves the manual effort. Our team chose to save all the variables separately in the resource file. Test Case execution: We can execute the test case or test suite directly from the RED Editor. Test cases or test suites can be run in sequence, and the ordering can vary using the tags. With Robot Framework, whole execution of test suite takes only few hours. Test report: Once the execution is completed, HTML report will be generated, and the report colour easily reveals whether the execution has been successful or not. Summary Information in the report displays the overall status of the test execution, Start Time, End Time, Elapsed Time and the link to the log file. If the Critical test cases are passed successfully, then regardless of the other test cases results, the report will be marked as OK. However, statistics will

Is Script-Less Test Automation The Future ? Part-2

Functional Testing / By Tri Rath

Wishing everyone a happy new year 2021! As I indicated earlier, this article is continuation from Part-1 where we discussed on how test automation tools have evolved in last few decades and how leading test automation tools are adopting latest technologies like AI-ML etc. and enabling script-less test development, test maintenance and intelligent test execution in order to accelerate the automation that is really the need of today’s Agile/DevOps world. If you have not yet read the Part-1 of this article then you can read it Here. In this part, we are going to see a listing of such 30 script-less popular test automation tools. Please note that the below listing is not a comprehensive one. Several more such tools are there in the market as well. If you are aware of such a tool(s), then don’t hesitate to give me a heads up on LinkedIn and I can add them in the below list too. Note: The below listing is not intended to rank the platforms as per their features. The brief description, features etc. of each of the tools/platforms below are taken from their official product info web-page. 1. Tosca Provider: Tricentis Licensed/Free: Paid Next-gen automated Continuous Testing. Tricentis Tosca, the #1 Continuous Testing platform, accelerates testing with a script-less, AI-based, no-code approach for end-to-end test automation. With support for over 160+ technologies and enterprise applications, Tosca provides resilient test automation for any use case. Gartner has recognized Tricentis as a “Leader” in Software Test Automation for 5 years in a row. 2. EggPlant (DAI) Provider: KeySight Technologies Licensed/Free: Paid EggPlant Digital Automation Intelligence: Test smarter with AI-powered automation that views technology from the users’ perspective, improving how you test, develop and deliver in the digital age. A modern AI-assisted approach to test automation that helps you create products that delight users, deliver at DevOps speed, test the full customer experience, test ANY technology, and predict the quantified impact of new product versions on the user before release. Eggplant uses scriptless models, AI, and analytics to expand automation beyond test execution across the full testing process. From test-case generation and test optimisation to results analytics.Gartner and Forrester have recognized EggPlant as “Leader” in Software Test Automation. 3. ACCELQ Provider: ACCELQ Inc. Licensed/Free: Paid ACCELQ is an AI powered Codeless Test Automation & test Management platform on Cloud. (Unified platform for Web, Mobile, API, DB and Packaged Apps) ACCELQ is listed as Leader in Forrester Wave™: Continuous Test Automation Suites, Q2 2020 This platform covers the below areas of testing:– Quality Lifecycle management– API and UI End-to-end validation– In-sprint and Codeless Automation 4. Ranorex Provider: An Idea Inc. Company Licensed/Free: Paid Ranorex Studio: Functional UI Test AutomationEveryone from novice to expert can build sophisticated tests for desktop, web, and mobile. Rapidly build and deploy reliable, maintainable automated GUI tests – with or without coding. Easy-to-use tools, a comprehensive set of test automation features, and built-in best practices. Ranorex Studio enables testers to create modular automation scripts that are efficient and maintainable, regardless of programming expertise. Reach your test automation goals faster with Ranorex. 5. Katalon Provider: Katalon LLC Licensed/Free: Paid Katalon Studio is an end-to-end automation platform. Below are the key features of the paltform: Test Generation– Productive IDE to generate automated tests easily for all platforms and OSs, regardless of application complexity.– Powerful recording utility for effortlessly storing all UI elements to maximize reusability.– Codeless experience for beginners. Infinite testing extension for experts. Test Execution– Smart engine with auto-heal and auto-wait capabilities.– Speed up scripting and execution processes.– Improve team performance, reduce maintenance costs. Test Analysis– Powered by artificial intelligence.– Track and monitor in real-time with holistic insights.– Instant end-to-end feedback loop for CI/CD pipelines 6. TestIM Automate Provider: Testim Licensed/Free: Paid Super-fast authoring. Amazingly stable tests. Your way—coded, codeless, or both! Key features of TestIM Automate:– Expand test coverage: Fast authoring increases coverage and quality across your application.– Reduce maintenance: AI-based stabilizers eliminate flaky tests that consume resources.– Troubleshoot quickly: Diagnose failed tests, pinpointing root cause to fix bugs and release faster. 7. Qualitia Provider: QualitiaSoft Licensed/Free: Paid QUALITIA – The test automation platform that does not put testers to test! Qualitia is one-of-its-kind scriptless test automation platform that’s designed to think like a QA Expert. Empower your functional manual testers, business users and subject matter experts with automation, without learning programming. Key features of QUALITIA:– Qualitia has easy and powerful point and click mechanism to identify UI objects.– Start in-sprint automation the moment your stories and wireframes are ready!– Automate complex test scenarios requiring conditions based on the run time results.– A detailed report for every step of execution that captures screenshots of the current application status.– Qualitia’s Automation Bots and Integration help you to confidently enable your DevOps pipeline.– You can author Custom Actions directly in Qualitia and the Custom Action code is maintained in Qualitia’s project library. 8. TestMagic Provider: Universal Technology Solutions Licensed/Free: Paid AI/ML-based End-to-End One-Stop Scriptless test automation Platform. Unified AI/ML capabilities across different stages of Test Automation – Test Creation, Test Execution, Reporting & Analysis. Key features of TestMagic– Script-Less and offline Automation– Single Sign-On– Offline Automation– Object Version Agnostic– Test Data Encryption– Inbuilt Version Management– Operational and Business Analytics– Exception Handling– OCR Capabilities– Computer Vision– Mouse and Keyboard Automation– STAAMAP And iAutoHeal– AI-Driven Automated Test Pack Execution – AI-Driven Test Reporting– AI Driven Scheduler– Smartphone Test Farm 9. TestArchitect Provider: LogiGear Corporation Licensed/Free: Paid TestArchitect is an integrated codeless Test Automation platform that focuses on scaling up Test Automation coverage & productivity. Key Features of TestArchitect:– Codeless Automation– Automation Analytics– Tools Integrations for CI/CD/DevOps Testing– Web & Web Service Testing– Legacy App Testing– Image-Based Testing– Data & Database Testing 10. TestCraft Provider: Perforce Licensed/Free: Paid Escape slow testing with TestCraft’s no-code automation boosted by AI. TestCraft accelerates test creation, execution, and maintenance. Designed for functional UI testing of web & mobile-web applications.– Automate stable regression & UI tests

Is script-less test automation the future ?

1 Comment / Functional Testing / By Tri Rath

Well, this article is going to be the last publication of 2020 in the forum. In this article, we’re going to discuss how script-less or no-script based automation tools (Functional) are dominating the Test Automation Tools market globally. Be informed that I’m going to discuss more than 25 such popular script-less tools and therefore, this is going to be a bit lengthy article, and hence, will be a multipart article being divided into 2 parts. In this part, I am going to provide some background on how test automation tools have evolved in last many decades, challenges with conventional tools, how script-less tools are making lives of test automation professionals easier and why we should consider that these script-less tools are going to be futuristic. Introduction As we step into 2021, our quest for speed around Test Automation will continue to grow. With everchanging business dynamics due to several environmental factors like COVID19 pandemic, has pushed the enterprises to adapt to the new age automation tools/platforms that are easy to adopt, intelligent and are also cost & time effective. In today’s DevOps/Agile world, delivering on 100% test coverage within the sprint cycle is not an easy task. Testing teams are always challenged to squeeze the testing window in order to keep up to the code delivery schedules. Test automation has 3 broad areas like Test Development, execution and script maintenance, the ideal automation tool is the one that brings the maximum speed around these 3 areas as one of their primary requirements. Have a look at the chart above that gives a snapshot of how we have come along with the evolution of Test Automation tools over last several decades. Challenges with conventional script-based testing tools In last few years, as DevOps started gaining popularity to accelerate the Digital Transformation, traditional automation test engineers upskilled themselves on open-source technologies like Selenium, Appium, JMeter, Robot framework, Jenkins etc. and also on several commercial tools, in order to build continuous testing solutions that are needed as part of the CI/CD setup for continuous delivery. Testing teams were able to complete the configuration seamlessly from automation standpoint however, “Speed” was still a concern as 100% in-sprint automation coverage was still not a reality all the time. The below points describe the issues with the conventional automation tools that need testing professionals to write scripts/code to build their automation suites: Scripting/coding in Test Automation takes time for development, dry runs and so on and it is difficult to achieve 100% automation in each sprint in a timely manner Significant rework in the script/code is involved when functionalities or objects change Automation coverage and overall efficiency (e.g. reusability etc.) is dependent on the experience/coding skills of the test engineer Integration with other tools/plugins might be difficult as that needs some level of technical skills How script-less testing tools address the above challenges? Script-less/codeless Testing tools don’t need coding or programming skills and are designed to create automated tests that are structured and customizable. The goal is to create the automated tests quickly, self-recognize the changes and auto-correct the tests and also to reduce the maintenance costs. Ultimate idea is to enable “Anybody can automate the tests”. Note: Not all script-less tools are really 100% code free. There might be a marketing gimmicks as well, some of them are marketed as script-less tools however, there might be low or very minimal coding required. Let’s try to understand how script-less tools work. Script-less tools use the below methods in order to create the Automated tests quickly: Object based testing NLP: Natural Language Processing (AI) Model based testing Key-word driven testing Record and playback Optical Character Recognition (OCR)/Image recognition AI Bots for business process automation Script-less Test Automation needs very minimal learning curve and are also simpler to use. Why script-less automation tools are thought to be futuristic? I hope you are familiar with Hyper-automation. So what is hyper-automation? In short, Hyper-automation is intelligent automation of automation i.e. cognitive orchestration of cluster of automations of several business processes. Hyper-automation enables end users (e.g., a field worker) with no coding skills to automate several low-level digital tasks thereby creating something called Digital Twin i.e. a virtual representation of one or several business processes. This is possible due to emergence of intelligent platforms like RPA (Robotic Process Automation) that leverages AI and Machine Learning models to automate bulky and repeated tasks which earlier were done manually. One of the Gartner’s Top 10 strategic predictions for 2021 and beyond is “Neuromorphic Computing” — a computer that thinks and acts more like the human brain which will begin to take their place, which will be a reality in next 5 years or so. So, it is evident that we are possibly hitting a Digital Era of No-Coding in next few years (unless you are developing your own software product) and the emergence of AI/ML powered script-less tools are just the beginning of this transformation process. In the part-2 of this multipart article I’ll discuss several emerging Script-less Test Automation tools in the industry that are gaining more and more popularity over last few years. Looking forward to connect with you on part-2 of this article in the New year, 2021. Thank you for your time on this article. Wishing you a bright and happy new year! 7+

API Testing – An approach to shift-left ideology

2 Comments / Functional Testing / By Saravanan Anantharaman

Introduction Software delivery teams have always faced multitude of challenges in an ever-changing landscape of tools, technologies & practices. Teams must constantly upgrade themselves in this multi-dimensional landscape as software development processes transitioned from traditional models to Agile & DevOps. The constant focus for the teams during this transition has been to have a potentially shippable product at end of every iteration. As iterations became leaner and thinner, releases became more frequent and reduced the distance between the user aka operations and the development teams. During this period of frequent releases, teams had to constantly adapt to having releases over several months to fortnights to days and more so hourly in certain industries. Over this long journey from past 2-3 decades, during which Software teams have significantly evolved & adapted to changes, the only constant which has remained unchanged through this enduring expedition has been Software Quality, which defines if a customer stays with you or walks out of the door. I would also like to bring your attention towards how applications have matured over decades. Software teams have constantly enhanced the architecture from being monolithic to distributed, services oriented and microservices to support user experience, customer needs, availability and security. Hence, our focus today is at intersection of two very important changes we have seen in our industry over the decades – Software development processes & Technology. Today, in this article we will not only look at one of the many different strategies which teams follow to drive quality mindedness, but also understand how we can achieve it. So, lets dive in – One of the important strategies we are talking about today is Shift-left ideology. Software teams come up with great jargons and you will hear many more as you read! The idea here is Software teams reduce the risk of finding failures as much by testing early, which in turn reduces overall of cost of quality among the various other benefits. Before going any further, let’s look at Martin Fowler’s Test pyramid. On a side note, I am a huge fan of Egyptian pyramids, since they are humanity’s one of the greatest inventions and their solid architecture has stood the test of times. So, in my view a strategy built around Test pyramid is so much relevant even today as we will see that this approach will help us build a solid approach to Software Quality. Our focus here is going to be on one of the components in Test pyramid – Service or API layer. We will look at one such approach for API layer – API testing and will specifically look at how an open source testing tool – Karate can help us achieve this goal. Interesting name for a tool isn’t it! Anyone who masters this tool will henceforth be called ‘The Karate Kid’…Just kidding! I would like to take this opportunity to thank founders of this tool for making easy it easy for Software teams. So, what’s an API?? In Service Oriented Architecture (SOA), an Application Program Interface (API) is a way for various applications to communicate with each other using a common language, often defined by a contract. Examples of these would be a Swagger document for RESTful services or a WSDL for SOAP services. APIs represent building blocks that developers can use to easily assemble all sorts of interactions without having to rewrite an interface every time they need machines to communicate. Additionally, since APIs have contracts, applications that want to communicate with each other can be built in completely different ways, if they communicate in accordance with the API contract. This allows different developers from different organizations in different parts of the world to create highly distributed applications while re-using the same APIs. We won’t dwell too much into types of APIs, but primarily web services/API development falls into either REST or SOAP based on the architecture style followed by Software teams. In this article, we focus on REST since it primarily relies on HTTP protocol and more widely followed. Let us consider a basic example – A REST based application would have a client send a HTTP request to an endpoint with either of 4 basic actions – GET, POST, PUT & DELETE (CRUD) and server responds back with a status and specific content type as we see below: Testing the service layer APIs in above example instead of UI takes us a step towards our Shift-left ideology. But as applications tend to get more complex based on several factors – functional demands, scale, architectural style like microservices etc., we start looking much deeper and the concepts of mocks & stubs will take us more closer in our journey to Shift-left ideology. [will discuss more about it in my upcoming articles] Now that we understand what’s API…Let’s see what & how to test it! One can perform automated API testing to test the application at the API level, design test cases that interact directly with the underlying APIs instead of UI and gain numerous advantages, including the ability to test the business logic at a layer that is easy to automate in a much stable manner. The API tests allow you to test application logic at a level that unit tests cannot. The best way to approach API testing is having an API contract in place. A contract is a blueprint [will discuss more about it in my upcoming articles], which will enable software teams to develop code and tests in parallel – referring to shift left approach I was talking about earlier. Test approach for APIs can vary too, some of most common types of tests include – Specification : Contract Functional : Component, Scenario Performance Security Teams can pick from plethora of tools available based on their need – Rest assured and Karate DSL are two commonly used frameworks. Here, we are going to discuss how to achieve API testing using Karate DSL framework. Few of the aspects which stood out during framework evaluation: Support to

Welcome to TAF - Your favourite Knowledge Base for the latest Quality Engineering updates.