How To Create An Automated Validation For The Functionality Of The Register Machine

Automating testing lets you save resources by automatically checking the functioning of nigh production components. Notwithstanding, the performance of each particular function can't ensure the successful execution of all business scenarios.

After adding new functionality to 1 of our customer's software products, we had to revise our existing acceptance tests every bit they were too reliant on previous functionality. In search of an efficient solution, we turned our attention to a new approach provided by beliefs-driven evolution (BDD). This methodology initially builds software evolution processes based on concern requirements and scenarios. To simplify our testing logic and improve the quality of our software development, we decided to implement BDD tools for acceptance testing on one of our recent projects.

In this article, nosotros'll share our experience of exam automation using the Estimate framework and provide a brief overview of its application.

This commodity will be useful for QA specialists who want to improve their software testing processes.

Contents:

Our initial arroyo to automated acceptance testing

In search of alternatives

Installing Guess

Principal components of Guess tests

Implementing scenarios in Gauge

    Implementing steps in Gauge

Estimate tests and reports

Our experience using Judge

Decision

References

Our initial approach to automated acceptance testing

Afterward developing a new product or calculation new functionality, software development teams usually behave functional testing. Functional testing of complex systems is a complex task, within which credence testing is the concluding step. Acceptance testing aims to confirm that all business requirements are met by the production and that it'due south fix to satisfy customers' needs.

Though this type of testing is normally performed right earlier a production is delivered to the customer, internal acceptance testing can be done at intermediate stages of product development.

When working on i of our projects, we used automated credence tests created by our developers specifically for the needs of the product. Our developers started acceptance testing later on building a new version of the product and discovered the nearly critical and blocking defects. As a consequence, testers got a stock-still version of the product with fewer bugs.

Our approach to acceptance testing provided skilful results – until the product began to grow. Subsequently adding new functionality, nosotros had to create new acceptance tests and rework the existing ones. We significantly reworked our existing tests several times considering they were likewise dependent on the product functionality.

Moreover, since our developers initially causeless that they would be the just ones working with the acceptance examination results, the data nosotros received in the cease was difficult to sympathize for our testers and managers.

All in all, the necessity to constantly maintain our tests and the complexity of the test results forced us to await for a third-political party solution.

In search of alternatives

When searching for the most suitable test automation framework, we took into account our top requirements:

• Assuasive testers as well as managers and clients to hands describe automatic test scenarios
• Simplifying automated examination support when a product grows
• Getting easily understandable reports of exam progress and results

After a brusque investigation, nosotros found that the first requirement (allowing all stakeholders to hands describe test scenarios) could be met with the behavior-driven development methodology. So nosotros decided to consider this arroyo and choose a BDD tool.

BDD is a software evolution procedure that's based on combining business interests with technical insights. BDD emerged from test-driven development (TDD) and also uses tests. The tests are created earlier product development and their successful completion is based on compliance with business requirements. This arroyo allows you to write tests in natural language, so our first requirement could be satisfied with any BDD tool. Later on analyzing several ones, we were drawn to a complimentary open-source project called the Guess framework, equally information technology offers a range of useful features that make acceptance testing much easier:

The near popular BDD instruments like Behave and Cucumber use Gherkin syntax, with the main clauses existence Given-When-Then. These clauses pose sure limitations for describing test cases. Fortunately, Guess allows yous to write tests almost without syntax limitations. So creating tests in Gauge doesn't require special knowledge.

Nosotros besides discovered that Gauge has all the functionality to come across our other requirements. To simplify test back up, this framework can quickly introduce changes to tests with the same steps. Possible bugs in Gauge are fixed by its developers, so Gauge integration makes our tests like shooting fish in a barrel non merely to develop simply as well to support.

Finally, Gauge allows usa to create test progress reports in HTML, so test results are like shooting fish in a barrel for anyone to understand. Additionally, nosotros can get a written report with a list of all existing tests that we can filter and search through.

Installing Gauge

Currently, automated testing with Gauge tin can be performed using programming languages including Java, C#, Carmine, JavaScript, Golang, and Python. Judge supports Windows, Linux, and macOS.

For our projection, we decided to run automated tests written in Python for Windows. Gauge is a panel-based application, then information technology has no graphical user interface.

The framework installer can exist downloaded from the official website.

You can too use a packet managing director, for instance, Chocolatey for Windows.

While Gauge has a module structure, the minimum fix of plugins can be installed for the desired programming language only a few commands. The gauge-python plugin and the getgauge module for Python can be installed with the following two commands:

Now, a new template can be created for a project. To exercise this, we initialize a new project in the language of our choice in an empty directory.

          
            
D:\projection> gauge init python            
          
        

After the template for the Estimate project is created, we tin monitor exam functioning. The project includes a sample specification written in Python, the language we've selected. If everything has installed successfully, nosotros can run the tests with this command:

          
            
> gauge run D:\projection\specs            
          
        

A plugin for HTML reports will be installed automatically before the project starts. In that location are also plugins for getting reports in JSON and XML formats that can be installed manually.

For storing testing objects (for example, DLLs), we'll create an Artifacts binder in the new project.

Main components of Gauge tests

Automated Estimate tests include three main components:

1. A specification represents a business test example that normally describes a particular software feature. Each specification includes at least one scenario.
2. A scenario is a single catamenia in a specification. Each scenario consists of one or more steps.
3. Steps are executable components of specifications.

Implementing scenarios in Gauge

Gauge uses Markdown as the markup language for writing scenarios. Thanks to UTF-8 back up, scenarios can be written in any natural language. We used English for our project.

Gauge doesn't utilise the Given-When-Then approach to writing acceptance tests, so all steps have equal ranking. Steps can exist divided into logical groups with the utilise of markers. Information technology's possible to combine scenarios with any common feature in a unmarried file, chosen a specification. The size of scenarios and specifications is unlimited.

Withal, specifications can merely be parsed if:

• the specification name is written as the kickoff-level header;
• the name of each scenario is written every bit the second-level header;
• scenario steps are written as markdown unordered list items (bulleted points);
• variables are written in quotes.

For example:

          
            
# Specification Heading   ## Vowel counts in single word   * The word "gauge" has "3" vowels.
          
        

This is the minimum ready of components for getting an executable scenario. However, you can besides utilize other components to expand the test capabilities:

• Tags are written after the prefix Tags: under the specification or scenario heading.
• Context steps are executed before whatever scenario in a specification. They are written before the heading of the kickoff scenario.
• Teardown steps are executed after every scenario in a specification. They are written at the end of a specification after three or more consecutive underscores.
• Concepts unite the most ordinarily used logical groups of steps into a single unit, which is defined in a carve up file. Concepts and headers are used in a specification just like other steps.
• Information and variable values can be written as strings or in a table inside scenarios. They also can be taken from a text or csv file.
• Comments are any line that isn't marked as a step or other element. Comments are not executed when implementing a scenario.

Here's a Guess test example using tags, context steps, and teardown steps:

          
            
This is a specification heading:   # Delete projection   This is a listing of specification tags:   Tags: delete, valid   These are context steps:   * Sign up for user "mike" * Log in as "mike"   This is a scenario heading:   ## Delete single project   This is a list of scenario tags:   Tags: delete, unmarried   These are scenario steps:   * Delete the "example" project * Ensure "example" project has been deleted   This is a clarification of the adjacent scenario:   ## Delete multiple projects   Tags: delete, multiple   * Delete all the projects in the listing * Ensure the project list is empty   ____________________ These are the teardown steps:   * Logout user "mike" * Delete user "mike"            
          
        

You may find that these two scenarios don't have matching steps. In practice, information technology'southward all-time to avert using different stride definitions for similar actions. This simplifies scenario support and minimizes the risk of duplicating the same steps.

In this case, steps tin represent concepts that contain sets of other steps. Concepts created from the implemented steps are executed immediately.

Hither's an example of a file with concept descriptions:

          
            
This is a concept heading:   # Delete the <project_name> project   These are concept steps:   * Observe the post-obit projects: <project_name> * Check that the user has admin rights in found projects * Delete all institute projects   This is a description of the side by side concept:   # Delete all the projects in the list   * Find the following projects: "all" * Check that the user has admin rights in found projects * Delete all found projects            
          
        

Scenarios with more abstract steps can be useful for all stakeholders, who tin also edit them. Detailed steps in concepts that are hidden from scenarios volition be bachelor simply for testers and developers.

You tin can notice more than data about how to create specifications in the Guess tutorial.

Implementing steps in Gauge

For our project, we wrote our steps in Python. A unproblematic step implementation looks similar this:

          
            
from getgauge.python import pace   @step("The give-and-take <word> has <number> vowels.") def assert_no_of_vowels_in(give-and-take, number):    assert str(number) == str(number_of_vowels(give-and-take))            
          
        

The @step decorator accepts the line with a step sample. After this, the bodily implementation of this step is provided.

A step that has already been implemented tin can exist added to any other project scenarios.

For a ameliorate experience, at that place are plugins that support several IDEs (Visual Studio, IntelliJ IDEA, Visual Studio Code).

Approximate tests and reports

For test execution, Gauge offers to use the path to either a specs directory, whatsoever subdirectory, a particular specification, or a scenario. In each of these cases, it'due south possible to specify several paths.

          
            
> judge run \specs\some_specs            
          
        

          
            
> gauge run \specs\some_specs\authentication.spec            
          
        

          
            
> gauge run \specs\some_specs\hello.spec \specs\another_specs\globe.spec            
          
        

A single scenario for a specification tin can exist executed by specifying the line number in the bridge of that scenario in the specification.

          
            
> approximate run \specs\some_spec\authentication.spec:28            
          
        

You tin also employ scenario and specification tags to execute certain groups of tests. For example, you can include or exclude scenarios with specific tags. Circuitous combinations of tags are also supported in Estimate.

Hither'south an instance of a control that executes just scripts with certain tags:

          
            
> guess run --tags "tag1, tag2"  D:\project\specs            
          
        

With this command, all scripts will exist executed except those containing the specified tag:

          
            
> approximate run --tags "!tag1"  D:\project\specs            
          
        

Later running the Judge projection, the panel displays the proper name of the currently executing specification.

In that location'south a separate line for each scenario, and each step passed is marked with the corresponding letter: P (passed) or F (failed). For failed steps, Gauge provides error names.

Earlier test execution, Judge checks the steps and their implementation. If there are issues such as absenteeism or duplication of whatsoever step, the respective error will be displayed.

After finishing all tests, Gauge creates a report in the Report folder. Here'south a sample HTML written report:

The report provides statistics on completed tests and detailed information on failed steps. If necessary, a screenshot made at the moment of failure can be fastened to each failed step.

Our experience using Judge

Using Gauge, our acceptance tests have become more transparent and require less maintenance. At present, automated acceptance tests can be written past our testers and not just by our developers, every bit was the case with our previous approach. Test results are like shooting fish in a barrel to empathise and are regularly reviewed by both testers and developers. And managers can too get the results of automatic tests when necessary.

If our automated acceptance tests find whatsoever serious defects in the release version of a product, the framework automatically notifies the client about information technology. The results of full tests are automatically recorded in TestRail.

The Estimate framework offers additional benefits for both testers and developers:

However, we also encountered some pitfalls when we used Estimate for test automation:

• Occasional unstable test performance and compatibility issues
• Bugs aren't fixed immediately
• Detailed scenarios increment test execution time

When working with the Gauge framework, nosotros noticed that test performance may be unstable. An update of ane of the plugins or to the Python interpreter sometimes leads to compatibility issues. Unfortunately, the framework developers don't ever prepare these problems apace, and so our specialists have to make additional efforts to cope with them.

We also faced difficulties because of our style of writing scenarios. Even though Estimate doesn't have syntax limitations compared to other frameworks, our scenarios appeared to exist too specific. This isn't an outcome for a small production, only as it grows, information technology becomes hard to work with a bang-up number of detailed steps. More than importantly, while giving us little to no benefit, such detailed tests significantly increment execution time.

Conscientious planning of acceptance testing processes might assist avoid some of these difficulties when using the Gauge framework. Particularly, you need to accept into account the possible growth of your production and develop a suitable and succinct architecture for test scenarios and their implementation.

Dealing with the bugs of the Gauge framework is a bigger claiming. But since the framework has simply recently come out of beta, these issues are quite understandable. We tin can only hope that the framework will keep to exist developed and its bugs stock-still.

Decision

Using the Gauge framework for automated acceptance testing, we successfully achieved transparency in our tests and test results. Now, our automated test scenarios can comprise detailed business requirements and product specifications. They're easy to review and edit and can exist understood non simply by our developers simply also by our testers and managers. Thanks to automated Judge tests, we tin can spend less time and endeavor on manual testing without reducing product quality.

Approximate has immune our developers to find and ready almost all blocking defects. Though our testers still have to deal with critical defects, their number is much fewer than in projects where automated Gauge tests aren't applied.

After this experience using the Approximate framework, we're planning to go on implementing the beliefs-driven development arroyo in our upcoming projects.

Apriorit has a team of QA specialists whose expertise is confirmed with ISTQB certification. Contact us if you need to ensure the quality of your software.

References

https://docs.gauge.org/latest/index.html

https://judge-python.readthedocs.io/en/latest/alphabetize.html

https://blog.getgauge.io/why-we-congenital-approximate-6e31bb4848cd

Source: https://www.apriorit.com/dev-blog/595-automated-acceptance-testing-gauge-framework

Posted by: foleycalipand87.blogspot.com

Share this post