Tutorial 11: Use Tags¶

Goal: Understand the usage of tags to organize the testsuite and optimize test runs.

Several test frameworks support a concept of tags to mark a number of tests (py.test markers, TestNG test groups, JUnit Categories, NUnit CategoryAttribute). This provides a simple, flexible and effective mechanism to:

select a number of tests
exclude a number of tests

for a test run. This mechanism is orthogonal to the static test package structure.

Hint

Predefined or often used tags:

Tag	Kind	Description
@wip	predefined	“Work in Process” (under development).
@xfail	predefined	Tests that are (currently) expected to fail.
@not_implemented	user-defined	Marks a test that is not implemented yet.
@slow	user-defined	Mark slow, long-running tests.
@glacier	user-defined	Mark even slower, longer running tests.

Hint

Tag Logic v2: Using cucumber-tag-expressions

Logic Operation	Command Options	Description
select/enable	`--tags=@one`	Only items with this tag.
not	`--tags="not @one"`	Only items without this tag.
logical-or	`--tags="@one or @two"`	If @one or @two is present.
logical-and	`--tags="@one and @two"`	If both @one and @two are present.
match-substring	`--tags="@one.*"`	Matches all tags that start with “one.”

Notes:

The tag name prefix ‘@’ (AT) is optional in tag options
Use --tags-help for a short description of the tag logic.
Use parenthesis to group tag expressions, like: @one and (@two or @three)

See also behave tags documentation for more information on tags.

Write the Feature Test¶

# file:features/tutorial11_tags.feature
@wip
Feature: Using Tags with Features and Scenarios (tutorial11 := tutorial2)

    In order to increase the ninja survival rate,
    As a ninja commander
    I want my ninjas to decide whether to take on an opponent
    based on their skill levels.

    @ninja.any
    Scenario: Weaker opponent
        Given the ninja has a third level black-belt
        When attacked by a samurai
        Then the ninja should engage the opponent

    @ninja.chuck
    Scenario: Stronger opponent
        Given the ninja has a third level black-belt
        When attacked by Chuck Norris
        Then the ninja should run for his life

Run the Feature Test¶

When you run the feature file by excluding the tag @wip, then any feature marked with this tag is skipped as well as all of its scenarios.

$ behave --tags="not wip" ../features/tutorial11_tags.feature
0 features passed, 0 failed, 1 skipped
0 scenarios passed, 0 failed, 2 skipped
0 steps passed, 0 failed, 6 skipped, 0 undefined
Took 0m0.000s

Note

Check the test summary for the skipped count for features and scenarios.

Case: Select-by-tag¶

When you enable the tag @ninja.chuck:

$ behave --tags=ninja.chuck ../features/tutorial11_tags.feature
@wip
Feature: Using Tags with Features and Scenarios (tutorial11 := tutorial2)   # ../features/tutorial11_tags.feature:2
  In order to increase the ninja survival rate,
  As a ninja commander
  I want my ninjas to decide whether to take on an opponent
  based on their skill levels.

  @ninja.chuck
  Scenario: Stronger opponent                    # ../features/tutorial11_tags.feature:16
    Given the ninja has a third level black-belt # ../features/steps/step_tutorial02.py:57
    When attacked by Chuck Norris                # ../features/steps/step_tutorial02.py:65
    Then the ninja should run for his life       # ../features/steps/step_tutorial02.py:69

1 feature passed, 0 failed, 0 skipped
1 scenario passed, 0 failed, 1 skipped
3 steps passed, 0 failed, 3 skipped, 0 undefined
Took 0m0.000s

Note

Now only the second scenario is executed and the first one is skipped.

Case: Exclude-by-tag (Logical-not)¶

When you disable the tag @ninja.chuck:

$ behave --tags="not ninja.chuck" ../features/tutorial11_tags.feature
@wip
Feature: Using Tags with Features and Scenarios (tutorial11 := tutorial2)   # ../features/tutorial11_tags.feature:2
  In order to increase the ninja survival rate,
  As a ninja commander
  I want my ninjas to decide whether to take on an opponent
  based on their skill levels.

  @ninja.any
  Scenario: Weaker opponent                      # ../features/tutorial11_tags.feature:10
    Given the ninja has a third level black-belt # ../features/steps/step_tutorial02.py:57
    When attacked by a samurai                   # ../features/steps/step_tutorial02.py:61
    Then the ninja should engage the opponent    # ../features/steps/step_tutorial02.py:69

1 feature passed, 0 failed, 0 skipped
1 scenario passed, 0 failed, 1 skipped
3 steps passed, 0 failed, 3 skipped, 0 undefined
Took 0m0.000s

Note

Now only the first scenario is executed and the second one is now skipped.

Case: Select Combinations (Logical-or)¶

When you select items with either tag @ninja.any or the tag @ninja.chuck (tag-or):

$ behave --tags="@ninja.any or @ninja.chuck" ../features/tutorial11_tags.feature
@wip
Feature: Using Tags with Features and Scenarios (tutorial11 := tutorial2)   # ../features/tutorial11_tags.feature:2
  In order to increase the ninja survival rate,
  As a ninja commander
  I want my ninjas to decide whether to take on an opponent
  based on their skill levels.

  @ninja.any
  Scenario: Weaker opponent                      # ../features/tutorial11_tags.feature:10
    Given the ninja has a third level black-belt # ../features/steps/step_tutorial02.py:57
    When attacked by a samurai                   # ../features/steps/step_tutorial02.py:61
    Then the ninja should engage the opponent    # ../features/steps/step_tutorial02.py:69

  @ninja.chuck
  Scenario: Stronger opponent                    # ../features/tutorial11_tags.feature:16
    Given the ninja has a third level black-belt # ../features/steps/step_tutorial02.py:57
    When attacked by Chuck Norris                # ../features/steps/step_tutorial02.py:65
    Then the ninja should run for his life       # ../features/steps/step_tutorial02.py:69

1 feature passed, 0 failed, 0 skipped
2 scenarios passed, 0 failed, 0 skipped
6 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.001s

Note

Now both scenarios are executed.

Case: Logical-and¶

When you select items that have the tag @ninja.any and the tag @ninja.chuck (tag-and):

$ behave --tags="@ninja.any and @ninja.chuck" ../features/tutorial11_tags.feature
0 features passed, 0 failed, 1 skipped
0 scenarios passed, 0 failed, 2 skipped
0 steps passed, 0 failed, 6 skipped, 0 undefined
Took 0m0.000s

Note

Now no scenario is executed, all are skipped.

Case: Match Tag Names by using Wildcards¶

When you select items that have a tag that starts with “@ninja.” prefix:

$ behave --tags="@ninja.*" ../features/tutorial11_tags.feature
@wip
Feature: Using Tags with Features and Scenarios (tutorial11 := tutorial2)   # ../features/tutorial11_tags.feature:2
  In order to increase the ninja survival rate,
  As a ninja commander
  I want my ninjas to decide whether to take on an opponent
  based on their skill levels.

  @ninja.any
  Scenario: Weaker opponent                      # ../features/tutorial11_tags.feature:10
    Given the ninja has a third level black-belt # ../features/steps/step_tutorial02.py:57
    When attacked by a samurai                   # ../features/steps/step_tutorial02.py:61
    Then the ninja should engage the opponent    # ../features/steps/step_tutorial02.py:69

  @ninja.chuck
  Scenario: Stronger opponent                    # ../features/tutorial11_tags.feature:16
    Given the ninja has a third level black-belt # ../features/steps/step_tutorial02.py:57
    When attacked by Chuck Norris                # ../features/steps/step_tutorial02.py:65
    Then the ninja should run for his life       # ../features/steps/step_tutorial02.py:69

1 feature passed, 0 failed, 0 skipped
2 scenarios passed, 0 failed, 0 skipped
6 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.000s

Note

Again, both scenarios marked with @ninja.any and @ninja.chuck are executed.