Cypress as of today (v 9.2.1) doesn’t provide any support for tagging tests. But fortunately due to the large open-source community that cypress has we have a few ways using which we can achieve tag tests. Let’s further deep dive into this:
Method 1: Using cypress-select-tests (This is DEPRECATED)
Step 1: Install the cypress-select-tests plugin using the command:
1 | npm install --save-dev cypress-select-tests |
Step 2: Under cypress/plugins/index.js write:
1 2 3 4 | const selectTestsWithGrep = require('cypress-select-tests/grep') module.exports = (on, config) => { on('file:preprocessor', selectTestsWithGrep(config)) } |
Step 3: The next step is to Tag our Tests. I will be using two types of tags for this article: ‘Smoke’ and ‘E2E’. I would be filtering the tests on the basis of the Test Title, but this plugin provides other ways to filter as well:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | ## run tests with "works" in their full titles $ npx cypress open --env grep=works ## runs only specs with "foo" in their filename $ npx cypress run --env fgrep=foo ## runs only tests with "works" from specs with "foo" $ npx cypress run --env fgrep=foo,grep=works ## runs tests with "feature A" in the title $ npx cypress run --env grep='feature A' ## runs only specs NOT with "foo" in their filename $ npx cypress run --env fgrep=foo,invert=true ## runs tests NOT with "feature A" in the title $ npx cypress run --env grep='feature A',invert=true |
In TC01.spec.js I am adding the keyword ‘Smoke’ to the title of the test:
In TC02_fixtures.spec.js I am adding the keyword ‘E2E’ to the title of the test:
In TC03_writeReadFile.spec.js I am adding both keywords ‘E2E’ and ‘Smoke’ to the title of the test:
Now, If we want to execute only tests with the ‘Smoke’ tag we will use the command:
1 | npx cypress run --env grep="Smoke" |
After Execution, you could see that Tests 1 and 3 that have the Keyword ‘Smoke’ in their title are executed, and the rest all are in pending (skipped/Not Executed) state.
Similarly, If we want to execute only tests with the ‘E2E’ tag we will use the command:
1 | npx cypress run --env grep="E2E" |
After Execution, you could see that Tests 2 and 3 that have the Keyword ‘E2E’ in their title are executed, and the rest all are in pending (skipped/Not Executed) state.
Now it seems a bit confusing that the tests that are not executed are marked under ‘Pending’. As it turns out, this is a known behavior of Mocha Test reporter and you can learn more about it from this Github Thread.
Method 2: Using a Custom Function
There is one more way you can apply tags to your tests as suggested by Marie Drake in her blog and in my opinion, this is a much robust way of doing it.
Step 1: Create a file called filterTests.js under the support folder and add:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | const TestFilters = (givenTags, runTest) => { if (Cypress.env('tags')) { const tags = Cypress.env('tags').split(',') const isFound = givenTags.some((givenTag) => tags.includes(givenTag)) if (isFound) { runTest() } } else { runTest() } }; export default TestFilters |
What this does, in a nutshell, is it filters the tests based on tags that we provide from the CLI. It searches for the tags in the js files where we have added the TestFilters function.
Step 2: For this to work you have to import the filterTests in all test files and add the TestFilters function.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | import TestFilters from '../support/filterTests.js' TestFilters(['smoke', 'regression'], () => { describe('Some Test Suite', () => { it('Some Test Case', () => { /..../ }) it('Some Test Case 2', () => { /..../ }) }) }) |
Now, if you don’t want to add any tags in that case just add an empty array:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | import TestFilters from '../support/filterTests.js' TestFilters([], () => { describe('Some Test Suite', () => { it('Some Test Case', () => { /..../ }) it('Some Test Case 2', () => { /..../ }) }) }) |
For our case, we have added the tags – smoke, regression for the following test suites. And for the rest, we have passed an empty array.
1. To execute tests with only one tag(regression) we will use the command:
1 | Cypress_tags=regression npx cypress run |
As per our implementation only test suites, 5 and 8 should be executed and the rest of them shouldn’t be.
2. To execute tests with two tags(regression, smoke) we will use the command:
1 | Cypress_tags=regression,smoke npx cypress run |
As per our implementation only test suites, 5,8, and 19 should be executed and the rest of them shouldn’t be.
3. To execute all tests irrespective of any tags we will use the command:
1 | npx cypress run |
Method 3: Using cypress-grep
Step 1: Install the cypress-grep plugin using the command:
1 | npm i -D cypress-grep |
Step 2: Under cypress/plugins/index.js write:
1 2 3 4 | module.exports = (on, config) => { require('cypress-grep/src/plugin')(config) return config } |
Step 3: Under cypress/support/index.js write:
1 | require('cypress-grep')() |
Step 4: Now, we can directly run the tests by grepping texts from the title. Let’s execute the test with the text “username and password in URL” in the title.
So we will use the command:
1 | npx cypress run --env grep="username and password in URL" |
Since the text that we are grepping is unique to one test, so only that test will execute and all other tests will be marked as ‘Pending’. You can read more about why the tests that are not executed are marked as pending from the cypress docs.
Step 5: We can also add tags to test suite or test case level and add tags and then execute the test cases. Let’s add the tags ‘smoke’ and ‘regression’ to test cases and test suites and execute them based on the added tags.
TC_25_filterFindWithin.spec.js Added ‘regression’ tag in one of the test cases
TC_26_jqueryCommands.spec.js Added ‘smoke’ tag in one of the test cases
TC_27_cypressRecursion.spec.js Added ‘smoke’ tag in the test suite
Now if we want to run the tests with ‘regression’ tag, we will use the command:
1 | npx cypress run --env grepTags="regression" |
As we can see above only the test having the ‘regression’ tag from TC_25_filterFindWithin.spec.js was executed, the rest of the tests were not executed and marked as ‘pending’.
Now, similarly if we want to run the tests with ‘smoke’ tag, we will use the command:
1 | npx cypress run --env grepTags="smoke" |
And, if we want to execute both ‘smoke’ and ‘regression’ tags together we will use the command:
1 | npx cypress run --env grepTags="smoke regression" |
As we can see above only test suites TC 25,26 and 27 were executed rest all were marked as pending.
Similarly, there are other ways also you can play around with tags and execute the test cases. I would recommend you to go through the readme page for the cypress-grep plugin.
Do check out 🙂
Github: https://github.com/alapanme/Cypress-Automation
All Cypress Articles: https://testersdock.com/cypress-tutorial/
Hi,
Thanks for posting this article. As it helped me a lot. However, I am facing an issue that I don’t want to show pending test result on the result table(For not executed test cases). Please suggest me how to handle this situation.
Thanks,
Khushal Gupta
I couldn’t find anything using which you can achieve that – https://docs.cypress.io/guides/guides/command-line.html
Hi, I followed the same steps but even tests without the smoke tag got executed. Is this due to the newer version of cypress (6.XX)
This can be confusing but this is a default Mocha behavior. You would get the impression that the tests are being executed, but it is just skipping the tests and only executing the tests that match the criteria for grep. You actually see it in the summary report that cypress generates (refer to the last screenshot on the article) at the end of the test execution. All executed tests will have entries under the columns Passing and Failing. All skipped or Non-executed tests will have entries under Pending column.
Also, I executed the above using cypress 6.4.0 and it is working as expected for me.
That worked for me, Thank you for the quick response. Just one more doubt,
if we want to run it in a CI tool like gitlab, how do we provide the command? Currently what I do is provide the “test” in package.json and add the command (“scripts”: “npx cypress run”) in the script section. And I run the command npm run test(which will run all pretest, scripts and posttest commands) . But I need to execute only smoke tags in one environment and entire suite in another. Could you please help me understand how do we achieve this in CI?
Got the CI part too! Thanks!
That’s Great! Keep up the good work 👍
Hi, Is it possible to integrate testlink using cypress?
I haven’t worked with cypress and testlink, so I won’t be able to say anything on this.
Hello everyone, I am seeing some issues with this implementation, for example, I have in each test before(), in that before I am doing some API calls and these calls are adding the data. Somehow, these before() and after() are not skipped at all. Any idea?
Where are you adding the grep texts on the describe() blocks or it() blocks? Also, is the before() and after() block declared globally or declared separately for each test (spec files) ?
I am adding the grep texts at the top of the file, main describe() block.
Yes, you are right, the before() and after() are declared globally, for all tests.
Hey, I am on Cypress 6.7.1 and I tried using this package. However, when I tried, it was throwing the following error:
Oops…we found an error preparing this test file:
cypress\support\index.js
The error was:
Error: Webpack Compilation Error
./node_modules/coffeescript/bin/coffee 1:0
Module parse failed: Unexpected character ‘#’ (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
> #!/usr/bin/env node
|
| var path = require(‘path’);
@ ./node_modules/coffeescript/lib/coffee-script/register.js 52:13-48
@ ./node_modules/coffeescript/lib/coffee-script/coffee-script.js
@ ./node_modules/coffeeify/index.js
@ ./node_modules/@cypress/browserify-preprocessor/index.js
@ ./node_modules/cypress-select-tests/src/index.js
@ ./node_modules/cypress-select-tests/grep.js
@ ./cypress/support/index.js
Could you please once?
Hi, Rajib from the error message, it is clear that the error is originating from the cypress\support\index.js, I would start looking from there. Also if you google ‘Webpack Compilation Error Cypress’ there are a lot of solutions available. In case none of this works, I will suggest you to post this on Stackoverflow.
Hi Rajib I’m having the same issue. Did you manage to resolve it?
Hi Team,
Thanks for the wonderful blog
I’ve tried to follow the same mentioned above and seems all good for me,
but i’ve a situation below, can you please help me how to do that
let’s say we have 3 specs, 2 tagged with FULL and one tagged with SMOKE
if we run the command to execute the SMOKE tag specs,
cypress is triggering all the 3 specs and based on the order when it reached the spec which assigned SMOKE tag it is executing it and for other two tagged with FULL it is just skipping
so i just want to run only the specs which are matched with the given tag and don’t want to trigger the specs that are not even mentioned… so, that I can get the test results sooner
is there a way to do that?
With Method 2 you can do that. It will only execute the tests whose Tag is mentioned.
I did according to the instructions of METHOD 2
I wanted – Cypress_tags = regression npx cypress run
And receives an error –
‘Cypress_tags’ is not recognized as an internal or external command,
operable program or batch file.
Hi Ricky, The Cypress_tags works perfectly for me with the latest as well as old cypress versions. Please try running the command – npx cypress open first and then this. Also, if this still doesn’t work I would suggest you create a new thread at Stackoverflow and provide more info for debugging.
Cypress_tags is not recognizable , any option how we can solve this. I want to filter tests with method 2 I am already using grep method as grep tags mark untagged tests as pending. This will reduce the pass percentage of passed test cases.
Are you running your tests on windows? Then in that case use ‘SET Cypress_tags=smoke’
Hi Alapan, thanks for sharing this, I’m running the command but the tests are not filtered. It seems like all the tests are picked up during the execution. Anything I can check?
Thanks
Hi Alapan,
It is picking up all the tests in the “integration” folder method-2. i’m using the cypress newer version 8.5.0
Hi Alapan,
I am trying to implement Method 2 and I am running the test on widows. I am gettign the same error the people above are mentioning. Please advise how to run the test in windown environment.
PS D:\GitHub\webapp> set CYPRESS_TEST_TAGS=regression
PS D:\GitHub\webapp> CYPRESS_TEST_TAGS=regression npx cypress run
CYPRESS_TEST_TAGS=regression : The term ‘CYPRESS_TEST_TAGS=regression’ is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the
name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ CYPRESS_TEST_TAGS=regression npx cypress run
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ FullyQualifiedErrorId : CommandNotFoundException
Hi Marie,
I figured it out. By default the terminal in VS Code is powershell, so this is the way to set it up and run it:
> $env:CYPRESS_TEST_TAGS=’regression’
> npx cypress run
It seems like thou, if a test does not have a tag, then it will be run regardless the value of CYPRESS_TEST_TAGS. Is there a way to prevent that?
Hi, i am running my test on window and using the second method.
i get this error message “The term ‘cypress_tags=fr’ is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again”. Why is this happening?
Cypress.env(“tags”), what is the value of the tags?
I am using Cyoress with cucumber BDD . How can I achieve tag base run with mentioned combination without test getting skipped in the report.
Great post Alapan, simplified