Great Video bro! I think this video answered all my questions that I have been looking for. I really appreciate all the hard work you put into the videos that you create.
Great video. I had a question though. Why would we integrate e2e tests with the build pipeline and not release pipeline instead. (post relealse mostly to check if code deployde to production is working all fine or not)
@@letcode Thank you for the prompt reply. But the intention of e2e tests is to verify regression in code getting deployed right. With testing in build pipeline, we won't achieve it. It will just verify if the code already deployed in production is working all good or not.
Depending on the application, for instance in my application testers should not create any data. So we do only the smoke in the product and our e2e run against the dev and staging.
I noticed if I used Other Git instead of Github I don't see the option to publish html report. As maximum I am able to attach the index.html report to an artifact which is published within the pipeline execution
hi! I have hundreeds of scripts classified by category with playwright, is there any way I can just run a specific category on the azure pipeline insetad of running everthing on each build?
Hi Koushik, html reports is not displaying under Published HTML tab as shown in the tab. JmeterReportsPath parameter value need to be passed for publishhtmlreport??. I think JmeterReportsPath parameter available in the latest Azure Devops, it is showing in your video
Hey Koushik, thanks for the video. New to playwright reporting - Is it possible to drop the test report HTML file into the artifacts section of Azure programmatically via yaml code?
Hi Koushik , am still struggling to get this to work? all my test run locally but doesnt execute on the pipeline even though the pipeline run and its have no errors? pls help
Hi koushik,I already have the pipeline ready and when I click on edit pipeline…I can see YAML,variables ,triggers,history but I don’t see the tasks tab to add the HTML reporter. Why is it missing. Do I need to enable it somewhere or should I need to get sort of permission from my manager to get it displayed ?
Hi Koushik, how to configure Testcase id in Pipeline level i.e. If I have mention testcase id or testsuite in pipeline build, those tcs are should be run..?? In this video Running the testcase from config file.
Hi Koushik - I have one little question. When I run selenium tests with Specflow, I see a tab "Tests" in the Pipeline and there is a lsit with all the tests and also a history. Is this also possible with Playwright?
Hi Koushik and All, Hope you doing well, In this video you clearly explained how to create CICD for playwright by installing dependencies. In my case I need to execute the power shell script that is: pwsh bin/Debug/net6.0/playwright.ps1 install when I tried running over the azure pipeline im getting some error which is: The argument 'bin/Debug/net6.0/playwright.ps1' is not recognized as the name of a script file. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. is there anything will you suggest how to run this command over the azure pipeline
Hi Koushik, I am able generate html reports Locally, When comes to Pipeline Getting Empty screen, Not able to view Html reports Under PublishhtmlReport Tab and Getting Description for Agent job command window like 'This task can be used to publish html reports to azdo. Currently Jmeter HTML reports are being transformed to be consumed into Azdo directly'
@@letcode Sorry Koushik, I have used Publishhtmreport task to Agent Job and I have not used any plugin. I have observed that there is no Plugin in this video. Can you please give html plugin name
Hi Koushik, When I try to add the publish HTML report in the Azure pipeline and trigger an execution. I'm unable to see the report in the HTML Viewer tab. I'm able to see the tab created but reports are not getting displayed in the tab. Do you have any updates on this?
Hi Koushik, thank you for the video, I want to integrate Maven+Java+Play Wright+Allure Report +Azure combination, but Allure Report is showing error in the Devops, can you help me in this? Allure: Target directory C:\allure-results for the report is already in use, add a '--clean' option to overwrite
Great infromation. how to download the external npm package with .npmrc file when executing the npm install this npmrc file located near to package.json and have the registry link. package name is given in dependencies section,
HI! Thanks for the vide. It is very helpful. I'm getting the following error: npm ERR! enoent ENOENT: no such file or directory, open 'C:\a\1\s\package.json' Could you suggest how to resolve it.
I have kept all my testscript.spec.js files under tests folder of playwright project , also in playwright.config.js file mentioned testDir: './tests',: But still I am facing below error while executing testcases thru npm cutom task : [command]C:\Windows\system32\cmd.exe /D /S /C "C:\agent\_work\_tool ode\18.6.0\x64 pm.cmd run ./tests/*.spec.js" 2022-07-19T14:05:35.3088640Z npm ERR! Missing script: "./tests/*.spec.js" , Can you please suggest a solution?
I am able tu run from pipeline, but one step is fail, the pipeline does not stopr just after the run timeout run . so the test execution step is fail Terminate batch job (Y/N)? ##[error]The operation was canceled. Finishing: CmdLine , but the pipeline is continue
hi Koushik it ismy yml: name: ci variables: VERSION: 7.2 timezone: Europe/Budapest
pool: vmImage: 'windows-latest' workingDir: '$(service_path)' schedules: - cron: "0 3 * * Mon-Fri" displayName: M-F 3:00 AM (UTC) daily build branches: include: - main always: true jobs: - job: AutomatedTests timeoutInMinutes: 30 # how long to run the job before automatically cancelling cancelTimeoutInMinutes: 33 # how much time to give 'run always even if cancelled tasks' before killing them steps: - task: NodeTool@0 inputs: versionSpec: '14.x' displayName: 'Install Node.js' - script: which node displayName: "Install framework" - script: npm i && npx playwright install #- script: npx playwright install --with-deps displayName: "Execute test" condition: succeeded() - script: npm run smoke - task: PublishHtmlReport@1 condition: succeededOrFailed() inputs: reportDir: 'playwright-report/index.html' - task: CopyFiles@1 condition: succeededOrFailed() # Run task even if previous ones fail inputs: #leave it empty, the copying is done from the root folder #sourceFolder: '$(Build.SourcesDirectory)/_daniavander_hydro-test/playwright-report' contents: | playwright-report/** screenshot/** test-results/** targetFolder: '$(Build.ArtifactStagingDirectory)' - task: PublishBuildArtifacts@1 condition: succeededOrFailed() displayName: 'Publish Artifact: drop' inputs: PathtoPublish: '$(Build.ArtifactStagingDirectory)' executed the tests: 4 failed 6 passed (9m) Serving HTML report at 127.0.0.1:9323. Press Ctrl+C to quit. Terminate batch job (Y/N)? ##[error]The operation was canceled. Finishing: CmdLine and the steps after cmdline PublishHtmlReport, CopyFiles, PublishBuildArtifacts do not execute, but i do not why
You need to make sure the environment variable CI is equal to true. If Playwright sees CI=true it will not hang after the test results. Here is the command with a Windows container. It's a bit different with Linux. - task: CmdLine@2 inputs: script: 'set CI=true && echo %CI% && npx playwright test'
I totally follow your steps, but do not start execute test " ; HOME = C:\Users\VssAdministrator ; Run `npm config ls -l` to show all defaults. C:\Windows\system32\cmd.exe /D /S /C "C:\hostedtoolcache\windows ode\16.16.0\x64 pm.cmd run smoke" ##[error]The operation was canceled. Finishing: Execute test "
please help error on installing playwright npm ERR! code ENOENT npm ERR! syscall open npm ERR! path D:\a\1\s/package.json npm ERR! errno -4058 npm ERR! enoent ENOENT: no such file or directory, open 'D:\a\1\s\package.json' npm ERR! enoent This is related to npm not being able to find a file. npm ERR! enoent npm ERR! A complete log of this run can be found in: npm ERR! C: pm\cache\_logs\2023-07-21T06_13_12_175Z-debug-0.log ##[error]Cmd.exe exited with code '-4058'. Finishing: Install Playwright test
Thanks for sharing your tips. Tried running Playwright with Azure Pipelines but most of the tests fail with: "page.goto: net::ERR_CONNECTION_RESET". Everything works fine locally. Also you run "run test" -> have you scripted that, if yes what commands are behind the script?
Great Video bro! I think this video answered all my questions that I have been looking for. I really appreciate all the hard work you put into the videos that you create.
Thanks bro :)
this is exactly what I have been looking for for a long time, many tnx :)
Welcome 😊
Thanks Koushik, I was exactly looking for this information
Welcome 😀
Super helpful, Thankyou.
Welcome 🤗
even when i request you for this video , i fixed in 2 days but great work man
Great 👍
Great work for this playwright playlist!
Thanks buddy ☺️
Awesome was waiting for this
Cool 😎
Great video. I had a question though. Why would we integrate e2e tests with the build pipeline and not release pipeline instead. (post relealse mostly to check if code deployde to production is working all fine or not)
Thanks,
You can do it in the release pipeline as well.
But no need to run every tests, just the smoke in the production will make sense.
@@letcode Thank you for the prompt reply.
But the intention of e2e tests is to verify regression in code getting deployed right. With testing in build pipeline, we won't achieve it. It will just verify if the code already deployed in production is working all good or not.
Depending on the application, for instance in my application testers should not create any data.
So we do only the smoke in the product and our e2e run against the dev and staging.
Super helpful thank you
Glad you like it ☺️
I noticed if I used Other Git instead of Github I don't see the option to publish html report. As maximum I am able to attach the index.html report to an artifact which is published within the pipeline execution
I don't think that is because of git.
Because the html report is standalone.
@@letcode i guess is related i am using azure dev ops git instead of gitbub. Not actual git itself but git form azure dev ops
hi! I have hundreeds of scripts classified by category with playwright, is there any way I can just run a specific category on the azure pipeline insetad of running everthing on each build?
Did u try tags from the command line argument?
Hi Koushik, html reports is not displaying under Published HTML tab as shown in the tab.
JmeterReportsPath parameter value need to be passed for publishhtmlreport??. I think JmeterReportsPath parameter available in the latest Azure Devops, it is showing in your video
U have to add that html report plugin.
@@letcode In Agent machine???
@@sivareddychepuri9260 yeah, from azure project settings
The best Koushik.
Thanks you Mao ☺️
Hi ji not moving on another task in our script is passed but not moved ...
Please give me any suggestions bro. Because out the terminal.
Hey Koushik, thanks for the video. New to playwright reporting - Is it possible to drop the test report HTML file into the artifacts section of Azure programmatically via yaml code?
Hi, yes i guess but i haven't tried.
I did on circle ci.
Hi koushik any setting requird in package.json as for me it is not working with no error
Hi Koushik , am still struggling to get this to work? all my test run locally but doesnt execute on the pipeline even though the pipeline run and its have no errors? pls help
Ok, I'll try to connect with u this weekend.
Hi Koushik, How we can do this pipeline setup for playwright java ? we have repos in azure devops itself. How can we get JSON or Config file ?
You have to configure from the maven.xml.
@@letcodeCan you give any reference koushik
@@letcode Thanks koushik. I have implemented using maven POM.xml.
Hi koushik,I already have the pipeline ready and when I click on edit pipeline…I can see YAML,variables ,triggers,history but I don’t see the tasks tab to add the HTML reporter. Why is it missing. Do I need to enable it somewhere or should I need to get sort of permission from my manager to get it displayed ?
I am not sure about it. I guess by default it should come.
Hi My test still doesn't execute, it passes but doesnt show the test what could be possiblt wrong
is it running in the local?
@@letcode Yes it does run local but when i have pushed my test to the branch and run my pipeline it runs and not sure the test.
It will execute but the browser won't be visible.
Did u get any console message ?
@@letcode Yes i wasnt excepting a browser to open. but to see the test cases like you have , i have no error message.
Hi Koushik, how to run Particular Testcase from Pipeline not from Config file
We need to add Variable??
Yes, from the variable we can achieve it.
Hi Koushik, how to configure Testcase id in Pipeline level i.e. If I have mention testcase id or testsuite in pipeline build, those tcs are should be run..??
In this video Running the testcase from config file.
Pipiline paramerts concept has to be used. I will make a video.
Hi Koushik - I have one little question. When I run selenium tests with Specflow, I see a tab "Tests" in the Pipeline and there is a lsit with all the tests and also a history. Is this also possible with Playwright?
Hi, yes it is similar only.
@@letcode I have test it but no tab with "Tests" 😞
Hi Koushik and All, Hope you doing well, In this video you clearly explained how to create CICD for playwright by installing dependencies. In my case I need to execute the power shell script that is: pwsh bin/Debug/net6.0/playwright.ps1 install when I tried running over the azure pipeline im getting some error which is: The argument 'bin/Debug/net6.0/playwright.ps1' is not recognized as the name of a script file. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. is there anything will you suggest how to run this command over the azure pipeline
Hi.
Not sure how the .net files get executed..
can you help out how can we not push passwords on github and use some CI enveriomental variables to login on url
For that we can use the environment variables in git and azure as well.
I will create a video.
Is it possible to provide support for updating test results(each test step with results) in Azure DevOps Test Plans ??
Not sure about it
Hi Koushik,
I am able generate html reports Locally,
When comes to Pipeline Getting Empty screen, Not able to view Html reports Under PublishhtmlReport Tab and
Getting Description for Agent job command window like 'This task can be used to publish html reports to azdo. Currently Jmeter HTML reports are being transformed to be consumed into Azdo directly'
Which plugin u used to publish html?
@@letcode I have not used any plugin. Please suggest plugin name and How to install
@@supriyareddy2998 in this video itself i have explained bro
@@letcode Sorry Koushik, I have used Publishhtmreport task to Agent Job and I have not used any plugin.
I have observed that there is no Plugin in this video.
Can you please give html plugin name
@@supriyareddy2998 yeah meant the task only.
Hi Koushik,
When I try to add the publish HTML report in the Azure pipeline and trigger an execution. I'm unable to see the report in the HTML Viewer tab. I'm able to see the tab created but reports are not getting displayed in the tab.
Do you have any updates on this?
Hi, i have received the same comments a few times, I'll check and get back to you this weekend.
@@letcode Thanks for the update.
any updates ?
@user-ux9nz2bz5s no issues from my side
Hi Koushik, Thanks for this video.
How to add Visual Studio Test for Agent Job.??
We dont have dll files for this code??
Dll is only for c# right ?
I'm using a typescript project here so not required.
how can i found this extension "publish html report" i found only "publish html reports"
Both same I guess, i will share the link.
Hi Koushik, thank you for the video, I want to integrate Maven+Java+Play Wright+Allure Report +Azure combination, but Allure Report is showing error in the Devops, can you help me in this? Allure: Target directory C:\allure-results for the report is already in use, add a '--clean' option to overwrite
I haven't used the Allure, so I am not sure about it.
Can you attach screenshot per tests?
I have already covered this. Pls check the playlist.
Can we run on Linux operating system, Any specific configuration required to run on linux operating system?
No changes required, you can choose the os as Linux.
Great infromation. how to download the external npm package with .npmrc file when executing the npm install
this npmrc file located near to package.json and have the registry link.
package name is given in dependencies section,
Thanks, not sure I haven't used that.
HI! Thanks for the vide. It is very helpful. I'm getting the following error: npm ERR! enoent ENOENT: no such file or directory, open 'C:\a\1\s\package.json' Could you suggest how to resolve it.
I guess you have given the wrong path.
I have kept all my testscript.spec.js files under tests folder of playwright project , also in playwright.config.js file mentioned testDir: './tests',: But still I am facing below error while executing testcases thru npm cutom task : [command]C:\Windows\system32\cmd.exe /D /S /C "C:\agent\_work\_tool
ode\18.6.0\x64
pm.cmd run ./tests/*.spec.js"
2022-07-19T14:05:35.3088640Z npm ERR! Missing script: "./tests/*.spec.js" , Can you please suggest a solution?
I use test match, I'll check the test dir option.
I am able tu run from pipeline, but one step is fail, the pipeline does not stopr just after the run timeout run .
so the test execution step is fail
Terminate batch job (Y/N)?
##[error]The operation was canceled.
Finishing: CmdLine
, but the pipeline is continue
In my case if there is any failure it's stops the execution
hi Koushik
it ismy yml:
name: ci
variables:
VERSION: 7.2
timezone: Europe/Budapest
pool:
vmImage: 'windows-latest'
workingDir: '$(service_path)'
schedules:
- cron: "0 3 * * Mon-Fri"
displayName: M-F 3:00 AM (UTC) daily build
branches:
include:
- main
always: true
jobs:
- job: AutomatedTests
timeoutInMinutes: 30 # how long to run the job before automatically cancelling
cancelTimeoutInMinutes: 33 # how much time to give 'run always even if cancelled tasks' before killing them
steps:
- task: NodeTool@0
inputs:
versionSpec: '14.x'
displayName: 'Install Node.js'
- script: which node
displayName: "Install framework"
- script: npm i && npx playwright install
#- script: npx playwright install --with-deps
displayName: "Execute test"
condition: succeeded()
- script: npm run smoke
- task: PublishHtmlReport@1
condition: succeededOrFailed()
inputs:
reportDir: 'playwright-report/index.html'
- task: CopyFiles@1
condition: succeededOrFailed() # Run task even if previous ones fail
inputs:
#leave it empty, the copying is done from the root folder
#sourceFolder: '$(Build.SourcesDirectory)/_daniavander_hydro-test/playwright-report'
contents: |
playwright-report/**
screenshot/**
test-results/**
targetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts@1
condition: succeededOrFailed()
displayName: 'Publish Artifact: drop'
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
executed the tests:
4 failed
6 passed (9m)
Serving HTML report at 127.0.0.1:9323. Press Ctrl+C to quit.
Terminate batch job (Y/N)?
##[error]The operation was canceled.
Finishing: CmdLine
and the steps after cmdline
PublishHtmlReport, CopyFiles, PublishBuildArtifacts do not execute, but i do not why
You need to make sure the environment variable CI is equal to true. If Playwright sees CI=true it will not hang after the test results. Here is the command with a Windows container. It's a bit different with Linux.
- task:
CmdLine@2
inputs:
script: 'set CI=true && echo %CI% && npx playwright test'
@@testihenkilo6911 Thank you bro you have saved my day. :)
Can this be done for playwright with cucumber?
It should as we have used node js
Call the appropriate script tag
Can you please post video for sending mail after execution on local as well as azure pipeline
I'll try
I totally follow your steps, but do not start execute test
"
; HOME = C:\Users\VssAdministrator
; Run `npm config ls -l` to show all defaults.
C:\Windows\system32\cmd.exe /D /S /C "C:\hostedtoolcache\windows
ode\16.16.0\x64
pm.cmd run smoke"
##[error]The operation was canceled.
Finishing: Execute test
"
What are the commands u used?
@@letcode I do not know what is the problem, but when i build the pipeline with the original method (yml file) it is ok
please help
error on installing playwright
npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path D:\a\1\s/package.json
npm ERR! errno -4058
npm ERR! enoent ENOENT: no such file or directory, open 'D:\a\1\s\package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! C:
pm\cache\_logs\2023-07-21T06_13_12_175Z-debug-0.log
##[error]Cmd.exe exited with code '-4058'.
Finishing: Install Playwright test
It's folder related issues.
Create a folder on the desktop then proceed with the installation.
@@letcode just an empty folder?
th-cam.com/video/AdFgipIPAlg/w-d-xo.html
@@letcode thank you so much!
@@letcode same error in azure devops pipeline, installing playwright
Thanks for sharing your tips. Tried running Playwright with Azure Pipelines but most of the tests fail with: "page.goto: net::ERR_CONNECTION_RESET". Everything works fine locally.
Also you run "run test" -> have you scripted that, if yes what commands are behind the script?
Yeah i have used the script tag, you can find that in my git repo.
Connection reset means u sure your application was up and running that time.