If the complete code, incorporating validation and data flows, is available and provided to the LLM with a suitable prompt, success is almost certain. The LLM can generate Page Objects and a comprehensive set of positive and negative tests. From my experiments, Claude 3.5 has shown excellent capability in creating quality Playwright artifacts, but the results rely heavily on the context and the clarity of the prompt (e.g., specifying locator preferences). The challenge becomes apparent with component-based frameworks like React and Angular, where validation logic and data flows might be scattered across components or APIs. Without clear business rules or fully accessible code, it often feels like a game of peek-a-boo, leaving QA to ask the eternal question: “Show me the requirements!”
Cool, But what happend when you have more complex issue With more files Like passing context etc - I have problem too keep browser connections When one test finishes and another should be run the same session, Any suggestions?
@@ChecklyHQ@ChecklyHQ, we are very grateful that you and your team keep releasing this quality of content! It is really useful, and I've learned a lot from your videos. Please keep them coming! 🎭
First mistake was to use gpt4, instead of o1. It would probably have been able to do more with less. Anyways, you do not use chatgpt for this, and you mention, you need custom built in tools that can on their own, in an iterative fashion, fetch the data they need to keep writing the test.
Thanks for the tip. I just played with o1 a little bit and I couldn't find a real improvement. Without enough context, they both fail to generate good code, and with enough context, roles and rules, they both succeed. > you do not use chatgpt for this I 100% agree with you; I took this example as the kick-off, because I've literally seen these instructions in some docs: "To get started with Playwright, head over to ChatGPT and tell it what it should do"... And I'm not kidding. 🫣
Nice video Stefan! you should try using the new claude computer use API, it will opens up a lot more possibilities, for example I think we can omit the html/css/js context to the prompt
If the complete code, incorporating validation and data flows, is available and provided to the LLM with a suitable prompt, success is almost certain. The LLM can generate Page Objects and a comprehensive set of positive and negative tests. From my experiments, Claude 3.5 has shown excellent capability in creating quality Playwright artifacts, but the results rely heavily on the context and the clarity of the prompt (e.g., specifying locator preferences).
The challenge becomes apparent with component-based frameworks like React and Angular, where validation logic and data flows might be scattered across components or APIs. Without clear business rules or fully accessible code, it often feels like a game of peek-a-boo, leaving QA to ask the eternal question: “Show me the requirements!”
Yeah, that's right and thanks for the comment. I would be very keen on seeing a prompt that works well for you.
Honestly I loved the content.
Thank you! Glad it was valuable! 🦝
Very Cool! the best content for Playwright!
Thank you! Great to see that you're still following along. 💙
the best content for playwright 👌
Very helpful. Greetings from Brazil
Cool,
But what happend when you have more complex issue
With more files
Like passing context etc
-
I have problem too keep browser connections
When one test finishes and another should be run the same session,
Any suggestions?
have you tried with snapshots feature which is released today ??
I haven't checked the new feature yet, but I might play around with it next week. :)
@ thanks for your reply
great content, perhaps you could showcase how, codegen generated code can then be updated to use POM and omit clicks() etc using CursorAI
Yes! This is only part one of a planned series. Cursor and Copilot are next. And I'm very grateful for every suggestion. Thank you! 🦝
@@ChecklyHQ@ChecklyHQ, we are very grateful that you and your team keep releasing this quality of content! It is really useful, and I've learned a lot from your videos. Please keep them coming! 🎭
Thanks for the kind words! 💙 Happy the videos are valuable.
Very impressive, Thank you for your efforts!
My pleasure! 🦝
Do you know the command for codegen in case of using .env files?
@@og4789 I'm not sure I understand. How do .env files relate to codegen?
So baseURL from env variables I have to type always manually into npx playwright codegen command right?
Yes I think so. :)
Thanks for your video.
What about searchgpt, that can access the web. Or maybe perplexity, that also can go to the web sources.
I haven't used any of these but I'll put them on the list for possible future videos. Thank you! 🦝
Tabnine ai extension can help as well
Thank you! I'll check it out!
First mistake was to use gpt4, instead of o1. It would probably have been able to do more with less.
Anyways, you do not use chatgpt for this, and you mention, you need custom built in tools that can on their own, in an iterative fashion, fetch the data they need to keep writing the test.
Thanks for the tip. I just played with o1 a little bit and I couldn't find a real improvement. Without enough context, they both fail to generate good code, and with enough context, roles and rules, they both succeed.
> you do not use chatgpt for this
I 100% agree with you; I took this example as the kick-off, because I've literally seen these instructions in some docs: "To get started with Playwright, head over to ChatGPT and tell it what it should do"... And I'm not kidding. 🫣
Nice video Stefan!
you should try using the new claude computer use API, it will opens up a lot more possibilities, for example I think we can omit the html/css/js context to the prompt
Great idea! I'll put it on the list because I'm planning a series and this video is only part 1. 🫣 Thank you!