22:10 you can use the other yaml multiline indicator that's perfect for writing a long one-line command. If you use ">", you don't need to escape your line breaks, they will just be collapsed into one line. :)
Two more ideas - a bit connected: Don't create pipeline/release per environment - it in fact differs only on some variables, so if that will be customized enaough you can just have some variables per environment. Was in project which had build/release per environment, so each production release was fun - you was never sure if you updated pipeine with all required step which you already tested in dev/test/uat environments - you could still make a mistake in Prod :-) And second idea which was like unloved child of first issue: since it was in Azure Devops with that nice UI version pipeline we had also a Task Groups per environment, so is not like 'environment variables' should be configured propertly, you should be sure to uses correct Task Group - for exaple for Test environment and there was also one more problem - in case there was new version of that Task Group you had to make sure that higher environments got that updated version - this was madness, but fun ;-) And maybe one 'DO' - since currently I do work on project which use gitlab and that nice yaml file and we build only from main - so no more build per environment - there is shared repo with gitlab jobs and sth which I do love, it had proper versioning - like if there is breaking change then it have higher version and I can use each job from different version, so there will be no problem like mentioned in speech that "you have to build new version to use it" - and in fact, I can use version from dedicated branch, becouse why not. But in general pretty nice speech, there is few ideas which I would like to introduce - especially that part with "one pipeline to rule them all" - initially wanted to get rid of some jobs from PR (or MR how gitlab calls it) or at least move it (linting annoying me xD - it fail fast, but during development I dont really care - just block it until there is MR)
Good point. I don't think I've ever seen that in the wild, but it's definitely a pattern. I'll see if I have time to include that in the future (literally, I've already had to cut things to stay within the allotted time).
The rules of thumb provided are good, but a former coworker had a similar but opposite idea. She worked at some financial institution in the 2000's, and they had a rule: developers were provided the least powerful machines in the company. The idea here was that if it could run on a developer's machine, it could run anywhere. So, instead of trying to keep your CI/CD agents as beefy as your developer machines, perhaps reducing the capability of development machines would make it easier to meet the demand in production. Side benefit here is that you likely end up with leaner runtimes due to developers having to fit within their local machine's constraints.
This is exactly what we do, but with our non-prod environments. If you can make it work on those machines with limited resources, then it will fly in production.
hobbling your devs is a terrible idea. every second waiting for a build to complete is lost productivity . instead, use vms with minimal resources for testing the deployed app.
By developer machines you mean the computers that they are supposed to work? Or development servers? If it is about developer computer then I would laugh loudly and resign the first day that idea is introduced. IDE, docker, some background processes - it all takes the CPU. And you can believe me - waiting 10 minutes for the enterprise solution to be fully loaded and indexed by IDE is something I don't wish anyone
28:00 4-5k lines of Jenkins pipeline code...I wish, lol. The company I work at has a lack of developer expertise on the team of engineers responsible for maintaining pipeline operations. The current line count for looking at the multiple projects is 19,810. Additionally, they don't use a typical bundle process, and instead put everything in the vars/ directory. For those unaware, the vars/ directory is supposed to represent easily referenced things, like a pipeline entrypoint, and they are initialized as globals for the entire pipeline. Instead, we made every supporting action its own entrypoint basically, lol. Half the pipeline code is seemingly composed of hard-coded Shell scripts for interacting with the OpenShift CLI, and making sure that the 3-5 discrete Kubernetes configurations are swapped correctly (because we have, per application, a standard template, a blue template, a green template, a canary template, an alpha template, a beta template...) Absolute madness to me. Also, to add insult to injury, they couldn't be bothered to write pipelines for specific builds, so we have The One Pipeline. And, on top of that, we have The One Build-Agent, which needs build tools for Java, Go, C#, Python, JavaScript, etc. And as if that wasn't already overloaded, we also installed a consumer browser on the agent because Selenium tests needed to share the same host for some dumb reason.
What is the proper alternative to putting everything under vars/ ? Asking since we had quite a bunch of Groovy under vars/ in my previous company -- and if this is actually a bad practice, I'd like to avoid it when I introduce shared libraries to my current place. I know that sharedlibs also support src/, but from what I understand it's not so different approaches...
Test should start containers only once, saves lots of time. Integration tests using bowsers could be speed up. Lot's of unneeded waiting, because 200 years ago things were slow
20:45 YAML is a human-readable data serialization language that is often used for writing configuration files. Depending on whom you ask, YAML stands for yet another markup language or YAML ain't markup language (a recursive acronym), which emphasizes that YAML is for data, not documents
Yeah, the batteries in the mic ran out, so we had to switch to a handheld thing. I dislike those because I never know how close to the mouth I need to hold it :)
02:48 - Anti-pattern#1 - The Ritual
10:18 - Anti-pattern#2 - Hoarding
12:50 - Anti-pattern#3 - The Aesthete
15:03 - Anti-pattern#4 - The One Pipeline
20:24 - Anti-pattern#5 - Gift wrapping
23:54 - Anti-pattern#6 - It's complicated
27:50 - Anti-pattern#6.5 - Entangled
31:31 - Anti-pattern#7 - Groundhog day
40:53 - Anti-pattern#8 - Interference
42:34 - Anti-pattern#9 - Overloaded
22:10 you can use the other yaml multiline indicator that's perfect for writing a long one-line command. If you use ">", you don't need to escape your line breaks, they will just be collapsed into one line. :)
Two more ideas - a bit connected: Don't create pipeline/release per environment - it in fact differs only on some variables, so if that will be customized enaough you can just have some variables per environment. Was in project which had build/release per environment, so each production release was fun - you was never sure if you updated pipeine with all required step which you already tested in dev/test/uat environments - you could still make a mistake in Prod :-) And second idea which was like unloved child of first issue: since it was in Azure Devops with that nice UI version pipeline we had also a Task Groups per environment, so is not like 'environment variables' should be configured propertly, you should be sure to uses correct Task Group - for exaple for Test environment and there was also one more problem - in case there was new version of that Task Group you had to make sure that higher environments got that updated version - this was madness, but fun ;-)
And maybe one 'DO' - since currently I do work on project which use gitlab and that nice yaml file and we build only from main - so no more build per environment - there is shared repo with gitlab jobs and sth which I do love, it had proper versioning - like if there is breaking change then it have higher version and I can use each job from different version, so there will be no problem like mentioned in speech that "you have to build new version to use it" - and in fact, I can use version from dedicated branch, becouse why not.
But in general pretty nice speech, there is few ideas which I would like to introduce - especially that part with "one pipeline to rule them all" - initially wanted to get rid of some jobs from PR (or MR how gitlab calls it) or at least move it (linting annoying me xD - it fail fast, but during development I dont really care - just block it until there is MR)
Good point. I don't think I've ever seen that in the wild, but it's definitely a pattern. I'll see if I have time to include that in the future (literally, I've already had to cut things to stay within the allotted time).
Can I just say.. this guys mic technique is hyper pro. Bravo.
The rules of thumb provided are good, but a former coworker had a similar but opposite idea. She worked at some financial institution in the 2000's, and they had a rule: developers were provided the least powerful machines in the company. The idea here was that if it could run on a developer's machine, it could run anywhere. So, instead of trying to keep your CI/CD agents as beefy as your developer machines, perhaps reducing the capability of development machines would make it easier to meet the demand in production. Side benefit here is that you likely end up with leaner runtimes due to developers having to fit within their local machine's constraints.
This is exactly what we do, but with our non-prod environments. If you can make it work on those machines with limited resources, then it will fly in production.
But you aren’t running your software in a cloud build pipeline, you’re building it. No need to make that longer than it has to be.
I think I'd have resigned in my first week
hobbling your devs is a terrible idea. every second waiting for a build to complete is lost productivity . instead, use vms with minimal resources for testing the deployed app.
By developer machines you mean the computers that they are supposed to work? Or development servers? If it is about developer computer then I would laugh loudly and resign the first day that idea is introduced. IDE, docker, some background processes - it all takes the CPU. And you can believe me - waiting 10 minutes for the enterprise solution to be fully loaded and indexed by IDE is something I don't wish anyone
28:00 4-5k lines of Jenkins pipeline code...I wish, lol.
The company I work at has a lack of developer expertise on the team of engineers responsible for maintaining pipeline operations. The current line count for looking at the multiple projects is 19,810. Additionally, they don't use a typical bundle process, and instead put everything in the vars/ directory. For those unaware, the vars/ directory is supposed to represent easily referenced things, like a pipeline entrypoint, and they are initialized as globals for the entire pipeline. Instead, we made every supporting action its own entrypoint basically, lol. Half the pipeline code is seemingly composed of hard-coded Shell scripts for interacting with the OpenShift CLI, and making sure that the 3-5 discrete Kubernetes configurations are swapped correctly (because we have, per application, a standard template, a blue template, a green template, a canary template, an alpha template, a beta template...)
Absolute madness to me. Also, to add insult to injury, they couldn't be bothered to write pipelines for specific builds, so we have The One Pipeline. And, on top of that, we have The One Build-Agent, which needs build tools for Java, Go, C#, Python, JavaScript, etc. And as if that wasn't already overloaded, we also installed a consumer browser on the agent because Selenium tests needed to share the same host for some dumb reason.
What is the proper alternative to putting everything under vars/ ? Asking since we had quite a bunch of Groovy under vars/ in my previous company -- and if this is actually a bad practice, I'd like to avoid it when I introduce shared libraries to my current place. I know that sharedlibs also support src/, but from what I understand it's not so different approaches...
Great talk
Test should start containers only once, saves lots of time. Integration tests using bowsers could be speed up. Lot's of unneeded waiting, because 200 years ago things were slow
really good!!
Pretty obvious conclusions, but I'm glad that they were highlighted
A pipeline running more than a couple of minutes is wrong...
It depends on project size
20:45 YAML is a human-readable data serialization language that is often used for writing configuration files. Depending on whom you ask, YAML stands for yet another markup language or YAML ain't markup language (a recursive acronym), which emphasizes that YAML is for data, not documents
Human-readable is debatable
33:21 no sound, not a skill issue
Yeah, the batteries in the mic ran out, so we had to switch to a handheld thing.
I dislike those because I never know how close to the mouth I need to hold it :)