Continuous Delivery Pipelines Webinar
ฝัง
- เผยแพร่เมื่อ 16 ม.ค. 2021
- This is an edited version of a Live-Streamed Continuous Delivery Pipelines Webinar recorded on Nov 10th 2020. Dave Farley explains "what is a Deployment Pipeline" and describes the objectives of a Deployment Pipeline and its role in Continuous Delivery and DevOps. Dave describes how to build a Deployment Pipeline from scratch, and some ideas to focus on, when building a Deployment Pipeline, so that you maximise the value that you get from your software development process.
Dave also answers the detailed questions of attendees at the webinar.
---------------------------------------------------------------------------------------
If you want to learn Continuous Delivery and DevOps skills, and How to Build a Deployment Pipeline, check out Dave Farley's courses here:
➡️ bit.ly/DFTraining
📚 BOOKS:
📖 Dave’s NEW BOOK "Modern Software Engineering" is now available on
Amazon ➡️ amzn.to/3DwdwT3
In this book, Dave brings together his ideas and proven techniques to describe a durable, coherent and foundational approach to effective software development, for programmers, managers and technical leads, at all levels of experience.
📖 "Continuous Delivery Pipelines" by Dave Farley
paperback ➡️ amzn.to/3gIULlA
ebook version ➡️ leanpub.com/cd-pipelines
📖 The original "Continuous Delivery" book by Dave Farley and Jez Humble
➡️ amzn.to/2WxRYmx
--------------------------------------------------------------------------------------
You can get Dave's FREE "How To..." guide on Acceptance Testing, when you join the CD Mail List
➡️ www.subscribepage.com/cd-atdd... - วิทยาศาสตร์และเทคโนโลยี
Disclaimer: I have deep respect for the work that you are doing.
When talking about deployment pipeline you should not mix everything together. Release pipeline and deployment pipeline are different things. I work with deployment pipelines for almost 15 years. When talking about deployment pipeline you should not focus on acceptance, code commits etc. Their are related but it is not about deployment. You loose focus when you mix everything together. From my experience you need deployment pipeline to achieve following goals or answer following questions:
- can you deploy? for testing, for release, for QA?
- measure performance degradations/improvements
- can you migrate data?
- what is your availability during deployment?
- security and compliance requirements violations? Mitigation process?
- quality of IaaC?
- speed of deployment?
First you use deployment pipeline during build, then use it in testing, then in QA/production with data migration tests. It means that deployment pipelines is a tool used throughout continuos delivery cycle. It is not a delivery process. Your videos seems to mix a lot of things and on the other hand does not draw clear line between different aspects. Does not emphasize three major challenges:
- data migration
- IaaC
- recovery
and three minor challenges
- performance impact measurement
- integrated automated testing
- availability impact measurement
So, I hope you could work more on deployment pipelines to bring it in more consistent package to wide public.
Well, first, thanks for the thoughtful response.
I don't claim that what you are describing are not useful ideas, they are. Much better than what went before. However, in this one narrow context I can be more definitive, maybe even dogmatic, than I usually am, because I invented the concept that we are talking about. The term "Deployment Pipeline" is one that I created for the Continuous Delivery book. So I can be sure of the definition for this one thing.
What I describe here is exactly what I meant, and have been applying for the last 20 years or so, on real projects.
One area where we may be getting confused on the approach, I am NOT saying that the pipeline has to automatically release into production.
The decision to release can be automated (Continuous Deployment) or manual (push-button). My point is that the "Deployment Pipeline" is definitive, in terms of what constitutes releasability. If the pipeline says everything is good, there is no more work to do apart from "pushing the button to release" and that is a choice.
So, where we differ, perhaps, the "Release Pipeline" and the "Deployment Pipeline" are not different, but "Release" and "Deployment" are.
I can choose to deploy and change into production that passes the evaluation in the "Deployment Pipeline", that doesn't necessarily mean that I have released a feature, the change may be part of a feature that is not yet ready for release (I talk about that idea in more detail here: th-cam.com/video/v4Ijkq6Myfc/w-d-xo.html).
My point here, is that we can use the idea of a "Deployment Pipeline" as THE organising principle for our development approach. It is definitive in terms of what it takes to get to a point where we can safely, and with confidence, deploy a change (hence the name) and that idea is a lot more valuable than breaking up the process into a series of different pipelines.
I hope that, at least, my thinking is clearer?
Hello Dave, can you please make a video about what a single cowboy programmer should look for ina CI/CD system. There are still lots of programmers who work on their own (maybe pet) project without a team. What special rules apply, where should i focus more and where less ?
Thanks for the suggestion
@@dixztube and me!
Would you have separate deployment pipelines for database schema changes, API layer, and backend layer of a single service? Or is it better to have one pipeline for all?
Because of the runtime dependencies of each layer on a specific version of the layer below, I would deploy all layers (including UI) together, so one pipeline for all. It's better to split vertically, so to have services by domain, where each logical service may have its own pipeline.
Bookmarked.
Paused, awaiting results 🤣
For each newly generated release candidate, would the current release candidate being tested in the acceptance test environment be overridden by the new release candidate? or do we provide each newly generated release candidate it's own acceptance test environment?
When I have done it, we have always used on acceptance test environment per pipeline.
My advice, for this approach, is to run this as kind of buffered process. Imagine, for simplicity, we have a commit-stage that takes 5 minutes and an Acceptance Cycle that takes 50. There may be 10 commits per-acceptance run.
If we naively test every commit, we build an ever-increasing backlog of changes.
Instead:
Implement a simple algorithm, when the Acceptance Test stage (gate to Acceptance Cycle) becomes free, identify the most recent successful release candidate and deploy and evaluate that.
In our example, this "most recent successful release candidate" could be the sum of 10 previous commits - it will, of course, include all of the previous commits. So the acceptance test stage can "catch-up" by surfing the leading edge of the changes.
If there is no release candidate ready when the Acc Test stage comes free, sleep for a few minutes and check again until there is.
I am trying to imagine any problems with creating an Acceptance Environment for every Committed release-candidate, and I can't think of any except that it would be expensive (not too bad in the cloud I guess) and a bit more complex to understand the results. You would have to search for successful (passed all acceptance tests) release-candidates that were associated with the most recent successful commit, in order to figure out what is the most up-to-date candidate for release. Not difficult, but a bit more work.
For this to work though, it would be VERY important to treat the Commit stage as a sync-point. You can't do this on separate branches. Interesting idea though.
1 hour... Time to grab the popcorn.
These sorts of videos to us sorts are what gaming videos are to today's gaming kids. Real, real interesting and pop-corn worthy.
@@16815 I mean... I'm one of today's gaming kids 😁 I think one of the dangers of this kind of content is that there's a risk of blurring productive learning with a kind of borderline infotainment. I've seen a metric crap ton of con and prez vids but I don't wanna risk claiming I learned anything from them until I've implemented something that someone uses using that info.
Hell even a tutorial that mixes that info into some other context/use case could count.
I'm still wondering if there's a 'minimum viable learning project' template for different fields. Kind of like implementing recursive descent parsers in programming langs to build familiarity with them for coders.
I prefer a glass of wine, but its "Dry January", so Popcorn is good 😉
Hi Dave! I really, really enjoy your content! I Think you do a great job in improving the overall mindset of how programmers think about their work. As a german programmer i am really happy that i can find good and free quality content about improvin the skills needed to grow as a developer at least in the english speaking community.
If i could make a wish on your content it would be that you improve your audio quality of your content. As i can see, your channels is rapidly growing and people will expect good audio and video quality at some point. Speaking in your words: Fail early! Improving your audio quality now will get you more subscribers. The longer you wait, the more subscribers you will missing out =)
Please excuse my bad english skills. I hope you understand what i mean =)
@@BugTrampler I'm not Dave, but I'm sure he understands. Your English is pretty good.
Dave, here is the citation from your book: "It is often useful to incorporate a simple smoke test suite into your commit stage. This smoke test should perform a few simple acceptance and integration tests to make sure that the most commonly used functionality isn’t broken-and report back quickly if it is."
Does this literally imply that the Commit Stage may incorporate a smoke integration test?
The idea of the commit stage is to get fast feedback (ideally in under 5 minutes) and a high level of confidence (about 80%) that, if all the commit tests pass, then any other tests in later stages will also pass (and so the change will be releasable). If the smoke tests can run in under 5 minutes, and give you increased confidence, then I think that they can be useful. However, I wouldn't compromise the 5 minute limit to allow for smoke tests.
@@ContinuousDelivery do you think running lightweight smoke integration tests that meet your requirements not to touch DB/network/disks, made with the same xUnit framework like the one used to accomplish unit testing might be too much costly to Dev and to the Deployment Pipeline velocity?
@@ashimov1970 I can't really answer that outside the context of your system. I recommend that, if you are unsure, try one simple example, and see how long it takes to run it. Worst case you now have a test that you can run in a slower, post-commit, stage. Use the time (under 5 minutes) and confidence (80% chance of passing if commit passes) as guides to make your decision. Commit tests are not necessarily all unit tests. The majority of them will be, because that is the most efficient form of testing, so you can do more unit testing in your 5 minutes than anything else. But you also need the confidence, and adding other kinds of tests to commit to increase that confidence is a good strategy, as long as you don't use it as an excuse to compromise on the time.
@@ContinuousDelivery Now I completely get your point, Dave. Thank you so much. BTW, I wonder what kind of updates your new book on CD Pipelines has got with respect to Deployment Pipeline architecture as compared to the old good one ("Continuous Delivery") by you and Jez Humble
@@ashimov1970 The new book is intended to be a more direct guide to getting a pipeline established. I think that most of the ideas in the new book were covered in the original, but the new book is meant to be a slightly more practical guide. The original book explores the ideas of CD in more detail and more breadth.