Branching Strategies Explained
ฝัง
- เผยแพร่เมื่อ 28 เม.ย. 2024
- What are different branching strategies? Which Git branching strategy should you be using? Should it be trunk-based development, feature branches, GitHub Flow, forking strategy, release branching, Git Flow, environment branches, or something completely different?
#git #gitflow #githubflow #featurebranches #forking
Consider joining the channel: / @devopstoolkit
▬▬▬▬▬▬ 🔗 Additional Info 🔗 ▬▬▬▬▬▬
🎬 Continuous integration, delivery, deployment, and testing explained: • Continuous integration...
▬▬▬▬▬▬ 🚀 Courses, books, and podcasts 🚀 ▬▬▬▬▬▬
📚 DevOps Catalog, Patterns, And Blueprints: www.devopstoolkitseries.com/p...
📚 Books and courses: www.devopstoolkitseries.com
🎤 Podcast: www.devopsparadox.com/
💬 Live streams: / devopsparadox
▬▬▬▬▬▬ 👋 Contact me 👋 ▬▬▬▬▬▬
➡ Twitter: / vfarcic
➡ LinkedIn: / viktorfarcic
▬▬▬▬▬▬ Timecodes ⏱ ▬▬▬▬▬▬
00:00 Intro
01:19 Trunk-Based development
04:20 Feature branches or GitHub Flow
06:53 Forking strategy
08:22 Release branches
12:06 Git Flow
14:04 Environment branches
15:46 Which strategy should you use? - วิทยาศาสตร์และเทคโนโลยี
![[LIVE] : ONE ลุมพินี 60 | คู่เอก "สุริยันต์เล็ก vs ฤทธิเดช"](http://i.ytimg.com/vi/2ls0lZplik4/mqdefault.jpg)
finally, someone summarizes these different branching strategies in an easy way to digest. thanks a lot.
This was outstanding, I loved every word of it. I’ve worked on most of these methods and cannot agree more with your evaluation.
A git branching strategies video that talks about git branching strategies.
And on top of that it explains sums them very well.
This should be higher on youtube's list.
This is a great meta topic in our industry, thank you so much for taking the time to dd on this.
Glad it was helpful!
@@DevOpsToolkit great video. Also you forget that in case of microservices it becomes insanity level 999999.
You are awesome. The visual & pictorial representations were awesome. Though I spent more than 2 hours understanding this complete video. But this was so much worthy and concentrated content. Thanks for the video 😃
Wow I spent ages reading on articles and other videos and I could never fully understand branching and versioning until I watched your videos. Thank you!
Exceptional! you are a legend, and the way how you explained these strategies, I could not control but ROFL. We started off with GitFlow, and then adapted Release branching and now taking the path of Trunk based!
Not only super instructive but also funny :p Excellent as always, thank you !
Excellent video. Thorough and clear enough for someone who has almost no experience with this!
you solved my confusion by summarizing the branching strategies once and for all tysm
Nice video, Viktor. Liked the simplified articulation. Great job!
You just saved me hours preparring my last notes for my exam.. thank you
"...because it will feel like I'm on vacation" I ROTFL there! Great explanation btw, thanks
Great rundown. I LOL'd when you mentioned people using complex strategies for job security. I have people like that at my company.
Hey Viktor, Trust me this is one of the best video explanations out in internet.
Super thanks for making information as well as fun.
I totally get your points and the options for different purposes.
And the last takeaway was awesome, Haha.
you make the topic interesting, thank you.
Well done for summarizing and comparing all of them 👏👏👏
Personal opinion: The testing needed for trunk based development isn't a lot of unit tests. What's needed, in my opinion, is to have a robot equivalent of a QA tester, meaning that the thing will be deployed in an environment just like production, and some component will drive the program like a human would. Teams I worked in didn't like this or even see it as a possibility because it sometimes requires thinking outside the box and being multi-disciplinary in order to implement.
Something like cypress? This seems to be the robot equivalent
Correct. Unless you are dealing only with a rigid API (and no UI involved whatsoever) where a very mature set of deep unit tests (whitebox/blackbox) and be unleashed that will verify all possible inputs/outputs to confirm behavior is not broken for each of the API interface elements, then sure. Unleash the robots. But for _anything_ else, if you are not putting a set of human eyeballs on it and actually manually stepping through the application, then you are 100% going to shoot yourself in the foot. That tiny little CSS change you made which does the right thing on one page, but is shared around 50 other pages, has now right-shifted your logos to the wrong location. That input drop list menu that worked just yesterday is still there and the first item that the automation test relies on is still there, but that automated test robot will not tell you that the same menu item is now repeated 20 times. That automation robot will also not tell you that the beautiful white color you chose for that css element not just made half that text on another page completely invisible on a white background. The automated tests still see that text as being there, so what's the problem?
Automation is and always will be just a backup sanity check. Real manual human testing is ALWAYS required before releasing publicly available software to the world, else you are going to have some very pissed off customers.
@@privatename3621 In your scenario it seems that the fear of walking on a mine is the main problem, not the failings of testing automation. Feature toggle is what allows you to check for a mine before stepping on it.
What you're saying is end to end (e2e) test, which is an integration test. Look at the testig pyramid, you'll need different test to cover.
E2E test are slow and it's white box test. Unit Test are super fast, gives your immediate feedback and helps you identify problems on the unit level, whereas if an e2e test fails, if doesn't tell you what's wrong, you only know the feature didn't work as expected.
And if you don't cover using unit test and instead write them in e2e test, you'll need to covert a lot of scenario which could is not ideal and not efficient.
Often the production-like environment ends up not being very production-like. It can also be very expensive. Production is always the most production-like environment. Feature toggles were mentioned and can enable dark launches, staged rollout, A-B testing, etc. Ongoing observability that includes health checks combined with a fast hotfix capability is also critical in my opinion.
There is a big chunk of strategy that was missed here in relation to trunk based development combined with release branches and environments. My preferred workflow uses trunk based where main/master deploys to a staging environment (or UAT). Then once acceptance testing has completed, a release branch is created to trigger a deployment to production. Hotfixes then can be cherry picked from main into the release branch, as needed, avoiding the branching mess described in this video. This is similar to Gitlab AutoDevOps, but inverted from their default (main/master targets k8s production and other branches target staging environments). This also works well outside of k8s deployment targets.
You're right. There is close to infinite number of permutations one can do with branching strategies.
It's a great video, very crisp and concise !! However, in a corporate development environment there can't be only 1 approach. Probably, it makes sense to use combination of Trunk + Fork + Feature Branch. Here is the reason :
Reason :
1) Trunk based - For frequent small feature changes to main branch.
2) Fork Strategy - To limit the developers access to main repository and allow merge into main repo only via PRs.
3) Feature Branch - To have development for major features which tends to break almost everything until they are entirely completed. e.g. TECH stack upgrade etc
I know there can be different approaches with Trunk based development, to disable/hide incomplete features (e.g. Dark launch, Branch Abstraction, feature toggles etc). but that requires very high level of maturity of team (which is not always possible with new developers joining once-twice year).
I don't think that there is one-fits-all solution to anything, including branching strategies. While some work better than others, it ultimately depends on what one is doing and how mature the teams are. What really matters is not to pick one, but to be able to make a decision what to do and how to do something based on circumstances. There is nothing wrong to use different tools and strategies depending on specific situations, hence, it's quite OK to choose the right strategy on per-case basis.
Very well illustrated. I don't think you should waste your time watching any other videos about branching strategies
Well done! You should do a talk with the guy from the channel "Continous Delivery", I just saw a video in which he is more for trunk-based development rather than branching. Would love to understand better in an in-depth talk between you two! Btw, the git flow part was really fun!
That's a good idea. I haven't seen him since one of the pre-COVID conferences. Let me check whether he has time for it.
Excellent presentation. Immediately subbed. Thank you from Ireland.
Luv this! I appreciate your humor ... very helpful.
excellent explanation! Thanks Victor for sharing!
Really nice overview. Thank you:)
Excellent video, I like the way you delivered truth with a pinch of sarcasm. Kudos!
This was a good explanation and illustrated even. Thank you for helping.
Very informative ! Thank you!
Appreciate this breakdown, thanks!
My team _started_ using git flow shortly after I joined the teama few months ago. I was brand new to the team, and don't have a tonne of experience with branching strategies and yet even without knowing exactly why, I just had an almost instinctive revulsion to the new strategy, but as the team newbie, I kept my concerns to myself. Tomorrow though, I think I'm going to share this video with my managers and just let them watch it and take from it what they will.
I'm curious to hear how it went.
@@DevOpsToolkit Well I shared the video with my team. I think one person looked at it and it was basically forgotten about. Don't worry though. I still haven't given up this particular fight. I just need to pick my time.
Loved the video, subscribing your channel :) , especially the extreme hatred for Git flow & Env. branches :)
Wonderful explanation! You have saved my day.
Thanks a ton, @milonga332.
I giggled a bit at the environment branching part, since the idea was pretty intensely circulated in our team meetings a few months ago, as a possible strategy for our IaaC repositories. In the end we went for something most closely resembling feature branching - looks like we dodged a git bullet there :)
you may have gone to a mental institute
@@cyclogenisis after working for few months you can call that a vacation lol.
Great explanation even today :) I've been feeling lately that every other company developed their own branching strategy with its own name :D Even in our team, I have no clue where to categorize our strategy. It is a mix of feature, release, and even maybe an environment strategy. I guess I will never know if there is a team that uses the similar technique :D Cheers!
What an easy and interesting explanation!👍
I'm on the Environment Crap.
and have deployment team... when I mentioned "branchless" they told me that doesn't scale or other idiocy.
Great video, thanks really found it to be informative.
Ah, good old branch per environment merging. I still have nightmares! Happy feature branch user now :). Great video as always.
I still remember spending weeks in creating diagrams, meeting with people, and doing whatever else should be done to convince decision makers to simplify the insanity we had with branches. Every once in a while, I remember those days and then I spend yet another sleepless night.
@@DevOpsToolkit relatable, going through the same.
So you don’t have staging environment? Or do you release manually?
@@barefeg That depends on the strategy. When doing feature branches, there is no staging, at least not in the traditional sense. Each PR from a feature branch can get an environment that is removed when the PR is closed. In a way, there are as many staging envs. as there are open PRs.
Thanx a lot for your work!
Great summary and actionable insights
Nice presentation! Thank you!
great explanation!
Thank you for the video!
This explains all the strategies very well. But I think it should also include rollback strategies and which branching strategy makes it easier.
I believe that, in most cases, we should roll forward. If you do need to roll back, tools like Argo Rollouts or Flagger are great choices and do not depend on git branching strategies.
@@DevOpsToolkit I use GitHub Flow in all my services. We have automated tests, but we still need to rely on manual QA. We try to make features backwards compatible, but that's not possible everytime. I am thinking to start tagging the main branch with a release tag upon a successful release and in case a rollback is required, just create a build with the older release tag and get it deployed. Looks like it would work in theory. I am still figuring out the edge cases that would increase madness (in your language ;) ).
Awesome video, nice and easy explanations
In our BI team we use release beanching because things are interconnected. The thing is… you have to commit often
Great video, very helpful. Thank you.
You deserve a million thumbs up!
Just to be fair to the Release Branches Strategy: Everything Victor said about it is true, but when it is applied in the Project or Product as a Service business models, when there is only one production environment.
However, when business model is Product based, where product is shipped to many customers and there are many production environments, then Release Branches Strategy is viable complementary branching strategy to cover customers that are using different versions of our product (for non-technical legal, regulatory, business... reasons). Actually it is must-have additional branching strategy, in addition to say trunk based development or feature branches that are used for the main stream of development on the current version of the product.
Agree 100%. When you do need to maintain multiple releases of a product, you do need each of those releases accessible separately, so you do need a branch for each active release, similar to what Kubernetes itself is doing.
Stayed for 14:45 my favorite branching workflow 👌😉
I liked this video very much. Very informative
thank you, this was great
7th of April, 2005. Release date of Git. However, Victor is using branching strategies for 30+ years. LEGEND, still think you're the best DevOps youtube and can't stop wathing and learning from you! Keep the great job mate!
You're right. That statement had some artistic freedom in it. I have not been using branches for 30+ years. If my memory serves me, and, this time, I calculated it correctly, I've been using branches for 27 years, so I should have said 25+ years.
I think I started working with CVS in 1995. The "pain" caused by working with it still haunts me. Discussions about branches started long before Git was born.
@@DevOpsToolkit WoW, my appolagies, I've haven't heard for CVS before and thought (my bad) that branching was invented with git (feel ashmed). Meaning your statement is more than correct!
Outstanding !!
Best video on guy I’ve ever watched
My dumb ass stumbled across the wrong video. I literally have no idea what your talking about and found no clue in the comments. I was just trying to figure out how to create more branches on my cannabis plants. Hats off to u geniuses.
I would gladly change my branches for yours :)
thank you for this
That madness caused by git-flow, I am experiencing it in my veins these days as I switched to a team that uses git-flow. When I share my opinions with the more experienced engineers, they keep producing arguments against me which freaks me out.
Great job on this video! It was spot on and also had some really funny moments. “Cake made of mud..” 😂
Thanks! 😃
More like a cake made of shit.
Great Video. Thanks Victor
Really nice overview. Thanks
I've been getting back into development after about 5 years off and wanting to bring my basic skills up to a usable level. Git specifically and branching (CI, CD, testing) strategies are turning out to be the biggest area of learning and can be rather overwhelming at times.
You have a direct, easy and fun to follow manner. The information comes across incredibly well. Kudos
Excellent video! I think you left out the biggest madness of all -> Not having a documented strategy, ending up with some crazy mix of all of the above and hundreds of branches -> Madness Level = Suicidal... Might be worse than using Outlook for source control.
It's planning and experience. The more you have of one of those, the less you need of the other.
Baby dev here: I learned git flow just yesterday and I was so proud of it for a complete day until this video broke my pride bubble :D
But seriously, git flow makes a lot of sense, it's not this difficult. Also there are tools that can automate its flow. But I'll trust this video b/c experience talks first.
Make sense…. For your context. Every solution works better in specific context, good engineer chose the right solution, not the most shinny tool
Interesting video; I liked it. With my 40 years of experience, I agree with most of it. There is an unstated assumption in what is presented here, especially with the trunk based dev. The assumption here is that whatever is in trunk/master/mainline = production. That is not always the case, and it is where I diverge from agreeing. This is due to Continuous Deployment, which I personally, am not a fan of. I prefer a human in the loop. I follow the Apache Maven approach. Yes, Forking and PR's are used [to good effect], but commits to master can (and sometimes do) create a dirty/broken master. But this is not a problem, as we choose when to perform a Release - it is not automatic - why I don't like Continuous Deployment. Once is a Release is performed, it's out in the wild, and we prepare for the next release. Always moving forward. Cut a Release, moving it through environments until it fails testing or hits Prod. You make this point well, where you point out that we're not deploying source into production (unless you're using Ansible!), we make packages and deploy them as need be. Why don't I care that trunk does not equal production? What if there is more than one? OS Libraries and some of my work places have more than one production, so having trunk represent 'production' what ever that is, is meaningless.
One thing that I'd add/challenge: I understand why you say that you need a disciplined team for trunk, but my experience shows that feature branching requires a higher level of maturity and discipline so that you DO limit branch lifetimes. Otherwise you end up in merge hell and everything comes to a sudden halt due to the inability to deliver any change.
So, I take the KISS approach for everything that I've ever done, trunk based dev has worked for me, and what I recommend. Only go more complex if you absolutely need too.
TBH, there was ONE exception, where there were monthly release branches with overlapping dev (due to testing resources), but we simplified things by never having to merge back to master (no point). R2210->R2211->R2212->R2301 etc.
You're right about "multiple productions" like, for example, when maintaining multiple releases of something (e.g., Kubernetes). In those cases, there are multiple branches that represent "production".
The last 2 blank minutes are for meditation over the code-to-subversion? :D Cool video as always
Introspection, I'd say. To ask yourself if the whole software development shebang is worth it.
Good catch. I must have messed up the editing part. It should be fixed in a couple of minutes.
Great video!
Could you clarify how we can handle with hotfix using feature branch strategy? I’m not sure how to not interrupt the development flow of new features? (In this case there is only one development environment)
Hotfixes are actually easiest ones to deal with since they tend to be very short lived. Create a branch from the mainline, push the hotfix, create a PR that will run whichever pipelines steps you might need to run, and merge it back to the mainline.
loved the conclusion at the end! The beginning was more like he was biased towards trunk based dev and thought that is what he was going to conclude. It was refreshing to hear otherwise
OMG, thx a lot Papa Varcic
hahaha loved the conclusion ! RUN !
Loved it ❤ thanks! But you're not completely fair on gitflow: it's not that complicated : it's like the feature flow but with extra steps (and, from what i've seen, some steps, like the release branch are, often, not even used !)...
... And, often (e.g. if you don't have strong tests, ci/cd ) you don't realy have the choice :(
I've worked in many teams that use short lived feature branches (that exist for a day or two) and they call it trunk based development. And I agree with them. What you call feature branching is essentially the same as trunk based development since you need feature switches because you are essentially pushing to the trunk regularly.
Technically, trunk based development means that you push directly to the mainline without branches and PR. In practice, short-lived branches are very similar in terms of the objectives (get to mainline and production fast).
Feature switches are not a requirement of trunk-based development. They help a lot, but are not necessary.
@@DevOpsToolkit Yeah, so making them different branching strategies is just academic at best. I still need to hear a good case for why using short lived feature branches with feature switches is not considered trunk-based. All I've heard so far is just splitting hairs. I think I am in good company given the previous teams that I've worked with and given some "authorities" like Martin Fowler who considers using short lived branches as trunk-based development too. Of course there is no official source of truth. In the end, whatever is adopted by most people will win out.
PD: Just in case, when someone says "trunk-based" it is not implied that you are pushing to the trunk. That is your personal definition. Another valid interpretation is to base your development on pushing to trunk as quickly as possible. So technically, it does not mean pushing to the mainline without PRs.
At the end of the day, what does matter is how frequently we push the code to the mainline. Whether that is done directly or through PRs is not that important. So yes, I agree. If you are pushing to the mainline frequently, you are effectively doing trunk based dev.
The only thing I would correct is when you say "as quickly as possible". For some, quickly could be once a month.
I think release branches can make sense for mobile applications or IoT software where you simply can't deploy updates on-demand, or force users to update remotely.
You're right on that one. I forgot to mention that my points do not apply to those orgs that distribute software in a way that users download/install some binaries. In those cases, multiple releases might need to be maintained and that requires release branches. A good example is Kubernetes which promises to maintain the last 4 releases and that means patches to all those. So, it's not that much whether you or others are updating but, rather, how many major/minor releases you promised to keep up to date with bug and security fixes. If there is no such promise, users can always install the latest.
There are millions of ways to branch.
All the strategies you mentioned fall into a single family.
Branching with git opens up so much potential.
Yet, the industry is stuck on one flavor, as always.
haha, thanks a lot for this video. Awesome take on this subject. I'ma send this video to all the old heads i know, who still believe Git Flow/Environment branching is the right way to go. Tired of convincing :P
I laughed at Git flow segment like I was watching a standup show haha.
What no one realizes is even in feature branching or trunk based development or release branching we all encounter merge conflicts etc., because the business team wants 3 big features in the same javascript or java or typescript file at the same time which will be taken in same sprint and the leads and devs are supposed to deliver that.
This is a great video! We used to follow gitflow, but now swited to release based braching. But the issue we only have 2 environment. As per gitflow, develop and prod. How can manage these in release based branching.
I don't think that there should be release branches. Tags are a better option since there should be only one release (one binary or one container image) that moves through environments.
So eature branches with feature flags. For me it has everything as soon as feature flags are being used. As soon as project is non-production you can give right to merge pull requests without a review. It's quite close to trunk based development - gives you more speed via simplicity while avoiding any dedicated pipeline logic.
You're absolutely right. Feature Flags (Feature Toggles) are the perfect companion.
I skipped mentioning them in this video mostly to avoid extending the subject too much. I am planning to have a video (or a couple of videos) dedicated to Feature Flags.
if we go with one branch only, I don't think we can hot fix? because main branch will have some features which we would not want to ship to prod. and if we take a branch off of master branch it would contain the work. if we take hot fix from release branch then we will need to merge it back to master. NO? Could you please share thoughts on that
That can be solved with feature toggles.
How do you handle big feature projects without release branches? I started with a very continuous strategy with light feature branches, but the product owner is pushing for not releasing until the big feature is “perfect “
The answer depends on your confidence in tests. If you trust them, you can release frequently, but disable the features from being visible to users by using feature toggles. On the other hand, if you do not trust your tests, then releasing frequently and using short-lived feature branches is not a good idea.
Thanks for your video. I really like it but I have to say your opinion on the Git-Flow strategy was a big surprise to me, probably because I worked at the place using it and it was the least painful environment for me as developer (and I know we had release management but it did not sound in any way complex compared to for example Release Branching strategy). So, I have a question for you. I know your opinion on it already but it looks like you favor environments with fully implemented CI/CDel/CDev pipeline and I do, too. However, the reality is these dont get implemented just over the night. The things that need to be set in the motion are not as simple. There needs to be an agreement "we need to change", that needs to be approved by the higherups, there need to be team agreement to move that direction etc, etc. In lean environments, these can take weeks or months, in not-so-lean environments, these can take years or never come to materialize. What do you recommend as a strategy fitting a team where (a) there is no CI/CD in place, (b) there is slow release cycle, 1-3 deployments a year, (c) teams (as you described for "release-branching" collaborate only once one team is ready to merge into the trunk), (d) there is no need to maintain multiple versions of the same software, and (e) Our features are not small to fit into 1-2 day living branch only, but last usually months? Thank you and much appreciated. Really like your video, so many good points. Thumbs up!
If there is no need to maintain multiple versions, there is, by default, no need for anything but mainline (production), feature branches (development), and PRs (testing). Those requirement would not fit you though since that assumes that each feature is released separately. If you have 1-3 deployments a year, you could still do it like that but that would mean that you would not have the possibility to create hot-fixes so you would need an additional branch.
Going back to your points:
a) The process does not (should not) change without CI/CD. What changes is the velocity.
b) that's bad but should not influence the process either unless that means that you are working on multiple releases in parallel. If you do, you need release branches and that's where things start getting complicated.
c) Not having collaboration until late stages is bad (really bad) and is probably what prevents you from increating velocity.
d) That makes things easy as long as you're not working on multiple releases in parallel.
e) If it takes months to deliver a feature is bad but does not necessarily have to influence the strategy. It just takes longer and results in a bigger mess when merge finally happens.
I can't give you a more concreate answer beyond "do not change anything if you're not ready to change things vertically". Saying things like "we're going to change X but Y and Z need to stay the same" typically results in a failure or low ROI. The reason behind that statement lies in the fact that every tool is a result of application of a specific process (or processes) which are a result of a specific culture. If you change the process, you need to change the culture and the tools. Changing one without the others is rarely a good option.
@@DevOpsToolkit much appreciated
Thanks!
That's very generous. Thanks a ton!
@@DevOpsToolkit Your content is so informative and high quality. Keep up the good work!
what do you think about vs code extensions that notify you in real time when you are about to make a conflict in a line so you can discus it with a colleague?
I am not sure how could that work if everyone has it's own branch. I guess that assumes that everyone is working in the same branch. Is that the case?
I'd like to try it thought. It could be indeed interesting. Let me try it... What's the name of th plugin?
Great video
I disagree with the "you don't care what goes to production" in trunk-based development, because the feature toggles are exactly to avoid that. Other than that, great video!
You are right as long, as you said, there are feature toggles. The only thing I would add is reliable and automated testing.
@@DevOpsToolkit also you don't have to release everything that you commit to main immediately. You can still test it as a release candidate. It's just that the review process is different, and that's why Dave Farley suggests pairing instead of pull requests for trunk based dev. I think this is a bit unrealistic for most teams
I use a single mainline where everything is merged. Any commit to it is deployed to staging environment, and any tag is deployed to production. Insanity level seems low even with this git flow. The only problem is when devs merge to the mainline but that change is not ready to be tagged at any moment. It doesn’t happen that often but then it’s better to keep the feature branch a bit longer until we know it could be deployed to prod right away or wait. I have to mention we do automatic deployment to staging when the PR is open too. This doesn’t use feature branches but instead it’s a new pod with slightly different service name. The infrastructure can route to that version at runtime by passing flags. It’s kind of a feature flag but for the microservice routing infrastructure rather than the code. This makes the deployment a bit more complex but it’s fully automated now.
I would still like to try code feature flags to remove the issue I mentioned of the mainline not being ready for deployment as well as not having to deploy multiple pods for each PR. Then we would release to both staging and production on commits to the main trunk. But the default state of the feature flags would be on in staging and off in production. A tag In the main trunk would then happen when a feature flag is flipped to on in production. I haven’t really thought how that looks precisely but it sounds interesting
If a change is not ready (to be tagged or anything else), it's probably not a good idea to be merged to the mainline anyways.
Oops I submitted before being ready. I edited the main comment. But I agree it needs to wait if it’s not ready, even if we try to be as agile as possible
What do you recommend when using GitOps style IaC? It seems to be often recommended as a branch per environment, or sometimes using a tag per environment. Love to hear your thoughts.
I do not like the idea of using a branch per environment. Branches are supposed to be mergeable to the mainline and not used as a permanent placeholder disconnected from the trunk.
I tend to use either a directory for each env. in the mainline or a separate repo for each env. In either case, it's the mainline that matters, and everything else is temporary.
Not sure how applicable this approach is to your use case, but if you're using something like Terraform Cloud or Enterprise, you can separately version your modules using their registry. Then you can reference those (sub)modules by version in a main IaaC repo, to which you can apply something like feature branching, since making a "code change" would simply imply a version bump to said modules.
@@Blkhole02 Exactly. That is a general approach to releasing something. Every release is versioned and stored somewhere. Using a release of something should have no relation to branches. That is the part that people mix. There is no need to have, for example, a staging branch with the source code of an app if you already built that app and created a release.
@@DevOpsToolkit Nice video as always!
We are using the following strategy for application deployment (application code itself is in a separate repository, can therefore be ignored here):
Feature branch for any change no matter which environment -> dev-branch for deploying to dev env -> main-branch for deploying to test env -> tag on main-branch for deploying to prod env
(Each branch has directory structure containing configuration of all environments)
80% of the time we are quite happy with this because of the following reasons:
- We can prepare changes for all environments simultaneously, but roll them out independently.
- The standard way of merging is predefined (dev->test->prod) so there are almost no merge conflicts
- We have multiple pull requests which get reviewed before deploying to prod effectively
Things which are problematic:
- Some applications have more environments than these 3, meaning more branches. This is where things get laborious.
- The code is effectively stored as many times as there are environments (-1 for prod, because its tagged) which could lead to confusion what is applied effectively.
What would you suggest for in our case?
Especially the first "advantage" described above is huge for us.
@@wodan742 I think that branches should be temporary and should be mergeable to the mainline. From that perspective, I don't like the idea of having a dev branch for dev env. Instead, I prefer having a dev directory in the mainline, or a dev repository. That way, mainline is always the source of truth and all PRs are made towards mainline. If sharing is important, different dirs in the mainline simplify things a lot. The alternative is to use some kind of libraries/modules/plugins that contain the shared code.
Your painpoints are one of the reasons why I love tools like Argo CD and Flux. Manifests can (and should) be in the repos of the apps while the references to those apps and env-specific values are in the repo of that environment.
Git Flow is the symptom of when having a very good hammer then everything looks like a nail.
I would also add to the list of unspeakable mad strategy git sub-modules as a way to police where developers push things.
I did not mention submodules in that video mostly because I worked hard to erase it from my memory. That was a cause for a lot of trauma.
What about the continuos delivery using feature branching or trunk based to dev, staging and production environments in an automated fashion.
Think of GitOps and GitHub actions to k8s cluster.
How would you do it? Git tags?
That's a great question. Would you mind repeating it in today's AMA session on the channel? I think it would be of interest for a wider audience.
@@DevOpsToolkit I did but my question got skipped several times. I even sent a super chat 😞
@@angelgutierrez240 Sorry for that. I can assure you that it wasn't intentional. Darin (he's putting messages on screen) probably missed it.
I'm in a meeting now... I'll answer later tonight in here or, if you prefer, pick any time available in calendly.com/vfarcic/meet and we can have 1-on-1.
Terribly sorry for missing your questions. I feel really bad about that. I hope there's no hard feelings.
@@DevOpsToolkit would be great if you could reply in here so that everyone with the same doubt be able to see it.
@@angelgutierrez240
I tend to keep application manifests in the repository of the application, and Argo CD or Flux manifests in env-specific repos. Those are referencing application manifests and overwriting variables with env-specific values.
Inside the pipelines (e.g., GitHub Actions), I checkout env-specific repos, modify Argo CD or Flux manifests, and push them back.
The logic is, more or less, the same no matter which branching strategy is used. If, for example, we have feature branches, I'd create new Argo CD or Flux manifests and push them to the previews repo whenever a PR is created. When a PR is closed, I'd remove those manifests so that the preview environment associated with that PR is removed.
The builds triggered by commits to the mainline (no matter whether those are made directly or by merging PRs) would perform some actions (e.g., building, testing, etc.) and, each time I'd need something deployed somewhere, push changes to the env-specific repos.
At the end of the day, it's all about modifying env-specific repositories that reference the repo with the manifests and overwrite stuff with env-specific values. It's a set of simple `git clone`, `sed`, and `git add/commit/push` commands.
I do not have a strong need for Git tags since the history of env-specific repos is giving me insights what was and what is running in my clusters. That being said, I do still use tags, but that's mostly for the purpose of distributing releases to third-parties.
You're really cool, it's a pleasure to watch
Thanks
Hey Dev need Assistance , we have One big monolithic application , we work on Agile PI model like 3 month work panned advance and requirements break in to stories and plan multiple spring , for that PI , there are many thing we can only test in integration environment because of hardware software etc , for Sure trunk base or feature base development wont work for us and release base is too complex , was thinking to modify bit on fork base , So we have 2 branch main and integration close on each other thn dev create feature branch from main work on it after test code review merge to integration branch , after integration testing core team decide if thy want to release the feature or not , if they want they can push to master and it goes to prod
The threat of migrating my code to subversion worked ;-)
A while ago, I used a similar strategy, except that my threat was migration from SVN to CVS.
Do you have videos explaining the workflows of feature-branches and release-branches little bit more in depth?
Unfortunately, no 😔
@@DevOpsToolkit perhaps a good reference to look at?
Much appreciated again
It's been many years since the last time I read a book about version control so i not even remember what it is. Since then, it's mostly personal experience. I other words, i don't think I have a useful reference 😔
I was gonna say best video but after reading some comments of Viktor, I am gonna say "good video" :)
I guess "good" is still good :)
What was the comment that lowered the score?
@@DevOpsToolkit I do not want to be misunderstood. I was just joking, it is a great video. (I was referring your comment about the term "best practice" & "good practice")
I missed my own reference :)
Great video thanks.
Though I will step in to defend git flow a little here since it seems no one else wants to :) ...
When I use git flow, the release branches / hotfix branches are like a stepping stone to main, then main (which one would tag / release from anyway) gets merged back to dev instead of the original branch. It makes things a lot simpler vs oft-documented git flow per-branch stuff and cuts out the "more or less sometimes eventually up to date" bit (1 tag = one merge back to dev rather than depending on the merger to know how to / remember to merge individual branches back to dev - which admittedly they wouldn't remember to do otherwise).
I think it depends a lot the sensitivity of the software you're writing, the manual testing / team involved, the stability / confidence required etc (after all, unit tests / integration tests get bugs too). I do a lot of specialist development so those factors have been important pretty much anywhere I have went. Effectively it depends on whether or not you need release branches to stage for extensive and/or expensive non-automatable QA while ongoing bleeding development testing can continue in isolation. It also gives you a more formal and well defined version of environmental branching too without doing an environment strategy (automated deployments for dev = integration, release-* = UAT / staging, master = production). It even can give you long release branches should you need them. That's the thing with git flow - it pretty much gives you everything you describe in the list, but gives you a well defined framework in which to do it.
As another example if your target is an embedded or specialist hardware system, you can't always automatically test every aspect of it, and testing everything in a test plan is resource intensive where you have a separate QA team testing it. So you don't want to do it too often (and as a developer you don't want to be pressing buttons on a hardware device either). If you're lucky enough to have a simulator for CI, even then it won't necessarily behave in the same way as the real thing so you ultimately still need manual testing anyway.
I think git flow gives you everything most mature projects need and it's ubiquitous enough that must developers don't need it explained to them. In saying that, I usually would start trunk-based for new projects, then move to git flow once the project starts getting deployed because it *does* bring with it more overhead, but I've used it in several companies, none of which needed a release manager and none of which caught fire after I moved on.
That's a great write-up.
Ultimately, it all depends on what you want to do and what your requirements are. I tried to provide general rules and ideas which certainly do not work for everyone. The important thing is to understand and have practice with different tools and processes. After that, one can make a decision what to do and what to use based on first-hand experience. It does not matter if that ends up being X instead of Y. What does matter is that decisions are not made blindly or "because that's how we were doing it for years". Beyond that, decisions differ from one organization to another.
Great video. Which one to use? Depends on how mad you want to get ;-)
You forget the backup aspect of the central repo. When you only push to the main line and do not create branches, then you do not have that backup component and at some point some work will get lost if you do not push your work to the central server. Often people want to push to a server before they leave the office or it is an requirement of the company.
That's true only if you work on something for a long time without pushing to the mainline.
@@DevOpsToolkit Not every team member will push every day day to the main line, not even every two days. Some features need a few hours dev time, but some also need a few days and you should not push half done work to the main line and you still have push requirements at many companies. Please don't say now that you should split up everything into a few hours tasks, because that is impossible to do and also no good idea to do that ^^
It would be a project managers wet dream to see that every feature and bugfix is done in 4 hours and fully tested and operational. Devs always completely understate their development time.
I do not even want to mention what figures I've heard in regards to development time at companies and what the reality was and this isn't due to inability of devs, but just the reality of hour industry that execution time is often only a small fraction of dev time and often devs are challenged with new things and problems arise. This scenario where we all only have execution time does not exist in reality, because then we'd all build an app in a matter of 2-4 weeks.
Also if somebody has to pack his things right now after 1 hour of work what then?
It's just a characteristic of this one branch model that it will not be easily possible to fulfill this requirement.
@user-du6hs8fe8x you're absolutely right. That's why we use feature branches.
@@DevOpsToolkit yes
Madness level pictures are aptly chosen 😂
hmmm. Were i work they want to do environment branching, but keep some branches in different repositories