AWS: Real-world CodePipeline CI/CD Examples

AWS CodePipeline helps us to orchestrate CI/CD pipelines. To implement real-world pipelines, CodePipeline calls additional AWS services to do the work. E.g., CodeBuild to run arbitrary scripts, CloudFormation to create or update stacks, or CodeDeploy to deploy software to running EC2 instances. A pipeline is connected to sources such as a git repository (Bitbucket, GitHub, and GitHub Enterprise Server, CodeCommit), an S3 bucket, or an ECR repository. If a source changes, the pipeline executes. But how do real-world pipelines look like? In this video, I share pipelines with you, from simple to complex.
Interested in more? Check out the blog post: cloudonaut.io/real-world-code...
Support us:
Have you learned something new by reading, listening, or watching our content? If so, we kindly ask you to support us in producing high-quality & independent AWS content. We look forward to sharing our AWS knowledge with you. cloudonaut.io/support-us/
Chapters:
00:00 - Intro
00:42 - What to expect
01:46 - Introduction to AWS CodePipeline
04:47 - Available Source Integrations in CodePipeline
06:50 - Common Architectures of Pipelines
09:50 - Missing Acceptance tests
14:44 - Real-World Example
17:19 - CodeDeploy
18:47 - Demo: marbot pipeline
24:27 - Pitfall: Update pipeline in pipeline
26:20 - Outro
#aws #amazonwebservice #cloudcomputing #cloudonaut #codepipeline #cicd #examples

[@ayencoscolfield3312]

just discovered your channel but am loving it , thanks and good job

[@cloudonaut]

Glad you enjoy it!

[@ansizfark]

You guys are so knowledgeable! I love your AWS content, so helpful to me studying for my DevOps Engineer renewal exam.

[@cloudonaut]

Thanks a lot for your kind words. All the best for your DevOps certification.

[@kaedien]

Fantastic content as always, keep it up!

[@cloudonaut]

Thanks!

[@adamghowiba]

Please make more real-world videos like this. I've been searching for a channel that actually dives into "real-world" project piece by piece instead of providing a simplified example.

[@cloudonaut]

Thank you very much for your motivating feedback.

[@christianibiri]

Great video guys

[@cloudonaut]

Thanks so much!

[@yangzhihong521]

amazing

[@arunramachandran8717]

Thanks for the amazing content regarding codepipeline. Where can I find the sample pipeline files for matbot and other samples you showed in the slides?

[@cloudonaut]

Check out cloudonaut.io/configure-your-cloudformation-managed-infrastructure-with-parameter-store-and-codepipeline/ for an example.

[@pjpj652]

I am not able to understand why would pipeline needs to update itself as you mentioned from 19:48 onwards.

[@cloudonaut]

In GitHub Actions, if you change the workflow, it just works. In CodePipeline, you have to update the pipeline yourself.

[@0xccd]

Is it possible to dynamically chnge the source before running the pipeline? I'm interested on running the same pipeline for different repositories

[@cloudonaut]

I don't think, that's possible. But it is possible to define multiple sources.

[@0xccd]

@@cloudonaut I found out that you can use codebuild. Basically, you can provide the project and repo info by using env vars, then trigger the build using the aws sdk and before each execution you overwrite the project and repo info as env vars. This is limited to codebuild, I'm still not sure how to integrate it with pipeline and multiple stages

[@XStrikeSuper]

What tools or script did you use to run integration tests?

[@cloudonaut]

It depends on the project. For example, we have been using JUnit for a Java project to write integration tests. For Node.js we are typically using mocha.

[@XStrikeSuper]

@@cloudonaut if I understand correctly integration tests are hitting real services? I typically use jest for unit testinf but never tried for integration tests

[@sbuckle1171]

Sorry, I'm not understanding why the 3rd stage is called Commit. Why not call it 'Build Image'? Commit often refers to a change in the source code which would be your trigger for your build, assuming you've configured your CI/CD system to automatically push and merge commits into your branch.

[@cloudonaut]

We use the terms introduced in the "Continuous Delivery" book by Jez Humble and David Farley. Check out www.informit.com/articles/article.aspx?p=1621865&seqNum=4 to learn more.

[@Sam-nn3en]

the wave deployments at 15:19 sounds like canary deployment

[@cloudonaut]

Yes, kind of.

[@prerakhere]

how to have that self updating pipeline stage?

[@cloudonaut]

You can find an example in CloudFormation here: github.com/widdix/aws-velocity/blob/master/deploy/pipeline.yml#L164-L181

[@luvrahooo]

How to trigger code pipeline in case of pull request?

[@cloudonaut]

I don't think that this will easily work. CodePipeline is designed to work with a single branch.

[@meras1549]

One remark. You don't do Linting only in pipeline unless you pay for cloud costs by yourself :) Linting should be done at very early stage so you should have scripts to lint you code locally and then next step is to lint code during push and last is lint before merge. So linting in build is as well OK but before that you have bunch of tests before you will trigger build.

[@cloudonaut]

Yes, lint before tests. I also agree that you should be able to run all the steps locally as well.