Localstack has also developed an AWS CLI wrapper call awslocal which will point the cli commands at your running localstack configuration. This is my preferred method over setting an alias.
This is exactly what I need. An AWS local Simulator / Mock / Stub. If it also calculates the cost of simulation. This would absolutely perfect to apply cost predictions.
Think this is the earliest I've been to one of your videos. Thanks for putting out quality content like this! Thanks to you I'm working on moving my company over to use kubernetes and you've helped show me tools to make that infinitely easier!
Do you think it might be useful to have functional mocks of AWS services running locally on your computer? The latest release (shipped only a few days after the video was made) comes with support for ARM64. You can now run LocalStack natively on your Apple M1 - or even your Raspberry Pi! Please check out the 0.13.0 release notes in the Github repo for more details. IMPORTANT: For reasons I do not comprehend (and Google support could not figure out), TH-cam tends to delete comments that contain links. Please do not use them in your comments.
Great video! In my opinion, local mocks are good since you can setup simple demo's of running AWS services and AWS solutions that you can show to clients or other people. This is useful for small app demos or just testing before you commit to provisioning the services on AWS cloud itself.
One thing I wish to see if there is or will LocalStack support other cloud providers like Azure or Google. I checked their site and the tool is more catered to AWS services. So it seems it will really focus on that. Still wishful thinking for other cloud services to have a tool similar to LocalStack to create mock services non AWS.
@@hamstar7031 From what I know, there is no ambition to expand LocalStack beyond AWS. That's a pity since it would be very useful to have the same ability with other providers. On the other hand, AWS alone is a huge challenge. There are quite a few AWS services that were not implemented or not yet stable or fully functional.
@@DevOpsToolkit I agree that AWS alone is a huge challenge having these run locally via LocalStack. Seeing their issue tracker a lot of AWS services like Lambda Functions (just to name one) needs improvements. But the tool has a lot of potential specially if the issues are addressed over time.
I could see this being an integral step in CI/CD pipelines now. Ensure that your IaC changes are successful on localstack, run tests, and deploy to production
Been using Moto since 2014 to do my AWS mock testing its a good mocking system and free. It only mocks API calls but can run Lambda code and store test objects in S3. Using it as Motoserver can have any SDK, Terraform or AWS CLI commands calling using the endpoints. Contributors welcome to extend its capability......
Looks interesting, thanks for sharing 👍. It's very disappointing to know that docker is still a prerequisite in order to use it. I think they should at least give the option to choose which container runtime to use.
For me as AWS consumer, most of the tests I would like to do simulate/emulate on my local machine mainly be around do/verify the security of my solution, understand the cost of my aws consumption rest like faking a db creation etc not sure make sense other than direct use from aws native api. When I checked what's in free tier none of them there although it says IAM included but not for security/policy enforcements :-(
@@DevOpsToolkit Thank you for your quick reply. Below is my scenario: Actually i have installed localstack as a pod in my kubernetes environment. My pod want to get the temp credentials from STS service.
does it have support for creating eks cluster using eksctl or we need to call the aws cli for creating infra labyrinth one after another to spin up an eks cluster? May be you have already started working on the XR for EKS using crossplane with localstack as the provider.
I tend to use Crosplane and Terraform most of the time so I haven't tried it with eksctl. A quick look at the params did not reveal the option to specify an endpoint (like what I did with the `alias` in the video). If there is no way to change it, eksctl will probably not work with LocalStack. As for Crossplane... We already added the option to specify the endpoint in ProviderConfig, so LocalStack and Crossplane can work together. The major problem now is that LocalStack relies on Docker making it not very useful inside k8s clusters. I will make a video about the combination, I'm just not sure when. Probably sometime in January. Also, I might publish it in the Upbound channel and not here.
@@DevOpsToolkit Another related update got deleted, I see a trend but that is not important here. From the list of registered services per your gist , I could only make s3 work , rest all failed citing no provider found although I installed localstack using the helm chart on a local k3d cluster, does it matter if the local k8s cluster is k3d, minikube, kind or rancher. It should be more friendly with docker daemon running underneath k3d but turns out sans s3 and to some extent ec2 all other services in the list you selected for demo failed. Especially disappointed with failure to provision RDS which worked for you. Failure with ecr and ekr are in line with your observation despite I am having docker running.
@@RideLikeAChamp I haven't tried it with k3d (yet). So far I used it with Rancher Desktop and GKE and those that worked in the video worked in both of those k8s distributions. That being said, LocalStack folks definitely need to transition from Docker-based solution to something that works in k8s (instead of just wrapping the binary into a container running in k8s but failing with too many resources). My understanding is that LocalStack team is aware of the need to improve it in k8s. However, I would still ask you to comment it in their Slack channel or, even better, create a GitHub issue. The more people report problems in k8s or complain about it, the more priority that will be for the team to work on.
@@RideLikeAChamp ... as for TH-cam comments... I honestly do not understand the criteria YT uses to choose what to keep and what to delete. For me, it's very painful since quite a few viewer's comments are deleted without any obvious reason.
@@DevOpsToolkit Open to discuss this / work on this with you if you like. I've got a rather advanced use case that I need to flesh out eventually. (lots of resources multiple terraform projects - folders -).
@@StephenRayner my main problem is time. When I look at the list of mandatory tasks I have to work on, I doubt I'd be able to get to that one any time before January or later. What I can do however is to get you in touch with the devs working on the project. Would that help?
@@StephenRayner That sounds great. We can do a video similar to those I do in this channel or we can do a webinar together with the LocalStack folks. They asked me to do something like that a few days ago.
throwing a dumb idea here... but... if you manage to do a symlink from.. let's say podman which should likely handle every docker command... Could that be a workaround for those services that don't work within kubernetes? I hope they are not trying to connect directly to the docker REST endpoint directly...
Those requests to Docker are happening on server-side inside a k8s cluster. That means that Podman or any other similar cannot help. If LocalStack is running in k8s, there cannot be any relation to Docker no matter whether that's through API, mounted sockets, or anything else. There is no Docker in k8s (except in the cases when people are prolonging the inevitable). As it is now, the only viable option is to run LocalStack directly or in Docker, but not in k8s. That's the main issue I have with it and I hope it'll be fixed soon.
It is true that Terraform and other IaC tools require time to learn. However, once you master them, they tend to save a lot of time, not to mention avoid random stuff from happening. The only case when such tools might not be a good idea is if it's a very small company with small operations and only a single person working on infra. The moment the number of resources in AWS scales, managing them all through the Web Console starts taking more (not less) time than with IaC. Also, I'm not sure that AWS is a good choice when dealing with a relatively small size. Something like DigitalOcean, Linode, etc. might be a better choice. AWS is complex mostly because it's based on building blocks rather than "final" solutions. That's intentional because larger orgs need building blocks to assemble exactly what they need but that results in elevated complexity that is made manageable through IaC. If that complexity is not needed, other vendors are a better choice.
@@DevOpsToolkit I did some experiments with localstack but was not satisfied. Also because I needed to restart it often for a test. At the end I went to mocking AWS in unit tests with moto. The benefit there is also a lot of unit tests which are also performed during codebuild. I will watch on localstack and hope there will be an alternative GUI from someone for AWS/Localstack which is also able to export all in cloudformation files.
I'm not sure I understood. Are you trying to create (simulate) AWS VPN? If that's the case, it is available (docs.localstack.cloud/aws/feature-coverage/).
For me trying to create the rds instance leads to this error: Unable to parse response (not well-formed (invalid token): line 1, column 0), invalid XML received. Further retries may succeed. Anyone can help why it is happening?
@@DevOpsToolkit I have created billing for 14 days trial but local stack does not generate the API key for me, Am I need to do any other step to get trial API key?
For me seems like they have a bug in the dashboard, they allow creating a trial plan without address and payment but they do not issue API key, if the user fulfills the address and card info later, the user is not able to peak the created plan!
@@mehdimohammadi1623 The team behind LocalStack made some changes to the API and it seems that there are some rough edges they still need to fix. Until then, you should see the API key if you go to Account > BIlling > Pro Edition - Free Trial.
That can indeed be very expensive if purchased for everyone. However, often only a handful of people need that license and in that case it's not that much.
Saw the video title and Subscribed! This the kind of content I want to watch on TH-cam
Localstack has also developed an AWS CLI wrapper call awslocal which will point the cli commands at your running localstack configuration. This is my preferred method over setting an alias.
Discovering your channel for the first time today - excellent stuff - subscribed!
This is exactly what I need. An AWS local Simulator / Mock / Stub. If it also calculates the cost of simulation. This would absolutely perfect to apply cost predictions.
Think this is the earliest I've been to one of your videos. Thanks for putting out quality content like this! Thanks to you I'm working on moving my company over to use kubernetes and you've helped show me tools to make that infinitely easier!
Do you think it might be useful to have functional mocks of AWS services running locally on your computer?
The latest release (shipped only a few days after the video was made) comes with support for ARM64. You can now run LocalStack natively on your Apple M1 - or even your Raspberry Pi! Please check out the 0.13.0 release notes in the Github repo for more details.
IMPORTANT: For reasons I do not comprehend (and Google support could not figure out), TH-cam tends to delete comments that contain links. Please do not use them in your comments.
Great video! In my opinion, local mocks are good since you can setup simple demo's of running AWS services and AWS solutions that you can show to clients or other people.
This is useful for small app demos or just testing before you commit to provisioning the services on AWS cloud itself.
One thing I wish to see if there is or will LocalStack support other cloud providers like Azure or Google. I checked their site and the tool is more catered to AWS services. So it seems it will really focus on that.
Still wishful thinking for other cloud services to have a tool similar to LocalStack to create mock services non AWS.
@@hamstar7031 From what I know, there is no ambition to expand LocalStack beyond AWS. That's a pity since it would be very useful to have the same ability with other providers. On the other hand, AWS alone is a huge challenge. There are quite a few AWS services that were not implemented or not yet stable or fully functional.
@@DevOpsToolkit I agree that AWS alone is a huge challenge having these run locally via LocalStack. Seeing their issue tracker a lot of AWS services like Lambda Functions (just to name one) needs improvements. But the tool has a lot of potential specially if the issues are addressed over time.
I needed these intricacies, thanks, Viktor
I could see this being an integral step in CI/CD pipelines now. Ensure that your IaC changes are successful on localstack, run tests, and deploy to production
... one model could be to use lovalstack with Para and AWS when merging those Pars.
Been using Moto since 2014 to do my AWS mock testing its a good mocking system and free. It only mocks API calls but can run Lambda code and store test objects in S3. Using it as Motoserver can have any SDK, Terraform or AWS CLI commands calling using the endpoints. Contributors welcome to extend its capability......
Amazing videos man! Please keep releasing more!
Great, I am going to pay Localstack, after setting up some trials
That sounds great. OSS projects need dedication. There is not dedication without income.
Looks interesting, thanks for sharing 👍. It's very disappointing to know that docker is still a prerequisite in order to use it.
I think they should at least give the option to choose which container runtime to use.
Agreed completely and transmitted that message directly to the team behind the project.
Great video!
Great overview. Too bad about the lack of features when running on k8s. Thanks!
For me as AWS consumer, most of the tests I would like to do simulate/emulate on my local machine mainly be around do/verify the security of my solution, understand the cost of my aws consumption rest like faking a db creation etc not sure make sense other than direct use from aws native api. When I checked what's in free tier none of them there although it says IAM included but not for security/policy enforcements :-(
are you going to do a new video on localstack 3.0?
Maybe later. It's not in my list for the next few videos.
Thanks!
Thanks a ton!
The starter paid license is $35/mo now and seems to have a lot of limitations still
And you need an annual plan for that price I think
Hi,
Is there any video on "localstack with STS" ?
I hardly see videos around using STS with localstack on youtube.
can you make one vide on this ?
I'm not sure I'll make more videos about localstack.
@@DevOpsToolkit
Thank you for your quick reply.
Below is my scenario:
Actually i have installed localstack as a pod in my kubernetes environment. My pod want to get the temp credentials from STS service.
Thanks ❤️
does it have support for creating eks cluster using eksctl or we need to call the aws cli for creating infra labyrinth one after another to spin up an eks cluster? May be you have already started working on the XR for EKS using crossplane with localstack as the provider.
I tend to use Crosplane and Terraform most of the time so I haven't tried it with eksctl. A quick look at the params did not reveal the option to specify an endpoint (like what I did with the `alias` in the video). If there is no way to change it, eksctl will probably not work with LocalStack.
As for Crossplane... We already added the option to specify the endpoint in ProviderConfig, so LocalStack and Crossplane can work together. The major problem now is that LocalStack relies on Docker making it not very useful inside k8s clusters.
I will make a video about the combination, I'm just not sure when. Probably sometime in January. Also, I might publish it in the Upbound channel and not here.
@@DevOpsToolkit Another related update got deleted, I see a trend but that is not important here. From the list of registered services per your gist , I could only make s3 work , rest all failed citing no provider found although I installed localstack using the helm chart on a local k3d cluster, does it matter if the local k8s cluster is k3d, minikube, kind or rancher. It should be more friendly with docker daemon running underneath k3d but turns out sans s3 and to some extent ec2 all other services in the list you selected for demo failed. Especially disappointed with failure to provision RDS which worked for you. Failure with ecr and ekr are in line with your observation despite I am having docker running.
@@RideLikeAChamp I haven't tried it with k3d (yet). So far I used it with Rancher Desktop and GKE and those that worked in the video worked in both of those k8s distributions.
That being said, LocalStack folks definitely need to transition from Docker-based solution to something that works in k8s (instead of just wrapping the binary into a container running in k8s but failing with too many resources).
My understanding is that LocalStack team is aware of the need to improve it in k8s. However, I would still ask you to comment it in their Slack channel or, even better, create a GitHub issue. The more people report problems in k8s or complain about it, the more priority that will be for the team to work on.
@@RideLikeAChamp ... as for TH-cam comments... I honestly do not understand the criteria YT uses to choose what to keep and what to delete. For me, it's very painful since quite a few viewer's comments are deleted without any obvious reason.
Is it like a sandbox?
Correct me if I am wrong.
It's more like a fully functional mock of AWS services that you can use for practicing, testing, etc.
Would like to see examples with Terraform
Adding it to my TODO list... :)
@@DevOpsToolkit Open to discuss this / work on this with you if you like. I've got a rather advanced use case that I need to flesh out eventually. (lots of resources multiple terraform projects - folders -).
@@StephenRayner my main problem is time. When I look at the list of mandatory tasks I have to work on, I doubt I'd be able to get to that one any time before January or later.
What I can do however is to get you in touch with the devs working on the project. Would that help?
We can compile the findings and turn that into a video.
@@StephenRayner That sounds great. We can do a video similar to those I do in this channel or we can do a webinar together with the LocalStack folks. They asked me to do something like that a few days ago.
throwing a dumb idea here... but... if you manage to do a symlink from.. let's say podman which should likely handle every docker command... Could that be a workaround for those services that don't work within kubernetes? I hope they are not trying to connect directly to the docker REST endpoint directly...
Those requests to Docker are happening on server-side inside a k8s cluster. That means that Podman or any other similar cannot help. If LocalStack is running in k8s, there cannot be any relation to Docker no matter whether that's through API, mounted sockets, or anything else. There is no Docker in k8s (except in the cases when people are prolonging the inevitable).
As it is now, the only viable option is to run LocalStack directly or in Docker, but not in k8s. That's the main issue I have with it and I hope it'll be fixed soon.
Re:invent 2021 announcement - Amazon LocalStack
I wouldn't be surprised if AWS does purchase LocalStack. That would be a relatively cheap acquisition.
The big problem is: there is no GUI. Setting up the env is a pain and you need to invest a lot of time in cloudformation or terraform.
It is true that Terraform and other IaC tools require time to learn. However, once you master them, they tend to save a lot of time, not to mention avoid random stuff from happening. The only case when such tools might not be a good idea is if it's a very small company with small operations and only a single person working on infra. The moment the number of resources in AWS scales, managing them all through the Web Console starts taking more (not less) time than with IaC.
Also, I'm not sure that AWS is a good choice when dealing with a relatively small size. Something like DigitalOcean, Linode, etc. might be a better choice. AWS is complex mostly because it's based on building blocks rather than "final" solutions. That's intentional because larger orgs need building blocks to assemble exactly what they need but that results in elevated complexity that is made manageable through IaC. If that complexity is not needed, other vendors are a better choice.
@@DevOpsToolkit I did some experiments with localstack but was not satisfied. Also because I needed to restart it often for a test.
At the end I went to mocking AWS in unit tests with moto. The benefit there is also a lot of unit tests which are also performed during codebuild.
I will watch on localstack and hope there will be an alternative GUI from someone for AWS/Localstack which is also able to export all in cloudformation files.
Is there a way to create a VPN and just use the local machine like you are in an EC2 instance?
I'm not sure I understood. Are you trying to create (simulate) AWS VPN? If that's the case, it is available (docs.localstack.cloud/aws/feature-coverage/).
I feel like this is good for students.
It's good for anyone who wants to practice or run tests (manual or automated) against AWS.
Is there anything similar for GCP or Azure?
I haven't seen anything similar outside AWS :(
For me trying to create the rds instance leads to this error:
Unable to parse response (not well-formed (invalid token): line 1, column 0), invalid XML received. Further retries may succeed.
Anyone can help why it is happening?
RDS is a Pro feature that requires an API key. Can you check whether you used a valid key as per instructions in lines 21 and 23 in the Gist?
@@DevOpsToolkit I have created billing for 14 days trial but local stack does not generate the API key for me, Am I need to do any other step to get trial API key?
For me seems like they have a bug in the dashboard, they allow creating a trial plan without address and payment but they do not issue API key, if the user fulfills the address and card info later, the user is not able to peak the created plan!
@@mehdimohammadi1623 The team behind LocalStack made some changes to the API and it seems that there are some rough edges they still need to fix. Until then, you should see the API key if you go to Account > BIlling > Pro Edition - Free Trial.
@@mehdimohammadi1623 You're right. It's a bug in the dashboard introduced by some recent changes. It should be fixed soon.
Viktor, what's in the box? :)
The one behind me? I don't think I ever opened it. My girlfriend put it there and I never bothered to check out what inside.
Hopefully not divorce papers :)
@@DevOpsToolkit 🤣
Can i create a VPC using this localstack?
As far as I know, VPCs are not supported directly. You can reference them in other resources, but you cannot create them.
Awesome product but crazy price for pro, $28 or $35 per user per month. :(
That can indeed be very expensive if purchased for everyone. However, often only a handful of people need that license and in that case it's not that much.
You have personified and glorified the role "Developer Advocate"
ancient meaning more than a year or two :')
ti si neki nas?
Jesam