Storing secrets CORRECTLY in .NET using AWS Secrets Manager
ฝัง
- เผยแพร่เมื่อ 15 ก.ค. 2024
- Get started with AWS for FREE: go.aws/3IRuSfW
Get the source code: github.com/Elfocrash/aws-videos
Check out .NET on AWS: aws.amazon.com/developer/lang...
Check out my courses: dometrain.com
Become a Patreon and get source code access: / nickchapsas
This video is sponsored by AWS
Hello everybody I'm Nick and in this video I will show you how you can work with secrets in .NET correctly and securely using AWS Secrets Manager. Secrets Manager has a lot of features that make working with it extremely easy and we are also going to use a third party package that makes the process even easier.
Give AWSSecretsManagerConfigurationExtensions a star on GitHub: github.com/Kralizek/AWSSecret...
Don't forget to comment, like and subscribe :)
Timestamps
Intro - 0:00
How configuration in .NET works - 0:39
Working with AWS Secrets Manager - 6:38
Taking a look at the SDK - 18:51
Social Media:
Follow me on GitHub: bit.ly/ChapsasGitHub
Follow me on Twitter: bit.ly/ChapsasTwitter
Connect on LinkedIn: bit.ly/ChapsasLinkedIn
Keep coding merch: keepcoding.shop
#aws #dotnet
![[LIVE] : ONE ลุมพินี 70 | คู่เอก "โฟกัส vs สเตฟาน"](http://i.ytimg.com/vi/_fVieQ89DRQ/mqdefault.jpg)
You must have acces for my browser history. Googled that an hour ago. Thank you for your video!
Thank you! Looking forward to any/all aws & dotnet content!
I learn so much from you in every video you bring to us!
Loving it. Thank you so much!
A tip. If you are setting user secrets on a unix terminal and that value has !, make sure to scape it because it gets interpreted by the shell. That happen a lot when setting connection strings where the password has special characters.
Yeah, I got around this issue by using single quotes in bash: dotnet user-secrets set "SecretName" 'SecretValueWith!'
Great vid Nick..thanks
The !! feature was stripped away from C# 11, you could touch that topic and give a few bits on decisions like this one
Thank you for sharing. Using SecretsManager for a while now and really love the concept of using a separate service for my secrets. Keep the AWS content coming.
Hi Wheeler, do you know how he sets up "dev" and "prod" and pairs it to "ConnectionString" in appsetting.json?. In the video it doesn't show where he gets those from but he's able to use them first-thing in the video.
Very well paced video! Do you have a video about authenticating developer machines for AWS / IAM ?
Is basically a credentials file in your ~/.aws folder.
Google AWS Configuration and credential file settings and it will walk you through it.
I will definetely like to see an AWS course
Hey, how would you use one secret key with many key value pairs containing the applications secrets? This would help with keeping costs down
Hi Nick, thank you very much for this video. I was wondering what would be the case if the Secrets Manager is unavailable or returns some sort of throttling error. How does the package deal with it? Is it possible to configure any resilience or retry pattern? For ECS or EC2 it would be fine, however for lambdas it would be important.
Hey Nick, thank you for the explanations. How the code knows which AWS environments to hit? All I was passing a AWS Region Endpoint. In my example it is East-1 for QA and West-2 for Prod.
I am not able to understand how nuget package able to download the secret by using a region endpoint?
Hello Nick, first of all great video love your content! I have a question, everything works as expected when running locally, however when I publish to my EC2 I'm still getting the default appSettings value in this case "prod". Any idea why this can be happening? Also I double checked and I'm authenticated against AWS through IAM creds.
Great video. Thanks for the information
Hey Nick, gr8 vid... quick question, how would this work in elastic beanstalk?
Oh I just commented that in another comment:
If you're using AWS Elastic Beanstalk, you can (kinda) set these LaunchSettings in the Environment properties: docs.aws.amazon.com/elasticbeanstalk/latest/dg/environments-cfg-softwaresettings.html
Reading it back in dotnet doesn't seem to work with the example they're giving (using ConfigurationManager.AppSettings) - but you can use this helper to merge it with your existing config: pastebin.com/wwzcbsty
@@ronsijm thx. Lying in bed with covid ATM, but will defo give that a shot
Works exactly the same way as demonstrated by Nick in the video. All you need to do is modify the default aws-elasticbeanstalk-ec2-role and ensure it has access to the secrets manager and hey presto, magic!!
I'm the author of AWSSecretsManagerConfigurationExtensions. Thank you for featuring my lib. Do you mind if I add a link to your video in the README of the repo?
Absolutely not, please go ahead and thank you for the library. You've done a great job.
@@nickchapsas It seems I need to add a tutorial on how to pronounce Kralizek tho :P
Hey this was a great video!
A slightly off topic question, for the nuget package you used (or in general), how do we know the code on github is the same as the package published through nuget?
You don’t. You have to manually check it, which is what I’ve done for packages like this. In fact for this one I am using a private fork because I needed to change some logic for my use cases
Thank you for this great video, but I have a question that's been on my mind. If a hacker were to gain access to my EC2 Windows server, implementing the solution of not allowing the connection string to be read from the appsetting.json file would prevent them from accessing it. However, the hacker could potentially use a tool like dnSpy to reverse engineer the code and extract the connection string. Using an obfuscator would also prevent the hacker from being able to read the connection string. So why would I need this solution?
What do you do if you have code that has secrets in the appsettings and you want to get rid of them?
Hi Nick, I have a question that hope you can reply me,
I'm following your video and its works on local but when I deploy it to production mode it got an error "Amazon.Runtime.AmazonServiceException: Unable to get IAM security credentials from EC2 Instance Metadata Service.". So how can I get a credential for my production mode? Tks for your time
Hey there. You would need to create a policy for your role that allows access from your EC2 to the Secrets Manager
Hello Nick, I hope you are well, you see I have the following error when starting the app, Unable to get IAM security credentials from EC2 Instance Metadata Service., I know I need to configure the iam role but I can't find exactly how to do it, do you have any resources that I can follow
Greetings from Mexico
Hi Nick, awesome vid,
Just having a bit of trouble making it work by deploying a docker container to AWS ECS.
Getting the error: Unhandled exception. Amazon.Runtime.AmazonServiceException: Unable to get IAM security credentials from EC2 Instance Metadata Service.
Do you have a video that goes through getting a db connection string from either this secret manager or ssm parameter store while running a container on aws ECR,
Would like to know the best practices for AWS, cause at the moment what I can think of is only giving the credential of an IAM user with only getparameter access, but still it would mean showing the secret key in code.
I don’t but you don’t need a connection string. You need to create a policy which allows your ECS task definition to see the secrets
How does this compare to aws parameters and app config?
Seems to be able to do almost the same thing but is more expensive?
Parameters store and app config is for general parameters and app configuration. Secrets manager is specifically targeted towards secrets and it has features that you’d only need for secrets
I think you did a great job here, but I have 2 issues. First, using pattern matching for names can be error prone. A simple naming error can cause problems. Second, from a security standpoint, you should not be grabbing all the secrets and discarding them. It is more secure to grab only the ones you need by name.
The filter will only grab the ones you need. It’s a server side filter. Not having a pattern causes conflicts. It’s a very standard practice. If you had a type the secret wouldn’t be loaded in the first place and the app wouldn’t start
Hi Patrick, do you know how he sets up "dev" and "prod" and pairs it to "ConnectionString" in appsetting.json?. In the video it doesn't show where he gets those from but he's able to use it first-thing in the video.
One of the very first things I built in my very first proper dev job was a program that had to keep the Dropbox API key a secret. Since a .NET program can be reverse engineered, that's no good. If for example you keep the secret in an encrypted sqlite database, now you have to keep the database password safe. If you hardcode it in your code, it can be discovered.
These secret managers i feel suffer from the same problem. You put your secrets in a secret manager, but then where do you put you secret manager credentials? Where previously you had to keep your secret safe, now you have to keep the credentials for you secrets manager safe. What am I missing?
You don't need credentials. In AWS you authenticate using IAM Permissions and security policies. You define that X microservice has access to Y AWS service and AWS knows who to allow in and who to deny access to based on that.
@@nickchapsas Thanks for the reply, Nick. Honestly I really appreciate the fact that you read the comments and take time to reply. TH-cam's comments are not exactly the best thing for a person's mental health so i hope you do take care of yourself.
Anyway, i should have clarified that i'm talking about a plane old desktop app. For example if a microservice is running on the cloud and you need to talk to it, you probably need some sort of API key or credentials. Now how are you going to keep those credentials safe? I ultimately ended up believing that managed languages simply can't keep secrets though i would love to be proven wrong.
@@superpcstation You can't securely authorize a desktop app directly to the secrets manager. You'd have to add some form of user authentication and then build an API that the app communicates to and that API is the authorized one. There are always ways but they are relative to your usecase.
@@superpcstation
This is not unique to managed languages. Any software can be reverse engineered, and if that software has credentials within, they can be extracted.
Would be nice if you would explain, why you did not need to provide credentials at 12:19. Especially, when the app would be in production...where are the credentials for the cloud? This is usually always a loophole, where you need to store credentials somewhere, just to access the secrets from the cloud.
There would be no credentials. Your pod (k8s), task (ECS) or VM (EC2) will be authed against the service via an IAM role that defines what the service can and cannot access.
@@nickchapsas I also would like to know how can i authenticate my production VPS (ubuntu) to access aws secrets
@@Qrzychu92 No they don't need to. You just set up access control on EKS. Here is the process: docs.aws.amazon.com/secretsmanager/latest/userguide/integrating_csi_driver.html
Nicely explained. The only thing I'm skeptical about is the 3rd party nuget package for configuration. I could see a potential supply-chain attack on a library maintained by a solo developer.
If you are genuinely worried about that then there are two options I can see you taking.
1) Fork the code or clone it to you machine, read it thoroughly and reference your copy where you fully understand and can ensure the code is not doing anything dodgy.
2) Pin the Nuget version to a specific version that you have code reviewed and are happy taking. If you target that specific version then any further updates to the codebase should not affect you.
@@oliverrivett-carnac5564 Thank you for this response. @Dalibor's concern was the same as mine.
Hi Dalibor do you know how he set up "dev" and "prod"? In the video it doesn't show where he gets those from but he's able to use it first-thing in the video.
Hello guys?
Can someone help me with this error here?
"is not authorized to perform: secretsmanager:listsecrets because no identity-based policy allows the secretsmanager:listsecrets action"
It's working when I'm running my API local and getting the secret from the AWS, but it's not working when I publish the API to AWS
Hi Nick, I am trying to solve a problem when loading secrets from AWS that change.
In the DI i use objects that use values from those secrets, and sometimes those are added as singletons.
How do i get those objects updated with the new values?
For example, i have an sqlsettings object that is loaded with values from AWS:
builder.Services.AddOptions().BindConfiguration(nameof(SqlSettings)).ValidateDataAnnotations().ValidateOnStart();
builder.Services.AddScoped(resolver => resolver.GetRequiredService().Value);
And later add the SqlClient
builder.Services.AddSingleton();
Where SqlClient constructor uses the SqlSettings to initiate a connection (or something like that)
After a change the secret value, the SqlSettings object changes, but the SqlClient does not.
Is there a solution to this, or is this singletons working as intended?
Is there always free services that we as developers can use, like services that do not have a trial an always free? If so can you please make a vid about those services and if they are usefull and how they can help us pleasee
I only make videos on things I have used in production, for a long time at scale. There is probably some free version to manage secrets too but I am not aware of it and I won't be making videos on things I am not proficient at.
For secret management you can use Hashicorp’s Vault.
Hey Nick Chapsas, can you do a similar video for Azure?
I have one on Azure KeyVault already
I am very beginner building c# windows form application with firebase database || how to secure my connection please help !!
Guyz I am trying to integrate the secret Manager on on premise web server…. We have jboss eap which is connecting to cloud database now I want to mask the id and password using secret manager on the on prem server… have tried multiple method online but nothing seems to work ……does anyone have document for this will by much help
Can these be accessed in ci cd Pipelines
Sure if you auth the pipeline to your environment
14:58 - MY approach?!
This video was dedicated to you Colin
@@nickchapsas I will remember this
If u can have a different implementation of that SecretManager packager, u can basically store ur secrets in your database(or seperate db) in a seperate table. So the question here, why do we need this AWS Secret Manager? Obviously your doesn't know where ur secrets are.
Why not just use something like table-storage or RDS or something else? What is the key benefit of SecretManager?
Because Secrets Manager is built to store and load secrets in a secure way and it also gives you features such as the secret rotation out of the box. If there is a breach in your database and you have the keys in there you are basically done. Also, how do you get access to a database when the keys for the database are in there? There are too many factors you need to take into account to make such an implementation yourself. It also gives you super fine-grained polices for managing the access to the keys and things like secure audit.
@@nickchapsas If you are all in within AWS, you can use IAM roles to access your resources instead of secrets. I did a .NET POC of this years ago where SQL RDS was using IAM roles as credentials instead of the standard username/password configuration.
Actually, is there ways to safe store stuff in mem?
Not really. As soon as they are in memory they will need to be accessed and used. Even if they are encrypted and decrypted in memory you can't guarantee that they key won't be there too
@@nickchapsas Thanks Nick, I was thinking of things like securestring which is now deprecated. I wonder is there anything in win API...
key vault master race
...and when AWS goes down, so does your app, no matter where you hosted. How do you mitigate this risk - use Azure KeyVault in parallel?
AWS isn’t a single entity. You have georeplication in place for high availability
i could understand the improvements on using the secret manager in local development, but in production where i use containers and k8s and my env is stored in an ,env file, why should i change to the aws secret manager?
Because you shouldn't store those in an env file, that's dangerous. The env files should point either to the secrets manager ARN or use K8s' secrets managers directly
@@nickchapsas got it, thanks for the explanation
Why don't you use Azure Key Vault?
Because I am not on Azure. If I was on Azure I would use Key Vault
@@nickchapsas Thanks for your answer! I think my question was more related to "why you chose Aws instead of Azure", a poor selection of words by me, I'm sorry 😅
@@victorgarcia3526 It's ultimately not a developer decision but a business one. Both offer similar services with small differences in functionality. Judging factions will be some cloud exclusive features (which are usually rare), business relations and pricing. Developers don't make those calls.
Press 5. It's essential that you know what the name of this package is.
No it's not. Context is way more important and understanding why we will do what we will do is way more important. The package is irrelevant
@@nickchapsas Sounds like someone didn't press 5
@@notmeprobably_ Wait was this a joke because I couldn't pronounce the name? 😂
Hi
One question, i saw you previous video th-cam.com/video/J0EVd5HbtUY/w-d-xo.html title "Managing your .NET app configuration like a pro". Both are solving same problem which is loading secrets securely in application. Can you please share yours though which one should be used over other in which scenarios.
They don't solve the same problem. AWS Parameter Store is, as the name suggests, a general parameter store and all you app's configuration can go in there. Secrets manager is built for secrets management and it involved encryption and native secret rotation. They solve different problems.
@@nickchapsas When creating new parameter In types we have option "SecureString" which will be encrypted using KMS. We could have plain text as well as encrypted string. Secret Manager is also encrypting using KMS. Please bear with me if my question sounds silly. Thank you.
Too lengthy lectures, please try to keep it short and straight
The length is relative to the information that is needed for people to understand the full context. If it was shorter it would be a bad video
The new thing is "cloud provider independence".
If you have libraries to support AWS features in the stomach of your codebase, you are dependent in a way your don't wish for your codebase.
What we've seen with Parler and other services hosted at AWS, they were illegally shut down, censored, for no other reason than political.
Supporting AWS after knowing this is one thing, but what if you get shut down? Can you take your codebase and host it another place? -If you wrote your code in an appropriate way, yes you can. Any service in fact. But using AWS specific features? No. You will have to re-write large parts of your codebase, and wait weeks if not months, before you are back online.
This is even worse for Firebase developers. I don't understand them.
You are missing the point. This isn't about supporting AWS, Azure, GCP or whatnot. This is about people who are getting into AWS or are already in it and they want to know how to do this. People can talk about "cloud provider independence" all they want but good luck finding a cloud agnostic version of DynamoDB or CosmosDB. And no, MongoDB or Couchbase are both a nightmare to manage and scale and I'm happy I don't work with them anymore. You think that Microsoft can't stop supporting aspect of .NET at any point? Where is .NET Framework? Where is WinForms? Where will Blazor be if it keeps being niche? This "tech independence" on any level is a lie people are believing.
@@nickchapsas I think his idea is creating libraries that are provider agnostic and easier to replace providers using the abstractions you create during development. Provider locks is also something I am tense over.
@@nickchapsas Agree, the concept here is "do not store secrets in your appsettings", you have showed one of many ways to do it and this can be easily rewritten if needed. Anyway, having a package and a few lines in your startup project ins't a vendor lock-in.
@@mateusnc It's probably the least invasive way to implement this. You can look at the Azure KeyVault equivelent video that I have out and the setup is basically identical since they're both built on top of a configuration provider
Xactly. While I totally agree with Nick that this was not at all the point of the video, you do make a valid point about the conflict between commercial profit and a cloud agnostic consumer friendly standard for a type of service. In the last 40 years in the business, every time I have seen a proprietary solution, within 2 years it has turned out to be an expensive option from which it can be hard to escape. It is the cost plus mentality.
Azure > AWS
This video is great, however, it would nice to know how to load that secret when you register your servces.AddDbContext(), instead of using it in the ApiController.
It’s the same since it is loaded in the configuration provider. You just point to the config name and it will be loaded assuming your app has access to secrets manager
Hi, have you find the solution for that?
Can you do a similar video but about azure key vault?
You don't need to replace __ to :. Dotnet eats double uderscore well as I know, at least it does it for my k8s secrets injected as a env variables.
.NET does indeed accept double underscores but for some reason it wasn’t working in one of the demos I did so I went with the approach that was working for all my examples