Also, a fun tale from inside Crapital One. The company decided to yeet their Microsoft software licensing agreement in favour of Google's services plus Zoom, because the execs of Microsoft and Cap1 fell out over a game of golf. Really gives you an insight into the minds of these corporate (b/w)ankers.
@@langmod What is Google’s video conference solution today named? It’s not like Teams is any good, but at least they don’t replace it every couple of years since they decided that they killed Skype dead enough.
I remember joking in a meeting of my fellow architects (at a company that processes a lot of personal/financial data) "It's a good job the $regulator is so underfunded". The laughter died away pretty quickly as we all looked at each other. Too many people in senior positions will see IT security purely as a cost.
Some are really good though but they're so good, they're too restrictive and you can't get anything done without going through a million different people and approvals and when things break... Well, good luck trying to troubleshoot where you're being denied.
Many companies are opening their eyes. My org uses something called the zero trust model paired with least privilege. That means you have to authenticate for everything you do, and if you need to do an administrative task you need to request specific permission in order to do so. Even devops are locked out. The idea is if you do get compromised the damage should be limited to that machine only. We actively scan for PII and PHI on the workstations, and any offender is immediately flagged. We have several layers of access before you even get into our network, and if you do happen to ingress somehow your ability to do damage is severely curtailed. We have put about 20% of our budget into security - hiring top level security engineers with experience with integrated systems (we have data centers too - not everything is in the cloud - and the same escalation is required to access anything on prem). Every single application that goes out into the wild (even if it's an internal application) has to pass a stringent security review where they review things like the access model I discussed previously, how data is transported, how and where it's stored, and other pertinent details around the proposed solution. We actively scan our code base and our web sites for vulnerabilities on a constant basis. We have started to incorporate red/blue teams as well.
Unless some material facts are missing from this video, if I was auditing this, I'd put the blame entirely on Capital One. That is not a reverse proxy. It looks like a simple HTTP proxy, and a blind, fully trusting one at that. I don't think a network TTL of 1 would've protected them. The incoming TCP request would've terminated at the proxy, and it would've been a new connection between the compromised server and the metadata server. The change to PUT would've probably worked, but developers that make "convenience choices" like creating this proxy, also do stupid things like "damn AWS doesn't let me GET, imma proxy it to PUT". Like I said, unless something huge is missing, it's entirely Capital One's fault. But they're a huge customer, AWS would make changes to allow customers that size to make stupid mistakes and still mitigate the loss. Azure is HUNGRY. I was only running an account with a couple of mill worth annual usage a few years ago and Azure sales guys were calling me to meet every couple of months.
Wow, I worked at C1 as a swe intern few months after the attack, but the company wouldn’t tell us what really happened to this level. Thanks for the info!
Dumb little visualizations like that are fantastic for actually getting the point across. Computer infrastructure terminology gets super abstract sometimes
@@ENCHANTMEN_ And it sometimes feels like trying to pull meaning out of a bunch of colored rectangles spilled all over a sheet of paper that got framed in a museum.
All the features in AWS are pretty well documented, they even explain how things works inside so you can understand the tools better. One of the biggest drawback of AWS is the lacking of advanced examples. They just provided the most simple example, so you may have a hard time to figure out what do they actually do, especially cloudformation and python aws sdk. They are well documented but painfully abstracted so you have to try and retry again and again to make something work as intended. It’s like they give you a death star lego set of 100000 lego pieces, with just a few pages to build some simple construction so you have to look at the images in the box to build the complete one.
Fair point. Then again almost no SDK, let alone any language contains advanced examples. Think about the Stream/Collection Javadocs and how often those two can be used together, but Oracle chose to give you only a holistic picture of what’s possible. There’s only so much to teach before you have to apply it yourself.
The thing is AWS being dominant became ignorant between 2016-2021. I work on multi cloud setup and AWS is the one that I am least interested to work with.
@@nemesisprime6727they are the most popular. You know, popular does not mean the best, like js and mongodb, very popular but scaling them is a true nightmare
Just finished a binge of a ton of your videos. Keep up the grind my brother and you will 100% have a thriving career as a youtube creator. These videos are clearly really high effort and also just good (those things arent necessarily correlated).
Bro your channel should be going places. I found it through your Cloudflare vid (of course) which currently has 1M views and idk how more people aren't subscribed. Really top notch content!
Claiming AWS is responsible is kind of ridiculous. Of course it could be done better and they DID improve it. Only because engineers are lazy and give too many permissions. I’ve seen it myself of course but I think it is lack of good practice or maybe outsourcing ,, and people just try to get something working by throwing more and more permissions at it instead of the more time consuming process to look for the root cause and do it properly.
AWS somewhat created a vulnerable point in their system, which could be avoided. It’s like a mom store the tide pods with candy packs, telling her children that tide pods is not edible. Somehow one day the child ate the pods instead of candies and go straight to the coffin. Yeah we can easily blame the child to be not careful enough, but the mom could have prevented that in the first place if she put the tide pods in a safer place. That’s why in everybox of liquid detergent said “keep away from children”. So AWS partly have some responsibilities in this case.
@@aperture147 It's like the grocery store has both tide pods and candy packs in the store and you're blaming it for having them both in the store instead of making people go to a separate store to get tide pods.
Just wanted to say - you do a great job breaking these events down and producing them - I hope you get time to make more - I find the malicious ones the most interesting, but even fail over fails are fascinating (probably most of us working on the periphery of the IT sector do too)
Couple of years ago my account was hacked. Fraudulent charges notifications so i called Capital one. They shut down my card but the fraudulent charges were still happening as i they shut down the card. They had the nerve to ask me if i gave my card to someone and I’m like, “you dumbasses, you just shut down my card so how are the charges happening as we speak?”
I love how the gist literally said: "Warning: use of these commands will get you arrested by the FBI, user discretion is advised" 😂 And there was me thinking it was a rookie mistake making such a script public...
I think it would have been quite clever to share the script had he used a more discreet account. A bunch of people downloading the data through Tor could create a lot of work for the investigators and thus help hide his identity.
Someone honestly should go through AWS (and other providers) list of IP addresses and attempt to get the instance ID. If you can, report it to the cloud provider so hopefully they can inform the customers that are affected.
I worked a contract there as a Sr. SW Engineer. Was never told I was put on a team/project for which I interviewed. Was supposed to be doing design/implementation of some new micro services. I start day one and the project manager didn't know I was coming on board and we had never spoken. I spend the day getting my env set up blah blah blah. Then they started pulling tasks off of the 'Agile' board and point me to the code base and it's a complete clusterfk of code that was about 2 yrs old and nobody was around who understood the problem domain. absolutely no discipline (SOLID, DRY, etc) was used. Thousands of lines of 'copy pasta'. Automated tests (Cuke or whatever?) that didn't pass simply had the input data commented out. AND, best of all, I'm told that the team is responsible for setting up AWS S3 and servers, networking, etc. No dedicated DevOps people. I don't do that stuff and when I've done it in the past, only in a 'dev' env. and not in 'prod'. Two other TBTF banks were bad as well, but nothing like CapOne. Un-real.
The real question is how the guy got away with it. That's a pretty textbook CFAA violation. I kinda died a little the second I saw that IMDS forwarded URL, anyone who's dealt with this before knew immediately what happened.
Dude this is one of the most informative yet hilarious channels I've come across related to cybersecurity. Awesome job. Love the in depth details of actually went wrong instead of just broad "got hacked" verbiage.
All the features in AWS are pretty well documented, they even explain how things works inside so you can understand the tools better. One of the biggest drawback of AWS is the lacking of advanced examples. They just provided the most simple example, so you may have a hard time to figure out what do they actually do, especially cloudformation and python aws sdk. They are well documented but painfully abstracted so you have to try and retry again and again to make something work as intended. It’s like they give you a death star lego set of 100000 lego pieces, with just a few pages to build some simple construction so you have to look at the images in the box to build the complete one.
Worked for a client that did work for Capitol One, prior to 2019. Capitol One was by far the most strict partner that we dealt with. Everyone complained about having to follow Cap One's processes and procedures but what we realized that it was for our own good.
I say capital one's fault. They're using the service and AWS is only responsible for accurately telling them what they get and don't get. If Amazon guarantees something or misconstrues what they provide or fails to provide, it's their fault, else it's not. If Amazon says they get 99% uptime and they get 99% uptime, it's on the customer. That being said, sticky situation and one could make a case for either. I like the braking analogy as it's definitely a spectrum for what's "expected"/"reasonable" for example automatic advanced braking system vs just a working one in general. The difference is just the scale and what's "reasonably expected"
AWS should have two types of S3 buckets (public immutable, and private immutable). And that would solve a lot of problems. What I see happen is devs get confused by all the security configurations for S3. This isn't an excuse but I'm just saying what I see happen. The problem is that a private bucket can be changed to public.
The amount of documentation you need to read to get the right permissions are just ridiculous if you don’t know what your doing. And most of devs who set this up aren’t experts in cybersecurity so it’s hard. That’s why pen testing is so damn important, even if it’s bloody expensive.
the process to unprivate a bucket is lengthy in itself. you need to uncheck/deselect/disable varying options across their submenus. all buckets are locked down by default with plenty of warnings screaming if something is public. company needs to also do their due diligence and actually prevent + detect anything thats been exposed to the internet.
S3 buckets are private by default for good reason. I really can’t see many good reasons to make a bucket public. Part of the billing is data transfer so allowing anyone to consume as much content in a bucket as often as they please is going to result in hefty bills.
I was interviewed several times by Captial One about 5 years ago. I was a certified solution architect and I had put in my resume at some point. They were really trying to poach me, but three minutes into the interview I knew it was a clown college. Regarding AWS security - they are only responsible for data inside their network. They tell you this, and it's part of the practitioner and solution provider exams. If you are 100% serverless then AWS is 100% responsible for your data. As soon as you agree to manage your own server via an EC2 instance then you are responsible. Honestly I don't know why any org would need an EC2 instance when ECS is a viable alternative, and makes scaling zero effort. Scaling EC2 instances can be done, but it takes work, and it's susceptible to all the problems a non-managed solution has.
that's not how shared responsibility work... AWS responsible for the underneath infrastructure. so it is true if it is serverless AWS responsible the server that run the software. but the IAM permission, the code you run inside serverless is still responsible by the user who create it
blaming aws for this is ridiculous. It's not their job to forsee every possible stupid decision their customers could make. If someone set that bucket to public global unencrypted access, it wouldn't be AWS' job to shut that all down
No. The problem is you'll get a pair of enterprising folks that will create trivial problems, then report them to get rewarded. Then someone will make a flaw they think is trivial, but turns out to be serious, and by the time it is corrected things go to hell very fast.
They should; it would incentivize people more, but the FBI and others will pay millions to get some of these exploits; there's no competition if people do it for the money.
I have no idea what the hell that video was about. You engineers make civilized life possible and don’t get an iota of appreciation from the rest of us.
The name of the VPN service "Ipredator" is.. one could call it "unfortunate", but it's probably deliberate. The service was created in direct response to the EU IPR directive, also known as IPRED, and the subsequent Swedish Intellectual Property law commonly known as the IRPED-law, which was basically about combating software piracy in general and, some would say, specifically made to attack The Pirate Bay.
Even though AWS isn't responsible for this, it still upgraded IMDS to try and help its customers from making mistakes - not because it was responsible. It didn't have to do this, but it did.
Simple Storage Service... no security concerns here, it is just a "simple" service going on... Looking back at some of these leaks that involved S3 and AWS where companies "rushed" into selling the cloud idea to their senior leadership, seems like people had their own set goals to get the biggest bonus possible, and sell fancy terminology how company is modernizing, how company is adopting intelligent technology, etc. - but behind all that "fancy" is simply the same technology that was available before - just now, you pay the company to host it for you instead of building your own data center... hence, since you are "contracting out" that piece, it is inevitable that once again convenience comes at the price of security... So how does this happen? It happens when AWS tells the company this service comes with the shared responsibility - AWS is responsible for a piece of it, while the company handles a piece of it... in other words, unlikely that AWS will do something wrong, as they are in business in providing this up to a certain level and you get the whole encyclopedia of it what they do... companies??? Apparently not so much in CapitalOne case... Too bad that data breaches continue to happen, and penalties and fines companies end up with are nowhere near realistic ones to make a difference... "it's a speeding ticket" given their profits that measure in billions each quarter...
Glad AWS made some patches to help mitigate this. However this was 100% the fault of Capital One. Insecure software. An instance designed to serve public data had an IAM role to get and decrypt all data across all S3 buckets. AWS denies access by default. Allowing open access to S3 is deliberate.
Some AWS consultant making $175k per year helped to cause this blunder. That's what pisses me off. 15 years in IT, never making big money and never making blunders that jeopardized users.
Some thing amaze mee is that who uploaded cred file to git repo and who made git repo public is operational team forgetting the severity of information
1:43 - "The engineers tested the commands" wait wat, I hope they tested it in a clean vm isolated from everything including not being in same network address range as corp network right?
They could have done it in their private network but still someone probably used the commanda in another network. Even if you are in the company's private network, you still not be able to access all the data.
No mention at all for the responsibility AWS had to review/remove vulnerabilities and access when employees are termed. We have been doing that loosely for more than 2 decades, but now there is a checkbox that initiates a thoroughly organized team review that includes a think tank style catch-all: "worst case / what else". We routinely have meetings that start: "as you all know Timmy is gone and we will be updating everything". If neither of these shops do that; shame, shame, shame.
Because AWS isn't responsible. SSRF can be performed by anyone as long as they can connect to a webserver that is vulnerable to it. The instance metadata server is only accessible from inside of the instance. If someone is able to obtain data from a publicly accessible website from the imds server, then you haven't secured your webserver very well. I don't have imdsv2 enabled on my instances but then again my web servers aren't vulnerable to it and my instance profiles are fairly well locked down. Having said that, if I do live to regret my words, it's on me. I'm not going to go crying to AWS... I don't pay for support anyway..
If you rearend someone because your breakline broke, then the company would be at fault, plus the company pays more so they should be targeted with the blow back. I think that it should be illegal for a company to save any sensitive information of someone for any reason besides needing it in that moment, the second that information leaves then they should be deleted. As for a punishment, i would not mind if every CEO and higher up who personally signed off on launching the site had every personal detail about them leaked, and allow the affected customers to be able to leak any new information whenever they want to, with it only stopping once every customer affected decides they're ready to. We need to be as cruel to big multi-national companies as possible
every financial company and credit bureau that's on AWS have been hacked. I wont say how because I still have Non-Discloser but i will say this who ever gets AWS jump ship fast.
How can it be AWS's fault when an employee of Capital One hired to set up their servers configured a role that could do anything on a public server AND apparently didn't configure the firewall at all? Lmao
![[MV] A leap of faith...ความเชื่อใจครั้งสุดท้าย (From "Petrichor the series")](http://i.ytimg.com/vi/U1piZH2CNXA/mqdefault.jpg)
Also, a fun tale from inside Crapital One. The company decided to yeet their Microsoft software licensing agreement in favour of Google's services plus Zoom, because the execs of Microsoft and Cap1 fell out over a game of golf. Really gives you an insight into the minds of these corporate (b/w)ankers.
Truly we live in a meritocracy.
Lmao, this is hilarious. Do you remember where you read this?
tbf anything is better that the MS Teams ecosystem.
@@langmod What is Google’s video conference solution today named? It’s not like Teams is any good, but at least they don’t replace it every couple of years since they decided that they killed Skype dead enough.
@@mikhailryzhov9419 Zoom
The sad part is that you could hardly look beneath the hood at any large or tech company and NOT find this kind of disaster waiting to happen...
Just imagine all the public exploits that have been published without the companies affected being any wiser ✡️
I remember joking in a meeting of my fellow architects (at a company that processes a lot of personal/financial data) "It's a good job the $regulator is so underfunded". The laughter died away pretty quickly as we all looked at each other. Too many people in senior positions will see IT security purely as a cost.
@Generaal the amount of unsupported servers with minimal backup plans is mind boggling 😂
Some are really good though but they're so good, they're too restrictive and you can't get anything done without going through a million different people and approvals and when things break... Well, good luck trying to troubleshoot where you're being denied.
Many companies are opening their eyes. My org uses something called the zero trust model paired with least privilege. That means you have to authenticate for everything you do, and if you need to do an administrative task you need to request specific permission in order to do so. Even devops are locked out. The idea is if you do get compromised the damage should be limited to that machine only. We actively scan for PII and PHI on the workstations, and any offender is immediately flagged. We have several layers of access before you even get into our network, and if you do happen to ingress somehow your ability to do damage is severely curtailed. We have put about 20% of our budget into security - hiring top level security engineers with experience with integrated systems (we have data centers too - not everything is in the cloud - and the same escalation is required to access anything on prem). Every single application that goes out into the wild (even if it's an internal application) has to pass a stringent security review where they review things like the access model I discussed previously, how data is transported, how and where it's stored, and other pertinent details around the proposed solution. We actively scan our code base and our web sites for vulnerabilities on a constant basis. We have started to incorporate red/blue teams as well.
Yes, we leaked your Social Security Number. Yes, we paid less than 2$ per SSN leaked. No, the $2 doesn't go to you, it goes to the state.
Unless some material facts are missing from this video, if I was auditing this, I'd put the blame entirely on Capital One. That is not a reverse proxy. It looks like a simple HTTP proxy, and a blind, fully trusting one at that. I don't think a network TTL of 1 would've protected them. The incoming TCP request would've terminated at the proxy, and it would've been a new connection between the compromised server and the metadata server. The change to PUT would've probably worked, but developers that make "convenience choices" like creating this proxy, also do stupid things like "damn AWS doesn't let me GET, imma proxy it to PUT".
Like I said, unless something huge is missing, it's entirely Capital One's fault. But they're a huge customer, AWS would make changes to allow customers that size to make stupid mistakes and still mitigate the loss. Azure is HUNGRY. I was only running an account with a couple of mill worth annual usage a few years ago and Azure sales guys were calling me to meet every couple of months.
@@asanokatana AWS did actually make a couple of changes on the back of the Capital One incident. Some of it was discussed in the video.
@@nachoIibrethey made (pretty simple) changes because it’s bad PR even if it’s not their fault
"pushing"
>Puts a picture of deadlift
"And pulling"
>Puts a picture of bench
Someone's a little confused, but they got the spirit!
2:35 I saw this at the same time that part started playing lmao
Wow, I worked at C1 as a swe intern few months after the attack, but the company wouldn’t tell us what really happened to this level. Thanks for the info!
I'm enjoying these videos a lot, they're informative and have some fun editing lol
How do you listen to this AI voice it just sounds odd
@@Henry-zw4xs is it the tone or speed? I put it on 1.25x 😅
@@Henry-zw4xsThe voice is perfect for the style of commentary and editing imo.
Dumb little visualizations like that are fantastic for actually getting the point across. Computer infrastructure terminology gets super abstract sometimes
@@ENCHANTMEN_ And it sometimes feels like trying to pull meaning out of a bunch of colored rectangles spilled all over a sheet of paper that got framed in a museum.
All the features in AWS are pretty well documented, they even explain how things works inside so you can understand the tools better. One of the biggest drawback of AWS is the lacking of advanced examples. They just provided the most simple example, so you may have a hard time to figure out what do they actually do, especially cloudformation and python aws sdk. They are well documented but painfully abstracted so you have to try and retry again and again to make something work as intended. It’s like they give you a death star lego set of 100000 lego pieces, with just a few pages to build some simple construction so you have to look at the images in the box to build the complete one.
Fair point. Then again almost no SDK, let alone any language contains advanced examples. Think about the Stream/Collection Javadocs and how often those two can be used together, but Oracle chose to give you only a holistic picture of what’s possible. There’s only so much to teach before you have to apply it yourself.
The thing is AWS being dominant became ignorant between 2016-2021. I work on multi cloud setup and AWS is the one that I am least interested to work with.
@@nemesisprime6727they are the most popular. You know, popular does not mean the best, like js and mongodb, very popular but scaling them is a true nightmare
This was very well put.
@@drakedoss1975 ah yes, Stream API and Collection framework, classic security risk mine.
your channel is criminaly underrated
I work at the company and this video explained why C1 has so many controls I've had to deal with in my day to day job.
Just finished a binge of a ton of your videos. Keep up the grind my brother and you will 100% have a thriving career as a youtube creator. These videos are clearly really high effort and also just good (those things arent necessarily correlated).
How to find videos similar to these?
Bro your channel should be going places. I found it through your Cloudflare vid (of course) which currently has 1M views and idk how more people aren't subscribed. Really top notch content!
This channel kicks butt. You're going to go places dude if you keep up with this content.
Claiming AWS is responsible is kind of ridiculous. Of course it could be done better and they DID improve it. Only because engineers are lazy and give too many permissions. I’ve seen it myself of course but I think it is lack of good practice or maybe outsourcing ,, and people just try to get something working by throwing more and more permissions at it instead of the
more time consuming process to look for the root cause and do it properly.
AWS somewhat created a vulnerable point in their system, which could be avoided. It’s like a mom store the tide pods with candy packs, telling her children that tide pods is not edible. Somehow one day the child ate the pods instead of candies and go straight to the coffin. Yeah we can easily blame the child to be not careful enough, but the mom could have prevented that in the first place if she put the tide pods in a safer place. That’s why in everybox of liquid detergent said “keep away from children”. So AWS partly have some responsibilities in this case.
@@aperture147 It's like the grocery store has both tide pods and candy packs in the store and you're blaming it for having them both in the store instead of making people go to a separate store to get tide pods.
Just wanted to say - you do a great job breaking these events down and producing them - I hope you get time to make more - I find the malicious ones the most interesting, but even fail over fails are fascinating (probably most of us working on the periphery of the IT sector do too)
Couple of years ago my account was hacked. Fraudulent charges notifications so i called Capital one. They shut down my card but the fraudulent charges were still happening as i they shut down the card. They had the nerve to ask me if i gave my card to someone and I’m like, “you dumbasses, you just shut down my card so how are the charges happening as we speak?”
I love how the gist literally said: "Warning: use of these commands will get you arrested by the FBI, user discretion is advised" 😂
And there was me thinking it was a rookie mistake making such a script public...
I think it would have been quite clever to share the script had he used a more discreet account. A bunch of people downloading the data through Tor could create a lot of work for the investigators and thus help hide his identity.
the description says "This is not the actual Github file"
Someone honestly should go through AWS (and other providers) list of IP addresses and attempt to get the instance ID. If you can, report it to the cloud provider so hopefully they can inform the customers that are affected.
@@Y2B123 They'll just look @ the 1st ip
@@danielo7985 would've worked if it weren't for the fact that the attacker used Tor.
I worked a contract there as a Sr. SW Engineer. Was never told I was put on a team/project for which I interviewed. Was supposed to be doing design/implementation of some new micro services. I start day one and the project manager didn't know I was coming on board and we had never spoken. I spend the day getting my env set up blah blah blah. Then they started pulling tasks off of the 'Agile' board and point me to the code base and it's a complete clusterfk of code that was about 2 yrs old and nobody was around who understood the problem domain. absolutely no discipline (SOLID, DRY, etc) was used. Thousands of lines of 'copy pasta'. Automated tests (Cuke or whatever?) that didn't pass simply had the input data commented out. AND, best of all, I'm told that the team is responsible for setting up AWS S3 and servers, networking, etc. No dedicated DevOps people. I don't do that stuff and when I've done it in the past, only in a 'dev' env. and not in 'prod'.
Two other TBTF banks were bad as well, but nothing like CapOne. Un-real.
The real question is how the guy got away with it. That's a pretty textbook CFAA violation. I kinda died a little the second I saw that IMDS forwarded URL, anyone who's dealt with this before knew immediately what happened.
@@raylopez99 well that's really fair
@@ramielsayed2614 Actually gamed the system.
@@f4ephilosophy691 would not surprise me if the dude squirreled away some money offshore and then pretended to have spent it all...
@@raylopez99 Oh it was someone even more institutionally privileged than a woman. Thanks for letting us know.
@@TheShamefurDispray
That's why 10:40 happened I suppose, girls looking out for each other 🤣🤙
Dude this is one of the most informative yet hilarious channels I've come across related to cybersecurity. Awesome job. Love the in depth details of actually went wrong instead of just broad "got hacked" verbiage.
Your videos just have a flow and dry humour to them that makes it very entertaining to watch while still being informative and not being demeaning
Keep posting, Kevin! These videos are awesome - I will be recommending to people
No one was hacked, that information was public, or rather the security keys to access the private information was publicly available.
It is literally dropping door pasword note on the floor in front of the door
😂
IAMs, VPCs and SGs are the most confusing part of AWS services. It's a labyrinth of configurations and very easy to screw up.
All the features in AWS are pretty well documented, they even explain how things works inside so you can understand the tools better. One of the biggest drawback of AWS is the lacking of advanced examples. They just provided the most simple example, so you may have a hard time to figure out what do they actually do, especially cloudformation and python aws sdk. They are well documented but painfully abstracted so you have to try and retry again and again to make something work as intended. It’s like they give you a death star lego set of 100000 lego pieces, with just a few pages to build some simple construction so you have to look at the images in the box to build the complete one.
completely disagree
@@aperture147 agree 100%
Love the editing in these postmortem videos!
Adding a comment to help engagement. This is a truly underrated channel.
Worked for a client that did work for Capitol One, prior to 2019. Capitol One was by far the most strict partner that we dealt with. Everyone complained about having to follow Cap One's processes and procedures but what we realized that it was for our own good.
their arch nemesis: the on-premises menace
I love this channel
Just want to say your channel is amazing and I’m so glad I found it before it blew up
Having a wide open reverse proxy on your corporate network seems like a terrible idea.
the minecraft cli XDDDD man your editing is the best and story telling is top notch.
As someone who knows cap1 senior devs I’m not surprised this video came out. Only it didn’t come out sooner.
I say capital one's fault. They're using the service and AWS is only responsible for accurately telling them what they get and don't get. If Amazon guarantees something or misconstrues what they provide or fails to provide, it's their fault, else it's not. If Amazon says they get 99% uptime and they get 99% uptime, it's on the customer. That being said, sticky situation and one could make a case for either. I like the braking analogy as it's definitely a spectrum for what's "expected"/"reasonable" for example automatic advanced braking system vs just a working one in general. The difference is just the scale and what's "reasonably expected"
AWS should have two types of S3 buckets (public immutable, and private immutable). And that would solve a lot of problems.
What I see happen is devs get confused by all the security configurations for S3. This isn't an excuse but I'm just saying what I see happen. The problem is that a private bucket can be changed to public.
The amount of documentation you need to read to get the right permissions are just ridiculous if you don’t know what your doing. And most of devs who set this up aren’t experts in cybersecurity so it’s hard. That’s why pen testing is so damn important, even if it’s bloody expensive.
the process to unprivate a bucket is lengthy in itself. you need to uncheck/deselect/disable varying options across their submenus. all buckets are locked down by default with plenty of warnings screaming if something is public. company needs to also do their due diligence and actually prevent + detect anything thats been exposed to the internet.
That’s not what happened here though 😊 but it has been the source of many leaks before
S3 buckets are private by default for good reason. I really can’t see many good reasons to make a bucket public. Part of the billing is data transfer so allowing anyone to consume as much content in a bucket as often as they please is going to result in hefty bills.
Fun and educative video. Hope your channel blows up
I was interviewed several times by Captial One about 5 years ago. I was a certified solution architect and I had put in my resume at some point. They were really trying to poach me, but three minutes into the interview I knew it was a clown college. Regarding AWS security - they are only responsible for data inside their network. They tell you this, and it's part of the practitioner and solution provider exams. If you are 100% serverless then AWS is 100% responsible for your data. As soon as you agree to manage your own server via an EC2 instance then you are responsible. Honestly I don't know why any org would need an EC2 instance when ECS is a viable alternative, and makes scaling zero effort. Scaling EC2 instances can be done, but it takes work, and it's susceptible to all the problems a non-managed solution has.
They probably just did a lift-n-shift from on-prem, before they had containerized their setup.
that's not how shared responsibility work... AWS responsible for the underneath infrastructure. so it is true if it is serverless AWS responsible the server that run the software. but the IAM permission, the code you run inside serverless is still responsible by the user who create it
blaming aws for this is ridiculous. It's not their job to forsee every possible stupid decision their customers could make. If someone set that bucket to public global unencrypted access, it wouldn't be AWS' job to shut that all down
Lmaooo I love the Lavish Tesla pic for "automatic braking". Excellent, subbed.
Capital one should probably incentivize giving people rewards for following their responsible disclosure agreement.
No. The problem is you'll get a pair of enterprising folks that will create trivial problems, then report them to get rewarded. Then someone will make a flaw they think is trivial, but turns out to be serious, and by the time it is corrected things go to hell very fast.
They should; it would incentivize people more, but the FBI and others will pay millions to get some of these exploits; there's no competition if people do it for the money.
Man, your way of explaining stuff is brilliant and easy to understand, even for a lay person. You deserve way more subs!!
you can't even really be mad at the hacker for exploiting such a trivial weakness
I have no idea what the hell that video was about.
You engineers make civilized life possible and don’t get an iota of appreciation from the rest of us.
What a fantastic video. Just goes to show that we need to be mindful of the security of our operating systems and applications.
HE POSTED THE CODE HE USED UNDER A PUBLIC REPO WITH HIS FULL NAME 💀💀💀💀💀
This is gold ! Thank you
The name of the VPN service "Ipredator" is.. one could call it "unfortunate", but it's probably deliberate. The service was created in direct response to the EU IPR directive, also known as IPRED, and the subsequent Swedish Intellectual Property law commonly known as the IRPED-law, which was basically about combating software piracy in general and, some would say, specifically made to attack The Pirate Bay.
Even though AWS isn't responsible for this, it still upgraded IMDS to try and help its customers from making mistakes - not because it was responsible. It didn't have to do this, but it did.
Underrated channel
Ooo! Another video 😊
Simple Storage Service... no security concerns here, it is just a "simple" service going on...
Looking back at some of these leaks that involved S3 and AWS where companies "rushed" into selling the cloud idea to their senior leadership, seems like people had their own set goals to get the biggest bonus possible, and sell fancy terminology how company is modernizing, how company is adopting intelligent technology, etc. - but behind all that "fancy" is simply the same technology that was available before - just now, you pay the company to host it for you instead of building your own data center... hence, since you are "contracting out" that piece, it is inevitable that once again convenience comes at the price of security... So how does this happen? It happens when AWS tells the company this service comes with the shared responsibility - AWS is responsible for a piece of it, while the company handles a piece of it... in other words, unlikely that AWS will do something wrong, as they are in business in providing this up to a certain level and you get the whole encyclopedia of it what they do... companies??? Apparently not so much in CapitalOne case...
Too bad that data breaches continue to happen, and penalties and fines companies end up with are nowhere near realistic ones to make a difference... "it's a speeding ticket" given their profits that measure in billions each quarter...
imagine legitimately using bing and actually expecting a quality response.
Great video! i've been looking for more channels like this! Subscribed!
Thank you for the great content to consume while snacking, from the editing to the info, good stuff :)
Like disruptTV without the distractions, just the info
I got an AWS ad on this video
I love all the explosions in your videos.
it was in the cloud, they should’ve known it would’ve rained down one day
Glad AWS made some patches to help mitigate this. However this was 100% the fault of Capital One.
Insecure software.
An instance designed to serve public data had an IAM role to get and decrypt all data across all S3 buckets.
AWS denies access by default. Allowing open access to S3 is deliberate.
8:07 it lived up to the WAF part of it’s namesake 😂😂😂
as in "what a fail"? 🤨🤨🤔
this channel will blow up soon
Some AWS consultant making $175k per year helped to cause this blunder. That's what pisses me off. 15 years in IT, never making big money and never making blunders that jeopardized users.
Capital One: Who's In Your Wallet?
Very good video. Love this type of story telling
Some thing amaze mee is that who uploaded cred file to git repo and who made git repo public is operational team forgetting the severity of information
This is great content, Keep up the good work
I love these videos, please keep making more
Alright just one more video before I go to sleep
The last video before I sleep:
1:43 - "The engineers tested the commands" wait wat, I hope they tested it in a clean vm isolated from everything including not being in same network address range as corp network right?
They could have done it in their private network but still someone probably used the commanda in another network. Even if you are in the company's private network, you still not be able to access all the data.
I literally just started working on their Cloud Security team and this is the first time I'm hearing of this smh
This is my new fave channel
2:41 that's legit how I learned how CLIs work lol
Amazon is not at fault. When the user can't do the proper things to secure their machines, then that's on them.
Don't forget the counter-strike cli
Love the content man! You've just earned a sub
Wow, what an amazing video. Thank you
Money under the mattress is looking better and better
No mention at all for the responsibility AWS had to review/remove vulnerabilities and access when employees are termed. We have been doing that loosely for more than 2 decades, but now there is a checkbox that initiates a thoroughly organized team review that includes a think tank style catch-all: "worst case / what else". We routinely have meetings that start: "as you all know Timmy is gone and we will be updating everything". If neither of these shops do that; shame, shame, shame.
Because AWS isn't responsible. SSRF can be performed by anyone as long as they can connect to a webserver that is vulnerable to it. The instance metadata server is only accessible from inside of the instance. If someone is able to obtain data from a publicly accessible website from the imds server, then you haven't secured your webserver very well. I don't have imdsv2 enabled on my instances but then again my web servers aren't vulnerable to it and my instance profiles are fairly well locked down. Having said that, if I do live to regret my words, it's on me. I'm not going to go crying to AWS... I don't pay for support anyway..
i love this video keep makingmore of it
In my 35 years as an IT tech, I have come to the conclusion, If a system can be written, it is a system that can be hacked.
The changes you've seen In your tech career must be astronomical!
This is a great channel
underrated channel
Wasn't the Bing model trained on data from like 2019?
GPT4 would be 2021, but I assume it can regurgitate more than it's training data since it uses the search engine.
Very interesting video, thank you!
9:37 "google en pasent" - I see you are a man of culture!
amazing
Imagine downloading 30gb of Capital One's data and then just going "lemme just publish how to do this on a gist and call it a day"
Love your slides lol
If you rearend someone because your breakline broke, then the company would be at fault, plus the company pays more so they should be targeted with the blow back.
I think that it should be illegal for a company to save any sensitive information of someone for any reason besides needing it in that moment, the second that information leaves then they should be deleted.
As for a punishment, i would not mind if every CEO and higher up who personally signed off on launching the site had every personal detail about them leaked, and allow the affected customers to be able to leak any new information whenever they want to, with it only stopping once every customer affected decides they're ready to. We need to be as cruel to big multi-national companies as possible
Why would most reverse proxies block PUT request. Doesn't make sense. Any webserver implementing RESTful would be using PUT requests.
Amazing! Instant subscribe + all notifications
Amazing quality man
I love your videos!
every financial company and credit bureau that's on AWS have been hacked. I wont say how because I still have Non-Discloser but i will say this who ever gets AWS jump ship fast.
Thanks for your sharing
So what happened to the person who published it on GitHub?
great video, thank you.
Great overview, thank you. Please do tone back the volume on the effects you use though, sometimes they drown out what you’re saying.
Please make more of these videos! They're great.
10:58 So what I'm hearing is that everything would have been fine if they had just stuck to COBOL like God intended.
AWS is over rated. It's amazing they still have the cloud market share they have.
How can it be AWS's fault when an employee of Capital One hired to set up their servers configured a role that could do anything on a public server AND apparently didn't configure the firewall at all? Lmao