They failed to give him tens of thousands of dollars for a bug bounty, and instead ended up paying potentially millions of dollars in lost clients. 👌🏼 That's fantastic business decisions right there.
Well, i've seen worse. Like literally sending lawyers + law enforcement after people who sent the vulnerability + the fix to the company. But of course that doesn't mean you're wrong.
For his personal gain yes, the kid made the smart move. But this is at the very least Grey Hat behavior. Frankly, he is, as most teenagers are, relatively ignorant and overconfident in his judgement (and as Prime said, this can be supercool if you manage to point him in the right direction and cut him some slack, plus maybe, show him some of his own limits if he needs to be pulled back down to earth). The usual thing would be to let the time go by for the company to address the issue, then tell them again that you are going public or inform third parties. The kid really did break responsible disclosure policies and could be sued and prosecuted in criminal court. You might not like it, but if you tread on dangerous grounds, you take some real risks. Of course this is not as fun as being a bit of talented smart ass who sticks it to a big unresponsive and frankly stupid corporate blob, but that's what ethical hacking is: Not that fun and exciting.
@@grafgrantula6100 I am not a hacker or programmer so forgive my ignorance, but I don't understand how it is ethical to allow a company to keep their customers in the dark about a security/privacy vulnerability that affects them. By giving Zendesk the full responsibility to fix the issue on their own and inform their customers, one has to blindly trust that Zendesk will do the right thing. The customers have a right to know that what they are using is not safe and secure, and there is no indication that they would have known at all. For all we know, Zendesk could have kept silent on their ends until they finally had a fix, and that is way more unethical in my opinion than telling affected third parties about security issues that can pose a legitimate risk or concern for them. Why should the makers of the software have a right to privacy about the flaws in their design? Why should they be trusted to act morally and ethically without any sort of external pressure? It is not like the kid originally started disclosing this vulnerability on the internet for all to see and abuse, he tried to keep the communication channels private and limited to the affected parties. Legally, I have no idea if there is an actual case to be made against the kid, but I would hate to imagine that. If this kid can be held liable or be put with criminal charges for trying to protect third parties from a security vulnerability by informing them, then something is seriously wrong.
@@grafgrantula6100Actually the reported have informed Zendesk via h1, but Zendesk claimed that enail spoofing isn't part of the bounty scope and decided that it is not a vulnerability. At that point the reporter's responsibility on responsible disclosure is already fulfilled, if the company say it is not a bug, he is free to perform further actions. In fact it is very responsible for him to only inform Zendesk's customers, if he do some public writeup the situation would have been much worse since it is a truly exploitable 0day.
OMG Zendesk. You just lost a sale. We were considering moving our ticketing system to them. They are one of the top 3 contenders. It's not the fact that they had an issue - anyone can have one. But this is a shitty way to handle a major security flaw. So what happens next time they have a major issue reported by someone who doesn't fit their exact specs? As customers, we don't care about their bounty requirements. All we care is that our private data and data of our customers (that they might share in the ticket) is kept secure.
Also, notice how they never accepted responsibility - it was the fault of the products doing the spam filtering, they said. As if a bunch of heuristics could ever substitute for proper security-by-design.
Truth. They lied in their statement that the hacker violated anything because they themselves told him to kick rocks. This is not a company that deserves its clients.
Same. We were about to get Zendesk, now I'll veto them. Screw that. Their statement is deceptive at best, and clearly their interest to save face is bigger than my interest to not have my company compromised.
@jamesarthurkimbell if its in shut up about it scope, its in bounty scope. Shut up about it scope requires shut up about it money, which is just a bounty anyway
Exactly, Daniel could’ve just responded “look you made it extremely clear we have no business relationship therefore I am not bound to any of these rules. Either admit this is in scope or I will keep doing the ethical thing of notifying affected parties so they can remedy the issue because their supplier refuses to”.
They weren't even publicly posting about the issue, they were reporting it to impacted clients. They were still acting ethically while being told it was outside of the bug bounty program. This is completely the correct way of handling it.
TLDW: Daniel: Zendesk, Problem! Zendesk: Eh, not the right kind of problem. Daniel: Companies, problem! Companies: Zendesk WTF. Zendesk: Oi. You told companies about the problem. Daniel: Now I’m gonna tell everyone about the problem and how poorly you handled it 😊
I think their ChatGPT instance misspoke, they meant "unwanted communications", because they didn't want to deal with it or fix the issue or pay out any money.
i feel like zendesk really misses the point of bug bounties. they’re not a kindness to the researcher, they’re a fucking ransom. this bug could’ve been worth millions
"I have already disclosed all necessary information to reproduce and address the issue, which is currently outside the defined scope. Please understand that I am unable to share further details with third parties, as the root cause lies within Slack’s authentication and the customer’s integration. Slack and any other affected parties are managing communication, and they will provide additional information if needed. Thank you for your understanding. If you need further assistance in the future, please feel free to reach out."
"You are welcome to hire me as a consultant. My hourly rate is 10k. It will take 4-5 h to write a detailed report on this vulnerability and I'll also be available for potential further assistance." (Not sure if that's legal due to his age)
@@vincent_sz why? Is he obligated to give it to them? Is there an upper limit of the rate? He tried going through their official channel and they said "no", so now it's his terms. If the price is too high for them, they don't have to take it, simple. Also, he's not selling the vulnerability, he's selling his services.
getting paid a 5 figure sum while retaining the ability to explain exactly where the money came from AND the bragging rights is not a bad deal, at all.
@@KemalArdilGulez this getting money and some fame is most likely realy more fun, than getting no fame and having to find away to launder a lot of money before eing able to use it. I even imagine, if you get some disclosure programm with no bounty attached for hacking law enforcement without getting into jail , where you are allowed to claim the fame after law enforcement fixed the breach you found, this would attract thousands of white hat hackers to do that just for the bragging rights.
@@KemalArdilGulezbro would reject 950k USD to brag to some internet randoms. Also selling such a vulnerability isn’t always illegal, especially when selling to a US/western governmental agency…
@integrierterbursche4062 why would a government agency buy this bug for a million dollars if they can just force Slack to give them customer information
I mean, if you knew ZD, you'd know how the 1st time would play out anyway, though of course you'd attempt to warn them because it's the right thing to do.
So the summary is - kid found the bug and reported it - they told him we can't pay you for this, it's 'out of scope', but it's serious - the kid finds out that he can backdoor into slack of major companies and reports it to them directly and they pay him - and finally zendesk claims that he violated the disclosure rules 😂
As a dev in a 100-person company that threw zendesk in the bin and started coding its own tool to implement a few specific features and not have to pay for zendesk while losing all the others and the most important stability, and this without having a dev team dedicated to it. And now they've hired me and are paying me to turn this in-house solution into a zendesk-like product with 0.1% of the budget. And when you think about it, it cost them $900 per day doing so...
Creating a new product line is different than creating an in-house only tool. It is perfectly reasonable to buy software tools for things you need that are not core competencies. An accounting and payroll program should be easy enough, but I bet most software companies use an external tool. I know all the companies I have been to use external tools.
@@Ganerrr Except that it's not just a ticketing system. You start adding features... Features that should be in an ERP or CRM. What's more, the code base wasn't written by a developer, so I'll leave you to imagine the state of things.
I found a bug in MS software and saved a customer probably 5-10 million year over year. I got nothing for it except a thanks from MS. So now i no longer fix bugs. I just sell workarounds.
Usually the scenario plays out like this: a product manager comes up with a brilliant idea on how to make the product better. It is then tossed over the wall to engineering who have been taught to just shut up and do what they are told. No one thinks over the security implications -- there is no budgeting for anything like this. The engineers aren't impowered so they just expect to be directed and the managers don't know any better and expect the engineers to just handle the security.
@@kuhluhOG More specific: When the engineers find a security concern, where an exploit is knowable or where watertightness is not proven, they bring it up to the managers. The managers then use their own "skills an expertise" and judge that the exploit is not very likely and thus doesn't need to be solved.
Holy shit Zendesk committing sudoku. The irony of a helpdesk service having the worst PR people and trying to shift blame to a literal 15 year old for poisoning the well when they're the ones responsible for screwing up again and again. Unbelievable.
This is not even an issue with SPF or any of the mentioned protocols. It is a logic bug in the application that invites people after the smtp protocol part already happened. Bad triaging.
That's why I stopped reporting bugs. Dozens of critical issues, remaining unaddressed for years or swept under the rug by stats massaging bots... Zero appreciation... So I learned to walk around the bugs instead.
Same. I am sitting on a handful of serious exploits I have stumbled upon because companies treat you like shit when you report things. I would advise staying away from Facebook and ManageWP especially.
@@biblebot3947 sure it can. Don't allow others to claim your time, resources and effort for nothing and be spiteful if you refuse to do their slavish bidding in the name of altruism.
They were saying the vulnerability is not policy because they say the vulnerability is in the email protocol, not their app. Which isn't true in this case
They way they do it literally ignores the requirements and recommendations of RFCs: * 4408/7208 * 7489 * 6376 * 5321 vs 5322 It's not the email protocols. It's literally never email protocols, they're *REALLY* solid. It's developers thinking they're too good for RFCs/too lazy to read RFCs/thinking RFCs "limit creativity". I have encountered countless examples of all three. It's always the developers.
The email protocol has nothing to do with the crap system that allows people to add themselves by simply knowing the ticket id. THAT is an insane feature.
@@brentsaner When the sender implements DKIM+DMARC, and both the sender and the receiver implements DANE or MTA-STS it's pretty solid. Otherwise, not so much. Unless you consider allowing from header spoofing or not preventing man in the middle attacks solid. The amount of domains implementing DMARC correctly is currently still in the minority. The amount of domains implementing DANE or MTA-STS is miniscule. I hope adoption will go up in the future. Saying it's never the email protocols is misleading, as at some point dmarc etc didn't exist, and at that point it was definitely the protocols that were REALLY NOT solid for preventing email spoofing.
I see where they're coming from as email security also relies on others setting up their domain correctly, but they should have designed their app in a way that accounts for email spoofing, instead of just throwing up their hands and saying "not our problem".
@@louis-lau You mean just DKIM. DMARC is purely for spam policy, not establishing trust. Likewise, *not* using MTA-STS doesn't magically enable a MiTM on STARTTLS cross-MTA. FURTHER, like DMARC, it just defines a decision policy for the remote end. It's up to the remote end to follow/enforce it, which you as a sender have no visibility into (aside from seeing which phishes make it through). ADDITIONALLY, DANE can be considered neutral for mail. I'd argue it has the ability to add legitimacy to an illegitimate sender by *not* relying entirely on a third-party anchor. Once again, it depends on *how validation is being done by the receiver*. Just like STARTTLS, just like DMARC. Lastly, no MTA nor MUA software maintained over the last 5 years that I know of uses the From header for any routing decisions unless explicitly configured to do so, they use the From from the envelope. Lastly part 2, NONE OF THOSE DO ANYTHING TO PREVENT THIS ATTACK. The majority of the mitigations you mention are very specific and very targeted that are more for security *polity* than they are for *guaranteeing safety*. They are by no means bad, I encourage sites to deploy them. But my point stands - there has been no successful attack on the SMTP protocol itself that I can think of that would allow this. It's going to be either receiving MTA configuration, which is far, far less common with transport being delegated to third parties, or MUAs in applications - WHICH IS PRECISELY THE CAUSE HERE. This, like every major vuln related to service-based email that made headlines over the past, what, decade are entirely past the MTA. It's the developers.
I disagree with the mantra of “just do the core thing and externalize the rest”. The “core thing” for Amazon was selling books, the only reason AWS exists is that they refused to use outside tools and invested in creating their own. That mentality has destroyed the european industry as top management got to the (wrong) conclusion that sales was the profit driver while engineering was a pure cost center.
the whole "sales is the engine" has taken over almost all industries now. people have forgotten that somewhere in the economy someone has to be producing the goods. a land of producers is wealthy land. a land of sales people is an impoverished land where everyone is fooling each other into thinking they're wealthy.
Jeff Bezos explicitly states in interviews that selling books was just his starting point. He never intended the company to be about books, he wanted to ride the big online shopping wave and saw a future where package distribution and delivery was going to be reinvented. And he wanted to build a vertically integrated company from the start. Books were just the easiest place to start. Amazon was never intended to be just a book store. If you want to build your own online bookstore today, you're not going to reinvent datacenters.
@ Yeah, he never wanted to sell just books, but he intended to just sell “stuff”. He never even considered web services as a business. Amazon is not the only example, Intel also stumbled into their whole modern business by doing something that was outside of their “core business”.
The whole reason the United States engineering industry is the PowerHouse it is, is because we have an entire society that values people saying I don't like the existing solution, I'm going to build something even better. And then I'm going to figure out a way to make it profitable. The whole point is to build a product that is so good that it solves someone else's problem better than the competition, and engineers are especially good at that, because engineers just get frustrated when things don't work right. And then they go build something better because they get annoyed. Never piss off an annoyed engineer, because they'll turn that annoyance into profit.
19:40 "Sorry, sending the proof-of-concept for a non-issue is out of my scope. This was already identified as a non-issue, so I don't see why it's a big deal now. I've already let a bunch of people know about it for their own security so I can't exactly keep it confidential now. Maybe discuss with the other companies and leave me out of it. It is, after all, not important. To remind you of this non-issue, I have attached a copy of the previous correspondence to this email. Kindest regards" All I've learned from their response here is that NOBODY should be doing business with Zendesk, and anybody who finds exploits with Zendesk should do anything but report them because Zenddesk won't do anything for you.
He knew he won't get paid so he at least had his revenge by making their lives a living hell and lose customers. Brilliant. Wholeheartedly support his method of making them look like complete morons.
This, and also arguably warning their customers was _the moral thing to do_ . They're the ones exposed to the risks, and ZD seemed unwilling to properly address the issue.
In this episode, a corporation shifts accountability into the nether as they slander the white hat hacker for "violating ethical principles" that were voided when his submission was rejected.
classic corporate unaccountability. if they had lawyers they would have played this way better and that says something about how incompetent zendesk is on this.
"Responsible disclosure" in the scope of Zendesk's T&Cs for the bug bounty is one thing. "Responsible disclosure" for the end customers is something else entirely. Zendesk was not the one being hacked here it was their customers. If your company supplies SaaS you have a responsibility to your customers, to fix in a timely manner, , to disclose if necessary and to keep the white-hats happy. This is part of what a SaaS company should be supplying as part of supporting their product. Zendesk let their customers down. Slack channels were the least of Zendesk's customers' worries (any service with a password recovery option involving email was at risk). In terms of "responsible disclosure" to those end-user companies impacted the white-hat did the right thing. If customers moved on from Zendesk, because of what happened it is because of how Zendesk handled the situation from their perspective. Those companies care, not because of the mistake (we all make them) or whether a 15 year old gets a couple grand or not, they care about a SaaS company they use who doesn't take their security seriously enough.
The asshole, when their failures are exposed, they will deny and when confronted with evidence will double down and project while refusing to acknowledge. Basically "blame someone else, otherwise it's your fault!" So when you find yourself in a situation like that, take a moment remember "Getting angry will probably make things worse!" A misunderstanding is easy to unintentionally create and extremely difficult to negotiate after. So chill out, and stay frosty.
Nothing in scope of an NDA is out of scope for bounty. The companies pay for the silence. If they don't cough up for responsible disclosure why bother, just blast it out and let them deal with the fallout. Fck big corp.
@@Blackbirdone11he is young, proud and smart. Refusing to disclose his PoC unless they pay him, right after he exploited those vulnerabilities, would have exposed him to criminal prosecution under extortion charges. As much as I love being petty, it’s not worth catching a felony.
Setting up an email server without SPF, DKIM and DMARC (maybe two out of three - it's been a while) is flagrantly incompetent. There aren't many things to fire a person for immediately, but this is on the list. You can't even get an AWS email server up and running without dealing with anti-spoof first. Now, was it a great idea to add some sort of auth to email? Hell no. They're depending on others to not being idiots.
The wild thing is that they excluded it on invalid grounds - it wasnt even bypassing email authentication. It was code of the zendesk product itself making bad assumptions about emails and using incremental IDs
well, a ticket system may not be "core" but what happens here shows that you still have to be very careful. it would've probably bee cheaper to do an in-house system compared to the fallout of that backdoor.
14:26 the triage acknowledged it as a serious vulnerability. OK don't pay but flag the report as such (at the very least IMPORTANT) and f@ing forward it the security team!
They haven't acknowledged that the bug is on Zendesk authz part, not SPF/DKIM/DMARC which are irrelevant to the issue. "Email = SPF/DKIM" is silly logic.
@@chupasaurus yep was thinking about the DKIM signatures to prevent spoofing. But it's basically very hard to set up unless you have someone else do it for you.
I built my own ticketing system for a product I managed while working at a fortune 500 company, and it worked great. Although the product I managed was only used by about 100 employees internally, not the whole company. I built it in WordPress creating my own custom plugins and it fit our needs well and I rarely had to work on it after the initial 3 days it took me to build it.
The craziest thing about this is that ZenDesk ticket threads very often contain PII, such as (ironically) for GDPR requests. Having the ability to add yourself to someone else's ticket chain and get access to all that PII is ... insanely dangerous, not to mention illegal if they do not disclose such a vulnerability when informed about it, and ZenDesk absolutely deserves the blame here for not realizing this.
For a ticketing system, you have a user facing number that is incremental, and an id that is a GUID or UUID. And the system only works with the id, and the agents only think about the numbers I work at a F500 that also sells a ticketing system to most of F500.
@@jasondoe2596 my guess would be for better partitioning in datawarehouse storage? (reduce query times by grouping like tickets together) or maybe only to have a human-friendly format for users to work with? (e.g. users can see at-a-glance that ticket-123 was created after ticket-111) i really dont know
34:59 I’ve heard about an old slot machine bug (infinite money glitch) where the slot manufacturer alerted the casino, and the casino owner drove to the next nearest (competitor) casino and used the exploit 😂. But yeah, you can’t control whether victims of an exploit become bad actors themselves. To the ethical question of who to contact, I feel like the kid made the right decisions. I think this is similar to discovering a major product defect, and if notifying the producer does not cause action to be taken, then it is “responsible disclosure” to notify the customers.
I can definitely say Netflix did not use Zendesk when I was there in 2014. It was for sure an in-house solution (using bog standard Bootstrap CSS at the time, too).
The Customers are no third parties. Why should he ever go back and try again if ticket/pii access is out of scope if its based on mail spoofing? He reported his finding to other vulnerable parties with vulnerable SLACK instances where his bug was in scope. It was was totally correct to do so. Even if Zendesk would fix the issue still slack and other tools would be vulnerable through other services.
I once created a certificate validation system based on the hashed name of the user. It worked fine for preventing attacks that want to enumerate participants. It is not worth attacking as it was just for an academic conference. But I didn't feel like exposing everyone's name to the internet.
@vidal9747 Probably good enough, for the purpose. But i still dont like the smell of just using the users Name. You might be able to replicate the generated id's somehow (If you know that it's the encoded Name) Maybe also add the time to that hashing algorithm or Something.
@@vidal9747 you are probably aware however how only using the user's name as input to a hash function can easily be reversed by lookup tables. If it's going to be an id, you could use the user's name, plus the user's name is UPPER plus the reversed lower ... which would still be obfuscation but a lot harder to look up.
@sk-sm9sh no, a hash function will consistently output the same output for the same input, it doesn't matter what hash function you use... it will still output the same value for "ybvb" every single time. I'm afraid you'll missunderstand but that's fine :D
the kid did exactly the correct thing. he went to clients who had the most to lose. zendesk has very little to lose in this and still managed to screw up. when they responded, he should have responded as they did. and referred them to clients. and respond through them. I reported to a client months back (via a deskdesk support ticket) where they were advising me to catch a flight to singapore. hopefully the company paid for the flight... I tried twice... sometimes you have to pay for experience.
50 people for building/maintaining a ticketing system? How about 5? It's not rocket science. We have a solid framework on which 2 people work, and the other 3 work on customer requests, support and QA, and it's used by multi billion dollar companies. It's a *lot* cheaper in-house than *anything* else on the market. And it's not even a product that we really sell, it's just something we use, not business critical as you say it.
To me, it seems like the slack takeover isn't a vulnerability. But that the vulnerability was the auto adding other emails to tickets from spoofed emails. The slack takeover was a proof of bigger impact.
I think the point about DKIM is that the companies email server should not have allowed email spoofing. Then 100% the second bug is an apple, slack, email, zen desk bug. So he is 100% right to disclose to each company. Not to mention that they paid out.
I work for an MSP. Our ZenDesk contract is almost up and we're actively bringing in another ticketing solution. This incident was part of the decision.
It would be as simple as to check dmarc, and record it on the opening of the ticket. Then, if you recieve any subsequent tickets, you simply compare the current status to the previous one.
I think the first chapter of "How to make friends and influence people" says "as you come up with better a PoC:s, keep the owner of the system up to date before setting their customers systems on fire, because that might prompt them to change category of a Hackerone report" or something along those lines
"Hey, did you know if a window is left open you can get through it?" Zd: "Yeah everyone knows that, but we don't sell windows" ..a few moments later.. "Hey, I'm in your house" Zd: WTFBBQ!?!
26:37 I 100% agree with this decision. He should have been paid before but he definitely shouldn’t be paid afterwards. It would set a really bad precedent to reject paying out a bug and then paying after they take it to the public. People would just be able to elevate the importance of their discoveries by telling affected people.
He forfeited his award of ZERO dollars for disclosing this big to the customers after being lead to believe Zend Desk was not going to take it serious. I am not surprised Zend is failing. They were big in PHP like 20 years ago and have just gone down hill since then.
its like saying I will pay anyone who protects your assets, then turning around and saying "oh sure, you reported that my side door was left open, but you didn't give me a step by step map on where all my valuables are so you get no compensation, nothing, zilch, GOOD-DAY!", then once the person reporting say ok fine steps inside the wide open door and goes up to the guests who also said they are willing to pay to be notified about risks to themselves crying that you should've only warned them about the issue, you have already demonstrated your unwilling to actually honour your promises and will use weasel tactics to get out of paying, the guy was well within the bounds of morality to instead take his chances alerting everyone else who was being exposed to risks due to your flagrant disregard to security and hope some of them are more honourable and actually show some appreciation that they were warned directly and able to avoid a potential absolutely devastating hack to take place due to party A's severe lack of interest in addressing real security issues, if anything his decision was potentially even more moral than going back to zen desk given their previous dismissal of the concerns raised, at least now the actual potential victims have a chance to do their own mitigation and not rely on the discretion of zen desk to care
If first party responsible disclosure doesn't work... talk to their customers. Especially the big ones. Someone will listen and you wil set the steps in motion. Has worked before.
This is what many people seem to misunderstand about bug bounties. They are not generally intended as a path to communicate with an organisation about vulnerabilities that need to be fixed. They are intended to steer the effort of no cure no pay pentesters. That is fine. But refusing to communicate with people who report valid security issues just because you decided not to give a financial reward for the report is harmful.
@@aravindpallippara1577 I'm not suggesting the person reporting should have done something differently. But it's important to realise that if you want to be able to do coordinated vulnerability disclosure and warn others, like the customers in this instance, it's probably not a good idea to enter into an NDA like the ones attached to practically every bug bounty. Most NDAs there will apply indefinitely, even if the report itself doesn't get a reward.
i think this is honestly responsible disclosure, as he was first dismissed, he informed the companies actually at risk that they were at risk. How is that not responsible disclosure? zendesk just didn't want negative publicity and choose to put other companies at risk because of it.
Their revenue for 2021 was $1.34B USD, and their operating in come was $-166.68M USD? Their net income, which I guess they mean net profit, was $-223.64M USD? So they spent $1.5B to run? What the hell are they spending money on? A billion and a half to run a software firm? No manufacturing costs, no shipping costs, just electricity, rent and staff? Holy hell.
19:25 I think it would have been very reasonable for him to request that the issue was re-evaluated as being in scope first before sharing those details. At that point they are literally requesting free labor. You can't have the cake and eat it too. Either the bug is in scope and you get the details or it's out of scope at which point you have determined it to not be important enough to need the details
To be fair, the reason scope and out-of-scope are defined is so that companies can avoid getting ganked from all sides at the same time and bring all their teams to a bug fixing grinding halt. Also, going out of scope and expecting a reward is unfair to the other security researchers who participate in the program and keep the ball between the lines
Personally I believe the bounty program scope needs to be simplified to "We don't want you to share this bug with our clients so here is the 'shut up' money for you." If the company doesn't care about the bug going public then no bounty for you. If they do, then there is a bounty.
Sweet! I just used this to piggy back myself on the persons ticket before the one I just previously made because their Id # Run concurrent to one another. I was able to get a checkout code for 75% off entire cart that was given to resolve a complaint about shipping. I'll be swimming in e-juice.
The reason why Zendesk is doing so bad financially is the same reason the honest man didn't get rewarded. Leadership is the opposite of what would have lead to opposite outcomes in both cases. In other words: If ( CEO == eval(-iq)) { effects(-iq) }
Not Invented Here for tooling, like everything, is about balance. How well does the thing do what you need it to do, and what would it take to do it inhouse, and to what degree would the inhouse version be better than the outsourced? Making an ecommerce site and remaking Shopify is dumb, even when you could get a more purpose built equivalent, because it's just so complex, and will basically never in reality be better (even when better in some ways). But remaking some of the basic shopify apps people install? Probably great, because most of them are terrible as is, and most aren't even THAT complex either.
@14:16; The issue is that they are marking as "Informative", and aren't saying "Thank you for the responsible disclosure - while we won't be paying you for this, we will work to fix this issue, and we will follow-up on when we have fixed this and you can fully disclose this issue that you found after we have patched it in." The person wasn't even given permission to just reveal this publicly, or told that they *will* be able to reveal it publicly. They effectively said "This doesn't count for the program you're reporting this from - you'll need to fill out the form in a different program, over there - it's a blue form.".
Most likely, the SPF, DMARC, DKIM out-of-scope in the policy was supposed to be for emails FROM zendesk themselves, not, an attack against their product that should depend on these email authentication schemes. Excluding SPF, DMARC, and DKIM is common for many policies, but very few companies actually build their products on top of email.
Making in-house solutions is just a matter of cost, licensing vs development. Though it can be very hard to predict what it means to develop and maintain a service, while a license cost is predictable.
At the end I thought, but they didn't even fix the problem. A system that checks email addresses for suspiciousness is of course never watertight. Which was also confirmed in the last 2 minutes of your video.
3:48 does an in-house ticketing system require 50-100 devs because ticketing systems require it, or because managers never ever read Fred Brooks? I think it would be interesting to see what would happen if you put a hard cap on a team of 3-5 people and forced them to build radically simpler software just to be able to ship anything at all.
I bet this probably never got above basic triage level and investigated an actual engineer never saw it until real customers started reporting it and management realized they were going to lose money and business and reputation. Exposing potentially very sensitive customer data? Holy hell is that a major red flag and needs to be one of the top things to stress to customer service / triage people.
If you've sent anything to a company that uses Zendesk for support, it's just out there in the wild. They don't apply any security or anything to ticket attachments. I'm sure there are some juicy screenshots out there.
I guess their reasoning is that the mailserver should have validated SPF/DKIM/DMARC and blocked it before Zendesk even gets to see it. Seems somewhat reasonable to me, actually.
They failed to give him tens of thousands of dollars for a bug bounty, and instead ended up paying potentially millions of dollars in lost clients. 👌🏼 That's fantastic business decisions right there.
save 10 grand to lose prob millions in contracts (and goodwill, or lack thereof)
Worth it
Spending dollars to save pennies
ZenDesks response is what makes a white hat 15 year old into a black hat 16 year old.
They grow up so fast.
Well put
He should have sent them a Zendesk ticket.
Well, i've seen worse. Like literally sending lawyers + law enforcement after people who sent the vulnerability + the fix to the company. But of course that doesn't mean you're wrong.
The kid made a smart play by informing Zendesks customers.
For his personal gain yes, the kid made the smart move. But this is at the very least Grey Hat behavior.
Frankly, he is, as most teenagers are, relatively ignorant and overconfident in his judgement (and as Prime said, this can be supercool if you manage to point him in the right direction and cut him some slack, plus maybe, show him some of his own limits if he needs to be pulled back down to earth). The usual thing would be to let the time go by for the company to address the issue, then tell them again that you are going public or inform third parties. The kid really did break responsible disclosure policies and could be sued and prosecuted in criminal court. You might not like it, but if you tread on dangerous grounds, you take some real risks.
Of course this is not as fun as being a bit of talented smart ass who sticks it to a big unresponsive and frankly stupid corporate blob, but that's what ethical hacking is: Not that fun and exciting.
Props to him on not doing a 4chan post
@@grafgrantula6100 I am not a hacker or programmer so forgive my ignorance, but I don't understand how it is ethical to allow a company to keep their customers in the dark about a security/privacy vulnerability that affects them. By giving Zendesk the full responsibility to fix the issue on their own and inform their customers, one has to blindly trust that Zendesk will do the right thing. The customers have a right to know that what they are using is not safe and secure, and there is no indication that they would have known at all. For all we know, Zendesk could have kept silent on their ends until they finally had a fix, and that is way more unethical in my opinion than telling affected third parties about security issues that can pose a legitimate risk or concern for them. Why should the makers of the software have a right to privacy about the flaws in their design? Why should they be trusted to act morally and ethically without any sort of external pressure?
It is not like the kid originally started disclosing this vulnerability on the internet for all to see and abuse, he tried to keep the communication channels private and limited to the affected parties.
Legally, I have no idea if there is an actual case to be made against the kid, but I would hate to imagine that. If this kid can be held liable or be put with criminal charges for trying to protect third parties from a security vulnerability by informing them, then something is seriously wrong.
@@grafgrantula6100 found the zendesk employee
@@grafgrantula6100Actually the reported have informed Zendesk via h1, but Zendesk claimed that enail spoofing isn't part of the bounty scope and decided that it is not a vulnerability. At that point the reporter's responsibility on responsible disclosure is already fulfilled, if the company say it is not a bug, he is free to perform further actions. In fact it is very responsible for him to only inform Zendesk's customers, if he do some public writeup the situation would have been much worse since it is a truly exploitable 0day.
OMG Zendesk.
You just lost a sale. We were considering moving our ticketing system to them. They are one of the top 3 contenders.
It's not the fact that they had an issue - anyone can have one. But this is a shitty way to handle a major security flaw.
So what happens next time they have a major issue reported by someone who doesn't fit their exact specs? As customers, we don't care about their bounty requirements. All we care is that our private data and data of our customers (that they might share in the ticket) is kept secure.
Also, notice how they never accepted responsibility - it was the fault of the products doing the spam filtering, they said.
As if a bunch of heuristics could ever substitute for proper security-by-design.
Truth. They lied in their statement that the hacker violated anything because they themselves told him to kick rocks. This is not a company that deserves its clients.
Same. We were about to get Zendesk, now I'll veto them. Screw that. Their statement is deceptive at best, and clearly their interest to save face is bigger than my interest to not have my company compromised.
Zendesk is awful lol. We never went with them.
What are the other contenders?
22:33 a bug outside of scope and without payment is perfectly fine to be reported to other companies.
It's out of "bounty" scope, but it's in "shut up about it" scope.
@jamesarthurkimbell if its in shut up about it scope, its in bounty scope. Shut up about it scope requires shut up about it money, which is just a bounty anyway
@@johndeaux8815 this is what the humans refer to as "humor"
Exactly, Daniel could’ve just responded “look you made it extremely clear we have no business relationship therefore I am not bound to any of these rules. Either admit this is in scope or I will keep doing the ethical thing of notifying affected parties so they can remedy the issue because their supplier refuses to”.
They weren't even publicly posting about the issue, they were reporting it to impacted clients. They were still acting ethically while being told it was outside of the bug bounty program. This is completely the correct way of handling it.
TLDW:
Daniel: Zendesk, Problem!
Zendesk: Eh, not the right kind of problem.
Daniel: Companies, problem!
Companies: Zendesk WTF.
Zendesk: Oi. You told companies about the problem.
Daniel: Now I’m gonna tell everyone about the problem and how poorly you handled it 😊
Daniel: what you gonna do? Fire me?
you missed last point:
Zendesk: posts a retrospective where they claim they made no mistakes
@@sk-sm9sh it's like "hey, we maybe screwed it up initially, let's make sure we really screwed it up" :D
"...and has caused unnecessary communications on our end."
Amazing and hilarious
Sounds a lot like they are just annoyed that they got talked to by people.
Damn. I thought a European company would be a bit less into the US corporate malignance-by-design
"Unnecessary communication" is the absolute funniest corporate-speak way of saying we had to do our jobs and didn't like it.
this kind of shit is why millennials and especially gen z just won't put up with corp speak.
I think it means: "We don't want to pay you, and we don't want to hear about it or deal with it, because that would also cost us money"
"unnecessary communications" LMAO
nice pfp
I think their ChatGPT instance misspoke, they meant "unwanted communications", because they didn't want to deal with it or fix the issue or pay out any money.
i feel like zendesk really misses the point of bug bounties. they’re not a kindness to the researcher, they’re a fucking ransom. this bug could’ve been worth millions
19:27 the best way to reply here is "I wont because it's out of scope, but I will be working with your clients to fix the issue."
"I have already disclosed all necessary information to reproduce and address the issue, which is currently outside the defined scope. Please understand that I am unable to share further details with third parties, as the root cause lies within Slack’s authentication and the customer’s integration. Slack and any other affected parties are managing communication, and they will provide additional information if needed.
Thank you for your understanding. If you need further assistance in the future, please feel free to reach out."
"You are welcome to hire me as a consultant. My hourly rate is 10k. It will take 4-5 h to write a detailed report on this vulnerability and I'll also be available for potential further assistance."
(Not sure if that's legal due to his age)
@@koncinar This is extortion and unethical due to selling the vulnerability.
@@vincent_sz why? Is he obligated to give it to them? Is there an upper limit of the rate? He tried going through their official channel and they said "no", so now it's his terms. If the price is too high for them, they don't have to take it, simple.
Also, he's not selling the vulnerability, he's selling his services.
He could have sold this vulnerability for millions given the information it couldve granted
getting paid a 5 figure sum while retaining the ability to explain exactly where the money came from AND the bragging rights is not a bad deal, at all.
@@KemalArdilGulez this getting money and some fame is most likely realy more fun, than getting no fame and having to find away to launder a lot of money before eing able to use it.
I even imagine, if you get some disclosure programm with no bounty attached for hacking law enforcement without getting into jail , where you are allowed to claim the fame after law enforcement fixed the breach you found, this would attract thousands of white hat hackers to do that just for the bragging rights.
If this was exploited he would be the first suspect considering the bug report and rejected bug bounty. Not worth the risk.
@@KemalArdilGulezbro would reject 950k USD to brag to some internet randoms. Also selling such a vulnerability isn’t always illegal, especially when selling to a US/western governmental agency…
@integrierterbursche4062 why would a government agency buy this bug for a million dollars if they can just force Slack to give them customer information
After they shunned him the 1st time, he decided he wasn't gonna go through that again. it's 98% their f up
I mean, if you knew ZD, you'd know how the 1st time would play out anyway, though of course you'd attempt to warn them because it's the right thing to do.
you mean the 2nd time - he followed up the initial "not in scope" with a "can you check with zen"
this kid has a bright future ahead
Man, I really hate unnecessary communication on my end.
So the summary is - kid found the bug and reported it - they told him we can't pay you for this, it's 'out of scope', but it's serious - the kid finds out that he can backdoor into slack of major companies and reports it to them directly and they pay him - and finally zendesk claims that he violated the disclosure rules 😂
As a dev in a 100-person company that threw zendesk in the bin and started coding its own tool to implement a few specific features and not have to pay for zendesk while losing all the others and the most important stability, and this without having a dev team dedicated to it.
And now they've hired me and are paying me to turn this in-house solution into a zendesk-like product with 0.1% of the budget.
And when you think about it, it cost them $900 per day doing so...
Now i see where their profits are going
just a doubt, is zendesk is like service now or ibm's bmc remedy?
yeah a ticketing system isn't crazy at all, could easily be done by one dev in a few months ngl
Creating a new product line is different than creating an in-house only tool. It is perfectly reasonable to buy software tools for things you need that are not core competencies. An accounting and payroll program should be easy enough, but I bet most software companies use an external tool. I know all the companies I have been to use external tools.
@@Ganerrr Except that it's not just a ticketing system.
You start adding features... Features that should be in an ERP or CRM.
What's more, the code base wasn't written by a developer, so I'll leave you to imagine the state of things.
I found a bug in MS software and saved a customer probably 5-10 million year over year. I got nothing for it except a thanks from MS. So now i no longer fix bugs. I just sell workarounds.
whaa even big C like microsoft didnt appreaciate this financiallyy?
naive to think otherwise though. what do you think MS would do? give you millions?
@@HolyMacaroni-i8e well atleast couple hundred or even grands still acceptable, but if only "thanks" then its f up
@@HolyMacaroni-i8e no, but bug bounties exist. I don't even try any longer, they never pay out.
@@HolyMacaroni-i8e no but atleast some thousands of dollars. It's Microsoft, ofc people will expect something 😂
Usually the scenario plays out like this: a product manager comes up with a brilliant idea on how to make the product better. It is then tossed over the wall to engineering who have been taught to just shut up and do what they are told. No one thinks over the security implications -- there is no budgeting for anything like this. The engineers aren't impowered so they just expect to be directed and the managers don't know any better and expect the engineers to just handle the security.
and when some engineer brings security up they are told to shut up and continue like it's not a problem
@@kuhluhOG More specific: When the engineers find a security concern, where an exploit is knowable or where watertightness is not proven, they bring it up to the managers. The managers then use their own "skills an expertise" and judge that the exploit is not very likely and thus doesn't need to be solved.
Holy shit Zendesk committing sudoku. The irony of a helpdesk service having the worst PR people and trying to shift blame to a literal 15 year old for poisoning the well when they're the ones responsible for screwing up again and again. Unbelievable.
did you mean seppuku?
@@cameronwebster6866Even if they did, this is not very honorable behavior from Zendesk.
@@cameronwebster6866 It's an old joke to say sudoku instead.
This is not even an issue with SPF or any of the mentioned protocols. It is a logic bug in the application that invites people after the smtp protocol part already happened. Bad triaging.
They are using spam filters to protect customer data. 🤣 wtf.
exactly my thought as well, how is it within the realm of either DKIM, DMARC or SPF.
That's why I stopped reporting bugs. Dozens of critical issues, remaining unaddressed for years or swept under the rug by stats massaging bots... Zero appreciation... So I learned to walk around the bugs instead.
Same. I am sitting on a handful of serious exploits I have stumbled upon because companies treat you like shit when you report things. I would advise staying away from Facebook and ManageWP especially.
Atlas Shrugged and Anthem. Should be mandatory reading for all "ethical" hackers. It can only be "ethical" if ethics is defined fairly.
@@send_loveand it sure as hell can’t be found in atlas shrugged
@@biblebot3947 sure it can. Don't allow others to claim your time, resources and effort for nothing and be spiteful if you refuse to do their slavish bidding in the name of altruism.
@@biblebot3947 why not?
2:57 My fave out of context Primogen quote ever: “I am an ignorant s**t”
They were saying the vulnerability is not policy because they say the vulnerability is in the email protocol, not their app. Which isn't true in this case
They way they do it literally ignores the requirements and recommendations of RFCs:
* 4408/7208
* 7489
* 6376
* 5321 vs 5322
It's not the email protocols. It's literally never email protocols, they're *REALLY* solid.
It's developers thinking they're too good for RFCs/too lazy to read RFCs/thinking RFCs "limit creativity". I have encountered countless examples of all three.
It's always the developers.
The email protocol has nothing to do with the crap system that allows people to add themselves by simply knowing the ticket id. THAT is an insane feature.
@@brentsaner When the sender implements DKIM+DMARC, and both the sender and the receiver implements DANE or MTA-STS it's pretty solid.
Otherwise, not so much. Unless you consider allowing from header spoofing or not preventing man in the middle attacks solid. The amount of domains implementing DMARC correctly is currently still in the minority. The amount of domains implementing DANE or MTA-STS is miniscule. I hope adoption will go up in the future. Saying it's never the email protocols is misleading, as at some point dmarc etc didn't exist, and at that point it was definitely the protocols that were REALLY NOT solid for preventing email spoofing.
I see where they're coming from as email security also relies on others setting up their domain correctly, but they should have designed their app in a way that accounts for email spoofing, instead of just throwing up their hands and saying "not our problem".
@@louis-lau You mean just DKIM. DMARC is purely for spam policy, not establishing trust.
Likewise, *not* using MTA-STS doesn't magically enable a MiTM on STARTTLS cross-MTA. FURTHER, like DMARC, it just defines a decision policy for the remote end. It's up to the remote end to follow/enforce it, which you as a sender have no visibility into (aside from seeing which phishes make it through).
ADDITIONALLY, DANE can be considered neutral for mail. I'd argue it has the ability to add legitimacy to an illegitimate sender by *not* relying entirely on a third-party anchor. Once again, it depends on *how validation is being done by the receiver*. Just like STARTTLS, just like DMARC.
Lastly, no MTA nor MUA software maintained over the last 5 years that I know of uses the From header for any routing decisions unless explicitly configured to do so, they use the From from the envelope.
Lastly part 2, NONE OF THOSE DO ANYTHING TO PREVENT THIS ATTACK. The majority of the mitigations you mention are very specific and very targeted that are more for security *polity* than they are for *guaranteeing safety*. They are by no means bad, I encourage sites to deploy them. But my point stands - there has been no successful attack on the SMTP protocol itself that I can think of that would allow this. It's going to be either receiving MTA configuration, which is far, far less common with transport being delegated to third parties, or MUAs in applications - WHICH IS PRECISELY THE CAUSE HERE. This, like every major vuln related to service-based email that made headlines over the past, what, decade are entirely past the MTA.
It's the developers.
Prime's reaction to first sentence exactly like mine.
I disagree with the mantra of “just do the core thing and externalize the rest”.
The “core thing” for Amazon was selling books, the only reason AWS exists is that they refused to use outside tools and invested in creating their own.
That mentality has destroyed the european industry as top management got to the (wrong) conclusion that sales was the profit driver while engineering was a pure cost center.
Boeing did the same thing. It's what happens to great companies when they put Wall Street in charge of the company.
the whole "sales is the engine" has taken over almost all industries now. people have forgotten that somewhere in the economy someone has to be producing the goods. a land of producers is wealthy land. a land of sales people is an impoverished land where everyone is fooling each other into thinking they're wealthy.
Jeff Bezos explicitly states in interviews that selling books was just his starting point. He never intended the company to be about books, he wanted to ride the big online shopping wave and saw a future where package distribution and delivery was going to be reinvented. And he wanted to build a vertically integrated company from the start. Books were just the easiest place to start. Amazon was never intended to be just a book store. If you want to build your own online bookstore today, you're not going to reinvent datacenters.
@ Yeah, he never wanted to sell just books, but he intended to just sell “stuff”. He never even considered web services as a business.
Amazon is not the only example, Intel also stumbled into their whole modern business by doing something that was outside of their “core business”.
The whole reason the United States engineering industry is the PowerHouse it is, is because we have an entire society that values people saying I don't like the existing solution, I'm going to build something even better. And then I'm going to figure out a way to make it profitable. The whole point is to build a product that is so good that it solves someone else's problem better than the competition, and engineers are especially good at that, because engineers just get frustrated when things don't work right. And then they go build something better because they get annoyed. Never piss off an annoyed engineer, because they'll turn that annoyance into profit.
19:40 "Sorry, sending the proof-of-concept for a non-issue is out of my scope. This was already identified as a non-issue, so I don't see why it's a big deal now. I've already let a bunch of people know about it for their own security so I can't exactly keep it confidential now. Maybe discuss with the other companies and leave me out of it. It is, after all, not important. To remind you of this non-issue, I have attached a copy of the previous correspondence to this email. Kindest regards"
All I've learned from their response here is that NOBODY should be doing business with Zendesk, and anybody who finds exploits with Zendesk should do anything but report them because Zenddesk won't do anything for you.
He knew he won't get paid so he at least had his revenge by making their lives a living hell and lose customers. Brilliant. Wholeheartedly support his method of making them look like complete morons.
This, and also arguably warning their customers was _the moral thing to do_ . They're the ones exposed to the risks, and ZD seemed unwilling to properly address the issue.
In this episode, a corporation shifts accountability into the nether as they slander the white hat hacker for "violating ethical principles" that were voided when his submission was rejected.
classic corporate unaccountability. if they had lawyers they would have played this way better and that says something about how incompetent zendesk is on this.
*EMAIL* is fine. People completely ignoring RFCs when *implementing* email servers is the problem.
"Responsible disclosure" in the scope of Zendesk's T&Cs for the bug bounty is one thing. "Responsible disclosure" for the end customers is something else entirely. Zendesk was not the one being hacked here it was their customers.
If your company supplies SaaS you have a responsibility to your customers, to fix in a timely manner, , to disclose if necessary and to keep the white-hats happy. This is part of what a SaaS company should be supplying as part of supporting their product. Zendesk let their customers down.
Slack channels were the least of Zendesk's customers' worries (any service with a password recovery option involving email was at risk).
In terms of "responsible disclosure" to those end-user companies impacted the white-hat did the right thing.
If customers moved on from Zendesk, because of what happened it is because of how Zendesk handled the situation from their perspective. Those companies care, not because of the mistake (we all make them) or whether a 15 year old gets a couple grand or not, they care about a SaaS company they use who doesn't take their security seriously enough.
The asshole, when their failures are exposed, they will deny and when confronted with evidence will double down and project while refusing to acknowledge. Basically "blame someone else, otherwise it's your fault!" So when you find yourself in a situation like that, take a moment remember "Getting angry will probably make things worse!" A misunderstanding is easy to unintentionally create and extremely difficult to negotiate after. So chill out, and stay frosty.
The issue was out-of-scope for bounty collection, but in-scope for NDA? That's just evil. He really shouldn't have shared the PoC with them for free.
̀he is young and proud. If he was a adult he would know they rip him up and noone would give them the Bug without atleast a 100k paycheck
The community should have his back and bombard zendesk with shit reviews.
Nothing in scope of an NDA is out of scope for bounty. The companies pay for the silence. If they don't cough up for responsible disclosure why bother, just blast it out and let them deal with the fallout. Fck big corp.
@@Blackbirdone11he is young, proud and smart. Refusing to disclose his PoC unless they pay him, right after he exploited those vulnerabilities, would have exposed him to criminal prosecution under extortion charges. As much as I love being petty, it’s not worth catching a felony.
Setting up an email server without SPF, DKIM and DMARC (maybe two out of three - it's been a while) is flagrantly incompetent. There aren't many things to fire a person for immediately, but this is on the list. You can't even get an AWS email server up and running without dealing with anti-spoof first. Now, was it a great idea to add some sort of auth to email? Hell no. They're depending on others to not being idiots.
Dmarc is by itself not a verification mechanism, it only tells email servers how to proceed.
30:25 - Funny he still thinks Flip watches his videos
The wild thing is that they excluded it on invalid grounds - it wasnt even bypassing email authentication. It was code of the zendesk product itself making bad assumptions about emails and using incremental IDs
well, a ticket system may not be "core" but what happens here shows that you still have to be very careful. it would've probably bee cheaper to do an in-house system compared to the fallout of that backdoor.
14:26 the triage acknowledged it as a serious vulnerability. OK don't pay but flag the report as such (at the very least IMPORTANT) and f@ing forward it the security team!
They haven't acknowledged that the bug is on Zendesk authz part, not SPF/DKIM/DMARC which are irrelevant to the issue. "Email = SPF/DKIM" is silly logic.
@@chupasaurus yep was thinking about the DKIM signatures to prevent spoofing.
But it's basically very hard to set up unless you have someone else do it for you.
@@aravindpallippara1577 I think Apple SMTP servers send mail with correct DKIM signatures🤣
I built my own ticketing system for a product I managed while working at a fortune 500 company, and it worked great. Although the product I managed was only used by about 100 employees internally, not the whole company. I built it in WordPress creating my own custom plugins and it fit our needs well and I rarely had to work on it after the initial 3 days it took me to build it.
The craziest thing about this is that ZenDesk ticket threads very often contain PII, such as (ironically) for GDPR requests. Having the ability to add yourself to someone else's ticket chain and get access to all that PII is ... insanely dangerous, not to mention illegal if they do not disclose such a vulnerability when informed about it, and ZenDesk absolutely deserves the blame here for not realizing this.
He failed “no hack November”
This was from October ;)
@@everyhandletaken yeah, because they failing to November from Octuber
wow, not only is it insecure software, theyre also rude
For a ticketing system, you have a user facing number that is incremental, and an id that is a GUID or UUID. And the system only works with the id, and the agents only think about the numbers
I work at a F500 that also sells a ticketing system to most of F500.
Why does the user-facing number have to be incremental? (Genuinely asking.)
@@jasondoe2596 my guess would be for better partitioning in datawarehouse storage? (reduce query times by grouping like tickets together)
or maybe only to have a human-friendly format for users to work with? (e.g. users can see at-a-glance that ticket-123 was created after ticket-111)
i really dont know
Corporate: Saved 10k and got a bug filed!
Reality: Lost millions in business deals and made themselves look greedy.
You love to see it
34:59 I’ve heard about an old slot machine bug (infinite money glitch) where the slot manufacturer alerted the casino, and the casino owner drove to the next nearest (competitor) casino and used the exploit 😂.
But yeah, you can’t control whether victims of an exploit become bad actors themselves. To the ethical question of who to contact, I feel like the kid made the right decisions. I think this is similar to discovering a major product defect, and if notifying the producer does not cause action to be taken, then it is “responsible disclosure” to notify the customers.
I can definitely say Netflix did not use Zendesk when I was there in 2014. It was for sure an in-house solution (using bog standard Bootstrap CSS at the time, too).
The Customers are no third parties. Why should he ever go back and try again if ticket/pii access is out of scope if its based on mail spoofing? He reported his finding to other vulnerable parties with vulnerable SLACK instances where his bug was in scope. It was was totally correct to do so. Even if Zendesk would fix the issue still slack and other tools would be vulnerable through other services.
I really hope Prime gets his own company one day. I'm sure it would do very well.
And thats another reason to just stop using incremental integers as ids.
I once created a certificate validation system based on the hashed name of the user. It worked fine for preventing attacks that want to enumerate participants. It is not worth attacking as it was just for an academic conference. But I didn't feel like exposing everyone's name to the internet.
@vidal9747
Probably good enough, for the purpose.
But i still dont like the smell of just using the users Name. You might be able to replicate the generated id's somehow (If you know that it's the encoded Name) Maybe also add the time to that hashing algorithm or Something.
@@vidal9747 you are probably aware however how only using the user's name as input to a hash function can easily be reversed by lookup tables. If it's going to be an id, you could use the user's name, plus the user's name is UPPER plus the reversed lower ... which would still be obfuscation but a lot harder to look up.
@@send_love depends what hash function was used. There are pretty safe hashing options
@sk-sm9sh no, a hash function will consistently output the same output for the same input, it doesn't matter what hash function you use... it will still output the same value for "ybvb" every single time. I'm afraid you'll missunderstand but that's fine :D
the kid did exactly the correct thing. he went to clients who had the most to lose.
zendesk has very little to lose in this and still managed to screw up.
when they responded, he should have responded as they did. and referred them to clients. and respond through them.
I reported to a client months back (via a deskdesk support ticket) where they were advising me to catch a flight to singapore.
hopefully the company paid for the flight... I tried twice... sometimes you have to pay for experience.
50 people for building/maintaining a ticketing system? How about 5? It's not rocket science. We have a solid framework on which 2 people work, and the other 3 work on customer requests, support and QA, and it's used by multi billion dollar companies. It's a *lot* cheaper in-house than *anything* else on the market. And it's not even a product that we really sell, it's just something we use, not business critical as you say it.
Duck typing for auth seems fine.
To me, it seems like the slack takeover isn't a vulnerability.
But that the vulnerability was the auto adding other emails to tickets from spoofed emails.
The slack takeover was a proof of bigger impact.
Email is not the problem, Zendesk logic on top is
and this is how prime got cited in court from cloudflare (min 2:19)
saved in the corner
I think the point about DKIM is that the companies email server should not have allowed email spoofing.
Then 100% the second bug is an apple, slack, email, zen desk bug. So he is 100% right to disclose to each company.
Not to mention that they paid out.
When he actually didn't zoom on his face.
😂😂😂😂😂😂😂
I work for an MSP. Our ZenDesk contract is almost up and we're actively bringing in another ticketing solution. This incident was part of the decision.
13:25 Imagining tech companies being good people to those doing work for them, what is this, 2010?
It would be as simple as to check dmarc, and record it on the opening of the ticket. Then, if you recieve any subsequent tickets, you simply compare the current status to the previous one.
I think the first chapter of "How to make friends and influence people" says "as you come up with better a PoC:s, keep the owner of the system up to date before setting their customers systems on fire, because that might prompt them to change category of a Hackerone report" or something along those lines
"Hey, did you know if a window is left open you can get through it?"
Zd: "Yeah everyone knows that, but we don't sell windows"
..a few moments later.. "Hey, I'm in your house"
Zd: WTFBBQ!?!
In 1993 I implemented PEM for Trusted Information Systems. At the time DES was not a comercially exportable system : )
26:37 I 100% agree with this decision. He should have been paid before but he definitely shouldn’t be paid afterwards. It would set a really bad precedent to reject paying out a bug and then paying after they take it to the public. People would just be able to elevate the importance of their discoveries by telling affected people.
"We have received tickets from our customers…" reply: "Yes, I have seen those" 😀
He forfeited his award of ZERO dollars for disclosing this big to the customers after being lead to believe Zend Desk was not going to take it serious. I am not surprised Zend is failing. They were big in PHP like 20 years ago and have just gone down hill since then.
Woah thanks for taking this up!
ahahahah "if there is a vulnerability, theres an asshole" amazing
its like saying I will pay anyone who protects your assets, then turning around and saying "oh sure, you reported that my side door was left open, but you didn't give me a step by step map on where all my valuables are so you get no compensation, nothing, zilch, GOOD-DAY!", then once the person reporting say ok fine steps inside the wide open door and goes up to the guests who also said they are willing to pay to be notified about risks to themselves crying that you should've only warned them about the issue, you have already demonstrated your unwilling to actually honour your promises and will use weasel tactics to get out of paying, the guy was well within the bounds of morality to instead take his chances alerting everyone else who was being exposed to risks due to your flagrant disregard to security and hope some of them are more honourable and actually show some appreciation that they were warned directly and able to avoid a potential absolutely devastating hack to take place due to party A's severe lack of interest in addressing real security issues, if anything his decision was potentially even more moral than going back to zen desk given their previous dismissal of the concerns raised, at least now the actual potential victims have a chance to do their own mitigation and not rely on the discretion of zen desk to care
Zendesk learned the principles of FAFO. 😂
is this some kind of competition to find the weirdest container to drink from? How about an oversized toothpaste tube?
If first party responsible disclosure doesn't work... talk to their customers. Especially the big ones. Someone will listen and you wil set the steps in motion. Has worked before.
This is what many people seem to misunderstand about bug bounties. They are not generally intended as a path to communicate with an organisation about vulnerabilities that need to be fixed. They are intended to steer the effort of no cure no pay pentesters. That is fine. But refusing to communicate with people who report valid security issues just because you decided not to give a financial reward for the report is harmful.
What else could have the guy done?
Their lack of imagination should be punished.
If you don't know what you are doing, pay somebody who can.
@@aravindpallippara1577 I'm not suggesting the person reporting should have done something differently. But it's important to realise that if you want to be able to do coordinated vulnerability disclosure and warn others, like the customers in this instance, it's probably not a good idea to enter into an NDA like the ones attached to practically every bug bounty. Most NDAs there will apply indefinitely, even if the report itself doesn't get a reward.
@@FloorTerra Good point! Thanks for the clarification.
@@FloorTerraI would bet good money that those NDAs are unenforceable
@@KeenanV Just keep in mind that even if you win your lawyers keep that good money.
“A Zendesk backdoor? I’m so surprised!” Said nobody
i think this is honestly responsible disclosure, as he was first dismissed, he informed the companies actually at risk that they were at risk. How is that not responsible disclosure? zendesk just didn't want negative publicity and choose to put other companies at risk because of it.
No bounty, then no NDA. Hes young he has learned it now. Dont trust anyone.
Correct. Fck corporate.
22:43 "people are too scared to be cruel, but too lazy to not become the villain"
Their revenue for 2021 was $1.34B USD, and their operating in come was $-166.68M USD? Their net income, which I guess they mean net profit, was $-223.64M USD? So they spent $1.5B to run? What the hell are they spending money on? A billion and a half to run a software firm? No manufacturing costs, no shipping costs, just electricity, rent and staff? Holy hell.
So it's both "out of scope" on the bug bounty program, and "in scope" in terms of responsible disclosure? Nice one Zendesk, very cool.
Zendesk also jacked their $$. Cancelled them a while back. Looks like we made a good choice.
19:25 I think it would have been very reasonable for him to request that the issue was re-evaluated as being in scope first before sharing those details. At that point they are literally requesting free labor. You can't have the cake and eat it too. Either the bug is in scope and you get the details or it's out of scope at which point you have determined it to not be important enough to need the details
To be fair, the reason scope and out-of-scope are defined is so that companies can avoid getting ganked from all sides at the same time and bring all their teams to a bug fixing grinding halt.
Also, going out of scope and expecting a reward is unfair to the other security researchers who participate in the program and keep the ball between the lines
To Zendesk: git gud😂
Personally I believe the bounty program scope needs to be simplified to "We don't want you to share this bug with our clients so here is the 'shut up' money for you." If the company doesn't care about the bug going public then no bounty for you. If they do, then there is a bounty.
Sweet! I just used this to piggy back myself on the persons ticket before the one I just previously made because their Id #
Run concurrent to one another. I was able to get a checkout code for 75% off entire cart that was given to resolve a complaint about shipping. I'll be swimming in e-juice.
The reason why Zendesk is doing so bad financially is the same reason the honest man didn't get rewarded.
Leadership is the opposite of what would have lead to opposite outcomes in both cases.
In other words:
If ( CEO == eval(-iq)) {
effects(-iq)
}
i have seen this with a spoofed email that got a company to wire money to a random location
Not Invented Here for tooling, like everything, is about balance. How well does the thing do what you need it to do, and what would it take to do it inhouse, and to what degree would the inhouse version be better than the outsourced? Making an ecommerce site and remaking Shopify is dumb, even when you could get a more purpose built equivalent, because it's just so complex, and will basically never in reality be better (even when better in some ways). But remaking some of the basic shopify apps people install? Probably great, because most of them are terrible as is, and most aren't even THAT complex either.
@14:16; The issue is that they are marking as "Informative", and aren't saying "Thank you for the responsible disclosure - while we won't be paying you for this, we will work to fix this issue, and we will follow-up on when we have fixed this and you can fully disclose this issue that you found after we have patched it in."
The person wasn't even given permission to just reveal this publicly, or told that they *will* be able to reveal it publicly.
They effectively said "This doesn't count for the program you're reporting this from - you'll need to fill out the form in a different program, over there - it's a blue form.".
Most likely, the SPF, DMARC, DKIM out-of-scope in the policy was supposed to be for emails FROM zendesk themselves, not, an attack against their product that should depend on these email authentication schemes. Excluding SPF, DMARC, and DKIM is common for many policies, but very few companies actually build their products on top of email.
Rules are made with intention. If the rule contradicts the intention, that is an exception and you should follow the intention instead.
Making in-house solutions is just a matter of cost, licensing vs development. Though it can be very hard to predict what it means to develop and maintain a service, while a license cost is predictable.
Yeah they never triple or quadruple or even ten fold more! Just never happens licenses are predictable!
So what they are telling, is that Zendesk bugs are for selling, not reporting?
At the end I thought, but they didn't even fix the problem. A system that checks email addresses for suspiciousness is of course never watertight.
Which was also confirmed in the last 2 minutes of your video.
3:48 does an in-house ticketing system require 50-100 devs because ticketing systems require it, or because managers never ever read Fred Brooks?
I think it would be interesting to see what would happen if you put a hard cap on a team of 3-5 people and forced them to build radically simpler software just to be able to ship anything at all.
I bet this probably never got above basic triage level and investigated an actual engineer never saw it until real customers started reporting it and management realized they were going to lose money and business and reputation. Exposing potentially very sensitive customer data? Holy hell is that a major red flag and needs to be one of the top things to stress to customer service / triage people.
If you've sent anything to a company that uses Zendesk for support, it's just out there in the wild. They don't apply any security or anything to ticket attachments. I'm sure there are some juicy screenshots out there.
I guess their reasoning is that the mailserver should have validated SPF/DKIM/DMARC and blocked it before Zendesk even gets to see it. Seems somewhat reasonable to me, actually.
Hiding behind technicalities is such a corporate move.
We were in the phase of starting to get inquiries form the zendesk sales team. I will stop the negotiations now.