This is a fascinating tale of just how reliant science is on the integrity of researchers. I review pharmaceutical data for accuracy, but there is literally no way for me to check whether the readings recorded by an experiment's author are not just made up. Some experiments require a witness sign off (mostly for controlled substances experiments), but that just decreases the chances of fabrication. There's no way to eliminate the risk of fraud entirely without making the process so cumbersome that it would no longer be feasible to do the work. Data that is used to determine the efficacy of drugs that we *put inside out bodies* could be fraudulent and we might not find out about it until years after the fact, if we ever do at all. This seems like a system that should quickly collapse due to corruption, but somehow it works well enough for new drugs to be discovered, distant black holes to be photographed, and perhaps, one day, for the personalities of spiders to be cataloged.
I am just shocked that repeated data used and copy&pasted wasn't simple more random. Like ok i studied computational simulation, so maybe i am biased... But it would he fairly easy to simulate data and make fake results look much more convincing... Obviously this wasn't needed because everyone just trusted the data.
@@alansujansky8591 thing is with random data you couldn't produce those beautiful plots as easy, unless you work more on how to control that randomness and get the plots at the same time, which maybe Jon was lazy about doing
How big are your datasets? Though tbh its part of why you should do some data exploration and descriptive statistics before you get to any actual statistical work, and then publish all data and code. @SonicluNerdGamer
Isaac Asimov’s first non-science fiction novel, _A Whiff of Death_ was a mystery about someone at a university being murdered for faking his data. It wasn’t very successful, and a lot of the non science reviewers criticized it for the motive, ‘cause that’s no reason to kill someone, and all the science reviewers were like “the little twerp got what was coming to him!” [Edited to correct a typo.]
the most insane thing about this is that if this guy would have been like 10% smarter he would have gotten away with it for his entire career, and there are absolutely without a doubt probably people out there getting away with it right now
Totally agree. I've experienced this on a smaller scale. In my work we commission sewer surveys and sometimes there are some missing measurements, when we ask the surveyors for the missing data occasionally they will come back with a number that is so far from the expected value that we instantly know it has just been made up. If they had taken the time to look at there own survey photos they could make up a more realistic measurement and we wouldn't even question it, and I'm sure that does happen.
I'd revoke their title because someone with PhD should simply be smarter... One of 1st things they thought us in statistics and experimental physics classes was how easy it's to spot botched data, and how it is virtually impossible to get caught faking if you know what you are doing. If you are publishing experimental results you should understand the statistics to be able to draw proper conclusions... OTOH... It's easy to catch cheaters lol
Hey man love your videos. Yeah imagine if he had just not left the data faking formulas in the public spreadsheets. If he had fuzzed the numbers a bit so it wasn’t obvious there were repeating exact sequences.
As a very evolved and bold spider I broke out in a sweat multiple times as I watched the video - I believed they uncovered us... cant believe I can sweat also but according to the paper about Sweaty Spiders there is data to back up the very existance of my subspecies.
I guess a follow up with them is in order, at least the opportunity to comment, or if they can't be found, going to Africa to do a follow up with their fellow members.
@@haldorasgirson9463 it really would, the thing is, he probably started fine didn't find anything and made it up for the perks. Redesigning experiments and make a few tests, maybe, redoing all his work to get valid data doesn't make much sense. Not to consider the meme of the subject, if somebody does try to explore it..
academia most certainly rewards lying. like you said, confirmatory studies never get funded, but even worse than that, i've been told by a Phys Rev D editor that they don't accept pure rebuttals - you have to write a whole new paper with new results and conclusions. and Pruitt's upward career trajectory proves the worst part of it - that had he simply published a few dozen papers showing weak results or no correlations, he'd have gone nowhere, but showing novel correlations over and over and over was the key to his successful career. it's the reason why juvenile forms of known dinosaur species are given new species names, it's why nothing can be reproduced in behavioral psychology, it's the reason why Berkeley had to retract its discovery of element 118. being "good at science" is *not* how you become successful in the field. being "right all the time" is, even if you have to lie about it. i wonder sometimes if the need to be right is responsible for the proliferation of so many competing and orthogonal statistical tests for novelty. in other words, not everyone is willing to fake their data, but they are willing to use a weak statistical test to prove their hypothesis.
It also rewards publishing many papers even if they don't really add much or anything at all for scientific progress instead of making fewer but more meaningful experiments. So we have lying + thinning out the meaning of geniue studies
The Russian language has a concept that we are lacking in English - Vranyo: I tell you a ridiculous lie. You know I'm lying. I know you know. I tell it with a straight face. You nod seriously and take notes. You repeat my lie with your own embellishment. The person you tell it to ... nods seriously and takes notes. As long as everybody lies, and everybody accepts each others lies ... all the liars get ahead. The troublemaking maverick who tells the truth gets left out. When all the lies get compiled and regurgitated by an official source, it becomes Gosvranyo. It's not that we don't do this ... our language just doesn't express it as concisely..
@@tomlxyz None of what was described above supports a claim of "thinning" Thin-NING implies an INCREASING TREND of this over time, which you've given no reason to believe is true. All these same motivations existed in the 1700s as well, or whatever, and would in fact have been even easier to get away with than now. So I see no reason to suspect some sort of snowballing catastrophe, as opposed to just a constant, static level of baseline corruption and inefficiency that we've been dealing with forever, but isn't enough to actually stop progress. Just makes it slower and more expensive than maybe it could be if we come up with better systems someday.
@@gavinjenkins899 I think you just don't know a lot of the crappy texts published today as scientific articles. It is certainly an INCREASING problem, nowadays even speeding up due to predatory open access journals, where one essentially just pays some money to get published with 'peer review' that does not deserve its name. It is also common nowadays to write papers about known stuff and pretend that they are about new things. And it all gets published without problems.
@@gavinjenkins899I think you misinterpreted this person. It's clear what they mean by thinning is not a change over time but rather that the papers are being thinned as a result of the incentives and are hence less meaningful. People are willing to write lots of nonsense rather than big publications. No time dynamics involved there just a statement of now
"Why would they use a spreadsheet?" Oh, I envy your entire field. Basically all of biology is done in Excel. They've had to rename genes in contravention of the Principle of Priority, even, because Excel interpreted the original names as dates, and errors were getting into the published literature!
There were literally multiple times during this I said "no...." and felt the hairs on my neck go up. "There were literally equations in the spreadsheets!!!"
How was Pruitt so prolific? Well, when you make up the data and don't actually do any experimentation or observation, it's AMAZING how quick it is to write a paper.
I cant believe physics Jenny Nicholson even had a spider arc. The comparison just gets more and more apt with every video on this channel I watch A...and I love that, to be clear
I had no idea what you were talking about so googled "physics Jenny Nicholson" and somehow found the youtube channel and watched the spider story which was relatively brief for her - but delightful! I'm almost hoping for another Covid so I'll have time to look at all her stuff as I'm quite the one for falling down the rabbithole as well.
This story is kinda extraordinary. Whilst you were explaining the surprisingly amateurish way he faked his data I was thinking "why would you do it that way, why wouldn't you write a simply Python script to create random data with the distribution you are trying to demonstrate?" (whilst feeling slightly guilty at the fact I was thinking of ways to fake this better than Pruitt did). But then something scarier occurred to me: perhaps there are scientists who did fake their data better than Pruitt did. Obviously some people are willing to fake data in order to further their careers, and I can see a way to do so that would be harder to detect than his was*, so how do we know someone else hasn't done exactly this? What other area of research might be built on fake data and will we ever know? How far will the tree of subsequent research grow before we do? That's scary. My sister worked in a Psychology lab years ago where the PI was selling educational software for kids, and whilst they weren't faking data (or at least as far as she was involved) I remember her being extremely uncomfortable with the manipulation the team's statistician was doing to make results look significant. I worry that this sort of thing is far more common than we imagine. Anyway, really interesting story. As you say, it's worth being compassionate towards Pruitt. Unless he does a tell-all, I suspect we'll never know the exact details of his fraud, but if it went all the way back to his PhD it's easy to imagine a panicking PhD student whose experiments aren't going anywhere making up data to avoid failing**, and when they got away with it doing it again, and again, and again. I can also imagine the panic he must have felt when he saw the whole thing unravelling. Obviously he shouldn't have done it, but I do have some sympathy for him**. *I suspect that it would still be detectable by someone clever enough really looking for it, but it wouldn't be "Excel formulae in the cells of the raw data obvious" and it might be harder to convincingly prove. **As someone who made themself really sick with stress writing a thoroughly mediocre maths PhD, I do understand just how difficult being a PhD student can be.
Oh, absolutely---same---I was thinking the same thing. So much better ways to fake it if that is what you want to do. So, the conclusion of the video is important. We must be willing to fund duplication efforts. And to pay attention if research is NOT repeatable in a different lab.
What you describe is what someone who sets out to be fraudulent would do. I'm sure with each study Pruitt did, he had "good intentions", but the pressure to publish something positive and significant pushed him to fudge the data. I mean, he had actual post-docs doing measurements. Many of the numbers in the spreadsheet were likely real measurements. But he needed to make adjustments until he got something worth publishing. I didn't take from the story that EVERY NUMBER in the spreadsheet was completely fabricated - just that he was making adjustments until he got what he wanted to publish. That might be a distinction without a difference, but it reflects the pressure these people are under to "publish or perish". In the end, he might not have been caught if he totally fabricated the data.
I was shouting exactly the same thing at the screen! I can only imagine he didn't really have a good understanding of data science and didn't realize how flawed his method for faking data was.
I am terrified at how much fake data there could be. My Post-Doc involved a lot of complex data manipulation (it was a differences-of-differences kind of study, so the level of manipulation is necessary and accepted in the field). And my results, as a post-doc, were mediocre. It occurred to me, when I was a post-doc, that it would be trivial for me to fake a result. The differences we were looking for were
@@alexanderbrady5486 Here's something even scarier: you published a mediocre paper and are out of academia. Pruitt faked his data and was an academic rock star. Given that only a tiny fraction of PhDs end up in academia, how many of these "success stories" do you think gained their success without fudging their data, or engaging in other unethical practices? The overpopulated academic bubble economy, combined with a metrics-based faux meritocracy, is a duo from hell suffocating science.
God this is an actual real-life nightmare. I already have nightmares about discovering an honest mistake in mine or my colleagues' data years after the fact. The level of betrayal to discover a colleague or an advisor knowingly falsifying data is just unthinkable.
What one expects from a scary story: Normal people: The super aggressive spiders escaped the lab and are now on the loose. Panic time! Super Nerd Scientist: THE DATA WAS FAKE!!!!!!!!!!! Seriously though, your a great storyteller.
Yeah after the intro I was expecting a story about an invasive species because of improper lab protocols. I think the fake data is less bad than that, only because it's possible to fix, whereas invasive species can be impossible to eradicate.
What no one realized was that the spiders were actually far more intelligent than anyone knew. The data, if it hadn't been faked, might have revealed this and perhaps the events that followed would have played out very differently. Could the Era of Humans have continued if we'd been aware of this threat earlier? Would we have failed this test anyway and ceded our primacy to the ultra-organized culture of sentient arachnids no matter what? Would Pruitt have become a spider collaborator either way? We'll never know, and soon, there will be no one left to ask.
Maybe I'm heartless, but I have no sympathy for Pruitt. Every time he took an academic position, he was taking that position away from other applicants whose research was perhaps less startling, but almost certainly more honest. And he knew he was doing it. Every time.
Its happening! Nearly 50k subs. Excited to share these awesome well told stories with everyone. I appreciate that this physicist is taking the time to make free content so I feel compelled to help spread the word even in the smallest way.
22:40 So they're *literally* doing peer review, and Pruitt is like "NOOO! Don't do that! That's not how science works!" It would be hilarious if it wasn't so maddening.
It’s astounding that the data fakery was so incompetent and so easy to detect. And it’s terrifying how easy it would be to fake the data in a way that would not have been detected. I’d heard about this scandal, but your telling makes it much more vivid. Great storytelling! And your observations about academia are right on.
To fake the data in an easy to detect fakery way, is part of the thrill. In the Netherlands we had the famous professor in Social Psychology Diederik Stapel. He started out embellishing the results from questionnaires, e.g. among school kids. Then he rented a cottage in the woods and would fill in the spreadsheets by hand, during the weekends. He would no longer bother to actually visit the schools for his questionnaires. In the end he would even fake the schools. That is, he would "visit" schools that not existed. He made up a school on 500 m distance from his university lab, and nobody discovered that this nearby school did not exist.
@@ParameterGrenzesomething something "smallest tinge of guilt and responsibility" for making sure whatever damage was caused could still be "reasonably reversed" even if that means decades if work end up going down the drain immediately upon discovery -- which I guess is better that they can find out they are chasing imaginary birds sooner rather than later when too much has been built up and the entire house of cards fall down
I quit academia before publishing anything because my advisors were so awful they literally told me "no one cares about the data, we won't include it anywhere, just publish a good result so we'll get attention" when I asked how to go about including my dataset, this was in like the early 2010s
Schön did make one mistake, his supposed research was readily commercializable and used equipment and techniques most people in the field would be at least familiar with. So, there were a lot of replication attempts almost immediately, and, while it took longer than it should for people to figure out that it wasn't them-problem, it was Schön-problem that there was no successful replications, he was doomed from the start. Behavioural research on animals that aren't your standard lab models, and so require some special arrangements to get and probably special expertise to simply keep in a lab... Even if you wanted to replicate, even if you somehow got funded, guess whose lab would you ask for that expertise? And if you don't, they always would have an excuse that your lab did something wrong because you didn't ask them. It's an almost perfect niche for an academic fraudster. Which is scary.
This is such an impressive story. Mad respect to Dr. Kate for her integrity. It would have been so much easier to things the wrong way, but she went through the excruciating process of correcting the problem. What a wonderful thing, to be pruning bad research out of the literature.
You hit the nail on the head in terms of the job season awfulness Pruitt's coauthors had to go through. I'm friends with a few of Jonathan's former collaborators and this has been devastating to their careers. One pre-tenure prof with half their research program called into question and the other a current graduate student who had to completely overhaul their dissertation in the middle of doing fieldwork. It absolutely sucks that for every fake data scandal there's a trail of honest people who've wasted years of their lives investigating a made-up phenomenon and had to question the validity of their entire subfield. (Also, yeah we use bioarxiv, but I don't think biologists post as many preprints as physicists, culturally speaking.) I completely agree that the incentives are completely messed up and it's easy to see why Pruitt did what he did...That said, this is second-hand hearsay, but I've heard that Pruitt treated his students terribly behind closed doors. So I don't particularly feel any sympathy for him.
The reality is academia has a huge problem and these kinds of stories are not uncommon. I ended up not doing a PhD because I was accepted into a program and the Professor running the program was caught publishing books which plagiarized his grad students, all he had to do was make them co-authors but I guess he was too greedy. They suspended him which shutdown the program and given the timing if I wanted to do my PhD I'd have to wait a whole extra year and get accepted into a different program somewhere else... This is despicable and universities need checks so that this kind of dishonesty can't go on for years before it is caught (which was also the case in my situation). If more people were caught and held accountable the first time they tried this BS then far fewer would ever attempt it in the first place. This also dramatically undermines the public's confidence in the whole scientific establishment.
OMG I love academic drama! It's like those criminal story podcasts, except nobody's dead, however, knowing that spiders may not have personalities is heartbreaking on itself💔 Thank you very much for this video!
I have had hard time explaining to people why learning Python or R is important when you can do a lot of stuff in Excel with seemingly less effort and this story encapsulates the faults of Excel so perfectly it's astonishing
When I hear stories like this or Schön, I wonder how many fake results are out there. How many people made up their data, but were smart enough to not make earth shattering claims and remembered to add noise, not recycle data and keeping the sample sizes realistic. I'd guess it's really really tough to prove data is faked when the person faking it understands what they are faking, is trying to make it hard to detect, is competent with computers and takes their time.
I think there is a category problem that limits this. The kind of researcher you describe needs to possess three traits; - Know, understand, and care about all the factors that are necessary for convincing fake data. - Be willing to put in the work to create high quality fake data despite knowing that it likely won’t be checked. - Be willing to fake data. Problem is, if you’re lazy and slipshod enough to be willing to fake data, you’re probably not willing to put in the work to do it really well. After all, you could have just gone and done proper science with that same level of effort.
@piedpiper1172 the point hammered home in this video is that the pressures inherent in academia would promote even hardworking non lazy scientists faking data. The best metaphor I can think of is that Dream minecraft speedrunning scandal. He was still a top mine craft speedrunner but the run itself had inherent probabibilities making good runs very unlikely. The same is true with science because you could be working on something that goes nowhere and limits your work and research oportunities despite doing the science perfectly on what, only in hindsight, became an inconsequential hypothesis.
I'm glad you got to the incentives. I wonder how many of the academics "at the top of their field" are there because faking data is the only way to keep up with the pace and compete, especially when there's a decent chance everybody you're competing with at that level is also faking data.
This sounds analogous to the problem of doping in cycling. The only way to be competitive against everyone else who is doping is to also rely on doping oneself.
Pruitt was the one who got caught because he was so bad at hiding his traces. If he just added a small normal noise to all his faked data it’s likely no one would question the data in the first place.
What I experienced was that a couple of the supervisors I worked with were very busy men, with minimal involvement in their research. The work was almost completely done by grad students and post docs. The supervisor's seldom participated, or even checked the work of their students, but on the other hand they were constantly pushing for publishable results. In my case, I was not only threatened, I was kicked out of a Ph.D. program for failing to come up with publishable results. So as a grad student there is a very strong incentive to come up with something that can be published, and a very low probability that if what you come up with is largely fabricated that your supervisor will either be familiar enough with your work, check your work carefully enough, or possibly even care enough to notice. This is significant because a person who cheats and gets away with it as a grad student is very likely to continue in the same pattern as they continue their career. Based on my experience what is really needed are graduate student supervisors who are intimately enough involved in their student's research, and sufficiently motivated, to catch the fake results before they get published. It needs to be difficult to write a research paper that is good enough to pass your supervisor's scrutiny. In Pruitt's case there is an argument to be made that his Ph.D. supervisor should bear some responsibility for failing to properly supervise his research. I might even go so far as to comment that research work is intense, and highly competitive. There is a personality type that thrives under such conditions, and people that have that type of personality are the ones that end up as university faculty. You don't have to spend too much time with those people to see the impact that has on their interpersonal relationships.
I’m not an academic, I got my masters degree in electrical engineering and got a job. I’m shocked that something like this can happen. I always assumed that because science is all about reproducibility, that people would be attempting to recreate results from published experiments. I guess that’s why people assume there is no way someone could fake data. They kind of go “how can you fake data? Someone could try to reproduce your data and expose you,” but nobody ever did
You want it to get scarier? read "Reproducibility of Scientific Results" from Stanford edu, Stanford Encyclopedia of Philosophy (for Philosophy of Science people, I presume). This could happen a lot in the current way of things, and that was 2018 - it is probably worse now.
@@masonwheeler6536Just a little while ago, a professor at U Minnesota intentionally tried to commit bad code to the linux kernel. many of his CS students were involved.
@@masonwheeler6536 I think it's somewhat wrong to compare this case with Heartbleed. Don't get me wrong, it's the same "type" of problem, but the degree is so much different that it turns into something else. Heartbleed sorta existed for two years give or take, in some systems, not all. It was detected and could be fixed. It was a vulnerability but it could only affect systems with said vulnerability. The Academia issue is that the vast majority of data in the past decades wasn't validated. Even some big shot papers don't get any meaningful analysis. It's simply too much work for few hands and the system does not encourage that menial labor. Overviewing open code is relatively cheap, quick and actually develops your skills. Replicating studies takes long, costs as much as the initial study itself, or even more, gets your career nowhere and doesn't make headlines unless it's a really trendy topic. And unlike code where even if replicated, each instance is unique, citations bring the original paper along with them. And to make matters worse, Heartbleed was a massive vulnerability, but wasn't a dumb mistake. It's crazy it wasn't noticed for two years, but to exploit it you needed the right conditions, expertise and some luck. What Pruitt did was the equivalent of you trying a basic SQL injection and drop all tables on the entirety of Alphabet services including backups. He did the lowest possible form of faking data, and it took a decade for someone to catch on. And if he wasn't stupid enough to leave so much evidence, it would have been written off as a data error. And (and again) in heartbleed, each affected instance is a singular issue. In Academia, each fake paper can lead to dozens or hundreds, maybe thousands of retractions, corrections ,and so on. Imagine that, across two or three decades, across several fields.
People are too naive. Everyone should know that the outside appearance of ANYTHING is not equal to the inside. We live in a capitalist society which is all about lying and promoting lies (usually it's called advertisement, public relations, outreach, election campaign, product information, to name just a few). Telling the truth gets you into trouble, telling lies brings you in the highest positions.
i love learning about drama in random fields im unfamiliar with. that chess cheating drama, the fishing contest cheating drama, now spider science drama
With the whole "scary" buildup and singular villain as well as your hip, matter-of-fact delivery gave this a Scooby-Doo vibe. Like you're Velma and dude's gonna pop up and say he'd of gotten away with it too except for you meddling kids. 😂
The really scary part about this is that this was only discovered because of both how open data sharing is in this field, and because of how wildly incompetent his data faking was. If he had simply written a Python script to generate random points from a desired distribution, nobody would have ever identified it. If his spreadsheets were kept private, no one would have any evidence. Other scientists are faking data right now, in every single field of study, and nobody knows who. It feels like the only way to catch this is with reproduction studies but as you say there's very little incentive to do those on small results like this.
The FIRST thing I was taught to do in grad school (and I was in grad school multiple decades ago) before I did any analysis in SPSS was to give my data a look with the human eye to see if there's anything abnormal, especially if I wasn't the one who recorded it. I was taught to sort it by different fields, look for missing values that may affect analysis, make sure it's all the correct unit, make sure it's all SI or English, check if a fellow researcher accidentally mislabeled something, etc... look at it before you have your stats software do anything to it, even if it's a million rows ... at least play with it a tiny bit to see if there's anything strange, do a couple countifs, vlookups, etc. just to make sure things are ready to analyze.
I was taught to do it doing research as an undergrad. Look at the raw data, try graphing data different ways to see if any anomalies pop up, do statistical analysis on the results, be suspicious if the statistical analysis comes back _too_ good... All of which is to say, the repeated data immediately caught my eye as something which could have been easily caught by plotting it. There would have been suspicious quantization artifacts. The topic of deliberate data falsification was never brought up though, nor the idea of doubting your co-authors. The concern was wholly focused on catching equipment failures, math mistakes, and so on. Something else that this story reminded me of: Taking notes on carbon copy paper. It's a good practice, but there's no point if the originals and copies aren't stored separately by different people.
@@AlabasterClay to be honest, I don't think that's too bad. Like despite faking his data (meaning he shouldn't be able to do academic research), I'm sure he has a perfectly fine grasp of high school level science concepts.
Dr. Kate Laskowski's action was heroic. I'm so pleased her career has gone on and she has a thriving lab at UC Davis. I'm glad Dr Dan Bolnick's career is going well and especially that the spider research is being restarted. I mean, spiders having personalities would be so cool if it turns out they do. If Pruitt wants to really repent, he needs to set out in detail what he did and what pressures he was succumbing to that made him do it. That would be really hard for a person to do, but it is what is needed for him to regain any honour or integrity.
2:44 - That is a dead or dying Agelenopsis, not a spider in a defensive or 'timid' position. If you see a spider in this position, try giving it some water.
This is criminally underviewed. I routinely watch mini-documentaries and essays about things like financial crises and machine learning and the rise of AI, and a physics PhD posts the most engaging TH-cam video (and about a publishing scandal in biology, no less) I've watched in a couple of years. Bravo And here's hoping the changes to the institutional landscape that allow for a reform of academia and academic publishing come sooner than later.
I find the other scientists involved in this story amazing - they are what we should expect scientists to be. The story is a good lesson to use for reforming academic publishing.
This is a mind blowing story, you tell it incredibly well and most of all I’m blown away by your empathy. I’m outside of academia but this story, how it’s told just humanizes it all so well. This is a seriously good video, thank you.
I feel like I've been hearing about this worrisome trend in academia for decades. A baseline expectation of trust just doesn't work when all the incentives align with intense pressure to take short cuts. Like even expressing interest in redoing an experiment and retesting data is thoroughly discouraged. If it's not novel, it's not worth anyone's time. That's such a reckless platform on which to build a foundation of future scientific discovery.
i feel like the scariest part of this story is how bad jonathan pruitt seems to have been at faking data. like, copy-pasting the same numbers and leaving calculations in your sheet feel like pretty basic errors, and yet he still got away with it for a decade. it's upsetting to think about how many people might be even mildly better at faking data, or even just mildly luckier.
Would you be able to keep up faking data over and over? He had to hard code thousands of "measurements" over dozens of articles. I don't think you appreciate just how tideous a process that is. Have you ever had to fill out entire collumns of data by hand? It's insufferable. Thank god i can put and equal sign and be done with it. As for cheats and frauds, good riddance.
@@sakesaurus 1) you know who else has to hard code thousands of measurements over dozens of articles? actual biologists doing actual biology. and they have to do that on top of actually doing the experiment and taking the measurements 2) i know how to copy -> paste special -> values only
Im a biologist. I got my PhD in 2013. I don't do research anymore, but I can tell you that I, and.most everyone else i knew when i was in grad school used spreadsheets for most things. I sent and received data in Excel files all the time and i mostly trusted that no one would fuck with it. Besides, i had a bajillion backups so i could always figure it out of anyone did something fishy with my data. Also, most of us aren't dealing with huge data sets and there were (and i imagine still are) plenty of programs with nice cuddly UIs that require no programming skills whatsoever to learn. In short, you could quite easily get by without knowing how to use much more than Excel and SPSS (e.g.), and many people , including the faculty, did just that. Students were more likely to know some coding or how ro use R, but if your advisor doesn't value it and doesn't think they need it, they're not going to push you to or even allow yiu thr time to learn it either. That said, if I were faking data (which I wouldn't) , id use a random number generator to produce values within the range required to generate the results I wanted. I'd also copy the values to a new spreadsheet, save over the evidence with a blank file , and delete the file.
As an archivist -- I understand the appeal of an Excel spreadsheet, but oh god that's a nightmare for data integrity and preservation. Thanks for giving me something new to keep me up at night. 😂
ปีที่แล้ว +3
@@captainnemo7690 We were always taught, and it made sense to me, to backup data in raw format always so in the worst case you can go back and edit it from scratch again.
It’s because biologists aren’t scientists 😂. Sincerely a physicist. But seriously how is it ok to use something without an audit trail, backups, and non-automated (rife for human error), as the default tool you use. It’s scary. (Just joking obviously, but tools matter.)
@@astronemir I invite you to do a seminar series teaching old crusty biologists how to do all of that. Just give me a heads up before you start so I can get my popcorn
These videos are amazingly well done. As an engineer, who isn't a scientist, it's extraordinary to see the insight into how academia works as a social structure and capitalist endeavor. Angela, you're a master storyteller.
How we say in German "Das war zu schön um wahr zu sein" (That was to good to be true). And it probably starts with the small things. When doing experiments in school, who is ever going with a more than 2sigma from the accepted result? Should happen often, but I guess really happens. Just remove a 'faulty' data point.
If the spiders can hunt exponentially bigger prey, then we should make a log graph of the mass of their prey. then we can predict when the spiders will have evolved to hunt a human toddler. It's important to be able to predict threshold when the human species loses its apex predator status on this planet to the arachnids.
Dr Collier because I also watched your video about AI, I decided to ask ChatGPT if spiders have personalities. Chat GPT said yes and recently researchers had detected evidence of personality. I asked who did this research and you will not be surprised that Pruitt was the researcher chat GPT relied on.
What surprises me is that this doesn't happen more often. It varies from lab to lab, but some academic labs involve so much pressure to produce publishable results. For anyone who is between their PhD and academic tenure (which has become immensely more difficult to achieve in the last half-century), there is constant pressure. Lab supervisors are often awful people-managers, and shit always flows downhill.
Love the storytelling approach and that you dug into the way the story unfolded. Such drama! Really entertaining :) Favorite quote: "That's an interesting question." ummmmmmmmm. In this situation, I always feel bad for the people who try to replicate the data and end up wasting epic amounts of time. Replicating work is hard enough as is, without it being FAKE! Also, it's interesting that it took someone looking at the specific data points and noticing the weird lack of variance as a red flag.
@@acollierastro - What also scares me is that, if he used a better method of faking (just randomizing the times slightly), it's likely that none of this would have been caught ever. Anyway, thanks for the fascinating account. I remember when the story made mainstream media headlines, but I did not follow it closely; and it's great to hear it analyzed in a way that captures my interest. Kudos!
There was something about youtube that started to annoy me and I couldnt put my finger on it. But whatever you're doing in these videos is the antidote for it. Thanks for storytelling. From one researcher to another I really appreciate the kind of stuff you're coming up with here
This kind of TH-cam content is my Tiger King. I don’t know how often these sort of shenanigans occur in research, but if there was a channel devoted to these kinds of stories, it would totally be my jam.
You might want to check out Bobbybroccoli scientific fraud is his jam. Schon, ninov, genetic cloning fraud, etc. All multiple hours long. Really good stuff. He also delves into some stories that are just incompetence/bad luck whatever like the Supercollider that was supposed to be built in Texas.
loving these video essays, your storytelling, the way you speak, and the editing are all spot on. Very pleasant to listen to. I definitely see this channel going far
My dad and I both watched this video separately and had a conversation about it. Happy to see you getting recommended more, I really love your style! :D
Angela is a dangerous woman, brilliant and brave. Doing the necessary work of exposing the dirty underbelly of academia no one else is willing to do. She should not have to sacrifice her dream career to warn her colleagues of the unseen dangers. This work needs to be recognized and rewarded. ❤
Angela, you're a fantastic TH-camr and a great help in bridging the gap to us normal non academia people, hopefully I'm not the only young mid 20s non academic that sees how awesome your coverage is of stuff like this. You're a great speaker, made things very clear without dumbing it down, and your sense of humour is fantastic. I love that you're unapologetically you and you not only talk about things that happen and cover the event, but also made a stand on sexual assault and harassment, mental health, and the reality of taking adjunct positions or taking a 3 year post doc to uproot your life and spend a decade making nothing to become a prof at an ivy league or even just become a professor in your chosen field and specialty. Really opened my eyes to the reality of the space and the things you talk about and it means more coming from someone like you who is very honest but also makes things digestible and fun for those of us not in the space. Not the more serious stuff, that should stay serious, I mean stuff like this with little jokes but also very serious when it comes to the mental health bit and how involved Dr Dan was in making sure Pruitt was always being watched by his family and friends because it's overwhelmingly too common of an issue that people commit suicide because they've made some major mistakes throwing their careers in the trash like Pruitt did Edit: the very last clip of you saying he's your favorite Disney villian and putting your clip of saying there were formulas in the spread sheets is what im talking about when i say i love your humour. Love your stuff, cant wait to see your next video!
Smart people can explain complicated subjects, to people, who aren't as smart or as knowledgeable, in a way that the listener understands. Egotistical people explain things, using words and phasing, that makes people think, "I don't understand that, so the speaker must be really smart". Angela is the former, which is why she's such a great speaker. And I also agree that her taking a stand, against sexual harassment and assaults, was fantastic and very much needed. It's great to see people using their platform to do good, in addition to the hours of entertainment she's freely provided for us. She's done a lot of good, and I also really appreciate her work 😊
Another paper retracted a few weeks ago --- going on 5 years since this story broke and his career ended, papers are still being retracted! Crazy. Your comment about never looking at datasets in spreadsheet form was really interesting - I help manage a huge catalog of data products (like datasets companies buy), and it's wild to me that spot-checking just isn't a thing in academia. Maybe I'm stupid, but I find I always have a better "feel" for a dataset, or a better mental image of the raw data, just by looking at spreadsheets or tables in a relational dataset, picking one record/event, and tracing it across all the relationships.
This story reminds me of Stephen Glass, the former reporter for The New Republic who was (eventually) caught fabricating his stories. But before he got caught he was a rising star in the journalism world. He wrote dozens of pieces that were published in major national publications. No one bothered to fact-check his work until someone (another journalist, Adam Penenberg) finally did. An interesting side note is that, like Pruitt, Glass's next move after being exiled from journalism was to write a crappy novel.
I guess the moral of the story is to check your ambitions and manage your expectations bc you never know whats going to set you back. You tell better stories than the boys in science communication. Thank you for your time.
I really appreciate that last section about the compassion / sympathy. I am finishing a masters in engineering and most of my results are simulations. It would be very very easy to design the simulation to support my argument, and I feel that temptation constantly, and I'm sure that I've let this happen to some extent. I wish we had a system where this was less likely to happen and where it was okay to fail. Honestly I think it'd be such a cool and satisfying career to just be an expert results replicator, to learn and keep up with all the papers others are writing and fact check them, but it can never really work the way the money is allocated right now.
idk, is it strange I have zero sympathy for someone who harms others to give the illusion of excelling in their career? Bc that’s sociopath crap right there. The pressure is on everyone exactly the same. The difference between the people who are tempted and don’t fake data vs the people who are tempted and DO is very simply whether or not they’re a good person or are they willing to damage science and the careers of countless people to get over. It is exponentially worse that time and again he was made quite aware of how much damage it was doing to individuals and the field! And not only did he not have the spine to take ownership, not only did he use aggressive, litigious threats to silence people trying to survive the damage he’d done, he liked that post which is gross bully egotism to the extreme. I love that she is a kind person in that way, to take the time to bring compassion to the conversation, but it’s impossible for me to have real compassion for predators. Everyone’s out here just trying to survive, I don’t have feelings for people who are sad bc someone forced them to stop taking advantage of/harming others to get ahead of them. Someone else deserved that job he had all that time, ya know? It’s like someone asking me to feel bad for Alex Jones if he someday actually faces fair consequences and is super sad about his legacy.
I think that Pruitt declined to offer an apology for his actions on the advice of counsel that doing so would facilitate criminal prosecution. After all he defrauded one university of a PhD degree and another university (or the same one?) of a faculty position. Sounds like it could be criminal fraud to me. Where are the prosecutors?
I'm not an academic anymore, however I am a data analyst and a developer, and so many aspects of how he faked the data is crazy. I honestly don't know if it's better or worse that he was so bad at faking it, because if you know anything about R, Python, data analysis etc. you can think of a bunch of different ways to fake data if you wanted to that are way less detectable than literally copying and pasting down a spreadsheet. It's fine to store data in spreadsheets as it's low budget and low technical skill, however data collectors need to take precautions to ensure that data can't be fixed at the source, or edited at any point down the analysis pipeline. Horrifying stuff for sure.
Imagine how many people have published papers with data that wasn't just copy pasted. They knew how to write some simple code to optimise random number seeding that provides support to their fraudulent data that absolutely no-one can pick up on and now everyone treats as valid and useful. Probably the same people that are sensible enough to not have a tab in a spreadsheet saying "I dun faked it dint I lol".
I'm pretty sure this is a systemic thing as long as reproductions of studies are so rare. In a field with similar pressure to dwliver meaningful results, German journalist Claas Relotius got caught having made up large parts of his long form articles. He got lots of awards and was kind of the rising star journalist and turns out it's a lot easier to find a compelling narrative if you go to a foreign country and just make up everything you see and encounter there. I think there was a similar somewhat lower profile case at the Intercept about 5 years ago, where a journalist made up quotes and people they interviewed. I think this is really common everywhere bc there is no mechanism being enforced to stop it.
very excited to watch this video as spiders and exposing fraudulent data are both so interesting to me! but I have to point out that at 0:31 when you say "20% of spiders are colonial," i think you must have meant 0.20%? There's about 50,000 spider species and only about 120 of them are known to be social! I know my arachnophobic friends would freak out if it was actually 20%, so I thought I would comment about it in case people were worried going forward lol. anyway on to the rest of the video! :)
Science cannot work when built on trust. It's built on obligatory presentation of data, peer review, and experimental confirmation. That said, many of all published works and suppossed findings are not confirmed yet, though already treated and accepted as scientific findings.
This was excellent! Riveting. Absolutely fascinating subject and a well-told tale. Your concluding chapter caused me to hear Chris Rock’s voice saying over and over “…but I understand!” Thank you for such an enlightening and engaging video.
Students deserve a professor like you. Please find a place in the academic world. You are thoughtful, thorough, brilliant and brave. I would take every class that you teach. Learners need you. You are my hero ( heroine), and I have never had one before. Stay well.
Im still over here wondering why there arent grant bounties for replication studies based on the citation count of the original paper. Obviously everyone has noticed the lack of replication being a problem, and this seems like a reasonable way to do it without having to manually issue a grant for every possible replication.
As a friend of someone involved in this - this was quite an insane episode. While this involves human deception, its obvious from the event that system wide you're mostly using the honor system (and excel) to assert the data you use came from a real place that was accurately represented as the procedure described. And and it only gets worse from there.
On one hand it is scary. But on the other it is wholesome how all (or most) of the other scientists acted immediately with integrity. The fact the editor who even had a personal relationship realized he has to get independent analysis also gives me hope. I am just surprised that blatantly faking data like that and benefiting from it is not illegal and the guy didnt have to face criminal charges.
I just read the Wiki article about Pruitt (which said that as of 2022, he was a high school science teacher in Florida and that makes sense for anti-science Florida). That article linked to the long sad list of “Incidents of Science Misconduct.” Scary stuff, but I do like your reporting.
hes my teacher, he works at tampa catholic high school. Hes actually a very good teacher despite this scandal. I dont believe he did this out of malice, more likely necessity with his personality.
That's not about science per se, that's about academia. Him faking data doesn't diminish his knowledge in the field of biology. As a former scientist I feel very bad for him but of course he shouldn't done this. He sabotaged his own career and, more important, he sabotaged careers of several grad students.
I stumbled across your channel the other day and I'm obsessed. Your storytelling skills are awesome and I appreciate your perspective on the topics you discuss. This story was absolutely bonkers.
I am a former experimental physicist that struggled in a lab for years trying to get just one data point. Finally I gave up and switched to theory lol. It would certainly have been easier if I just invented data. It is shocking how inept he was at the fraud. It makes me worried about fraud from someone with a little more competence.
The part about not re-doing experiments is really interesting, i hadn't thought enough about the funding side and how a lot of older data could be relied on to be correct in these circumstances!
He was extremely careless in how he fabricated the data, which is why he got caught. Imagine how much data faking is going on by those who are just a little bit more diligent, diligent enough to use a random number generator and slightly more careful to not leave evidence of data fabrication--formulas--in the spreadsheets.
But why do it at all? There are thousands of better paid jobs other than science. The single thing that keeps one being idealistic about science is to find out some tiny true fact about Nature after years of hard work and frustration. Taking that away by faking the data takes away the only meaning of working as a scientist.
This was such a well made video. With each chapter I had brewing questions that were immediately brought out in the next chapter. A sad story and disappointing but as a late comer into academia and scientific pursuits and the perspective of an outsider, I can see how this happened. I have big questions about Mr. Pruitt’s PhD and the supervision and mentorship he received. I hope/wish we shift towards validating science that says “I wondered x and did the science, my results showed no significant relationship” or even better “I was wrong, here’s my data”
As one of the people involved in this story, I want to complement you on how accurately and clearly you tell this story,
Thank you so much for watching! I really enjoyed your excellent write up of the situation.
This is a fascinating tale of just how reliant science is on the integrity of researchers. I review pharmaceutical data for accuracy, but there is literally no way for me to check whether the readings recorded by an experiment's author are not just made up. Some experiments require a witness sign off (mostly for controlled substances experiments), but that just decreases the chances of fabrication. There's no way to eliminate the risk of fraud entirely without making the process so cumbersome that it would no longer be feasible to do the work.
Data that is used to determine the efficacy of drugs that we *put inside out bodies* could be fraudulent and we might not find out about it until years after the fact, if we ever do at all. This seems like a system that should quickly collapse due to corruption, but somehow it works well enough for new drugs to be discovered, distant black holes to be photographed, and perhaps, one day, for the personalities of spiders to be cataloged.
I am just shocked that repeated data used and copy&pasted wasn't simple more random. Like ok i studied computational simulation, so maybe i am biased... But it would he fairly easy to simulate data and make fake results look much more convincing... Obviously this wasn't needed because everyone just trusted the data.
@@orathaic fr it literally would have been trivial for him to randomly generate data that would be basically impossible to distinguish as fake
@@alansujansky8591 thing is with random data you couldn't produce those beautiful plots as easy, unless you work more on how to control that randomness and get the plots at the same time, which maybe Jon was lazy about doing
This channel is my new hyperfixation. TH-cam boosted "string theory lied to us" and now I'm binging everything.
SAME!!
Same here.
Yup
Same.
Same
The irony of a spider scientist getting caught in a tangled web of deceit.
sounds what chat gpt would say
I mean, everything is an interconnected web of things, so it checks out
LOL
“You would never open a spreadsheet.”
Tell me you’re a physicist without telling me you’re a physicist.
common @acollierastro mathematical L
Yeah, I don't get this sentiment at all
I will dig through heaven and earth to find the most simplistic detail ever in any spreadsheet I am given lol
Is there any chaotic phenomenon occuring in your computer? So what's the problem?
How big are your datasets? Though tbh its part of why you should do some data exploration and descriptive statistics before you get to any actual statistical work, and then publish all data and code. @SonicluNerdGamer
If they are small enough for excel they aren't big @@Isaiah_McIntosh
Isaac Asimov’s first non-science fiction novel, _A Whiff of Death_ was a mystery about someone at a university being murdered for faking his data. It wasn’t very successful, and a lot of the non science reviewers criticized it for the motive, ‘cause that’s no reason to kill someone, and all the science reviewers were like “the little twerp got what was coming to him!”
[Edited to correct a typo.]
Isn’t that a science fiction novel?
@@SHA-3qua No, it’s a straight mystery novel.
If this is a pun then I applaud you
I have copy of that Asimov novel :-)
@@donsample1002 you sure?
the most insane thing about this is that if this guy would have been like 10% smarter he would have gotten away with it for his entire career, and there are absolutely without a doubt probably people out there getting away with it right now
Totally agree. I've experienced this on a smaller scale. In my work we commission sewer surveys and sometimes there are some missing measurements, when we ask the surveyors for the missing data occasionally they will come back with a number that is so far from the expected value that we instantly know it has just been made up. If they had taken the time to look at there own survey photos they could make up a more realistic measurement and we wouldn't even question it, and I'm sure that does happen.
I'd revoke their title because someone with PhD should simply be smarter... One of 1st things they thought us in statistics and experimental physics classes was how easy it's to spot botched data, and how it is virtually impossible to get caught faking if you know what you are doing.
If you are publishing experimental results you should understand the statistics to be able to draw proper conclusions... OTOH... It's easy to catch cheaters lol
Hey man love your videos. Yeah imagine if he had just not left the data faking formulas in the public spreadsheets. If he had fuzzed the numbers a bit so it wasn’t obvious there were repeating exact sequences.
You are right.
There are others out there for sure .... it is scary.
@@masonwheeler6536 his name is probablyjacob after all
I feel the most for the spiders, who may never have their personalities or lack of personalities revealed to the arachnologist community
As a very evolved and bold spider I broke out in a sweat multiple times as I watched the video - I believed they uncovered us... cant believe I can sweat also but according to the paper about Sweaty Spiders there is data to back up the very existance of my subspecies.
That may, or may not, be really stressful for them!
I guess a follow up with them is in order, at least the opportunity to comment, or if they can't be found, going to Africa to do a follow up with their fellow members.
Somebody needs to repeat his experiments. It would be ironic if they really do have personalities.
@@haldorasgirson9463 it really would, the thing is, he probably started fine didn't find anything and made it up for the perks. Redesigning experiments and make a few tests, maybe, redoing all his work to get valid data doesn't make much sense.
Not to consider the meme of the subject, if somebody does try to explore it..
I can't express how much I love the lines, "This was my 'Tiger King.' This was my sourdough."
academia most certainly rewards lying. like you said, confirmatory studies never get funded, but even worse than that, i've been told by a Phys Rev D editor that they don't accept pure rebuttals - you have to write a whole new paper with new results and conclusions. and Pruitt's upward career trajectory proves the worst part of it - that had he simply published a few dozen papers showing weak results or no correlations, he'd have gone nowhere, but showing novel correlations over and over and over was the key to his successful career. it's the reason why juvenile forms of known dinosaur species are given new species names, it's why nothing can be reproduced in behavioral psychology, it's the reason why Berkeley had to retract its discovery of element 118. being "good at science" is *not* how you become successful in the field. being "right all the time" is, even if you have to lie about it.
i wonder sometimes if the need to be right is responsible for the proliferation of so many competing and orthogonal statistical tests for novelty. in other words, not everyone is willing to fake their data, but they are willing to use a weak statistical test to prove their hypothesis.
It also rewards publishing many papers even if they don't really add much or anything at all for scientific progress instead of making fewer but more meaningful experiments.
So we have lying + thinning out the meaning of geniue studies
The Russian language has a concept that we are lacking in English - Vranyo:
I tell you a ridiculous lie. You know I'm lying. I know you know. I tell it with a straight face. You nod seriously and take notes.
You repeat my lie with your own embellishment. The person you tell it to ... nods seriously and takes notes.
As long as everybody lies, and everybody accepts each others lies ... all the liars get ahead. The troublemaking maverick who tells the truth gets left out.
When all the lies get compiled and regurgitated by an official source, it becomes Gosvranyo.
It's not that we don't do this ... our language just doesn't express it as concisely..
@@tomlxyz None of what was described above supports a claim of "thinning" Thin-NING implies an INCREASING TREND of this over time, which you've given no reason to believe is true. All these same motivations existed in the 1700s as well, or whatever, and would in fact have been even easier to get away with than now. So I see no reason to suspect some sort of snowballing catastrophe, as opposed to just a constant, static level of baseline corruption and inefficiency that we've been dealing with forever, but isn't enough to actually stop progress. Just makes it slower and more expensive than maybe it could be if we come up with better systems someday.
@@gavinjenkins899 I think you just don't know a lot of the crappy texts published today as scientific articles. It is certainly an INCREASING problem, nowadays even speeding up due to predatory open access journals, where one essentially just pays some money to get published with 'peer review' that does not deserve its name. It is also common nowadays to write papers about known stuff and pretend that they are about new things. And it all gets published without problems.
@@gavinjenkins899I think you misinterpreted this person. It's clear what they mean by thinning is not a change over time but rather that the papers are being thinned as a result of the incentives and are hence less meaningful. People are willing to write lots of nonsense rather than big publications.
No time dynamics involved there just a statement of now
"Why would they use a spreadsheet?" Oh, I envy your entire field. Basically all of biology is done in Excel. They've had to rename genes in contravention of the Principle of Priority, even, because Excel interpreted the original names as dates, and errors were getting into the published literature!
Wow
Me watching this video while my computer struggles to keep open an excessive amount of excel sheets recording my lifespan data from my lab: 👁️👄👁️
Oh yes. Want a new nightmare? Browse a biology journal and grab a bunch of supplemental data volumes.
There were literally multiple times during this I said "no...." and felt the hairs on my neck go up. "There were literally equations in the spreadsheets!!!"
A story about spiders and something else is more scary...
How was Pruitt so prolific? Well, when you make up the data and don't actually do any experimentation or observation, it's AMAZING how quick it is to write a paper.
And we thought there was peer reviewing and reproducability. Most published research is wrong.
I cant believe physics Jenny Nicholson even had a spider arc. The comparison just gets more and more apt with every video on this channel I watch
A...and I love that, to be clear
The need for a collab is skyrocketing
I had no idea what you were talking about so googled "physics Jenny Nicholson" and somehow found the youtube channel and watched the spider story which was relatively brief for her - but delightful! I'm almost hoping for another Covid so I'll have time to look at all her stuff as I'm quite the one for falling down the rabbithole as well.
THAT’S WHY HER VOICE SOUNDS SO FAMILIAR!
Iconic.
Jenny’s spider reviews have more data in them than Pruitt’s
@@seasidescott is this the start of your villain arc? 😂
This story is kinda extraordinary. Whilst you were explaining the surprisingly amateurish way he faked his data I was thinking "why would you do it that way, why wouldn't you write a simply Python script to create random data with the distribution you are trying to demonstrate?" (whilst feeling slightly guilty at the fact I was thinking of ways to fake this better than Pruitt did).
But then something scarier occurred to me: perhaps there are scientists who did fake their data better than Pruitt did. Obviously some people are willing to fake data in order to further their careers, and I can see a way to do so that would be harder to detect than his was*, so how do we know someone else hasn't done exactly this? What other area of research might be built on fake data and will we ever know? How far will the tree of subsequent research grow before we do? That's scary.
My sister worked in a Psychology lab years ago where the PI was selling educational software for kids, and whilst they weren't faking data (or at least as far as she was involved) I remember her being extremely uncomfortable with the manipulation the team's statistician was doing to make results look significant. I worry that this sort of thing is far more common than we imagine.
Anyway, really interesting story. As you say, it's worth being compassionate towards Pruitt. Unless he does a tell-all, I suspect we'll never know the exact details of his fraud, but if it went all the way back to his PhD it's easy to imagine a panicking PhD student whose experiments aren't going anywhere making up data to avoid failing**, and when they got away with it doing it again, and again, and again. I can also imagine the panic he must have felt when he saw the whole thing unravelling. Obviously he shouldn't have done it, but I do have some sympathy for him**.
*I suspect that it would still be detectable by someone clever enough really looking for it, but it wouldn't be "Excel formulae in the cells of the raw data obvious" and it might be harder to convincingly prove.
**As someone who made themself really sick with stress writing a thoroughly mediocre maths PhD, I do understand just how difficult being a PhD student can be.
Oh, absolutely---same---I was thinking the same thing. So much better ways to fake it if that is what you want to do.
So, the conclusion of the video is important. We must be willing to fund duplication efforts. And to pay attention if research is NOT repeatable in a different lab.
What you describe is what someone who sets out to be fraudulent would do.
I'm sure with each study Pruitt did, he had "good intentions", but the pressure to publish something positive and significant pushed him to fudge the data.
I mean, he had actual post-docs doing measurements. Many of the numbers in the spreadsheet were likely real measurements. But he needed to make adjustments until he got something worth publishing.
I didn't take from the story that EVERY NUMBER in the spreadsheet was completely fabricated - just that he was making adjustments until he got what he wanted to publish.
That might be a distinction without a difference, but it reflects the pressure these people are under to "publish or perish".
In the end, he might not have been caught if he totally fabricated the data.
I was shouting exactly the same thing at the screen!
I can only imagine he didn't really have a good understanding of data science and didn't realize how flawed his method for faking data was.
I am terrified at how much fake data there could be. My Post-Doc involved a lot of complex data manipulation (it was a differences-of-differences kind of study, so the level of manipulation is necessary and accepted in the field). And my results, as a post-doc, were mediocre.
It occurred to me, when I was a post-doc, that it would be trivial for me to fake a result. The differences we were looking for were
@@alexanderbrady5486 Here's something even scarier: you published a mediocre paper and are out of academia. Pruitt faked his data and was an academic rock star. Given that only a tiny fraction of PhDs end up in academia, how many of these "success stories" do you think gained their success without fudging their data, or engaging in other unethical practices?
The overpopulated academic bubble economy, combined with a metrics-based faux meritocracy, is a duo from hell suffocating science.
God this is an actual real-life nightmare.
I already have nightmares about discovering an honest mistake in mine or my colleagues' data years after the fact. The level of betrayal to discover a colleague or an advisor knowingly falsifying data is just unthinkable.
What one expects from a scary story:
Normal people: The super aggressive spiders escaped the lab and are now on the loose. Panic time!
Super Nerd Scientist: THE DATA WAS FAKE!!!!!!!!!!!
Seriously though, your a great storyteller.
That’s really nice! Thanks for watching
Yeah after the intro I was expecting a story about an invasive species because of improper lab protocols. I think the fake data is less bad than that, only because it's possible to fix, whereas invasive species can be impossible to eradicate.
But actually, it was the spider colony testing how the scientists react to strange data. After all, there are experts string theorists.
when she opened it with "non native spiders gathered from Africa and brought in for study" i totally thought that was the angle it was going haha
What no one realized was that the spiders were actually far more intelligent than anyone knew. The data, if it hadn't been faked, might have revealed this and perhaps the events that followed would have played out very differently. Could the Era of Humans have continued if we'd been aware of this threat earlier? Would we have failed this test anyway and ceded our primacy to the ultra-organized culture of sentient arachnids no matter what? Would Pruitt have become a spider collaborator either way? We'll never know, and soon, there will be no one left to ask.
Maybe I'm heartless, but I have no sympathy for Pruitt. Every time he took an academic position, he was taking that position away from other applicants whose research was perhaps less startling, but almost certainly more honest. And he knew he was doing it. Every time.
You are 100% going to blow up huge soon. You’re a great storyteller. INSANE story about something I’ve literally never even thought about!
That's nice, thank you!
@@acollierastrohere congratulate you on blowing up. (Positive)
Its happening! Nearly 50k subs. Excited to share these awesome well told stories with everyone. I appreciate that this physicist is taking the time to make free content so I feel compelled to help spread the word even in the smallest way.
And here we are, thanks algorithm ^^
Now nearly 60k subs. Time for another oscilloscope soon!
I think it's rather beautiful how Pruitt found his passion making up stories about animal personalities... as a sciencefiction author of course.
22:40 So they're *literally* doing peer review, and Pruitt is like "NOOO! Don't do that! That's not how science works!" It would be hilarious if it wasn't so maddening.
It’s astounding that the data fakery was so incompetent and so easy to detect. And it’s terrifying how easy it would be to fake the data in a way that would not have been detected. I’d heard about this scandal, but your telling makes it much more vivid. Great storytelling!
And your observations about academia are right on.
To fake the data in an easy to detect fakery way, is part of the thrill.
In the Netherlands we had the famous professor in Social Psychology Diederik Stapel. He started out embellishing the results from questionnaires, e.g. among school kids. Then he rented a cottage in the woods and would fill in the spreadsheets by hand, during the weekends. He would no longer bother to actually visit the schools for his questionnaires.
In the end he would even fake the schools. That is, he would "visit" schools that not existed. He made up a school on 500 m distance from his university lab, and nobody discovered that this nearby school did not exist.
@@ernstraedecker6174 I suspect that there must be a psychological explantion about this situations that make the perpetrators want to be caught.
@@ParameterGrenzesomething something "smallest tinge of guilt and responsibility" for making sure whatever damage was caused could still be "reasonably reversed" even if that means decades if work end up going down the drain immediately upon discovery -- which I guess is better that they can find out they are chasing imaginary birds sooner rather than later when too much has been built up and the entire house of cards fall down
I quit academia before publishing anything because my advisors were so awful they literally told me "no one cares about the data, we won't include it anywhere, just publish a good result so we'll get attention" when I asked how to go about including my dataset, this was in like the early 2010s
Bobbybroccoli has a great video about Henrik Schön, who might have also faked data because of academic pressure
This was great, thanks for the rec!!
Schön did make one mistake, his supposed research was readily commercializable and used equipment and techniques most people in the field would be at least familiar with. So, there were a lot of replication attempts almost immediately, and, while it took longer than it should for people to figure out that it wasn't them-problem, it was Schön-problem that there was no successful replications, he was doomed from the start.
Behavioural research on animals that aren't your standard lab models, and so require some special arrangements to get and probably special expertise to simply keep in a lab... Even if you wanted to replicate, even if you somehow got funded, guess whose lab would you ask for that expertise? And if you don't, they always would have an excuse that your lab did something wrong because you didn't ask them. It's an almost perfect niche for an academic fraudster. Which is scary.
I think he has 3 video series about scientists who faked data by now. The others are on Victor Ninov and Hwang Woo-suk
Schön did not have any academic pressure. He could have done honest science and get promoted as well. He decided not to do the hard work.
This is such an impressive story. Mad respect to Dr. Kate for her integrity. It would have been so much easier to things the wrong way, but she went through the excruciating process of correcting the problem. What a wonderful thing, to be pruning bad research out of the literature.
She was saving her skin, like spider in a box, no poetry.
as of 2023, pruitt actually wrote his own fiction novel "The Amber Menhir". good to see his information fabrication skills put to better use.
You hit the nail on the head in terms of the job season awfulness Pruitt's coauthors had to go through. I'm friends with a few of Jonathan's former collaborators and this has been devastating to their careers. One pre-tenure prof with half their research program called into question and the other a current graduate student who had to completely overhaul their dissertation in the middle of doing fieldwork. It absolutely sucks that for every fake data scandal there's a trail of honest people who've wasted years of their lives investigating a made-up phenomenon and had to question the validity of their entire subfield. (Also, yeah we use bioarxiv, but I don't think biologists post as many preprints as physicists, culturally speaking.)
I completely agree that the incentives are completely messed up and it's easy to see why Pruitt did what he did...That said, this is second-hand hearsay, but I've heard that Pruitt treated his students terribly behind closed doors. So I don't particularly feel any sympathy for him.
That's what we get for making a competition in productivity out of everything. Love your channel. You're a genious storryteller.
The reality is academia has a huge problem and these kinds of stories are not uncommon. I ended up not doing a PhD because I was accepted into a program and the Professor running the program was caught publishing books which plagiarized his grad students, all he had to do was make them co-authors but I guess he was too greedy. They suspended him which shutdown the program and given the timing if I wanted to do my PhD I'd have to wait a whole extra year and get accepted into a different program somewhere else... This is despicable and universities need checks so that this kind of dishonesty can't go on for years before it is caught (which was also the case in my situation). If more people were caught and held accountable the first time they tried this BS then far fewer would ever attempt it in the first place. This also dramatically undermines the public's confidence in the whole scientific establishment.
For the first time in human history, the scary thing was that there were no spiders involved.
OMG
I love academic drama! It's like those criminal story podcasts, except nobody's dead, however, knowing that spiders may not have personalities is heartbreaking on itself💔
Thank you very much for this video!
I'm only a third of the way through, but so far Dr Kate is a fucking hero with guts of steel.
I have had hard time explaining to people why learning Python or R is important when you can do a lot of stuff in Excel with seemingly less effort and this story encapsulates the faults of Excel so perfectly it's astonishing
When I hear stories like this or Schön, I wonder how many fake results are out there. How many people made up their data, but were smart enough to not make earth shattering claims and remembered to add noise, not recycle data and keeping the sample sizes realistic. I'd guess it's really really tough to prove data is faked when the person faking it understands what they are faking, is trying to make it hard to detect, is competent with computers and takes their time.
I think there is a category problem that limits this.
The kind of researcher you describe needs to possess three traits;
- Know, understand, and care about all the factors that are necessary for convincing fake data.
- Be willing to put in the work to create high quality fake data despite knowing that it likely won’t be checked.
- Be willing to fake data.
Problem is, if you’re lazy and slipshod enough to be willing to fake data, you’re probably not willing to put in the work to do it really well. After all, you could have just gone and done proper science with that same level of effort.
@piedpiper1172 the point hammered home in this video is that the pressures inherent in academia would promote even hardworking non lazy scientists faking data. The best metaphor I can think of is that Dream minecraft speedrunning scandal. He was still a top mine craft speedrunner but the run itself had inherent probabibilities making good runs very unlikely. The same is true with science because you could be working on something that goes nowhere and limits your work and research oportunities despite doing the science perfectly on what, only in hindsight, became an inconsequential hypothesis.
I'm glad you got to the incentives. I wonder how many of the academics "at the top of their field" are there because faking data is the only way to keep up with the pace and compete, especially when there's a decent chance everybody you're competing with at that level is also faking data.
This sounds analogous to the problem of doping in cycling. The only way to be competitive against everyone else who is doping is to also rely on doping oneself.
Pruitt was the one who got caught because he was so bad at hiding his traces. If he just added a small normal noise to all his faked data it’s likely no one would question the data in the first place.
What I experienced was that a couple of the supervisors I worked with were very busy men, with minimal involvement in their research. The work was almost completely done by grad students and post docs. The supervisor's seldom participated, or even checked the work of their students, but on the other hand they were constantly pushing for publishable results. In my case, I was not only threatened, I was kicked out of a Ph.D. program for failing to come up with publishable results.
So as a grad student there is a very strong incentive to come up with something that can be published, and a very low probability that if what you come up with is largely fabricated that your supervisor will either be familiar enough with your work, check your work carefully enough, or possibly even care enough to notice. This is significant because a person who cheats and gets away with it as a grad student is very likely to continue in the same pattern as they continue their career.
Based on my experience what is really needed are graduate student supervisors who are intimately enough involved in their student's research, and sufficiently motivated, to catch the fake results before they get published. It needs to be difficult to write a research paper that is good enough to pass your supervisor's scrutiny. In Pruitt's case there is an argument to be made that his Ph.D. supervisor should bear some responsibility for failing to properly supervise his research.
I might even go so far as to comment that research work is intense, and highly competitive. There is a personality type that thrives under such conditions, and people that have that type of personality are the ones that end up as university faculty. You don't have to spend too much time with those people to see the impact that has on their interpersonal relationships.
That’s really upsetting to hear
I’m not an academic, I got my masters degree in electrical engineering and got a job. I’m shocked that something like this can happen. I always assumed that because science is all about reproducibility, that people would be attempting to recreate results from published experiments. I guess that’s why people assume there is no way someone could fake data. They kind of go “how can you fake data? Someone could try to reproduce your data and expose you,” but nobody ever did
You want it to get scarier? read "Reproducibility of Scientific Results" from Stanford edu, Stanford Encyclopedia of Philosophy (for Philosophy of Science people, I presume). This could happen a lot in the current way of things, and that was 2018 - it is probably worse now.
@@masonwheeler6536Just a little while ago, a professor at U Minnesota intentionally tried to commit bad code to the linux kernel. many of his CS students were involved.
@@masonwheeler6536 I think it's somewhat wrong to compare this case with Heartbleed. Don't get me wrong, it's the same "type" of problem, but the degree is so much different that it turns into something else.
Heartbleed sorta existed for two years give or take, in some systems, not all. It was detected and could be fixed. It was a vulnerability but it could only affect systems with said vulnerability.
The Academia issue is that the vast majority of data in the past decades wasn't validated. Even some big shot papers don't get any meaningful analysis. It's simply too much work for few hands and the system does not encourage that menial labor. Overviewing open code is relatively cheap, quick and actually develops your skills. Replicating studies takes long, costs as much as the initial study itself, or even more, gets your career nowhere and doesn't make headlines unless it's a really trendy topic.
And unlike code where even if replicated, each instance is unique, citations bring the original paper along with them.
And to make matters worse, Heartbleed was a massive vulnerability, but wasn't a dumb mistake. It's crazy it wasn't noticed for two years, but to exploit it you needed the right conditions, expertise and some luck. What Pruitt did was the equivalent of you trying a basic SQL injection and drop all tables on the entirety of Alphabet services including backups. He did the lowest possible form of faking data, and it took a decade for someone to catch on. And if he wasn't stupid enough to leave so much evidence, it would have been written off as a data error.
And (and again) in heartbleed, each affected instance is a singular issue. In Academia, each fake paper can lead to dozens or hundreds, maybe thousands of retractions, corrections ,and so on. Imagine that, across two or three decades, across several fields.
People are too naive. Everyone should know that the outside appearance of ANYTHING is not equal to the inside. We live in a capitalist society which is all about lying and promoting lies (usually it's called advertisement, public relations, outreach, election campaign, product information, to name just a few). Telling the truth gets you into trouble, telling lies brings you in the highest positions.
i love learning about drama in random fields im unfamiliar with. that chess cheating drama, the fishing contest cheating drama, now spider science drama
I recently found your channel and I gotta say while your storytelling is compelling, your comment is what makes it most worth it.
Thanks for watching!
With the whole "scary" buildup and singular villain as well as your hip, matter-of-fact delivery gave this a Scooby-Doo vibe. Like you're Velma and dude's gonna pop up and say he'd of gotten away with it too except for you meddling kids. 😂
The really scary part about this is that this was only discovered because of both how open data sharing is in this field, and because of how wildly incompetent his data faking was. If he had simply written a Python script to generate random points from a desired distribution, nobody would have ever identified it. If his spreadsheets were kept private, no one would have any evidence. Other scientists are faking data right now, in every single field of study, and nobody knows who. It feels like the only way to catch this is with reproduction studies but as you say there's very little incentive to do those on small results like this.
I like to think that while all this happened, the spiders continued having a happy life.
The FIRST thing I was taught to do in grad school (and I was in grad school multiple decades ago) before I did any analysis in SPSS was to give my data a look with the human eye to see if there's anything abnormal, especially if I wasn't the one who recorded it. I was taught to sort it by different fields, look for missing values that may affect analysis, make sure it's all the correct unit, make sure it's all SI or English, check if a fellow researcher accidentally mislabeled something, etc... look at it before you have your stats software do anything to it, even if it's a million rows ... at least play with it a tiny bit to see if there's anything strange, do a couple countifs, vlookups, etc. just to make sure things are ready to analyze.
I was taught to do it doing research as an undergrad. Look at the raw data, try graphing data different ways to see if any anomalies pop up, do statistical analysis on the results, be suspicious if the statistical analysis comes back _too_ good...
All of which is to say, the repeated data immediately caught my eye as something which could have been easily caught by plotting it. There would have been suspicious quantization artifacts.
The topic of deliberate data falsification was never brought up though, nor the idea of doubting your co-authors. The concern was wholly focused on catching equipment failures, math mistakes, and so on.
Something else that this story reminded me of: Taking notes on carbon copy paper. It's a good practice, but there's no point if the originals and copies aren't stored separately by different people.
According to Wikipedia Jonathan Pruitt is, as of 2022, currently a Florida high school science teacher.
oh no
@@AlabasterClay to be honest, I don't think that's too bad. Like despite faking his data (meaning he shouldn't be able to do academic research), I'm sure he has a perfectly fine grasp of high school level science concepts.
@@amentrison2794 Did you test his knowledge of high school level science concepts? Did you look at the data? Is history going to repeat itself?
@@justteathankyou.3642 Are you making a joke rn or are you trying to raise a serious concern? /gen
Well, his bachelor's degree is probably still legit.
You are a really engaging storyteller. I hope your content gains traction!
I appreciate that!
Dr. Kate Laskowski's action was heroic. I'm so pleased her career has gone on and she has a thriving lab at UC Davis. I'm glad Dr Dan Bolnick's career is going well and especially that the spider research is being restarted. I mean, spiders having personalities would be so cool if it turns out they do. If Pruitt wants to really repent, he needs to set out in detail what he did and what pressures he was succumbing to that made him do it. That would be really hard for a person to do, but it is what is needed for him to regain any honour or integrity.
Your closing thoughts really took this already engrossing tale to another level. Great video!
2:44 - That is a dead or dying Agelenopsis, not a spider in a defensive or 'timid' position. If you see a spider in this position, try giving it some water.
This is criminally underviewed. I routinely watch mini-documentaries and essays about things like financial crises and machine learning and the rise of AI, and a physics PhD posts the most engaging TH-cam video (and about a publishing scandal in biology, no less) I've watched in a couple of years.
Bravo
And here's hoping the changes to the institutional landscape that allow for a reform of academia and academic publishing come sooner than later.
I find the other scientists involved in this story amazing - they are what we should expect scientists to be.
The story is a good lesson to use for reforming academic publishing.
Angela, this is still by far my favorite video in your channel. I keep rewatching it. It’s so entertaining!
This is a mind blowing story, you tell it incredibly well and most of all I’m blown away by your empathy. I’m outside of academia but this story, how it’s told just humanizes it all so well. This is a seriously good video, thank you.
I feel like I've been hearing about this worrisome trend in academia for decades. A baseline expectation of trust just doesn't work when all the incentives align with intense pressure to take short cuts. Like even expressing interest in redoing an experiment and retesting data is thoroughly discouraged. If it's not novel, it's not worth anyone's time. That's such a reckless platform on which to build a foundation of future scientific discovery.
i feel like the scariest part of this story is how bad jonathan pruitt seems to have been at faking data. like, copy-pasting the same numbers and leaving calculations in your sheet feel like pretty basic errors, and yet he still got away with it for a decade. it's upsetting to think about how many people might be even mildly better at faking data, or even just mildly luckier.
Would you be able to keep up faking data over and over? He had to hard code thousands of "measurements" over dozens of articles. I don't think you appreciate just how tideous a process that is. Have you ever had to fill out entire collumns of data by hand? It's insufferable. Thank god i can put and equal sign and be done with it. As for cheats and frauds, good riddance.
@@sakesaurus
1) you know who else has to hard code thousands of measurements over dozens of articles? actual biologists doing actual biology. and they have to do that on top of actually doing the experiment and taking the measurements
2) i know how to copy -> paste special -> values only
Im a biologist. I got my PhD in 2013. I don't do research anymore, but I can tell you that I, and.most everyone else i knew when i was in grad school used spreadsheets for most things. I sent and received data in Excel files all the time and i mostly trusted that no one would fuck with it. Besides, i had a bajillion backups so i could always figure it out of anyone did something fishy with my data. Also, most of us aren't dealing with huge data sets and there were (and i imagine still are) plenty of programs with nice cuddly UIs that require no programming skills whatsoever to learn. In short, you could quite easily get by without knowing how to use much more than Excel and SPSS (e.g.), and many people , including the faculty, did just that. Students were more likely to know some coding or how ro use R, but if your advisor doesn't value it and doesn't think they need it, they're not going to push you to or even allow yiu thr time to learn it either.
That said, if I were faking data (which I wouldn't) , id use a random number generator to produce values within the range required to generate the results I wanted. I'd also copy the values to a new spreadsheet, save over the evidence with a blank file , and delete the file.
As an archivist -- I understand the appeal of an Excel spreadsheet, but oh god that's a nightmare for data integrity and preservation. Thanks for giving me something new to keep me up at night. 😂
@@captainnemo7690 We were always taught, and it made sense to me, to backup data in raw format always so in the worst case you can go back and edit it from scratch again.
It’s because biologists aren’t scientists 😂. Sincerely a physicist.
But seriously how is it ok to use something without an audit trail, backups, and non-automated (rife for human error), as the default tool you use. It’s scary.
(Just joking obviously, but tools matter.)
@@astronemir I invite you to do a seminar series teaching old crusty biologists how to do all of that. Just give me a heads up before you start so I can get my popcorn
@@astronemir Yeah, you wouldn't want this sort of chicanery with airframe stress calculations!
15:45 “Retraction. Not a pretty way to invite in 2020.”
Buddy, if you only knew
These videos are amazingly well done. As an engineer, who isn't a scientist, it's extraordinary to see the insight into how academia works as a social structure and capitalist endeavor. Angela, you're a master storyteller.
So well done, including not only identifying the systemic issues and demonstrating human sympathy, but also the cap prop and the selected music.
If you liked the Pruitt story you will love the story about single molecule transistors invented by Jan Hendrik Schön.
How we say in German "Das war zu schön um wahr zu sein" (That was to good to be true). And it probably starts with the small things. When doing experiments in school, who is ever going with a more than 2sigma from the accepted result? Should happen often, but I guess really happens. Just remove a 'faulty' data point.
Bobby broccoli has Ana amazing series on this if anyway is curios
If the spiders can hunt exponentially bigger prey, then we should make a log graph of the mass of their prey.
then we can predict when the spiders will have evolved to hunt a human toddler.
It's important to be able to predict threshold when the human species loses its apex predator status on this planet to the arachnids.
@@euming It's also a wonderful premise for a film ... working title 'Arachnid Park'.
Jabberwock I think it should be called Billy and the Arachnosaurus.
Dr Collier because I also watched your video about AI, I decided to ask ChatGPT if spiders have personalities. Chat GPT said yes and recently researchers had detected evidence of personality. I asked who did this research and you will not be surprised that Pruitt was the researcher chat GPT relied on.
What surprises me is that this doesn't happen more often.
It varies from lab to lab, but some academic labs involve so much pressure to produce publishable results. For anyone who is between their PhD and academic tenure (which has become immensely more difficult to achieve in the last half-century), there is constant pressure. Lab supervisors are often awful people-managers, and shit always flows downhill.
Love the storytelling approach and that you dug into the way the story unfolded. Such drama! Really entertaining :)
Favorite quote: "That's an interesting question." ummmmmmmmm.
In this situation, I always feel bad for the people who try to replicate the data and end up wasting epic amounts of time. Replicating work is hard enough as is, without it being FAKE! Also, it's interesting that it took someone looking at the specific data points and noticing the weird lack of variance as a red flag.
It is so wild how obvious the fake data was but unless you were specifically looking for fake data you wouldn't see it.
@@acollierastro - What also scares me is that, if he used a better method of faking (just randomizing the times slightly), it's likely that none of this would have been caught ever. Anyway, thanks for the fascinating account. I remember when the story made mainstream media headlines, but I did not follow it closely; and it's great to hear it analyzed in a way that captures my interest. Kudos!
There was something about youtube that started to annoy me and I couldnt put my finger on it. But whatever you're doing in these videos is the antidote for it. Thanks for storytelling. From one researcher to another I really appreciate the kind of stuff you're coming up with here
This kind of TH-cam content is my Tiger King. I don’t know how often these sort of shenanigans occur in research, but if there was a channel devoted to these kinds of stories, it would totally be my jam.
You might want to check out Bobbybroccoli scientific fraud is his jam. Schon, ninov, genetic cloning fraud, etc. All multiple hours long. Really good stuff. He also delves into some stories that are just incompetence/bad luck whatever like the Supercollider that was supposed to be built in Texas.
loving these video essays, your storytelling, the way you speak, and the editing are all spot on. Very pleasant to listen to. I definitely see this channel going far
My dad and I both watched this video separately and had a conversation about it. Happy to see you getting recommended more, I really love your style! :D
Angela is a dangerous woman, brilliant and brave. Doing the necessary work of exposing the dirty underbelly of academia no one else is willing to do. She should not have to sacrifice her dream career to warn her colleagues of the unseen dangers.
This work needs to be recognized and rewarded.
❤
Angela, you're a fantastic TH-camr and a great help in bridging the gap to us normal non academia people, hopefully I'm not the only young mid 20s non academic that sees how awesome your coverage is of stuff like this. You're a great speaker, made things very clear without dumbing it down, and your sense of humour is fantastic. I love that you're unapologetically you and you not only talk about things that happen and cover the event, but also made a stand on sexual assault and harassment, mental health, and the reality of taking adjunct positions or taking a 3 year post doc to uproot your life and spend a decade making nothing to become a prof at an ivy league or even just become a professor in your chosen field and specialty. Really opened my eyes to the reality of the space and the things you talk about and it means more coming from someone like you who is very honest but also makes things digestible and fun for those of us not in the space. Not the more serious stuff, that should stay serious, I mean stuff like this with little jokes but also very serious when it comes to the mental health bit and how involved Dr Dan was in making sure Pruitt was always being watched by his family and friends because it's overwhelmingly too common of an issue that people commit suicide because they've made some major mistakes throwing their careers in the trash like Pruitt did
Edit: the very last clip of you saying he's your favorite Disney villian and putting your clip of saying there were formulas in the spread sheets is what im talking about when i say i love your humour. Love your stuff, cant wait to see your next video!
Smart people can explain complicated subjects, to people, who aren't as smart or as knowledgeable, in a way that the listener understands.
Egotistical people explain things, using words and phasing, that makes people think, "I don't understand that, so the speaker must be really smart".
Angela is the former, which is why she's such a great speaker. And I also agree that her taking a stand, against sexual harassment and assaults, was fantastic and very much needed.
It's great to see people using their platform to do good, in addition to the hours of entertainment she's freely provided for us. She's done a lot of good, and I also really appreciate her work 😊
Unironically, this is STEM horror. Amazing video, well presented and to the point; thank you for taking the time to talk about this
Another paper retracted a few weeks ago --- going on 5 years since this story broke and his career ended, papers are still being retracted! Crazy.
Your comment about never looking at datasets in spreadsheet form was really interesting - I help manage a huge catalog of data products (like datasets companies buy), and it's wild to me that spot-checking just isn't a thing in academia. Maybe I'm stupid, but I find I always have a better "feel" for a dataset, or a better mental image of the raw data, just by looking at spreadsheets or tables in a relational dataset, picking one record/event, and tracing it across all the relationships.
This story reminds me of Stephen Glass, the former reporter for The New Republic who was (eventually) caught fabricating his stories. But before he got caught he was a rising star in the journalism world. He wrote dozens of pieces that were published in major national publications. No one bothered to fact-check his work until someone (another journalist, Adam Penenberg) finally did. An interesting side note is that, like Pruitt, Glass's next move after being exiled from journalism was to write a crappy novel.
I've watched this something like 4 or 5 times now and I'm not in academia at all. Your skills of story telling are incredible.
I hate how so many key parts of our society has horrible incentives including science.
I guess the moral of the story is to check your ambitions and manage your expectations bc you never know whats going to set you back. You tell better stories than the boys in science communication. Thank you for your time.
"This was my Tiger King. My sourdough bread!" 🤣 I'm gonna use that.
I really appreciate that last section about the compassion / sympathy. I am finishing a masters in engineering and most of my results are simulations. It would be very very easy to design the simulation to support my argument, and I feel that temptation constantly, and I'm sure that I've let this happen to some extent. I wish we had a system where this was less likely to happen and where it was okay to fail. Honestly I think it'd be such a cool and satisfying career to just be an expert results replicator, to learn and keep up with all the papers others are writing and fact check them, but it can never really work the way the money is allocated right now.
idk, is it strange I have zero sympathy for someone who harms others to give the illusion of excelling in their career? Bc that’s sociopath crap right there. The pressure is on everyone exactly the same. The difference between the people who are tempted and don’t fake data vs the people who are tempted and DO is very simply whether or not they’re a good person or are they willing to damage science and the careers of countless people to get over. It is exponentially worse that time and again he was made quite aware of how much damage it was doing to individuals and the field! And not only did he not have the spine to take ownership, not only did he use aggressive, litigious threats to silence people trying to survive the damage he’d done, he liked that post which is gross bully egotism to the extreme.
I love that she is a kind person in that way, to take the time to bring compassion to the conversation, but it’s impossible for me to have real compassion for predators. Everyone’s out here just trying to survive, I don’t have feelings for people who are sad bc someone forced them to stop taking advantage of/harming others to get ahead of them. Someone else deserved that job he had all that time, ya know? It’s like someone asking me to feel bad for Alex Jones if he someday actually faces fair consequences and is super sad about his legacy.
I think that Pruitt declined to offer an apology for his actions on the advice of counsel that doing so would facilitate criminal prosecution. After all he defrauded one university of a PhD degree and another university (or the same one?) of a faculty position. Sounds like it could be criminal fraud to me. Where are the prosecutors?
Not to mention all the grant money he received while running a lab.
This channel is amazing. My ADHD absolutely loves how you present, especially when you play a game while discussing a topic. Absolutely perfect lol
I'm not an academic anymore, however I am a data analyst and a developer, and so many aspects of how he faked the data is crazy. I honestly don't know if it's better or worse that he was so bad at faking it, because if you know anything about R, Python, data analysis etc. you can think of a bunch of different ways to fake data if you wanted to that are way less detectable than literally copying and pasting down a spreadsheet. It's fine to store data in spreadsheets as it's low budget and low technical skill, however data collectors need to take precautions to ensure that data can't be fixed at the source, or edited at any point down the analysis pipeline. Horrifying stuff for sure.
Such a good video that this is my third watchthrough. Even scarier on rewatches aaaah! 🕷
Thank you for the video! I just found out about this, and it’s a huge topic we’re currently discussing in my class and you were a big help!
Glad it was helpful!
Imagine how many people have published papers with data that wasn't just copy pasted. They knew how to write some simple code to optimise random number seeding that provides support to their fraudulent data that absolutely no-one can pick up on and now everyone treats as valid and useful. Probably the same people that are sensible enough to not have a tab in a spreadsheet saying "I dun faked it dint I lol".
I'm pretty sure this is a systemic thing as long as reproductions of studies are so rare.
In a field with similar pressure to dwliver meaningful results, German journalist Claas Relotius got caught having made up large parts of his long form articles.
He got lots of awards and was kind of the rising star journalist and turns out it's a lot easier to find a compelling narrative if you go to a foreign country and just make up everything you see and encounter there.
I think there was a similar somewhat lower profile case at the Intercept about 5 years ago, where a journalist made up quotes and people they interviewed.
I think this is really common everywhere bc there is no mechanism being enforced to stop it.
Perverse incentives+ no oversight/accountability
Perverse incentives+ no oversight/accountability
As an archivist who works with research data periodically, the recordkeeping on this is a nightmare to me. I intend to take it as a personal offense.
very excited to watch this video as spiders and exposing fraudulent data are both so interesting to me! but I have to point out that at 0:31 when you say "20% of spiders are colonial," i think you must have meant 0.20%? There's about 50,000 spider species and only about 120 of them are known to be social! I know my arachnophobic friends would freak out if it was actually 20%, so I thought I would comment about it in case people were worried going forward lol. anyway on to the rest of the video! :)
"science is built on trust" is the scariest statement in the video, given the economic system we live under.
Science cannot work when built on trust. It's built on obligatory presentation of data, peer review, and experimental confirmation. That said, many of all published works and suppossed findings are not confirmed yet, though already treated and accepted as scientific findings.
This was excellent! Riveting. Absolutely fascinating subject and a well-told tale. Your concluding chapter caused me to hear Chris Rock’s voice saying over and over “…but I understand!” Thank you for such an enlightening and engaging video.
"I'm not saying he shoulda killed her..... but I _understand."_
As someone who has never gone to university and has no idea about academia, I just love this science drama lol
Keep em coming!
Meanwhile science: the replication crisis
Students deserve a professor like you. Please find a place in the academic world. You are thoughtful, thorough, brilliant and brave. I would take every class that you teach. Learners need you.
You are my hero ( heroine), and I have never had one before.
Stay well.
Im still over here wondering why there arent grant bounties for replication studies based on the citation count of the original paper. Obviously everyone has noticed the lack of replication being a problem, and this seems like a reasonable way to do it without having to manually issue a grant for every possible replication.
As a friend of someone involved in this - this was quite an insane episode. While this involves human deception, its obvious from the event that system wide you're mostly using the honor system (and excel) to assert the data you use came from a real place that was accurately represented as the procedure described.
And and it only gets worse from there.
So you're saying that the bolder, attack-happy spiders have been completely unobserved the entire time? Training. Getting stronger. Biding their time.
On one hand it is scary. But on the other it is wholesome how all (or most) of the other scientists acted immediately with integrity. The fact the editor who even had a personal relationship realized he has to get independent analysis also gives me hope.
I am just surprised that blatantly faking data like that and benefiting from it is not illegal and the guy didnt have to face criminal charges.
I just read the Wiki article about Pruitt (which said that as of 2022, he was a high school science teacher in Florida and that makes sense for anti-science Florida). That article linked to the long sad list of “Incidents of Science Misconduct.” Scary stuff, but I do like your reporting.
hes my teacher, he works at tampa catholic high school. Hes actually a very good teacher despite this scandal. I dont believe he did this out of malice, more likely necessity with his personality.
That's not about science per se, that's about academia. Him faking data doesn't diminish his knowledge in the field of biology.
As a former scientist I feel very bad for him but of course he shouldn't done this. He sabotaged his own career and, more important, he sabotaged careers of several grad students.
I stumbled across your channel the other day and I'm obsessed. Your storytelling skills are awesome and I appreciate your perspective on the topics you discuss. This story was absolutely bonkers.
I am a former experimental physicist that struggled in a lab for years trying to get just one data point. Finally I gave up and switched to theory lol. It would certainly have been easier if I just invented data.
It is shocking how inept he was at the fraud. It makes me worried about fraud from someone with a little more competence.
Ultimate nightmare: Experimenter teams up with a Statistician to fake the data competently.
i just wanted a video to fall asleep to man why’d it have to be so interesting for 47 minutes
You are really fun! I love the video !
The part about not re-doing experiments is really interesting, i hadn't thought enough about the funding side and how a lot of older data could be relied on to be correct in these circumstances!
He was extremely careless in how he fabricated the data, which is why he got caught. Imagine how much data faking is going on by those who are just a little bit more diligent, diligent enough to use a random number generator and slightly more careful to not leave evidence of data fabrication--formulas--in the spreadsheets.
But why do it at all? There are thousands of better paid jobs other than science. The single thing that keeps one being idealistic about science is to find out some tiny true fact about Nature after years of hard work and frustration. Taking that away by faking the data takes away the only meaning of working as a scientist.
This was such a well made video. With each chapter I had brewing questions that were immediately brought out in the next chapter. A sad story and disappointing but as a late comer into academia and scientific pursuits and the perspective of an outsider, I can see how this happened. I have big questions about Mr. Pruitt’s PhD and the supervision and mentorship he received. I hope/wish we shift towards validating science that says “I wondered x and did the science, my results showed no significant relationship” or even better “I was wrong, here’s my data”