How We’re Fooled By Statistics
ฝัง
- เผยแพร่เมื่อ 22 พ.ย. 2013
- Is punishment or reward more effective as feedback? Do new medical treatments really work? What about streaks in sport? Without considering regression to the mean, we are prone to making significant errors.
Check out Audible.com: bit.ly/ZJ5Q6z
Filmed at Perimeter Institute: pitp.ca
Is punishment or reward more effective for helping people learn. A lot of people would say different incentives motivate different people, or in different circumstances, but in psychology there is a sizable body of evidence that in order to learn skills, positive feedback is more effective. This fining has been verified not just with humans, but also with other species.
It was strange then that after Daniel Kahneman discussed this research with Israeli fighter pilot instructors that he was met with resistance. They found the opposite was true: when they reprimanded a cadet for performing poorly, he invariably improved, but if they praised a cadet for an excellent performance, the next attempt was not as good. In order to solve this apparent contradiction we first need to understand regression to the mean.
Teacher study: bit.ly/1h8puVT
Rugby player study: bit.ly/1aNSrBI
One of your better videos
It is unfortunate that your next likely will not be as good.
let's hope skill plays a much larger role in my video creation than luck! ;)
***** I don't think you understood the video
and you were right ;)
+Veritasium exactly what i thought of when i saw the RtotheM concept. where does skill come in? - Carol Dweck's science seems to indicate one must praise the effort (though that too with a pinch of salt). Chess GMs become better by consistent application of certain principles - why not fighter pilots too? how does RTTM stack up in case of skilling?
how much can things be applied to luck? is it always a lack of skill?
+Ravi Srini I guess skill determines how high the average is. If you can achieve a high average quality in videos, as it is on this channel, your youtubing skills can't be particularly bad. XD
The book "the talent code" talks about the appropriate style of feedback according to skill level. A beginner needs more encouragement to improve, regardless of immediate outcomes, but relative expert is engaged and needs no encouragement, but they still require detailed feedback in order to correct their mistakes. Reversing those, I could see beginners giving up out of frustration, and experts rolling their eyes, feeling like their time is being wasted.
Also check out "Thinking Fast and Slow." It's an amazing book that discusses the way our mind operates and how we trick ourselves constantly with data-related events.
Mmmm, think it works better to pick apart every fail and flaw of beginners, they improve faster, work harder, and the ones you don’t want, (people who give up) leave due to frustration.
Everyone is frustrated, everyone is unhappy, but results improve.
@@notyetdeleted6319
This just seems silly. As Veritasium mentioned, most research indicates that positive feedback is more effective than negative feedback. Obviously if something is being done wrong it needs to be corrected, but leaning more towards praise than reprimands is widely believed to be the way to go. This idea that beginners will work harder when harshly criticised MIGHT have some merit in cases where they were already highly motivated to begin with, since at least then they are unlikely to quit immediately, but even then it seems suspect.
Besides, results are never the best when everyone is frustrated and unhappy. Unhappy people are unproductive people in the long term, misery isn't good and misery isnt attractive. If the place makes everyone feel bad then people will want to leave. Why do something/work somewhere that makes your life worse when you could do something more enjoyable, right. Those who have the option to leave likely will.
Makes a lot of sense.
@@notyetdeleted6319 That's not going to work. Beginners (in a sport, say) have options. They have no interest in working harder or improving faster if they're being nitpicked at every step. They'll pick a different sport, or a different coach.
Can we all take a moment to appreciate how good Derek is at drawing bell curves?
Just look at that steady hand 2:15
@@haulin SHESH!!!
Unfortunately after he draws a particularly good one the next few seem to be not quite as good...
@@ultimateman55 Pretty cool how this kinda applies to the topic of the video. A cool coincidence.
@@lucienz1846that’s the joke
“The negative feedback was a harsh reprimand and the positive feedback was a 👍🏻”
Seems uneven but ok
Exactly my thought. The school policy of emphasizing positive feedback is as effective as negative feedback. How about a balanced approach: you got it wrong your are told, "you got it wrong". You get it it right, "you are told you get it right"
.
As for 'encouragement': the more you practice, the luckier you get.
@@markplain2555 As a Maths teacher (not in the US), I feel devastated every time I hear of people simply being told "you got it wrong". To me, that's a terrible teacher. Whenever I use that red cross of doom, there is always a comment next to it which explains what the issue is. A cross by itself is ridiculous and never helps anyone learn. But then, maybe I'm the exception... I don't know if many other teachers do the same. All I know is that many of my students tell me that I give the best feedback of any teacher they've ever had.
Negative feedback sounds like an insult. Getting something wrong should still have positive feedback with it.
@@TheBaggyT I feel like you're kinda "detrementing" them for the future of their education (no offence). Imagine that same class going on to have another teacher who puts a cross instead of any positive feedback on wrong answers. Anyways, have your students performed better in other tests because I find this topic very interesting? lol
@@deletioninducedin7days919 Not sure what you're getting at, but yes, my students regularly perform better than other students in their year. The only exception is when I'm teaching a very low ability class, but even then, they often out-perform the expectations of the head of maths.
@@TheBaggyT Tests/exams are a learning experience too. Obviously you should already know the material for best results but that's not always realistic. People learn from their mistakes but not being told what went wrong hampers that learning.
2:13 Question #1 was really, really, really, really unfair.
Most of them actually have no answer, that one is just a more well known paradox.
And the first student (the good one) circled both answers in that question and scored 1/2 point. WTF? I should try this tactique at school.
***** but if it is true that it is a lie and a lie is another word for wrong, the statement is also wrong. So you can't really say it is true. Can you?
***** The point is - how can you correct an answer if there is no correct answer? So which one in the test was true? Would I have to tick true or false to get full points. Becasue in theory you would need to tick both. But Albert Polák said for this you would only get 1/2 points. So what is someone expected to do?
So I thought you might have proof, if you state with that much conficence that it was true. But it doesn't seems like it ;-)
Alex Jones The answer is simple it's false. Because the statement is referring to itself saying that it is both true and false (whenever you state something your implying that it is true [there is a bear here = it is true that there is a bear here]). Since this breaks the law of noncontradiction, the statement is false.
I appreciate that you had to film yourself 49 times for those classroom shots!
I appreciate you counted.
I would like this comment but it has 49 likes
@@SuperMaDBrothers then what's the problem?_
I appreciate your appreciation for him!
I appreciate that you appreciate the appreciation
I read about this exact example in Daniel Kahneman's book "Thinking, fast and slow". It is an amazing example of a fallacy we're primed to believe since a young age, which actually proves detrimental to both the performance and mental well-being of a person always being reprimanded. It is just one of the many quirks of our mind that Daniel explores in the book, a big recommendation.
This is ALOT of effort to explain our last Golf outing, dude...
lol
So, theoretically, we could watch or not watch this video, like it or dislike it, share it or not share it, and the next Veritasium video will either be better or worse? :)
Only if you find it to be either, respectively, significantly worse or significantly better than usual.
meh LOL
@morris994 Hi there time to clean the browser history again :/
:))))))
It's Okay To Be Smart I love your channel Bro
2:22
Name: Jimmy Neutron
Student # : 1.675 x 10^(-27)
lol
Thank you
Clever
Thats a good eye you got
@@deletedchannel9945 And he got a point for answering that Veritasium is a real element in question two. You can see the questions at 2:10.
Selecting both answers on the question "This statement is a lie" and receiving 1/2 points is next level IQ
Or maybe just leave them blank
Those questions aren't all truly meaningless. For example, someone with a moderate knowledge of the periodic table (or Latin) will know that Veritasium is NOT an element. So there is a non-random factor in this allegedly random test.
@@stevenscott2136 The premise was that the people taking the test do not know any answer and therefore have to guess everything, thus making it completly random. The questions provided on the test prop were a gimmick for your entertainment, or lack thereof.
@@guilhermebarcelos115 Not answering usually means 0 points.
But if you choose both, one of which must be true, by definition, you will necessarily be awarded 1/2 points. And this is the maximum you can obtain on this question; after all, if you only choose one, it will instantly become the wrong answer.
@@VVayVVard I think it would be paradoxical and neither of which are correct, so leaving them blank would be “more correct”, specially if the wrong answer has a negative connotation. With that said, when choosing both you would not be half right, you would be double wrong!
I feel like regression of the mean is something I’ve applied so much in my life but never actually knew about.
Yeaa Yeaa, you know everything
yes! @@swaminathan_r1
I wish my dad was smart enough to know this when I was growing up.
:'(
dbgrfdg Your profile picture contradicts your comment.
With most negative parents it wouldn't have mattered.
dad....I wish my teachers in school and college were smart like Derek while I was growing up. Fathers being smart is a matter of luck. Teachers being smart is a matter of government policy.
@@srinivaskari government policy often fails.
I wish my teacher would say "Great work!" whenever I got a 61/100.
Well that's pretty bad soo... Donno why he would (if he genuinely said that instead of just blurting put good work to every student regardless of their scores).
@@archockencanto1645 It would be a lie to make him feel better. The original comment is clearly for humorous purposes anyway, so I'm not sure why you're taking it so seriously and acting rude.
@@TrickShotKoopa What a snowflake generation. Now this is called rude and everything ever is a joke so some weasel has a way out always. God help us from the hole we're going to.
@@archockencanto1645 Not everything is a joke. I find it unfortunate that you are incapable of distinguishing between a joke and a statement. Regardless, you seem quite pessimistic about life, so I hope you find something which makes you happy.
@@TrickShotKoopa You just lack the experience because of your age. You will realise reality in 10-15 years.
I started a new job and my confidence in my skills isnt that high yet, but the guy teaching me believs in me. He sends me off doing stuff as if its nothing, even though it makes me nervous. That makes me proud, and I want to prove myself. So I try very very hard until I get it right. If he wouldnt believe in me, I'd be discouraged.
I'm surprised he didn't bring up the fact that the test was poorly executed anyway. What they should have done was to give negative feedback to both groups and then, to other test groups, give positive feedback to both groups. That would help reduce "regression to the mean" as a factor.
I was thinking along these lines as well. It's not even that controlling the test would mitigate regression to the mean, but it would eliminate the bias of selecting pilots for praise/reprimand based on their skill level.
Wouldn't this just increase the group size, rather than solving regression to the mean?
@@melkbot Yes but it would also reduce undesirable factors that would taint the results.
He did briefly mention that this is why control groups are so important
And the testosterone thing... Chances are that the players with higher testosterone generally are better.
That distribution at 2:18 was drawn brilliantly
unfortunately it wont be drawn as brilliantly next time
This is why I love science. I clicked on this video thinking... why is a physicist talking about psychology? Why?
Because science has to take into account all sides, this is one many people probably overlook.
A great video. I kept wishing that you would clarify that it isn't that *the next* event is likely to be better (after doing unusually poorly) or worse (after doing unusually well). It is that *any* event is likely to be average (by definition). There is nothing special about the next event. Thanks for a thoughtful video!
Yep I caught that too it isn’t up down up down like wave lengths around the mean it is that most outcomes will be around the mean with more infrequent extreme highs and lows that’s literally what the bell curve shape represents
There is a great likeliness of the next performance being somewhat different though. Different within the average, but still.
Like your channel. As a manager and scientist, the most important idea is to maintain communication with direct reports. Staying out of blame and asking questions help staff focus on the important things like values and group culture.these studies are important yet they isolate behaviors without context to the whole. Thus the point is well made. And what is also important includes perception of and individual performance to the group. And if done correctly, group influence actually means more than just my impact.
I have to say constructional feedback is the best I hate when teachers give me an a- without explaining why it's not an a+ just because they say there is room of improvement or it's a way so I would keep try harder. It's better to actually know what I did wrong and how it can improve.
You need to figure out for your self what you need to do to improve. You won't always have a teacher there to guide you.
I had a teacher that would give you a 93 whenever you had a perfect score, there was no way to score higher.
It bred a hostile environment where no student strived to even try leading to her class eventually having the worst grades of all the other teachers of her subject. Fact is, if you want to motivate students you can't use negativity to hope to get anywhere, that myth that "there's always room for improvement" makes no sense on a test where a perfect score is a 93, all it does is mean that one failure is a B, making the goal beyond perfection leads to being a demotivational tool. It's like forcing your first year science kids to land a probe on the moon, they are going to take one look at the goal and blow your class off to watch cartoons since doing the impossible isn't worth thinking about.
@@SherrifOfNottingham Yes, that is what happened as I clearly took a lot of time for a project and made it as best as possible and next kid who just flunk something together get same score. I was like damn I don't need to do my best, just deliver something that is good enough and not even care about that class anymore. It only demotivated the good students and not even motivate the bad students to do any better. While teacher that really improved the class motivation to improve their work showed the best projects to the class after, so all can learn differences between their own project with theirs and know how to improve or strive for.
@@Agnemons I hate the "You won't always have a ___ with you" argument
@@Agnemons , yep, nothing teaches better than the ole "figure it out yourself" method... \s
The entire point of having the teacher there is for them to teach. Don't excuse them not doing their job.
I was told, by many psychology professors, that punishment is very effective at promoting or deterring behavior, it's actually too effective. However, negative reinforcement and positive reinforcement are usually better at keeping the desired behavior more habitual. When using a punishment paradigm you risk the chance the undesired behavior comes back when the punishment is no longer present. It's true to positive and negative reinforcement, I believe they call it extinction, but it isn't as detrimental as punishment.
Just to clarify, negative reinforcement is not punishment. It is escaping a stimulus that is aversive, which ends up becoming a behavior. Ex. Some people put their seat belt on because they don't want to hear the sound of the seat bell alarm. They are trying to escape that stimulus but aren't being punished by it. Punishment is a stimulus given after a behavior that's meant to be aversive but isn't present before the the behavior.
The United States Army has already developed a solution to this. It's called the "After Action Review" and I find that it's a wholly superior method.
It consists of the following phases.
1. Review of what our goals were.
2. Review of what actually happened. Did we accomplish those goals?
3. What did we do well that helped us accomplish those goals?
4. What did we mess up, and how can we improve more for next time?
5. Develop a more refined game plan for next time.
This takes the 'punishment' element and puts it in a more constructive light. One takes an honest look at what actions could have resulted in failure, but ONLY with the understanding that we're in a learning process.
Punishments/chastisement rarely ever logically work anyway, because:
1. All chastisement runs the risk of demotivating anyone.
2. Those who are truly motivated to do better will continue to do so in spite of the chastisement.
3. Those who are NOT truly motivated to do better will not improve anyway... and require a better incentive (i.e a reward) in order to improve.
This is why the Army focuses on positive feedback, and straightforward constructive criticism.
Now, before anyone starts telling me "Then why do Drill Instructors always yell at new enlistees?"
That is because they are being trained for stress resilience. It is mental hardening. That's a different exercise entirely.
It's interesting, I actually read Kahneman's book, but I still find your way of presenting this idea very fresh. You are a really good educator.
Man these veritasium videos from the ancient times sure are starting to surface again on my feed.
Oh man I hadn't even noticed this video was so old
This concept is very underrated and is really hard to integrate from a raw thought into an explainable concept
Having spent decades training people in both technical fields and martial arts, my experience is that it depends on the type of person you are. Most people are encouraged by positive feedback, some are encouraged even more by negative feedback while others are discouraged.
This is what the topic should be about.
Personally i hate to get positive feedback. it makes me uncomfortable. Constructive negative feedback is my bag.
Sure it makes sense to be subjective with each interaction but we need to have a method that statistically works better with a greater population, in this case positive feedback, so even in this example you have a higher chance of hitting the right method from the start
Here's a thought from the webcomic prequel:
You find a four-leaf clover, this increases your chance of finding another four leaf clover, which increases the chances of you finding another, and another, and eventually you build up so much luck that the only clovers you can find are four leaf clovers.
This was such a serious lecture. Congrats for the work done
THE DIFFERENCE IS GROUP VS SINGLE CADET.... a group is able to converse and feel an emotion towards something in unison, thus strengthening said emotion. A single cadet is alone and unable to feel with his comrades, making it an all around internal battle, rather than external,(as a team would do). It's really easy to think about if you just put your self into those situations.
This book completely changed my outlook on life. It’s pretty crazy.
When watching your videos I usually like them, then start to watch in full screen. Then for 3 or 4 times during the video I will go out of full screen wondering if I already liked the video. I wish I could like it multiple times. Excelent video.
Most of my thought pattern on this video went from "Hey, this is like what Jeffrey Lin is doing for Riot and League of Legends!" to... "Woo, where did he find chalkboards?"
2:20 That's a nicely drawn curve, Derek. Well done.
But then the next curve he drawn was pretty crappy. I guess his video right then ehh?
@@nathanwaltrip7220 Circle of lif- regression
I love this video, it's amazing. Though i understood every word of it. I am replaying it over once again.
If statical models are these deceptive then imagine the opinion we form on our individual experiences.
I'm torn between my love for Veritasium and my love for Vsauce.
Toss a coin and see which one you like best.
Subscribe to both so that youtube can hide both of their latest updates from you.
Seriously youtube. I subscribe and my homepage is entirely full of "Watch it again" "Recommended" and other crap OTHER THAN MY ACTUAL SUBSCRIPTIONS!
Treblaine well change your settinfs
Andres .C I fiddled with them before and got nowhere. I guess I'll try again.
Treblaine There is a thing called "My subscriptions." it will show the latest video of the channel that you subscribe. th-cam.com/users/feedsubscriptions
"This statement is a lie" True/False
LMAO dude you got me 😄
"Nothing is true all the time."
Is Veritasium a real element?
@@capitaopacoca8454 no
@@capitaopacoca8454 Veritasium is a made-up element name. It's a pun on the Latin word for truth, and is meant to convey the meaning of "an element of truth". For some trivia, they gave it the atomic number 42 as a reference to Douglas Adams' book The Hitchhiker's Guide to the Galaxy. However, 42 is actually the atomic number of the real element molybdenum.
@@bruhbroham8760 I didn't ask this. It was on the test.
"How we're fooled by statistics"? show me some stats on how this is
I feel as though we as humans love to have shortcuts and clear answers that are either black or white, but in real life, finding a balance in the middle is key in almost everything. I feel like the positive/negative feedback debate shouldn’t be about which one is better or which one helps more. I think it should be about which one you should choose in each different situation that requires feedback. For example, if you have two pilots training for the same exercise and one is doing great while the other isn’t, you shouldn’t use exclusively positive feedback or vice versa. You should adjust your method of giving feedback based on the performance of the person. Because even though in this exemple, both people are doing the same exercice, they do not have the same needs in their understanding of the exercice for them to be better at it. I feel like for example, the one who’s doing good would benefit more from (not negative but) constructive criticism, so that the person can still find ground to evolve from and not get stuck in a lazy place because they were great from the start. Giving exclusively positive feedback to someone who’s doing good can easily make them fall in a confort zone because the mind stops working to find ways to surpass itself (and with time it can regress) and same goes for the person who’s struggling and how negative feedback can affect their ability to trust themselves and concentrate on doing better. Sometimes when you are too stuck in a negative cloud, you can’t see anything positive anymore and you can get lost in a desperate state where you think you are inherently bad at something because of that initial experience and response that you had. Finally, i think the danger lays in thinking that there is ONE better way, its either positive or negative but no, we should always adjust. (I havent watched the video yet so I might change my mind after this if I discover something i didn’t think about :3 )
It’s important to qualify that the concept of regressing to the mean is only as powerful as the inherent variability in case-by-case performance. Human behavior is not entirely dictated by random chance so this concept doesn’t necessarily speak to the effectiveness on negative or positive feedback in relatively controllable circumstances (such as encouraging or discouraging behaviors brought about by conscious choice)
as a highschool student i can tell you now that we tend to do better if promised a reward
I see the same thing in my school, when a teacher promises us a reward, everyone tries his best to be the winner, even when we find out the reward isn't really valuable.
But when the teacher whoever doesn't do well in an exam, or a class, it makes most students don't even try answering, because they fear to make a mistake, which really doesn't help them.
But a teacher once told a student who has bad grades and never does a good thing in class to solve a problem(in math) to let him go out of class early, he handed his solved problem before any other student in the class.
I've noticed there are so many videos on this channel about the book, Thinking Fast and Slow. I'm reading it right now and these are super cool to watch and recap!
Thank you for explaining! This has helped a lot.
"Not Quite my Tempo!"
If you get this reference, you the man.
"why do you suppose I just hurled a chair at your head Neiman?"
What I learned from this video: Derek has the ability to take something which I found really dull (statistics) and turn it into the most interesting thing I've seen today :-)
Statistics ifs far from dull. The only difference is it's not a 60 year old downie reading facts from a book.
benblue3 I don't disagree with you after watching this video :-)
The thing is, he started with the motivation to a subject, and it is a good motivation, so the way in which it relates to statistics actually is interesting.
I was just thinking about this today. Positive feedback reactions may be inspiring for learning, but i think trusting someone to do a task will really boost their confidence and ability to perform.
I think that's an interesting opinion but I doubt that it's universally true.
I think I ran to this on my hobby project doing player performance analysis. I was comparing the players’ performance in a vehicle to the average of their similar vehicles. Initially I excluded the vehicle itself as it seemed logical. But the algorithm worked better (more reasonable results) when I added the vehicle back into the comparison point. It only occurred to me after this video that the unexpected behavior of the algorithm was probably due to the regression to the mean. Since the “player’s other vehicles” dataset was not that large, likely those players who did very poorly on average in the other vehicles, did better in the vehicle being compared to.
Its simple.
When you get good feedback, you think you do really good at it, and because of that, think you don't have to try as hard. But it can help you have some
confidence in yourself.
Negative feedback will make you try to get better to make sure you do it better the next time, but to much negative feedback can make you give up.
The best is a mix between the two.
What if it has to do with how much respect people have for the instructor? I know that for teachers I dislike and don't respect, negative feedback makes me contentious and positive feedback makes me annoyed, but with teachers I do like, negative feedback makes me want to try harder and positive feedback makes me lax. In the case of the fighter pilots, the pilots are trained to respect and not fail the instructors, so maybe they'll try a lot better to not disappoint and to not be reprimanded. Meanwhile, many students loathe their teachers, so they wouldn't care. However, for the sports team, it could be a case where the players don't want to let their coach down, and those who are castigated actually feel letdown and they perform better because they are depressed a bit. Just a thought.
My cousin and I used to play a lot of backgammon, which has elements of luck and skill. We were evenly matched, having pretty much taught each other the strategy that we used.
We started keeping track of our scores, including the use of the doubling cube.
The results swung rather widely, with one of us being well in the lead one week, and the other being well in the lead another day.
But neither of us ever got a consistent lead.
I read about this like two weeks before seeing this video in Thinking Fast and Slow by Daniel Kahneman, good stuff.
One thing that could also influence the effectiveness of of positive/negative feedback is the personality of the people involved.
For example, fighter pilots tend to be super alpha males with big chips on their shoulders. Positive reinforcement may not have much effect on them as they already view themselves in a very positive light. Getting them to change via smacking them down a peg or two may in fact be what works better.
On the other hand, using boot camp techniques on a bunch of grandmas in a yoga class is probably not going to have a very good result.
EXACTLY!!
To extend on your idea, there are probably a variety of factors involved. Skill might also be an issue.
People of low skill (students) might need positive reinforcement to continue to engage in the activity as their results don't show competence. People of high skill (fighter pilots) might need negative reinforcement because the only thing they could improve on is how much their "head is in the game."
Ego, Skill, what they had for breakfast and the precession of Venus might all be factors which must be controlled to gain the best outcome.
UniversalPotentate there is even more. you should take into account what the person is doing. in school you might get bored students to work in a subject with using positiv feedback, but you only become a pilot when you are really into that and you really want to do that so if you did something poorly you want to improve next time.
I believe in consistent honest feedback. Consistency is actually the key to whole thing. Also, the notion of one disciplinary technique is the best fit for all people is flawed at the most basic level.
Couple points - I agree consistency is important. But when you say "the notion of one disciplinary technique is the best fit for all people is flawed at the most basic level" it sounds like a hypothesis. Studies across species have shown that to help individuals, say, find their way through a maze, rewards work better than punishments. There may be exceptions, but I don't think they break the rule.
Veritasium As a matter of fact, I recently heard tell of an article wherein a study cherry picked well behaved children to see how they were disciplines. 60% had parents who spanked them, this discipline normally dies out between 6 and 8 years of age. One must be careful that their understanding of disciplinary studies isn't colored by media editing based on flawed assumptions regarding normal human development. If, for instance, you're under the impression that children 'learn' to misbehave, rather than are born with their own personality, you would appear to be making a mistake, although one inexplicably common amongst many intellectual elites, but, not shared by the actual clinicians who do the actual work.
I have experienced this a lot with my video games, if someone praises me i feel like i can do anything and it's now expected of me, but when i do wrong and someone calls me out i now feel the need to protect my pride by proving them wrong and doing what they thought i couldn't.
Thinking Fast and Slow is a great book and the Israeli pilot training is a memorable analogy. I wonder what the studies say about the type of positive feedback and the longer term affects. If you give students a participation award and praise them for average or even below average work then maybe they do find encouragement in that the first couple of times but they might soon adjust and realise that the praise is empty.
An important video for people currently in university or otherwise writing scientific papers. You'll understand why when you watch it.
Also when it comes to positive vs negative feedback to students and players, keeping someone's attitude and outlook on life feels more important than rubbing in their performance on previous tasks(both positively and negatively).
I can personally relate to this statistical phenomenon.
After not having done so for months or even years, I've gone out to shoot a couple rounds of Trap (its shotgun shooting sport) or had gone rolled a couple games of bowling.
In every case, my first round of Trap, I would shoot 23-24 trap out of the 25 targets and likewise my first game of bowling, I'd score of 195 or above, however, any following attempts at either sport, always yielded a less impressive score....15 for Trap and 123 for bowling.
After years of seeing this phenomenon, (I'm 60 yrs old) I discovered that there were two common denominators in each sport.
Firstly, I had developed the biomechanical skills from years of practice. Secondly,
I had experienced a sense of greatness with the results of the first attempt relative to each sport.
As I became more intent in subsequent attempts, my desire to improve the results had shifted me away from my naturally developed skills.
I realized that by not being trained as an professional Athlete, I did not have the concentration skills necessary to maintain the results and would consistently do much better by utilizing my naturally developed skills, rather then forcing myself to perform.
IMO, I believe that some of the effects that you are assigning to chance are actually due to not considering how individuals are motivated. Here's how I would conduct the pilot experiment based on info in the video and assuming sufficient pilots were available to be part of the trial(s): 1) Establish objective criteria for success/competence if not available, and measure each pilot before the trial begins. 2) Administer personality tests to determine the extent to which pilots are motivated by external feedback--either positive or negative. 3) Develop the statistical plan including group comparisons before the trial begins. I would propose assigning pilots randomly to 6 groups (more or less motivated by external feedback) and then randomly assigned to positive, negative or neutral feedback (either the instructor delivers results without emotion or pilots are just given their results by computer with no positive or negative feedback). Conduct the trial, measure improvements, analyze and then determine if changes are needed for subsequent trials.
I think there is many factors involve in learning. But in the case of positive and negative feedback, the biggest difference between positive and negative feedback, is that positive can increase your motive to learn what your learning. Because positive feedback respond to the human's need to feel good, feel love, get attention and believe in our potential, even if all of this is intense, it is also more pleasant to be suround with positiveness than negativeness. So in general, in your group, the mean should increase if you use positive feedback because you augment the motive and the feel good feeling of the student. A student motivated or that feel good gonna put more effort, more focus and maybe gonna practice on is own. Feedback is feedback so if you have relevant feedback, you still gonna be listen by those who are motivated. But positive is in the most case the best. But why then the jet student has different results. I like the explanation of the video, but still, I believe there is a causal explanation. I tougth first, that the motivation of a jet class, since you have a hard process to be there, would not be a random sampling of the population motivation that you would have most of the time in other studies. But then a remember that in my high level badminton class I would see the impact of a coach with positive vs negative feedback. So that is not it because a pleasant environment is still really important. However there is an extern variable in the case of the jet learning. Driving a jet is dangerous, we all know that if you do a bad move, you can die by screwing up. So I think that a negative feedback when you do those class make you remember the relevance of the information that's gonna come if you do not wanna die. So in that case, remembering you could die, gonna put your focus on the highest you have. The think is a negative feedback in a situation so relevant like that doesn't event feel unpleasant and it increase focus. The idea actually come from my lesson from lead in climbing. I had that focus that I do not have as high as when I knew the feedback couldn't save my life. It would be interesting to test another jet class or another class of something dangerous to see if we observed the same behavior of if the statistique gonna win again ;) It is not base on any paper or something, and I'm open to change my tought :)
The difference between Regression to the Mean and the Gambler's Fallacy is sample size. In GF, it's only one random event, so the probability isn't changed between the past and present. But in RM, it's a collection of random events. So while one event isn't dependent on it's past, it IS dependent on the average performance of the present, because the probability will be that it is in the middle, regardless of where it was before.
just because many of the data we collect fits a normal distribution, doesn't mean that the normal distribution is some kind of natural law that governs performances of humans. complex actions cannot be easily quantified in that respect- it's very hard to evaluate objectively which performances are good and which are bad. also: it doesn't violate any physical law, if a novelist is getting "better" with each book, or an athlete with each competition et cetera.
First thing, much, not many. Second, statistics in general are about as objective as it gets. Lastly, using artists, as that is what authors are artists of the written word, in an analogy about statistics is about as subjective as it gets given one's enjoyment of their craft is dependent entirely upon each individual, however the performance of an athlete can be quantified given the rules of the game they play, so comparing them in the same analogy is a fallacy.
Thanks for correcting my grammar. Being not a native speaker of the english language, it's always helpful to get advice in that regard, so please also keep an eye out for potential mistakes in this comment. I have to add though, that "data" can be seen to be either plural or singular, so my version is also correct, albeit maybe a little unfamiliar to you.
Now: 1) What did you mean to imply with the remark "Second, statistics in general are about as objective as it gets."? I don't see any connection to what I said, or any meaning behind these words in general. Did you mean the mathematical field of "statistics", or the results of satistical analysis of certain phenomena? I certainly didn't criticize statistics in general, just that the conclusions about reality, that were drawn in the video, were drawn from mathematical concepts, not empirical measurement. That is: Regression to the mean is not a universal law of nature- I could for example built 5 towers in my life, each one bigger than the next. This is an obvious fallacy in my eyes.
2) The evaluation of the performance of a pupil in class is also massively subjective (I'm not talking about tests here). As is the evaluation of most complex human activities- this was my point. So whereas the performances of individual athletes in certain disciplines can be quantified with ease, in many other disciplines they cannot- for example: football, boxing, dancing, etc. Of course quantification of many single aspects of the performances is straightforward, but it's certainly not easy to attach a single number to each performance. Therefore talking of athletes and novelists in the same sentence - I didn't compare them explicitly, by the way, they were just part of the same enumeration - doesn't constitute a fallacy, and I could have also legitimately added scientists and painters and whatever happens to cross your mind.
Regression to the mean works with any distribution, not just the Normal distribution.
And if everyone is getting better over time, it means that the average itself is changing. The lesson still holds at any given time that the ones scoring much higher than average today will probably not score as high above average tomorrow.
+Elliot Collins
I'm not an expert in math, though I'm not sure if what you said holds true for power laws.
Also: Your comment doesn't touch on the issue, my main argument, that it's hard to quantify most human actions (I'm not sure, for example, how you could rigorously quantify the performance of jet pilots for example...), therfore you mostly don't even get a distribution (as you have no data).
We don't have "laws" of human behavior, that mathematically model our actions, so the whole thing about regression to the mean is more of a metaphor, than an actual law that could be violated. Therefore I don't take it too seriously.
No more than that, did I intend to say! If you still disagree, it's okay
***** I agree that it's hard to quantify behavior, but that doesn't mean it's impossible or unreliable. We quantify performance on tests, amount of money made or spent, popularity of youtube videos, percentage of times a fighter jet's maneuver works, etc. etc.
There aren't deterministic laws for social science, but when we measure outcomes of behavior, we see distinctive and useful patterns. And when we use those patterns to improve fighter performance or road safety, the principle of Regression to the Mean is important to remember.
I think something that's different but related would be the fact that we tend to naturally give more feedback in respond to a negative performance (from traditional skills to social ones) than a positive one because we naturally intuit that less feedback is needed when things are going well than when they're going poorly
Veritasium your videos are ridiculously good to watch. It’s crazy to think this was published 9 years ago, when life was simple.
both positive and negative feedback work. it's not a matter of which is better, they BOTH should be used for maximum effect. punishing failure in addition to rewarding success provides a better gain than simply rewarding alone
"punishing failure in addition to rewarding success provides a better gain than simply rewarding alone" got any data to back that up?
take a shot everytime he says feedback
I'm not 21...
+Sam Blough Poor american...
Wahngott Or 19
+Sam Blough 18*
Duke Of Spook Don’t.
Wow! you've been at this for this (and more) longer!? no wonder you got viral, your content is clean.
connection!
Regression to the mean begs redemption day by day.
Thank you for your work, the message at the end was great too
Coming back to clarify some concepts, still an excellent explanation full of relatable examples. Thanks!
This video was fascinating, but I feel like it wasn't really about punishment vs. reward so much as positive feedback vs. negative feedback. I guess I'm just curious to see how (for example) kids might react to discipline or candy.
not trying to be a generic non-answer, but i truly believe they need both. Although some will respond more to another based on their personality. Overall they need love and acknowledgement, but they also need to know failing is ok and part of life and that in the end they aren't less because of it.
If you seek to programm behaviour into people, I suggest you look into ridiculing [The Effects]. Straight off the bat this sounds evil but its effective and can be used for good.
Make sure to check out Plethrons . They have some pretty good content!
Itsover ninethousand That comparison is ridiculous. Punishment is meant to teach a lesson to the kid. You don't punish your girlfriend or your friends because it's not your job to teach them how to behave.
It's a completely different relationship.
alejandrinos
Children deserve to be treated as people, too, though. When they misbehave it's not like they're doing it out of malice. It's that it takes time for them to learn. Punishing them distracts them from the lesson and teaches them to look at situations selfishly, as in "how will this affect me?" instead of its larger impact. Talking with kids is a better way to improve their behavior, and it promotes a better parent-child relationship as well, which is important if you want your kids to keep listening to you as they get older.
I love the thumbnail because the time duration partially covers the text such that it reads "Regression To The ME"!!!
I love this episode- Awesome. Accurate. thank you for making this!)
"three kinds of lies. Lies. Damned lies. And statistics." -Mark Twain.
@@ENXJ oh wow, he sounds cliche, how terrible. At least he doesn’t sound like an asshole.
@@ENXJ Oh wow, how kind of you to be an asshat to someone who at least wanted to share an idea, douche.
@@dinglesworld three kinds of ideas. Constructive/helpful ideas, neutral ideas, and destructive/dumb ideas.
@@ENXJ Lol "It's fun to be mean"? Ok troll
Last time I checked, Mark was no mathematician. Statistics are pure. Raw. And one of few things we can truly trust.
People with an agenda can use then to serve that agenda (good or evil).
I believe that the only thing learned through reward is how to get a reward.
I suppose it's all about what we are learning as well. When we learn not to do something, it is usually through negative re-enforcement and the opposite through positive re-enforcement. However there are so many variables that it becomes hazy to know when negative or positive re-enforcement should be used for improvement.
It is also expected that by using both, eventually an equilibrium is created and each persons equilibrium can be different and can change from time to time, that equilibrium being the point where one is content with oneself at the moment where one doesn't feel the need or expect a reward for learning something, and is also aware of possible consequences which is reward in itself, because one then avoids that which may cause harm, be it mental, physical, or psychological.
I realise as well, that those who always seek reward tend to be self-centered, and greedy and often are not aware of or care for consequences resulting in their need for reward. These people I believe to have great difficulty with the negative aspect of learning and have emotional imbalances.
When I became aware of this subject, it was hard not to notice it, see it in action and the results created, in my workplace and with the daily interaction of people. This also allowed me to know each persons state of equilibrium and the amount of balance or imbalance, and how it affects the person which in turn seems to effect their personality traits and reveals the level of aptitude they may have.
Thank you, i appreciate your videos. You have a great job!
I’m no mathematician but I actually have “regression to the mean” tattooed on me 😂
As an IDF former fighter I'd propose a different reason for the success in negative feedback.
The fighter jet recruits have a very hard competition which determine who will stay in the program and who will be kicked out. Their for they are looking to improve their flight technique and prefect it. Additionally positive feedback may encourage them to do better, but they won't know what they need to improve either.
And last but not least, the army system works completely different then any other system. You can not make a soldier out of a civilian only by positive feedback, you have to have negative feedback. For those reasons and more the ground for comparing the army and any other system is very small. Thou I do agree with your call for positive feedback, it simply differs from the army system.
Excellent video, man. You nailed it.
Positive reinforcement with dogs has proven to me a lot better at teaching them new tricks, while negative reinforcement has proven more effective at interrupting unwanted behaviour. So for me it would be "if you are trying to teach how to do something, use positive reinforcement; if you are trying to teach not to do something, use negative reinforcement" .. Of course you could say "we are teaching pilots not to make mistakes".. Still I think my idea has a little something to it..
Well explained. I had expected to be fascinated but partially mystified, so this was a pleasant surprise.
Whenever i become aware of faulty ways of thinking i always do my best to avoid them in the future... but this... this is gonna take a while to wrap my head around...
Explained perfectly
However, when i get a bad grade usually i lose motivation whereas if i get an A or 100% i will be more motivated to study harder.
That's not the same thing as what's being talked about though. When you consider the true or false test is about questions students wouldn't know about, their luck is just that, luck, and that changes from day to day. If anything, this only proves that luck and knowledge, like the knowledge you gain from studying is are not something that go hand in hand. You can get lucky and guess a few answers right here and there, but if you know more than you will get a higher score. The positive or negative reinforcement being talked about here is also something that you get from people as opposed to self-accomplishment or something you did by yourself, so I imagine how it affects what you do is also different.
Really? When I get a bad grade, I'll realize I need to step up my game, whereas a good grade tells me I know my stuff and can slack.
For me, negative feedback can really demoralize me over time. For the first or second times, it can make me feel like I'm proving them wrong and I do better, but after too long I just shut down and say "I don't need to please this person" and end up not wanting to do anything they say.
On the other hand, sometimes with positive feedback I know I'll have to up my game to impress again, so I'll try to do so. Of course eventually that becomes difficult and I level off again to reestablish myself before trying again.
Both forms of feedback are beneficial for me in small amounts. Too much negative and I'm less likely to complete the task in the first place, too much positive and I know I can't always do better.
Truly successful people don't give care about feedback. They will do their best no matter what. Hard to achieve that confidence though. Set your own goals and congratulate yourself. People who rely on other's opinions are screwed.
@@mikeyo1234 That's too oversimplified buddy. It is dependent on your expertise what kind of feedback you prefer to a certain task, together with your confidence. So basically the felt confidence to successfully deal with something is the important thing. People low in expertise and confidence obv need positive feedback to reassure them they are doing fine. But people with very high expertise are different. They (usually) really happy when they get constructive negative feedback because it rarely happens when you are super good in something and this gives you a chance to even get better. So when expertise increases in people there comes a turning point where they actually prefer negative feedback.
Still, surely the negative feedback should be phrased nicely like "I think you should try dadada "
I work well with negative feedback personally.
Something people has to always remember, you can't keep a steady 110%, but it is easy to exceed low expectations.
Speed cameras and red light cameras reduce speeding and running red lights, but often increase rear-end collisions by causing drivers to slam on their brakes to avoid tickets.
yup, an example of forgetting the goal in favor of the means
I noticed this when playing an RPG. Everytime I would get a "miss" I would shortly later get a "Critical hit", so it all evens out.
Something was bugging me about the logic of this video and it just hit me what it is. (Bear with me - this is a monster of a "comment.")
Assuming that you're right with your standardized test example, it makes sense that if - for instance - you get 100/100 on a test, and then you retake it, chances are (speaking in terms of probability) you're going to get less than 100/100 the next time.
This works for quantifiable ceilings (and floors - such as 0/100) because the inherent belief is that you can do no better than 100/100 and no worse than 0/100. But if we delve into more subjective examples, where the ceilings and floors are highly arbitrary and ill-defined (such as acting) the logic of it becomes increasingly psychological.
Matthew McConaughey, for example, won an Oscar this year. Imagine if he were to say "that was the peak of my performing ability" (the acting equivalent of a perfect score - 100/100). Using the logic from before, that would mean that he's likely to do a somewhat worse job in his next big performance.
However, the real McConaughey said that his hero is himself in 10 years. What this means is that there's an older, all-around-better version of himself that he believes he can always reach. In other words, he doesn't believe that there is a ceiling for his abilities. So the "ceiling" logic of his next performance being worst than his last may not actually apply because he doesn't believe it was his best possible performance.
I think this psychological distinction is important to note because standardized testing internalizes this idea that you have a floor and ceiling to your abilities, and you can't stretch above or below those areas, which is incredibly limiting to believe! And if you apply the logic of your video here to a ceiling that you believe you have, then it makes sense why some people fail before they even get started: They don't believe that they can grow beyond a certain point.
It also adds some weight to the motivational tough-coach routine, like the techniques used in the movie "Miracle." The coach provided some harsh feedback and some kind feedback, but that ultimately wasn't the point. The point was that he wanted them to believe that they could get better. And better. And better. And he insisted that he knew they had more ability in them than what they were offering.
In short, he broke their limiting, psychological ceiling.
wow, you just blew my mind. Thanks.
In line with your thinking, I feel that much of my success has come from not setting limits on my own capabilities. I do not always have the ability to do something, but I believe I have the capability to acquire that ability if I desire to. "That sounds hard" and "I could never do that" are not thoughts I entertain.
I think I've traced this thought process back to a single moment of realization. When I was in high school (went to a computer science school), a friend of mine said to me, "You know, as programmers, we're just not creative people." I didn't disagree, but on the inside I rejected that idea completely. Later that year I went and got an internship as a graphic designer. I learned how to use illustrator and photoshop. I learned how to be "creative." I majored in Computer Science and Film. I can program, draw, animate, write, film. Every hobby, skill or talent I've acquired crosses over and lends itself to every other area. The end result is that I am capable of developing games entirely on my own. Better still, when working in teams I have an understanding of the work that each element of a game requires and I understand how to fit everything together cohesively on a project because I've done those jobs. The idea that a programmer and artist cannot do both well because it requires two different types of minds is something I reject.
I truly believe this comes down to personal philosophy. All it takes is refusing to set those limits for yourself. Anytime I've needed to do something, I've opted to learn how to do it. On the other side of the coin, I think there's a lot of positive feelings to take away from this video. I was not familiar with regression to the mean before watching this video. I am happy that I did. As a very serious 10-pin bowler, I have the occasional 250+ game. When I bowl in the 170s after that I feel like I might be getting worse. That I'm not improving. I think having a conscious awareness of regression of the mean may eliminate those feelings. I'll have bad games and I'll have good games, but you can't measure the highs and lows relative to each other the way that I've been doing. So I thank them for this video, and I also thank you for your thoughtful comment.
Idong Itauma
Happy to. Blew my mind too when it occurred to me.
Bryan Mitchell
Thanks for sharing your experience. I agree with you that not setting limits greatly changes what you can and can't learn. I think people who have disabilities and learn to work around them are a great example; they could opt for saying they're crippled and avoid doing most things, but instead they often push past the hardships and figure out ways to complete the tasks they want to get done.
Bryan Mitchell Thanks for sharing -- I'm really impressed with what you've accomplished! Your comment on bowling scores reminded me of what my music teacher told me about performing. He basically said that you have good performances (like a 250+ bowling score) and bad ones (in the 170s), but whatever happens, every guitarist has a "bracket." Brackets have both a range and a central tendency. For example, if you rated someone's performance on a scale from 1-100, the very best guitarists will almost always hit 95-100; small range (consistent) at the top of what is possible. Lesser-skilled performers might range from 70-90; larger range (less consistent, having more to do with luck) at a lower over all level of skill. And even good students shouldn't be surprised to range from 40-80, until they learn to hone their craft.
This is fantastic! I have done many statistical studies, and you have challenged my thinking!
Shawn R. Hughes, Ph.D.
This very interesting video could be even more interesting if it addressed how the mean moves over time. Since nothing exists in a vacuum, our attempts to measure and influence do, eventually, move the mean one way or another. The way this video was presented, and I'm sure that wasn't the intent, kind of made it sound like the mean eventually overpowers any attempt to derive meaningful information from cause and effect studies. One place where this becomes extremely obvious is in the movement of price for currencies on the foreign exchange market. There, there is a very strong tendency for price to revert to the mean and revisit previous price levels, but the mean is constantly moving, since price is constantly moving. It becomes much more challenging to figure when things are at an extreme, and when they are actually not because the mean has slowly shifted due to the repetition of certain events (price points) being over represented.
Derek you are an excellent communicator and you make amazing videos. Even though this video is 8 years old, I would love it if you made one addressing the points I mentioned above, namely the movement of the mean and moving averages over time, as sample sizes increase and feedback loops are introduced.
This video literally just made my day. I've been struggling with depression a lot lately and it just makes me feel like things will always average out to be better. Like I'm getting all the bad days out of the way now, so I can live a happier life later, ya know?
So, 4 years later, did it work that way or what?
@@giampaolomannucci8281 unfortunately Zachary has passed away a few months ago
@@windowsxseven How do you know?
Seems like he updated one of his playlists a couple days ago
I hope the correct answer for the question "There are 6 continents on earth" was TRUE.
The question doesn't ask if there are only or exactly 6 continents.
Well, it depends. There are 4-7 continents depending on what convention you swear by.
You think like a mathematician
Is the statement "There are 0 continents on earth" true then?
@@jeremyquentin42 no. 0 is definitive and means there can not be any more. If a thing has 7 pieces, it also has 6,5,4,3,2, and 1 pieces (and also fractions of those pieces, but that's going down the rabbit hole and there is a point where the math gets really weird), but nothing can have zero and non-zero pieces.
This is why questions that are designed to test intelligence need to be very, very specific, or else you punish the test taker for thinking critically, something that an intelligence test shouldn't be doing. There ARE 6 continents on Earth, just like there are 6 hours in a day.
@@jeremyhulbert3343
We can prove otherwise by recursion though.
If A has n B's, then it has (n-1) B's.
So technically, Earth also has -1000 continents.
This is one of my favorite videos of all time!
As a trainer I found this very useful man, thanks a lot
It would have been so much better if everyone understood this. I've been thinking about this for a while and tried to explain it my acquaintances, but you can't change that in a day. Thank you for making the first steps to bringing that understanding to the majorities.
A lot of people think that a 50% change of getting head or tails means that if you flip a coin 100 times it would result in 50 heads and 50 tails, but in practice that is not exactly true. There is even a small change that all of 100 flips it will result in only one of conditions.
As they say - there is a big lie, a small lie and statistics. And true and at the same tame false this statement is.
That's it, schools should stop punishing for bad performance but only give awards for good performance. If you keep punishing people then they will do anything not to get punished but they won't care about performing more than just the minimum because the not getting punished is the focus. If you add reward together with punishment then there will be something to strive for but not getting punished is still the main focus which interrupts with the idea of striving for the reward . If you have only rewards then people don't have to worry about being punished and because someone get's more rewards than others competition emerges. The best thing is that no punishment means freedom to choose how to compete and which rewards should the person be going for.
Good video, I learned something. In real life, like the fighter pilots or team players, it seems that constructive criticism is key to improve skills. Highlighting good behavior can be a buffer for the constructive criticism "You did this very well, but you need improvement in . . . "
I was talking about this today and didn't know what it was called, and here you are recommendations to fill the void of my soul again
the question is: do people get more afraid of consequences than excited about rewards?, or viceversa
Yeah, and he didn't even try to answer the actual question. He's just pointing out that "oh, you know, luck also plays a role in these things", which is not big news. I'm sure we can find a psychological answer to the question, this video was just about statistics.
But I still subscribe him cause he's hot.
Just look at the gaming industry and you get your answer, because they employ good psychologists. ^^
Short answer: excitement wins. So hard that you get addicted.
So from watching your videos, I gathered that the most efficient way to success or improvement is to receive rewards but keep stepping out of your comfort zone 😊😊
The difference between the coin and the pilots is that the coin tosses are (theoretically) independent, that is, past tosses don't affect the outcome of future tosses, whereas the pilots' performances obviously depend on the pilot, so future performances by a pilot are expected to be dependent of past performances by the same pilot. That's the difference between the gambler's fallacy and regression to the mean.
I really wanna say you do a fantastic job at making your videos timeless. I can never tell when they were made until I look at the date.