Can you trust your smartwatch?
ฝัง
- เผยแพร่เมื่อ 16 พ.ค. 2024
- Want to know how your smartwatch measures your heart rate? Well, the next thing you should watch is this video I made with the amazing @SteveMould! • The bizarre flashing l...
And these videos are both part of my Smartwatches series. Check out all the instalments here! • Smartwatches series
Smartwatches make some grand claims, counting steps, heartbeats and sleep, and offering insights into our health and wellness. But how accurate are the numbers? Which is the best smartwatch? (Out of these three, anyway…) Using the latest scientific research (and some of my own experiments!), I ask: can you trust your smartwatch?
In this video, I pit an Apple Watch SE, a Fitbit and a Mi Band 6 against one-another. Obviously there are lots more brands on the market (Garmin, Huawei, Oura Rings and many more!) and if I had a bit more budget it would be great to do a more comprehensive comparison of their accuracy. One day… :)
Chapters
00:00 Introduction
00:31 Counting steps
04:56 Heart rate during exercise
07:40 Heart rate: all day
09:01 Heart rate: all night
10:47 It knows when you are sleeping…
13:12 Conclusion
Sources and further reading
Steps
‘Accuracy of Fitbit devices: Systematic review and narrative syntheses of quantitative data’ mhealth.jmir.org/2018/8/e10527
Heart rate
‘Investigating sources of inaccuracy in wearable optical heart rate sensors’: www.nature.com/articles/s4174...
Study comparing the Polar H10 to research-grade equipment, finding that it’s suitable for use as a ‘gold standard’: pubmed.ncbi.nlm.nih.gov/31004...
Did I mention the video I made with @SteveMould?! • The bizarre flashing l...
Sleep
Systematic review of Fitbit sleep analysis: www.ncbi.nlm.nih.gov/pmc/arti...
Two more articles looking at different devices’ sleep performance: www.ncbi.nlm.nih.gov/pmc/arti... www.dovepress.com/evaluations...
Apple Watch sleep stage tracking performance from @TheQuantifiedScientist: • Apple Watch SE 2022 : ...
White dudes
‘Limiting racial disparities and bias for wearable devices in health science research’: academic.oup.com/sleep/articl... (I’m going to make a full video about this at some point but it would be remiss not to at least include an introductory link…)
Credits
Thanks to Tran Nguyen for filming and letting me take her expensive doctor stethoscope to Steve’s studio, and thanks to Linh for letting me borrow her Apple Watch for testing!
And finally…
Follow me on Twitter / statto
Follow me on Instagram / andrewjsteele
Like my page on Facebook / drandrewsteele
Follow me on Mastodon mas.to/@statto
Read my book, Ageless: The new science of getting older without getting old ageless.link/ - วิทยาศาสตร์และเทคโนโลยี
Go check out my video with Steve Mould about how smart watches use green light to measure your heart rate: th-cam.com/video/BFZxlauizx0/w-d-xo.html
And hello to everyone coming here from Steve’s channel! I hope you’ll stick around. :) Edit: Woo! Just passed 10k subs! Onwards to 16,384… Thanks, Steve Mould crew!
Edit: just passed 16,384 subs! And Steve passed TWO MILLION, massive congrats!!
How did you make the code you used? You had a nice graphics and graphs which are beyond my programming abilities so Im curious 😅
@@SomeKidFromBritain Haha! The video-wrangling code for Steve’s heart rate was all Python, but everything else was in R (spot the former bioinformatician), and all the graphs for this video I cleaned up a bit in post. But the ggdark library for ggplot does make some pretty graphs straight off the bat!
@@DrAndrewSteele Ok cool, so you took data from your camera and lights, got it on the computer, and then plotted it?
@@SomeKidFromBritain Exactly, I took the video recorded from the camera, extracted the pixel colour data from Steve’s face, then plotted the changes to find his heart rate!
@@DrAndrewSteele Well explained, thank you very much.
At 9:00, you said that during sleep is when the HR measurements are at their most accurate. I would like to point out that is also when their step count is at their best as well.
I love that 😂
LOL
But do they count the steps i do when i run in a dream 🤔
🤣🤣🤣
imagine you waking up and discovering you made 600 steps during sleep lol
I feel something that has to be pointed out is how cheap the Xioami Mi Band is compared to the others in this comparison. While it may not be the most accurate, it is definitely more than enough for people to get a good picture of their fitness level and make changes accordingly.
Very true, I did mention it briefly and I am impressed for the price!
10:27 mi band price mentioned here
Another thing to point out is that Xioami is owned by the Communist Party of China so if you dont want them spying on you or dont want to support a genocidal dictatorship then you might want to consider a different brand.
While I was paying attention to the video, I also didn't notice the moment he mentioned the price. So I think it's good that you pointed it out here :)
Yep, the Apple Watch can be 10 to 30+ times as expensive
A couple of things to consider in your tests.
Most of tracks are relatively sensitive to the location on the wrist, how tight it the tracker is wrapped around it and any ability to slip and move around. In my personal test I also found out that metricks were much closer to my chest strap (consistency) when I shaved my wrist. Optical sensors that have the ability to glide on skin don't play well with hair moving across it's optics. It's hard to conduct parallel tests without disadvantaging a brand. Perhaps a row of 3 tests with each watch in primary position would compensate for that.
I appreciated the collam with Steve explaining how actual sensor tracks the pulse, as it explained to me the early miband controversy over recording heartrate of inanimate objects.
If you don't want to shave your wrist and look like a dork in the swimming pool, you can also just wear the watch on the inside of your wrist. I tested the differences and found the watch (Garmin Forerunner45) to be more reliable and accurate when worn on the inside. It wasn't the purpose of my test to find that it worked better, so I didn't try to find out why. I guess you just explained why =)
My reasons for testing were dual:
I have a habit of damaging lots of stuff by getting stuck behind them with my watch, and the reciprocal damage to the watch and sleeves hanging over the watch is also relevant, so I needed to find out a better way of using the watch. The other reason was to settle a discussion =) I asked my brother who is an active athlete and avid user of tech gadgets, and he immediately said he assumed that it would not work or not be reliable at all, given that it was designed for the blood vessels on the outside of the wrist... Luckily I know more about anatomy than he does, so his remark actually made me think that it could definitely work. To prove me right, all I had to find out was if the data was consistent and usable (I'm only interested in observing my own evolution, so it wouldn't matter if the numbers weren't absolutely correct, just as long as they were consistent). I had also considered wearing it on my ankle/calf if the inside of the wrist didn't work. Just anything to not have to wear it on the outside of my wrist.
So I tested, using heartrate monitor and blood pressure devices for the norm values and found out that not only does it work perfectly, but the actual values measured matched the more bulky old-school devices' values much closer and much more consistently. On the outside of the wrist there seems to be much more variance in the signal (both compared to itself as well as compared to the norm values), while on the inside the difference between the measurement and the norm values was always the same.
@@lukearts2954 Interesting, thanks for sharing that! I also tend to wear my watch on the inside of my wrist because of practical reasons, similar to yours (and a habit now) and was often curious if a sport watch measurements could be wrong because of that. Unfortunately my watch is a total garbage, so I can't have any consistent results, but your experience assures me that I can wear it in a convenient way.
Thanks for the comment and sorry I missed it for a few days! I’ve read this too but interestingly didn’t find the effect very strong… I wore the watches on the same and different wrists, in different orders etc so they were in lots of different positions, but it didn’t seem to make a great deal of difference to their relative accuracy. That said, I’m white and have thin pale hair on my arms so the watches are definitely playing on easy mode!
I did manage to get some strange effects which may be due to the watch sliding up and down the wrist but, as I said in the video, these didn’t appear on the test run I did with all the devices. D’oh. We discuss it a bit on the extra content for Steve’s Patreons, and I’ll probably talk a bit about it in a future video too! This one was already quite long…
@@DrAndrewSteeleeasy mode! 😂 But seriously this brings up an interesting question in my mind of might these watches suffer from a hidden racial bias? Since they use optical sensors....
It would be interesting to see if the designers/programmers could greatly improve accuracy if the added sensors at different points along the watch band.
I wonder how a Garmin watch would have performed in your tests since I heard that their sensors and measurements are one of the best.
I have a Vivosmart 5 and sadly, last time it did horribly. I was trying to do some cardio and once I felt reasonably taxed I repeatedly checked the wach, but in never went over 120 bpm. After a lot of wheezing I checked on the machine I was on and it said I was over 160 most of the time. Maybe it would have helped if I told it I was on a cross-trainer. Possibly the relatively slow movement threw it off.
They are among the best, though Apple tend to be the most accurate.
But really, using optical sensors to measure heartrate just isn't very accurate. *None of them* are great. They are a trade-off between comfort and convenience, and accuracy.
Go watch "The quantified scientist" on youtube, he does actual tests of most mainstream smartwatches. This test here doesn't even come close to what he does.
Edit: Garmin isn't great. Apple does what they offer the best, simply due to the fact they have more money to collect and write a better algorithm. Sensors are less important than algorithm when it comes to smartwatches.
When I do a bicycle workout, my optical sensor is near dead on perfect compared to wearing a chest heart rate monitor. First few minutes are iffy, but after that (still in my warm up), it will be within a couple BPM. This is using a Garmin Fenix 6.
Accurate enough that I don't worry about wearing a monitor for planned workouts lack that.
I have the VEVO Smart 2 and it's dead on the money. I check every Friday at IViG treatment. I do make sure I keep the app updated as well with weight etc...
The point about measuring some of these only at night is very important one as people get a false impression of their resting heart rate, HRV etc (everything appears better when you’re asleep!)
Feels appropriate to heart your comment on this subject
(Also it’s a good point!)
Like you said the absolute numbers might be off but the relative trends day to day (assuming decent precision in measurements) is the aspect that can help us improve our lives. They’ve come a long way though! I remember getting thousands of steps on the original fitbit in ca. 2013 when all I was doing was mixing samples on a vortex mixer. At the end of the day if someone wants a device that is cheap and encourages them to stay fit or sleep better, maybe accuracy isn’t the be all and end all. On the flip side, if you’re into tech like me, you’ll end up watching the quantified scientist and looking like a cyborg wearing a whoop, oura ring, apple watch, polar etc in order to get ALL the data! Question is, how many years until implanted trackers? Scary thought.
Haha yeah I found The Quantified Scientist while researching this video! Serious dedication! I admit reading all about these made me a bit cyborg-esque/tracking obsessed too…
I appreciate how thorough and clear this is. Answered every question I was having. Great editing and use of graphics.
I come from my long time friend Steve Mould's channel, i really liked your collab and this video and you have earned my humble subscription. Great work
The amount of hard work done for this video is commendable. Loved your work
As MI Band 7 user actually satisfied with the performance. Actually my use case is more like smartwatch than smartband, with small size and cheap prices actually it's a good device. Thanks for reviewing and compare.
I really appreciate your hard work to get these data keep the good work going.
Great video, loved the depth you go in.
Waiting for the next part! Subscribed.
I've found with my Garmin watch that it's pretty accurate on step counting under _most_ scenarios. It gets it pretty close (tested by actually counting 1000 steps) when walking normally, when carrying a shopping bag, when walking with my hands in pockets. The one time it absolutely fails to count any steps has been when I've been pushing my cycle (puncture) and keeping my hands on the handlebar. On the flip side, it absolutely fails on stair climbing LOL, either registering 0 flights or stupid numbers like 50 flights when walking alongside a river, which definitely isn't on a slope.
These are some fascinating insights to how smartwatches work. I hope Andrew will do more on fitness trackers, both smartwatches and chest strap wearables. I think people would be very interested to know about caloric burn calculations also. Great video, Andrew!
Definitely, I’m going to make a video about calories so watch this space!
A great series idea! Great video production and info!
Such a great video. Haven’t seen anyone doing such a great analysis. ❤️❤️
This is a video that I have needed for about a year, thanks 🙏
Andrew, love the video! Would be very interested to know how accurate these measure total calories throughout a day. A common use case is calorie tracking for weight loss and this would be helpful to know.
These are some good quality comparisions. The tests, visualization and conclusions are well thought. Great work :)
Great video - keep up the good work Andrew👏.I agree with you that there should be more open test results available especially given the increasing use of smartwaches.
Also, I have a Garmin instinct 2 and it measures this thing called HRV (Heart rate variability). I was wondering if you knew, given your research whether it's actually an important metric?
Great question! And one I’ll be discussing with a cardiologist in the next part of this series… :)
Interesting! I own a Mi Band, that I mostly got because my heart rate was doing funny things and for €30 I figured it was worth the risk to see if it would help with my anxiety about that. (Turns out I was super anemic.) And it does help with that, because I don't have to actively count my heartrate when I'm dealing with a panic. The seem pretty accurate for that, compared to my counts. Only thing it does weirdly is occasionally it drops to 45 when I can count my pulse and it's higher then that.
I don't use the fitness tracker, and even try not to use the step counter, because it's not good for my mental health at all. Which I wonder how that is influenced for other people too, since these trackers became so common.
9:44 That shock is also what allows me to jump out of the bed in the same second I wake up, sometimes I start the jump while sleeping and land on my feet, awake.
But I don't jump every day, only if I have slept good and something I classify as important is happening soon after my alarmclock goes off.
Thanks @Steve Mould for referring you. God Bless.
I love the way you present de data on these videos, really helps understand everything
Thank you!
I noticed a problem with the heart rate measuring when doing sports. When I sweat a lot under the watch the reading can be wrong and I mean completely .. like suddenly I had 20 beats. So maybe thats another factor to include. Nevertheless my Mi band has certainly the best performance for that price, because I just want to know the rough amount of steps and distance when I go for a run. Lastly I think the different brands try to stretch their battery time with messureing the heart beat less often.
But nice channel, will watch more videos :).
Fantastic video! Truly relevant, and of value!! Much appreciated for your efforts and insight
I came over from the collab with Steve and I really enjoyed your video, so I subscribed!
Great video!!! I came here from Steve's channel and subbed!
Thank you!
Such an underrated video! I've wanted to know all of this for so long! I really want to know if the "sleep quality" and/or "stress" measurements are accurate. Mostly because I don't think they are!
I’ve collected a bit of data to check out sleep quality, but stress is a nightmare…what can I compare it against? Anyone got a good-standard stressometer I can borrow?!
@@DrAndrewSteele Your alarm clock. :D I still don't understand why people like to wake for a heart attack.
This was very helpful, thanks
Great review keep up the good work
As once teaching Science, then an IT Trainer and now an old man prescribed Sleep Apnoe who has gone down the getting 'super fit' path in the hope of being not having to use CPAP machines, which I hate and have problems wearing - I am currently using such skills looking at how well, accurate and cost of getting a smartwatch would be to me - so thank you, a very good, informative and far better than anything else I have seen on here
The Oura ring would also be nice to check out. I don't think there are too many ring wearables out there. But even to see how they perform compared to the big bulky watches and other medical grade equipment could be very interesting. Especially since they are about the same price or more than something like the Apple watch, and they claim to track just about all of the same data points if not more!
Yeah I’d love to get hold of a few more devices and try them out too! The Oura Ring and the Whoop Strap are both interesting for not being wrist-mounted…
@@DrAndrewSteele Whoop Strap? Ooer, sounds a bit rude. What do you strap that to, dare I ask?
@@Nilguiri Your, er, upper arm, disappointingly!
@@DrAndrewSteele Oh well, to each his own! haha.
@@Nilguiri They also have sports bras, underwear, swimsuits, thongs, and shorts that you can put the Whoop sensor into!
I can't believe I didn't know about your channel before you teamed up with Steve Mould! It's right up my alley. I'm going to show this to my high school stats class. I'm trying to show them that there are lots of ways that a basic familiarity with statistics and the methods of collecting data can help answer really interesting questions. I really want to help them see stats, and math in general, as a powerful tool they know how to wield, not just an esoteric puzzle system that math teachers make you learn.
Nice! Hope they enjoy it! I really wish I’d done more stats at school. My A-level maths teacher forced us to do S1-3 (no idea if that still exists!) instead of mechanics M4-6, and as a bunch of physics nerds we weren’t happy about it…but I’m very grateful for it now! Even some fairly simple stats plus a bit of thinking can get you a long way. :)
Loved it!
While I'm not in the market for a smartwatch now, when I eventually will be in the future, I'll be sure to check for the accuracy of data as much as the features - something I might have overlooked had I not watched this video.
However, for consumer devices not posing to be a replacement medical equipment, I believe there is a point after which these devices can get away with being inaccurate. As long as the data they provide is actionable for normies like me and the methods they use to gather that data are not very intrusive, I can allow them to be inaccurate.
Definitely, there’s a trade-off when choosing between accuracy, convenience and other functions when choosing which one to buy. Fingers crossed they all keep improving-and most of them seem to be good enough to be actionable now!
Literally got my first smartwatch yesterday and was wondering how accurate those measurements were.
Now thanks to you and for once, the YT algorithm, I instantly got what I was looking for.
That alone is reason enough for a sub.
is it Apple or Garmin?
I've always felt like I could trust the pulse rate and 02 percentage but the step counter has been very optimistic on every smartwatch watch I've ever had
What a superb video. Thanks for putting this together, very nice production value (and extra points for not adding annoying background music like every other utube video).
This helped reassure me that the incredible number for my heart rate while shoveling snow was accurate. Last week I was pushing large chunks of ice and snow with a prowler shovel some distance. Stuff is heavy and I kept a constant eye on the Apple Watch's heart rate monitor.
At one point it hit an incredible 196 bpm. Wasn't sure if that was a fluke but thanks to your comparisons, I now realize it probably was accurate. This is kinda scary. Could explain why so many people have heart attacks while shoveling snow. Let this be a cautionary tale for anyone that has to deal with the white stuff. Your bpm can spike hugely without you realizing how much strain you're putting on your clunker.
I totally agree that we need more testing on these things. I've had the same issue with safety equipment; it's great that it's got pretty colors and nice foam, but if it's going to be 18% less likely to prevent me from getting a concussion, I care about that a whole lot more.
I think my favorite weird issue with fitness trackers has been that my Fitbit will report garbage but seemingly valid heart rate data...while sitting on my bedside. Or left in my pocket on a hanger. It seems like if the algorithm can pull data from literal white noise, that might not bode well for its accuracy heh.
Totally agree about safety equipment! Though I may leave testing that to the experts…
And good point about garbage data when not on…I’ve got a great day of garbage data from a time my Fitbit strap broke and it spent the day in my jeans pocket which I’ve been meaning to get around to looking at!
Thanks for the informative video. Which model treadmill are you using?
What a fantastic video Andrew, I found this while I was looking for reviews regarding the new Galaxy watch 5, since I just bought one and waiting for it to arrive, so I would love to see a similar test on newer watches, like the Galaxy watch 5, the new series 7 apple watch and a new Huawei watch if possible, thank you in advance and keep up the great videos bud.
I do a lot of crochet and while winding a rather large ball of yarn I realized my smartwatch was reading it as steps. I guess I got a lot of exercise that day.... I now take it off when I crochet or wind yarn.
This video was really useful, please keep making these awesome videos. thank you
This is amazing! Thank you for this!
Great video. I wear a Garmin Vivoactive 3 while running and have found the built-in HRM to be quite reliable for me. I've verified that over the past couple of weeks where I've been tracking my treadmill runs with a chest strap and the Garmin. The Garmin is usually within 1-2 beats at all times, even up to 160BPM. Don't see much sun in the Northern hemisphere where I'm located, so you could say that I'm pretty white. 🙂 In terms of the sleep tracking, it's total rubbish, but that's not important to me.
I gotta say, I 100% vouch for the underestimation of the fitbits heart rate monitor during high intensity workouts. I typically have a pretty high heart rate during exercise, and i can say with certainty that I’ve never seen my Fitbit actually track my peak heart rate accurately. It’s quite frustrating.
A problem I've had with step counters is similar to your water test - walking when pushing a shopping cart or pulling a disc golf cart. My Garmin watch seems to handle that a bit better than the fitbit.
I feel like price comparisons should be done, also would be interesting adding other brands to compare, specially Samsung and Garmin
It would be interesting to see a comparison between modern smart watches and Nintendo's Pokewalker (and its later iteration, the Wii U Fit Meter), which at the time was one of the most accurate pedometers. Also just comparing arm based watches to hip or leg based pedometers in general would be cool too.
I loved your insight on Steve's video!
Glad you enjoyed it!
Awesome video, I'll be waiting for the next part.
Amazing video!!! How were you able to download the data from your Apple Watch ? Also are any of apples algorithms public or all protected / hidden?
Apple watch saved my Grandmothers life. I advise anybody who's older or has CoPD, sleep apnea, etc to have one!!! Her oxygen was in the 70s and we had no idea....I grabbed my watch and put it on her... called an ambulance. She was in the hospital for 3 weeks... now she's on oxygen, has a CPAP and I got her her own apple watch to help track her oxygen levels.
biggest lies 🤥
@@AlexMkd1984 I’m her caretaker. It 100% is true. If I didn’t have that watch she could be dead. -_-
Low oxygen isn’t something you’d notice till it’s too late or you have a way to measure it.
She still has her Apple Watch 6. Which I got for her on EBay for $165.
It’s a life saver and well worth the $$.
Thanks Doctor ❤
I have an Amazfit watch, and you can set the frequency of tests, and that can help. I have a condition where my heart rate raises too much, so it's nice to know what affects it and what to be careful of. I do find the sleep one pretty inaccurate, but I move around a lot in my sleep.
There is a setting for Apple Watch heart rate, you have to switch the tracking on & check the app background refresh, plus there are a ton of extra things the heart tracking does that you didn’t mention.. like phoning your doctor if it thinks your having a heart attack, it’s not really comparable for those extra but I think even the cheap ones are worth having for basic health tracking.
Thank you that was a very informative video
For me I find that it is a great starting point to a fitness journey. I found within the 4 years of having a smartwatch I’ve managed to have more motivation to workout and move my body. Even if the accuracy is there or not 100% at times I’ve gotten into the habit of a better lifestyle.
6:35 I used a fitbit and samsung galaxy watch 3 in the past. The fitbit is very inaccurate if you jog up hill. I suspect they get very noisy data from the heat-rate monitor and use the step count to help compensate for it. If the heart-rate goes up but the step rate goes down they get confused. The samsung galaxy watch was a lot better in this but it often stops recording entirely because it doesn't get a well enough reading.
A follow up with Oura, Samsung Galaxy watch and Garmin would be amazing
I got my first fitness tracker because I was diagnosed with sleep apnea, but the CPAP wasn’t making me feel any better, so I wanted to measure my overnight oxygen saturation.
I got a Garmin watch, which at the time was the cheapest one I could find with that feature. It gave me some very alarmingly low results which sent me back to the doctor for another sleep study. The clinical pulseox machine showed the results from the watch were complete garbage, and had been stressing me out for no reason for months.
I now have a FitBit which doesn’t claim to measure an absolute O2 number, just variability, and seems to be far more consistent. (Not saying Fitbit is necessarily better than Garmin in general, as it was also a higher price point)
I still don’t know why I’m so tired. The Fitbit says I don’t get much deep sleep, but as you point out, this may not be very accurate.
Thank you for doing these kinds of tests. These devices affect people’s lives and health decisions, and we need to know how far to trust the results.
Sorry to hear about the low oxygen readings…but I’m afraid I think it’s quite typical of these devices. I didn’t test these in the video because it’s quite hard (I’d need to either go to high altitude or do a test on a plane to change my sats and, even worse, a lot of these watches don’t allow a one-off measurement, they only do it when you sleep or whatever…the reason being, spoiler alert, they’re not very accurate!)
They all seem to give readings that are too low for one reason or another, and in a way that’s obvious because most readings are 95-99% with a ceiling of 100% so any error is going to be in the downward direction! It’s not even clear based on my reading around that there’s much correlation between the readings and the actual values… Interesting, and glad the Fitbit is a bit more consistent. I hope they continue to improve. :)
I assumed that the step count was super accurate as there would be multiple ways to cross reference steps.
although I always wondered how they actually determined the steps from general movement, I always feel like just accelerometer alone would be super inaccurate.
I would have expected them to use as many sensors available and do something like below.
In theory you can:
- determine steps and steps per time interval from the accelerometer
- determine steps and steps per time interval from the gyroscope (most accelerometers these days are 6-axis or 9-axis so they do both accelerometer and gyroscope)
- Use GPS to determine distance traveled per step
- if the heart rate tracker picks up step as well get steps from that as well (which I'm not sure about but with heart rate detecting steps sometimes would suggest that the steps is some kind of noise in the heart rate sensor)
- then if you have a phone in your pocket you have another source of information for accelerometer, gyro and GPS
Then you can put that through a filter algorithm like a Kalman Filter (which all GPS units use already use to determine location) using both input data from the 7 inputs and prior state data to filter out noise in the signal. Which should be super easy for a device with as much processing power as a Smart Watch or a Smart Phone to do.
While all data will be not be accurate like GPS when you're on a treadmill or not carrying your phone those sensors would probably be excluded by the filter (with the downside of being slightly less accurate) when everything else is telling your device you are taking steps but the more sensors you can use the more noise the device should be able filter out making the steps determined more accurate.
I agree about reviews almost never checking accuracy.
Amazing content!
It's a pity you did not try a so called "sport watch", as well.
It would have been interesting to see how it compares with other devices during your run...
Very informative. It saddens me educational videos tend to get lower views vs garbage content that doesn't improves ones self grow.
Thank you very much.
Thanks for your sharing
That the Mi Band is missing out on the heartbeat on intense run part doesn't surprise me. I have been using a third-party tool for it from version 2 till 4 and it gave interesting insight. The sensor used sometimes misreads. For example if you have a stable heartbeat of 70 a reading of 160 or 40 once every 30 minutes can happen. Mi Band 3 (the first with heartbeat reading) didn't filter out these. With the Mi Band 4 they added support for continuous reading (once every minute) but they also filter out possible misreadings. This off-course has a side effect, a stable heartbeat below 48 in my sleep can't be registered, and the same for the extreme high values. But even though I still think it's the best value option
When I switched from a Fitbit to Apple Watch, I think I read that the Apple takes less frequent HR measurements to help with battery life. But I was glad for the improvement in step counts when driving. I was getting some crazy step counts and even calorie burn numbers while driving with my Fitbit which was really frustrating.
Thanks for the video, it was very informative!
Do you know of any smartwatch that monitors breathing? Like giving an alert when i start to breath very flat? Is this even possible without a breastband?
As a ecg/heart rate recorder while working out an H10 polar strap is the way to go. It's always good
Most of the problems with heart rate monitoring disappears during exercise if you set the proper exercise for the fitbit(in my experience)
Good video. Do u update them and will compair if some is more correct?
I wish you'd tested a Garmin. I have the Fenix 6x pro and subjectively it seems like it's impressively accurate. However a friend with a different model said it didn't do very well for them. Maybe it's a difference in hardware, or them having thicker wrists than I. Would be nice to see a video that tests more variables like these.
I just watched you over at steves channel, subscribed here. good luck on your channel, love the content
Quality content right here 👍 keep it up
You should include battery consumption. In most of portable/ wearable device, the infrequent measurement is made to conserve battery.
I have a Polar H10 chest strap, Apple watch series 6 and an Oura ring. The series 6 correlates strongly with with the Polar Re heart rate. The Oura not so much. They all use different algorithms for calorie burn. At the end of the day, I think the apple watch is the best bet when it comes to accuracy Vs convenience of wear.
Great work. I was a bit distracted in the conclusion with the no blinking and little movement. Teleprompter issue maybe
Many Smartwatches today use Gps to track distance and steps more accurately, BUT they usually only do that if a workout is started first. So that should also go into consideration.
Great video though!👍🏻
So, I got a Pixel watch and one thing that a lot of people don't like is that the battery life is low and that's typically attributed to the watch taking a measurement of your heart rate every second. I'd be curious to see how that data matches with other watches and see if it really helps get better measurements, making it worth the hit to battery life.
Also, I once went on a walk with my child, so my hands were on the stroller the whole time and my watch ended up tracking very few steps. So, I unintentionally did a similar experiment and learned that step tracking isn't the most accurate in some situations.
How did you export the diagrams of each watch and import it to a computer? Great video by the way!
Great video! Would be interesting to also compare the new Samsung watches to the Apple watch or chest strap, as well as some Huawei watches which claim good heart rate tracking.
Interesting - I just wrote firmware for some comparable device. The 'secret' sauce, or reason behind the difference, lays in the power management (frequency of measurements), smoothing / averaging of measurements. Normally there needs to be some averaging and low pass filtering to eliminate random values. Depending on how many measurements are taking to do the smoothing, will lead to how reactive the device is to changes. Higher power sample rates -> higher power consumption -> more accurate / responsive readings.
So its all an trade-off between power and how many different activities such a device can track.
Batteries only need to last a week if you are using is to track sleep. Otherwise you can just charge it at night so it would be good to have a high performance mode that was more accurate but only lasted say, 15 hours. The thing is I know that gui people don't like options and marketing people don't like it when you admit that your other mode isn't maximally accurate
Thats why i would choose a Garmin or Apple above the Chinese watches.
That was great content. Thx a lot
I recognise that outdoor location, that’s Windsor’s Long Walk 🙂
Lived in the town for 6 years, that’s a great location for walking/running, zero cars for hours 😎
It's cool you can see visibly with tech that you are getting healthier without having to pay attention to how you feel
This is really good experiment
Interesting vid. Would have been interesting to see the samsung Galaxy watch included on this.
I'd be interested to see a follow-up to this with a few different watches (I own a Withings Scanwatch, and I'd like to see how it compares to others).
Hello! What do you think if you test the Garmin Epix Gen 2 and the Apple watch Ultra for the same tests (Heart rate, sleep tracking, running, walking...etc). That would be very much appreciated. I've seen many people looking for tests for those two watches and especially the accuracy. Thank you!
I have a Fit Bit Charge and also use a Garmin. The former I wear all day and the latter mostly when I am exercising. There can be a wide variation between the two particularly with heart rate during exercise. Each has its pros and cons and the Fit Bit in particular has a few quicks. Neither is completely accurate so you have to use them as a guide monitoring your performance over a period. Also as long as they are consistently wrong in the same way over a period comparisons are useful. These devices certainly spur me on and encourage me to exercise more than I otherwise would.
My Fitbit charge 4 has good information on REM sleep I think because I can compare when I was dreaming to what the report says and it seems to be accurate most of the time
I have a Galaxy Watch and a Mi Band 6 and I use them for a rough estimation of my steps though the day and for registering workouts. It can be fun to see how often I exercise and how often I move more than average but I don't care much for the other "health" features.
I really don't understand why people are so interested in tracking their sleep, most aren't going to use that data for anything anyway. Also checking heart rate during the day is useless for most people. These are features the companies use for selling more products and apparently those features can be wildly inaccurate.
To me the best things about smartwatches is to be able to change watch faces and to see notifications with my phone on silent. Almost all other features are just "nice to have".
Would love to see these tests on the Oura ring :)
Great video! it would nice to see compression between garmin, galaxy and pixel watches too
Really would. I’m going to need a bigger budget. (And a longer arm…)
I think Pixel Watches should use the same tech as Fitbits :)
@@huawafabe they (might) use some of the same, but not all of it, at least according to the Fitbit website, and the range of functions each tracker/watch can and cannot do.
Would love to see this done with some other watches like the Garmin's.
Thanks! My Garmin watch is great for speed and distance on my racing bike but if it's
Interesting! I’ve seen temperature talked of as an issue (less blood flow near the surface of the skin when it’s cold) but I’ve never tested it! Only data point I’ve got is, it’s 0°C in Berlin at the moment and my Fitbit still seems to be working OK…
Informative video! Was going to mention The Quantified Scientist's channel, but see you're already familiar with him.
Can I ask where the filming was done for the green screen and your run? Looks like a good place for a walk.
Thanks! And yes, I’m a Quantified fan thanks to the research for this video! It’s the Long Walk in Windsor. Highly recommended, it’s lovely!
I see a bit of a methodology problem in your tests. I noticed that the accuracy of the measurement of my FitBit varies depends on where I wear it on my arm. so for more consistent Data for the three fitness-trackers/smart-watches, I think it would be relevant to perform those tests 3 times, each time moving the FT/SW in a different position on the same arm.
Very interesting video nonetheless. Shoutout to Steve Mould for making me discover your channel :D
Off-camera, I did try the watches in various different orders/on both my arms and I found it made surprisingly little difference! I’ve also done the max heart rate test with my Fitbit REALLY FIRMLY attached slightly higher up my arm as you’re meant to and…still nothing. More data are always good though! It would be very cool to do this systematically with a bunch of devices/testers/arm positions/etc…