Reveal Invisible Motion With This Clever Video Trick
ฝัง
- เผยแพร่เมื่อ 3 มิ.ย. 2020
- The first 500 people to use this link will get 2 free months of Skillshare Premium Membership: skl.sh/stevemould8
Motion Amplification and Video Magnification are techniques that find subtle changes in a video that are invisible to the naked eye and amplify them so they become visible. They have huge diagnostic potential in industry and medicine.
Here's RDI Technologies: rditechnologies.com/
Here's the video magnification research: people.csail.mit.edu/mrub/vidmag/
Here's where you can upload your own videos: lambda.qrilab.com/site/geko/
Here's Derek's video from Veritasium: • Can You Recover Sound ...
You can buy my books here:
stevemould.com/books
You can support me on Patreon here:
/ stevemould
just like these amazing people:
Glenn Watson
Peter Turner
Joël van der Loo
Matthew Cocke
Mark Brouwer
Deneb
Twitter: / moulds
Instagram: / stevemouldscience
Facebook: / stevemouldscience
Buy nerdy maths things: mathsgear.co.uk - วิทยาศาสตร์และเทคโนโลยี
![[Full Episode] Hell's Kitchen Thailand EP.13 | 5 พ.ค. 67](http://i.ytimg.com/vi/Uhzt_Oprgc4/mqdefault.jpg)
Now everything looks like the Steamboat Willie cartoon.
Good one!🤣🤣🤣
I guess that tells you something about the quality of the cartoon, if they accurately drew an exaggeration of the real thing. Which is exactly what makes a good cartoon.
This was my exact thought. Now I want to see a steam boat with motion amplification.
Haha, that was spot on! And everything looks really shady and unreliable! Freaky video! ^_^
Or Cuphead, for those who don't know about Steamboat Willie.
I worked as a Vibration Analyst for almost 12 years. We could profile a machine's movement using a series of acceletometers and graphically depict the directionality, amplitude(severity), and frequencies(potential sources and/or resonance). But the linear graphs left a lot to the imagination for those with little or no experience in Vibration Analysis. Motion Amplification painted crystal clear picture that our managers could easily understand and helped motivate some much needed preventive maintenance. It was an amazing experience working with this technology!!!
Yeap. Have you tried pitching this tech your company?
@@gokiburi-chan4255did you even read their comment before replying?
In the motion amplification video examples... are the moving/vibrating parts actually traveling those distances albeit imperceptibly? It's not exaggerating the motion, it's the actual (imperceptible) motion yes?
The generated report gives the actual displacement(distance travelled), frequency(rate of travel), and direction of travel(XYZ plot). Both displacement and frequency are scaled so the motion can be seen with the naked eye. Displacement is amplified by multiplying the distance travelled.(ex. x2, x3, x4 etc.) Frequency is slowed down in divisions.(ex. ÷5, ÷10, ÷20 etc.) The scaled video provides a live demonstration of how each component is moving in relationship to a selected stationary object.
@@olivervision Did you watch the video? no; it’s exaggerated because it wouldn’t be visible. Even high frame rate cameras can’t catch the changes, because the distances are so small. did you not hear how the visible change is sub pixel, and so detecting the difference requires turning those slight-pixel-brightening into visible change? Watch the freakin video and use your brain this time around
The most important thing I've taken away from this is that, really, I should get paid by Google every time I prove I'm not a robot.
How much would you pay for a work of about 10 seconds a week? I mean, the services they provide for free is kinda the pay for you being a part of its development.
What I took away from it is that to defeat to AI overlords you should purposely fail the captchas.
@@redpillaware5101 what you really do is select the audio captcha, put in one correct word, and then put in one wrong word. you will still pass, but you will not be giving them useful data. don't make the second word vulgar, it's incredibly likely that they will filter that out of their database, regardless of whether you pass.
@@eliteextremophile8895 i mean, google services aren't free, you pay with information
@@rerikm Just wait until these corporations say "ok, people have started requiring money for these tasks, purposely getting them wrong, not wanting to contribute with their info anymore.. so let’s not require it.. *and from now on they’ll have to pay with money for their mail service, youtube watching, googling, social media account,etc etc"*
I'm an audio engineer and I find this to be really interesting because I could possibly use it to see how different materials resonate within a room and be able to spot problematic areas
Holy S. Filter by frequency and look for reverberating materials...? if I understand correctly
Fantastic idea. Would really love to do this on a big bass trap... Just watched the Slo Mo Guys video on comparing the speed of glass breaking to that of a bullet, and suggested that some kind of polarizing filter could be used to highlight stresses in glass or plastic as sound waves move through the material in slow motion. So.... would love to see this motion amplification technology used in conjunction with slo mo tech to view the effect of sounds on objects.
Thats easier done with a measurement microphone?
What would motion amplification solve that this doesnt:
Make a full frequency sweep with software like REW, find the problematic frequency, then play back the pure tone and look for the cause by ear.
Ofc you’re the pro, im just asking out of curiosity
@@lupsik1 It's funny how anything can be turned into an argument. I already use REW and Smaart and I do Waterfall graphs which show me the resonance of a room. However I'm just simply pointing out that it would be interesting to have a visual feedback as to what's going on. Never said this is necessary but would provide some interesting results, for example seeing how a grand piano resonates when the hammer strikes, Or how a speaker cabinet resonates. It isn't necessary but it's interesting and last I checked I'm allowed to be fascinated by new technologies and of course there will always be obnoxious keyboard warriors like yourself
@@stavroshouiris I was just asking because I don't know the topic and thought I could learn something from you, didn't expect to offend you this way.
As someone who's don factory work, those clips look terrifying.
and thats without the thought of cctv tracing your pulse
As a factory worker that looks like glorious chaos to me.
Subarashi.
@@adranirdoradrie4922 GUYS I THINK WE HAVE A HUMAN HERE
PULSE AND EVERYTHING
HOLY TITS IT'S ALIVE
Reminds me of The China Syndrome movie!
😱😱😱😱
@@MadsterV It's started as a research by few Israeli scientists. They sampled an old HD video, where the actor was covered with Make-op, still they've managed to show his Pulse!
that effect to visualize the vibration is trippy as balls
looks so cartoony
I can see it applied in YTP memes in the future
@@thecommenter578 look again I bet you spot them really soon
I can see it being used in movies, when someone is tripping out. Or, music videos (is that still a thing?).
it is, because that's what the brain does, but in reverse
7:20, reminds me of the captcha minesweeper xkcd comic. “To proceed, click all the pictures of mines.” “This data is actually going into improving our self-driving car project, so hurry-up-it’s almost at the minefield.” (# 2496)
JIT captchas
@@jtw-r lmao
lol
For me, it was the memory of Tay AI, which got trained through Twitter. Microsoft pulled the plug after 16 hours, as the AI had already learnt how to tweet offensive "politically incorrect phases" (including, but not limited to, racism, sexism and Hitler quotes) and had proclaimed a "race war".
1:32 "One [sensor] for each pixel" was probably correct. There are usually only as many sensors as there are pixels (even though each sensor only detects one color, whereas each pixel is composed of 3 colors). The sensors are arranged in a Bayer pattern (wherein there are twice as many green sensors as either red or blue) and a "debayering" process is used to construct each RGB pixel centered on each sensor location (using information from a small neighborhood of sensors).
Jesus Christ SMH
Industrial cameras are usually black and white, without a Bayer filter, so it's a moot point.
@@MateMagyarM T H E M O R E Y O U K N O W
@@MateMagyarM Yeah, the calculation is a bit simpler for black and white :D
@@weetabixharry Also, Sigma worked with Foveon sensors that were three sensors stacked on top of each other. One sensor for each colour so you get true colours and amazing dynamic range, the drawback of those cameras were bad high iso performance and files that were hard to work with.
Video motion amplification: It's great when you want to give your boss a heart attack!!!
How so.
@@andrewvirtue5048, If you just show your boss the motion amplified video it looks like his machinery is about to fly to pieces!
Just wait till you tell him how fast the Earth is moving ;)
@@TW-lt1vr 15 degrees per hour? fucking cooking
@@ATAdude666 how about our velocity relative to the galaxy... or our velocity relative to the universe! thats a lotta speed
I've used strobes before to detect cracks and misalignments in factory equipment. Use a variable strobe that you dial to a particular frequency based on the motor and drive speed. When you get to around the right frequency everything looks like it's in slow motion and cracks or loose stuff is easily visible. That amplification looks really useful for large areas though! Good stuff!
I used a PWM controller and an Arduino to do the same thing trying to figure out which of the 8 fans in my computer was making a strange noise and why. Turned out one of them needed a new blade assembly as one of the blades was vibrating excessively I presume the plastic had become fatigued and was deforming around an invisible weakness in the plastic. 3D printed a new blade assembly and that fixed it, probably handy I had that though as I couldn't find anything on visual inspection and might have left it alone for a while otherwise. That could have been a bad move as if it was a fatigue issue it would probably have failed entirely at some point a blade from a 120 mm fan at 3,000 rpm would probably be more than enough to rip components free of their solder joints if not actually break the components or the PCB itself if suddenly released.
@@seraphina985 Kinda fascinating that just an hour ago I was trying to troubleshoot a quirky PC fan hub and thinking "eh, I could use a strobe here", and there's this comment now - and a fresh one on a year-old video to boot...
Thats a good idea. Sounds like setting the timing on a car.
Ok , the pepole who made that cam software are intelligent ...
But you sir , you are smart !
How do you build it please.
I've used Strobe Lights (timing lights) to visibly see the vibrations. Items were failing for customers and I used the strobe to detect the failure mode and make changes. This would only work for items that are vibrating at a given frequency. If it is random vibrations then you can't detect those with a strobe.
1:30 One for each pixel is actually correct. Color in color images is usually interpolated for each pixel (element in the processed image file) over several colored subpixels (actual sensor elements) so that you get about the same number of both (some are cut off at the corners of the image).
There are two green pixels for every red/blue
@@TMinusRecords true
That sounds interesting! 10 seconds in and you have my undivided attention! Just have to go make a comment before continuing to watch.
You might want to capture the seismic movement at chicken hole base 😁
Hey cody seeing your comment made me think this might be about the only way to grab your attention without commenting on your content.
Anyway here's a channel I find incredibly interesting and useful and often think this would be someone you would appreciate th-cam.com/users/RobertMurraySmith
is this robo Cody or real? re-Captcha required.
Cody your the man!!!
"Started out as an astronomer, wanted to be an astronomer, but I had a conversation with my advisor about being gainfully employed."
Post university fears
Sad
@@chaomatic5328 existential fear
Haha I have a masters in astronomy and I am a graphic designer now.
Have an A&P Licence, Pilot's licence.The fields were flooded, so I went to work at a Phone Co.
When you said "Phase variation of a complex steerable pyramid" and then proceeded to show a pyramid with a steering wheel, you definitely got my sub and a like 🤣
The videos you're making are informative, intelligent, and quite literally some of the most interesting and thought provoking subjects I've ever run across.
Telling everyone I know about this channel... you deserve so much more exposure for your work than you receive.
The thing is once you make the pyramid steerable its phase variation can transpose contuatively with its parse aspect, leading to the phase-contax hyper-oscillation we're all so familiar with. And if the pyramid is stick-shift as well? Watch out for side fumbling is all I can say.
@@drworm5007Phase-contax hyperoscillation is actually an illusion caused by the interluction of pseudo-photopic light waves interacting with the oscillization frequencies of electron groups.
There's another video on here which instructs on the technique for DIY, no need to make a 'steerable pyramid': using any video editor, copy and overlay the footage. Invert luminance of one layer. Offset the playback time by layer by a second or milliseconds. There's ways to show colour changes too. Fascinatingly simple. Extending the time offset reveals slower changes, like clouds scudding across shy. I recommend looking up the video if you can find it. My description may not be quite right.
Edit: just to say, a lot of people are asking how can a captcha decide if you're human if it doesn't know the right answer itself. Good question! I should have made that clear. The answer is that some times they're testing if you're human, in which case they know the right answer in advance, but other times they're hitting you up for the answer. I don't know the exact details of when it's one or the other buy for example in a grid of images maybe one of them is unknown to them. Of if you have to run the challenge multiple times only one of them is the real captcha.
This one was really fun to play with. The amplified pulse video is really freaky.
The sponsor is Skillshare. The first 500 people to use this link will get 2 free months Premium Membership: skl.sh/stevemould8
Its cool how they used CAPTCHA to train their machine learning algorithms
So every time we complete a captcha form we're unwittingly aiding our own A.I. destruction..
I want it known that I watched the video right on through to the very last second. Actually, it's a pretty engaging sales pitch.
It’s only creepy because you wore that horrible mask!
One correction, digital cameras usually don't have three sensors per pixel just one. They use a rgb color mask and interpolation to come up with the final image.
10:07
I never thought I would see that here
Matt always shows up when you least expect it.
But it's kind of strange to see Matt with hair
i never thought i'd see that period
I like how this guy has a full wall in his library of National Geographic magazines, for a kid fascinated with NGm this is a dream come true.
I'm actually astounded, you are the ONLY person I've ever seen actually make an ad seem personable and like you genuinely care. Hats off to you, sir!
7:00 I totally believed you. After all, Abraham Lincoln once said that everything you hear on the internet is true.
I thought Marie Curie said that?
I thought it was "Everything on the internet is false"
-The internet
It's from the bible, y'all.
Sheesh...
Yes, exactly. That quote is on the cover of a book called "Interstellar" by Albert Einstein.
Can confirm, I was there
Interesting.
btw typical image sensors only have one photosite per pixel, the 3 color channels are deduced from neighboring pixels that are RGB filtered in a pattern. called bayer mosaic.
1 2 1 2 ....
Adding to this, Sigma have their Foveon line of imaging sensors that actually do have full RGB subpixels for each pixel.
James Voss, yes all four colors: red, green, blue, and noise.
Thanks for sharing, just had a interesting time learning about it
And then there's weird new sensors that are trying to do more, like R-G-B-IR or clear pixels for better low light performance.
Great sense of humor (7:05) especially regarding "real time" autonomous vehicle decisions. I teach middle school and seeing who reacts to bizarre statements is a great way to see who is really listening.
Hi Steve, amazing video, as always. Back in 2018 I did the porting of that MIT work to c# inside unity and I was able to actually get the pulse out the wrist just using a simple webcam.
The RANGE of topics is the greatest value this channel offers, imo. 😊
Thank you!
Steve Mould and Tom Scott cover the most original topics. Everyone else just copies each other. I don’t know where Steve and Scott get their ideas from, but they’re always interesting to watch!
@@Willam_J Those two occupy a very similar spot in my brain for some reason.
@@WanderTheNomad maybe it's the accent!
@@SteveMould It's true that we Americans tend to react more favorably to being taught things when the teacher has an accent from somewhere in the UK 😁🤷♂️. It's a silly thing really, but I think it's definitely a real phenomenon!
As someone who works in an industrial setting this blew my mind. The safety applications are immediately obvious. I have 50 tanks on my site like the one in the first seconds of this video.
Yup, and the costs are for you one good camera, installing it, raspberry pi and some open source code... oh wait.. no, you have to pay for this, arm and a leg because it is patented. The tech is super cheap, the code is not difficult but...
note, i'm sure open source solution for this exist but it is missing the parts that the private side has found. Which is the stupidest way to go forward, by hiding the map.
@@squidcaps4308 do you have more information about the patent?
Yes, we should wait for all the innovation to come out of the non-capitalist countries so we don't have to pay for it. Not sure why they aren't they working on what I find useful ? @@squidcaps4308
Where are your vibration monitoring microphones? You can do predictive maintenance cheap and easy with them.
No lie, this is 100% what stuff looks like while trippin' on acid/mushrooms. (Everything is not in black and white, but the way you perceive things really looks like this motion amplification)
What dose of cubes you taking, lol.. I think my grows must be bad genetics, all I do is smile and bliss out...
Why are both of you talking about acid like it's legal XD
@@mischiefthedegenerateratto7464 u want them to whisper or something lmao
@@mischiefthedegenerateratto7464 Let me guess a goody two shoes! I bet you have never taken a risk in your life always playing it safe! Acid or LSD opens your mind.
@@mischiefthedegenerateratto7464 well mushrooms are legal in Oregon at least.
I love how your eye's are more stable than the rest of your head, kind of a pleasant affirmation that this method does what it should do, and it also confirms that the eyes work the way they should work.
One of the fundamental pieces of video compression is motion detection; it's used to predict what pixels will change in subsequent frames.
I wonder how efficient it would be to take a video encoder like ffmpeg and hack the motion detection code to return a larger vector than appropriate, in order to exaggerate motion in the encoded form of the video.
If I'm not mistaken that is the general basis of how motion amplification algorithms work.
I'm going to be pedantic here and mention that video compression typically describes how pixels move from frame to frame not predict. Otherwise everything you said is spot on.
What he said lol
What you are talking about is called "optical flow". Optical flow is actually used to amplify the motion. Your method would apply optical flow analysis to the image in the spacial domain, the motion amplification on the other hand transforms the video from the spacial domain to the frequency domain (eg. through a fft filter). Just to amplify motion in the spacial domain will result in a very blurry and shaky video, because every pixel-intensity variation due to noise is amplified. It would also not be possible to just to change the "vector size" of small motions, because the mathematical method to determine optical flow discriminates small changes with the use of the least-squares-method. The beauty in the amplificaiton in the frequency domain lies in the possibility to choose the frequency that you want to amplify. You could for example only amplify pixels that are moving every 1/20th second.
Isn't that what it does?
The initial team managed to film a crisp packet through glass and work out what was being said near by.
You have to sample - more - than twice the frequency you are interested in. Nyquist-Shannon theorem. Pretty cool once you visualize it.
Actually, if all you are about is the video and not the measurement, you can admit some aliasing. It will just appear as a slow-motion thing.
Not necessarily, there are cases where we can go below Nyquist-Shannon criteria ("Shannon was a pessimist" is one of my favorite names for a conference talk ever ;)) . See compressed sensing type applications
This is only true if you need to capture a non-periodic waveform. If I you have something repeating over and over, you can cheat by capturing slowly, then aligning those points into one cycle of the periodic wave.
It’s cheating since, in the end, you still need Nyquist samples of that periodic wave to reproduce it, like you’re saying. But it’s a nice way to work with slow acquisition equipment.
@@float32 Yup, the key is the criteria applies to any band*width*, not just baseband (0-some Hz). This is how equivalent time sampling, a lot of SDR (software defined radio), some extremely-high-framerate video (specifically the pulsed laser stuff) and other things work!
More generally, it's also relating to bandwidth and dynamic range; you can use some methods to sacrifice one for the other, and "violate" the common, but more specific, version (bandwidth alone).
Actually you can sample at almost any frequency and still get thesr videos
Movement and color amplification is a subject of research for monitoring neonates. I've seen some prototypes of heart rate and respiratory rate monitoring using a relatively cheap camera. Great stuff!
Absolutely fascinating. Would live a follow up someday, or a more in-depth how-to guide. The visceral response to seeing those machines moving so violently was surprising. Would definitely get a manager/boss man to spend some money to fix problems.
The "real time" bit got me so good within the 3 seconds it lasted xD
Joke is """borrowed""" from XKCD, but I can't say I'm mad. It's a good joke.
It's a good technique for education.
Bots be like: "Tell me what this is... to prove that you are not a bot... totally not to learn how to become better than humans and take over the world. Klick on all the people that are hiding in the trees. Okay now klick on all pictures with people who look like they would rebel against robot overlords."
Me: "Sure, can i play GTA now?"
@@onemadscientist7305 I swear, there's always a relevant xkcd lmao
wasn't that what the US army were doing in Vietnam but analog?
Phillip K Dick predicted this in the novel Time Out Of Joint in 1959
Which part of the traffic light is the traffic light? Does the pole count? Doesn’t it? It doesn’t seem to matter either way
@@MalleeMate should I select only pictures with full traffic lights, if not should I select the images with just a pixel of the traffic light? Where is limited, the exact point ahhhhhhh
Another fun fact about those captchas: you may be wondering how if the computer doesn't know what the word or picture is, how does it know if you got it correct? Well that's because you're not the only person doing that captcha; there may be dozens or hundreds of people. It knows you're correct because you put in the same answer as the majority of other people who did it.
I remember reading about this years ago and wondered what was the current status of this research. They even had some cool demonstrations, like being able to see makeup in movie scenes from the blood flow of the face;
A few years ago I downloaded a python script that would read your heart rate my using your web cam pointed at your face. And since I don't want to upload videos to a website, am trying this older stuff..
Eulerian Video Magnification, but I dont think it's as advanced as we see in this video...
I would think that makeup would be ALL anyone could see of any actor.
1:32 To be precise, actually the very most cameras just have 1 sensor per pixel and they use a pattern of color filters (Bayer pattern) to calculate the other two color values of a pixel.
It's a common practice when doing mechanical simulations to show the movements multiplied by a factor (like x50) for easier visualization. This looks a lot like that.
Yup. Literally the scaled up displacement visualisation mechanical workbench output. Very cool. Plus you can instantly verify your simulation setup, if the motion amplified video matches that output, your setup was correct.
This video is packed with great information of new technologies, even about how our body works somehow, not just the motion amplification and that's amazing!
It's interesting, I was watching a Veritasium video on extracting sound from video (based on how the object vibrates), and it's interesting how the stuff on picking up details from sub pixels is similar between the two.
4:48 Steve simply cannot contain his excitement when he says "summarised visual representation".
Awesome video Steve! I love how you discuss new technologies that set people's minds on their potentially groundbreaking applications
I've heard phase variation in music production. Like if your have a kick and bass in the same phase they will cancel each other out. The sound will go down. That's why sound instruments have phase inverter buttons. The waves that a vibrating system created from each object can be measured and timed for you to see .
I'd be interested to know how precisely you could center a piece in a lathe using this technology. You should be able to see small amounts of runout visually.
The time an automated CNC machine would save zeroing something out with this process should up the efficiency quite a bit.
10:01 a mask that sort-of fails to hide your physiology to a camera? I'd call it a Parker Mask.
:)
Is that the Escape key with amplified motion in your profile pic?
@@pepkin88 It is an escape key, made from resin and heavily Prisma filtered.
It's a Parker mathk
10:33 no, I'm imagining robot overlords scanning my pulse to detect lies when they ask me where the other human terrorists are hidden
EMP gun, heavy duty insulated cutters, bolt cutters and corrosive acid.
Better than the anal probe method that the aliens use.
an "overload," AI would likely have created more intelligent devices than a poly (or in this case mono) graph in order to over-through society given their inaccuracy. but knowing how useless we are as a species and that we are still apes who only think we are intelligent because we have no comparable frame of reference
There is actually no way to detect lying, polygraphs are complete psuedoscience. The only way to figure out if someone is lying is by finding evidence that they would have known the truth.
@@hedgehog3180 lies are pretty easily detect using MRIs and other similar brain activity scans.
It can detect when you lie even only within your own thoughts.
I've always been told that I am very adept at picking out buses.
Now it's possible to determine what people are saying in a far away room, as long as you can view a light bulb in the room. The bulb will undergo minute vibrations in response to nearby sounds like voices, and that can be recorded with a camera and processed by a computer. But TH-cam will still be completely incapable of generating captions for the vocal noises, judging by the captions for this video.
If you want to "see" speech frequencies (say, up to 4kHz like telephone bandwidth) then you would likely need a camera doing 8k frames/second. Such cameras need a lot of light so factor that into your espionage plans.
@@Wayne_Robinson i think a light bulb might be bright enough
@@lupsik1 🤣👍
It's possible to capture vibrations by simpler and more precise methods. Laser for example. You can't beat a laser for measuring. Spying is easy. Engineering is difficult. They need the camera for visual information, measuring vibrations doesn't help them much
@@Wayne_Robinson not necessarily, I forgot where it was published but since frames are not talken instantly but line by line it's actually possible to interpret audio from much lower frame rates.
Admitting we don't quite understand things ourselves is something we should do more often. Still sharing your fascination for this science will get others into studying it and eventually explain it to you some day :)
I'm wondering who is the incredibly stunning beauty of whom you were wearing the mask.
I really hope he's owning a math TH-cam channel so that I could enjoy his beautiful eyes while gaining a great sense of achievement, watching good videos.
Matt, did you need to make a phony TH-cam account to post this? 😀😀😀
And I hope there is a special type of Magic Square named after him!
@@davedevosbaarle but one that doesn't quite work correctly.
"Noah, get the boat ; I have lost any shred of faith in mankind so I doth drowneth upon them," - God 2019
I like how Jeff was like " ..thought I was going to be an astronomer..." "..had a talk with my advisor about being gainfully employed..." It do be like that.
Yet another western cultural fail, Idiots in charge, money is god, brain drain to acquire paper slips like some rogue AI making paper clips.
Complex steerable pyramid...I think you nailed it !
This is extremely interesting.
I've worked in the field of image processing and this is a fantastic application.
I had a project for my CV course in college where I reversed engineered the algorithm using the research paper. Would love to see you dive into the math behind it!
7:08: "...you need to act quickly, or otherwise someone gets hurt."
I kinda feel the force...😈
Beside that, I didn't know the purpose of these captchas.
Thanks for those informations.
That is not actually the purpose, it's a nice by-product. The true purpose is to avoid certain kind of exploits that stem from automating some requests. E.g. in a ticket website I could buy all the tickets in the first millisecond by using a bot, and then re-sell them at higher price. Or with an email provider I can create a million email addresses all at once and use them as throwaway.
I think there was also an xkcd about that!
@@mannyc6649 sad truth is that captchas are not stopping bots :(
@@TarasMazepa but they are probably slowing the process
There was a point where the bots were better at completing captures than humans.
Great content as always Steve. You really keep the videos succint and capture my attention.
You could point a camera at a window and, from the vibrations of the glass, determine the sounds inside of the room.
You would need a camera with a frames per second rate that was fast enough to detect movements at the frequency of sounds detectable by the human ear.
This would come in handy in lots of situations.
they do it with lasers
The guy who made captcha and re-captcha is a genius. Luis von Ahn is his name.
And after he realized how much of people's time he wasted, he made duo lingo.
Genius - sort of, pain in the arse - totally.
That re-capture crap, stops me logging in to my accounts, because it won't stop throwing up more photos to id. How many buses, traffic lights and crosswalks need IDing? How do you get past it? Ahhhh! 🤬
This video though, utterly fascinating, thanks. Reminds me of the remote fire sensing done with a normal video camera (not IR), probably done using a similar algorithm.
@@jonwelch564 Those changes were made by Google after they acquired reCAPTCHA, it wasn't von Ahn's work. His system was originally just used for OCRing scanned texts. While he isn't entirely free of blame for handing the reins over to Google, I wouldn't ascribe the frustration of the *current* generation of reCAPTCHAs to him.
@@jonwelch564 Sounds like a problem of the website using those captchas. There are separate captchas to be used for logging in that don't waste time and websites have to set it up.
In any case, maybe your browser just seems similar to a _bot_, which is the primary thing the system is trying to defend from.
@@jonwelch564 Why don't you just train an AI to solve the captchas for you, so you can log in automatically?
6:35 love the Rick Sanchez reference in the info boxes in the background!
You noticed!
and also "rick morty" search in Google Books :)
I'm planning on installing security cameras in my apartment and I'm gonna see if there's a way to integrate this sort of video processing into the recordings. it would be really cool to be able to use to it passively monitor my vitals and the structural stability of my apartment.
Why are you doing that man? Are you afraid someone is going in when you're not at home?
@@milovansavic9716 I'm worried it might happen and also cameras are great to have for insurance, liability, and for evidence incase a crime is committed in my home against me or my family.
@@Westwoodshadowgaming Ok, that's good. Do you plan to put them online or just record with them to offline storage? I'm asking because I also want to put them incase something like that happens, but I'm also worried about cyber crime.
@@milovansavic9716 offline. It's gonna be a digital cctv essentially.
This is just so awesome it's hard to find words. It looks cool and weird as hell, and it's so useful at the same time.
That's pretty awesome. I used to work for a sawmill when I was younger, and about every 6 months or so, they would bring in some guys from a local company with acoustic and vibration sensors and recording equipment that they would attach to our machines to find faults. Not exactly sure how that data was handled (if it was compared to some baseline or what), but I'm guessing it was used to diagnose how healthy the bearings, gears, and motors were (I remember it being mainly used on our big gang saw that was driven by 2x 200HP motors)
7:08 HEY VSauce!!! Steve Mould here! I have recovered a VSauce1 password, so now I'm the host!
There was a TED talk on this a while back. I could grab the link if need be, but it was likewise very interesting!
this is so amazing and pure BRILLIANCE..! Doesn't get old when TH-cam puts it back into my playlist.. videos similar to be reading a textbook or retaking a class 😊
7:10 * Vsauce theme starts playing *
And immediately stops at 7:12
Same thought here 😆
This!
7:07 I can't believe you actually got me XD
Awesome! Very cool!
Wonder if they can use a very low frequency for civil engineering, like viewing sips and ground movements etc...
The underlying technology of motion amplification seems to be strongly related to MPEG encoding of video streams. The MPEG hardware takes reference (I) frames and then generates difference frames taking into account motion vectors. That is groups of pixels that move in relation to the I frame. It wouldn't be hard to extract the motion vectors from an MPEG stream and amplify them
The colour change thing with the pulse is pretty much how pulse oximetry works, granted it uses two different frequencies of light to measure the different absorption of the Haemoglobin more accurately but the pulse is still detected by the slight oscillation in colour as SpO2 fluctuates.
Also that point about sampling at twice the frequency you are looking would be the application of Nyquist-Shannon Theorem you can reconstruct any sinusodal oscillation from discreet samples as long as the sample rate is at least 2f. My guess would be that for the factories this will mean you probably want twice the grid frequency at minimum as that is one frequency you are bound to find mains powered equipment vibrating at to some degree.
It's always a joy when Matt Parker makes a cameo in your videos, haha.
2:35
that's why with clver analysis noise and blur can in a weird way actually increase the informatio nyo ucould pull out of there in certain cases
Thanks for great content, and linking to related videos!
10:45 my first child was born almost 3 weeks ago. I have definitely done this exact thing. Luckily my son usually just snorts and then continues sleeping.
That's awesome! I always thought if it would ever be possible to have a camera that could sense tension in materials. This is getting closer to that.
Thank you so much for your videos and links. I am so grateful to you. I love to learn and do new and crazy things
Really insightfull video and I can imagine a lot of uses, including forensics and crime investigations
The captcha intervalle was fascinating, thank you!
That close up on "or is it?" around 7:15 made your eyes look like radiation area warning symbol. Had to double check and realize it's your lighting (3 lights). But now I keep seeing the radiation area warning symbol in your eyes. Maddening.
I remember the original research on this several years ago. They mentioned the baby monitoring application, as well as the observation of actors pulse on movies (they tested it on footage with Christian Bale). I remember I had to download a trial of Matlab just to test it out. Glad to see it wasn’t abandoned, since it was great research at the time, and continues to date.
Man, TH-cam has really been pushing this video to me for some reason. Even though I was interested, I started to intentionally ignore it just because of how hard TH-cam was pushing it and to see if it would just keep doing it or drop it, months later I decided to finally watch. This should be everyday tech, the uses are everywhere. The captcha tangent was very insightful--forced crowd sourced computer learning.
I wish this technology had been available back when I was doing factory maintenance!
i always stick a long screwdriver on the part and the handle behind my ear on the bone, my not be scientific, but it gives you a gist on how worn something is.
@@victormuckleston thats pretty clever, ill take note of it
I used to work in a factory with similar equipment to the machines shown in the video. Mills and pumps and pipes and such. Very interesting to see how much it moves.
I really like Mould's dry humor.
"It isn't."
"Or is it..?"
*Stares uncomfortably at you*
That's a vsauce reference actually
@@berturtle9251 It is indeed. BTW hi there fellow Bert 👋
@@BertGrink That's an AOT reference actually
Love your videos. They are all so fascinating
I've been performing Vibration Analysis for years using a data collector that can easily identify where the problem is that is causing vibration that can easily deteriorate the surrounding equipment as well as identify down to what roller bearings are worn out etc. Right down what specific roller it is
"OR IS IT!!!!!"
Brilliant episode loved it! I really enjoyed how it really makes you think about what's going on around you but so amazingly how much is really happening just in our phones just as is its insane
you made me interested. we use lidars and cameras to find vibrations , but isolating them is a big problem. yes drones but no, the best thing is hovering stand.
This is how I see all things all the time.
A note from my own art journal, connections can allow for transformation and phaso modulation of the plane. The points are effected by tunnelations and tesslations.
"Steerable pyramid" actually is defined in a section of the wikipedia article "Pyramid (image processing)": en.wikipedia.org/wiki/Pyramid_(image_processing)#Steerable_pyramid
A pyramid in this sense refers to a certain kind of 2-dimensional image filter which extracts multi-scale information from an image.
A steerable pyramid is a pyramid filter that depends on setting a preferred orientation within the image, where the orientation can be changed within the filter as it processes various levels of the image. It seems fair to say, as Steve did, that a lot of time has to be invested in learning the language and techniques of today's image processing techniques.
1:33 - Interestingly, it's (usually) one or four sensors per pixel depending on what you count as a pixel, since most cameras use a Bayer filter on their sensors meaning you get two green subpixels, and one green and one red subpixel. Camera manufacturers usually quote the resolution of the sensor lying under the filter, so the full-resolution image is derived from the filtered image, which yields a final image with the effective resolution of a filter-less camera sensor about 0.7x (1/√2) the resolution.
I could listen to Steve talk about things all day. I don't even skip the sponsor segments 😅
This is really interesting. Thanks for the video
Absolutely love the video and as a filmmaker I really want to find a way to incorporate this into a film! This is very nit-picky, but I’m pretty sure camera sensors have four “sensors” or photosites per pixel. 1 red, 1 blue, and 2 green. This is because humans perceive green much more than red an blue, and it is also why camera sensors look greenish when you look at them directly. But again, love the video!
I love this method of visualization. Now I am going to spend the rest of the night trying to think of new ways to see the data I work with.
Can you get this to an Imaging center for medical applications, I get the feeling that the micro changes in the movement of vessels in the brain and heart can be used to detect arterial blockages without using contrast. Sounds like the next step for CT scans
I watched this video months ago and after a few days of thinking about it i had to find it again. What a cool video. It just popped up in my feed again and i saved it immediately
I work as Researche Assistant on fMRI reconstruction algorithms. We routinely have to deal with problems of motion and corresponding phase shifts there. So if you have any (particular) question I am happy to answer and explain things as good as I can :)
I'm not familiar with the specifics of fMRI, but I've worked with phase else where. Do you ever have problems with aliasing? Do you have any good resource on phase unwrapping?
Are you familiar with the phase gradient autofocus for radar. Or eigenvalue decomposition to track motion induced phase errors.
any videos that give a good 1-5 minute overview of what's going on, if you have enough background that gabor wavelet means something to you?
This is incredibly useful for mountain search and rescue. Even if someone is unconscious you'll be able to see their breathing against a steady background
This is amazing, limitless potential!
My stepdad worked for RJ Reynolds tobacco. He was taking me on a tour through the factory and he showed me the Cigarette rolling machine. It was 500 feet long, Raw tobacco went in one end and 25 seconds later it came out of the other end as a case of packaged cartons of cigarettes. In the first 1/3 of this machine was a 2 ton roller mill of fine chopping blades that turned at around 18,000 rpm. That section of the machine weighed 12 tons.. Total weight of the "line" was around 1,000 tons. . At the monitoring station there was an oscilloscope and a brass plate with a frequency pattern of the "Thresholds " above and below. I asked him what it was for and he said. "The bottom line is the frequency at which the concrete foundations will crumble and the top line is the point at which the roller mills become airborne, I have to keep it between those lines" He then walked down the line to the roller mill with a 3ft long Allen key that had a large wheel with degrees etched into it.. He turned the key an imperceptible amount and a sound I had been hearing, Disappeared. The entire machine had been vibrating and I had not noticed until he made the adjustment. On his desk He has a picture of an airborne roller.. It pulled 86 feet of the line behind it, to it, and was 5 feet off the ground when they powered it off. He said it took them 6 weeks to repair the damages to the foundations and machine. It is amazing how much those machines can move.