ไม่สามารถเล่นวิดีโอนี้
ขออภัยในความไม่สะดวก
Synthetic Sensors: Towards General-Purpose Sensing
ฝัง
- เผยแพร่เมื่อ 8 พ.ค. 2017
- Project Page: gierad.com/projects/supersensor/
We explore the notion of general-purpose sensing, wherein a single, highly capable sensor can indirectly monitor a large context, without direct instrumentation of objects. Further, through what we call Synthetic Sensors, we can virtualize raw sensor data into actionable feeds, while simultaneously mitigating immediate privacy issues. We deployed our system across many months and environments, the results of which show the versatility, accuracy and potential of our approach.
This is HUGE. I downloaded the research paper I was so excited. You are on the verge of a breakthrough in sensor based automation.
I need this in my life right now... Please keep developing this and bring it as a consumer product!
This is very cool!
I could imagine having multiple of these collaborating to increase the confidence level of event identification...
Great work !
Combine this with a personal assistant A.I. & WOW the possibilities are endless. I want, I want, I want.
The video is so easy to understand. Very well produced, and super idea.
Thank you for the kind words, Lois!
Really impressive! It's gonna be a huge improvement in the tech community with this little object :D
Very rare that I comment on youtube but this is worth the exception. Incredible work Gierad.
Wow, we are honored. Thank you! : )
This is absolutely incredible work, guys! All of us over at r/homeautomation are rooting for you!
Love it. Shouldn't be hard to solve ambience issues (living by a freeway, or in a city), but those Nth sensors are awesome. Detecting the ways device signatures change would be sweet: If the condenser in your fridge is running more often after a year of use, filters in your HVAC need replacing, all kinds of cool things you could do with that.
Awesome, congrats Particle too et al., been a fan since your kick starter:)
I'm only just coming across this in April, 2020. Wow! This is awesome. The greater implications are a little scary, but this... this is cool.
That is like creating life itself. Sensors feed data into the developed neural network which makes choices. Brilliant.
Wow. Interesting approach!
This is Brilliant Work!!
I've never seen something this great before, big thumbs up for you guys, now this inspires me more into AI...
Great project !
This is a game changer!!!!!
Having an IR blaster on board would give a lot of flexibility with automation output. Also easy for vendors to add without needing full internet hardware, software stack and associated vulnerabilities that are now being exploited.
Oh my god this is so cool!
very impressive!
Amazing. Truly. When will you release this ?
good idea!!!
Great, now where can I buy one?
So, when will I be able to get these on Kickstarter? ;)
amazing
I wonder if I can use them in a commercial building to see if the boilers burners kick on, pumps are kicking on, sense hot water pipe temp, boiler flue temp, water leak, unusual mechanical noises etc.
Great work guys. Let me know if you need any beta testing done in an Australian environment. :)
great video and impressive tech! the bg music is too loud.
I think the music adds character to the video :)
Does anyone know what are they using to plot the data? I mean language and if they are using OpenGL or how they are drawing the spectrum?
We used Processing, which is a Java based framework. You can download it at www.processing.org
Thank you very much Gierad :)
planning sell the product?
Where can we purchase the sensor and software?
Where can I get some of these! It would be really interesting to use these inline with generative AI!
Dear Gierald, Congratulation per innovation product, do you have samples for testing?
Cool project. But what are the possible applications of the sensing device? How can we use this data?
@gierard Laput We have done quite accurate prediction of meeting room usage by developing an algorithm that uses CO2 and noise data collected by our sensors. Please let me know if you are interested to dicuss and share experiences.
Awesome! Where can I buy this super sensor?
Artificial intelligence, here we go, can't wait to see what's coming in next years, we shoul dknow a great boom into innovation.
This is totally cool tech. Is this in market for purchase?
Very impressive, but I get the impression, the soundsensor (and maybe a heat sensor) is the only one used/most important one. Can it sense writing in a blackboard and erasing it? I really have strong doubts about that or at least doubt the accurateness of that. But maybe I am wrong
In a nutshell, data from acoustics, vibrations, EM noise, temperature, light color, and thermal image are able to capture a wide variety of events. But for some events, you need other sensors (e.g., magnetometer). It depends on what you want. "Fast" events (seconds-scale) are best suited for "fast sensors," while slow events (e.g., hours to days) are best suited for "slow" sensors (e.g., temperature). Our paper contains a table that describes "sensor contributions." You might find it useful!
Gierad Laput will have a further look
This is quite brilliant and elegant. I'm surprised the multiclass SVMs worked so well -- is it really that easy to train? Will the code or data be available? The demo in the video is very smooth. Also, what's the song playing in the background? It's great. Shazam fails to identify it.
Thank you! SVMs work well for this application, but other approaches are also possible (e.g., we are looking at deep learning, which would make this system even better). The background music is called "Imagination in Action" by Olive Musique. Licensed from PremiumBeats.
thanks for the reply! I also wondered (I should really be digging through the paper) how much of the activity classification was being done with many sensors vs a just few (one or two). For example, a lot of the activities can be classified through the acoustic signature -- straight-forward, labeled training on that alone could probably give you decent performance. The fundamental question being, how much more information do you gain from each sensor for different activity classes? Thanks again!
[Don't bother answering. I read your response below in the comments :-) ]
You get a lot of useful correlation from other sensor channels, and it generally helps improve robustness. It also depends on the event. For example, a microwave running triggers the accelerometer (it vibrates), mic (it makes a humming sound), and the EMI (it radiates electromagnetic noise). You can get rid of 2 / 3 of these channels and still get good accuracy, but having 3 / 3 is certainly better. Hope this helps.
Thanks!
Please tell me if this work on nfc tag just tell me thank you I am studying about Bluetooth a lot now please tell me
No
Sound sensor most useful...what’s potentially creepy is you could figure out what someone is writing based on how long each marker/pen stroke takes with accurate enough timing and knowing cadence of the person’s writing pace with a good enough clock and sensitive microphone...but this is awesome as the levels go up and is combined with AI applications...you could probably figure out what is being put in the microwave based on sound...a mug with different fluids will make different sounds...this is definitely the future...
Seems like sound is the primary sensor. How does this deal with multiple sounds and background noise?
Nice Work.. ? Would you tell us which sensor are you using for this. ??
We are using a combination of different sensors (accelerometer, microphone, EMI, light, color, humidity, temp, magnetometer, directional heat), fused into a single package. Details: gierad.com/projects/supersensor/
When will it be available? I'd buy it if the price is right ;)
Hello!!! I certainly admire, the coolest in the algorithms of recognition events of course, but I as an engineer of security systems (and not only:) would like to ask why if You have AMG8833 to put more and PIR. Well, here is the moment where when you turn on the water, the signal coming from the accelerometer, is it so the pipes make a noise?)
I suspect a lot of this video is bullshit. How do you use a remote accelerometer to detect water from a faucet?
Amazing work!. What are you using to visualize the data?
Thank you. We're using Java to visualize the data
Thanks for the answer
Verbale Interface is the next step.
Speech recognition is one of the hardest problems in AI, that's why TH-cam's subtitles gets it entirely wrong at times.
This is very impressive technology, but what are the applied uses for this sensory data?
Who cares when my fridge is open or when my microwave is on? I mean, I already know that, so what benefit do I gain when a sensor is tracking this data? What's the end goal ?
I'm genuinely asking because I am obsessed with sensors and IoT , just trying to understand the value of this sensory data.
Does it use sound spectrum to detect which event is working? Sound pattern?
Yes, as one of its features. We link to the full paper in the video description, which offers more details.
The system uses the audio spectrogram as part of its feature set. Turns out you can derive useful patterns from it. Other useful features from our findings include the vibration spectrum, EMI spectrum, light + color patterns, directional heat, etc.
I agree that it might be useful to think in terms of a hierarchy of sensors, but I disagree with the implication that the hierarchy starts with a nice simple "1". At the chip level, your "real" sensor is arguably more of a "virtual" sensor in a chain which only ends in voltage thresholds. Hierarchy, yes, but clearly numbered integer levels? Not so much.
Music detracted from the message.
Must be an engineer
Wow
Does this work when multiple events take place simultaneously?
Yes. You can see it in the last minutes of the presentation.
Yup, we show multiple event detections in the last minute of the video!
Where can I buy one?
This is unreal, ill take 500
OMG
"What could possibly go wrong"
Woohoo shadside. So, did y'ins make that cup of coffee?
Actually, we did :)
How the f.........?! :D (Y)
AAAARRRRGHHHH!!!! THAT BACKGROUND MUSIC IS SOOOOOOOOOOOOOOOO ANNOYINGGGGGGGG!!!!!!!!!!!!!!!
Yeah, this isn't disturbing at all. I want the internet of things to be completely aware of everything that's going on in every room of my house. No thanks.
well they said that nothing is uploaded to the cloud?
without an source code, how to be sure?
@pymai