These sensors encapsulate so many possibilities, yet it is important to remember that the platform on which the camera/sensor is positioned must remain completely still. it would be very interesting if a next gen version of these sensors can compensate for platform motion using either IMU or additional sensors pointed at other directions. Having the sensor receive such motions as input could perform the ambient subtraction within the sensor.
It's true that a moving camera increases the data rate, but still our measurements with driving scenes show that in typical 20ms "frames" of accumulated brightness change events, less than 10% of the pixels are activated, making these activity-driven frames a nice fit to AI hardware that can exploit activation sparsity.
I am really trying to decipher the concept. Maybe you can help with a bit more elaboration. As far as I know, a video compression like HEVC analyses moving subject with great accuracy already, admittedly after the capturing by the sensor, through the encoder. Is this a similar concept but within the sensor itself?
No HEVC is a codec, it needs frames to make a video. This sensor doesn't capture frames, but pure data. The " temporal resolution" on video its the framerate, a video of 30 fps is limited because you can't see more than 30 images in a second, you can't see more if you play it in slow motion. With DATA you dont have that limitation, because there are no frames, just points that move from point A to point B on a canvas, you can slow down the speed as you like.
Think of it as a grid of analog brightness sensors but instead of measured brightness at each point they output the derivative (or instantaneous change) of the brightness value. If that isn't right someone please comment because I know that's not how it actually works but I thought it might help grasp the concept.
Can someone explain to me why this is inherently better than a high frame rate sensor with some processing to focus on change in luminance. Is it just the fact that it's equivalent to a super high frame rate? Or could some give a specific case where this would work better than a high frame rate sensor and the processing I mentioned, besides not having to do that processing. I'm not knocking the tech I just don't see the benefit besides offloading the processing. I'm sure these examples exist or the product wouldn't exist.
I know this is an old comment but basically its pixels are async and very dense in time, you get very high resolution and continuous temporal gradients for things like fast motion tracking in robots, factories, and AR/VR. The dynamic range is also better than many so called 'night vision' cameras I've used, and it works in the daylight just as well. Great for vehicles, robots, outdoor AR/VR. It also has a few features like per-pixel hardware bandpass filters to either filter out or deliberately capture flickering / vibration for industrial applications and robotics. I have been playing with them for a few years now.
@@Blag_Cog a typical and common sensor with electronic shutter speed can capture up to 12000 "images". An event sensor would give the illusion of unlimited framerate
@@ES-qy2ju I could imagine this would be amazing for a wide variety of uses. It is going to be really great data for neural networks and interpolation technology.
Sony + Prophesee = Great technology!
WOW 👑 SONY GOD FOREVER#1 ✌️ 👑
Head raising so many doubts. Lets wait for hands on preview. Hope for the best 💕💕💕
Congrats Sony!
This is future of robotics vision
This tech can be a great asset to paranormal research.
Way better than my old one. Perfect height
These sensors encapsulate so many possibilities, yet it is important to remember that the platform on which the camera/sensor is positioned must remain completely still. it would be very interesting if a next gen version of these sensors can compensate for platform motion using either IMU or additional sensors pointed at other directions. Having the sensor receive such motions as input could perform the ambient subtraction within the sensor.
It's true that a moving camera increases the data rate, but still our measurements with driving scenes show that in typical 20ms "frames" of accumulated brightness change events, less than 10% of the pixels are activated, making these activity-driven frames a nice fit to AI hardware that can exploit activation sparsity.
Te amo Sony
I am really trying to decipher the concept. Maybe you can help with a bit more elaboration. As far as I know, a video compression like HEVC analyses moving subject with great accuracy already, admittedly after the capturing by the sensor, through the encoder. Is this a similar concept but within the sensor itself?
No
HEVC is a codec, it needs frames to make a video.
This sensor doesn't capture frames, but pure data.
The " temporal resolution" on video its the framerate, a video of 30 fps is limited because you can't see more than 30 images in a second, you can't see more if you play it in slow motion.
With DATA you dont have that limitation, because there are no frames, just points that move from point A to point B on a canvas, you can slow down the speed as you like.
Think of it as a grid of analog brightness sensors but instead of measured brightness at each point they output the derivative (or instantaneous change) of the brightness value. If that isn't right someone please comment because I know that's not how it actually works but I thought it might help grasp the concept.
When this sensor would be used in camera's? Looks like a decent quality.
can self-driving cars operated by these
wow this is basically capturing ground truth optical flow
Sony you're biggest fan here. I live in Tokyo I would love to work for you. Message me!
Me too. A Lifetime Sony Fan ❤️❤️❤️
Love from India 🇮🇳 ❤️❤️❤️❤️❤️ 🙏🏻🙏🏻
Trying to understand what is the benefit of this technology
Ai efficiency, surveillance, obtain data without the need to analyze the image in post process
2:30 ...
Sony, metahero and WDW. Future is comming.
Amazing technology!
Awesome
Can someone explain to me why this is inherently better than a high frame rate sensor with some processing to focus on change in luminance. Is it just the fact that it's equivalent to a super high frame rate? Or could some give a specific case where this would work better than a high frame rate sensor and the processing I mentioned, besides not having to do that processing. I'm not knocking the tech I just don't see the benefit besides offloading the processing. I'm sure these examples exist or the product wouldn't exist.
It has the USP that you can beat the latency-power tradeoff of frame cameras, plus you get really large DR and minimal motion blur.
I know this is an old comment but basically its pixels are async and very dense in time, you get very high resolution and continuous temporal gradients for things like fast motion tracking in robots, factories, and AR/VR. The dynamic range is also better than many so called 'night vision' cameras I've used, and it works in the daylight just as well. Great for vehicles, robots, outdoor AR/VR. It also has a few features like per-pixel hardware bandpass filters to either filter out or deliberately capture flickering / vibration for industrial applications and robotics. I have been playing with them for a few years now.
Cool stuff
😍😍😍
Vr?
Let me find out this is project Skynet..
Wow
👍
👍🏽
3
This I done when I got pass out... 🤨
How many frames per second? Or should I say states per second? How many hertz lol.
it probably depends on the shutter speed because there are no frames.
@@ES-qy2ju yeah thats what I was thinking. How many "events" per second
@@Blag_Cog a typical and common sensor with electronic shutter speed can capture up to 12000 "images".
An event sensor would give the illusion of unlimited framerate
@@ES-qy2ju I could imagine this would be amazing for a wide variety of uses. It is going to be really great data for neural networks and interpolation technology.
sounds like lcd vs oled !!
Sony Alien Company 😁