I've been waiting for someone to piece this together. All the parts of virtual sound synthesis have existed for a while. Binural spatializers, physical gas simulators and tools to interpret wave patterns as sound. These are very good results!!
No, I mean the code/algorithm that generates the sound, it would mean you'd have to create ui for the artist to set element type and other attributes, it'll be quite technical no doubt, but so worth it.
things like this NEVER make their way out, don't kid yourself hahaha. Every SIGGRAPH or MIT demonstration I've ever seen was just for these guys to wank over while they do their thesis
it seems more like it's meant to accentuate foley work on animated movies or something similar, however it might be possible to pre-render convolution impulses for a game and use that in real time applications instead, at least for some use-cases
These videos are cool and all and I'm amazed by the work that's gone in to all of these techniques but this is the first time a siggraph demonstration has really made me question my grasp on reality.
3:33 When the plunger went in front of the trumpet the first time (just to show the animation) my brain was automatically changing the sound I heard... When I watched the clip again without looking at the screen this time, I heard it as it should sound. very interesting
Very interesting. Great work. I wish to see the results of this development applied to sound for automotive (loudspeakers playing music in a complex cockpit).
For anyone unfamiliar with latin or how professional study papers are written, "Et Al" means "And Others." So, it is researcher Langlois and others being cited, implying that more than one person was deeply involved in or helped write the paper.
I Noticed that the fan blocks the voices and maybe reflects it but doesn't seem to "chop" the breath carrying the voice like it would in this dimension.
Oh god... I was watching another siggraph video while at the same time doing something else. It was late at night, that said it was quite quiet. And suddenly i hear those sounds. I was sure that something is happening with my stomach. Really realistic sounds.
THIS WILL BE REVOLUTIONARY FOR MUSIC PRODUCTION. I am imagining a VST synthesizer where you can model sounds from real life objects!! Or create virtual objects in a 3d space, than generate sounds produced by hitting them, blowing on them, etc.
This is amazing. I've been wondering when we would be able to truly "render" sound for a while now. How long does it typically take to output some of these demonstration files?
Selected numbers: • Dripping Faucet: duration 8.5s; 18.6 hours render time on 32 CPU cores • Bowl and Speaker: duration 9s; 45 min on 320 CPU cores • Trumpet: duration 11s; 33 min render time on 640 CPU cores Source: graphics.stanford.edu/projects/wavesolver/assets/wavesolver2018_opt.pdf pg.10 Table 1
Maybe Nvidia can create a new soundcard using this technology with advanced AI for producing near real time synthesized sound, like today they are doing with raytracing with the RTX series. That would open a whole new world of opportunities in the music industry.
just to make sure I get this: a) this synthesizes sound in real time from thin air depending on materials, physics & force or b) this takes 3d sound sources (mono sound) and propagates them physically correct according to nearby objects with set materials?
Well, sort of. It simultaneously performs the sound synthesis and simulates the *effects* of a 3D environment on the vibration of air in its volume, which kind of incorporates a and b at the same time, but more. Hence the comments on the pitch shift of the spalling bowl being due to near field effects - that part was not a result of synthesizing the sound of the bowl, but of the synthesized sound *interacting with itself* due to reflections off the bowl and the floor. Adding the 3D sound sources to that is as simple as simulating a speaker cone vibrating in response to that data (exactly as happens in the real world - speaker just works by wiggling a cone back and forth in response to the data). Very cool stuff.
sad this is targeted at offline synthesis. Next step would be an ANN approach to do this in f*ing-seconds? @FellowScalars: is this published yet?! Link?!?
See the project webpage in the video description: graphics.stanford.edu/projects/wavesolver/ . The page contains a low-res and a high-res paper that you can download.
I can imagine a time where video games will have real time sound simualtions This would require a lot of computing but would save companies a lot of money cuz they save on sound recording and stuff… Maybe idk
totalermist ... "For now" I really think it won't take that long for this sort of thing to become commonplace in production (5-7 yrs) Not as a replacement but at least as an aide. CAD doesn't replace engineers and architects. Surely as with any other tech, the processing time will drop significantly the more people use it.
Daniel Shealey - I wasn't too sure about "Surely as with any other tech, the processing time will drop significantly"-remark so I went to check what happened in terms of processing time during the last 5 years. I took the Intel Xeon E5-2640, a mid-range 6 to 8 core 90ish W data centre CPU as an example to estimate what happened to processing power in the past 5 years. The model went from 6 cores at 2.5 GHz in its first incarnation to 8 cores at 2.4 GHz in its current version. Performance went up from 9500 Pts [1] to 15331 Pts [2] for an increase of *62%* in about 5 years (there is no direct successor and the somewhat similar Xeon Gold 5115 yields no significant performance gains). If we just take these 62% and round them up to 100% we get from 24 hours using 36 CPU cores *down to 12 hours* processing time for the _10 second metal sheet shake_ sound simulation in the next years [3]. Now I don't know about you, but I'd like to see a production company that saves time and money by having a pretty beefy server system render for half a day instead of just letting a Foley guy/gal shake a metal sheet for half a minute and have a sound engineer mix it... [1] bit.ly/2GUTqnL [2] bit.ly/2IKQHTM [3] graphics.stanford.edu/projects/wavesolver/assets/wavesolver2018_opt.pdf
totalermist Sorry, I wasn't clear. I meant this type of rendering will get faster in the future. Software will get more efficient. Hopefully they'll find a way to change over to something like a GPU based. Othwise it would have been a pretty useless venture to develop in the first place. For making sounds simulate water... Yeah, come on. That's nonsense for now. It will probably be first used in product design. 3D modeling high end speaker systems and simulated binaural coustic design for expensive vehicles. One I could see right away is acoustically engineering auditory "dead" spaces into architecture. Placing and testing baffles to create quiet spaces. Sound effects for movies is still a long way away. Civil engineering for putting buildings next to highways with solutions other than "a giant concrete wall"
totalermist but I agree. Rendering the sound of metal sheets alone would be kind of silly. But the render time of a 3D square as a proof of concept was probbaly met with the same eyerolls until Pixar came along. The people making this seem like they have a little more in mind that these few proof of concepts.
![[TH] 2024 PMGC League | Survival Stage วันที่ 1 | PUBG MOBILE Global Championship](http://i.ytimg.com/vi/FVFT-PFwU80/mqdefault.jpg)
im going to start calling cymbals non linear thin shells from now on
heheheheeeeeeee
Luke, I am your fawawawawawawawa therrrrrrrrrrrrr.
Bruce Luo darth vader had a really high ping
This is by far the most convincing solver so far. Amazing work!
I've been waiting for someone to piece this together. All the parts of virtual sound synthesis have existed for a while. Binural spatializers, physical gas simulators and tools to interpret wave patterns as sound. These are very good results!!
Woah, this really blows all the previous sound-synthesis work I've seen out of the water. Congratulations!
I hope this gets integrated to a game engine or a 3d software soon. Thank you all for your hard work, This's amazing!
This is *not* real-time, though. That dripping tap took almost 19 hours to render on 32 CPU cores...
No, I mean the code/algorithm that generates the sound, it would mean you'd have to create ui for the artist to set element type and other attributes, it'll be quite technical no doubt, but so worth it.
things like this NEVER make their way out, don't kid yourself hahaha. Every SIGGRAPH or MIT demonstration I've ever seen was just for these guys to wank over while they do their thesis
it seems more like it's meant to accentuate foley work on animated movies or something similar, however it might be possible to pre-render convolution impulses for a game and use that in real time applications instead, at least for some use-cases
Someday it will be.
But not in this decade.
These videos are cool and all and I'm amazed by the work that's gone in to all of these techniques but this is the first time a siggraph demonstration has really made me question my grasp on reality.
3:33
Is that a cat on the reflection on trumpet?
Cant wait for this to be implemented into gaming. But I will wait.
5 years later still nothing
This kind of stuff is really cool, it gets even crazier when you think about it in the context of things like video game application.
I've imagined something like this for awhile now, crazy to see it for real
This is really awesome! we need the bucket over head sound fx for the next elder scrolls game.
Brilliantly conceived and executed. Where science meets art requiring indepth of visual and auditory modalities. Kudos.
Why the hell this is not available for sound designers/Visual Artists/Musicians to use?!!!! The World Needs It.
3:33 When the plunger went in front of the trumpet the first time (just to show the animation) my brain was automatically changing the sound I heard... When I watched the clip again without looking at the screen this time, I heard it as it should sound. very interesting
Simulating compression waves in a virtual space to generate real-world organic sounds? Incredible.
Very interesting. Great work.
I wish to see the results of this development applied to sound for automotive (loudspeakers playing music in a complex cockpit).
I've been wondering whether this is possible with current technology for ages!! Amazing
For anyone unfamiliar with latin or how professional study papers are written, "Et Al" means "And Others." So, it is researcher Langlois and others being cited, implying that more than one person was deeply involved in or helped write the paper.
Thanks. I was wondering about that for a moment
Acoustic shaders sounds incredible!
Wow this technology is awesome. I have the vision in 20 years its normal to use it in videogames and interactive video game movies.
I Noticed that the fan blocks the voices and maybe reflects it but doesn't seem to "chop" the breath carrying the voice like it would in this dimension.
Oh god... I was watching another siggraph video while at the same time doing something else. It was late at night, that said it was quite quiet. And suddenly i hear those sounds. I was sure that something is happening with my stomach. Really realistic sounds.
This is just perfection. Why is this not implemented in games? Or is it?
I want this in my DAW
Creating VSTi instruments will be a sure way to monetize this research when render times reach near RealTime.
You absolutely don't need real-time rendering for this to be totally revolutionary for recorded music.
@@gloverelaxis ok
This is what I call excellent understanding of world around us. Great job guys!
REALLY excellent! Great work!
Love the space odyssey trumpet :D
It's called "also sprach Zarathustra".
THIS WILL BE REVOLUTIONARY FOR MUSIC PRODUCTION. I am imagining a VST synthesizer where you can model sounds from real life objects!! Or create virtual objects in a 3d space, than generate sounds produced by hitting them, blowing on them, etc.
Could even lead eventually to an accurate guitar emulator. Guitar sample instruments dont come close to the real thing but this could change that
I knew this is possible, but probably takes too many resources for being useful nowadays. However, it's good to know that the algo already exists.
IS THIS THE AUDIO VERSION OF RAY TRACING???!
This is incredible, great job
man these 3d research papers are the most surreal shit ive ever seen
What an amazing future you are buildind
we need this technology.. integrated sound with 3d applications.
This is amazing. I've been wondering when we would be able to truly "render" sound for a while now. How long does it typically take to output some of these demonstration files?
Selected numbers:
• Dripping Faucet: duration 8.5s; 18.6 hours render time on 32 CPU cores
• Bowl and Speaker: duration 9s; 45 min on 320 CPU cores
• Trumpet: duration 11s; 33 min render time on 640 CPU cores
Source: graphics.stanford.edu/projects/wavesolver/assets/wavesolver2018_opt.pdf pg.10 Table 1
Oh my god, this is amazing!
It makes me wonder where this tech could take us in the worlds of simulation. Material sciences, engineering, product design, medical research even?
You could literally say that about any novel technology. Why even say something like that. ugh....
Outstanding work!
Damn, I REALLY want this.
Really great work. Keep it up!
Absolutely incredible!!
4:34 inspiration for psycho pass by Xavier wulf
This is extraordinary...!!!
Maybe Nvidia can create a new soundcard using this technology with advanced AI for producing near real time synthesized sound, like today they are doing with raytracing with the RTX series. That would open a whole new world of opportunities in the music industry.
wait how do they make the examples
and is the software free
Okay that is amazing
So... the processed voice and trumpet are... what? Recorded sounds or completely synthetic?
They were both pre-recorded sounds processed by this system
We need this shit in games
just to make sure I get this:
a) this synthesizes sound in real time from thin air depending on materials, physics & force
or
b) this takes 3d sound sources (mono sound) and propagates them physically correct according to nearby objects with set materials?
Well, sort of. It simultaneously performs the sound synthesis and simulates the *effects* of a 3D environment on the vibration of air in its volume, which kind of incorporates a and b at the same time, but more. Hence the comments on the pitch shift of the spalling bowl being due to near field effects - that part was not a result of synthesizing the sound of the bowl, but of the synthesized sound *interacting with itself* due to reflections off the bowl and the floor. Adding the 3D sound sources to that is as simple as simulating a speaker cone vibrating in response to that data (exactly as happens in the real world - speaker just works by wiggling a cone back and forth in response to the data). Very cool stuff.
Can someone make a music out the the sounds in this video?
This is absolutely fucking groundbreaking.
"This DOI cannot be found in the DOI System"
Amazing!
This is amazing....
How do I get this? Seriously xD
Woah amazing!
acoustic and asthetic
AMAZING
すげー魔法みたい
And after all of this, we have among us
insaaaneee
Relatable
yall this shit sounds moist
sad this is targeted at offline synthesis. Next step would be an ANN approach to do this in f*ing-seconds? @FellowScalars: is this published yet?! Link?!?
See the project webpage in the video description: graphics.stanford.edu/projects/wavesolver/ . The page contains a low-res and a high-res paper that you can download.
Ante Qu i meant the link to the paper for the online method...
this is mind blowing. Raytracing for Light and now this for sound!? holy shit
The legos sound kinda soft
1:31
Archeologists they like bones and ancient civilisations, archaeologists!
My man. World Doctors is hilarious.
@@HarmoniChris I just noticed the character model in the video hahah.
now simulate two drums and a cymbal falling from a cliff
WTF you guys created a world following our physics inside a computer
Only if someday computers get really really fast
1:40 Because this is how physics works
It actually kinda is. It's animated rigidly to demonstrate the change in pitch and timbre when bending a vibrating sheet of metal.
Wowwww
The metal sheet and bowl were great. Cymbals not at all.
2:59
*_B_*
🅱️
Any chance of coming to softwares like blender? Is it possible that it is already applied in games of the 9th generation?
This isn't for games, blendlet.
@@Dr.W.KruegerHum... Not yet. But, How longe? 😅
fucking wow, im baffled
DAMM
It's not : "Luke, I am your father"
But rather : "No, I am your father"
Nicholas R.M. We don't need no stinkin' badges
MattMexor2
You're gonna need a bigger boat!
@@nic12344 That's a lot of fish!
Mandela effect
Reddit?
No! I am your father.
fuck im high
well the cymbal was kinda disappointing.
Now i totally believe we could be living inside a simulation
2:30
Arch-ae-ol-o-gists they like bones, and
Ancient civilizations
arch-ae-ol-o-gists
(And one of them's gay)
I can imagine a time where video games will have real time sound simualtions
This would require a lot of computing but would save companies a lot of money cuz they save on sound recording and stuff…
Maybe idk
500th like
wOw
😄
Wat.
i wish i can ABCD in a barrel
a
*B*
c
(d)
🅱️
Bye bye foley people?
Unlikely - it requires longer to render the sounds than for a Foley artist to create the sounds and a sound engineer to mix it.
totalermist ... "For now" I really think it won't take that long for this sort of thing to become commonplace in production (5-7 yrs) Not as a replacement but at least as an aide. CAD doesn't replace engineers and architects. Surely as with any other tech, the processing time will drop significantly the more people use it.
Daniel Shealey - I wasn't too sure about "Surely as with any other tech, the processing time will drop significantly"-remark so I went to check what happened in terms of processing time during the last 5 years.
I took the Intel Xeon E5-2640, a mid-range 6 to 8 core 90ish W data centre CPU as an example to estimate what happened to processing power in the past 5 years.
The model went from 6 cores at 2.5 GHz in its first incarnation to 8 cores at 2.4 GHz in its current version.
Performance went up from 9500 Pts [1] to 15331 Pts [2] for an increase of *62%* in about 5 years (there is no direct successor and the somewhat similar Xeon Gold 5115 yields no significant performance gains).
If we just take these 62% and round them up to 100% we get from 24 hours using 36 CPU cores *down to 12 hours* processing time for the _10 second metal sheet shake_ sound simulation in the next years [3].
Now I don't know about you, but I'd like to see a production company that saves time and money by having a pretty beefy server system render for half a day instead of just letting a Foley guy/gal shake a metal sheet for half a minute and have a sound engineer mix it...
[1] bit.ly/2GUTqnL
[2] bit.ly/2IKQHTM
[3] graphics.stanford.edu/projects/wavesolver/assets/wavesolver2018_opt.pdf
totalermist Sorry, I wasn't clear. I meant this type of rendering will get faster in the future. Software will get more efficient. Hopefully they'll find a way to change over to something like a GPU based. Othwise it would have been a pretty useless venture to develop in the first place. For making sounds simulate water... Yeah, come on. That's nonsense for now. It will probably be first used in product design. 3D modeling high end speaker systems and simulated binaural coustic design for expensive vehicles. One I could see right away is acoustically engineering auditory "dead" spaces into architecture. Placing and testing baffles to create quiet spaces. Sound effects for movies is still a long way away. Civil engineering for putting buildings next to highways with solutions other than "a giant concrete wall"
totalermist but I agree. Rendering the sound of metal sheets alone would be kind of silly. But the render time of a 3D square as a proof of concept was probbaly met with the same eyerolls until Pixar came along. The people making this seem like they have a little more in mind that these few proof of concepts.
Does this system have any accounting or calculations based on position of the microphone
Wow. Excellent work.