Making Procedural Music in Unity
ฝัง
- เผยแพร่เมื่อ 14 ต.ค. 2024
- Hey There! 👋
Making procedural audio is hard. But hopefully this can help shed some light on the interesting processes behind generating audio at runtime!
github: github.com/rhe...
blog: blog.hedgecock...
I love how humbled I get when I think i'm a decent unity coder, and then I see this sort of wizardy
You dug way deeper than I expected to produce this, This was a blast to watch.
I was delighted to see you writing native code for something that sounded so simple in the beginning, love seeing massive optimizations.
Thanks for the kind words! I'm a sucker for optimization
First video of yours I've seen. You're getting my sub and a like cause you inadvertently taught me the existence of Expression-Bodied Properties and that feels like forbidden knowledge that I've wanted forever.
Really interesting topic; I had in plan to include Procedural Music in my own project, but the amount of work for the system itself would be too much. Instant respect for thinking of optimization and quality of your work instead of just blasting towards the outcome, it really pays back at the end, when your code makes your work easier ;) Patiently waiting for future videos/devlogs especially if in the same format
Thank you for your kind words! If there's one job a software engineer has, it's making our own lives easier right? 😅
@@rhedgeco Yeap, noone will help us, but surely will blame us for it 😆
First video and such great quality!! 👏🏼
This is incredibly good! I am pretty new to Unity and was investigating synthesis here for a music game. Lovely - you saved me so much time with very educating video :)
Thank you!
Great tutorial! I hope to see more from this channel, this is one of the most in-depth tutorials I've seen that use such complicated performance optimizations!
Would be great to see you do further developments on this, really appreciate you sharing the knowledge
Clean editing, interesting topic, and well-made video. Subscribed.
Thank you 😁
THIS IS ONE OF THE GREATEST TH-cam VIDEOS OF ALL TIME CAN'T WAIT FOR YOU TO BE BACK WITH MORE
"Something that makes a good software engineer is knowing you're a bad software engineer" - I've never heard anything more true than this. Am software engineer working for 6+years now. Subbed.
great video! Did not expect you to have 138 subcribers. Keep it up!
This is such an elegant tutorial, subbed and want more
Great video! I followed along very well an sparked lots of ideas! :D
For those who do not want/can use the burst compiler, you can precalculate the sinewave for example and store it in an array and read from there. I have to check how this interacts with filters and such, but I got similar or better performance as with burst compiler and its super easy to implement :)
Thanks for the comment! That is definitely a good way to go. In fact, I have a synth filter that caches the input from a runtime synth that does pretty much exactly this! If the data you are using for sounds can be prerendered, it's always a good idea to do so. Great tip! Glad you are enjoying my content 😁
That was very interesting, please make some more videos of this subject! Subscribed.
Good quality, good information, guess I gotta add you to my Unity watch list
It would be an honor ❤️
wow this is so exciting, as a game dev who has recently gotten into some modular synth stuff maybe i'll try & do some synth coding with this! thanks so much for making this & sharing
Would love to see where this is headed!!! Thanks for the video and the code!
Hey thanks for this great video! Eager to watch more
Nice video Ryan!!
Really great video, would love to see more on this topic.
I was excited to try this - I cloned your repo origin/main as of 2023-11-10 but I'm getting compiler errors in the Synthic.Handlers classes upon opening the project in the Unity editor. "error CS0234: The type or namespace name 'Native' does not exist in the namespace 'Synthic' (are you missing an assembly reference?)". It feels like the Rust code needs to be compiled to reference "Lib", but I'm totally guessing and have never compiled Rust code. Any tips you can provide will be very appreciated!
I probably need to put a note of this somewhere. But the original synthic did not use rust. So if you want to try out the version used in this video. Check out the old-synthic branch. Let me know if that helps!
@@rhedgeco thank you, I will give that a shot! Are there performance differences between the two?
@@KentBrakewell Nothing that should be noticable. Plus the rust version is more incomplete anyways. Should probably make the rust version an off branch instead of main
wow man this is really advanced. how do you think you would approach creating a standalone drum program that exports the tracks after the drum parts have been mapped out in an interactive way similar to a drum plugin? i doubt a unity app could plug into a daw so i think unity would need to export wave files as separate tracks of the drum kit.
Yeah unfortunately unity does not have a lot of good ways of embedding itself into another app which would stop you from building your own VST plugin with it. But you could create a VST that relies on another process and uses whatever your choice of IPC to talk to your unity program.
suuper insteresting! Please continue this series!
Thanks! I'm back to it so maybe I'll make another one soon!
hell yeea!
This is incredible!!
Love the video, but I'm super curious how you implemented the millisecond timer in the inspector?? Was this just some nice video editing for a visual helper, or did you actually get it to show in the inspector? I've been working in Unity for years but never seen any debug tool like that and I'm fascinated.
That's actually built in! When you use the OnAudioFilter read method, it displays the millisecond bar under that script. I agree it's very cool!
Very enlightening video, thanks. I did some work and found that if I use the OnAudioFilterRead to implement some procedural sound effects, such as playing the MiddleC sound when the keyboard is pressed. But I found that due to the different scheduling logic of OnUpdate and OnAudioFilterRead, there is an obvious delay from key press to sound playback. Is this implement not suitable for interactive sound effects?
Interesting that you found the delay noticeable. In my experience even if it takes a while frame to register the key press and set the audio it's only a 16ms delay. How long is the delay for you?
How did you show the latency in the script component? Nice video BTW!
It just happened by default in the unity version I was using at the time when the script had a OnAudioFilterRead method. Seems like it doesn't now which is a bug. There are a few workarounds if you look it up on Google lol
awesome. really enjoyed this, and your explanations were easy to follow!
How did you get that channels bar (The green dynamic bar that shows on play)?
Just curious
It happened by default in previous unity versions. They took it out for some reason. There are some hacks I've seen to bring it back though
Wouldn’t it be faster to calculate the samples in a compute shader?
Very great first video!
Great question! It would not for a few reasons.
First being that compute shaders can usually only be dispatched from the main unity thread so it an sync with the GPU. All the audio processing happens on an audio thread. Which is also why I didn't use jobs.
And the other reason being that data copying across the CPU GPU boundary actually takes a really long time. (In the order of a few milliseconds or more depending on the situation)
At the end of the day, GPU is better for higher volumes of data processing anyways, and we are only calculating 2048 samples. An incredibly small number when it comes to GPU parallelism.
why does unity use 48000 and not 44100 sample rate?
It's set to 48000 by default purely because it's slightly higher resolution. But you can change it to 44100.
So uh, making optimized procedural sine waves in Unity is cool too.
Lol yeah 😅 Im making more, there's just so much boilerplate and I forget that it takes some mental leaps to get to more complex sounds even though once you have programmatic waves you're pretty much there. Stay tuned for the next one!
@rhedgeco Hope it's soon! I know these things are a ton of work, but I'd really like the see the process and result.
@@barisaxo anything you are interested in in particular? I'm still flushing some things out so now's you're chance lol
using puredata is also another solution
Super interesting! So glad I found your channel!
I'm curious if you considered the "native audio plugin" route?
I'm not asking because I think you should be considering it - just because I want to learn how to create my own synths for use in Unity and struggling to make a decision about how I can go about it. I'd love to hear your input. :)
Here's another youtube video on the topic: watch?v=6SBjFPDrdu8 ( Jorge Garcia - Building game audio plugins for the Unity engine )
Yeah totally! I've actually done a fair bit of native plugin development, especially in rust. The only issue is that it's fairly difficult to reload native plugins at runtime. There are some ways, but its not super friendly. I found that getting used to the burst compiler was the easiest for fast and built in integration. Also the types that i make get to be shared between codebases instead of having to make duplicates.
Such a good video.
5:53
comic mono?
@@blepr good eye!
I feel like the title is a bit misleading considering that you don't actually make music in the video. Maybe something like "Setting up the burst compiler to make procedural music in Unity" could be more accurate.
Yeah that's a good point. Tbh I planned on doing more in the video, but it was getting to be a lot so I left it where it was. I've got new stuff coming soon so stay tuned!
You need a higher volume
Thanks for the feedback. I'll push those levels a little more next time!
Feels like Unity should really use C++, like the Unreal engine.
Yeah it's all about what they are going for. C++ is harder to write and easier to crash the program while C# offers a faster development cycle and better errors at the cost of some performance. So while I agree, I see why they went with C#.
First
If there's one thing I learned in kindergarten, its that if one is first, one might also be the worst. 🤓 Good luck fren
How can i contact you? Insta or something?
My email is linked in a few places like on my GitHub. I just don't actively advertise it on my TH-cam so I don't get too much spam