วีดีโอ

well - i guess i'm still discovering it - theoretically, it's a strongly timed language that has structural similarities to Java and C# - at least compared to Csound or Faust anyway. the main reason is that Unity CANNOT under any circumstances generative individually timed note events reliably on a grid - you can arrange clips or loops but not individual note events. so that timig needs to come from an external clock. i've heard of folks using OnAudioFilterRead to generate a timing grid clock of sorts, but don't have a recent example. the previous prototype version of Constellation used Csound in Unity and i got very excellent timing on that - it was on PC VR on Vive and this will be on Quest 3 so we'll see. also the main ChucK developer has been dark on Unity/Chunity for obvious reasons and i'm not getting much in the way of responses back from my help queries yet. so i might switch back to Csound. anyway, long winded answer - hope this helps a bit!

thanks so much! glad you enjoyed it and found it informative!

@@gameaudioinstitute Quick question if you don't mind... do you happen to have a video where you show how to make it so that audio is reactive to the environment rather than being triggered? In my use case, I want to make essentially a sound proof, or heavily occluded room, but its a multiplayer experience so accessing the audio sources of each user with a trigger or collider is not ideal nor scalable. It would be nice if there was essentially an audio source that "took care of the occlusion on its own" so that if you or other users enter a room, the room would essentially mute or heavily occlude the people inside for users outside the room, and users inside the room can hear each other normally. I appreciate any advice you can give!

@@jordanmuniz6167so that sounds like a social VR issue. i think it should be able to be handled interactively. you're already talking about about a room that will be, in effect, a 'quiet room'. as long as the audio from outside sources or other users is accessible and routed into Unity you can stick all of the objects outside that room to set their Audio Sources to a specific Output group that would be muted or filtered. you could probably snapshot that effect if needed. I may be able to make a video but this year has been exceptionally dense with other work so i may not get to it in time. it's a cool topic though - the main challenge as i see it is getting the outside user sources to be routed to the Mixer. once you can get that happening, everything else should be easy.

and you can also set 3D falloff curves on the AudioSources if you're using 3D - but that's a fairly basic thing. you can add Audio Filters to objects with Audio Sources too and set some control based on distance. but these are simpler approaches.

Yea don't worry about the video! I was just wondering if it was a topic you already covered. Thank you so much for the advice! You've given me some stuff to work with 👍@@gameaudioinstitute

absolutely agreed on Helm, but i think it goes to what i've said about procedural audio - it may save on RAM but its harder on the CPU and there isn't that much of a market for it either. Matt was charging $80 for it which is a bit high for an asset. and thanks for the tip on RNBO! - didn't know much about it at all though i've heard of the name. will definitely check it out.

okay, so i did a bit of research on it, and i don't think it will be included. or i can't include it anyway since it requires MaxMSP which is not cheap these days. all the tools described here and in the second half coming out are free and they have permissive licenses that are open to allow anyone to sell an app or game made with them. i do not think that RNBO has this option and it also requires commercial software to create the patch. on the other hand, i can't recommend Supercollider either, because it's license is too open (GPL) and too viral to be used in the AppStore. it might be okay in Google Play but i'm not at all confident about working in VR though i can research it. i suspect very few people are using it for game audio.

no problem! let us know what other tutorials you'd like to see on the channel!

yeah - it seems like it would be a really cool thing, and i think it's useful in certain outdoor cases, but it's quite tricky to manage for square shaped indoor rooms, plus it had bugs for while that made me not reach for it, especially once the Mixer came around with its own plugins. glad you enjoyed the video!

i'm glad you found a solution for your issue! in this case there wasn't any chance that any other object would trigger it so i still feel it's a bug and not working properly - although it might be because i had Edit In Play Mode enabled which would disable snapshots. glad you're enjoying the series! i'm using ChatGPT to write scripts in the latest video in the series. check it out!

glad it helped you out!

reverb zones aren't really there for music, they are more there for general 3D audio. music is usually 2D in nature, and Unity has a mixer with plugins where you can add reverb if you need to - sometimes it is useful to add a bit of reverb but yes you can easily go overboard.

thanks Kevin! i will certainly try my best for sure, but it has been a rough adjustment coming from Unity. the early part of this (about the first 6 episodes) aren't even covering MetaSounds and i expect this to be true for a while, because the app relies on managing a lot of physical objects on the screen. the TL;DR is that UE likes to do things its own way - a way that works for standard games and this may end up being enough of a blocker for my case that i won't be able to do it. and that's not even considering the performance issues. i think even if this project doesn't work for me i'll still have at least a few videos covering using MetaSounds in a generative context anyway. we'll see - game dev isn't a linear process though.

@@gameaudioinstitute I totally understand where you're coming from when it comes to Unreal quirkiness. I'll enjoy the series more watching you figure it out while applying your vast knowledge of Game sound design. The Lyra sample game has a good deal of Metasound implementation you could take inspiration from, in case you haven't checked it out already.

@@kamazooliy thanks a bunch for the suggestion! once i get a layer going and operational i'll be in a position to start triggering generated sounds so i'll start looking for simple synth options. not there yet though...

thank you for watching and subscribing! well we've already got one up covering Persona 5 so you you can go watch that on our channel, and we are going to drop one on Tetris Effect either at the end of this month or the beginning of the next and you'll be notified of that if you have that turned on. stay tuned for more! we'll keep Witcher 3 in mind though. thanks for the suggestion!

thanks Rick! we have another one coming up on Tetris Effect pretty soon - these in depth ones take longer to produce but we have lots of other stuff to check out in the meantime.

thanks so much for your wonderful and supportive comments! best thing you can do for us is spread the word. we've got Tetris Effect coming in January and more cool games after that to analyze. i'm also planning a deeper dive into more advanced generative music concepts and tackling Unreal for the first time on the channel. more will be coming!

yep - he might not have all that great of reputation in person, but his music composition skills are definitely not in doubt. check out Jean Sibelius if you want to hear what composers he was inspired by for this score.

thanks for the comment! we'll keep 'em coming!

yeah, i understand - i did mention my method only works in that specific situation. i'm not 100% sure but i believe that your controller is pretty old. i can give a basic idea of what you need. in my situation the footsteps script is able to find out information from the Controller (which has a rigidbody and a collider built in). now you can have separate components on the Player and you can find out info by having the surface report collisions and then send that data to the footsteps script. i don't remember whether your controller script has any variable pertaining to grounding but i would probably guess it's got something that can detect a foot on the ground vs in the air. but i can't help much more without seeing the problem in more detail. you can join our Discord and post there (it's in our Linktree links).

@@gameaudioinstitute ok Sir

hi there! thanks for your interest, and glad you are finding these helpful! since the demos don't have enemies we didn't cover how to be selective in triggering sounds. this is fairly easy to add though. what you'll want to do is instead of a boolean switch what you need is the CompareTag function. in the OnTriggerEnter function you'll want to change it slightly: void OnTriggerEnter(Collider other) { if(other.gameObject.CompareTag("Player")) { //play sound or whatever here } } so in this case, we make sure to get information from the Collider that enters the trigger area. then we get the Game Object that is associated with the Collider (gameObject), and then we use the CompareTag method to check if it matches a specific tag, which is a string (you can see it's in quotes). you do need to make sure your Player object is tagged as Player - it's a default tag option usually, but it does need to be set for the object that has the collider and rigidbody entering into the trigger. hope this helps out! i'm happy to do a video on selective triggering of audio if you think that would be helpful!

@@gameaudioinstitute Thanks For the quick response Sir It's Working Fine Now.

hi there - thanks for your enthusiasm and we'll try to cover FF 7 at some point. in regards do Nintendo they are the absolute kings of super complex music management that's made to sound cute and fun no matter what behind the scenes magic is going on. in response to your clip (thanks for that - always good to see other music analyzers out there) i would say the musical electric zap traveling loops are managed entirely with MIDI triggered sample instruments - often called sound fonts. this way the system is capable of changing key or mode very easily where as if they handled with a custom music loop you'd have to have one ready regardless of what change the player was making - or decided to choose from a list as they demonstrated earlier. definitely a cool application of the idea! we'll be looking at some similar systems later on.

@@gameaudioinstitute ​ @Game Audio Institute Thank you! I too suspected that they often use MIDI for this kind of applications in their own game engine(s). I'm not super familiar with MIDI inside Wwise though, I thought you could just trigger different MIDI files, not modifying them in real time. But I'm probably wrong. I can't wait to see what you'll be covering 👍

@@MarcoLongoMusic well, by the time you get into triggering MIDI procedurally, you're well outside of Wwise's wheelhouse. as popular as Wwise is, it's definitely not the best tool for every kind of music application. for example games like Peggle 2 indeed use Wwise, but to make that game happen, a WHOLE lot of straight up C# scripting had to happen to make the music system respond intelligently to game state changes in terms of key and/or mode. and games like Mini Metro would not be possible period - Wwise isn't anywhere near ideal for some kinds of procedural music.

@@gameaudioinstitute that makes sense. But what if you "accompany" different pieces of audio with a hidden MIDI track that follows tempo and structure while playing an arpeggio like the one in the video? Could you switch on the fly in Wwise? Or there could be some stuck notes issues?

@@MarcoLongoMusic well, tempo can be tracked outside of Wwise. i know that usually it's not a common thing to use multiple audio systems. but i am pretty sure in this case that Wwise can do a lot more than play a MIDI file. you can create sample banks and trigger MIDI events on them. but you would have to inform Wwise when the change takes place and in what way so it can alter the pattern or use other patterns. keep in mind that some action takes place and these levels aren't procedurally generated so there can be triggers that would tell the music engine to change mode at a certain point, or in the case of the 'jukebox' selection to play a mode that harmonizes with it. it's not incredibly easy but it is possible, and much easier to generate notes that way than altering existing ones. and my guess is they do their best to bulletproof the system to avoid hanging notes, etc. that all being said i am fairly certain that Nintendo does not use Wwise in their main releases as they are a large company and have their own in-house audio engine that handles music as well. and it is quite possible that this engine allows MIDI events triggering sound fonts or DLS banks. that's certainly what was possible back in the SNES days (indeed that was the only option actually at the time).

Im not so sure yet for the price + Ill need 3 more sets of classes but not the board or wand but I cant find the price for just the classes on the site

@@Durrum1 you'll likely want the group XE pack at that point. they are catching up on the XE single Kickstarter backers right now. after that comes the XE group pack backers and then they'll be getting to preorders. i think once the XE group packs are prepared they'll likely be orderable on the web. i'm not sure of the hardware support just yet for multiple glasses, but it's definitely on their immediate roadmap.

actually, with evening room lighting it's fine - you can easily see the board with your eyes. but web camera placement is tricky and the result isn't as bright as you can see with just the glasses.

well, i think that's a misreading of the product. for about $350 you get a pair of AR glasses that are roughly 8-10X cheaper than Magic Leap or HoloLens with brighter images than either of them. and of course it has its limitation of having to be projected onto a surface - but that makes it not all that dissimilar to current computers bound by their screens - although at the moment T5 can only do horizontal orientation of the board. and there's so much more sensor and camera math going on behind the scene to make the experience work smoothly. my demo is restricted because i couldn't hold the glasses near the webcam and play the game at the same time. and it's incredibly portable compared to a desktop computer. the board folds up into an area not much larger than a typical small laptop. and as far as price i don't know where you're able to get reliable 720p projectors for $70 but i think the prices are closer to T5 or closer to 2/3 or 1/2, not 5 times cheaper. and you'd still have to have add IR cameras, sensors, and firmware to track the board position incidentally a Steam Deck is actually an excellent product to pair with T5 - although its GPU isn't good enough for VR it's got plenty of power to drive the 720 per eye resolution of T5.

To me it looks just as portable as a board game pretty much. Not to mention besides just paying for hardware, you are also paying for many years worth of R&D and software/firmware development to get it all working. Comparing it to the price of a projector is like comparing oranges to apples as you can't do this with just an off the shelves projector.

thanks so much Patrick! really appreciate that you found it useful!

indeed! that was definitely one of my thoughts for sure when seeing it.

We need that! Action!

its all because of physics. the reason why those VR headsets are so bulky is that they have to cram the optics, battery, mirrors, electronics, etc all into that headset. I like my quest 2, but really putting it on is exceedingly off-putting. These look light. And being able to see your surroundings looks like a big plus. Having moved out of state, I've ordered 8 of these for family and friends, simply because I want the experience of playing games the same way as I did when they were local. We could get together and play boardgames for hours, and I'm really missing that feeling.

@@gregmattson2238 Really smart to get so many headsets. Is there maximum number of headsets that can look at the game board at the same time? Or controllers that can be used on the same board at once? It's interesting that although none of the headsets will be reflecting back to the exact same space, if they did, they'd actually be creating the same image, so there is no chance that they'd interfere with each other on the board. If anything, it would just boost the brightness.

@@joelface from a couple of interviews I've seen from Jeri, the headset itself ups the framerate. So it basically takes any input source that you have, and does interpolation between frames to churn out 120 fps. so locally I'd think you could have 1 pc and multiple devices and be able to play a video game pretty readily. For remote games, each local pc would responsible for its share of the board so here I don't think the CPU/GPU would be the bottleneck, and any network lag would be the handled by the same onboard interpolation. In any case the games I'd like to play - eurogames - I don't think this is going to be taxing my pc at all, and I'm so looking forwards to getting my devices.

@@joelface Theoretically you could use an infinite number of headsets on the board due to the way it reflects, but since nothing is perfect, there will be some light bleed around the angle of your head so it might be possible to interfere if two people are right next to each other. There's also a chance it will be negligible enough to not matter and just work though.

no problem. i was debating about jumping right into the coding or starting right from the beginning with installation. glad i made the right decision for you!

i'll try! - the challenge is to try to get some way of mounting the webcam so that i can have access to the controller. with this video i was carefully holding the glasses in front of the webcam with both hands which mean i could not actually operate the controller.

yeah it's definitely useful for first timers, and it's important - originally Unity came with its own script editor called MonoDevelop, but they dropped that and went with Visual Studio which is fine but VS is a much more broad based and widely used IDE than MonoDevelop, and the simple Code variant is geared to be more open-ended and available to a variety of languages (like JavaScript, C++, Python etc). however the code highlighting and auto completion features have to be enabled for C#. the snippets are just more Unity function/method oriented, again very useful to know though. like training wheels.

That would be an interesting idea.