Wow, that is super cool! I'm very excited about this new frontier of stage show; I've seen a lot of interesting stuff on twitch, but being in a space/interacting is next level!
Thank you! Yeah even though there's some tech similarities, it feels really different to an online stream - there's something about having the audience physically there that makes a big difference!
WOW. I am a singer/songwriter/musician, I set up and performed live shows in my backyard venue with a full band, studio quality sound, and programmed stage lighting, all handled by me alone (except for the other band members' playing their instruments). So I KNOW this was an incredible amount of research, planning, trial and error, tech nightmares, and it's just amazing that you pulled this off. Congrats! I have been planning to setup something similar for a while now, but I want to livestream my solo looping performances as my custom avatar. I don't need cinematic scenes or remote data transfer, but I am getting bogged down in trying to find a solution that lets me play instruments (drum kit, guitars, keys, sing at a mic) as my avatar, in real time, on a DIY budget. I am very tech savvy, but between creating the avatar from my 3D art images, to rigging, to motion capture, it's daunting. And the hardest part seems to be having my avatar play a guitar live. I'd prefer it be AR , with the avatar projected onto my body/fingers as I play my own real guitar, but free/cheap mocap cannot track limbs and bodies when blocked by a guitar's neck or body. Do you have any ideas or potential directions I could pursue this? My closest solution so far is a combo of my Valve Index VR system, some vive trackers, an iphone for face capture, and then AR props of the instruments in MoCap Fusion VR, with live video output handled by OBS....
Thank you! And your project sounds super cool, thanks for sharing! If you haven't already, check out WoodyDevs - he recently did a project where he animated himself playing drums & keyboard using a combination of motion capture and MIDI triggers: th-cam.com/video/9cNR1mz167A/w-d-xo.html Long story short, rather than having the finger/hand animation driven by mocap, he used the audio input to drive the hand/finger animation. It seems pretty hard, and not sure how MIDI would work with a guitar, but I think you'd definitely get some inspiration from the video, and I'd recommend joining the Discord! I've only really used Unreal Engine so I'm not super familiar with Valve Index or MoCap Fusion, but I do 100% recommend Unreal + motion capture. There's lots of free assets on the Unreal Marketplace, including instruments and various plugins that might help you with what you're after. Good luck and I'd love updates!
@@bigsandband Thank you so much, my weeks of research never crossed that video due to the ambiguous title! Definitely interesting, I will check these out and report back. :)
Nice! So glad it all came together well. Is there more to the team besides yourself and Emma? I always wonder how many people go into something like this.
Mostly just me and Emma for this one! Lots of late nights 😅 But also shoutouts to the team at S1T2 who mentored me with UE stuff, Nic Ferrar for being a brains trust, Powerhouse staff for letting us do heaps of testing and the Data Arena team for backstage support. I also had the prototype show and prior assets to build from, so it's always a growing list of credits!
![ร้อยพ่อพันแม่ - WanMai [Official MV]](http://i.ytimg.com/vi/XY2pNYASEbg/mqdefault.jpg)
Wow, that is super cool! I'm very excited about this new frontier of stage show; I've seen a lot of interesting stuff on twitch, but being in a space/interacting is next level!
Thank you! Yeah even though there's some tech similarities, it feels really different to an online stream - there's something about having the audience physically there that makes a big difference!
WOW. I am a singer/songwriter/musician, I set up and performed live shows in my backyard venue with a full band, studio quality sound, and programmed stage lighting, all handled by me alone (except for the other band members' playing their instruments). So I KNOW this was an incredible amount of research, planning, trial and error, tech nightmares, and it's just amazing that you pulled this off. Congrats!
I have been planning to setup something similar for a while now, but I want to livestream my solo looping performances as my custom avatar. I don't need cinematic scenes or remote data transfer, but I am getting bogged down in trying to find a solution that lets me play instruments (drum kit, guitars, keys, sing at a mic) as my avatar, in real time, on a DIY budget.
I am very tech savvy, but between creating the avatar from my 3D art images, to rigging, to motion capture, it's daunting. And the hardest part seems to be having my avatar play a guitar live. I'd prefer it be AR , with the avatar projected onto my body/fingers as I play my own real guitar, but free/cheap mocap cannot track limbs and bodies when blocked by a guitar's neck or body.
Do you have any ideas or potential directions I could pursue this? My closest solution so far is a combo of my Valve Index VR system, some vive trackers, an iphone for face capture, and then AR props of the instruments in MoCap Fusion VR, with live video output handled by OBS....
Thank you! And your project sounds super cool, thanks for sharing!
If you haven't already, check out WoodyDevs - he recently did a project where he animated himself playing drums & keyboard using a combination of motion capture and MIDI triggers: th-cam.com/video/9cNR1mz167A/w-d-xo.html
Long story short, rather than having the finger/hand animation driven by mocap, he used the audio input to drive the hand/finger animation. It seems pretty hard, and not sure how MIDI would work with a guitar, but I think you'd definitely get some inspiration from the video, and I'd recommend joining the Discord!
I've only really used Unreal Engine so I'm not super familiar with Valve Index or MoCap Fusion, but I do 100% recommend Unreal + motion capture. There's lots of free assets on the Unreal Marketplace, including instruments and various plugins that might help you with what you're after. Good luck and I'd love updates!
@@bigsandband Thank you so much, my weeks of research never crossed that video due to the ambiguous title!
Definitely interesting, I will check these out and report back. :)
Nice! So glad it all came together well. Is there more to the team besides yourself and Emma? I always wonder how many people go into something like this.
Mostly just me and Emma for this one! Lots of late nights 😅 But also shoutouts to the team at S1T2 who mentored me with UE stuff, Nic Ferrar for being a brains trust, Powerhouse staff for letting us do heaps of testing and the Data Arena team for backstage support. I also had the prototype show and prior assets to build from, so it's always a growing list of credits!
Hi can we connect? I would like to invest in my own rokoko system, but i have some doubts, that you could help me to solve
Hello! No worries - feel free to drop me an email at hello@bigsand.band and I'm happy to answer any questions :)