Long time fan and love what you were able to do here. I remember a few years back looking to see if there was a USB emulator for the ESP32 and only finding one for the 8266, then seeing that it was you who had written it. Its a small world out there, keep up the amazing work!(I ended up using a Arduino with some modified firmware to act as my HID)
Dude you're the best kind of insane person, I mean, you put so much effort into absolutely amazing things. All of this is making the world a better place, we're glad you're on our side, lol!
Its crazy how you just go "and yeah then i did xyz" like it's some trivial task, but xyz is really more like the coding equivalent of designing and building a fusion reactor. Always look forward to your videos! you are a huge inspiration :)
"14:28 Because it was using the actual skeletal data, for all the players in VRC who use full-body tracking, you could see them dancing around on the swadge. Yes, even this potato of a game system supported legs." absolute LOL 14 minute windup but absolutely brutal burn
I've been subscribed since 2015 and I have to say that you keep impressing me every time! This is a marvelous feat of design, engineering , integration and hackery Awesome job!
the register bruteforce hacking is so painful and yet awe-inspiring to read! i had a thing that could've used something like your long session but it was just too frustrating.. great job on this project! (more specifically, using both DACs on the ESP32 in accursed ways, there's some bitluni videos to watch on surrounding things if you haven't)
@@CNLohr my memory is hazy but yes. bitluni bitbanged composite video along with audio on the ULP in separate videos and i was hoping to combine those to get a cute little application platform, pico-8 esque but in hardware -- after brushing my head into it for a while i halted the project pending access to an oscilloscope and similar nice-to-haves i love writing programs for machines where i'm close to a piece of relatively-simple hardware that could conceivably be understood by one person, the low latency feels great and the security in not having the rug pulled under me too and it's expensive but i'd like to try again eventually. emulators are okay and less e-waste is great, but if they aren't perfect emulators then it's only added more layers of stack to pop and comprehend, which is not okay for an experimental, toy machine. i want to have a bit of me embodied in every layer of it maybe?
this is the kind of stuff i wish i could make some day, interconnecting so many little things and projects youve made over the years to make a really cool thing
when I was subscribed this channel, I did't expect to get infomation that VRC intergration with esp32 from this channel. but these info are what I need. cool!
It is surprising how much more powerful a system that can execute code is than systems that are hard-coded into gates. It is really surprising just how much can be done with just a little code.
This is basically the epitome of something I'd want to also make, so it's amazing to see all the work done and explained so succinctly. Amazing work. I remember even just trying to get low-latency streaming working with my own RTMP server, learning later about RTSP, as well as trying to consider encoding delays. This type of ultra low-level programming and hardware solutions are mindblowingly awesome, and only makes me more sad that VRChat and many other games run the way they do.
I see the notification for this vid and am like "Oh goodness Charles' at it again". not even 3 minutes into the vid I'm already amazed and haven't even _seen_ the main point of the video yet
I've been working on a wireframe flight simulator targeted at similar hardware on and off for a long time, super excited to look at the source code! Very cool the way you allowed these devices to interact as well. Fantastic job.
Everytime you post something, you show another unbelievably insane project. Btw have a look at standalone VR headsets (Quest 2, Pico 4 They cost like 400~500 and can run VRChat without the need for any other hardware, though I guess you probably know that already). You could make even more intervined VR projects with that. The headsets run android and can communicate via serial over usb-otg with a microcontroller. So hooking up an ESP32-S2 to work as an espnow receiver/sender would work. Also the Quest 2 in particular has kind of good handtracking and a black and white AR-mode.
I've tried using VRChat on stand-alone and it is just so utterly uninteresting to me. Running it on PC for me is the difference between looking at a VHS video of an event and being there. :-/ Maybe the next round of stand-alone.
I can not believe the incredible amount of work that went into making this amazing, I don't even know what to call it, system? gadget? interface? Amazing job!
What a dope system! Integrating the ESP32 like that is awesome and gives me hope for some really awesome spacial XR experiences in the future. I wonder if the ESP32 S2 supports Wifi RTT for some wifi-based localisation systems, allowing to integrate the position of the devices into the experience.
@cnlohr few days ago I found your espusb project while trying to find a way to implement usb mouse and keyboard funcnality to esp8266 . Can you please make a video how anyone can customise and use espusb project
@@CNLohr you my friend, you don't go... You are basically free falling 🤣 You can open up a club together with Sebastian Lague, Stuff made here and the Thoughtemporium lol
@@CNLohr Don't downplay your self. I respect the persistence to finish stuff you guys do. I'm the sort of guy who starts projects just to abandon them on the last mile... I get to easily distracted, and lose interest after a while.
Hey i know this is an older video but im very curious to know how you find all of the surrounding ESP mac addresses to send data to and receive from. I had a look through the code but couldn't find where that happens. Could you explain briefly how you accumulate those addresses or discover them, or even point me to the function that handles that! Awesome project btw
With ESP-NOW, you can send to "everyone" by broadcasting to FF:FF:FF:FF:FF:FF, then when receiving a packet you can decide what to do with it. So in this project, everyone just sends everything to everyone. The main node sends info about the VRChat players, the individual players just info about their ships/boolets.
do you take comissions for technical projects in VRC? I want to hire someone to help make a working book that can be pulled out of an avatar or area of a world and flipped through, probably having multiple books contained in the same one, having the main book change aspect ratio for the PDF loaded (they could be stored as relatively low res textures?) would want to eventually be able to pull out pillows, incense, books, and maybe even a basic music player all from an avatar to put in public and private worlds to hold spiritual programs
I regret that I do not take commissions, but, if you join my Discord, there is a #vr channel, and there may be someone who can point you in the right direction.
sorry if im missing something, but couldn't you get around the performance issues of gpu -> cpu data transfer issues by rendering your data somewhere on your screen and then grabbing that from an external program? Or was that data only local to users machines in vrchat? Sick project though, looks like so much fun :o
How compressed can you make a game 101 right here I also have a question this is coding efficiency right? if so is it possible to make a bigger game for modern computers with those tricks
You can make games for modern computers this way. It just is a trade off between expedience and effectiveness. As soon as you use an abstraction you trade some efficiency for coding expediency. Like if you use a library to do something. But the issue is they also made those trade offs. And they use abstractions. And so on and so on. And eventually in modern computers, we generally achieve
Depends on how you count it - EVERYTHING, thousands, if you include all the work on the ESP, rendering systems, . But just this part of the project shown probably only about 250.
This is one of the most impressive things I have ever seen!
Thanks!
"Yes, even this potato of a game system supported legs"
Meta felt that burn 🤣
I was concerned that your genius would be lost into a VR abyss but nope! Extremely impressive and inspiring!
I much prefer this conception of the metaverse to Zuckerpants' one.
@@jimmy2drinks he did better with a billion times less funds
“Similar but legally distinct” hahahahahhahahah
¯\_(ツ)_/¯
@@CNLohr Its something I would've said as a kid about my homework I copied from a friend.
Long time fan and love what you were able to do here. I remember a few years back looking to see if there was a USB emulator for the ESP32 and only finding one for the 8266, then seeing that it was you who had written it. Its a small world out there, keep up the amazing work!(I ended up using a Arduino with some modified firmware to act as my HID)
Thanks. And btw - the S2 has really good native USB support.
This is really incredible! Love the data encoding!
That was the part people kept getting annoyed at me for, "just use a real encoder" but I think it was my favorite part at the end of the day, too.
@@CNLohr What even would be a "real encoder" here?
Dude you're the best kind of insane person, I mean, you put so much effort into absolutely amazing things. All of this is making the world a better place, we're glad you're on our side, lol!
Thanks :).
cool project, love your solution to dealing with video compression
Just don't compress ;)
This is insanely impressive, awesome work like always.
That means a lot.
you are a wizard! been keeping up with your esp obsession for a few years and you are just making more and more insane stuff!
The ESP obsession isn't going anywhere.
this is genuinely insane
good job
Thanks!
Incredible amount of effort and a great result! Glad I was able to be on the VR side to see it work.
:)
Man, this is so impressive.
The creativity and engineering is insane.
Its crazy how you just go "and yeah then i did xyz" like it's some trivial task, but xyz is really more like the coding equivalent of designing and building a fusion reactor.
Always look forward to your videos! you are a huge inspiration :)
You can look through my code. "Did XYZ" is really not that much code. Each major part is probably
This is incredibly impressive, the amount of effort that must've gone into this is crazy. Great job!
It was a lot, but it was over several months so it didn't feel like a lot.
This is maybe one of the coolest things I've ever seen, basically magic to me -great work! :O
Hah I can see myself at 1:42!
The swadges were super cool this year. Thanks for all the work you put in, both in meatspace and the digital world!
"14:28 Because it was using the actual skeletal data, for all the players in VRC who use full-body tracking, you could see them dancing around on the swadge. Yes, even this potato of a game system supported legs." absolute LOL 14 minute windup but absolutely brutal burn
Always a pleasure to see what you've been up to Charles :)
I try to make it at least once every 2 months!
This is an amazing piece of work and a great end result!
I am really happy with how it turned out. I don't think I "need" to top this for next year's MAGFest :-p
I'm always amazed at the wonderful things you can do
Thanks
I've been subscribed since 2015 and I have to say that you keep impressing me every time!
This is a marvelous feat of design, engineering , integration and hackery
Awesome job!
Thanks!
Amazing! You explain your process well in a way that's fun to watch
It often feels harder to talk about what I did than to do it.
This is absolutely incredible. A few years ago I could never have imagined that such a thing would ever be possible.
Now, it's not just possible but something I was able to throw together in about 2 months.
Amazing work! Thanks for taking the time to document in video form as well. ^-^
That is the longest part.
I just watched this video now. What you do is beyond expectations! Thank for share your work!
Thanks for the view!
incredible stuff. love the creative interfacing
Jebus Craps, that's impressive. The amount of effort you (all) went through is astonishing!
Little pieces. Little Pieces.
the register bruteforce hacking is so painful and yet awe-inspiring to read! i had a thing that could've used something like your long session but it was just too frustrating.. great job on this project!
(more specifically, using both DACs on the ESP32 in accursed ways, there's some bitluni videos to watch on surrounding things if you haven't)
I haven't yet needed to use the DAC, is there some trick to it?
@@CNLohr my memory is hazy but yes. bitluni bitbanged composite video along with audio on the ULP in separate videos and i was hoping to combine those to get a cute little application platform, pico-8 esque but in hardware -- after brushing my head into it for a while i halted the project pending access to an oscilloscope and similar nice-to-haves
i love writing programs for machines where i'm close to a piece of relatively-simple hardware that could conceivably be understood by one person, the low latency feels great and the security in not having the rug pulled under me too and it's expensive but i'd like to try again eventually.
emulators are okay and less e-waste is great, but if they aren't perfect emulators then it's only added more layers of stack to pop and comprehend, which is not okay for an experimental, toy machine. i want to have a bit of me embodied in every layer of it maybe?
Honestly this video is so cool the thumbnail doesn't do it justice.
Any recommendations for the thumbnail? I struggled to make this one.
It literally makes my week to see these project breakdowns. So inspiring.
I think "breakdown" is a good way of putting it. Because all the individual pieces are so simple.
I'm glad I'm subscribed to this channel. You always show off the coolest (most bodge-y) projects!
Thanks!
Mind Blowing!
Thanks!
Man, you are smart. Smart and creative.
Thanks!
this is the kind of stuff i wish i could make some day, interconnecting so many little things and projects youve made over the years to make a really cool thing
If you just start making today you'll have a library of powerful tools tomorrow.
You've really outdone yourself, Charles! What a fun and impressive project 😁
It took a lot of disciplines, but more importantly, it took dozens of people, all doing their own parts to make it happen.
Very cool! I loved being able to contribute even a small part of this. I look forward to your next project!
You know I have a discord! I talk about my projects while I'm working on them in case someone can help.
This is incredible, really pushing the esp32's limits. awesome work
I'm sure in 20 years people will be doing far more than this!
when I was subscribed this channel, I did't expect to get infomation that VRC intergration with esp32 from this channel. but these info are what I need. cool!
If you're into both you should do projects with both, too!
Amazing video. I love these little cpus, esp32s2 are great.
I know!!! They're so much more powerful than people give them credit for.
This is incredible... I wish to become as good of a programmer as you are, this made me feel as if nothing is impossible in code...
It is surprising how much more powerful a system that can execute code is than systems that are hard-coded into gates. It is really surprising just how much can be done with just a little code.
This is basically the epitome of something I'd want to also make, so it's amazing to see all the work done and explained so succinctly. Amazing work. I remember even just trying to get low-latency streaming working with my own RTMP server, learning later about RTSP, as well as trying to consider encoding delays. This type of ultra low-level programming and hardware solutions are mindblowingly awesome, and only makes me more sad that VRChat and many other games run the way they do.
RTMP is just an awful protocol no matter how you spin it. Don't feel bad on that front. Keep researching.
I see the notification for this vid and am like "Oh goodness Charles' at it again". not even 3 minutes into the vid I'm already amazed and haven't even _seen_ the main point of the video yet
Well it's been an hour. How did you like it when you did see the point?
@@CNLohr Even more, awesome video as always!!!
This is amazing. Incredible
Thanks!
I've been working on a wireframe flight simulator targeted at similar hardware on and off for a long time, super excited to look at the source code! Very cool the way you allowed these devices to interact as well. Fantastic job.
Thanks. And the code isn't laid out great, but it should give you some building blocks.
This is amazing I’m in awe of what you were able to do.
It was a big team effort.
I wish my projects were half this impressive. Keep up the good work!
Amazing project, would never have imagined that was possible
My projects stand as an example of what you can do. Not necessarily what you should do.
Never fail to amaze.
Thats really insane! Good job!
Thanks!
Everytime you post something, you show another unbelievably insane project.
Btw have a look at standalone VR headsets (Quest 2, Pico 4 They cost like 400~500 and can run VRChat without the need for any other hardware, though I guess you probably know that already).
You could make even more intervined VR projects with that. The headsets run android and can communicate via serial over usb-otg with a microcontroller. So hooking up an ESP32-S2 to work as an espnow receiver/sender would work. Also the Quest 2 in particular has kind of good handtracking and a black and white AR-mode.
I've tried using VRChat on stand-alone and it is just so utterly uninteresting to me. Running it on PC for me is the difference between looking at a VHS video of an event and being there. :-/ Maybe the next round of stand-alone.
@@CNLohr ah okay, yeah due to the way vrchat works performance is kinda meh on mobile
Amazing stuff, never heard of Magfest untill this video.
Exceptional job, very impressive!!
Should check it out. It's a really cool convention.
@@CNLohr I since have, and it is very cool indeed!
This is really cool, way cooler than any of the Arduino video game projects I made in college.
Same here! The difference is I didn't stop making projects after college.
Amazing stuff. It's like a master class in problem solving 101.
Or in how to create more problems than you ever would imagine :)
This is insane!
It's what I strive for.
This breaks my brain, so many software and hardware stacks to make this happen 🤯 👍
I know it's a lot of stacks, but when you break things down, and don't let them be complicated, it becomes pretty straightforward.
I can not believe the incredible amount of work that went into making this amazing, I don't even know what to call it, system? gadget? interface? Amazing job!
I'm not sure what I'd call it, either.
Every video i say the exactly same words i sad long time ago, when i subscribe to this channel .. ."this guy rock!!!"
Thanks!
You absolutely awesome legend are still uploading! 😍😍😍😍😍😍😍
16 years and counting!
What a dope system! Integrating the ESP32 like that is awesome and gives me hope for some really awesome spacial XR experiences in the future. I wonder if the ESP32 S2 supports Wifi RTT for some wifi-based localisation systems, allowing to integrate the position of the devices into the experience.
The S2 supports 802.11mc FTM
Thank you for being a wizard, dude.
🧙
Great to see your videos again !
I am going to try to make one every 2 months.
@@CNLohr that's great news for me !
Once I helped you with a little bug during a Livestream and I felt great :)
@@eVITORIOe probably no more of those because of TH-cam's copyright music issues
I saw the future in your video, thank you for sharing
Now go. Build it.
This video blew me away, no joke
Just a bunch of small pieces.
14:34 - Zuckerberg, remember to apply ointment to the burn area! :D
because you are a genius mate.
I hoped you'd eventually do something like this ever since your videos on CNOVR! Your hacks are truly legendary.
I spent a LOT of time in VRChat. A lot of my research went into worlds in VRC.
@cnlohr few days ago I found your espusb project while trying to find a way to implement usb mouse and keyboard funcnality to esp8266 . Can you please make a video how anyone can customise and use espusb project
This video melted my brain oh my god this is super cool!
太逆天啦，手搓各种最原始的协议，大佬牛皮
wow impressive!, I love the data encoding
That was my favorite part, too!
This is amazing creative work
Thanks.
This is incredible, holyy
Such good stuff!
this is really impressive, although for some reason i expected a headset/position tracking to be involved
The video was a little click-baity.
Holy shit... This is absolutely epic
will this be done in future MAGFest as well?
We will probably be doing the portal, but probably not as ambitious of a cross-play next year.
Ah, here is he going again...
Doing magic...
Charles is one of those people that find rabbit holes within rabbit holes.
"We must go deeper."
@@CNLohr you my friend, you don't go...
You are basically free falling 🤣
You can open up a club together with Sebastian Lague, Stuff made here and the Thoughtemporium lol
@@DerSolinski oh I'm not as smart as them
@@CNLohr Don't downplay your self.
I respect the persistence to finish stuff you guys do.
I'm the sort of guy who starts projects just to abandon them on the last mile...
I get to easily distracted, and lose interest after a while.
@@DerSolinski gotta drive en to the end!
Hey i know this is an older video but im very curious to know how you find all of the surrounding ESP mac addresses to send data to and receive from. I had a look through the code but couldn't find where that happens. Could you explain briefly how you accumulate those addresses or discover them, or even point me to the function that handles that! Awesome project btw
With ESP-NOW, you can send to "everyone" by broadcasting to FF:FF:FF:FF:FF:FF, then when receiving a packet you can decide what to do with it. So in this project, everyone just sends everything to everyone. The main node sends info about the VRChat players, the individual players just info about their ships/boolets.
Mind blown 🤯
do you take comissions for technical projects in VRC? I want to hire someone to help make a working book that can be pulled out of an avatar or area of a world and flipped through, probably having multiple books contained in the same one, having the main book change aspect ratio for the PDF loaded (they could be stored as relatively low res textures?)
would want to eventually be able to pull out pillows, incense, books, and maybe even a basic music player all from an avatar to put in public and private worlds to hold spiritual programs
I regret that I do not take commissions, but, if you join my Discord, there is a #vr channel, and there may be someone who can point you in the right direction.
"Yes, even this potato of a game system supported legs." xD
I had no idea you were in the area, neat
i normally am on the other side of America.
I was wondering how you got the skeletal data onto the swadges! I didn't know you had the full data
I talk about that later in the video.
This is mind blowing
I just love showing people _what is possible_
Goddamn dude, you're like a tech god. When you're in your 60s your gonna have nobel prizes and shit forsure.
Honestly, just a patent or two is all I've ever hoped for.
Insanity trade off, impressive
Insanity?
This is Killer game of it made commercially, I always dream of network shared small gaming device that player can get into easy and fast.
This is crazy dude
Thanks!
Bro if this was a thing i could buy, id buy it on sight of release.
sorry if im missing something, but couldn't you get around the performance issues of gpu -> cpu data transfer issues by rendering your data somewhere on your screen and then grabbing that from an external program? Or was that data only local to users machines in vrchat?
Sick project though, looks like so much fun :o
Sure! But then users need to run stuff locally too! But soon async readback will exist.
BONKERS, as always
Thanks!
awesome!!
03:49 - What font is this?
Asking the real questions XD
Is there anywhere to go from here? The dude defeated technology with this one!
audiolink.dev
Something entirely different. Or the same. Who knows.
14:36 laughed so much on this!!!
How compressed can you make a game 101 right here
I also have a question this is coding efficiency right? if so is it possible to make a bigger game for modern computers with those tricks
You can make games for modern computers this way. It just is a trade off between expedience and effectiveness. As soon as you use an abstraction you trade some efficiency for coding expediency. Like if you use a library to do something. But the issue is they also made those trade offs. And they use abstractions. And so on and so on. And eventually in modern computers, we generally achieve
so how can i connect to the rave world again? :)
Which one?
Well done!
Thanks! hopefully I'll be able to apply this more in VRC.
This is incredible! how many hours did it take to design and implement?
Depends on how you count it - EVERYTHING, thousands, if you include all the work on the ESP, rendering systems, . But just this part of the project shown probably only about 250.
Hi could you join stereo channels together next time?
Oh? This was the first video where I didn't. I preferred it on my system. What is the issue? Or is there anything I can do to make it better?
@@CNLohr In my experience listening on headphones the audio was pretty inconsistent and sometimes disorienting
@@jakeyyyyyyyy OH!!!! I didn't think about how voice-overs would work with that. I can be more careful in the future.
thats some crayz shit - gg
thanks