that's super cool ! it's amazing to see how far you're pushing your voxel engine, it's one of the most impressive ones out there :) (especially since i think you're the only one i've seen who actually invents new graphics techniques, that's genuinely so cool)
Veloren Easter Egg at 7:05? FWIW that model uses our 'index replacement' system: to properly render it into the world, you'll need to swap out voxels with the inner block's index with an empty voxel. Regardless, great changes. It seems to me that you should be able to store not just the whole edited chunk, but only the diff: and then automatically save LoD versions of that diff so that they can be applied live without the downsampling step, meaning that you don't revert to pessimistic behaviour for worlds with many player edits.
Awesome as always! As soon as you mentioned using the seed/world generation data to generate LODs, I thought "what about player generated structures!?" I like your solution. Much more seamless than the LOD Minecraft mods I've seen, which only generate LODs on the client side, and can become out of sync with the server. It's coming along great!
I've been following this series since the beginning and the project is getting better and better every time. Would be cool to see epic biomes and buildings using this engine!
you could trace though the transparent material. It seems to have a bright color on the edges. You could combat this by making the color contributed by the transparent voxels dependent of the "distance" to the backface (by raytracing or just rendering again with the backfaces).
I started learning rust because of you. I was skeptical of how will this project be useful, but now seeing the economy, and that this engine runs in the browser near native speeds, it's safe to say this could be an engine that a new runescape game uses - or something that made even more money and ran in the browser.
I forget what method 7 Days to Die uses Voxely - Octree type destructive environments... But once players hit that level and are able to go ham building wise and what not. I'm always seeing this where a building wont show up till your really close or some building that has been totally torn down shows up as you get further away from it. Great stuff you got going on here!!
Oh, is the model from Veloren? I really had no idea haha, it was just a random file called mage_tower.vox that one of my viewers sent to me. (I think that I recognize you from Tantan's game jam video, by the way - nice to meet you!)
Oh yeah, but feel free to use it. Keep in mind though that those assets are licensed under GPL3. The blocks sticking out from the door at the tower etc, are turned into air blocks in veloren and override terrain when placed. And hi!
An interesting solution for physics could be to have each voxel have a density value. So when a tree falls, its trunk has a density of 1.0, but the leaves have a density of 0.25, and voxels of lower density can combine on collision up to a maximum value of 1.0. So it would look like the leaves are compressing together when the tree falls. If you restrict it to only compressing with other voxels of the same material, it would be easily reversible. Leaves have a density of 0.25, and will revert to that when no forces are applied, but forces applied can increase that density to 1.0 It could allow for some interesting effects
That's an excellent idea! For performance reasons, I don't plan on supporting something quite like this - it would probably require some kind of per-voxel simulation, which can be pretty taxing on the CPU.
I think you are forgetting about another problem with LOD and editable terrain. From what you said you only send out updated versions of LOD data when another player requests those chunks (which is definitely a great choice for performance). But what if they don't? What if they stand still, never requesting new data from the server? Also great to see, that you finally fixed your weighted transparency. It looks a lot better now. But honestly I still prefer the things you can do with ordered transparency ;)
Ordered transparency lends more flexibility. The volumetric fog effects in Cubyz are amazing :) Anyway, that's a detail that I glossed over. @punchster289 is correct; when a chunk is dirtied all nearby clients are notified. This causes the clients to discard those old LODs and re-request them from the server. This ensures that LODs always remain properly up-to-date.
@@DouglasDwyerhonesty following your engine development has been really enjoyable. your solutions are extremely clean and logical, and your videos are high quality. once the engines done, i hope you enjoy making what will inevitebly be an extremely unique and intriguing game.
@@DouglasDwyer Do you think it's possible to defer these updates so they don't happen every time an edit is made? I can see that being a bit of an issue if multiple players are building near you, forcing your client to recompile the LODs constantly. Sorry if you've already considered this/don't need it, it was just a question that popped into my head during the LOD section.
Haha, I'm just trying to keep up with everyone! Your voxel project seems pretty cool too - the non-Euclidean rendering is quite different than any other engine out there.
I've been meaning to start on a voxel game pretty soon, and I have been thinking of a way to do this and I am curious what others might think. I was thinking that the best way to do this would be to create a chunk loading system similar to minecraft with 16x16 chunks, but each block within a chunk can be optionally divided into 16x16x16 sub chunks. Each block can have a divisior variable that decides how many sub voxels to use. That way I don't need to code all those blocks in areas that are solid material and materials like stairs don't need to use all 16x16x16 voxels, maybe just divide it into 3x3x3 for all the steps. When loading chunks near the player, I would like the game to first load the 16x16 chunks, then add the sub chunks near the player. Chunks that get further away would only allow smaller divisor values to create this level of detail and as the player gets closer, the full divisor can be requiried. I worry that reloading variables like this may not be the most efficient way to do things though and things in the distance may look too blocky.
One more thing about the divisor, I intended to have full solid blocks have a divisor value of 1, and the full 16 voxels would have a value of 16. In the event that a player is looking at a chunk made up of blocks with a divisor of 16, but the game only allows a divisor of 4 at that distance, every fourth block in that subchunk would be stored into memory and as the max divisor value increases as the player gets closer, every second block would be stored into memory, unless another player has updated the chunk, but only alternating second blocks would be queried because half those blocks are already in memory. Feel free to use this idea if it sounds like it would work.
I think that this is a great idea, since it skips processing redundant data (i.e. for normal blocks, you only process the entire block, nothing smaller). Most voxel data representations employ a compression scheme like this, in one way or another!
Another improvement to solve is grouping voxels next to other voxels with same material and make a mesh with the same border, that should draw less triangles than all its visible voxels faces.
Nanite and my LOD system serve similar purposes. But LODs are a much older technique than Nanite! Nanite does some very innovative things (like GPU data streaming and software rasterization) in order to achieve performance on high-end graphics cards. It's cool, but I would say different than this :)
@@DouglasDwyer it's still a cool technique non the less definitely similar to nanites though, im a gameplay programmer and game designer so I don't deal with the lower level issues like rendering, however I've always admired people how can do the lower level stuff, for me I don't even know where to start learning that type of stuff, was hard enough learning how unreal and godot works
Adding an atmosphere and volumetric clouds would be nice, but I'm going to leave those tasks to a real artist haha. I need to focus on functionality first!
@@DouglasDwyer Volumetric clouds would be a little overkill, I was thinking maybe a second layer of clouds that are the same as the bottom layer but slightly smaller and a lighter color to imply thickness. But yeah it's a nitpick and definitely shouldn't take priority
You can consider using something like Marching Cubes to make LODs render with even less triangles. At a distance, this effect should not be noticeable. But I understand if you'd want to stay away from this, as the LODs would technically no longer qualify as voxels. Maybe something like this: th-cam.com/video/3gEIUaZ05Ec/w-d-xo.html The benefit being that you could increase the render distance further.
It's a good thought. I don't plan on doing this right now, however, since it would require adding a completely different rendering pipeline for the triangular meshes :)
Awesome progress! However, I'd like to mention something about the chunked LOD approach. There are much better alternatives to octrees. Octrees are a very slow data structure, reading and writing. Especially if most simulation stuff occurs at the highest detail level. They're also a lot more annoying for multithreading, potentially causing a lot more contention. Furthermore, they're not very flexible. At any given spot you can only have ONE LOD type loaded. This is problematic... e.g., for your dirty approach, if you could just keep certain LODs loaded into memory anyways (as needed), then you can very quickly update them (in conjunction with the dirty approach). Among other issues like having to make significant changes to the octree just by moving, you get the point. The most simple data structure that solves this is literally just do what you did before but duplicate it for each LOD level. So either a flat array (good for client-side) or hashmap (good for server-side) in each level. In such approach, simulation stuff at the normal LOD has no performance impact, obviously much less contention too. They're much more flexible, you don't *need* to have a single LOD loaded in any given location. The only time you need to ensure this invariant is for rendering -- which is easy. In my engine, I actually use this potential overlap as a very natural way to transition between the LOD levels. Although, I'd say the best thing about this approach is that it's simple! Less moving parts.
Thanks for the suggestion! I concur. The octree explanation was mainly to highlight the way that LODs are organized into a cubic hierarchy. Internally, I do use hashmaps to store the LODs that are loaded. Although, I don't store any LODs that aren't currently meant to be shown. That's an interesting idea :)
@@DouglasDwyer That's great to hear! Also, you may want to clarify that somewhere. Doubtless a new voxel developer would be inspired by your videos and may implement octrees instead (It doesn't help that chunked octrees are often recommended online).
Haha, well I actually do use octrees for the representation of the voxel data itself! My voxel octrees are very battle-tested and robust at this point, and so I'll be keeping them for now. But I won't try to argue that they are ideal. I want to experiment with other formats in the future, so that I can support per-voxel color and normal data.
Great video as always! Out of curiosity what rendering method are you using? Are you still doing parallax ray marching or have you switched over to greedy meshing?
In this video, I was using fully greedy meshing. However, if you check out my latest video, you'll see that I've switched my rendering technique once again :)
Hi Douglas! Great work as always. I began following your channel about a month ago and have binge-watched all of your videos. I’m fascinated by your work but there is something I don’t understand and wondered if you could explain to me: I get that SVOs are compact, but your voxels are quite a bit smaller than those of - say - Minecraft and so it follows that you must have many *more* voxels than Minecraft for the same area (as each increase in voxel resolution, cubes the number of voxels required). So I’m wondering how on earth you manage to increase the voxel resolution as much as you have and yet still be able to store that many voxels in memory. My understanding of SVOs is that each child node is a single bit of an 8 bit parent and subsequent children are themselves single bits of 8 bit nodes…but the *type* of voxel has to be stored somewhere (otherwise all voxels would be the same type, right?!) and so that voxel id has to be stored somewhere, increasing the amount of memory required to store the voxels…and as you have *so many voxels* in your higher voxel resolution set up…it’s mind boggling to me how you’ve achieved it! Maybe I’m being dumb but however I run the numbers, storing the visible part of a Minecraft like world (high voxel resolution equivalent of 32minecraft “chunks”) quickly enters the hundreds of gigabytes realm…
Hey, thanks for your interest! The idea of a sparse voxel octree is very general, and they may be structured in different ways (including using bitmasks, offsets to children, or pointers to children). In my implementation, I store 16-bit material IDs in my octrees. I may, however, iterate and switch to another data structure in the future to better support per-voxel colors and normals. I think the key part that you may be missing is the "sparse" in sparse voxel octrees. If a node in an SVO is completely homogenous, then the children of that node are not stored. As such, large regions of empty space, grass, dirt, and the like don't take up much memory. Really, it's only surface voxels (or any voxels on the border between regions of differing ID) that take up appreciable memory, since there are far more of them than non-leaf nodes. As a simple example, let's imagine that we had a 32^2 world with chunks of 256^3. As a rough estimate, let's assume that there are 6*256 surface-visible voxels in each chunk (about the number of voxels that would be visible if the chunk were a solid cube). Then the amount of memory required is about (32^2 chunks) * (6 * 256^2 voxels per chunk) * (2 bytes per voxel) = 805 MB, a very reasonable size. This is neglecting the extra data necessary for the non-leaf nodes, but those could not increase the size by any more than 1/7 or 14%. For more information on SVOs, maybe this video will explain them: th-cam.com/video/gNZtx3ijjpo/w-d-xo.html
That looks insane! Im goint to have to look at order independent transparency for my engine, definitely goong to make things a whole lot easier. One question i had was will you publish the engine source code as open source?
I eventually hope to turn the project into a publishable game or service, so I don't plan on open-sourcing the complete engine in the near future. However, many custom components of the engine (like the event library, networking library, etc.) are available and open-source on my GitHub!
Adding an atmosphere and volumetric clouds would be nice, but I'm going to leave those tasks to a real artist haha. I need to focus on functionality first! Great suggestion :)
How does the downsampling itself work? How do 8 voxels turn into 1? Or are you doing downsampling on something other than voxels, some intermediate format? What would happen, for example, if I built a wall in a checkerboard pattern, where every other block is air? What would be the LOD representation?
Excellent questions. I am downsampling the voxels directly, so there may be some information loss. The system is just programmed to pick one voxel of the eight (preferring non-air voxels if possible) and use that as the representation. So, if you built a checkerboard, it would turn into a flat, opaque plane in the LOD representation.
@@DouglasDwyer Sometimes I wonder whether it is possible to make an LOD system from *pictures* instead of voxels. As in, the game takes a hundred pictures of the chunks in question from different angles, then JPEG-compresses them into pretty much nothing (the WORST compression quality) and displays to you a billboarded picture, chosen according to the angle you're looking from.
@@DouglasDwyer Come to think of it, another question: are "partial" voxels completely forbidden in your system? You know, like Minecraft's slabs/stairs/panes? And what about transparently textured ones, like leaves?
You're not the first person to suggest pre-rendered images as an LOD system. However, I think that it would be difficult to integrate changing sunlight shadows into them; you would need to re-render each time. Yes, the system does not have support for partial voxels of any kind. I could support transparently-textured ones easily, but I don't plan to do so. My goal is for the voxels to be small enough that detailed models can be created with them.
@@DouglasDwyer may have commit missunderstanding but it sounded like if a chunk is altered in any way the game skip it and all its octree neibors in lod pass
Since the graphics rewrite I am still missing some keys things (namely entities). Once those features are added in, I am going to post a new version of the demo. Probably next video.
Adding an atmosphere and volumetric clouds would be nice, but I'm going to leave those tasks to a real artist haha. I need to focus on functionality first!
Oh man, how I wish that this code were open source... I am so interested in how you do the meshing, as I understand you're not doing raytracing/raymarching but greedy meshing. It's so incredible impressive and sadly there is no open source implementation to peek at (I am a completely different dev by day, so I start from scratch with games)
Thanks for your interest! I would love to share the code with the world, but my ultimate dream with this project is to turn it into a game or product that I can work on professionally. The only way I can imagine this ever becoming my full-time job is if I keep the code proprietary.
@@DouglasDwyerI understand that sentiment, but I still hope you can open source some chunks... I am not trying to convince you, you've set your mind and it's your decision to make, but I personally think that while open sourcing has some risks (and it does!) you could use a license you could theoretically enforce in Court. But nobody but judges and maybe lawyers wants to go to court, so I guess my main argument would be: You're making that game. Even if you open source the technology (or parts) behind it, nobody can take your game from you, your special idea. An engine doesn't make a fun game, so even if you open sourced and some people made some cute demos, unless someone really puts in the ground work for making an actual game, it wont hurt you. But again: You've made up your mind and that is fine, I just wanted to share my perspective, because over the years I've seen too many great ideas never being finished and not open source, so nobody even had any chance to complete them or give them new life.
The LOD system is designed to always select opaque voxels while downscaling. So it's impossible for walls to disappear, although if you had a wall built of thin layers of different material, those inner materials might show through.
will you/would you release your engine (whenever) as pure engine? ... like minetest leans more into being an engine than minecraft? I am insanely impressed of what your engine is capable of.
Thanks for the kind words. My dream is to release the engine as a platform (like Roblox or Fortnite) where users can code their own experiences. This choice is compounded by the fact that the engine is quite special-purpose; for example, it only supports voxel volumes with 16-bit material IDs (no other kind of per-voxel data right now). Maybe someday I'll work on creating a more "general-purpose" voxel engine with the knowledge that I've gained, and release it as a pure engine.
Hey Douglas, does Webgpu support mesh shaders? The Nvidium rendering engine for Minecraft has achieved some crazy results with the technique, (it only works on Nvidia because only Nvidia supports mesh shaders on OpenGL.) Could be something to look into, I know you said at the end you were done with optimization for now but this could be a tempting route, haha.
WebGPU doesn't support mesh shaders, unfortunately. I think that when I do make another voxel renderer, it will probably be using ray marching again - so mesh shaders wouldn't be directly applicable :)
I watch a lot of voxel engine devlogs, and every time LOD is brought up, I have the question: wouldn't it be better to render polygonal LOD's than voxel LOD's? When far away, a voxelized hill will appear to be smooth. With voxel LOD's, voxels get bigger as they get further from the camera. Doesn't this mess with the player's perception of the distance to faraway objects?
You have a point. Technically, with "perfect" voxel LODs you should only switch once a single voxel is smaller than a pixel onscreen - this ensures that the user cannot see any difference in quality. But in practice, this is not feasible (at least in my engine) and the LODs need to appear sooner. I don't think that the perception of distance is damaged, since the faraway voxels are still textured normally (so a single voxel in a 1:2 LOD will look like it is composed of eight individual 1:1 voxels). But using polygonal LODs could perhaps yield better visuals in this case. The implementation would be complicated in that it would require a completely different rendering path for the LODs, which is why I opted for blocky LODs.
The approach we take in Veloren is to use a shader that emulates the behaviour of voxels on LoD objects. It can be quite convincing, even relatively close to the camera.
So you edit the lods when rendering ? I'm afraid it might cause lag spikes. If the player builds a whole city, and then teleports away to look at it from a distance, the engine will have to calculate all the changes at once. Instead, you could have your engine calculate the changes to lods while editing. It unfortunately means some changes will go back and forth (player unhappy with the result -> destroy their build and make something slightly different...), but the work of the engine will be smoothed over the time it takes to the player to edit, which should be long enough, right ? Also, if a specific lod receives no change, lower resolution lods don't either, I guess ?
Not quite. The LODs are recalculated on the server thread (which is not tied to the game's framerate) when the client requests changed LODs, and the work is spread over the course of multiple server ticks. As such, there is no lag!
GPU instancing is a useful technique for certain scenes, but all voxel data is unique here. As such, I don't think that there is a place to apply instancing (which requires having multiple models with the same vertex data).
Hey man! Very cool video! I have a question. I'm developing a game, I've been working so hard on it for the past two years. My vision is to make a voxel world, which is a recent decision. I've been using Zbrush to sculpt all of my models so far which I've put so much time into and I have a good workflow going. I've been using blender to convert my models into voxelized models and I've been testing the world I have built into unreal engine. My problem is that the world I have made is 26 mil poly's when it's converted into voxels and it just won't load into unreal engine 5... It always crashes. So I used a less detailed world, but it looks like minecraft which I really don't like. I was also going to do pose to pose animation to achieve a certain look. I guess my question is: Is it possible to create a voxel engine so I can have a super detailed world in which I can import OBJ files and the program (Or engine I should say) would convert them into voxelized models? I want to build my own engine too because I just think it would be cool to have only my logo, not unreal, pop up on the startup of the game. I also want a specific feel to the gameplay and I worry about using unreal, I know you can control that, but I wonder how much control you actually have over how the game feels to play.... Also I want to keep all the voxels uniform.
It sounds like you want to create a game utilizing voxels mainly for the art style? If so, have you considered using a shader or post-processing effect to turn models into voxels? Such solutions do exist, and that would allow you to continue the same workflow with ZBrush and Unreal while achieving that voxel look :)
Haha. In a sense, it was true - I had hit the limits of what felt possible with rasterization. To overcome those limits, I had to start a new renderer :)
Excellent question. Voxel engines are a complex topic, and I've built my knowledge from experience, looking at other engines, and talking to the community. A good place to start if you're looking to learn 3D rendering is to follow the learnopengl.com tutorial series, or watch some online videos about how to make a Minecraft clone. Once you feel ready to tackle something more advanced, the voxel.wiki website looks like it has some good links to resources. As for physics, I still have a lot to learn in that department. I'm planning to read Millington's "Game Physics Engine Development" since I want to integrate a constraint-based solver for my engine. Overall, the best teacher is experience! Pick a project or goal that you'd like to achieve, and pursue it at all costs :)
It depends upon the language/tools that you want to use, but the tutorials by Low Level Game Dev, b3agz, and obiwac all look good! As for John Lin, indeed, you would be hard-pressed to find someone in the voxel community who **hasn't** heard of him. His work was an inspiration to me and many others! If you want to create something as visually involved as Lin, you'll definitely need to use ray tracing, so I would recommend looking more into that.
@@DouglasDwyer thanks for the replies! does he share his voxel project on his github? and is there a difference in voxel size that he uses compared to what your doing? like with making the trees and grass?
I don't see why you need an octree, that just seems like a lot of memory allocation every time the player moves, you need to recalculate the octree, don't you? If you're already looking at each chunk and figuring out where it goes on the octree by distance, why not just use that distance instead of storing it in an octree?
I'm not really looking at every chunk - the octree structure itself is what encodes the position of the data and helps me determine what needs to be loaded. That said, the mention of octrees was meant mostly to explain the hierarchical way that the LODs are organized. I actually use hashmaps to store the data internally, which are a bit more flexible.
@@DouglasDwyer Ah, I see, in the few implementations of voxel LOD I've done, it's just a per chunk thing, and so the minimum LOD is dictated by the chunk size. However, both were finite, and more of a simulation type thing, so the entire world has to be small enough to fit into ram anyway so the LOD was only for GPU memory, not ram. But for yours it's both because your voxels are so small. I think I see now, very cool! PS sorry for rambling I was just playing DnD and I'm a little faded haha great video though man
That's very kind of you! But I am not yet ready to promise anything regarding this project, so I am not accepting donations. You can support me right now by subscribing and continuing to follow my work.
@@DouglasDwyer Understandable I would have given you money anyway even if you couldn't guarantee anything for the project. But I understand you don't want to take money from people if you cant promise anything, I'm like that my self.
Yep, that's my understanding too! I would guess that he still has implemented a custom data representation and LOD system for his voxels too, so his work is impressive from both a technical and visual standpoint.
@@DouglasDwyer I still want to know what he did. I tried Voxel Plugin one time and it was kinda janky. I really wanna make a voxel game (maybe also engine) but Im a noob in every engine and I have good understanding of C++ and no understanding of Rust but Rust is the objectively better choice...
I use WGPU, which is a graphics abstraction layer that allows me to target many backends (including Vulkan)! However, I target DX12 on desktop Windows since swapchains are less buggy in it.
I do not understand why anyone would ever use voxels, the graphics sucks so badly, so even what gameplay they might add will not matter in the long run.
![Emissive voxels and fancy lighting [Voxel Devlog #19]](http://i.ytimg.com/vi/VPetAcm1heI/mqdefault.jpg)
![Emissive voxels and fancy lighting [Voxel Devlog #19]](/img/tr.png)
![Coding rigid body physics for voxels [Voxel Devlog #20]](http://i.ytimg.com/vi/byP6cA71Cgw/mqdefault.jpg)
I love that you featured Ethan, he’s one of my favourite indie voxel devs
that's super cool ! it's amazing to see how far you're pushing your voxel engine, it's one of the most impressive ones out there :) (especially since i think you're the only one i've seen who actually invents new graphics techniques, that's genuinely so cool)
Thanks Dotted!
Seeing de_nuke at the end is truly surprising. It works so well with voxels.
Veloren Easter Egg at 7:05? FWIW that model uses our 'index replacement' system: to properly render it into the world, you'll need to swap out voxels with the inner block's index with an empty voxel. Regardless, great changes. It seems to me that you should be able to store not just the whole edited chunk, but only the diff: and then automatically save LoD versions of that diff so that they can be applied live without the downsampling step, meaning that you don't revert to pessimistic behaviour for worlds with many player edits.
Awesome as always! As soon as you mentioned using the seed/world generation data to generate LODs, I thought "what about player generated structures!?" I like your solution. Much more seamless than the LOD Minecraft mods I've seen, which only generate LODs on the client side, and can become out of sync with the server. It's coming along great!
This is a really cool and exciting project! It's definitely one of the most impressive indie voxel engines out there.
I've been following this series since the beginning and the project is getting better and better every time. Would be cool to see epic biomes and buildings using this engine!
Thanks! I do feel that some better art (more detailed terrain and assets) would really take things to the next level.
Man casually shows Nuke without any mention 😮
I don't remember seeing anything interesting other than the terrain with trees so far...
Haha, I upgraded my model loader and wanted to show off how a larger model looked with the new render distance.
Just wanted to say this is *astounding* work! Really, really, *really* impressive stuff!
Cool :D I was curious how to handle player-created structures in a LOD system, and your explanation made a lot of sense.
you could trace though the transparent material. It seems to have a bright color on the edges. You could combat this by making the color contributed by the transparent voxels dependent of the "distance" to the backface (by raytracing or just rendering again with the backfaces).
For performance reasons, I don't plan to use ray marching or backface rendering right now. But those are assuredly good suggestions :)
I started learning rust because of you. I was skeptical of how will this project be useful, but now seeing the economy, and that this engine runs in the browser near native speeds, it's safe to say this could be an engine that a new runescape game uses - or something that made even more money and ran in the browser.
You should do a collab with ethan gore, his game has very good render distance, it's like seeing 50 thousand earths at the same time no joke!
Man this project is amazing, can't wait to see more updates!
Glad that you enjoyed it!
Ethan Gore's engine is crazy
real
I forget what method 7 Days to Die uses Voxely - Octree type destructive environments... But once players hit that level and are able to go ham building wise and what not. I'm always seeing this where a building wont show up till your really close or some building that has been totally torn down shows up as you get further away from it.
Great stuff you got going on here!!
Just wanted to say I really enjoy your videos :)
Saw the tower at 7:06 and was like wait a second... (I'm a veloren coredev)
Oh, is the model from Veloren? I really had no idea haha, it was just a random file called mage_tower.vox that one of my viewers sent to me. (I think that I recognize you from Tantan's game jam video, by the way - nice to meet you!)
Oh yeah, but feel free to use it. Keep in mind though that those assets are licensed under GPL3.
The blocks sticking out from the door at the tower etc, are turned into air blocks in veloren and override terrain when placed.
And hi!
love these videos, thanks for spending the time putting them together
Thanks for appreciating!
Awesome Work! keep those voxels rockin!
An interesting solution for physics could be to have each voxel have a density value. So when a tree falls, its trunk has a density of 1.0, but the leaves have a density of 0.25, and voxels of lower density can combine on collision up to a maximum value of 1.0. So it would look like the leaves are compressing together when the tree falls.
If you restrict it to only compressing with other voxels of the same material, it would be easily reversible. Leaves have a density of 0.25, and will revert to that when no forces are applied, but forces applied can increase that density to 1.0
It could allow for some interesting effects
That's an excellent idea! For performance reasons, I don't plan on supporting something quite like this - it would probably require some kind of per-voxel simulation, which can be pretty taxing on the CPU.
I think you are forgetting about another problem with LOD and editable terrain.
From what you said you only send out updated versions of LOD data when another player requests those chunks (which is definitely a great choice for performance).
But what if they don't? What if they stand still, never requesting new data from the server?
Also great to see, that you finally fixed your weighted transparency. It looks a lot better now.
But honestly I still prefer the things you can do with ordered transparency ;)
ah true. the server should send lods of distant chunks modified by other players when they are updated
Ordered transparency lends more flexibility. The volumetric fog effects in Cubyz are amazing :)
Anyway, that's a detail that I glossed over. @punchster289 is correct; when a chunk is dirtied all nearby clients are notified. This causes the clients to discard those old LODs and re-request them from the server. This ensures that LODs always remain properly up-to-date.
@@DouglasDwyerhonesty following your engine development has been really enjoyable. your solutions are extremely clean and logical, and your videos are high quality. once the engines done, i hope you enjoy making what will inevitebly be an extremely unique and intriguing game.
@@DouglasDwyer Do you think it's possible to defer these updates so they don't happen every time an edit is made? I can see that being a bit of an issue if multiple players are building near you, forcing your client to recompile the LODs constantly. Sorry if you've already considered this/don't need it, it was just a question that popped into my head during the LOD section.
@punchster289 Those words mean more to me than you can imagine, my friend :)
congrats on the sponsorship that's awesome
It's definitely a nice opportunity.
Blowing the competition out of the water
Haha, I'm just trying to keep up with everyone! Your voxel project seems pretty cool too - the non-Euclidean rendering is quite different than any other engine out there.
Amazing work!
I've been meaning to start on a voxel game pretty soon, and I have been thinking of a way to do this and I am curious what others might think. I was thinking that the best way to do this would be to create a chunk loading system similar to minecraft with 16x16 chunks, but each block within a chunk can be optionally divided into 16x16x16 sub chunks. Each block can have a divisior variable that decides how many sub voxels to use. That way I don't need to code all those blocks in areas that are solid material and materials like stairs don't need to use all 16x16x16 voxels, maybe just divide it into 3x3x3 for all the steps. When loading chunks near the player, I would like the game to first load the 16x16 chunks, then add the sub chunks near the player. Chunks that get further away would only allow smaller divisor values to create this level of detail and as the player gets closer, the full divisor can be requiried. I worry that reloading variables like this may not be the most efficient way to do things though and things in the distance may look too blocky.
One more thing about the divisor, I intended to have full solid blocks have a divisor value of 1, and the full 16 voxels would have a value of 16. In the event that a player is looking at a chunk made up of blocks with a divisor of 16, but the game only allows a divisor of 4 at that distance, every fourth block in that subchunk would be stored into memory and as the max divisor value increases as the player gets closer, every second block would be stored into memory, unless another player has updated the chunk, but only alternating second blocks would be queried because half those blocks are already in memory. Feel free to use this idea if it sounds like it would work.
I think that this is a great idea, since it skips processing redundant data (i.e. for normal blocks, you only process the entire block, nothing smaller). Most voxel data representations employ a compression scheme like this, in one way or another!
Another improvement to solve is grouping voxels next to other voxels with same material and make a mesh with the same border, that should draw less triangles than all its visible voxels faces.
Is hard to implement that on complex shapes but easier on flat surfaces
So greedy meshing?
I have a greedy meshing scheme which does this. It's part of the reason that I can rasterize these scenes efficiently!
I feel like you need to use transparency voxels in structures instead of sphears to really test how it looks
Very interesting engine, i would love to make some short games on this.
My end goal is for users to be able to build things with the engine!
Seems like you made your own voxel nanites solutions that's cool
Nanite and my LOD system serve similar purposes. But LODs are a much older technique than Nanite! Nanite does some very innovative things (like GPU data streaming and software rasterization) in order to achieve performance on high-end graphics cards. It's cool, but I would say different than this :)
@@DouglasDwyer it's still a cool technique non the less definitely similar to nanites though, im a gameplay programmer and game designer so I don't deal with the lower level issues like rendering, however I've always admired people how can do the lower level stuff, for me I don't even know where to start learning that type of stuff, was hard enough learning how unreal and godot works
Really cool, small nitpick is the clouds look really flat
Adding an atmosphere and volumetric clouds would be nice, but I'm going to leave those tasks to a real artist haha. I need to focus on functionality first!
@@DouglasDwyer Volumetric clouds would be a little overkill, I was thinking maybe a second layer of clouds that are the same as the bottom layer but slightly smaller and a lighter color to imply thickness. But yeah it's a nitpick and definitely shouldn't take priority
You can consider using something like Marching Cubes to make LODs render with even less triangles. At a distance, this effect should not be noticeable. But I understand if you'd want to stay away from this, as the LODs would technically no longer qualify as voxels.
Maybe something like this:
th-cam.com/video/3gEIUaZ05Ec/w-d-xo.html
The benefit being that you could increase the render distance further.
It's a good thought. I don't plan on doing this right now, however, since it would require adding a completely different rendering pipeline for the triangular meshes :)
Awesome progress! However, I'd like to mention something about the chunked LOD approach. There are much better alternatives to octrees. Octrees are a very slow data structure, reading and writing. Especially if most simulation stuff occurs at the highest detail level. They're also a lot more annoying for multithreading, potentially causing a lot more contention. Furthermore, they're not very flexible. At any given spot you can only have ONE LOD type loaded. This is problematic... e.g., for your dirty approach, if you could just keep certain LODs loaded into memory anyways (as needed), then you can very quickly update them (in conjunction with the dirty approach). Among other issues like having to make significant changes to the octree just by moving, you get the point.
The most simple data structure that solves this is literally just do what you did before but duplicate it for each LOD level. So either a flat array (good for client-side) or hashmap (good for server-side) in each level. In such approach, simulation stuff at the normal LOD has no performance impact, obviously much less contention too. They're much more flexible, you don't *need* to have a single LOD loaded in any given location. The only time you need to ensure this invariant is for rendering -- which is easy. In my engine, I actually use this potential overlap as a very natural way to transition between the LOD levels. Although, I'd say the best thing about this approach is that it's simple! Less moving parts.
Thanks for the suggestion! I concur. The octree explanation was mainly to highlight the way that LODs are organized into a cubic hierarchy. Internally, I do use hashmaps to store the LODs that are loaded. Although, I don't store any LODs that aren't currently meant to be shown. That's an interesting idea :)
@@DouglasDwyer That's great to hear! Also, you may want to clarify that somewhere. Doubtless a new voxel developer would be inspired by your videos and may implement octrees instead (It doesn't help that chunked octrees are often recommended online).
Haha, well I actually do use octrees for the representation of the voxel data itself! My voxel octrees are very battle-tested and robust at this point, and so I'll be keeping them for now. But I won't try to argue that they are ideal. I want to experiment with other formats in the future, so that I can support per-voxel color and normal data.
@@DouglasDwyer Ah, that's fine. To clarify I meant specifically using octrees for the chunk structure.
Crazy!
Looks great!
Thank you!
3:10, randomize trees, orientation and make more variations !
"meet gorgeous latina women", ahh yes, just what i was looking for
1:28
Next video idea: adding gorgeous latina women to my voxel engine
@@DouglasDwyer yeeeesssssss
@@DouglasDwyer 10/10 originality
this game engine i'd imagine would be amazingly fun to mod. well and maybe suffer.
Great video as always! Out of curiosity what rendering method are you using? Are you still doing parallax ray marching or have you switched over to greedy meshing?
In this video, I was using fully greedy meshing. However, if you check out my latest video, you'll see that I've switched my rendering technique once again :)
As someone interested in making 2m x 2m voxel World, this is of great interest to me because with my scale that's basically infinite distance 😂
Hi Douglas! Great work as always.
I began following your channel about a month ago and have binge-watched all of your videos. I’m fascinated by your work but there is something I don’t understand and wondered if you could explain to me:
I get that SVOs are compact, but your voxels are quite a bit smaller than those of - say - Minecraft and so it follows that you must have many *more* voxels than Minecraft for the same area (as each increase in voxel resolution, cubes the number of voxels required).
So I’m wondering how on earth you manage to increase the voxel resolution as much as you have and yet still be able to store that many voxels in memory.
My understanding of SVOs is that each child node is a single bit of an 8 bit parent and subsequent children are themselves single bits of 8 bit nodes…but the *type* of voxel has to be stored somewhere (otherwise all voxels would be the same type, right?!) and so that voxel id has to be stored somewhere, increasing the amount of memory required to store the voxels…and as you have *so many voxels* in your higher voxel resolution set up…it’s mind boggling to me how you’ve achieved it!
Maybe I’m being dumb but however I run the numbers, storing the visible part of a Minecraft like world (high voxel resolution equivalent of 32minecraft “chunks”) quickly enters the hundreds of gigabytes realm…
Hey, thanks for your interest! The idea of a sparse voxel octree is very general, and they may be structured in different ways (including using bitmasks, offsets to children, or pointers to children). In my implementation, I store 16-bit material IDs in my octrees. I may, however, iterate and switch to another data structure in the future to better support per-voxel colors and normals.
I think the key part that you may be missing is the "sparse" in sparse voxel octrees. If a node in an SVO is completely homogenous, then the children of that node are not stored. As such, large regions of empty space, grass, dirt, and the like don't take up much memory. Really, it's only surface voxels (or any voxels on the border between regions of differing ID) that take up appreciable memory, since there are far more of them than non-leaf nodes.
As a simple example, let's imagine that we had a 32^2 world with chunks of 256^3. As a rough estimate, let's assume that there are 6*256 surface-visible voxels in each chunk (about the number of voxels that would be visible if the chunk were a solid cube). Then the amount of memory required is about (32^2 chunks) * (6 * 256^2 voxels per chunk) * (2 bytes per voxel) = 805 MB, a very reasonable size. This is neglecting the extra data necessary for the non-leaf nodes, but those could not increase the size by any more than 1/7 or 14%.
For more information on SVOs, maybe this video will explain them: th-cam.com/video/gNZtx3ijjpo/w-d-xo.html
That looks insane! Im goint to have to look at order independent transparency for my engine, definitely goong to make things a whole lot easier.
One question i had was will you publish the engine source code as open source?
I eventually hope to turn the project into a publishable game or service, so I don't plan on open-sourcing the complete engine in the near future. However, many custom components of the engine (like the event library, networking library, etc.) are available and open-source on my GitHub!
hey, it looks like you've made clouds out of a single plane with a noise shader. you can try using shell texturing to make clouds look much nicer!
Adding an atmosphere and volumetric clouds would be nice, but I'm going to leave those tasks to a real artist haha. I need to focus on functionality first! Great suggestion :)
How does the downsampling itself work? How do 8 voxels turn into 1? Or are you doing downsampling on something other than voxels, some intermediate format?
What would happen, for example, if I built a wall in a checkerboard pattern, where every other block is air? What would be the LOD representation?
Excellent questions. I am downsampling the voxels directly, so there may be some information loss. The system is just programmed to pick one voxel of the eight (preferring non-air voxels if possible) and use that as the representation. So, if you built a checkerboard, it would turn into a flat, opaque plane in the LOD representation.
@@DouglasDwyer Sometimes I wonder whether it is possible to make an LOD system from *pictures* instead of voxels. As in, the game takes a hundred pictures of the chunks in question from different angles, then JPEG-compresses them into pretty much nothing (the WORST compression quality) and displays to you a billboarded picture, chosen according to the angle you're looking from.
@@DouglasDwyer Come to think of it, another question: are "partial" voxels completely forbidden in your system? You know, like Minecraft's slabs/stairs/panes?
And what about transparently textured ones, like leaves?
You're not the first person to suggest pre-rendered images as an LOD system. However, I think that it would be difficult to integrate changing sunlight shadows into them; you would need to re-render each time.
Yes, the system does not have support for partial voxels of any kind. I could support transparently-textured ones easily, but I don't plan to do so. My goal is for the voxels to be small enough that detailed models can be created with them.
great work!
Thank you!
My man, what was that ad at 1:20 ?
so how do you get the color weighting if you don't know the order of the objects? Distance to the camera is the same as ordering of objects, no?
I wonder if a nanite like approach would also work, where a object seens to get greddy remeshed when you get further from it
Nanite is a very specialized and hyper-optimized form of LODs, so they're certainly similar :)
@@DouglasDwyer may have commit missunderstanding but it sounded like if a chunk is altered in any way the game skip it and all its octree neibors in lod pass
Cool stuff.
Also, do you intend to update the online demo? I would really like to try out those newfound performance improvements on my hardware.
Since the graphics rewrite I am still missing some keys things (namely entities). Once those features are added in, I am going to post a new version of the demo. Probably next video.
voxel clouds would be cool
Adding an atmosphere and volumetric clouds would be nice, but I'm going to leave those tasks to a real artist haha. I need to focus on functionality first!
Oh man, how I wish that this code were open source... I am so interested in how you do the meshing, as I understand you're not doing raytracing/raymarching but greedy meshing. It's so incredible impressive and sadly there is no open source implementation to peek at (I am a completely different dev by day, so I start from scratch with games)
Thanks for your interest! I would love to share the code with the world, but my ultimate dream with this project is to turn it into a game or product that I can work on professionally. The only way I can imagine this ever becoming my full-time job is if I keep the code proprietary.
@@DouglasDwyerI understand that sentiment, but I still hope you can open source some chunks... I am not trying to convince you, you've set your mind and it's your decision to make, but I personally think that while open sourcing has some risks (and it does!) you could use a license you could theoretically enforce in Court.
But nobody but judges and maybe lawyers wants to go to court, so I guess my main argument would be:
You're making that game. Even if you open source the technology (or parts) behind it, nobody can take your game from you, your special idea.
An engine doesn't make a fun game, so even if you open sourced and some people made some cute demos, unless someone really puts in the ground work for making an actual game, it wont hurt you.
But again: You've made up your mind and that is fine, I just wanted to share my perspective, because over the years I've seen too many great ideas never being finished and not open source, so nobody even had any chance to complete them or give them new life.
Beautiful! But won't thin voxel walls disappear at coarse LoDs?
The LOD system is designed to always select opaque voxels while downscaling. So it's impossible for walls to disappear, although if you had a wall built of thin layers of different material, those inner materials might show through.
will you/would you release your engine (whenever) as pure engine? ... like minetest leans more into being an engine than minecraft?
I am insanely impressed of what your engine is capable of.
Thanks for the kind words. My dream is to release the engine as a platform (like Roblox or Fortnite) where users can code their own experiences. This choice is compounded by the fact that the engine is quite special-purpose; for example, it only supports voxel volumes with 16-bit material IDs (no other kind of per-voxel data right now). Maybe someday I'll work on creating a more "general-purpose" voxel engine with the knowledge that I've gained, and release it as a pure engine.
Hey Douglas, does Webgpu support mesh shaders? The Nvidium rendering engine for Minecraft has achieved some crazy results with the technique, (it only works on Nvidia because only Nvidia supports mesh shaders on OpenGL.) Could be something to look into, I know you said at the end you were done with optimization for now but this could be a tempting route, haha.
WebGPU doesn't support mesh shaders, unfortunately. I think that when I do make another voxel renderer, it will probably be using ray marching again - so mesh shaders wouldn't be directly applicable :)
I watch a lot of voxel engine devlogs, and every time LOD is brought up, I have the question: wouldn't it be better to render polygonal LOD's than voxel LOD's? When far away, a voxelized hill will appear to be smooth. With voxel LOD's, voxels get bigger as they get further from the camera. Doesn't this mess with the player's perception of the distance to faraway objects?
You have a point. Technically, with "perfect" voxel LODs you should only switch once a single voxel is smaller than a pixel onscreen - this ensures that the user cannot see any difference in quality. But in practice, this is not feasible (at least in my engine) and the LODs need to appear sooner.
I don't think that the perception of distance is damaged, since the faraway voxels are still textured normally (so a single voxel in a 1:2 LOD will look like it is composed of eight individual 1:1 voxels). But using polygonal LODs could perhaps yield better visuals in this case. The implementation would be complicated in that it would require a completely different rendering path for the LODs, which is why I opted for blocky LODs.
@@DouglasDwyer I figured it would add a lot of complexity. Thanks for responding!
The approach we take in Veloren is to use a shader that emulates the behaviour of voxels on LoD objects. It can be quite convincing, even relatively close to the camera.
my god hes gods prophet of voxel engines...
Now make it nice to look at!
Let's gooo
Cool vid
So you edit the lods when rendering ? I'm afraid it might cause lag spikes. If the player builds a whole city, and then teleports away to look at it from a distance, the engine will have to calculate all the changes at once. Instead, you could have your engine calculate the changes to lods while editing. It unfortunately means some changes will go back and forth (player unhappy with the result -> destroy their build and make something slightly different...), but the work of the engine will be smoothed over the time it takes to the player to edit, which should be long enough, right ?
Also, if a specific lod receives no change, lower resolution lods don't either, I guess ?
Not quite. The LODs are recalculated on the server thread (which is not tied to the game's framerate) when the client requests changed LODs, and the work is spread over the course of multiple server ticks. As such, there is no lag!
does it have gpu instancing?
i might have missed if you added that in but gpu instancing is verry cool
GPU instancing is a useful technique for certain scenes, but all voxel data is unique here. As such, I don't think that there is a place to apply instancing (which requires having multiple models with the same vertex data).
@@DouglasDwyer oh ok
Hey man! Very cool video! I have a question. I'm developing a game, I've been working so hard on it for the past two years. My vision is to make a voxel world, which is a recent decision. I've been using Zbrush to sculpt all of my models so far which I've put so much time into and I have a good workflow going. I've been using blender to convert my models into voxelized models and I've been testing the world I have built into unreal engine. My problem is that the world I have made is 26 mil poly's when it's converted into voxels and it just won't load into unreal engine 5... It always crashes. So I used a less detailed world, but it looks like minecraft which I really don't like. I was also going to do pose to pose animation to achieve a certain look. I guess my question is: Is it possible to create a voxel engine so I can have a super detailed world in which I can import OBJ files and the program (Or engine I should say) would convert them into voxelized models? I want to build my own engine too because I just think it would be cool to have only my logo, not unreal, pop up on the startup of the game. I also want a specific feel to the gameplay and I worry about using unreal, I know you can control that, but I wonder how much control you actually have over how the game feels to play.... Also I want to keep all the voxels uniform.
It sounds like you want to create a game utilizing voxels mainly for the art style? If so, have you considered using a shader or post-processing effect to turn models into voxels? Such solutions do exist, and that would allow you to continue the same workflow with ZBrush and Unreal while achieving that voxel look :)
“The renderer is almost done”
Haha. In a sense, it was true - I had hit the limits of what felt possible with rasterization. To overcome those limits, I had to start a new renderer :)
goes to show how much minecraft java would benefit from a new rendering engine aswell as LODs
Luckily, there are many performance-based mods out there nowadays. But it would be nice if Minecraft Java were, say, rewritten in Rust... 🦀🦀🦀
Hey that is de_nuke.
octree goes brrrr
What do you mean the rendering is done? You're not going to add Ray Tracing Global Illumination MW 2019 gun animations??? /s
EPIC
what tutorials did you use to learn about voxel rendering and physics for it? i can not find any resources on the subject matter.
Excellent question. Voxel engines are a complex topic, and I've built my knowledge from experience, looking at other engines, and talking to the community. A good place to start if you're looking to learn 3D rendering is to follow the learnopengl.com tutorial series, or watch some online videos about how to make a Minecraft clone. Once you feel ready to tackle something more advanced, the voxel.wiki website looks like it has some good links to resources. As for physics, I still have a lot to learn in that department. I'm planning to read Millington's "Game Physics Engine Development" since I want to integrate a constraint-based solver for my engine. Overall, the best teacher is experience! Pick a project or goal that you'd like to achieve, and pursue it at all costs :)
@@DouglasDwyer any good minecraft clone tutorials you know of?
@@DouglasDwyer also have you seen the voxel videos from youtuber john lin? I would like to make something similar to that with.
It depends upon the language/tools that you want to use, but the tutorials by Low Level Game Dev, b3agz, and obiwac all look good!
As for John Lin, indeed, you would be hard-pressed to find someone in the voxel community who **hasn't** heard of him. His work was an inspiration to me and many others! If you want to create something as visually involved as Lin, you'll definitely need to use ray tracing, so I would recommend looking more into that.
@@DouglasDwyer thanks for the replies! does he share his voxel project on his github? and is there a difference in voxel size that he uses compared to what your doing? like with making the trees and grass?
Subbed
Nifty
And mf microsoft struggling with a 15 yr old voxel game
I don't see why you need an octree, that just seems like a lot of memory allocation every time the player moves, you need to recalculate the octree, don't you? If you're already looking at each chunk and figuring out where it goes on the octree by distance, why not just use that distance instead of storing it in an octree?
I'm not really looking at every chunk - the octree structure itself is what encodes the position of the data and helps me determine what needs to be loaded. That said, the mention of octrees was meant mostly to explain the hierarchical way that the LODs are organized. I actually use hashmaps to store the data internally, which are a bit more flexible.
@@DouglasDwyer Ah, I see, in the few implementations of voxel LOD I've done, it's just a per chunk thing, and so the minimum LOD is dictated by the chunk size. However, both were finite, and more of a simulation type thing, so the entire world has to be small enough to fit into ram anyway so the LOD was only for GPU memory, not ram. But for yours it's both because your voxels are so small. I think I see now, very cool! PS sorry for rambling I was just playing DnD and I'm a little faded haha great video though man
Are you switching to webgpu or wgpu? Because those aren't exactly the same thing.
Switching to WGPU, so kind of both in a way. On desktop I am using just WGPU, on web it is WGPU + WebGPU.
they are pretty much the same thing, firefox actually uses wgpu for its WebGPU backend
Where can I pay for you to continue making this?
That's very kind of you! But I am not yet ready to promise anything regarding this project, so I am not accepting donations. You can support me right now by subscribing and continuing to follow my work.
@@DouglasDwyer Understandable I would have given you money anyway even if you couldn't guarantee anything for the project. But I understand you don't want to take money from people if you cant promise anything, I'm like that my self.
Tooley1998 is using UE5 afaik
Yep, that's my understanding too! I would guess that he still has implemented a custom data representation and LOD system for his voxels too, so his work is impressive from both a technical and visual standpoint.
@@DouglasDwyer I still want to know what he did.
I tried Voxel Plugin one time and it was kinda janky.
I really wanna make a voxel game (maybe also engine) but Im a noob in every engine and I have good understanding of C++ and no understanding of Rust but Rust is the objectively better choice...
1:20 "Meet Gorgeous Latina Women"
xD
1:30 Meet Gorgeous Latina Women 😂
very nice Ads from YT lol
needs bazookas
agreed
Nuke❤
you should really render in vulkan is at all possible
I use WGPU, which is a graphics abstraction layer that allows me to target many backends (including Vulkan)! However, I target DX12 on desktop Windows since swapchains are less buggy in it.
dont know what all that means but 👍@@DouglasDwyer
meet gorgeous latina women ;-)
"Promo sm"
first
I do not understand why anyone would ever use voxels, the graphics sucks so badly, so even what gameplay they might add will not matter in the long run.
Please for the love of God stop flashbanging us Everytime you show some code.
This video looked interesting but I had to stop about 35% of the way in
Buy sunglasses? 😂
@@DouglasDwyer on order