Nice results are you planning to revisit this project? Vulkan raytracing support dropped recently... well the evaluation dev branch :-) Could be really interesting to see your engine running on different hardware.
I would really like to continue working on this. But for now I switched to making a game in Unreal. I asked for the Epic Megagrant to port this technique to Unreal where I could make a fair comparison against their technique. I would also have DXR support already implemented. But sharing my time between normal gamedev job and indie dev makes it impossible to continue until I get some financing. It's a heartache to leave it hanging like that :( So many ideas in my head...
@@icytower103 Yeah I can relate to that I have so many personal projects dead in the water. I can choose between financial shortcomings and too much time or a well endowed purse and no time -_- The sweet-spot occurs seldom... But I'm happy that you still have interest in it, time will come. It is a interesting approach and we have interesting times in hw with all that paradigm shifts towards modular architectures so who knows what to come. Maybe this could really a viable way to do things. Look at ID SW the new engine for Doom Eternal can do on paper 2000fps, that is freaking insane.
Amazing work! Results look wonderful, and fps is great for the results. I really do hope you continue polishing it and maybe hook it into Blender -- Cycles, though producing good results, might take minutes or hours for one frame of an interior scene
Thank you. All important operations are run in compute shaders or pixel shaders. Depending on what's more convenient. Most things are sequential as each reflection/refraction layer depends on the previous one. Same when merging them from back to front. But some things could be done in parallel for sure. Also, shadows from multiple, non-intersecting lights could be processed at the same time. And of course the ray tracing part is ultra-slow with multiple passes (one per object) and no acceleration structure for the scene. I plan to switch to DX12 and DXR this winter. I can't wait to see the speedup...
Thanks for the quick response! I'm curious how you're able to query mesh information to determine if an intersection occurs with the ray test for the shadowing part. It seems like you have to have all that information available in the compute shader, but not sure what that looks like.
@@johnjackson9767 That's exactly the problem in DX11 - you have to pass the proper vertex buffers, textures etc to the compute shader and then you access them as raw buffers. By calculating the offset within those buffers - and those offsets are stored in BVH nodes that you intersect. But unfortunately you cannot access multiple buffers or textures by index or address - so I process mesh by mesh, one at a time. In Dx12 you can index buffers/textures as well, based on object-id.
Those shadows are very impressive!! At first I figured they must be sample-based, but the lack of noise is astounding. Are the soft shadows an approximation? Also, how did you solve the shadow-terminator problem on your diffuse spheres, since they are discretized into triangles but the faces are smoothed.
Hard shadows are first generated with just one ray directed at the light source center. And then a post-processing step blurs them into soft shadows, which is a very tricky problem. Blur radius is based on distance to occluder. And that distance texture is also processed slightly. Most of the shadowing is also done in 3 layers (hard, medium, soft shadows) to improve intersections between hard and soft shadows. Blur on a single layer never could handle that despite so many tries. But overall I don't advice following that solution. Large radius blurs are hard in many ways and expensive. It's better to achieve initial bluring through deviating shadow ray a bit to get temporally fixed shadow pattern. And then blur that. That comes with some problems as well (hard shadows and edges), but those are much easier to solve. And performance is much better too. I wish I had time to finish that implementation and show the results. It looks so much better... I think I have the shadow terminator problem for hard shadows. Maybe it's not visible here. I haven't tried to fix it yet. Thanks for the kind words :)
I'm interested-what kind of approach is being used here? some sort of 'deferred raytracing' setup or just rasterizing primary rays and then secondary stuff being raytraced through a bvh and all that? Been looking into doing some sort of hybrid raytracing approach but haven't found so many resources yet
Hey, key points: - tracing rays without randomness - filtering ("advanced" blur) per intermediate layer of shadow/reflection/refraction (unlike path tracing where filtering is done at the end on the combined image) - using rasterization for primary rays and tens of other important bits and details which make or break the technique.
@@icytower103 nice thank you very much! will look into those^^-your demo here is very impressive edit: I think I've got something down-I have a fairly basic deferred renderer down already which I could port to direct3d 12, and then use the gbuffer information to send rays asynchronously in some sort of compute shader and calculate raytraced properties, etc. for the scene. could be an interesting project for a few weekends.
I would if I could :) But to use ray marching I'd need to describe geometry using mathematical distance functions (nearly impossible) or voxelize geometry which is memory heavy. Not mentioning animation... Triangles are pretty good in that context.
@@icytower103 Yeah, you're right, there's only a limited set of distance estimator functions currently, and describing triangle meshes like distance fields is pointless. But I'll have to argue the "nearly impossible" part. Inigo Quiles( th-cam.com/video/z_xM_jD08OM/w-d-xo.html ) showed that you can make arbitrary scene geometry using simple functions, he did it manually though. The question then becomes "Can you find a distance function that approximates a given mesh automatically?". This is a nonlinear optimization problem, I'm currently working on it. My approach would be to describe a DE function in the RPN notation and optimize it using evolution strategies to minimize the difference. Since we are converting meshes to DE functions beforehand - the optimization speed doesn't really matter that much.
@@icytower103 there's actually. Whole list of them online, and they are easy to define just hard to get with out Lipschitz discontinuitys. Here's a whole list of them including triangle interesections. www.iquilezles.org/www/articles/distfunctions/distfunctions.htm Triangle intersection maybe be fast marched + if you use a grid.
@@mykhailomoroz5296 I somehow missed that comment. It sounds very interesting, but I would assume that DE for a complex mesh would also be quite complex and expensive to compute intersections against. Plus the mentioned animation problem. And the cost of precomputation could be very large. However, the problem of finding DE for any mesh is very interesting. And usage of evolutionary algos would be cool. Especially with good visualization of the progress :)
Hey, thanks! I'm going to continue developing it, adding GI, improving shadows, reflections, performance, adding physics etc. And at some point I'm going to release a public demo and some simple game to showcase how it works. If all goes exceptionally well, then I may consider creating a plugin for 3DS Max, Maya, Blender for rendering. We will see... :)
@@LuckyST The engine is available on GitHub. It's easy to find if you search my name. You can fork it if you want. I think it's quite hard to pick such project after someone. However, for anyone interested, I have over hundred pages describing this method precisely and with lots of images :) I wish I could check if my hunch was correct that this method could be pretty powerful, but I guess the time window has closed - path tracing with denoising became the standard.
@@icytower103 thanks I had already found it in the comments. I'm more interested in it for educational purposes since I'm just a junior and an enthusiast. And this seems like a really cool project to get a grasp of things, and maybe learn a thing or 2 in cpp. if you have any documents you are willing to share I'd be more than happy to read them.
Thanks, yes, you can find it here: github.com/danielkrakowiak/Engine1. I had to stop development for now, because I switched to making a game, but I hope to resume it in the future.
@@icytower103 story of my life. You got a-svgf for filtering? I'm still working on porting over my opengl to vulkan so I can do rt without compute, also a lot of work... so much boilerplate
@@nielsbishere It's a complex subject. Reflection/refraction is filtered separately from shadows. But shadow filtering is a part of reflection/refraction filtering. Then r/r are filtered sperately for each light bounce, together for all screen pixels. We start from the last bounce, filter then and this becomes the input for 3rd bounce. Repeat. Input image from ray tracing is sharp and has no temporal noise. So much easier to filter, also because we filter incoming light from further bounces only, not texture details. Lots of details in every aspect of it, impossible to describe here, really. I linked my master thesis somewhere, which describes it in detail.
Any GPU supporting DirectX 11.3 can run it. So almost anything these days. Current gen would run it at 3-4x the framerate compared to the video, I guess.
I wish I had time for this. That would be great fun. But as Unreal progresses quickly towards ray tracing, it's hard to justify making your own engine these days. Especially when modifying Unreal is also possible. So we use Unreal for our game.
@@icytower103 Ahhh fair enough! The work you've done on it so far is pretty amazing! Unreal is an amazing engine also and I'm really excited for the ray tracing implementation and what it means for the future of gaming! Best of luck on your game!
This is running on a 1080? Look what I'm doing with my 1060: th-cam.com/video/wgqc15mJqrI/w-d-xo.html (realtime pathtracer 1080p, OpenGL core profile 3.3 no RTX)
Looks cool. Maybe try to show more of the dynamic aspect of it? Moving shadows, objects, lights? Changing GI? When it comes to performance mine leaks perf badly in some aspects as I described. I'm pretty sure it could run way faster. Good luck with yours! :)
This is what THE future looks like
I can watch this all day long !
That's what animation CGI looked like in the 2000s. Ray tracing hybrid.
Looks close to perfect :)
2:38 I like how the music gets scarier as the framerate drops
Nice results are you planning to revisit this project?
Vulkan raytracing support dropped recently... well the evaluation dev branch :-)
Could be really interesting to see your engine running on different hardware.
I would really like to continue working on this. But for now I switched to making a game in Unreal. I asked for the Epic Megagrant to port this technique to Unreal where I could make a fair comparison against their technique. I would also have DXR support already implemented. But sharing my time between normal gamedev job and indie dev makes it impossible to continue until I get some financing. It's a heartache to leave it hanging like that :( So many ideas in my head...
@@icytower103 Yeah I can relate to that I have so many personal projects dead in the water.
I can choose between financial shortcomings and too much time or a well endowed purse and no time -_-
The sweet-spot occurs seldom...
But I'm happy that you still have interest in it, time will come.
It is a interesting approach and we have interesting times in hw with all that paradigm shifts towards modular architectures so who knows what to come.
Maybe this could really a viable way to do things.
Look at ID SW the new engine for Doom Eternal can do on paper 2000fps, that is freaking insane.
can you remake this using the 3080?
Amazing work! Results look wonderful, and fps is great for the results. I really do hope you continue polishing it and maybe hook it into Blender -- Cycles, though producing good results, might take minutes or hours for one frame of an interior scene
Thanks! I hope to continue exploring algorithms and tech to improve significantly quality and performance.
Or you could just use Eevee
@@AlCatSplat dude I commented this two years ago way before eevee was a thing lmao
Great work. How much of this runs in compute shaders or in parallel passes?
Thank you. All important operations are run in compute shaders or pixel shaders. Depending on what's more convenient. Most things are sequential as each reflection/refraction layer depends on the previous one. Same when merging them from back to front. But some things could be done in parallel for sure. Also, shadows from multiple, non-intersecting lights could be processed at the same time. And of course the ray tracing part is ultra-slow with multiple passes (one per object) and no acceleration structure for the scene. I plan to switch to DX12 and DXR this winter. I can't wait to see the speedup...
Thanks for the quick response! I'm curious how you're able to query mesh information to determine if an intersection occurs with the ray test for the shadowing part. It seems like you have to have all that information available in the compute shader, but not sure what that looks like.
@@johnjackson9767 That's exactly the problem in DX11 - you have to pass the proper vertex buffers, textures etc to the compute shader and then you access them as raw buffers. By calculating the offset within those buffers - and those offsets are stored in BVH nodes that you intersect. But unfortunately you cannot access multiple buffers or textures by index or address - so I process mesh by mesh, one at a time. In Dx12 you can index buffers/textures as well, based on object-id.
Those shadows are very impressive!! At first I figured they must be sample-based, but the lack of noise is astounding. Are the soft shadows an approximation? Also, how did you solve the shadow-terminator problem on your diffuse spheres, since they are discretized into triangles but the faces are smoothed.
Hard shadows are first generated with just one ray directed at the light source center. And then a post-processing step blurs them into soft shadows, which is a very tricky problem. Blur radius is based on distance to occluder. And that distance texture is also processed slightly. Most of the shadowing is also done in 3 layers (hard, medium, soft shadows) to improve intersections between hard and soft shadows. Blur on a single layer never could handle that despite so many tries. But overall I don't advice following that solution. Large radius blurs are hard in many ways and expensive. It's better to achieve initial bluring through deviating shadow ray a bit to get temporally fixed shadow pattern. And then blur that. That comes with some problems as well (hard shadows and edges), but those are much easier to solve. And performance is much better too. I wish I had time to finish that implementation and show the results. It looks so much better...
I think I have the shadow terminator problem for hard shadows. Maybe it's not visible here. I haven't tried to fix it yet. Thanks for the kind words :)
I'm interested-what kind of approach is being used here? some sort of 'deferred raytracing' setup or just rasterizing primary rays and then secondary stuff being raytraced through a bvh and all that? Been looking into doing some sort of hybrid raytracing approach but haven't found so many resources yet
Hey, key points:
- tracing rays without randomness
- filtering ("advanced" blur) per intermediate layer of shadow/reflection/refraction (unlike path tracing where filtering is done at the end on the combined image)
- using rasterization for primary rays
and tens of other important bits and details which make or break the technique.
@@icytower103 nice thank you very much! will look into those^^-your demo here is very impressive
edit: I think I've got something down-I have a fairly basic deferred renderer down already which I could port to direct3d 12, and then use the gbuffer information to send rays asynchronously in some sort of compute shader and calculate raytraced properties, etc. for the scene. could be an interesting project for a few weekends.
Good job man!!!
Amazing stuff. You should probably try ray marching, it should be way faster than finding intersections between a lot of triangles.
I would if I could :) But to use ray marching I'd need to describe geometry using mathematical distance functions (nearly impossible) or voxelize geometry which is memory heavy. Not mentioning animation... Triangles are pretty good in that context.
@@icytower103 Yeah, you're right, there's only a limited set of distance estimator functions currently, and describing triangle meshes like distance fields is pointless. But I'll have to argue the "nearly impossible" part. Inigo Quiles( th-cam.com/video/z_xM_jD08OM/w-d-xo.html ) showed that you can make arbitrary scene geometry using simple functions, he did it manually though. The question then becomes "Can you find a distance function that approximates a given mesh automatically?". This is a nonlinear optimization problem, I'm currently working on it. My approach would be to describe a DE function in the RPN notation and optimize it using evolution strategies to minimize the difference. Since we are converting meshes to DE functions beforehand - the optimization speed doesn't really matter that much.
@@icytower103 And about animations. You could technically distort space or interpolate between several DE's. Well, sadly it isn't that simple =)
@@icytower103 there's actually. Whole list of them online, and they are easy to define just hard to get with out Lipschitz discontinuitys.
Here's a whole list of them including triangle interesections.
www.iquilezles.org/www/articles/distfunctions/distfunctions.htm
Triangle intersection maybe be fast marched + if you use a grid.
@@mykhailomoroz5296 I somehow missed that comment. It sounds very interesting, but I would assume that DE for a complex mesh would also be quite complex and expensive to compute intersections against. Plus the mentioned animation problem. And the cost of precomputation could be very large. However, the problem of finding DE for any mesh is very interesting. And usage of evolutionary algos would be cool. Especially with good visualization of the progress :)
Amazing how it's running on a 1080
Awesome, what do you intend to do with this incredible engine? sell ? release or something?
Hey, thanks! I'm going to continue developing it, adding GI, improving shadows, reflections, performance, adding physics etc. And at some point I'm going to release a public demo and some simple game to showcase how it works. If all goes exceptionally well, then I may consider creating a plugin for 3DS Max, Maya, Blender for rendering. We will see... :)
Daniel Krakowiak awesome, continue this amazing job. PS: in Blender would be so cool
@@icytower103 consider opening the source for it. will gain following raelly fast. how to commercialise that, not sure. but id be a fan instantly.
@@LuckyST The engine is available on GitHub. It's easy to find if you search my name. You can fork it if you want. I think it's quite hard to pick such project after someone. However, for anyone interested, I have over hundred pages describing this method precisely and with lots of images :) I wish I could check if my hunch was correct that this method could be pretty powerful, but I guess the time window has closed - path tracing with denoising became the standard.
@@icytower103 thanks I had already found it in the comments. I'm more interested in it for educational purposes since I'm just a junior and an enthusiast. And this seems like a really cool project to get a grasp of things, and maybe learn a thing or 2 in cpp. if you have any documents you are willing to share I'd be more than happy to read them.
this was done on a 1080?
Epic project. Is the software open source?
Thanks, yes, you can find it here: github.com/danielkrakowiak/Engine1. I had to stop development for now, because I switched to making a game, but I hope to resume it in the future.
Looks good, now it's time for DXR?
Yes, switch to DX12 and then DXR :) But no time for that unfortunately.
@@icytower103 story of my life. You got a-svgf for filtering? I'm still working on porting over my opengl to vulkan so I can do rt without compute, also a lot of work... so much boilerplate
@@nielsbishere No, it's very different from SVGF. Those techniques share almost nothing, really.
@@icytower103 hmm, how did you filter it then? Like EA SEED did?
@@nielsbishere It's a complex subject. Reflection/refraction is filtered separately from shadows. But shadow filtering is a part of reflection/refraction filtering. Then r/r are filtered sperately for each light bounce, together for all screen pixels. We start from the last bounce, filter then and this becomes the input for 3rd bounce. Repeat.
Input image from ray tracing is sharp and has no temporal noise. So much easier to filter, also because we filter incoming light from further bounces only, not texture details. Lots of details in every aspect of it, impossible to describe here, really. I linked my master thesis somewhere, which describes it in detail.
Which api was this made in? Opengl vulkan dx?
In Dx11.
@@icytower103 cool! waiting for updates!
What is the minimum requirement for a PC to run this. Just curious
Any GPU supporting DirectX 11.3 can run it. So almost anything these days. Current gen would run it at 3-4x the framerate compared to the video, I guess.
Nice. Is the source code publicly available?
Yes, you can find it here: github.com/danielkrakowiak/Engine1.
Are you developing a game with this engine?
I wish I had time for this. That would be great fun. But as Unreal progresses quickly towards ray tracing, it's hard to justify making your own engine these days. Especially when modifying Unreal is also possible. So we use Unreal for our game.
@@icytower103 Ahhh fair enough! The work you've done on it so far is pretty amazing! Unreal is an amazing engine also and I'm really excited for the ray tracing implementation and what it means for the future of gaming! Best of luck on your game!
This is running on a 1080? Look what I'm doing with my 1060: th-cam.com/video/wgqc15mJqrI/w-d-xo.html (realtime pathtracer 1080p, OpenGL core profile 3.3 no RTX)
Looks cool. Maybe try to show more of the dynamic aspect of it? Moving shadows, objects, lights? Changing GI?
When it comes to performance mine leaks perf badly in some aspects as I described. I'm pretty sure it could run way faster. Good luck with yours! :)
Rip fps
Silvio Pelizzoli Yep, for now... But I have only scratched the surface in terms of optimizations.