well so im just starting to get into this topic and i didnt quite get where you get the other texture from, surely im missing something important :D But thanks for the Introduction to this! :)
The reason you can't plug the mesh paint texture object node into height to normal function is that, the node outputs a Virtual Texture 2D and the texture samplers inside the function are regular samplers which can't sample a virtual texture. This is true for any texture sampling in general (in any game engine), and isn't anything specific for mesh paint texture object node. You always need a virtual texture sampler to sample a VT, and a regular one to sample a regular T2d.
All of the samplers used in this video were set to color, not to virtual texture color. Perhaps the engine is doing something invisible on the back end to change it. Changing the texture sampler inside the function to virtual texture does nothing. The problem is that the Function Input only accepts texture 2d, and there is no Virtual Texture 2d or Mesh Paint function input. The function input node, not the sampler, is the sole limitation and this is easily confirmed due to the fact that it works outside the function regardless of setting, and cannot be input to the function regardless of setting.
@@TechArtAlexdouble checked this. You're right. The function input is the limitation here. Regarding other samplers, I think the texture object type (which is virtual) overwrites the sampler type and it gets sampled as virtual color anyway. Just as you can throw a linear color texture object into a color texture sampler without issue. I maybe wrong though. But I'm not aware about how and where this virtual texture is stored. You can open it, but there's no location for it and so it can't be reused. Also found that it's attached to the nanite fallback mesh and is actually tied to the fallback mesh vertices count. If you change the fallback percentage, the memory cost for the painting changes. Not totally sure what's really happening on the backend.
Did you have any issue where the VT in the details panel would update but the texture on your mesh wouldnt? youre the first video i found on fixing this issue and im still not getting a proper result in the viewport. i hope unreal fixes this properly soon
I didn't have any issues with it, although I haven't tested it on the official 5.5 release. Certain settings changes like updating the resolution seemed to require deleting and re-creating the VT. But as far as painting goes, updates were instant and appeared correctly as long as I had assigned the UVs properly in both the details and the material. I would double check the UVs first, as it's the easiest error to make.
@@TechArtAlex would you consider doing a in depth video on this in the 5.5 release? hard to find material on it as its so new, and im sure im not the only one with similar issues or questions :)
@tylermadigan7631 not sure if I'll do new, more in depth tutorial on this. I don't really see any major differences between the preview and full version worth covering - but I do expect it'll come up in other videos as part of other workflows I do want to talk about.
The next tab, "textures" over can paint directly on textures, altering the actual selected texture itself, if that's what you mean. I didn't experiment with this much. Beyond that it works just like normal vertex colors when it comes to layering, you have multiple color channels to work with so you can use them as layers.
Presumably you can select an existing mesh paint virtual texture if you want to reuse it, although I didn't try it. Painting unique vertex colors is not all that different than having a unique virtual texture per asset, either way there is memory being consumed but in the case of the texture, the memory cost can be much lower since there are far fewer pixels in the texture than vertices in the mesh.
@@TechArtAlex It's an interesting way to tackle the problem for sure, need an actual project to evaluate its pros and cons I guess, but this does make the vertex colors free to use for other effects which can lead to interesting combinations.
"Either way there is memory being consumed but in the case of the texture, the memory cost can be much lower since there are far fewer pixels in the texture than vertices in the mesh." - so funny that it used to be the other way around :D
@harrysanders818 haha yup, definitely didn't expect to see the script flip on this one so soon, but here we are. I do think virtual texture samples are a bit of a hoop to jump through compared to just sampling the vertex colors, but even that's not such a big deal these days.
Editing a mesh texture object at runtime would essentially be the same as using a render target, which the engine can already do. Although it might be convenient to have that wrapped into this feature too, so you don't have to use two separate systems for static and dynamic things.
well so im just starting to get into this topic and i didnt quite get where you get the other texture from, surely im missing something important :D
But thanks for the Introduction to this! :)
Which other texture are you referring to?
The reason you can't plug the mesh paint texture object node into height to normal function is that, the node outputs a Virtual Texture 2D and the texture samplers inside the function are regular samplers which can't sample a virtual texture. This is true for any texture sampling in general (in any game engine), and isn't anything specific for mesh paint texture object node. You always need a virtual texture sampler to sample a VT, and a regular one to sample a regular T2d.
All of the samplers used in this video were set to color, not to virtual texture color. Perhaps the engine is doing something invisible on the back end to change it.
Changing the texture sampler inside the function to virtual texture does nothing.
The problem is that the Function Input only accepts texture 2d, and there is no Virtual Texture 2d or Mesh Paint function input.
The function input node, not the sampler, is the sole limitation and this is easily confirmed due to the fact that it works outside the function regardless of setting, and cannot be input to the function regardless of setting.
@@TechArtAlexdouble checked this. You're right. The function input is the limitation here.
Regarding other samplers, I think the texture object type (which is virtual) overwrites the sampler type and it gets sampled as virtual color anyway. Just as you can throw a linear color texture object into a color texture sampler without issue. I maybe wrong though.
But I'm not aware about how and where this virtual texture is stored. You can open it, but there's no location for it and so it can't be reused. Also found that it's attached to the nanite fallback mesh and is actually tied to the fallback mesh vertices count. If you change the fallback percentage, the memory cost for the painting changes. Not totally sure what's really happening on the backend.
Did you have any issue where the VT in the details panel would update but the texture on your mesh wouldnt? youre the first video i found on fixing this issue and im still not getting a proper result in the viewport. i hope unreal fixes this properly soon
I didn't have any issues with it, although I haven't tested it on the official 5.5 release. Certain settings changes like updating the resolution seemed to require deleting and re-creating the VT.
But as far as painting goes, updates were instant and appeared correctly as long as I had assigned the UVs properly in both the details and the material.
I would double check the UVs first, as it's the easiest error to make.
@@TechArtAlex would you consider doing a in depth video on this in the 5.5 release? hard to find material on it as its so new, and im sure im not the only one with similar issues or questions :)
@tylermadigan7631 not sure if I'll do new, more in depth tutorial on this. I don't really see any major differences between the preview and full version worth covering - but I do expect it'll come up in other videos as part of other workflows I do want to talk about.
is it possible to paint layers, and textures ? For example, i take a texture and paint my color "red" on it? and then bake somehow?
The next tab, "textures" over can paint directly on textures, altering the actual selected texture itself, if that's what you mean. I didn't experiment with this much.
Beyond that it works just like normal vertex colors when it comes to layering, you have multiple color channels to work with so you can use them as layers.
A thing that you didn't really cover and looked interesting is that you can change the resolution of the VT tiles in Project Settings.
So for every instance of a mesh I want to paint, I will have to add a new VT for that asset?
Presumably you can select an existing mesh paint virtual texture if you want to reuse it, although I didn't try it.
Painting unique vertex colors is not all that different than having a unique virtual texture per asset, either way there is memory being consumed but in the case of the texture, the memory cost can be much lower since there are far fewer pixels in the texture than vertices in the mesh.
@@TechArtAlex It's an interesting way to tackle the problem for sure, need an actual project to evaluate its pros and cons I guess, but this does make the vertex colors free to use for other effects which can lead to interesting combinations.
"Either way there is memory being consumed but in the case of the texture, the memory cost can be much lower since there are far fewer pixels in the texture than vertices in the mesh." - so funny that it used to be the other way around :D
@harrysanders818 haha yup, definitely didn't expect to see the script flip on this one so soon, but here we are. I do think virtual texture samples are a bit of a hoop to jump through compared to just sampling the vertex colors, but even that's not such a big deal these days.
It just sucks that it paints to the UV’s or be done at runtime.
Editing a mesh texture object at runtime would essentially be the same as using a render target, which the engine can already do. Although it might be convenient to have that wrapped into this feature too, so you don't have to use two separate systems for static and dynamic things.