NVIDIA’s Crazy New AI Paints With Images!

[@J-c-d8f]

What a time to be alive

[@jackyvivid]

In 10-20 years, these things will be considered lame!

[@IM2awsme]

​@@AI_native if you're not loosing your job to ai, it's because you work in a stressful environment.

[@jacquelebecquer3396]

You’re damn right

[@AgroFro]

@@AI_native If you could lose your job to AI, you should probably find another profession. Better off using AI to your advantage

[@cabanford]

As the US gets ready to reelect Trump... 🤦

[@tlemo5859]

As an artist, this is pretty huge, especially if you can use your own textures. Like you said, the textures this method generates will need modification (for now), but you can just draw an image and plaster it onto a model. I see this being implemented in Blender or a dedicated texture editor, and I see myself using it for actual work.

[@andrejtetkic7169]

I develop addons for Blender, most notably Drag & Drop Import. If they release the code, I might create the addon for it

[@Think666_]

It's a shame that "artists" seem to be attacking anything AI and anyone who integrates it into process.

[@DigiMyst]

@@Think666_ I agree, and that's why I'm trying to find a tool that's makes perfect code with little to no editing. It'd be much easier to make games since I can just make my own art and let the tool do the heavy lifting instead 💪🏾

[@Think666_]

@@DigiMyst Couldn't agree with this pragmatic approach more, I just wish more people shared this view.

[@DigiMyst]

@@Think666_ It's my hope that one day we will get functional AI game engines and tools that can fix their own code. It wouldn't even stop at games really. All modern tech relies on some form of code, even cars, so if I could diagnose my own computer and look for parts then I won't need to stress over repairs or skill gaps

[@test-uy4vc]

What an AI time to be painted alive! 🎉

[@user-tm7eq3jx4h]

😂❤

[@RagBrokeHisThumb]

As opposed to painted dead

[@skintillion]

Paint me like one of your girls

[@contentfreeGPT5-py6uv]

Best comment

[@user-tm7eq3jx4h]

@@skintillion 🤣 AI oobs and all

[@heckensteiner4713]

When I was a kid, my grandma always told me diffusion texture painting was the work of Satan. 30 years later, I still have no idea what she was talking about.

[@tuxxyy1]

Thank you; my morning was a little too sane, before your comment came along.

[@midnightblue3285]

It is true, it is a technology of cyborg eptelians

[@Jofoyo]

Now this is nice. It's like AI texture brush sampling rather than purely generative dribble. I can see this being really nice when painting environments with a megatexture approach if it can ever paint with PBR maps.

[@V3RTIGO222]

Now this is a good application of AI in art, we can use it directly as a tool for texturing digital worlds easier and not always resort to using sctrict asset libraries for textures... this is an excellent tool for game developers and 3D artists alike.

[@angrychompski]

this is the exact usecase for an ai, boring and repetitive tasks that dont need much consideration, not ai movie writers or smth that takes creativity

[@Matt-bp5vy]

Without the flashy stuff people wouldn't want to invest in this boring stuff

[@Caellyan]

Waiting for Adobe to patent something based on this and then sue anyone using something similar with their army of lawyers to maintain market dominance.

[@Rocketos]

lol, have that happened before?

[@MAN_FROM_BEYOND]

@@Rocketos
Yeah, I think they just recently made it such that anything you make using any of their software belongs to them, not yourself. This was like a couple weeks back I think.

[@hound_of_justice]

@@MAN_FROM_BEYONDthats not true, they dont own you assets you make in their software.

[@StephenRansom47]

😊 This a wonderful use of this technology … Skinning has always been a difficult part of Modeling in 3 Dimensions.
Now a director waves a brush, and AI set decorators sketch the first draft.
Brilliant 👏

[@orang1921]

We are seeing progress today that many just a few years ago would believe to be decades away

[@sikunowlol]

facts

[@drowzy2309]

It's baffling to me how many think AGI is still 6-10 years away. Although, I'm sure no matter how close we get people will still argue it's not AGI.

[@dibbidydoo4318]

@@drowzy2309 because the human mind is the most complex structure in the universe. People think it will take 20 years or more to solve fusion but you think human-level AI comes earlier?

[@drowzy2309]

@dibbidydoo4318 Those 2 technologies are not a good comparison. AI is helping every single field advance at speeds we could never achieve without it. That means it could even solve fusion if we have a breakthrough with physics and AI, which makes sense if you understand that AI can learn through simulations. Q* or strawberry is rumored to be able to do these things.
There's also the possibility that AI may never achieve human level, because sentience isn't in our control, but the same can be said about fusion. Sentience and fusion both occur in nature, but that doesn't mean we can achieve it, so who knows.

[@drowzy2309]

@dibbidydoo4318 my original comment was mostly referring to the speed of which AI is moving. If it continues even at its current pace of progress, and just using ChatGPT 4o's new SKY voice as an example, 6-10 years or more just seems silly.

[@yorzengaming]

This is absolutely breathtaking. Oh My Graphics

[@FengXingFengXing]

Awesome they open source this. I want use this in Blender. 😁

[@user-qc4xy8wb4y]

Maya paint allow you to draw in 3d any texture, since 28 years

[@kneel.downnn]

R.I.P adobe substance painter

[@PotatoClips]

my guess is this project continues for another couple iterations before there are clones and variations implemented into substance, photoshop, etc. with the circular nature of generating ai images, ai imagery tools within existing creative applications makes way more sense in the long run. its how we got rotoscoping and background fill tools. those were originally independent software developed for specific use cases.

[@Mireneye]

Thank god. So tired of Adobe ^- ^
Sad that Substance started as such a good suite for the people before.
Adobe didn't ruin the software itself, but the payment model..

[@MrGTAmodsgerman]

It doesn't do it as PBR as far as i can tell. So...

[@Mireneye]

@@MrGTAmodsgerman I forsee that development. Actually very likely a selection of all known texture types to generate so it can fit any workflow.

[@jenkem4464]

Naw. Will speed up your painter workflow. You still need all the bells and whistles for polish. Maybe pair it up with Quixel.

[@puffinjuice]

A lot of cleanup is still required. I hope they develop this further so that the edges of the brush are natural and so that the whole piece blends together at the end!

[@Razumen]

@4:31 shows that they do a really good job with blending between brushes already. Anything further than that would have to be done by the artist.

[@Hepza08]

Evolution of the stamp brushes, awesome!

[@Oscaragious]

I'm not an artist, but this seems a little too uncanny - like it's right at the line where using it won't lose you any time nor save you any time because you're spending the rest of the time prompt engineering to tweak the output just right.

[@jeffg4686]

Don't you just love how they use candy in their demos?
Everything coming down from the corps always includes either sugar/candy, animal products, pharmaceuticals, caffeine, and/or eye candy.

[@keybraker]

What a time to get paid by NVIDIA! 😊

[@Mulnader]

What is name of the 3D texturing software? Is it available for tests?

[@PotentChr0nic]

Nvidia Canvas?

[@morecarstuff]

What a time to be AI-live

[@travotravo6190]

I don't know, a lot of ways AI just feels like Photoshop of the Future... with the same pitfalls as old tech. If you rely too much on a software's filters and gimmicks, then you can easily run into dead ends where your "I did it" button stops working.

[@V3RTIGO222]

This is absolutely true, it is a good time saver but it can't replace the fundamentals... A real artist can make the determination on what looks good within their style or in terms of realism that an amateur still couldn't with the same tools.

[@thestruggleisreal9626]

It's amazing how you say the word "and" the exact same way each time XD it almost sounds like a soundboard...

[@derekgamer1978]

That's actually sick like wow wow wow.

[@Sekir80]

wow

[@duffsdevice]

Wow

[@peanutnutter1]

Waow

[@FlameForgedSoul]

Something in this space that won't have people immediately reaching for the torches and pitchforks. About time.

[@metakron]

Only NVIDIA develops masterpieces of this kind, maybe even Microsoft and meta.

[@user-qc4xy8wb4y]

Maya paint allow you to draw in 3d any texture, since 28 years

[@IcyLucario]

That's incredible, I really liked the moss example.

[@GyattGPT]

This is the kind of AI image generation that blurs the line between human-created and computer-created.

[@toututu2993]

I don't considered stealing images on the web to be used on the generated system anywhere near the term of "created". Get your English right people!!

[@zaughk]

Of all the use cases of AI that seem like more ways to just undercut artists, as someone who makes textures I could see a few situations where having this kind of cut and fill brush would be very useful. Reminds me of the way procedural painting works for foliage in games but moreso just on texture. If you could define the seams even more precisely this would be supremely useful to artists.

[@bluecarrotrm]

I only ever attained a GED but I've taught myself many of the tools used for game design. What I can't do is give myself the TIME to make games. My REAL job takes almost all of every work day. Tools like these would let me and others like me cut years out of a game dev project.
Moreover, one of my projects would benefit greatly from tech like this. Mask layers and then a "painted" diffusion technique that could fill in each layer with textures would be magic for procedurally-generated, naturalistic game environments.

[@rockrobertson5776]

Fractal design Painter had a suite of tools like this.

[@art-creator]

Never had. It was inconvenient and ridiculous subsystem even didn’t integrated to painter itself.

[@rockrobertson5776]

@@art-creator lol. They were a bit ahead of the times.

[@user-qc4xy8wb4y]

Maya paint allow you to draw in 3d any texture, since 28 years

[@art-creator]

@@user-qc4xy8wb4y do you read maya documentation or even start maya last 28 years?

[@art-creator]

@@rockrobertson5776 yes, it was a bit futuristic. But now there are no modern industrial tools which seam texture painted through brush-based interface with such degree of accuracy.

[@willguggn2]

If the examples shown aren't just the cream of the crop, this is quite huge. The blending is outstanding for an automated process.

[@jeffg4686]

It's time NVidia starts making some of their services available.

[@danielsanichiban]

Unlike most applications of AI, this is actually cool. I have to say tho, as a person who made music and painted etc in the 80s going into the 90s, then getting heavily into using software like deluxe paint, lightwave, photoshop, cubase and so on, I’m now so sick of how virtualised everything is, and how every damn thing is done thru the same damn computer interfaces, on a display. On one hand this stuff is revolutionising all kinds of design, textiles etc, but on the other hand it is very very different to pinning some material to a dummy and cutting it up, or knitting a psychadelic dino-turtle. We have good algorithms now but our hardware is sorely lagging behind

[@jean_mollycutpurse_winchester]

I don't understand. I could do this with, Painter back in the early 2000s.

[@elivegba8186]

🤔

[@michaelleue7594]

There are two improvements: 1) The important one is that the blending between textures is better, this should have been the primary focus of the paper, and 2) the less important one is that the textures themselves are ostensibly more consistent without repeating, which I guess is fine but hardly a big deal, or even really noticeable.

[@jean_mollycutpurse_winchester]

@@elivegba8186 Painter 3.0
Author: Mark Zimmer
Publisher: Fractal Design Corporation
Type: Applications
Category: Visual Arts & Graphics

[@Johnnyynf]

would there also be a possibility to generate alpha and depth map matched to the material too?
Like that would be kind of cool

[@MrTomyCJ]

To improve the seams, you could use a special dedicated texture for them, each half corresponding to their respective texture at each side of the seam. The halves to make the seam texture could be combined in the middle using stable diffusion inpainting (this doesn't need to be real time because it's done just one). Extra points if you could feed this inpainting with a real life picture of the seam when possible.

[@stereo-soulsoundsystem5070]

This would be so useful for games and animations. Really great for backgrounds and textures on models

[@KungFuChess]

Procedural texturing has been around for a while now in most pro software

[@MrSongib]

I think we could use Repaint in ControlNet on Comfy similar to this and for 3d it just need to be implemented.
Maybe it's better or worse or already in the video here 2:08. idk

[@cefcephatus]

The only mistake they made is that they appended the image in rectangle. If they made it into brush shape, it will be the biggest leap across industry. I think, the issue for that matter is that they just want to show that it's a demo, not a final product. So they don't have to hold a grudge against any major tool developers. So yeah, they might already did it, but they just didn't want troubles.

[@jameshughes3014]

This is the kind of thing I've been waiting for with AI. Tools designed from the beginning for artists, finally. yay

[@ThankYouESM]

I created a similar version whereas mine is called "ai scroll tiling" in which we can select a tile image to project then create an "approximate" mask then use a throttle scrollbar to cast every possible combination of similar patterns -80% to 100% to %80 similarities.

[@teenspider5207]

What a time to be an AI

[@MikevomMars]

Correct tiling is not only a matter of textures, but also UV-coords. So this tool also modifies UV-coords on the model itself? 🤔

[@WifeWantsAWizard]

(3:29) You said "text-to-image". It would be an "image-to-image" AI. Also, you can just do that now. Take a screenshot of that dinosaur using the "PrtScn" key and then hand it to a Web site like Krea or Dezgo and you can have an actual toy dinosaur texture to pull back into your 3D lab there within 60-seconds.

[@kamaur01]

This could be a great Substance Painter alternative.

[@franciscomagalhaes7457]

This sort of thing seems really cool for us non-artistic types to be able to whip up some assets for games and other little side projects.

[@blengi]

should paint with bump maps, levels of detail and physics etc...

[@musicalbug.]

Why no one uses any nvidia ai powered tool in the gaming and entertainment industry.

[@radekmojzis9829]

Its a pain to set up and the time you would save would be wasted on doing all the exporting between the programs.
We will see a massive surge in usage once it gets implemented into blender or something.
I think big studios already use it/are working on it, but they dont really want us to know - because there is still massive stigma attached to this type of tech.

[@akopval]

It's pretty terrible and useless right now because of the output it gives you, but these are the demos. We will need to wait a while before these are useable. What you see visually is not the important part

[@evolvedant]

Which tool do you use to take one picture, and get another picture in return that is a better of the first? You showed this for the plushie.

[@daniel-henrisparmont8253]

Hello dear Dr. Károly Zsolnai-Fehér ! I'm starting to move away from TH-cam since it's impossible to get rid of the shorts, slowly moving over to Nebula and i was wondering if there's any chance you would post your videos there, really love your channel ! What a time to be alive !

[@sassss5984]

If they can create also normal map, metallic map etc, then thats it. I am very surprized how Substance Painter still does not have AI. Those companies are very slow and I will be happy to see alternatives

[@Noizzed]

When everyone is capable of drawing good quality art there will be more interest in uniqueness. Back then anyone could pass as a good artist because their drawings were done with good quality, but many times the art was generic, basic and too shallow to captivate anyone. Now that everyone could do a good quality painting/drawing, only those who can push creativity to new horizons will be able to eart the title.

[@dr9299]

How do you actually print a article of clothing with those patterns?

[@j-b-l8147]

Finally a good use for image ai.

[@soup2634]

Oh man! It's amazing seeing technology progress from the Hose tool in Corel Painter to this :D

[@Chronomatrix]

This is huge for game developers and archviz. HUGE.

[@blacklightgaming9326]

3:12 nessiiii!!!

[@madcatlady]

Blender plugin?

[@madcatlady]

to be fair Quixel Mixer can sort of do this

[@erickzetina193]

3:12 Nessie 🧐

[@LarryPanozzo]

Cool! Smart design workflow

[@Juan-qv5nc]

I would use something like this in my work not with images but with audio, say to extend a musical idea starting with a short one. Usually you do that basically by copy-pasting, but this technique applied to audio could give a more interesting result by introducing small variations in the subsequent instances of the intial musical idea. Maybe someone has already done it, I haven't checked the literature as I have been holding to my own papers. For completeness: What a time to be alive! 🙃

[@lylek8933]

Not a brush artist but do see this being a cool tool for cleaning up quadrant seams when creating seamless textures. Cool stuff. Shared this video at a site that I frequent as well. :)

[@jasondads9509]

woah its like really fast inpainting

[@art-creator]

Shit, I thought try to use it with new Houdini Copernicus, but sources lack :/

[@CalmDownJack]

i’ve never heard so many spaces between words

[@jryde421]

UV wrapping is gonna change

[@RebelSee]

This is an actual cool and artists friendly way to use generative AI.
I really like it!

[@FitnessCoachAustralia]

Finally I can become the artist I always wanted to be but couldnt be bothered learning how to draw

[@vi6ddarkking]

I have a workflow to mass produce 3D clothing with Comfy UI.
So yes a better texturer to add to the workflow is definitely welcomed also With Adobe being Adome the more alternatives to their software the better.

[@unveil7762]

Hei hei, is this extension available in omni ?? Right now ?? 😮

[@dogrularable]

It s 3d version of a previous nvidia tool but this time lets you use custom materials and a 3d canvas... no public access i asume...

[@aledmb]

digital art is going to a very dark place really soon.

[@SP-ny1fk]

Unfortunately Adobe will come to own this.
The excitement levels drop considerably.
All of this needs to be made open source.

[@SIBUK]

I don't see the big deal. This is basically every computer game map editor for the past decade.

[@sudheergoutham]

🎯 Key points for quick navigation:
00:00:00 *🎨 Using AI to Paint With Images*
- Introduction to the concept of using AI for painting,
- Explanation of diffusion-based AI for generating images from text.
00:00:57 *🖌️ New Technique: Painting With Images*
- Transition from using brushes to using images,
- Issues with repetitive tiling and the solution with the new technique,
- Examples of painting with the technique, like adding a rocky road or moss.
00:02:15 *🛠️ Comparisons to Existing Methods*
- How the new technique compares to Photoshop,
- Advantages in tiling, consistency, and image versatility,
- Emphasis on control over detailed image elements like rooftops and trees.
00:03:11 *🦖 Future Potential and Applications*
- Acknowledgment of current technique limitations compared to expert artists,
- Future possibilities such as integrating powerful text-to-image AI,
- Encouragement to use current tools like Stable Diffusion for creative projects.
00:04:06 *🚀 Enabling Creativity and Looking Forward*
- Optimism about future developments,
- Encouragement for wider creativity among users,
- Mention of detailed comparisons in the paper and availability of the code.
Made with HARPA AI

[@peanutnutter1]

3:54 is this Blender?

[@jayjadotte1683]

I would used it to paint on 3d objects for video games like Unreal Engine

[@llRoBoBinHoll]

The results look awesome!

[@nutshell5494]

Blender procedural material be like "not using math? pathetic"

[@cavemann_]

This is so crazy, all my interests becoming obsolete in the job market!

[@supersonictumbleweed]

Hold on to your projects!

[@azeriff]

Can’t imagine this one would be improved in the next 3 yrs! Just Carzy!

[@Soar-hw2rw]

I like the new editing of the intro.

[@dieterjosef]

Can you mar video about AI and photogrammetry? I am wondering why this isn't something where AI isn't pushing any other solution out of the way.

[@JosephD79]

zbrush did this about 10 years ago

[@oliverfunk]

Why does it sound like the narrator voice is AI generated?

[@CaptainRx-ss3rt]

Because it probably is

[@TheAwesomeAtom3]

He's from another country (Hungary). He is a published author, has a PhD, and is probably the leading AI expert in his country.

[@Kinosei30]

Hold on to your brushes!

[@user-qc4xy8wb4y]

Maya paint allow you to draw in 3d any texture, since 28 years

[@flyingdogsoft]

I am getting a 404 after clicking on "Code" . So how can I use it now?

[@SB-yh2uk]

Excellent tool for Architecture!!

[@elgodric]

Meanwhile me as a painter trying finish this paint since Jun

[@dXXPacmanXXb]

But when is all this nice stuff actually coming to engines like Unreal or Blender

[@Suthriel]

Something similar, not so advanced is already there for use, Stable ProjectorZ.

[@akopval]

It's not ready for performance. This needs to be implemented into trim sheets, custom UVs, etc. Maybe for Blender, for a prerendered image, but they haven't even began to start making this useable for games

[@AllisterVinris]

I mean AI brush for background textures, why not I guess.

[@Iridium.]

Time to go back and enjoy traditional painting and drawing . digital stuff is now accessible to any potato

[@looooool3145]

In every AI paper, you can always find a Russian or Indian or Korean names in it.

[@michaelleue7594]

This is the reverse of what I want. If we could start with text to image and then use this to paper over incoherencies, that would be incredible, but this is just taking the same loss of control that's the problem with every other AI art assistant and staging it so you ease into a bad image rather than getting one to begin with. I want to ease upwards into more control, not ease downwards into the same amount as before, from a higher starting point.

[@itskittyme]

what a time to be alive!

[@Josephkerr101]

First draw a circle then complete the owl.

[@kacperxt371]

I'm against generating a completely new image with AI. However even as someone who's against AI I must admit that this tool is an awesome tool because instead of taking away the creativity (like AI usually does) it actually makes creative works easier to make.