A.I. High Resolution Texture Design (Wave Function Collapse #7)
ฝัง
- เผยแพร่เมื่อ 14 ม.ค. 2023
- 2023-01-14. Continuing uploading higher resolution results from the Wave Function Collapse (WFC) algorithm. I implemented it myself in software (single core, no GPU). Outputs to 256x128 texture results. Most of the source images are originally mine, and a few are based off of some trivial examples of the algorithm. Some generate more slowly than others due to more possible patterns to choose from.
The sidebar shows the parameters, including the pattern size, whether the texture wraps or not, if the patterns accepted are mirrored in X and/or Y, and if X and Y can be swapped (which allows for rotations).
Playlists:
--------------------
- Voxel Software 3D: • Voxel Engine #8 - Corr...
- Ray Casting 3D: • 3D Ray Casting Engine:...
- Graph-All Calculator 2D: • Graph-All Calculator #...
- Parallax Side-Scroll Shooter 2D: • The First Pixel: Dev V...
- Road Pseudo 3D: • Pseudo 3D Road #8 - in...
- Arena Shooter 2D: • Arena Shmup Demo #3 - ...
Websites:
---------------------
- GitHub: github.com/JDoucette
- Indie Game Studio: xona.com
- Blog: thefirstpixel.com - เกม
I think when collapsing a cell with multiple options, the probabilities should be weighted so that values coming from sub-patterns that are more frequent in the example image get favored. This might help with allowing more "empty space" if the example image has it instead of filling everything in with detail.
You are exactly right. Not only would this match the source image more closely (spacing and such), but it implies directly that the source image is part of the design: this may be obvious, but to put it into words... it means the source is no longer just how things connect, but in its totality, it's the general idea of what you expect as a result. This changes the way you "design" the input textures.
So I have this probability setting programmed in now... but I have not showcased it yet. It's mathematically working correct, and I even show the resultant percentages to prove it. But there are 2 serious issues:
Issue 1. the attempt to maintain near percentages means as soon as it reaches a near stable state, the remainder of the image "sticks" -- is restricted to -- keeping directly at that line. For a bad analogy: It's like a racing track to model the race line: if, on average, the car is in the center of the track, once this is achieved, it just stays there forever, without ever using the left or right sides of the track.
Issue 2. Sometimes the decisions made at the start dictate large portions of the image. You'll see in some of my other videos where the input texture is not ideal (like Mario characters, or NES games textures), and it gets stuck into a pattern it cannot get out of. The only path "out" is to undo, and other implementations have this sort of "back tracking" to handle this. I have an idea to handle this without back-tracking: Simply allow "correction" (changes to exiting data) to force a different result. The "engine" will have to handle conflicts (in this case, purposely introduced to "get back on track"), and resolve them.
@@JDoucette Awesome. Can't wait to see the results :) Maybe it is enough to just slightly increase the probability of the 'empty' color for things like sky or space. Or more precisely: when collapsing a cell where all previously collapsed neighbors have the same color, and that color is also an option for the current cell, have a separate (increased) probability for using that color again, otherwise proceed as normal. This might give an adjustable setting that allows for larger solid color regions without increasing pattern size.
This is great! I wonder if something similar can be done with audio, cutting up a musical piece into tiny segments and then stitching them back together with a WFC.
Oh wow. Interesting thought! If we split up music into separate tracks, and deal with each one individually (just for simplicity), then we have a 1D Wave Function Collapse, which is essentially just what note can follow others. But the 1D pattern to match can be large -- just like the 2x2 or 3x3 or 4x4 patterns I showcase -- so it can enforce several prior notes enforce a limit of what can follow. Maybe this is enough to possibly compose a decent riff?
If we added the other tracks, with additional rules, we'll have 2D Wave Function Collapse. This can enforce the various tracks are still in sync. Maybe this second dimension has to be less like space, and more like multi-dimensional -- since you sort of want every track to be aware of every other one... I guess that could be done with simple 2D, as long as the pattern is enough to cover all -- but you may want those track interactions to be less dependent on each other.
For 1D data you just end up with a Markov chain. Which used to be very popular back in the day and I've seen cool results when applied to music. But 1D doesn't let you jump around like in 2 dimensions, so it's more limited in what it can do.
Really nice to see some wavey, organic looking patterns!! 👍
Often I see this implemented rather grid like.
Oh interesting. I just uploaded another video that will surface in a couple days, and I find the same thing with most of my textures -- most seem to align to grid. Sometimes it's very cool, since it can be used for neat texturing or even level design (like dungeons). But of course, organic level design is very cool too (caves & caverns). I'll have to consider more organic forms for input.
This also suggests that maybe the algorithm itself -- given that the input texture produces sub-section (cells) or normally 3x3 squares, and the collapsing always impacts the neighbours of overlapping squares -- simply naturally produces such grid like structures. Even the wrapping, mirroring, and rotation are all aligned to grid or 90 degrees. Perhaps an algorithm that allows more organic flow (thinking floating point vs. integer) and having more fuzzy logic for matches (i.e. near matches could be accepted) would produce some wild organic outputs.
@@JDoucette Near matches make sense!
@@GemExtractor I have a feeling the IPM (Iterative Partial Match) algorithm does something like this. I also have a few ideas on how to add a new algorithm, based on the core quantum state foundation, that may be similar to IPM. The idea is to just collapse all patterns to 1 state, randomly. They will all fail to match their neighbours. Then implement cellular automata update style, where each "cell" (i.e. pattern) attempts to update to something better suited (near matches) to its neighbours. Perhaps a stability will grow out of this.
Would also be interesting to see how this works at completing the void around pre-seeded structures. For example, drawing a horizontal crossing wavy road in a forest using one WFC, and letting the second WFC generate the forest around that road with random clearings.
I think I understand -- so the void filling in your case may just be forest and trees, but not roads. Where it is filling the emptiness of an image that is mostly filled with pre-drawn roads.
The main reason I programmed this was I thought it could provide me a way to generate tiled textures that wrap correctly -- not just one texture to itself, but to various texture tiles. If you see my platformer game videos, some of the newer art has multiple square textures drawn at random, and they all match each other, regardless of how they are placed. This is quite a limitation. I had to do that by hand... so I wanted an algorithm to help me. While I never got that far, the idea was to seed it with the edges, and let it fill the insides.
Zero repeating/duplicated parts in the generation would be interesting to see ;)
I think I know what you're saying -- there are sections that are visibly repeating, and it would be nice to avoid them. However, the very algorithm is pulling a finite amount of data from a finite number of (usually 3x3) subsections from the original image, and thus it has to repeat.
However, perhaps we can avoid it from becoming visual. Maybe a range could be specified where the system dislikes the same patterns. However, the same pattern doesn't dictate a repetition will be visual -- it's more like a section of patterns (a large pattern made up of the smaller one) appears to be too similar to others. Nonetheless, maybe attempting to not allow specific patterns from being the collapse state too close to others may help.
@@JDoucette I think a random-or-exact (normal generation being the middle) pixel color density/distribution from the original input could change this, something like that. You could even have an oscillation/speed option where as the canvas is growing the pixel densities are going between random and exact, and you could even have a random option for the oscillation itself? Maybe oscillation could be applied to direction of the generation as well...like you have multiple starting points all doing different, but relative to each other things...just spitballin, great video :)
@@GemExtractor Thanks for all of the ideas. There are simply so many parameters you can tweak and as you said, perhaps some parameters could be themselves a wave of values across the texture. Simplicity is desirable here, and perhaps a good use of this algorithm would be for level design where the game dev author gives some hints, and lets the algorithm fill in the rest. Hints could be certain parameters that prevent (or invoke) repetition, or simply saying this part of the texture is already collapsed to water, so don't make buildings or paths here.
@@GemExtractor For repetition avoidance, I feel that the pattern density calculation is a good spot to merge this into. Right now, it just collapses to any of the available quantum states. But for density control, it should be aware of what has already collapsed, and attempt to self-correct (like a Tetris game that notices squares are lower count, therefore increase chance of a square -- p.s. Tetris acts more like Scrabble bag of 7 tiles, that is refilled every 7 plays). Part of this analysis on which pattern it should prefer to collapse to could also do a quick scan of a surrounding range and dislike tiles it already sees. I'll make a note of this.
would it be possible to store the time of creation for each "chunk" of texture and then make an animated texture that grows?
Could you explain more details? It sounds like an interesting thought. Each chunk of the texture has the time of creation as you see it in real-time -- what you're watching is the algorithm unfold in real-time, and the blurriness is the result of many quantum states averaged together that have not collapsed yet. So you are watching the texture "grow" to some extent -- thus I must be missing the core concept you're speaking about. To be clear, the starting "empty" state is actually all quantum states added together -- so the entire texture exists from the very beginning, and it "settles" (or collapses) to more concrete states.
@@JDoucette wow this is very complicated I have no idea about any of the math and stuff I was mostly curious about if it would be possible to use this to make procedural animated textures
@@wrighteously Since there is a butterfly effect here, where a single pixel difference at the start causes the resultant collapse to be vastly different by the end, the process by itself cannot create animation. However, due to the propagation of changes, this could be used to slightly animate the texture from one form to a slightly different form, continuously, showing a gradual change.
@@wrighteously This reminds me of a demo using cellular automata to generate textures. Cellular automata is well known with Conway's Game of Life, where the entire grid of cells constantly updates, so the whole grid is animating (even if it appears stable, it's still "computing"). Take a search for "Self-Organising Textures - Neural Cellular Automata Model of Pattern Formation" for a fantastic real-time demo of this being used to generate a texture that never quite settles, and therefore "animates".
this is really nice, but where is the AI
Thanks, Duality. A.I. is used really loosely here... basically it's the idea of the core concept of the algorithm itself, which is mimicking how a human would "design" something they are drawing or "rendering"... which is: If you were painting such a design, for a texture or a level design, or combination, you are using what's already drawn to better drive forward the design further. The stuff that naturally (is most easily added) is what you can do first -- in some sense, this is how a partial design can "design itself". The algorithm follows this. There is randomness for sure, just as in natural design, and there is inspiration from the seeded image, but the flow of creation is all about taking the part that is most "solid" (or least variable) given its possibilities, and it attacks and settles that part first. In this sense, given it is mimicking the way a human would do it, it's A.I. However, given people don't think calculators or chess engines are A.I. (at least not today), I doubt anyone would agree with me... even though this algorithm it is very human like. That's a long way of saying "it's not A.I.". Or maybe I should have said "the A.I. is in the algorithm". ;)
Why it hasn't error correction? Some of tiles are not collapsed because it impossible.
Good finding! I simply have not coded in error correction. I know some of the classic algorithms use backtracking, which has other uses such as limiting the number of certain objects. But I am more interested in getting a result close to the input image. So I have untested ideas... one is to simply allow the impossibility (which, as you see, happens), then to correct it on the spot -- using an "iterative improvement" algorithm that fixes the bad spots, creating more, but continually doing so. This could result in a significant recreation of much of the image.
The patterns fascinate me. Is the software available?
I don't have my ugly code available for the world to view quite yet. :) I should perhaps clean this up and make it available. There are online sources available from other people -- the algorithm is called "wave function collapse", and one of the first github results is the original implementation.
@@JDoucetteYes, I've watched countless TH-cam videos on the subject and installed the GitHub WFC software some months ago but found it awkward to use.
@@JellyMonster1 There are some tools made from this that are easier to use. Unsure if they are just GUI for non-programmers, or simply libraries that you can link in and use with a single API call. Try Google for some of these libraries, or look at the original WFC GitHub, and scroll down to see links to numerous other versions.
@@JDoucette Thanks for the advice Jason. It's mainly procedural map generation that interests me most.
@@JellyMonster1 You're welcome. One of my other videos is a collection of input textures that I thought would work well for level or map generation. I find that's the coolest thing about this algorithm as well!
The 1:05 is just awesome!
Thank you. It seems many people are enjoying the more natural, organic results, rather than the typical 90 degree mechanical, structured designs it normally creates.
Heh, the comment directly below states exactly what I was getting at...
> "Really nice to see some wavey, organic looking patterns!! 👍 Often I see this implemented rather grid like."
:)