Block Texture Noise
ฝัง
- เผยแพร่เมื่อ 7 ก.ย. 2024
- Trying to quantify the distinguish blocks based on their textures, not just color.
Thanks to Grassy for the Ultrakill video.
Discord Link: / discord
BisectHosting was gracious enough to provide us with the Minecraft server we use for the community. If you are in the market for a server then you can use my code gneiss to get 25% off the first month when using any of their gaming servers.bisecthosting....
Howdy - Image Processing nerd here.
The term you were looking for, in an image processing sense, is kind of confusing, "Frequency" - Low Frequency patterns vary in value over larger periods than high frequency patterns.
Kernels are a pretty standard way to calculate this, with more robust approaches using stuff like Fourier Transform and DCT Analysis, but when working with students I tend to explain it as, "Blur the image, then sum the differences for every pixel (Absolute values!)"
This comes up in computer graphics pretty often, human eyes will "pop" towards high frequency patterns and will recognize high frequency repeating patterns much more easily. Years ago Minecraft added the ability to randomly rotate and variate textures into resource packs to help limit this visual repetition, grass blocks being the main example of this.
Love your work as always
Hey! Thanks for the info. I look forward to getting back from my trip and trying DCT out on this.
Thank you for the info!
I was thinkinɡ, what happens if you use the square sum of this difference?
To add to this, the black/white patterns you presented are really easy to select for using a 2D Fourier transform, for example vertical columns would show up as dots on the horizontal axis. You can do the same with RGB images, but the results will be sensitive to the color space you use.
Also, related to @shadamethyst1258 's question, summing the the squared differences is equivalent to summing the squares of the whole fourier transform, except the center pixel (which is the average). This is related to the square of the image's RMS (average instead of sum), and to its energy (the sum of squares, without removing the average).
You can also look into spectral power density if you want units that are easier to work with, but Fourier is best if you need to combine multiple blocks.
Great work!
@@gneissname Maybe go even further an do a wavelet transform, which is kinda an optimal in-between of kernes and fourier, in order to get better space resolution at high frequencies. Aka stand outs can be pinpointed. This means that you can still include additional information relevant for builders, such as how noise is distributed. The downside is that the visualisation is higher dimensional. (A minecraft texture has two space (and technically one for time) and three (color) frequency (technically one for transparency) dimensions. Plus one dimension per analyzed wavelet for measuring how strong that combination is present. This means that for one wavelet we get already six dimensions. This is just about the amount you can practically visualize in minecraft, with three spacial dimensions and three color dimensions. Maybe a bit more including transparency.) So maybe this makes it a bit less practical for displaying as an infographic in a minecraft world, but would still be practical when the spaces or colors are projected down, for automatic generation or for sorting by certain properties, like appying filters to the transform depending on from which distance a build is intended to be viewed and then showing the block color space accounting for that.
The name actually makes a lot of sense when thinking about sine waves (sound, light, etc.). On a graph they always average to zero and different frequencies of the same amplitude (intensity/energy) are percieved differently and can even be distinguished and filtered by simple electrical circuits
The color distance is very close to a string concept called edit distance. If you aren't already familiar you may enjoy learning about that. We used it for DNA sequence comparison in my old job.
To explain edit distance from a programmer's point of view, for those who are too lazy to look it up -
Think of it as "What is the lowest number of single-character changes that would it take to transform one string into the other?"
Those changes are typically defined as:
- Additions; adding a letter, making the string longer (e.g. "case" to "chase")
- Deletions; removing a letter, making the string shorter (e.g. "dame" to "dam")
- Substitutions; replacing a letter with a different letter, keeping the string the same length (e.g. "gain" to "pain")
I can see this quality in particular making edit distance useful in working with DNA, as those are three of the basic effects that mutations tend to manifest as.
For example, "edit" and "distance" have a fairly high edit distance, by my count 6 - starting from the beginning of "distance," add "e", (skip past "d" and "i",) delete "s", (skip past "t",) and delete "a", "n", "c", and "e".
On the other hand, "edit" and "editor" have a much lower edit distance of 2, because "edit" is a sub-string of "editor", which only adds "o" and "r".
It's heavily used by spell-checkers and search algorithms. Google's "did you mean?" is a good direct example of this in action.
@@TheKd8lvt I think I saw it being used in google's diff-match-patch library, makes sense
@@TheKd8lvt did somebody say chase :)
is everyone who watches these a fucking field scientist
It's so fascinating to see how concepts like this are used in so many scientific fields.
You may want to distinguish between color noise and value or luminosity noise. For a block like cobblestone or a redstone block, the noise is mostly in value, whereas ore blocks have a lot of color noise (hue and saturation).
In a perceptual color space like Lab, value noise is just the noise in the luminosity component, and color noise is the other two components together.
Value noise is more useful for adding texture but color noise is very situational.
Oh that’s a good point!
You have the colour jargon I couldn't figure out. I definitely agree with the concept. In practice, this would be calculating the average distance of the colours after a projection from a 3d colour space into some sort of 2d surface, it could look quite interesting animated into the colour world. I wonder if there is much value to other projections, so instead of projecting onto the hue/saturation plane, what would projecting onto the hue/value plane or saturation/value plane tell us, if anything?
I was gonna comment a similar thing. Planks appear less noisy than the
ores
@@bwjclego Not really. The value axis simply contains a lot more information than the chroma axes. Hue vs. saturation doesn't matter much in this respect. Think about a black and white image - visually, almost all the important information is still present. You can make out almost all of the important details. You can usually easily detect the edge of objects and still read text. Details that are only distinguished by hue and saturation are much more subtle, even when you add the color back. If you remove the luma axis, the image is unrecognizable. Try reading red text on a green background when they're the same brightness.
This makes sense when you think about it from a physiological perspective. Your rods and all three types cones contribute to your perception of brightness. Color is only driven by two aspects - the difference between the long- and medium-wavelength cones ("red" vs "green"), and the difference between the short and long+medium wavelength cones ("blue" vs "yellow"). The sensitivity of the cones overlaps substantially, which further reduces the information in that space. The contribution of all three cone types to brightness means it also has higher spatial resolution - individual types of cones are spread out because they have to share space with the other types.
Lossy compression (e.g. jpeg) takes advantage of this by allocating fewer bits and lower resolution to the chroma channels. Likewise for analog forms of image transmission. Analog TV allocates a lot less bandwidth to chroma, as do analog recordings. Artists also take advantage of this. They use differences in value to draw users attention to the important parts of the subject. Areas will differences in color but not in value add context but aren't the focus.
Also, while chroma is two dimensional, there isn't one objective way to organize it into two axes. Hue and saturation make sense from the point of view of communicating color, or designing a color scheme, but physiologically you perceive color on red-green and yellow-blue axes. The different ways to organize brightness are roughly equivalent in terms of conveying information.
wtf the ending lol
I think someone made an Ultra kill map for him
For him? Nahh it's of him its endless survival mode in Ultrakill and you can change things such as sky texture and the title and as you can see well it's all him
I thought it was a bug with TH-cam.
its prob to reach the 10 min mark for ads
@@FE0003 you actually haven't needed 10m for years. It's 8m now.
I love that you have the mind to tackle unknowns head first, and that there's such smart people working in similar fields ready to help in the comments.
It’s great to have a community like this. I like to at least try to find a solution my own and maybe approach a problem from a different angle, but then when I post my attempt I get a lot of great feedback on other ways to do it and then I can learn from that.
This would be hella useful for pixel art generators. The ability to algorithmically translate texture on an image to texture on a block would be really useful
the ultrakill bit was fun
real
I think your "kernel noise" is trying to quantify the frequency profile of the noise/texture, you'd want to look into Fourier transforms and JPEG compression.
I thought the same thing, computing a DFT for the blocks and using that to distinguish high-frequency vs low-frequency noise would probably yield more reasonable looking results.
The kernel approach is essentially counting the high frequency noise. Changing the kernel size would change the frequencies you care about. That's a pretty good approach, though there are a lot of options.
> Using a kernel that is a line instead of square would pick out vertical or horizontal features. E.g., a vertical log texture vs a sideways log texture. Think about how jarring it is to have a wall of vertical logs and then have one which is sideways.
> You are using the average color distance from the average color. This is the L1 norm. If you used the mean squared distance, you'd get the variance, which is sometimes a more useful number. That would be the L2 norm. There is also the L-infinity norm, which is a fancy way of saying take the maximum color distance from a pixel to the average color. All three of these metrics could be useful in different scenarios.
> One more thing to consider would be to apply different weights for chromaticity and luminance. In some builds, having different hues is far more jarring than having lighter or darker variations of the same hue.
A log-scale on the kernel-noise axis would probably make blocks on this graph easier to understand. The regular noise axis might also benefit. Alternatively, maybe a -1/x scale could also even things out and make differences in noisyness more obvious.
For the first part, where you get the average distance of the pixels from the average color is also called the standard deviation. One way you could get more info out of that is by doing it separately for each axis of a color space. For example, while copper ore and dried coral are similar in average color and are both noisy, we would care about the noise in the saturation more than the noise in the value/brightness
What immediately came to mind when you were showing the checkered blocks that averaged to grey was the jpeg family of image compression.
It might be useful to covert the raw texture 'matrix' (either by color channel or some scalar factor like luminance/saturation) into a frequency matrix (using DCT) then calaculate the range and peaks of that.
i somehow always forget how well made your videos are. thank you for putting actual effort in your videos!
Thank you, this one was a bit rushed for my liking but it’s going to be almost a month before I can get the next one out so I wanted to at least post what I had here. I knew that my kernel noise wasn’t exactly what I wanted but I knew if I posted it someone would tell me how I should do it within the first ten min lol.
Two things that came to mind while watching:
1. Trying the average squared distance might be interesting, since it gives high "weight" to outliers in the computation.
2. Trying the average distance between all pairs of pixels, instead of the average distance from each pixel to the average. The result would be similar but would let you distinguish between cases where the pixels are clustered into a few tight points and where the pixels are all spread out from each other.
had this in the background and got jumpscared by the ultrakill gameplay lmfao
another fantastic video! I couldnt hear you over Ultrakill, but i loved watching it lol
Yooo another vid to remember the whole school program
DEVIL'S VORTEX SAWS?!?!?
😭😭😭 I can’t escape it
@@mrman12w YOU CAN'T ESCAPE
You can never escape the Devil's Vortex saws
time? i can't find it 😭
@@zorpglorpthumbnail (reversed)
Legitimately so happy you made this video, I've been thinking about this conceptually for years and you managed to put it to video form phenomenally.
I think a good experiment would be to change the size of the kernel noise sampling to be smaller (likely to 3 x 3), since I personally consider blocks which have defined areas of low noise which contrast with eachother (such as some of the glazed terracottas) to be less noisy than blocks which have more sustained noise (such as cobblestone or warped/crimson planks). The large kernel sampling brush size you chose likely biases towards considering these blocks with separate low contrast regions as high contrast due to being more likely to catch both regions in one sample, whereas it moves blocks with more localized contrast such as cobblestone lower on the contrast hierarchy because it mixes in more datapoints that average out the contrast which would be percieved on a local scale.
In any case, I think it could be quite interesting to see how changing the sampling area affects results
3:39 im not a color specialist either, but the term “Local Contrast” popped into my head, Kinda like edge detection type thing
It might be a good idea to add a second plot where the vertical axis is the ratio of your "kernal distance" to "color distance" This might spread the blocks out more instead of them mostly lying on a line (due to the 2 measures being highly correlated). Also your "color distance" sounds a lot like varience or standard deviation. If you square the distances before adding them up and dividing you will have varience, if you take the square root of the varience you will have standard deviation.
Woah new Gneiss upload. What a blessed day it is.
One thing that would be very interesting to see is the textures with the most and least amount of hexes. I would expect to see something like snow with only 3 hexes being the least. I'm also pretty sure that the concrete powders have an insane amount of hexes.
True. I was manually changing each pixel in a concrete block's texture, and sampling 6 pixels in a vertical line, NONE of them has the same Hex... So I gave up working on it.
@@groundedgaming lmao
I for one would love to hear about what sort of paleontology work you are doing in montana. Very gneiss video as always.
One thing I want to bring up is in regard to all that z-fighting! I have an idea of a way to fix it.
You take a radius around each block's centre, and for each block within that space, you stack blocks outward. This would add data that the unused axis would display.
For blocks that overlap too little to be stacked, you take the space of a block's side and apply it onto each block's origin point. Then, for each overlapping block, you move a block inward by the slightest bit. This idea might be *best* with some manual work, even if an algorithm makes it work.
I think a neat way to use this color distance is to use it to sort the colors into groups. For example, the copper ore block is divided into three distinct groups: the grey, the green, and the orange. I don't know how you would set about making an algorithm to identify these groups, but being able to plot blocks by these color groups (perhaps by having one instance of the block per color group) would be quite useful for selecting palettes
Extract all the individual colors and represent them as rgb coordinates, then increase the distance threshold to find points where a meaningful set of nearby points forms.
Just wanted to say you’re the highest quality mc channel i’ve ever seen on youtube
Fourier transform is what comes to mind.
You won't (initially) get a single value out of it. Instead you get the spectrum of the noise.
But it feels like you may be ultimately seeking info on how that noise is distributed per block.
High frequency noise (changing a lot every pixel) versus low frequency noise (small changes between adjacent pixels).
I think the right kernel/convolution could be essentially the same thing.
You can’t escape. It is subtle, but the devil vortex is there.
I had a mathematically similar issue doing some computational protein research. I found that with things like this, it's really easy to just add more and more layers of differentiation until you go insane.
In my particular case, I was looking at a large family of proteins called nuclear receptors and categorizing them because they serve different functions.
To analyze them further, I divided each of those nuclear receptor sub-families into groups based on which coregulator protein they were bound to.
Then, each of the nuclear receptor sub-family coregulator groups into their bound peptide groups.
Then each of the nuclear receptor sub-family coregulator-peptide groups into their bound ligand.
Then each of the ligand-categorized nuclear receptor sub-family coregulator-peptide groups into the amino acids that the ligand would form a hydrogen bond with
Then organized each of the bonding-residue separated ligand-categorized nuclear receptor sub-family coregulator-peptide groups into the probability that the hydrogen bond would form in an experimental setting...and so on.
At some point, even though there are more variables that could be changed in theory, you just start to run out of data to analyze and your sample size gets too small to really tell if the statistics you're doing actually show anything meaningful.
With the color world, you've gone from separating blocks into their different color spaces, then noise values, then kernel noise. With there being only a couple hundred blocks in the game, I feel like you'll hit a point where there's not enough data to see if the visualizations you're making really mean anything.
But of course that's what science is all about: trying to figure things out and sharing what you did with the world, no matter how unimportant it might seem.
Just don't spin your head too much with abstraction and sub categories lol.
another way to classify blocks that might be interesting would be something like "highlight color" or "contrast color" where instead of ordering blocks based on their average color, you order them by the color that's most different from their average color. e.g. the copper ore block would be classified as having orange highlights and the red mushroom block would be classified as white highlights
Somethings that seems a good distinction in terms of noise would be noise in terms of variations in the saturation of the same colour, in lightness/darkness of the same colour (like the redstone block that only has variations of red), and variations of colour (like the copper ore that has grey, orange and green).
long shot, but i would love to see you take a look at the geology of the mod TerraFirmaCraft as it completely overhauls world generation to include detailed stone types (including gneiss) with mechanics base around the style of rock. for example, Malachite will only generate within regions composed of marble or limestone and stone tools made from sedimentary rock will degrade faster than tools made from igneous intrusive.
If you're going to leave the noise profile in the world, since it's only 2D you should leave it up against that wall and have it display at the same time as the regular color visualization so the user doesn't have to switch back and forth and can just fly around looking for what they want
It's been said before, but the frequency distribution of a 2D-Frourier Transform would probably be the best way to quantify noise, at it would tell you how fine the noise is, and how much of it there is.
I think some instances such as the red mushroom block or the diamond block would be considered more solid given that their color is almost entirely uniform. While there are some specific spots of significant change, the overall color is largely the same. The few white dots or the white "shine" lines don't change the fact that the red mushroom block looks red and the diamond block looks blue.
I'm not sure that this is relevant in a strictly mathematical/color theory sense, but if the idea is to look at the noise in terms of the context a given block fits, it could be worth considering.
tldr; red mushroom blocks and diamond blocks look pretty much solid red and blue respectively, even if they technically have some noise.
This reminds me of that guy who hid barely visible entities with a noisy background to make a disappearing block trick
Thank you for explaining this!!! Me and my friends have been working on a map for a while, And we have a friend who gets on occasionally to hang out When we're building. He likes to try to help out In different ways One of which is by coming up with color pilots for builds But he only ever focuses on the color of the block 😅 We've tried to explain to him that The texture of the black is important to but he just doesn't seem to get it. Definitely will be sharing this videonext time it comes up. He really likes your color videos so hopefully this speak to him in a way he understands Cause I don't think we're explaining it in a way that He gets. Or you know he's just messing with us.
You're probably gonna wanna used root mean squared difference for the kernel noise, so the more noisy kernels have more impact-that way the mushroom block won't count as lower than diamond.
The rest of the video was so calm I nearly panicked I was so caught of guard by that ending! Definitely gave me a chuckle
One idea I had was to calculate color volume, for blocks with more than two distinct colors. Essentially placing the colors of the texture in that 3D display space, taking the convex hull of all the points, and then calculating the volume of that. Essentially it’d give an estimate of the perceptual color spread of the texture
That’s a cool idea. It would distinguish textures that have high contrast but basically two colors, like glazed terracotta from something that has a more diverse palette.
Not to overcomplicate things, but I feel that separating by Hue and Saturation is much more important than lightness in some cases, as this tool is for colour harmonies. Blocks with a lot of disparate colours like unseparated ore, glazed Terracotta etc. are going to be harder to work into "block as a pixel" type building vs something like a Note Block which is essentially 100% shades of brown but varies quite a bit in terms of local lightness. The Mushroom block should be lower for that reason, maybe, but if you're already taking this into account it could explain why the Diamond block was higher as it seems to shift between shades of aquamarine, turquoise etc.
love a good graph. shoutout gneiss
My idea on processing the noise is to take the 3d graph from the beginning of the video, flatten it by removing the y axis, then performing a similar "average distance from the average" that happened at about 1:20 (except one that cares about direction). You're left with a polar coordinate that determines what colour the noise is, and how intense it is. For example, gold ore would have large yellow noise, however a diamond block would have smaller orange noise, because moving from light blue towards white means adding orange.
However, thinking my method through for things like copper ore and a diamond block makes it seem pretty difficult to use. Copper ore has two types of noise, orange and green, so without some method of splitting up the different colours of noise, you'd end up with it having some weird in-between colour. For the diamond block, it's just unintuitive that it has orange noise. It's a mostly blue block, you'd assume it has 'white' noise, but that's not available in this current system, you'd need to make it even more complex to account for this. Another problem is that it's impossible to graph every block at once like this, since each block's 'origin' is a different colour, trying to put them all together would be a complete mess. Honestly, my proposal probably isn't worth it for that many blocks (maybe just use it for the various gray blocks, and perhaps the types of wood).
Man i love your minecraft animations
ty grassy for blessing us with the gneiss ultrakill
I think organizing by brightness and colorfulness would be a pretty interesting and potentially useful graph
this reminds me of when i had to make a mosaic out of emojis so i got average color values of emojis. had to incorporate a lot more logic than i thought i would.
I love these color videos! But I must say I miss the geology stuff I hope you make some more soon. Whatever you do keep it up cause these videos are awesome
Thanks, geology stuff is coming. These shorter color videos take a few days to do from start to finish but the geology ones take much longer, at least the meaty ones do. So these are usually me just taking a break for a bit. Been busy with real work and life lately but I’m still making progress on the next one and have many more planned out.
this feels like the type of minecraft content etho watches. hope he sees this
found an interesting mod recently called MCRGB that lets you put in a hex code or use color sliders to find blocks that contain that color or colors close to it, even compatible with blocks from other mods or resource packs
Brutal, Anarchic, Supreme, Sadistic... GNEISS
You can think of the kernel size as a spatial frequency, so your first noise method measures low frequency color difference, and your second method measures high frequency color difference. Instead of increasing the size of your kernel, you can take the Max or Median local color difference after a small kernel pass.
You could categorize blocks by “directionality” like some blocks look like they are oriented in one direction, some not, some the other, etc
I think it'd be kinda cool to do a version of this but specifically only with blocks that show up in worldgen. Not really for any specific reason, I just think it'd be cool to try and quantify the average "noisiness" of the minecraft world with the textures you find!
You know, I have had a video idea sitting around similar to this. I wanted to look at the color distribution by biome and then maybe structures too.
Hm, I feel like the reason why it seems like the blocks with higher noise have higher "kernel noise" is mostly just because the blocks with more noise overall will tend to have more localized noise also, such as with how you're sampling for the value in repeated 5x5 areas.
I think something to improve your noise measure would be to measure changes in hue as more significant than changes saturation or luminosity. Copper ore seems very noisy because the orange has a lot more contrast against the gray. Whereas something like emerald blocks might be decently noisy by your current metric, but feel decently uniform color wise when used in a build.
My thoughts on how to sort them:
- First like a lot of people are saying, some kinda fourier transform or something might be good since frequency is what you're trying to quantify and that won't tie the axes together like you see
- My gut reaction for quantifying how spread out they are is to instead use standard deviation, it's how you'd typically represent that and might give cleaner results
- But another thought I had: what if you did a linear regression and then had the error on that be plotted? You have a third dimension to work with, and one thing I noticed is some things have a their colors all in a line, like stone variants, but other things have the colors all over the place, and quantifying whether the differences are scattered between many colors or just points on a continuum of colors seems like a useful thing
For that art term, I’d refer to that as a form of contrast, or focus contrast.
Having filters of 0-0.75 or 0.25-0.75 something of noise would be super useful to have less clutter you're not planning to use, and for the latter letting you pre-filter blocks that might look too flat from the start
I don't understand what you're talking about half the time, but I think I would appreciate having a graph like that in the color world anyway. I think it makes it easier for me to understand the general uniformity of the blocks and makes it easier to pick out specific blocks as a one-off to break up a recurring texture in a build.
For someone as color deficient as me, I'm always very interested in all your videos about color theory in minecraft, but I do miss having your geology videos too lol. I'll always look forward to your uploads man
The analysis method you seem to be yearning for here is the Discrete Cosine Transform. The 'average color' is just the (0,0) 'dc offset' term in the DCT representation, while higher order terms give correction values that get progressively finer (higher frequency) along each axis to reconstruct the image. You can then take sums over regions of the DCT to derive how much "energy" a texture has at the associated frequencies.
One thing that might be interesting would be to look into jpeg compression and discrete cosine transforms. This splits an image into higher frequency and lower frequency components and might be a useful way to figure out how noisy a block is. (It's basically a 2D fourier transform for images)
Im so excited for this video, it’s an issue with palette generators Ive seen since forever, but nobody has so much as mentioned before
I admire the breadth of your knowledge
if/when you do add the option to sort by noise (or whatever its called) you might want to change it to be on a log scale so its easier and more intuitive to specify the difference between the large cluster of blocks towards the bottom
this dropped just as i freaking needed it
minecraft and ultrakill. the perfect combinaion
The official minecraft website says "Dripstone, in the real world, is called limestone" and someone on reddit explained that "minecraft stone isnt an exact stone but could be Basalt/Basalt-Andesite/Trachybasalt/Basaltic-trachyandesite"
Maybe it could be interesting to do one where there’s a color wheel disc on the floor representing true unchanging color (representing x and y), and then blocks raise “higher” in Z above their average color to represent noise? It doesn’t represent the two “kinds” of noise you just showed but it’s at least a mix of average noise + color in a way that might be useful for spotting “gaps” in places that could be filled (like if non noisy block of x colors are empty, it would be a good thing to know when making a build, or if one color had lots of similar blocks filling out the noise range, that’s useful for gradients)
Yeah that would be interesting and pretty easy to code with what I have already.
Noise could be interesting to look at in separate terms as well, rather than complete difference in colour it could be cool to separate it into hue noise, saturation noise, and value noise. They all are useful in different ways in building too
The diamond block doesn’t feel noisy. You should adjust it so it compares against a larger radius. Additionally add a layer of noise over the texture first so single pixel difference are less effective
Hello Gneiss, could you do a video on how ores are generated in nature ? Genuinely curious to know, in a digestible way like yours, how concentrated and localized ore veins are created in certain spots and not in others !
i like to imagine the same analysis, at mojang, new blocks are to be added, artists maybe instructed to remain within a narrow range of these meta metrics
A lot of people are talking about Fourier this and that but I think what you're looking for is just a covariance matrix of the distribution of pixel colors in 3D color space. That preserves information both about how much it varies and in what direction(s).
Sure it discards information about the physical position of the pixels, but I don't think that's actually super relevant to a build. The fact that they can be viewed from different distances gives a sort of scale-invariance, and if a build is small enough compared to the size of a block that that's not the case, then the builder is probably just going to eyeball it. (edit: to clarify, what I mean is that fourier analysis is looking at wavelengths, but these blocks are only a meter in size, so the wavelengths are all going to be some fraction of a meter, meaning for a decently-sized build they're just "high frequency" and the distinction between them doesn't matter)
At the very least it's worth looking at covariance alongside the local neighborhood stuff, it's sort of a 2nd-order correction to the 1st-order average color of the block.
I’ll look into it. I would love to have a few criteria to discriminate the texture of a block, with three you can display them all at once with each axis. Depending on what I can figure out there might be a few options so you select what you want on what axis or something. Orientation or linearity of color groups might be something too.
@@gneissname Yeah I guess the problem is that the covariance matrix is six degrees of freedom by itself lol, it captures both orientation and linearity though.
I think it makes sense to plot the magnitude and direction of the biggest eigenvalue, so that way you get a sense of how the color varies and in which direction, I think that's the way that preserves the most info.
@@ohno5559 @gneissname If we are talking covariance we might as well take it a step further and do PCA. That will be very effective at compressing things down, like you said the data loss doesn't matter a lot here but might as well lose as little data as possible while reducing the dimensionality. He could start by calculating the frequencies with DCT (what jpeg uses) he is going to get 256-dimensional data from this, but he can then throw out the dimensions corresponding to high frequencies that wouldn't be visible to a person anyway (just like jpeg does lol) and then do PCA to reduce the rest of the dimensionality. PCA should be able to crush this down to low dimensionality without losing much of the data's variance because the patterns of most Minecraft block textures are the same in both directions. Once he has the principal components, instead of taking the top few like you normally would he could manually fish out the first to correspond to low frequencies and the first to correspond high frequencies and if he wants a third axis to plot with, he could take the first component he hasn't taken yet. Also, you would want to do this a few times throwing out more and more dimensions from DCT before doing PCA for different build distances since the further away a texture will be viewed the less the high frequencies matter.
@@Jack-lp3gc What I described is PCA, PCA is just diagonalizing a covariance matrix. I think throwing in pixel position data would hurt more than it would help because that would mean there are more degrees of freedom than there are data points.
We already have a natural 3 dimensions with the color space, I don't think it would help to add a bunch of dimensions just to remove them. Might be worth trying though, I dunno.
Oh oops, the cutoff for when youtube puts the rest of the text behind "read more" perfectly landed where your first paragraph ended and I never noticed or read the second one. Also, I got curious and started working on this myself last night, pca was able to bring the 786 dims ( I used all 3 channels) down to 3 dimensions while preserving about 80% of variance. With pc1 containing 60% of it. @@ohno5559
Have fun with archeology! That's super exciting!
A directionality kernel sorting (a separate vertical, horizontal and maybe diagonal) would be cool to see!
you could use local and global noise instead of kernel and block texture since the former is less implementation-specific (and that's generally something you should aim for imo)
I should add that noise PERCEPTION, vs actual rgb values are two different things. A yellow block with somehwat noise seems way more muddy than a red one, as the luminosity of yellow skews the actual percieved noise a lot.
The issue being... I can't tell you have to adjust for that other than to use the relatively linear scaling used to judge the brightness difference of colour, which I still don't think is *wholly* accurate.
Same thing with noise from hue shifting, is different to the darkness value.
Ultrakill is such a great game.
I just wanna say, thank you for these videos they are very infomative! And very impressive work doing all the flares of animation moving things around! it all looks amazing!
Wait, Paleontology? BEST MINECRAFT TH-camR!!!
- since the x value is the overall noisiness and the v value is the local kernel noisiness there should be a maximum y for a given x. maybe one could try to "normalize" the y axis by multiplying it by 1/x² (eyeballing it, idk if thats correct) with x and y scaled to [0,1] since otherwise most of the graph would just be dead space.
- another idea would be to display the graph by kernel size, where x is the kernel size and y is the resulting value for that kernel size. each block would of course be plotted once per x value and one could trace out the path of a single block
Using the alt marksman is pretty based, can’t lie.
You should try looking at the hue noise for each block it might give a better result than the kernel noise. For example the diamond block and copper ore have similar kernel noise but they should have different hue noises. Or it could be a new axis or combined through some function with the kernel noise.
It feels like hue differences should be weighted heavier than value here, since a block like gravel or dead coral, while noisy, is a lot more homogenous than something like glazed terracotta, ores, etc. Put another way, a 17% difference in value might be expected for a given texture, but a 17% difference in hue (i.e. red/orange or blue/green) would be quite notable and stand out more.
You are probably right, but what I didn’t say was that I did all of the distance calculations in oklab which should alleviate some of the difference in perception changes?
3:14 you would love JPEG compression algorithm
Maybe standard deviation done over multiple frequency ranges. Duplicate layer, blur it, subtract that from the original, get the standard deviation. Downscale the blurred layer, duplicate it, blur that, subtract it, get the std, etc. constantly downscaling each iteration. Maybe also scaling up low frequency std values more. Then you could do a graph of std at each frequency range.
Could also look into discrete cosine transforms.
There also neural networks for visual feature extraction and then using u-map to bring the high dimensional data into 3D with some clustering if desired. Would be a bit of an issue using these tiny textures so you'd want to upscale them with some sort of edge directed method like for pixel art or use a premade upscaled texture pack. Stuff like VGG has a lot of downscales internally so 16px would probably be a challenge for it. 128 or 224 might be alright.
I wonder how dynamically placing animated blocks would look
lava seems to switch between low variance and high variance, so I feel like it would float around the graph a bit
It might be more natural to average the squared distance from the average colour, which is how you'd calculate the variance.
You could also calculate the covariance matrix to see the correlations between the different colour components, but im not sure how you'd use that.
6:28 I think the diamond block might be an outlier because of how its diagonal stripes intersect kernel: large uniform areas, but many colours within each square sample?
*the kernel
(can't edit comments on my phone?!)
With a bit more time on this it would have been interesting to try different kernel sizes to see how that affected everything. I did my initial test on those extreme cases and they might not be a good test case bye to them being just black and white. I think there is a third way to look at this problem and see about textures that have an orientation and not just “grouped” colors. Like the diamond being diagonal or the observer having linear areas.
I’d say small amounts of high contrast (most notably, ore) is far more noisy than very textured blocks of similar colours, and I feel like those cases don’t get caught by average noise or kernel noise (i guess it would be colour ‘range’?)
yeah there is something to capture there and I might try looking for bimodal distributions of color, basically seeing if there are two peaks when looking at a histogram. maybe seeing how things fall into a standard deviation plot, if its not a gneiss bell curve then something is up.
i was tabbed out listening to your video and i heard the slide and i was like WHAT I THOUGHT AUTOPLAY WAS OF- ohhhh its the same video, WAIT HE PLAYS ULTRAKILL HELL YeAH
Feels like I'm watching a SethBling video from way back
It might be useful to compare the hue of the noise to the hue of the rest of the block (eg. if it's just lighter/darker spots of the main colour vs a different colour. )
I think some other options for ways to differentiate textures might just be different weights for luminance and chroma, and perhaps a measure of the variance between pixels, or that same value but starting with blurred images of the textures. (to capture different scales of noise)
If possible a measure of how structured the noise is would also be amazing. (so like, a brick or plank textures vs gravel) but I'm not so sure how to go about implementing such a measure.
Something that could be interesting to read up on, is the Structural similarity index measure
It would be really cool to have three meaningful measurements so it could be plotted in the dimensions.
Not related, but I'm currently working on the entire 24-bit color palette pixel by pixel (contains 16,777,216 colors) in minecraft 16-color combinations, Delta E included with list of glass (pane)(/dye) color mixture (sequence of 6 in total). It's unknown when the project's conclusion is going to be revealed, but I'm feeling a bit courageous about this.
I think it would also be interesting to sort the blocks by the maximum slope between neighbouring pixels, basically which blocks have the highest contrast. Also I feel like that you would want to find the median difference of the kernel values and not the average, since the colors are logically not randomly aranged/grouped. I'm not a statistician though so that might just be a feeling.