Mean Shift Clustering
ฝัง
- เผยแพร่เมื่อ 7 ก.พ. 2025
- Mean shifting is the answer to my long problem of how to best determine the dominant color of a block. This episode has been a long time coming. A few months ago I was going to make a shorter video that was basically the first half of this but i got busy with other things and then learning about mean shift changed everything about the episode so I just put it off until I could spend the time on it.
More art by Whirdle: linktr.ee/whirdle
Mean Shift Pack: drive.google.c...
Discord: / discord
Patreon: / gneissname
Bluesky: bsky.app/profi...
BisectHosting was gracious enough to provide us with the Minecraft server we use for the community. If you are in the market for a server then you can use my code gneiss to get 25% off the first month when using any of their gaming servers. bisecthosting....
cant wait for polite shift clustering
Polite Static Spacing
kind shift clustering when
Hehe
polite static dispersion
Formal shift scattering
Gneiss is the only person ever to accidentally create a copy of a resource pack, but with an incredibly fast and surprisingly accurate method
Damm. I'm a programmer and the other day i've spent hours trying to find color clustering algorithms that solves prety much the same issue as you do and i couldn't find a thing that look like they would work for me.
If you would have told me i find the answers i was looking for in a minecraft video, i would have laughted at you !
Vector jump scare was iconic. Another great video!
ohh yea
We have Ben Vector’d
1:12 You are the leading expert on Minecraft color theory and geology!
probably the world record holder for "Geologist Who Has Thought The Most About Minecraft Color Theory"
5:19 NOOOO YOU SPAGHETTIFIED HIM
me walking a bit too close to the gorgeous black hole
Wasn’t expecting to see a reappearance of clustering algorithms in my life today 💀
We never do
i spent the whole video thinking about how cool a texture pack of this would look and lo and behold my prayers were answered at the end of the video before i could even ask😭
Oh I knew something was wrong with that cornflower!!
Minecrafts diagonal models are actually scaled to stretch the whole diagonal, effectively making all pixels 1x in height but root2x in width. It was hard to see clearly, but i swear you must have missed that for your fake version haha
Yep. My version was just text entity squares and I just rotated each 45 degrees. I figured nobody would catch me. 😅
this is extremly interesting for posterization with a known target palette. this has pointed me in the right direction for looking into that. thank you!
Watching this in my linear algebra class rn
probably learning more here
Always a great day when my favorite nerd uploads
I don't think the entire Minecraft art team has thought about Minecraft colors as much as you
I'm impressed each time you go to this world, it always looks cooler than it did before, its awesome seeing the gradual progress of this.
Another amazing video, the block entity stuff is so cool and unique
Always a joy to watch these videos! Since your OkLab video, i've implemented it for my own scripts, and have just been putting it in every project where i work with color. I think i'll play around with making this algorithm as well 😁
i have always wondered about this _exact_ thing when watching the previous color world videos. i had no idea how you would solve this. the solution is genius. thanks for the great videos. keep it up
This is the stuff I play with in my free time, but amplified to levels I couldn't even dream of approaching in my little Excel workbook of color math. I love seeing these somewhat behind-the-scenes style of videos in between larger updates for these ongoing projects. I always get exposed to some new-to-me concepts that I can rotate around in my brain when I need to have a relaxing diversion.
I watched multiple of your videos in the past and just now realized they were all suggestions on my feed, I wasn't actually subscribed... Had to quickly change that! The quality of your videos is just outstanding and easily understandable as someone who's knowledge about color theory ends at knowing that there are other color models than RGB :D
your videos are soo cool keep doing what youre doing
Seeing how you've turned what you've researched into a texture pack, what if you made a texture pack that limits the color and palette to the limitations of older video game consoles? Seeing how some blocks have such little color variation, console targets such as the Sega Genesis and Super Nintendo might not look that different, even when confined to a 16 color palette. But consoles like the NES and Gameboy Color could be more visually interesting.
Just play minecraft shareware
12:14 I'd recommend using a dynamic sphere size, growing or shrinking them until you have the desired count of colors.
Start with defining the range of radius: the maximum radius is the furthest distance 2 points can be from each other and the minimum radius is 0. Then use a binary search to hone in on a radius value between these extremes that matches the desired color count.
Don't think you need binary search in such low values and it might skip some magic number since clustering isn't quite linear.
I think we could iterate some range for the sphere and choose the result that minimizes/maximize a function (something like " cumulative difference of the clusters"/"number of clusters")
2:26 bro just disassembled a flower 💀
Image processing is one of my biggest fascination within programming, and even more specifically quantization, I hadn't explored this specific algorithm before, very interesting!
I was gonna ask for a resourcepack of this, but you already did that. Amazing!
I'm actually planning to do a mass ingame building project in the distant but foreseeable future, so your color theory videos are very helpful for some of the minor details. Thank you, Gneiss.
the displays looking so good
Amazing as always, thanks for putting together this presentation!
I have read academic papers that use k-means clustering and didn't know what it was, great explanation! Also that was the first time I saw a sniffer egg, did not realize that was a block
Amazing ep! Learnt so much in so little time! Love the work put into this one!!!
It's 2025 and even gneiss is doing ML content now.
That was a joke before anyone misses the point. But k-mean clustering is introductory to more complex ML.
it's funny that it is a term that comes from graph theory
@qexat yeah from geology to math. how the channel has changed.
The proof and theory is graph and the application is ML.
"Mean shift clustering" is such a clunky name for such a relatively straightforward averaging method
leave it to mathematicians to make a stupid name for something relatively simple
It’s precise but unilluminating, like many technical terms. But I’m not sure what else you would call it really, it does make sense once it’s explained.
1:52 "You average the numbers from smallest to largest" 😅
🙃
These color space videos are some of my favorite things to watch! Will you eventually make a video on the math or code you use to work out these arrangements?
I’m amazed at how much work this must have taken!
4:04 he uncotta the terra
I was expecting something like DBSCAN or OPTICS to be mentioned, only because those are what I've personally used. Glad to see ones I haven't before, especially since some of these seem pretty simple to use in comparison.
Great video as always!
I know that with the mean-shift algorithm, you can also use the coordinates (x,y) along with the 3 colors to calculate the distance in a 5D-space and group together pixels that are not only close in color but also in distance, which might be useful to produce smoother textures. If you have tried it, did you notice any difference ?
I haven’t tried that yet. I did some test doing that with kmeans and it’s interesting. Definitely useful for something.
Ayyy Whirdle my man!!
12:28, havent finished the video. i wanted to say this now. the size of the object encompassing the color points needs to be dynamic, to fit the clusters reasonably. i dont know if it always works, but a simple way of doing this is to use a sphere with radius some ratio of the diagonal of the bounding box of the color space.
That's really cool! I don't know if you already know, but what you are trying to do is a very know problem in machine learning. Those kind of algorithms are called classification algorithms. In your examples you used mean squared error and mean squared error with an hard cut, there are other functions that could be tried. If you are interested we can talk about it!
We got Gneiss baking yay
I think that instead of comparing the colors as if they're an array, it would be more accurate to cluster the colors based on their contrast to surrounding pixels on the texture map. Then you can even average clusters that are close in distance for consistency. This leads to method that is much more in line with our own visual perception, which is very much based on shape.
It might also be a good idea to look at different kinds of color spaces for calculations, since color distances that are mathematically equal might not be perceptually equal. Correcting this would make the results more in line with what we percieve
9:09 this reminds me of barebones and even my own SimplyCubed pack, minimal with the basic colors of each block
Edit: 12:57 well, would you look at that. Amazing
stats class coming in clutch on this one
“How did you learn colour theory” Minecraft
Bro is posting this from Bdubs's basement...
I don't know what's happening.
By the way, there is actually another way to apply the concept of a median to multidimensional space. I doubt it'd be any more useful for your purposes, but it's called the geometric median. It's the point which minimizes the sum of the Euclidean distances between it and the points in your set. Might be difficult to calculate within a Minecraft world though
YES WE GOT ANOTHER FLASHBACK USER LESGOOOOOOOOOOO
cool! Last year i also worked on my program which used k-means clustering to divide an image and then it converts it into an STL files for 3d printer based on the color. But never saw a mean-shift algorithm.
a while back i wanted to turn images into color palettes and was dissatisfied with k-means, this looks promising!
I loved this video
yes i will have gneiss day
This guy is so underrated. But maybe that's a good thing. The best channels are usually that smallest ones, who build their own community from scratch rather than jumping on the latest trend. Not that jumping kn trends are bad. But there's something more to them than just.. yknow?
Did I mention I love your videos? Because I love your videos.
Thanks! Be sure to join me next time when I talk about my top 5 hardcore mod packs made by ai!
@gneissname That actually got me LOL
Fantastic Video!
The averaged colours look way duller than the original blocks, maybe because bright colours stand out more but get dragged down in the average.
So if the goal is to find averaged colours that feel the same as the original varied colours, it might help to use a “perceptual mean” that’s biased toward brighter colours (or maybe more saturated or something idk).
Well done.
Funny I have an exam in a few days featuring the K-means algorithm :D Another method that might be appropriate for these small color pallets is K-medoids, which is basically K-means but it forces the "means" to be actual colors (samples) in the data.
it would be so cool to have a texture pack with mean shift clustering textures
13:02 ohhh neat
*@[**08:32**]:* In computer science, this is called non-determinism, and it can be annoying.
*@[**01:52**]:* I assume you mean "sort"?...
I love your mug
So cool, I love the way this uses machine learning to help quantify all this color data! Just neat!
great video
This texture pack with a ps1/ps2 style shader would go hard
I like this video
Did you look into self organizing maps? The're a type of neural network which reduce the dimensionality of data and in basic terms they find clusters in data, so that sounds very applicable
Gneiss Video.
Great video! You are the only person whose videos I watch right away...
and also...
Axiom has added a gradient maker and colour palette UI... would there be a more optimal operation than using the OKLAB average (which it uses rn atleast for the Gradient Maker)?
Welp, im not entirely sure what happened or what the main goal was, but i learned some things
Have you perhaps played around with the geometric median? It's in my opinion the most natural extension of the median into >1-dimensional data.
It turns out that in general, the n-dimensional mean is the point that minimizes the sum of squared distances from the input points.
And in 1 dimension, the median is a point that minimizes the sum of absolute distances from the input points. This definition naturally extends to >1 dimensions, giving the geometric mean.
Very cool.
Maybe someday Ill do this to a modpack like Gregtech new Horizons ?
kudos for showing a cool things.
As an idea: you could release the tool that you used to generate the blocks but for people to use on their skins, so if people want to play on "simplified" servers they can match the informational density of their surroundings.
Whoooo new vid
Damn that's a cool algorithm, thanks for the demo and explanation (also this would've massively helped with one of my post grad projects years ago, darn it!)
I wonder how much scaling the axes hurts the algorithm? Since it's looking at a sphere of data points, if one of your axes was scaled much more extremely than another it bought struggle to find good clusters (though this is more of a data cleaning concern than an algorithm limitation per se) (and this is also not an issue for pixel data since RGB are all using identical scales)
babe, wake up, our geology minecraft guy just posted new autistic colouring science video. (but seriously, amaizing video
Might it be an idea to try and blur the texture before the clusters are made, then take the vernoi cells that those cluster points create and apply that to the original texture?
NNNNNERRRRRRDDDD
(good video tho)
I TWICE misread the title as "Mean Shit Clustering"
hell yeah i love this shit
goddamnit, who the hell put 2 dislikes.
Barebones explained.
Spare!
it makes sense to me if you'd only use this more complex method for blocks which don't lend well to the other method where you just smoothly blend all of the values, if the bandwidth is too small then you have too many clusters, but if the bandwidth is small then they're all very similar anyway, right?
yeah, there is something I didn't show in the video but before i have organized textures based on their total root mean squared distance, each pixel to the mean, and i think that comparing that to mean shift will show some point where all of the colors are close enough that clusters don't work well. So maybe there is some rms value below which you just average the entire block and thats the dominant color
it would be nice if there's a way to apply this in building in game without having to use the pack
I feel like you could brute force this by just making a wall of a specific block, and then taking screenshots of it at increasing distances, and then just using the dropper in paint.
Then again you said this was for close distance sculptures so that wouldn't work here.
I think a better aproach to this would be processing first the rgb values to get what I will just call on the spot "wycgmrbb" (white, yellow, cyan, green, magenta, red, blue, black) values, the reason to this is that rgb values don't really represent the color of something, for example rgb(255, 255, 0) is not red neither green, is yellow and only yellow; now with these wycgmrbb values you can separate the colors in groups with colors who share the same color pattern, on top of this I would calculate the brightness of the colors and subdivide the groups based on how close in brightness the colors are.
If someone wants a more long explantion with more details and the why of this, just ask for it and I will write one
You should try the DBSCAN algorithm with variable epsilon, it's infamously known for these kinds of N-dimensional data set problems (like that in an image or 3D object), which is employed throughout the entire internet for recommendation algorithms, like that here on TH-cam.
.
What if you use k-means to determine how many clusters are needed before using mean shift clustering to determine values?
Sounds like the mean-shift clustering algorithm you're using estimates the variance of the data from the given points, but since you have a defined range, you could instead set constant variances to be used across all blocks. This would eliminate the issue with red carpet, as it would understand that the points are all close together and not spread out.
mean average? i was hoping for a gneiss average
Based clustering in minecraft lul
You just automated the process to make the Minecraft trailer resource pack
Commented this before I got to the part where you literally did this oops
8 views after 37 seconds.
Gneiss you gotta give it YOUR ALL
I disagree that median represents the bulk of the data because all the values lie close to it. If you had a bimodal distribution of numbers the majority of your data points could be very far away from the median. Medians are useful because 50% of the data is on either side of it and it's not affected by outliers (as I'm sure you know but just for everyone else). (I assume this was just a misspeak in the moment, but wanted to point it out as incorrect)
And for those who don't know about all these statsy things, since it comes up a lot, all of mean, median and mode are important, there's no one best measure because it depends on the context you're talking about. And good statisticians will always combine a measure of centrality (mean, median, mode, etc) with a measure of spread (range, interquartile range, variance, etc).
what if you combine mean shift with the first average method??
Funny, I just recently learned about k Means clustering algorithm.