Lesson 9B - the math of diffusion

[@ssw4m]

Thanks for explaining this math, concepts, and some of the difficult notation.
If I'm understanding this correctly, I will try to express the "add some noise" function in a simpler way:
let x be the current state (image),
x' the next state (noisier image),
n a unit normal distribution (the noise) with mean 0 and sd 1,
β a blend factor in (0, 1), where 0 gives no change and 1 total change to noise,
α = 1 - β, the corresponding blend factor seen from the other side; 1 gives no change and 0 total change to noise,
b = √β the weight given to the noise,
a = √α the weight given to the current state.
Note that a² + b² = 1,
so (a,b) would be (1,0), ~(0.7,0.7), (0,1) for no change, halfway, and full noise.
Then:
x' = ax + bn
I.e. The next state is a weighted average or blend between the previous state and unit noise, such that the sum of the squares of the weights equals 1.
This choice of blend function reminds us of Pythagores' theorem, and is appropriate for blending perpendicular / orthogonal / uncorrelated values, like a clock hand turning from 12 o'clock to 3 o'clock, 0° to 90°, goes through a point sin(45°), cos(45°) = √0.5, √0.5 ≈ 0.7, 0.7, or blending red and green to make "rainbow yellow" (with 0.7 red and 0.7 green). I guess noise is uncorrelated with everything, so that makes sense.

[@timandersen8030]

Thank you for this explanation. Would you happen to know also the reason ax is called the 'mean' in the video if it is just the pixels data in the current image being weight/blended down? Is it called 'mean' because it is the anchor value before noise (i.e. variance) shifts it either left or right?

[@michaelmuller136]

Great bonus material, easy to understand, thank you!

[@bf2825]

I’ve watch this again and again. Then I read the paper and codes again and again. Finally I can understand this video 100%. It’s the best explanation🎉

[@howardjeremyp]

Great job!

[@geekyprogrammer4831]

Can you make it easy for us like which paper and codes did you refer?

[@digishgabhawala7861]

Thanks for comments towards end.. I really did not understand math much but I will come back to this again during 2nd run and hopefully get it more with practicals.

[@timandersen8030]

Thanks for this math overview of the diffusion process and its history!

[@odilmode]

I appreciate amount of work done here, Thank you !!!

[@amortalbeing]

Thanks a gazillion times. can someone please tell me what Jeremy is saying in @19:25 ? if we subtracted the means from those "what"? first?

[@howardjeremyp]

Subtract the means from each of the vectors your taking the dot product of.

[@amortalbeing]

@@howardjeremyp thanks a lot doctor greatly appreciate it.🥰

[@danlewis92]

@@amortalbeing To add to that, if a random variable is normally distributed with mean mu and std sigma, then subtracting the mean gives you a random variable normally distributed with mean 0 and std sigma (translating the original to the origin)

[@amortalbeing]

@@danlewis92 thanks a lot really appreciate it

[@bosepukur]

wow, appreciate the effort

[@mreron]

In my understanding a pearson correlation coefficient (PCC) of 0 implies only linear independence. However, it does not mean that two variables (pixels) are independent. For instance two random variables X and Y which are related by the unit circle equation X^2+Y^2=1, have a PCC of 0, i.e., they are linear independent. However they are nonlinearly depending on each other, cf. en.m.wikipedia.org/wiki/Pearson_correlation_coefficient#/media/File%3ACorrelation_examples2.svg for an illustration.

[@无可奉告-n5o]

I can't access the discussion forum for this video and the forum of 2022 part 2, both pages say "Oops! That page doesn't exist or is private." I think it's because it is private, when will it be publicly accessible?

[@MirrorNeuron]

same here

[@matveyshishov]

Can we please have TanishqGTP, to interactively discuss papers and ideas with him?

[@Ip_man22]

Really cool video!!