A viewer made the excellent point that we technically need to sample at "strictly greater than" 2*omega. For the example, if you sampled at exactly 2*omega at the wrong phase, you could easily get something that looks constant.
4:20 It's a little bit nit picky, perhaps, but f is in Hertz or cycles per second. Omega is angular frequency and measured in radians per second. Omega=2*pi*f where the units on pi are radians per cycle and the units on f are cycles per second. The cycles part, being in the denominator and numerator respectively, cancel out leaving radians per second.
Fun fact: In Solid State Physics, the periodic spatial arrangement of atoms in a crystal basically "samples" the physical properties related to waves traveling in the solid. One then uses so called "reciprocal space" which is just a 3D fourier transform of the crystal for easy mathematical description. The Nyquist frequency in that 3D reciprocal space denotes the so called "Brillouin Zone", which plays an important role in Solid State Physics. For example, it dictates a condition on the occurence reflexes in diffraction experiments for determining crystal structures.
Steve, I really never comment on videos, but you are the best. If this can serve as a bit of motivation. please keep doing these amazing videos. You nail them every time. Let me finish by simply saying: Thank you!
4:20 - in a few textbooks the theorem is also named after another scientists: Whittaker and Whittaker (supposedly unrelated!), and Kotelnikov. So it would sound really serious: "Today we will prove the Nyquist-Whittaker- Kotelnikov-Whittaker-Shannon theorem!". BTW, in seventies (Masry, Shapiro and Silverman?) proposed "alias-free" random sampling. This was, as far as I remember, used (together with ocular microtremor) as a model of our alias-free vision due to random distribution of our rods and cones.
Great anecdote about why MP3s files are sampled at 44kHz. However, the extra 4kHz are there not because we can hear frequencies in the 21-22 kHz range, but because of how filters behave in practice. The extra frequency padding provides a good transition band for anti-aliasing filters. Also, love your videos btw. Been watching since taking AMATH 301 :)
The point at which frequencies fold is at omega over two e.g. Nyquist. In the PSD plot you show the folding to occur about the sample frequency of omega. For instance a frequency at 0.6*omega would fold down to 0.4*omega. Great video though with a great explanation. Subscribed - looking forward to watching more of your stuff!
You can get a flat signal if you measure at exactly twice the frequency. Imagine a pure sine wave at 1 Hz. You measure at 2 Hz and only find a flat signal. Then you managed to start measuring at the inflection points of the sine wave. Thats why I sample above twice the frequency. Excited about the compressed sensing series!
Thank you so much for your great explanations! You are making very complicated topics easy to understand - without losing information and oversimplifying it. Please never stop teaching :).
Actually 22khz is pushing it. The range is from 20hz to 20khz but by the time we are adults most of us lose those highest highs. 14khz is considered normal for adults over 30. The actual reason for 44.1khz is to allow for filtering at the high end. A low pass filter is put onto the signal starting at 20khz and sloping down to null at 22khz. This is needed to prevent aliasing.
Thanks for the video with intuitive. After watch your video, I think this way: Think the system as a black box, sample it at the 1X of highest frequency means you get a amplitude with no phase information, but with 2X sampling rate, you get the phase information, this amplitude and phase information gives you everything about this signal. It is like some aliens want to check what season the earth it is now, they need to check twice per year(they need to make sure within a year, the season). My two cents. Thanks again, I benefit a lot from your video.
Steve, these are great videos. However, at ~10:20 you misstate the nature of frequency folding. Recall that if you sample at omega, then the folding occurs around the omega/2 line.
excellent teaching, i should say! I was troubling with sampling theory and cam across this video. Certainly this is a great kickstart for me to continue! Thanks Steve and please don't stop making videos!
Great job by adding attractive background history. You explain nicely, and relate it with real life examples. I understand and learned something new because of you! Thank you!
Thanks to this video I understood in 10 minutes theoretical points that I would probably have to meditate on during 2 weeks if I had to perfectly grasp the ideas behind it on my own :)
Thank's for the video. Only thing I'd like to point out, is that saying "the highest frequency you care about" can be confusing. It should rather be "the highest frequency you receive as Signal". Otherwise one might think, that you automatically only measure the band below f/2, but actually you have to add a lowpass to manually cut off all frequencies, that are above f/2. Of course you explained aliasing (quite well), which is why I said "confusing" and not "misinformed". ^^
Steve, I like your videos so much. Thank you! In this video, however, there might be a small incorrectness: The angular frequency Omega is given in 1/s, which is technically the same as Hz. However, the unit of frequency can be given as both 1/s and Hz. Whish you all the best!
So having watched this one and the one on beating Shannon-Nyquist with compressed sensing, it looks like the reason that works for sparse signals is that the random measurements by dumb luck will have very high sampling rates within sub-windows of the overall measurement window. Would a measurement schedule with a fractal pattern be "optimal" in terms of highest reconstructible frequency for a given number of measurements made?
Can you please talk a bit more about aliasing in general, and how to detect that it might be taking place? I've noticed on modern mid-high end oscilloscopes that they seem to be able to detect that the signal is aliasing even though they have a limited sample rate. If I feed a spectrally pure (50+ dB over the noise floor) sine wave at 30GHz into a scope that can only sample at 8Gs/s it can still somehow detect that the signal is aliased and warn the user. I wonder how it does that? Is it heuristic or is there a deterministic way to tell?
I’m not sure how oscilloscopes do it, but you can also highpass signal in the analog domain and compute it’s power spectral density. If there is a lot of energy - you have severe aliasing. Another sign of aliasing is if changing the sampling phase slightly, you get different frequency power.
if i have a really short (duration) signal i am trying to catch, i go all the way to 10x if its a highly unpredictable signal (noise issues tho). If it is a predictable short duration signal, i like 5x.
Very concise. Thank you very much professor. I am just wondering that in the example, if I sample @ω，and the actual freq is 2ω, will the "aliased" signal be still 0.5ω ?
I have a question: how does the theorem account for the theatrically infinite frequencies between 2 Hz values? For example, between 100 and 101 Hz there is also 100.1 and 100.01 and 100.001 and so on, forever. How do we choose the "precision" of how finely we sample the amplitude, and which values, and the distance between the values? Can we pick between 100.00000001Hz and 100.00000002Hz? What if we stored information within that range? Could we accurately recreate it using a sampling rate of 202Hz?
apparently, for nonuniform sampling intervals, you can reconstruct a signal if the average sampling rate is twice the max frequency of the signal, as long as you have enough samples.
Wouldn’t the sampling clock be constantly falling in and out of phase with frequencies close to nyquist? How come u don’t get amplitude modulation artifacts from that?
If I have a signal sampled at 2*omega, then I will expect anything shown below omega in psd plot as real. But how can I be sure that it is not created by frequency folding of a real signal between omega and 2*omega? Say, I sample at 2kHz, and in psd plot I see a peak at 500 Hz. It can be either a real component at 500 Hz or caused by frequency folding of a real signal at 1500 Hz. In such case there may not be even a component at 500 Hz. Right???
Ok, cool, it wasn't just me that thought that was crazy. Although now that I think about I'm pretty sure he didn't, but instead just flipped the video after recording it. The gig is up Steve, looks like I just caught you left handed!
I would really like to see a better grouping of your videos or an continious online course. There are some playlists but certain items like this don't appear in any.
can you say anything about the tech how you filmed this? I was really confused when you could see that compressed was misspelled as compred. I assumed all those graps where just edited in in post? or did you actually project it somehow so clearly visible onto the glass? Or did you just have the graph on a teleprompter behind the camera and then didn't bother to fix the mistake in editing when you actually put the graph into the video?
Remember folks, N-S only applies to an ideal case, and frequencies near the limit get distorted (aliasing & non-ideality) with any real signal sampled on any real machine.
Great lecture. Surely random sampling is actually sampling at a higher sample rate than 2W. If you look at the period between samples (or the delta from an integer nyquist) you effectively are sampling above nyquist and just picking up aliases! You just aren't doing it periodically but over time you will build up the periodicity. The tradeoff that it takes longer to gather the data. So really you're not disobeying Nyquist sample rate. You are using a stroboscope effect where the strobe is non periodic so you will eventually see the whole signal.
That seemed like a bad explanation from my understanding... You only get mirroring across the y axis (and only for real signals), aliasing causes shifts by multiples of the sampling rate: 1.5-1=0.5 (note that there'd be two peaks being summed there actually if the signal was real, the other being from -1.5+2) *Edit: they would be square rooted, summed, and squared since you're dealing with power in this case I believe.
Also, if you're familiar with fourier transforms, a useful way to think about sampling is as multiplying your signal by an impulse train, where each impulse is spaced by the sampling period. That then turns into convolving with an impulse train spaced by your sampling frequency in the frequency domain, which in turn explains the whole shifting and adding process. Then you can think of aliasing as having spectrum components that will overlap when shifted (such that after adding you can't tell whether it's part of the original or a copy), so to not have aliasing your signal has to be band limited to half the sampling frequency. Edit: explaining the conclusion a little better: if you're signal is properly band limited to f_s/2 (a.k.a you're sampling at twice the maximum frequency component of the signal), your original signal will be centered around 0 between -0.5f_s (from reflection (ish, potentially different phase but same magnitude if we're talking straight fourier transform) across y axis for real signal) and 0.5f_s, a copy will be at f_s between 0.5 and 1.5, a copy will be at -f_s between -1.5 and -0.5, etc. No overlap, no aliasing!
America's Alan Turing. (Shannon) Communication Theory and cryptography are tied together. Twice the highest frequency is the sampling rate. Nyquist rate = optical sampling rate. Broadband signals works with Nyquist sampling. Compressed sampling.
Yes but it’s also done with two adc’s in parallel, sampling a so-called I an Q signal (The result of zif downconversion). At half the nyquist rate indeed but with twice the amount of samples. Comes down to the same overall amount of samples in the end…
A viewer made the excellent point that we technically need to sample at "strictly greater than" 2*omega. For the example, if you sampled at exactly 2*omega at the wrong phase, you could easily get something that looks constant.
The equality holds only for even functions, e.g., a cosine.
This is actually the reason I came to look at this video and it answered that. Thanks!
@Leland Hugh seriously? Is this the place for this? 😂
4:20 It's a little bit nit picky, perhaps, but f is in Hertz or cycles per second. Omega is angular frequency and measured in radians per second. Omega=2*pi*f where the units on pi are radians per cycle and the units on f are cycles per second. The cycles part, being in the denominator and numerator respectively, cancel out leaving radians per second.
Wow, I was just thinking about this having finished the video. Thank you for all the info!
it's crazy how clear everything becomes when the teacher knows how to make pretty drawings.
Thank you so much 😀. Who knew art was so important for math!
I have been a graduate student and have seen many presentations by Docs, PhD's, experts but this was a superb explanation and best backdrop.
Why isn't this man getting millions of views?!
How many people study signal processing??
Thanks -- I appreciate it!
actually in average around 1 million views for all videos per year for the last 4-5 years
@@raghavinder2161 compared to the total population they are likely sparse LOL
I am really impressed by how you teach, your students are lucky.
You make science really attractive. Thanks prof.
Thank you!
Fun fact: In Solid State Physics, the periodic spatial arrangement of atoms in a crystal basically "samples" the physical properties related to waves traveling in the solid. One then uses so called "reciprocal space" which is just a 3D fourier transform of the crystal for easy mathematical description. The Nyquist frequency in that 3D reciprocal space denotes the so called "Brillouin Zone", which plays an important role in Solid State Physics. For example, it dictates a condition on the occurence reflexes in diffraction experiments for determining crystal structures.
cool! thank you!!
Hands down, one of the best presented videos. Tools used + narration = Game on point.
Glad you liked it!
What he said and clarified at 10:56 made me understand what I could not understand from so many formal texts. Thanks a lot.
I come to this lecture after trying to deal with FFT. This is an amazing thing. Thank you very much Prof.
You should conduct an entry level communication module or signal and systems, this is really a masterclass quality explanation
Steve, I really never comment on videos, but you are the best. If this can serve as a bit of motivation. please keep doing these amazing videos. You nail them every time. Let me finish by simply saying: Thank you!
WOW...I have spent 3 years at university and they have failed to explain control/signals anywhere close to this. Love the energy dude!
4:20 - in a few textbooks the theorem is also named after another scientists: Whittaker and Whittaker (supposedly unrelated!), and Kotelnikov.
So it would sound really serious: "Today we will prove the Nyquist-Whittaker- Kotelnikov-Whittaker-Shannon theorem!".
BTW, in seventies (Masry, Shapiro and Silverman?) proposed "alias-free" random sampling. This was, as far as I remember, used (together with ocular microtremor) as a model of our alias-free vision due to random distribution of our rods and cones.
Great anecdote about why MP3s files are sampled at 44kHz. However, the extra 4kHz are there not because we can hear frequencies in the 21-22 kHz range, but because of how filters behave in practice. The extra frequency padding provides a good transition band for anti-aliasing filters.
Also, love your videos btw. Been watching since taking AMATH 301 :)
The point at which frequencies fold is at omega over two e.g. Nyquist. In the PSD plot you show the folding to occur about the sample frequency of omega. For instance a frequency at 0.6*omega would fold down to 0.4*omega. Great video though with a great explanation. Subscribed - looking forward to watching more of your stuff!
You can get a flat signal if you measure at exactly twice the frequency. Imagine a pure sine wave at 1 Hz. You measure at 2 Hz and only find a flat signal. Then you managed to start measuring at the inflection points of the sine wave. Thats why I sample above twice the frequency.
Excited about the compressed sensing series!
That's why Nyquist frequency upper border is not included in what it is possible to be measured without aliasing.
How much higher than twice? Is it appropriate to do 2.5X or something? Must it an integer multiple?
@@ericyip947no, it doesn't need to be an integer multiple. even just a couple hz would most likely suffice
Thank you so much for your great explanations! You are making very complicated topics easy to understand - without losing information and oversimplifying it. Please never stop teaching :).
Thank you Steve for making the most of this COVID year.
I've learned something new today. Human hearing is up to 22 kHz. I've always found Nyquist Samping Theorem very fascinating. Thanks as always Steve!
Actually 22khz is pushing it. The range is from 20hz to 20khz but by the time we are adults most of us lose those highest highs. 14khz is considered normal for adults over 30. The actual reason for 44.1khz is to allow for filtering at the high end. A low pass filter is put onto the signal starting at 20khz and sloping down to null at 22khz. This is needed to prevent aliasing.
Thanks for the video with intuitive. After watch your video, I think this way: Think the system as a black box, sample it at the 1X of highest frequency means you get a amplitude with no phase information, but with 2X sampling rate, you get the phase information, this amplitude and phase information gives you everything about this signal. It is like some aliens want to check what season the earth it is now, they need to check twice per year(they need to make sure within a year, the season). My two cents. Thanks again, I benefit a lot from your video.
Steve, these are great videos. However, at ~10:20 you misstate the nature of frequency folding. Recall that if you sample at omega, then the folding occurs around the omega/2 line.
Thanks David, great catch!
One of the greatest lecturers on youtube. If possible can you please take few classes on nonconvex, convex optimisation?
14:28 so FFT spectrum analyser is best example for that part ??
Your classes here are looking really good!
Awesome, thanks!
Your videos are really good. Keep up the good work. It's fun to watch your videos.
I never heard it explained so well!
So lucky to have found this channel, top notch 💎
Soooooooooooo cool!!! OMG I loved it! Thank you Professor, you saved my year with your videos!
excellent teaching, i should say! I was troubling with sampling theory and cam across this video. Certainly this is a great kickstart for me to continue! Thanks Steve and please don't stop making videos!
6:10 as you promised you will explain it in many ways 6:10 this part is Helpful to me and still watching..
Great job by adding attractive background history. You explain nicely, and relate it with real life examples. I understand and learned something new because of you! Thank you!
Thanks to this video I understood in 10 minutes theoretical points that I would probably have to meditate on during 2 weeks if I had to perfectly grasp the ideas behind it on my own :)
you would not dream for better explanation than this
At the example at 15:00, I presume you're using the "Fourier Series" version of the sampling theorem (since that function is very far from being L1)?
Thank's for the video. Only thing I'd like to point out, is that saying "the highest frequency you care about" can be confusing. It should rather be "the highest frequency you receive as Signal". Otherwise one might think, that you automatically only measure the band below f/2, but actually you have to add a lowpass to manually cut off all frequencies, that are above f/2. Of course you explained aliasing (quite well), which is why I said "confusing" and not "misinformed". ^^
Steve, I like your videos so much. Thank you! In this video, however, there might be a small incorrectness: The angular frequency Omega is given in 1/s, which is technically the same as Hz. However, the unit of frequency can be given as both 1/s and Hz.
Whish you all the best!
So having watched this one and the one on beating Shannon-Nyquist with compressed sensing, it looks like the reason that works for sparse signals is that the random measurements by dumb luck will have very high sampling rates within sub-windows of the overall measurement window. Would a measurement schedule with a fractal pattern be "optimal" in terms of highest reconstructible frequency for a given number of measurements made?
Very simple and clear explanation. Thanks Prof Steve
Can you please talk a bit more about aliasing in general, and how to detect that it might be taking place? I've noticed on modern mid-high end oscilloscopes that they seem to be able to detect that the signal is aliasing even though they have a limited sample rate. If I feed a spectrally pure (50+ dB over the noise floor) sine wave at 30GHz into a scope that can only sample at 8Gs/s it can still somehow detect that the signal is aliased and warn the user. I wonder how it does that? Is it heuristic or is there a deterministic way to tell?
I’m not sure how oscilloscopes do it, but you can also highpass signal in the analog domain and compute it’s power spectral density. If there is a lot of energy - you have severe aliasing. Another sign of aliasing is if changing the sampling phase slightly, you get different frequency power.
i like 2.2 -> 2.4 x. I find the 10-20% oversample really great in helping shorten sample time to ensure full wave replication
if i have a really short (duration) signal i am trying to catch, i go all the way to 10x if its a highly unpredictable signal (noise issues tho). If it is a predictable short duration signal, i like 5x.
Great video. I understand this stuff so much better.
Awesome, I'm so glad!
Also called Kotelnikov theorem
Whittaker-Nyquist-Kotelnikov-Shannon!
Thank you for the tutorial. Can you explain the Nyquist sampling rate needed for a still CCD image used in Astrophotography?
Great Breakdown! Deserves more views!
Thank you for creating this very clear explanation. Very helpful!
Hi, I really liked your presentation, but I wonder how your "blackboard" actually works. I love it.
Very concise. Thank you very much professor. I am just wondering that in the example, if I sample @ω，and the actual freq is 2ω, will the "aliased" signal be still 0.5ω ?
I have a question: how does the theorem account for the theatrically infinite frequencies between 2 Hz values? For example, between 100 and 101 Hz there is also 100.1 and 100.01 and 100.001 and so on, forever. How do we choose the "precision" of how finely we sample the amplitude, and which values, and the distance between the values? Can we pick between 100.00000001Hz and 100.00000002Hz? What if we stored information within that range? Could we accurately recreate it using a sampling rate of 202Hz?
apparently, for nonuniform sampling intervals, you can reconstruct a signal if the average sampling rate is twice the max frequency of the signal, as long as you have enough samples.
oh, didn't know he'd say that at the end...lol
I was doing HandmadeHero and the sound part was giving me some trouble, this helped a lot. Thanks.
Thank you so much! A few minutes needed to understand it intuitively!
Whittaker-Nyquist-Kotelnikov-Shannon saplings theorem. 👍
Wouldn’t the sampling clock be constantly falling in and out of phase with frequencies close to nyquist? How come u don’t get amplitude modulation artifacts from that?
Very informative, thanks a lot. I would love to see some content on digital signal filtering.
10:30 that's why Audio Engineers use oversampling when creating music.
You’re the absolute man.
Very good. I was playing along at home in Mathcad.
If I have a signal sampled at 2*omega, then I will expect anything shown below omega in psd plot as real. But how can I be sure that it is not created by frequency folding of a real signal between omega and 2*omega?
Say, I sample at 2kHz, and in psd plot I see a peak at 500 Hz. It can be either a real component at 500 Hz or caused by frequency folding of a real signal at 1500 Hz. In such case there may not be even a component at 500 Hz. Right???
Such a great video, thank you! 💪
Awesome, thanks so much!
Two times the angular frequency or two times the cyclic frequency. I am a little bit confused
I love it all and yet the stupid question I'm left with is : wait how did you write aliasing backwards? 0.o
Ok, cool, it wasn't just me that thought that was crazy. Although now that I think about I'm pretty sure he didn't, but instead just flipped the video after recording it. The gig is up Steve, looks like I just caught you left handed!
I would really like to see a better grouping of your videos or an continious online course. There are some playlists but certain items like this don't appear in any.
If we measure at 2omega, but not on peaks, then we also have some data corruption?
Very nice illustration, thank you.
This video is gold. thanks !
can you say anything about the tech how you filmed this?
I was really confused when you could see that compressed was misspelled as compred. I assumed all those graps where just edited in in post? or did you actually project it somehow so clearly visible onto the glass? Or did you just have the graph on a teleprompter behind the camera and then didn't bother to fix the mistake in editing when you actually put the graph into the video?
Remember folks, N-S only applies to an ideal case, and frequencies near the limit get distorted (aliasing & non-ideality) with any real signal sampled on any real machine.
This is Vladimir Kotelnikov's (USSR) theorem (1933).
Именно
Great lecture. Surely random sampling is actually sampling at a higher sample rate than 2W. If you look at the period between samples (or the delta from an integer nyquist) you effectively are sampling above nyquist and just picking up aliases! You just aren't doing it periodically but over time you will build up the periodicity. The tradeoff that it takes longer to gather the data. So really you're not disobeying Nyquist sample rate. You are using a stroboscope effect where the strobe is non periodic so you will eventually see the whole signal.
Love the new intro!
No actually THANK you . You sir are a very good teacher !
09:35 is confusing me: the true signal is reflecting around omega, not omega/2? Love your content btw! Will be getting your book for Christmas =]
That seemed like a bad explanation from my understanding... You only get mirroring across the y axis (and only for real signals), aliasing causes shifts by multiples of the sampling rate: 1.5-1=0.5 (note that there'd be two peaks being summed there actually if the signal was real, the other being from -1.5+2)
*Edit: they would be square rooted, summed, and squared since you're dealing with power in this case I believe.
Also, if you're familiar with fourier transforms, a useful way to think about sampling is as multiplying your signal by an impulse train, where each impulse is spaced by the sampling period. That then turns into convolving with an impulse train spaced by your sampling frequency in the frequency domain, which in turn explains the whole shifting and adding process. Then you can think of aliasing as having spectrum components that will overlap when shifted (such that after adding you can't tell whether it's part of the original or a copy), so to not have aliasing your signal has to be band limited to half the sampling frequency.
Edit: explaining the conclusion a little better: if you're signal is properly band limited to f_s/2 (a.k.a you're sampling at twice the maximum frequency component of the signal), your original signal will be centered around 0 between -0.5f_s (from reflection (ish, potentially different phase but same magnitude if we're talking straight fourier transform) across y axis for real signal) and 0.5f_s, a copy will be at f_s between 0.5 and 1.5, a copy will be at -f_s between -1.5 and -0.5, etc. No overlap, no aliasing!
Thank you so much, very clear explanation.
Is this video mirrored and lecturer is left handed, or is he actually writing in mirror mode (like Leonardo)?
How you done the video? Did you mirrored it in post?
Great explanation, thank you very much. gave my gratitude.
Is he actually writing on that board invertedly?
A very nice intro! Thank you.
Nice presentation!
Is this like putting 531441 as 1005260995 and6272254744 as 9947665379. Running at the right
Awesome as usual!
America's Alan Turing. (Shannon)
Communication Theory and cryptography are tied together.
Twice the highest frequency is the sampling rate. Nyquist rate = optical sampling rate.
Broadband signals works with Nyquist sampling.
Compressed sampling.
Starts at 7:30
You saved me the pain!
wow the explanation! Silky smooth. How you do that
it folds at w/2, not w. (minute 11)
Whow. More and more signal processing theory from you. Please
Is the video flipped vertically to show the direction of notes correctly?
Yes the video gets mirrored/flipped to show the notes correctly to the audience.
wonderful explanation!!
6:40 holy shit he draws on a screen, and he drew a perfect sinusoid
thank you very much Sir.
Great explanation, thanks.
Buen video. Me gusta la forma de la presentación. Perfecta. Y el contenido de maestría.
thank you very much !
Amazing stuff.
very interesting and anschaulich, thank you.
So good!
Video: *explains*
Me:
I can't see the lower half of this man
Flipping GOAT
Wow great video
How comes that a 5g signal is 100 MHz and it is sampled at 122.88 MHz?
Yes but it’s also done with two adc’s in parallel, sampling a so-called I an Q signal (The result of zif downconversion). At half the nyquist rate indeed but with twice the amount of samples. Comes down to the same overall amount of samples in the end…