Denoising Data with FFT [Python]
ฝัง
- เผยแพร่เมื่อ 6 เม.ย. 2020
- This video describes how to clean data with the Fast Fourier Transform (FFT) in Python.
Book Website: databookuw.com
Book PDF: databookuw.com/databook.pdf
These lectures follow Chapter 2 from:
"Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control" by Brunton and Kutz
Amazon: www.amazon.com/Data-Driven-Sc...
Brunton Website: eigensteve.com
This video was produced at the University of Washington - วิทยาศาสตร์และเทคโนโลยี
![Computing Derivatives with FFT [Matlab]](http://i.ytimg.com/vi/BCexJ-yfdi8/mqdefault.jpg)
![Computing Derivatives with FFT [Matlab]](/img/tr.png)
the way this video is setup, overlaying him, the code and a whiteboard, is really slick
Dude this guy is crazy, I still cant believe these videos are free.Thank You for making it free means a lot.❤️
I havent even watched the video yet and I can tell Im on for a ride. The content looks so good.
I can't even believe these videos are made
I've spent many hours trying to apply FFT to my data and I've finally done it with your amazing explanation. For sure, the best Fourier Transform video.
Pure gold. Fourier analysis still feels a bit like voodoo to me as I'm just learning the basics, and your videos have been very helpful. The python examples are really handy. Thanks for taking the time to do these things in both MatLab and Python.
This has been very useful for me. I am a Mechanical Engineer and I am working in dynamic studies of steel structures. This method is very practical to apply to the acquisition of accelerometer data in dynamic tests. Thank you so much Steve!!!
I found it very refreshing to watch in MATLAB, try to code it in Python myself, then actually watching the explanation in Python to correct my mistakes! Thank you very much for all you did!
Steve Brunton, I may never meet you in person but you helped me a lot with these videos. I wish you a good health and a prosper life.
People like you make the world a better place. Free education helps everybody in the end. Thank you.
Professor Brunton, I love all your videos.
Brilliant. Just brilliant. The quality of this lecture is off the charts.
super-duper, as always. Python examples are what we need!
Thanks a lot again Prof! Just wanted to mention that, when calculating the power spectrum (PSD), the data type of the production result of complex number with its conjugate is actually: complex with 0j (Python3)
Your videos are crystal clear! I cant thank enough for sharing this high quality content. Loved the approach you took of writing parallel to the code!
This is so excellent. Absolutely the best, and most comprehensive video regarding real world FFT on youtube. Probably going to buy your book now. Thanks a lot!
Wow, thanks!
This is amazing! I am finally getting rid of the feeling that the topic is complicated thanks for these videos! I mean it is, but I also am capable of getting it with your explanation.
Amazing Prof. Brunton. These videos are extremely helpful for DPS researchers
this is mind-boggling-ly useful! (written after hours of scrolling in vain attempting to understand both the transform itself and the code that implements it .) Thank you!
Hello Steve, I just wanted to say thank you. It's been really helpful.
Very nice video professor Brunton. I think the idea behind is FOurier transform of white noise is a constant depending its amplitude. So for large signal-to-noise ratio , you will see a plateu + the real signal, then picking those peaks will work.
Hi Steve, I watched many of your videos in Control the comments there are disable so I took this opportunity just to say thank you
me too
Thanks very much! This really helps students who are struggling with denoising.
Thanks for such great videos! Been following you for years and I’ve learned so much and am truly inspired by your work. Just ordered the book and am excited to apply!
Beautifully explained ! Wonderful to learn from Steve !!
Thanks much Prof Brunton! This is gold.
just in love with the way u teach concepts
Thank you a lot, Steve! Your lectures are absolutely amazing and extremely helpful!!
one of the best channels I could've ever stumbled upon...immediately subscribed😅
Thanks for contributing! Exactly what I was looking for.
Thanks for your videos Steve.
Great application of the convolution theorem. Well explained.
One of the best explanation I have ever seen ❤️❤️
Thank you very much Prof. Brunton for the interesting video!
Thank you very much for this amazing video! I was struggling to understand this, but you made it look easy.
Thanks Steve for your knowledge.
This video is adorably stylish!
Thanks for showing us how useful is learning Math!! Amazing!!
Very good explanation. GG for the backwards writing, REALLY nice.
Many thanks!
Your lecture gives insight to the content what is in there books.
Thank you teacher for the awesome explanation!
Steve, you are amazing. Thank you a 1000 times.
You are so very welcome!
Wow, I am super impressed, why didn't I find this channel early. Thanks so much for sharing this valuable information.
Thank you for the clean explanation
Agreed. A wealth of knowledge.
Many many thanks professors this was very clear.
You are awesome, dude. Thanks a lot. I wish I had a professor like you at the college.
what a great instructor! Thanks!
Wow great content. Python example just boosted my interest in signals processing by many folds. Thanks a lot Mr. Steve, I really wish to see your behind the scene setup by the way :p
epic lighting. subscribed.
damn the video quality is amazing here. got a sub
thank you very much, you're a really good teacher!!
Effective! Your amplitude at per unit time teaching is above threshold.
A fantastic video with great informational content...and mindblowing production!!! How do you do all the overlays???
WOW, thank you for sharing this video!
I love all your videos.
This work really helps, Sir!
Thank you for your videos
Glad to hear that!
Very helpful content. I have used MATLAB but not python. I want to use FFT and some basic signal processing techniques to learn how to use python, so these videos are very helpful
thanks for great explanation! You guys are doing it on a different level :)
Thanks so much!
i liked and subscribed before even starting the video.. i knew it was going to be great!
Brilliant. Just brilliant. Thanks
Amazing lecturer!
thank you!!! amazing graphics!!! clear explanation) best wishes
Thank you Dr. Brunton.
Very nice work! Thanks!
thank you for the education kind sir!
math + code example is a great idea
this!
So helpful. Thank you so much!!
First I liked, then I watched. Amazing!
You just made my day. THANKS :)
I'm glad!
I told my self lets see what this guy is talking about...
Realized I discovered a wizard. I will have to see all your videos. I always was afraid from fft. Thank you for simplifying it. Thanks again.
I gotta say, watching these and re-remembering uni courses has been quite a blast. Thanks for your lectures!
Also, what exactly do you use to make these videos? I assume you write on glass and then flip the video? It always turns out surprisingly clean, with the screen projection as well...
Thanks! Yep, glass and flip.
Thank you so much sir ,i was really confused before this video
Thanks for your amazing videos and book. I'm watching them all, and I hope they'll help me with a Kaggle competition on detection of gravitational waves from time series obtained by LIGO and Virgo.
Take note: f is the noisy data
1. f_hat = FFT(f)
- f_hat is a vector of complex Fourier coefficients (increasing frequency: low to high) with its magnitude.
2. PSD = |f_hat|
- vector of real magnitude (power) for each frequency.
3. Filter: on PSD, keep the power > Threshold, and set 0 to the rest (because they are the noise frequency) and we have new f_hat.
4. f_denoised = iFFT(new f_hat)
absolute king!
Thank you..very good explanation.
wow , ure just great !!!! thank you !!! never pressed that sub button this fast.
really helpful thanks a lot!
precise and clear!
Thanks Steve, great footage! Any chance to extend this series with a explainer on how to detrend a periodic signal in 2D data?
The magnitudes of the signals with 50 and 120 Hz depends on the random numbers that you generate at the beginning of the code. I had this problem that my magnitudes where different from those presented in this video.
I ❤ FFTs. Thank you!
Thank you so much for those lectures! I love it! Is it actually possible to use Fourier series to create time series forecasting?
thanks professor, for this detailed video. I
Thank you very much Prof. Brunton for the interesting video! It is amazing this way of making videos. Which software and devices do you use for that? Is that a glass board? or something similar?
Yes, a glass board
@@Eigensteve and yet the writing isn't back-to-front... clever! 🤔
Dude, thank you! You saved me!
Good video, very helpful
鏡文字書けるのすごいですね!(
It's amazing to be able to write mirror writing!)
This video is amazing.. Just to the point !! I'm speechless on how good the video is.. ❤
Much respect brother!!!
Thanks .....great teaching ....
Well explained... 👏
You forgot to multiply your np.fft.fft(...) output by dt. np.fft module assumes the sampling spacing is 1. So we have to fix for this if we have a different sampling spacing. Also, you don't have to give n as the second argument of your np.fft.fft because n is also the length of your signal, so it's effectless.
Brilliant!!!
I had Steven Brunton as a professor at University of Washington in Applied Mathematics, and he was excellent. So glad I just stumbled upon his channel. It looks really good. Will be tuning in! I am hoping you will do a video on linear programming and optimization; I have a problem at work I want to apply this to. :)
Awesome, thanks Joe! I have some optimization videos in the works, so stay tuned!
awesome! :) Loving it! :)
Yes, THIS IS WHAT I WAS LOOKING FOR
THANK YOU TH-cam ALGORITHM
Love your presentation.
Interesting though that you got your code to work without errors.
Casting complex values to floats ist verboten! 😏 Here's how it should be for the graphs to display without error:
plt.plot(freq[L], np.absolute(PSD[L]), color='c', lw=2, label='Noisy')
However,
this does not solve the problem at it's root, because the Boolean return for a complex like `PSD` referenced to and integer as in:
indices = PSD > 100 does only return a Boolean array filled with `False`, unless np.absolute(PSD) gets implemented.
The best place to do this without undue repetition throughout the code is where the problem starts.
Here we see the amendment that forces absolutes and brings all the trouble caused by attempting to handle complex values like floats to an end:
PSD = np.absolute(fhat * np.conj(fhat) / n) # Power Spectrum Density (power of FFT)
However,
the error finds itself repeated in the last set of plots with the `Inverse FFT for filtered time signal` and is best corrected like this:
ffilt = np.absolute(np.fft.ifft(fhat)) # Inverse FFT for filtered time signal 😉
All in all, a very clear and understandable `toy example`. 💚
Loved the video
Thank you so much professor
Thanks for the explain, it is very details and easy to understand. One question, how can we demonstrate/apply the 95% significant test onto the plot? Thanks!
Superb video!
Thanks!
This is one of the best videos on how to apply FFT in Python!!! Thank you so much! Is it possible to make a video with Time Series data denoising? And could be applied directly on the original Time Series Data Or first we should obtain the difference on 1st grade?