It doesn't. He should have done this with op amp circuits and it would be fine. Even if he skipped the window fudging thing to cut the pop, he could pick better transforms and meet success. I mean, why didn't he match the song effectiveness with what he demands from the filter; blam, a sing wrong fixer that has funkadelic qualia.
Funny that you say that, because this seems rather simple to me. No, I don't understand it at all, but the result is great for how "little" he did. I expected this quality to already need a huge amount of processing that only the gods at Antares (autotune company) have figured out.
@@LuLeBe The wizardry in autotune is all about interpolation between notes. The notes they sung, and the notes they wanted to sing. The repitching part probably isn't all that great if you just want a static effect. Like, imagine the math in this video, except the ratio between original pitch and target pitch is a function of the frequency content of the signal you started with.
@@Cineenvenordquist Ehm, no, what he's shown in the video is pretty much as simple as it gets for performing pitch shifting, which is just inherently difficult. I can't even guess what you could be referring to with the extremely vague "op amp circuits"... the function of an op amp in basically every circuit that uses one (except when used as a comparator) is to solve an equation, but the actual equation it solves is entirely defined by rest of the circuit, so describing something as an "op amp circuit" is utterly non-descriptive. Regardless, the fundamental problem of pitch shifting are not specific to how the audio is represented, and so are the solutions except they're much harder to implement using analog hardware than digital filtering, and getting phase to match for transient sounds seems like it would be particularly painful.
The original Alvin and the Chipmunks albums took advantage of speeding up the playback to change the pitch because they weren't being digitally processed. This means that the artists had to sing at an appropriately slower beat to account for the time shrinkage.
@@charleslambert3368 the way these things were recorded the singers sung lower notes at a slower beat. The instruments were played at normal rate. The singer track and the instrumental track were combined later.
Last semester did a project about phase vocoders at an audio signal processing course. Your explanation about is absolutely superb, we had to give a presentation at the end of the course and it was train wreck compared to what you managed to explain in 15 minutes. Well done!
6:28 The reason pitch modification is listed as "yes" for PSOLA is because it's the only algorithm that doesn't change the formant, which is important to produce natural sounding speech. Unlike SOLA and WSOLA, it doesn't introduce a chipmunk effect when speech is sped up. Because of this, it's heavily used in many autotune plugins today. It's possible to tell if a plugin uses PSOLA by shifting the vocal down: - If the formant sounds the same and there's gaps in the waveform, this indicates PSOLA was used, as it has no way to fill in those gaps. - If the formant sounds the same and there's no gaps, this indicates a frequency-based approach was used, as it can fill in those gaps. - if the formant sounds different, it's likely another approach was used.
Super smooth. Clear, concise and thorough... and then having a real demo audio to follow examples is a great presentation feature. As someone interested in spectrograms I'd like to see a video of which audio properties create the most problems for a shifter vs which properties are easiest to work with. I've got some guesses but I imagine you could demo it in a similarly illuminating video.
This video is truly amazing. I hope you make more coding content and simplify other random topics for literally anything. If it's just a one-off you still did amazing and I wish you well on future endeavors
Nice. My degree is in Physics. I assumed this was done with Fourier transforms, but I had always wondered how the finite sample sizes were handled. Basically, how they were glued together. This was well presented and I can see how much more there is to learn.
I lost it right around 11:00-ish. Nice to see where my current math knowledge tapers off. Hopefully I can revisit in a couple years and be able to understand the whole video. :)
This is an incredible explanation! The visuals are great, and the way the process is laid out ("lets try this -- oh that causes such and such problem") really helps understanding the rationale behind every aspect of the algorithm. I knew, as a physicist and electronic musician, that pitch shifting was hard. But I never knew how it worked, just that from a math side I couldn't come up with a good solution and from a musician side that choosing the right algorithm for a given sample was often difficult. This explains so much!! What a satisfying algorithm and a satisfying explanation.
9:15 expanding on this, for N samples of time domain data, you get N samples of frequency domain data. however each frequency is a complex number, using two samples each, dc offset gets one. the nyquist frequency also gets only one, because phase shift between nyquist sampling frequency and input frequency at nyquist results in amplitude change. for every phase shift there is an amplitude that will make it fit the samples, and hence only one can be represented (and we never know which it is).
Though note that FFT outputs N-1…N/2+1 (note descending order) are the complex conjugates of outputs 1…N/2-1 and therefore are redundant, so if you discard those you've got a real numbers for DC and N/2-1 complex numbers for frequencies between DC and nyquist, and a final real number for nyquist, which all combined conveniently fits in the same amount of memory as the original N real samples :-) (All this is assuming N is even; if N is odd then there's (N-1)/2 frequency samples between DC and nyquist and no frequency sample _at_ nyquist.)
@@MatthijsvanDuinExactly, values are mirrored with nyquist bin being the center point, real part of mirrored values is the same, while for imaginary part the sign flips. So mirrored value of x = a + bi would be y = a - bi. For example, with 1024 frame size, 0th coefficient is DC, 512th is nyquist, and every other coefficient 1024 - n, where n is coefficient number from 1 to 511, is a mirrored version of coefficient n. 513 is mirrored 511, 514 is mirrored 510, etc. I assume that you know this, but maybe someone will learn something new.
Content presented around 8:00 is a bit misinformed. STFT preserves all info. Discrete time-domain repr doesn't include any info above Nyquist freq either. When pitch shifting, freqs higher than Nyquist is supposed to be lost, so scaling the frequency axis is not problematic. HOWEVER! The video up to 8:00 was very cool and taught me a lot of time-domain methods I didn't know about. Very inspring, and I appreciate that!
5:32. To be precise, auto-correlation is a convolution between signal and itself, but delayed, this is what auto- means, in this case it is just "correlation"
This is something I’ve wondered about for a long time! Nicely done. It gives me hope when I see quality videos from small or new creators show up in my recommendations 😃
Excellent video! I've implemented these algorithms many times by now but having this video 5 years ago would have saved me a huge amount of time and headache :)
Wait, why do you reimplement this bunches and why don't you just use an op amp circuit or get a singer who likes working to spec? Or a marimba with motorized dampers? Do you have to teach elliptical equations on python driven instruments?
Long before the existence of auto-tune there were pitch shifters that were used in live performance situations to eliminate audio feedback from the speakers to the microphones. These units had a very clever approach which I wonder why is this not used for auto-tune? The approach was based on something very similar to super-hetrodyne theory. The audio was multiplied by a carrier signal and the modulated signal was filterd to remove one of the side bands and then this filered signal was multiplied by another carrier signal that was slightly frequency shifted from the previous carrier frequency. This demodulation proces gave an output signal that was slightly frequency shifted from the original.
What you describe are frequency shifters, not pitch shifters. When you apply too much of that, it messes up the harmonic relations of the sound. (A couple Hz of shift is not very audible, but removes the audio feedback.)
I remember watching a video some time ago where this guy deconstructed a square wave with FFT, phase-shifted the frequencies to minimize the constructive interference between them, and then put them back together. The result was a signal that still sounded like a square wave, but could be pushed far louder. I wonder if certain "compressor" type filters also use this approach, and what they might do to transients having seen after seeing the results of naive pitch shifting with phase shifting.
isn't a pure square wave the loudest possible waveform to begin with? i would guess that you could only decrease its loudness with this technique (ie increase its crest factor)
Love this video. I took a college level course on digital signal processing, and we never even covered those hann windows. I definitely saw some leakage in my projects, but thought it was unavoidable (similar to the Gibbs phenomenon).
My DSP experience is in digital telecommunications (OFDM! Costas loops! LMS equalizers! Oh my!), where this kind of operation isn't really ever necessary. I'd always wondered how my compatriots on the audio side of the DSP world managed to develop an algorithm to do this. Turns out it's an even more complex problem than I thought and I should perhaps rethink saying "I do basically the same thing, just at a higher sample rate." Excellent video editing and visualizations! With production this good, I'm surprised you only have ~1.57K subscribers - but count me among them now. :)
Subscribed instantly! This is an incredible video, thank you for visualizing something I've been incredibly curious about but had no clue how to find out! Beyond just being a math nerd I'm also a musician so a lot of this information might come in handy in the future...
Going to have to watch this a few times to really grok what’s going on, especially after the DFTs enter the picture. Been a while since I’ve done math to any serious level
This is really well done, I just feel like you could have given a quick recap of DFT, since you took a long time with the time domain explanation, but went super fast in the frequency domain explanation.
I wish i found this video sooner tbh, would have saved me some time, i believe. :3 After toiling for quite a bit of time trying to implement a realtime pitch shifter/time stretcher (hence i didn't want to touch fft or other things that would eat performance), i arrived at a solution where i only ever use two windows overlapping exactly 50%, but for each step to generate a new chunk, the windows are both resized to a size that's the chosen one plus/minus a small (uniformly or normally, i tried both) random deviation; Considering that a static window size was producing a very noticeable tone with my code with window sizes that were small enough to make the crossover points fast enough to go into audible pitch territory, the randomness introduced seemed to make that less noticeable quite nicely, basically akin to turning a pure tone into white noise. I still have issues with fluttering though, that i couldn't find time to implement a fix for yet.
I am watching this video at about 4 Times speed. This actually becomes very important to me because these artifacts dictate how fast I can watch a video on a device (or in general)
That was a concise walkthrough of the steps necessary to pitch shift. I never thought there were so many, and apparently you left some out. I can say that your end result still sounded way better than what Cool Edit Pro (predecessor to Adobe Audition) could accomplish. Of the left out steps, I would like to see formant preservation as the topic of a follow-up. It would be interesting to hear what happens when you move formants and pitch independently in a vocal sample.
Analog Semi does a conference? No, it's just audio not radar etc. so ...whoever they are, they have a YT Channel ADC. Or not, but some talks are there.
That was me, here you go 😄 th-cam.com/video/fJUmmcGKZMI/w-d-xo.html Yeah, this video hits many of the same points in a pretty similar order to mine - but presentation counts for a lot, and Jent's done a good job here. My goal is always for a train of thought to feel natural and obvious in retrospect, so if that's what happened here I'd be flattered.
4:22 The fluttering is related to how much the output blocks cross over. Since you've fixed the analysis hop size (instead of deciding the output hop size and then figuring out what the input one should be) compressing in time means the output blocks overlap a lot more.
This seems right; I just tried since fixing the output hop size, and it does introduce fluttering when shifting down. But shifting down decreases hop size, which creates more overlap, so shouldn't it have had more fluttering?
3:30 your envelope normalisation doesn't take into account the incoherent averaging between each chunk. If you took care of that correctly you wouldn't have "flutter" 9:01 It's not half the frequency, it wraps around between -0.5 and 0.5, so a frequency of 0.9 (1 being the sampling rate) will come out as -0.1.
I’ve struggled to get anything with pitch shifting done without sounding bad which is why I prefer the “just doing it in the right key/in tune during recording” strategy. At least now I know why it’s so complicated 😭
I studied physics for 3 years, and I’m currently studying computer science. That was wayyyyyyyy more complicated than I was expecting and a lot of it went over my head 💀
Regarding why you don't hear fluttering when pitch-shifting down (at 4:25), I would have thought the intuitive explanation is that to pitch down or compress time (which are equivalent as you point out) is a kind of resolution reduction (in this instance, sample resolution). Reducing resolution is a much simpler task to do faithfully and without artifact than increasing resolution, which will involve some kind of interpolation, which is much more likely to introduce conspicuous distortion (other than what would be inherent to the resolution).
Your knowledge level is intimidating... Also talent to present it clearly. Thank you for your work. I once tried "pitch shifter" using doppler effect method, hardware couldn't even dream of Fourier . I wonder if you have knowledge how pitch preserver works like in youtube speed change
Great video! I got a little lost with some of the Fourier stuff, but I haven't found a resource yet that doesn't lose me on that topic 😂 I'm surprised to see python doing this! I did a project doing DSP in python a few months ago and started hitting the limits of what could be done fast enough. Although I was doing real-time, not sure if you are. And I wasn't making extensive use of libraries written in C.
WAIT! How the hell do you only have 174 subscribers!? I only noticed after watching the whole video and then going to check your channel for more xD Consider it 175 now
Two years ago, I had to do this project. The explanations were less than clear. So, I started to dig into the academic papers. Once I was done I wanted to do a video just like this one for SoME2. Unfortunately, I never finished the project.
Its funny because this was always my first thought on how to do it (only overall not the details) but it always seemed too complex and like it would never work, so I assumed it was something crazy like fourier transforming then changing the pitch then changing back or something
Soon: Oh just use a phase mux SBN (sulfur boronitrite) inlet and bias vacancy decay with the new pitch. Or: I only use transforms named after musicians, but I'm still surprised when SZA is inverted through ASZA. Seriously though, did he have to not pitch shift the percussion?
![Real-time Trumpet Simulation [C++/Vulkan] [WARNING: Flashing Lights]](http://i.ytimg.com/vi/rGNUHigqUBM/mqdefault.jpg)
This is really well made! I had no idea pitch shifting required this much complication.
Same here. I really did not suspect so much math was needed for something this (apparently) simple.
It doesn't. He should have done this with op amp circuits and it would be fine. Even if he skipped the window fudging thing to cut the pop, he could pick better transforms and meet success.
I mean, why didn't he match the song effectiveness with what he demands from the filter; blam, a sing wrong fixer that has funkadelic qualia.
Funny that you say that, because this seems rather simple to me. No, I don't understand it at all, but the result is great for how "little" he did. I expected this quality to already need a huge amount of processing that only the gods at Antares (autotune company) have figured out.
@@LuLeBe The wizardry in autotune is all about interpolation between notes. The notes they sung, and the notes they wanted to sing. The repitching part probably isn't all that great if you just want a static effect. Like, imagine the math in this video, except the ratio between original pitch and target pitch is a function of the frequency content of the signal you started with.
@@Cineenvenordquist Ehm, no, what he's shown in the video is pretty much as simple as it gets for performing pitch shifting, which is just inherently difficult. I can't even guess what you could be referring to with the extremely vague "op amp circuits"... the function of an op amp in basically every circuit that uses one (except when used as a comparator) is to solve an equation, but the actual equation it solves is entirely defined by rest of the circuit, so describing something as an "op amp circuit" is utterly non-descriptive. Regardless, the fundamental problem of pitch shifting are not specific to how the audio is represented, and so are the solutions except they're much harder to implement using analog hardware than digital filtering, and getting phase to match for transient sounds seems like it would be particularly painful.
The original Alvin and the Chipmunks albums took advantage of speeding up the playback to change the pitch because they weren't being digitally processed. This means that the artists had to sing at an appropriately slower beat to account for the time shrinkage.
And then you can play it on a 16RPM turntable to hear the singer sing really slowly and the band play down an octave or so
@@charleslambert3368 the way these things were recorded the singers sung lower notes at a slower beat. The instruments were played at normal rate. The singer track and the instrumental track were combined later.
Last semester did a project about phase vocoders at an audio signal processing course. Your explanation about is absolutely superb, we had to give a presentation at the end of the course and it was train wreck compared to what you managed to explain in 15 minutes. Well done!
6:28 The reason pitch modification is listed as "yes" for PSOLA is because it's the only algorithm that doesn't change the formant, which is important to produce natural sounding speech.
Unlike SOLA and WSOLA, it doesn't introduce a chipmunk effect when speech is sped up. Because of this, it's heavily used in many autotune plugins today.
It's possible to tell if a plugin uses PSOLA by shifting the vocal down:
- If the formant sounds the same and there's gaps in the waveform, this indicates PSOLA was used, as it has no way to fill in those gaps.
- If the formant sounds the same and there's no gaps, this indicates a frequency-based approach was used, as it can fill in those gaps.
- if the formant sounds different, it's likely another approach was used.
As a semi-professional producer and programmer, I am deeply fascinated by this.
Super smooth. Clear, concise and thorough... and then having a real demo audio to follow examples is a great presentation feature. As someone interested in spectrograms I'd like to see a video of which audio properties create the most problems for a shifter vs which properties are easiest to work with. I've got some guesses but I imagine you could demo it in a similarly illuminating video.
This video is truly amazing. I hope you make more coding content and simplify other random topics for literally anything. If it's just a one-off you still did amazing and I wish you well on future endeavors
Thanks
Nice. My degree is in Physics. I assumed this was done with Fourier transforms, but I had always wondered how the finite sample sizes were handled. Basically, how they were glued together. This was well presented and I can see how much more there is to learn.
Do check out wavelets (including the continuous wavelet transform), as well as spectrograms. It’s all so cool.
@@bentpen2805 Wavelets are on my list of things to check out, this is probably a fun place to start.
I've wondered about this for decades. Great explanation, thank you!
I lost it right around 11:00-ish. Nice to see where my current math knowledge tapers off. Hopefully I can revisit in a couple years and be able to understand the whole video. :)
This is an incredible explanation! The visuals are great, and the way the process is laid out ("lets try this -- oh that causes such and such problem") really helps understanding the rationale behind every aspect of the algorithm.
I knew, as a physicist and electronic musician, that pitch shifting was hard. But I never knew how it worked, just that from a math side I couldn't come up with a good solution and from a musician side that choosing the right algorithm for a given sample was often difficult. This explains so much!!
What a satisfying algorithm and a satisfying explanation.
9:15 expanding on this, for N samples of time domain data, you get N samples of frequency domain data. however each frequency is a complex number, using two samples each, dc offset gets one. the nyquist frequency also gets only one, because phase shift between nyquist sampling frequency and input frequency at nyquist results in amplitude change. for every phase shift there is an amplitude that will make it fit the samples, and hence only one can be represented (and we never know which it is).
Though note that FFT outputs N-1…N/2+1 (note descending order) are the complex conjugates of outputs 1…N/2-1 and therefore are redundant, so if you discard those you've got a real numbers for DC and N/2-1 complex numbers for frequencies between DC and nyquist, and a final real number for nyquist, which all combined conveniently fits in the same amount of memory as the original N real samples :-)
(All this is assuming N is even; if N is odd then there's (N-1)/2 frequency samples between DC and nyquist and no frequency sample _at_ nyquist.)
@@MatthijsvanDuinExactly, values are mirrored with nyquist bin being the center point, real part of mirrored values is the same, while for imaginary part the sign flips. So mirrored value of x = a + bi would be y = a - bi. For example, with 1024 frame size, 0th coefficient is DC, 512th is nyquist, and every other coefficient 1024 - n, where n is coefficient number from 1 to 511, is a mirrored version of coefficient n. 513 is mirrored 511, 514 is mirrored 510, etc. I assume that you know this, but maybe someone will learn something new.
The Overlapped Add (OLA) algorithms also form the basis of several text-to-speech vocoders like MBROLA, PSOLA, etc.
Content presented around 8:00 is a bit misinformed. STFT preserves all info. Discrete time-domain repr doesn't include any info above Nyquist freq either. When pitch shifting, freqs higher than Nyquist is supposed to be lost, so scaling the frequency axis is not problematic. HOWEVER! The video up to 8:00 was very cool and taught me a lot of time-domain methods I didn't know about. Very inspring, and I appreciate that!
I just finished the video. 12:00+ was imprssive as well! You are utilizing assumptions about natural music to strike beyond theratical limits.
5:32. To be precise, auto-correlation is a convolution between signal and itself, but delayed, this is what auto- means, in this case it is just "correlation"
One of the best signal processing videos I have seen yet
This is insane quality and content for such a small channel. Hoping you get the attention you deserve!
This is something I’ve wondered about for a long time! Nicely done. It gives me hope when I see quality videos from small or new creators show up in my recommendations 😃
Best video on an audio DSP topic I've ever seen tbh!
I've understand so much more and so much better about pitchshifting by Your video. Thanks for it, really great job done!
Thanks. I've always wondered how this was done. I didn't realize it was so complicated.
This is fantastic. The animations and code are phenomenal for keeping up with the concepts. Great job!
Excellent video! I've implemented these algorithms many times by now but having this video 5 years ago would have saved me a huge amount of time and headache :)
Wait, why do you reimplement this bunches and why don't you just use an op amp circuit or get a singer who likes working to spec? Or a marimba with motorized dampers? Do you have to teach elliptical equations on python driven instruments?
this taught me more than a whole half a semester of uni. good video. sent it to everyone and their grandma
Thanks for sharing this! Easy to follow and really interesting! Your pitch shifter sounded great!
Long before the existence of auto-tune there were pitch shifters that were used in live performance situations to eliminate audio feedback from the speakers to the microphones. These units had a very clever approach which I wonder why is this not used for auto-tune?
The approach was based on something very similar to super-hetrodyne theory. The audio was multiplied by a carrier signal and the modulated signal was filterd to remove one of the side bands and then this filered signal was multiplied by another carrier signal that was slightly frequency shifted from the previous carrier frequency. This demodulation proces gave an output signal that was slightly frequency shifted from the original.
What you describe are frequency shifters, not pitch shifters. When you apply too much of that, it messes up the harmonic relations of the sound. (A couple Hz of shift is not very audible, but removes the audio feedback.)
That's a kinda DAW in the making, just add a tape loop or 7. Knocked away common mode and room resonance noise?
amazing video, took something i thought was magic and made it simple enough to the point where i feel i can implement it myself
I've been watching a lot of SoME videos recently and this is one of my favs. Well done!
I also don't like reading words. Thanks for the pretty animations and explanation!
Yeah, so why weren't the Jupyter notebooks in Sugar or a new Jaron Lanier language instead of python?
the demonstration are what really make this video. truly well done.
I love when I see something I take for granted (viewing youtube at x1.5) has a ton of math behind it. Awesome video
I’ve always wondered how this was done. Thank you. I could only really understand if I did it myself, but I get the gist.
This video is so well put together! I’d love to see this but with formant shifters. Love the content
I remember watching a video some time ago where this guy deconstructed a square wave with FFT, phase-shifted the frequencies to minimize the constructive interference between them, and then put them back together. The result was a signal that still sounded like a square wave, but could be pushed far louder.
I wonder if certain "compressor" type filters also use this approach, and what they might do to transients having seen after seeing the results of naive pitch shifting with phase shifting.
isn't a pure square wave the loudest possible waveform to begin with? i would guess that you could only decrease its loudness with this technique (ie increase its crest factor)
@@gabrieldoudna6570 indeed, phase-shifting the components of a square wave by a quarter turn produces in theory a signal with infinite crest factor
Love this video. I took a college level course on digital signal processing, and we never even covered those hann windows. I definitely saw some leakage in my projects, but thought it was unavoidable (similar to the Gibbs phenomenon).
Really enjoyed this! I would love a video on formant shifting please.
My DSP experience is in digital telecommunications (OFDM! Costas loops! LMS equalizers! Oh my!), where this kind of operation isn't really ever necessary. I'd always wondered how my compatriots on the audio side of the DSP world managed to develop an algorithm to do this. Turns out it's an even more complex problem than I thought and I should perhaps rethink saying "I do basically the same thing, just at a higher sample rate."
Excellent video editing and visualizations! With production this good, I'm surprised you only have ~1.57K subscribers - but count me among them now. :)
Subscribed instantly! This is an incredible video, thank you for visualizing something I've been incredibly curious about but had no clue how to find out!
Beyond just being a math nerd I'm also a musician so a lot of this information might come in handy in the future...
Great video! I always thought pitch shifting was just translation in the frequency domain. I like how you explained the problems with the example.
Going to have to watch this a few times to really grok what’s going on, especially after the DFTs enter the picture. Been a while since I’ve done math to any serious level
The piano tunes in the background were a nice touch!
This is really well done, I just feel like you could have given a quick recap of DFT, since you took a long time with the time domain explanation, but went super fast in the frequency domain explanation.
WOAH this is incredible for your first video! i hope you blow up!
I wish i found this video sooner tbh, would have saved me some time, i believe. :3
After toiling for quite a bit of time trying to implement a realtime pitch shifter/time stretcher (hence i didn't want to touch fft or other things that would eat performance), i arrived at a solution where i only ever use two windows overlapping exactly 50%, but for each step to generate a new chunk, the windows are both resized to a size that's the chosen one plus/minus a small (uniformly or normally, i tried both) random deviation;
Considering that a static window size was producing a very noticeable tone with my code with window sizes that were small enough to make the crossover points fast enough to go into audible pitch territory, the randomness introduced seemed to make that less noticeable quite nicely, basically akin to turning a pure tone into white noise.
I still have issues with fluttering though, that i couldn't find time to implement a fix for yet.
Jupiter notebooks unlocks alot of the mystery to this video.
Very technical details that I love, clrear explanation and very inspiring. The vocoder result sounds so high quality.
best explanation of pitch shifting I've seen so far, this is a 3blue1brown level of quality video, awesome work !!
Amazing video! Thanks for putting so much effort into it!
I am watching this video at about 4 Times speed. This actually becomes very important to me because these artifacts dictate how fast I can watch a video on a device (or in general)
the amount of math videos i had to rewatch before returning to this
really nice video. I appreciate that you took the time to define the jargon words.
Excellent video, I hope to see more from you soon
Damn, this channel is an underrated gem... I've been here at 661 subs, see you at many thousands soon
What a video. I just saw its your first video in your channel, so, congratulations and keep it up!
That was a concise walkthrough of the steps necessary to pitch shift. I never thought there were so many, and apparently you left some out. I can say that your end result still sounded way better than what Cool Edit Pro (predecessor to Adobe Audition) could accomplish.
Of the left out steps, I would like to see formant preservation as the topic of a follow-up. It would be interesting to hear what happens when you move formants and pitch independently in a vocal sample.
Wait there was an almost identical presentation at the ADC22, just 4 Months ago! Coincidence? I think not!!
Analog Semi does a conference? No, it's just audio not radar etc. so ...whoever they are, they have a YT Channel ADC. Or not, but some talks are there.
That was me, here you go 😄 th-cam.com/video/fJUmmcGKZMI/w-d-xo.html
Yeah, this video hits many of the same points in a pretty similar order to mine - but presentation counts for a lot, and Jent's done a good job here. My goal is always for a train of thought to feel natural and obvious in retrospect, so if that's what happened here I'd be flattered.
4:22 The fluttering is related to how much the output blocks cross over. Since you've fixed the analysis hop size (instead of deciding the output hop size and then figuring out what the input one should be) compressing in time means the output blocks overlap a lot more.
This seems right; I just tried since fixing the output hop size, and it does introduce fluttering when shifting down. But shifting down decreases hop size, which creates more overlap, so shouldn't it have had more fluttering?
@@JentacularGent More overlap reduces fluttering - the more blocks you're adding together, the more their fluctuations average out to be smooth.
I've been hoping to see a video like this one day, thank you so much for making it
Do more video like this one please!! Incredible work!
Wonderful video. Excellent idea to calculate the volume ❤❤
Wow I've always wondered how pitch shifting algorithms work and you did a really great job illustrating how some of them do so with this vid!
Thank you very much, the ideas in this video are useful for more than just music.
Amazing video! I was actually looking for this type of video a year or so ago and couldn't find one. Thank you for making this!
Yo, I remember you from Khan Academy! Keep making awesome stuff dude.
Excellent video. I really hope the phases didn't smother your interest in making more videos. Instant subscription.
only watched 4mins as of yet and i already learned more than i did in school lmao you deserve a sub
super great video and now im dizzy about phases...
i will come back someday
I like how almost every hurdle in audio processing comes back to phase when you trace it back far enough.
So glad I clicked on this video, beautifully made!
wow! "Hey kid, you know those things you didn't care to learn about? I'm going to show you what they're for and why they're cool"... fascinating.
Fantastic visualisations! Tnx for the video!
I’ve been wondering about this for a long time, but haven’t gotten around to reading a digital audio book to understand it
The silence after "the guy's name was just... Hann?....." is so perfect haha
3:30 your envelope normalisation doesn't take into account the incoherent averaging between each chunk. If you took care of that correctly you wouldn't have "flutter"
9:01 It's not half the frequency, it wraps around between -0.5 and 0.5, so a frequency of 0.9 (1 being the sampling rate) will come out as -0.1.
I’ve struggled to get anything with pitch shifting done without sounding bad which is why I prefer the “just doing it in the right key/in tune during recording” strategy. At least now I know why it’s so complicated 😭
I studied physics for 3 years, and I’m currently studying computer science. That was wayyyyyyyy more complicated than I was expecting and a lot of it went over my head 💀
Incredibly interesting and well explained!
Excellent video! So clearly explained. I hope you will do many more like it!
"It's full of big word" brother, most of the words you used flew right over my head I understood almost nothing
This is amazing!
Awesome video and easy explanation of such complex stuff👍
I'm a nerd when it comes to sampling. You are a legend for doing this, I hate words too ^_^
Regarding why you don't hear fluttering when pitch-shifting down (at 4:25), I would have thought the intuitive explanation is that to pitch down or compress time (which are equivalent as you point out) is a kind of resolution reduction (in this instance, sample resolution). Reducing resolution is a much simpler task to do faithfully and without artifact than increasing resolution, which will involve some kind of interpolation, which is much more likely to introduce conspicuous distortion (other than what would be inherent to the resolution).
Your knowledge level is intimidating...
Also talent to present it clearly.
Thank you for your work.
I once tried "pitch shifter" using doppler effect method, hardware couldn't even dream of Fourier .
I wonder if you have knowledge how pitch preserver works like in youtube speed change
Informative with nice animations, thanks for making this.
Great video! I got a little lost with some of the Fourier stuff, but I haven't found a resource yet that doesn't lose me on that topic 😂
I'm surprised to see python doing this! I did a project doing DSP in python a few months ago and started hitting the limits of what could be done fast enough. Although I was doing real-time, not sure if you are. And I wasn't making extensive use of libraries written in C.
WAIT! How the hell do you only have 174 subscribers!? I only noticed after watching the whole video and then going to check your channel for more xD
Consider it 175 now
Two years ago, I had to do this project. The explanations were less than clear. So, I started to dig into the academic papers. Once I was done I wanted to do a video just like this one for SoME2. Unfortunately, I never finished the project.
Very nice video. I would really like to see the code C or C++, maybe even using the JUCE platform.
This is awesome and I need so much more, you’re like if 3blue1brown was a producer
Super great video, thanks!
Absolutely amazing
great video!
Wow, great work
Well, that clears that up! (For Albert Einstein!)
My Favorite SoMe
i would not have the balls to do such a well made video at 1,7k followers
Its funny because this was always my first thought on how to do it (only overall not the details) but it always seemed too complex and like it would never work, so I assumed it was something crazy like fourier transforming then changing the pitch then changing back or something
Soon: Oh just use a phase mux SBN (sulfur boronitrite) inlet and bias vacancy decay with the new pitch. Or: I only use transforms named after musicians, but I'm still surprised when SZA is inverted through ASZA. Seriously though, did he have to not pitch shift the percussion?
this video is seriously awesome and inspiring thank you for putting in the effort! 😊
Very helpful. Thank you!