i dunno bro, i wrote the first lyrical song generating engine in '94, but this software works like crud, and everything sounds like your examples. this is an important thing - if you're going to give something the possibility to generate music infinitely, you want to train on the mills brothers singing smoke rings, not dubstep. there are good music generators, but it sounds like someone has no interest in music making this one. they don't even know that when you enter a duration, the numbers should start at the little end, so your song is 30 minutes long instead of 3. that's just crap development.
I typed "Farts and burps" into riffusion with the negative prompt "Guitar" and those hellish sounds will haunt me for weeks.. also try "Monkey screams" if you need some nightmare fuel
The vocals remind me of human dreams where you hear a melody or see that you're reading something, but if you try to focus on it, you can not comprehend what has actually been said.
Am I the only one who feels it sounds like chinese? Like that the training sample had too many chinese music in it and it all ended up sounding chinese.
@@videodaniel8945 yeah to me the vocals sound pretty authentic like a human could be singing it. i don’t get any creepy or dream like feeling from it tbh, it just sounds like someone singing in a foreign language to me.
They could totally transform this into an AI audio upscaler, to make low quality music sound crystal clear! I've been wanting something like that for years
Not sure how this video’s content relates to that, everything in this video has sounded like so-so vinyl or a somewhat worn Tape…so unless you’ve got some **really** bad MP3s idk where you got that idea
audio upscaler... throw it through a fourier transform, take the mag bins as a series and extend that series. haven't done it but i figure a cepstral transform may be used to "advance" the fft frame and double the bandwidth.
Warning. I would not dismiss this technology based on your personal biases. AI learns at an exponential rate and only gets better over time. It seems like AI developers are hell-bent on cutting people out of the art and music they consume.
There are already some audio tools that do something similar - with audio to MIDI convertors - but that example was strikingly good and serves as an example of something that will become commonplace. It will be unnecessary to have guitar lessons when you can just hum a tune and then get a computer to play it back and sound realistic.
Crazy that the model learned this from just ~5K labeled songs. If I didn't know that, I'd guess the dataset had to be hundreds of thousands of such datapoints at least.
Actually apparently (copy-pasted from comment on Fireship's video on the topic): the 5.5k music samples are only the *evaluation* dataset, which they've published along with the paper, but the paper states that "the semantic and acoustic modeling stages are trained on a dataset containing five million audio clips, amounting to 280k hours of music"
As a music producer, I'm really impressed by the track at 2:01 ! It's coherent, stays on track and is catchy as well. It's better than some people who have made musics for a while
@@joemama-bu5ue What's a bit ironic with that comment is that these are often the songs that sticks the most. It's often tracks that what feels off on the first couple sets of listening that get addictive and can be listened to over and over again. As a meddling guitar player, playing in key with scales is easy; it's what you learn when you start off. Not saying the AI is genius-level composer, but don't dismiss such creations so fast, what seems worse to you at first might be something you're just not 'getting' on the first couple of plays.
@@joemama-bu5ue Or just take jazz...... But yeah I don't mean that your comment is wrong, I'm sure there's going to be a lot of garbage generated, if it's like image AI, what is it, maybe one out of 10 is usable? 50? 100? You'd expect music to be similar... Can't all be homeruns all the time! But a bit of off-key off-scale might be a great thing; not a bad thing.
The jazz examples are an interesting contrast. They're completely incoherent even though the superficial elements that might make something "sound jazzy" are present. Jazz is often called "musician's music" because of its technical complexity and the level of music theory involved. Maybe that makes it particularly challenging to recreate convincingly. If you're not into jazz, the best way I can describe it is it kind of sounds like a non-musician with a really superficial understanding tried to imagine some jazz in their head. The overall vibe's right, there's drums and electric piano playing chords and a solo guitarist noodling away. But the harmony and changes are confused, the melodic lines are incoherent random notes, there's no groove to speak of and no real structure or idiomatic jazz vocabulary.
I wonder if part of the reason for creating this was the aggressive music industry copyright claims on TH-cam. Now future videos can have custom music without getting their earnings taken by some random rights management company.
New technologies always threaten the hegemony, so it will be interesting to see how the Big 3 record companies react to the destabilisation of the industry that AI can cause. I could imagine the big record companies doing things like threatening to pull their songs off Spotify or TH-cam if the streaming services allowed people to upload AI-created "fakes" of real artists. Any "democratic" technology that puts creative power in the hands of consumers scares the hell out of existing rights holders. I wonder whether Universal and Sony will see AI as an enabler that it can harness for its own profit, or as a potential destroyer of the industry. (Years ago, the record industry of America basically prevented minidiscs taking off as a recording medium, and it also tried to shut down online piracy via napster, so it will be interesting to see how it reacts to random people being able to produce and share "good" music without paying a record company for it).
»The book of the generation of Jesus Christ, the son of David, the son of Abraham. Abraham begat Isaac; and Isaac begat Jacob; and Jacob begat Judas and his brethren; And Judas begat Phares and Zara of Thamar; and Phares begat Esrom; and Esrom begat Aram; And Aram begat Aminadab; and Aminadab begat Naasson; and Naasson begat Salmon; And Salmon begat Booz of Rachab; and Booz begat Obed of Ruth; and Obed begat Jesse; And Jesse begat David the king; and David the king begat Solomon of her that had been the wife of Urias; And Solomon begat Roboam; and Roboam begat Abia; and Abia begat Asa; And Asa begat Josaphat; and Josaphat begat Joram; and Joram begat Ozias; And Ozias begat Joatham; and Joatham begat Achaz; and Achaz begat Ezekias; And Ezekias begat Manasses; and Manasses begat Amon; and Amon begat Josias; And Josias begat Jechonias and his brethren, about the time they were carried away to Babylon: And after they were brought to Babylon, Jechonias begat Salathiel; and Salathiel begat Zorobabel; And Zorobabel begat Abiud; and Abiud begat Eliakim; and Eliakim begat Azor; And Azor begat Sadoc; and Sadoc begat Achim; and Achim begat Eliud; And Eliud begat Eleazar; and Eleazar begat Matthan; and Matthan begat Jacob; And Jacob begat Joseph the husband of Mary, of whom was born Jesus, who is called Christ. So all the generations from Abraham to David are fourteen generations; and from David until the carrying away into Babylon are fourteen generations; and from the carrying away into Babylon unto Christ are fourteen generations. Now the birth of Jesus Christ was on this wise: When as his mother Mary was espoused to Joseph, before they came together, she was found with child of The Holy Ghost. Then Joseph her husband, being a just man, and not willing to make her a publick example, was minded to put her away privily. But while he thought on these things, behold, the angel of the Lord appeared unto him in a dream, saying, Joseph, thou son of David, fear not to take unto thee Mary thy wife: for that which is conceived in her is of The Holy Ghost. And she shall bring forth a son, and thou shalt call his name JESUS: for he shall save his people from their sins. Now all this was done, that it might be fulfilled which was spoken of the Lord by the prophet, saying, Behold, a virgin shall be with child, and shall bring forth a son, and they shall call his name Emmanuel, which being interpreted is, God with us. Then Joseph being raised from sleep did as the angel of the Lord had bidden him, and took unto him his wife: And knew her not till she had brought forth her firstborn son: and he called his name JESUS.« The Gospel of Jesus Christ - Matthew - Chapter 1
It would be quite interesting to transform this type of generated music into actual sheet music (or midi in a DAW), and analyze what it did from a "traditional" music theory perspective. Also could be interesting to compare how those tracks would sound if played by a real band.
That sort of thing was done at least thirty years ago (and possibly earlier) when primitive AIs were fed the manuscripts of Beethoven and Mozart and prompted to take bits and create new tunes. According to taste, some of the pieces were "quite good", but lacked the emotional qualities of the human-created pieces (and in parts sounded "weird") when played by orchestral musicians. With today's deep learning, the bots can create music that would be indistinguishable from the work of the all-time greats, but humans just don't want to accept it. They will though. The market will dictate that "Beethoven" should release a new symphony. The AI that creates the best one will be top of the classical charts.
You could just make the AI run the midi through a plugin like Kontakt where it'll play real sounding violins, bass, piano etc etc it'll sound much more convincing that what we have so far. Something like classical music or Jazz will sound very convincing soon.
@@LAFELIXMUSIC @AutPen38 This is a different technology, it does not work with "notes", this thing analyzes spectrograms and does not work with melodies and harmony in any comparable way to that. It does not generate midi nor can it be fed with manuscripts. It goes from finished sound wave to finished new sound wave, without ever going trough writing parts or anything like that.
@@koraamis5568 ableton can already do audio to midi. if its one simple instrument it converts it very well and accurately. soo all you need to do is take the audio generated and convert to midi then run that midi through a good sounding plug in.
This AI music revolution will truly be having an impact, not when a company makes a breakthrough (like this), but when they make that breakthrough public and usable by everyone.
It won't change much. When tools are available, they are available for everyone, music producers as well. Who do you think is going to make the best out of it? The same people who are making the best out of the billions tolls that already exist today. The AI revolution will have a huge impact, that's true, but I would be much more concerned if I was delivering stuff in a van, moving box in a warehouse on answering the phone for a company. Musicians will be fine, you won't go to a concert to see an empty stage with a laptop in the middle, randomizing notes.
@@ChristianIce Disagree. There are a lot of talented people who just dont have the equipment to do it. That AI provides this opens it up to numerous people. Same for AI pictures. Its more accessible and removes a lot of barriers. It will change a lot.
@@TheH1st0ry You must be unaware of the infinity of tools available to make music for free. Anybody with a computer and a microphone can, for decades. Yes, this tool will add to that, but you could produce loops and songs without knowing anything about music since the Amiga Tracker. To think that the effect would be the same in music industry compared to visual art means not knowing either.
@@ChristianIce but it is going to place where you even do not need microphone - soon AI will sing for you with your voice, and after that it will be able to create any song in any style, e.g. "play me new Elvis song" - you still think it will not change anything and there is no danger for artists who will lost their audience?
@@bzdr lol i guess the truth is that people not only listen to the music of other people but also they follow the personality, how them act, look etc. AI will make a revolution but only in production making, like any already existing VST. you’re not gonna listen to robot lol because it can do billions of GoOd songs and blah blah blah the perception of art will change
Whenever I was listening to that Drake- Greece song, I always suspected it being AI generated from beat to vocals. Sounds very computerized. I think these labels have been doing this kind of stuff for a minute now.
There are several music creation tools that already use AI, but nothing quite as comprehensive as this. There was a lot of hate for "Autotuned" vocals, but the record sales spoke for themselves. (Young) people like hearing new sounds and they don't care if a computer was used. In Korea they have pop bands that exist in virtual avatar form, and there are "vocaloid" plugin instruments that are virtual singers that you can program to sing your own lyrics. There's bound to me more and more of this over time, as listeners want to hear exciting new sounds and the best way to get them is to use the latest technology.
Have it generate music based on your heart rate, body temp, and brainwave frequency. Spend your life living in 'the zone' of perfectly appropriate music set to every life experience you have.
@@vmusatov I don't mean just the same beat and melody, but like everchanging, possibly with mood or whatever. I doubt it'll ever get boring if done correct.
@@snakeyeslp How though? The entire point of this imaginary thing is to make an indefinitely ongoing song (maybe based on some you already like?) that keeps it interesting. Neither you or me know what will be possible, but I know that if it becomes a commercial product there will be a lot of effort to make it not boring.
@@GamingDad lol, I remember calling Google about 10 years ago trying to see if we could pay them to license use of Google Maps and got what sounded like an intern who said wow, we aren’t really set up for that, you see we’ve got this advertising model and we don’t have a mechanism to sell things, so when our license with Delorme ended, we switched to Microsoft maps, then Google finally woke up and figured out how to sell things and we switched to them, was such a big deal the CFO got them to throw in a Google Glass kit. We gave it to a colleague as a going away present when he left to work for Indeed.
Just saw the video o the same research paper by another youtuber last night, but man this video feels something so "natural" (don't know if it's the right word,maybe soothing¿) and I don't know how to put it but "lofi like", yeah. Bycloud, your videos have a hard to define soothing feel to it, can't put a finger on what it is.
I had a lot of struggle getting it to make anything I asked it to, it always defaults to some modern EDM crap with my requests, but surely it will get better. Pretty impressive.
Unless you're a researcher working on this model, as far as I know no-one else has access to this. Were you using some other AI-Music-Generating site? There's a whole bunch of them now, but none of them seem to work the way MusicLM (THIS video's AI) does. A lot of them generate midi songs from building blocks it's given; not making songs from scratch. They both have their pros and cons, but the ones available to the public are often much simpler and same-y sounding.
I don't get why not train AI on music notation like midi or sheet music, and then use synth/samples to actually play the sounds? You would be able to put the notation output into a music program and have total control over tweaking it, instead of getting a raw audio output which is basically set in stone...
the same reason image ai's haven't been trained to make use of separate layers for cohesion/motion that would work well with an adobe/gimp/krita-based workflow answer: that tech leans closer to proceduralism & well-tuned algorithms, the sort of boring yet effective work that won't get the same "ai" marketing label
That workflow is quicker using midi packs, you don't need AI to reinvent the wheel. But I agree that in this way you can just use some loop, which again, you can make for yourself quicker through libraries. Let's get back to it next year and see if they managed to double the bitrate, at least :)
for starters musical notation completely fails to capture what's necessary to recreate the complexity of the human voice or rap flow. but also popular music is increasingly moving away from things that can be expressed in western musical notation and the creative focus is more on tone/timbre and subtle unquantizeable rhythms/grooves. so it'd be completely inadequate for that as well as international/ethnic music that never used it in the first place.
This is exactly what the company AIVA did. Imo as a musician/ producer, it was the most compelling one of all the music generation AI that I've looked into. Some genres come out terrible but a lot of them are pretty impressive. This Google MusicLM is one of the first ones that piqued my interest since AIVA tho. Way more impressive than Riffusion.
Hehehe! 'Hold on to your 2 Minute Papers'! That's probably how this ended up on my Home page recommendations! Interesting stuff! That hum conversion really impressed me!
Besides the 24kHz sample rate it actually sound quite good to me. Humans are really going to have to start thinking about things like intent vs technical skill.
What all these Algorithmic generators have seemingly neglected is AI self-consumption, that at some point the models will break down as AI generated content gets into datasets. What we have currently will be the purest sets from here on out. Soon the boundaries will blur until the AIs start to break and not function so well.
Yes this exactly this. I especially worry about the implications of using these models in the medical industry, manufacturing, and to write software, for exactly the reason you are describing. They accuracy is bad as it is. I expect it will get a lot better over the next couple of years, but then they will have exhausted the relevant/usable datasets and AI-generated content will start to feed into the training data, at which point the results will likely start degrading. I just hope society hasn't become too dependent on the technology by the time that happens!
Training data will become unbelievably critical and valuable. All of the massive public repositories of information and art will become locked behind paywalls at best, if not privatized and locked down entirely. Hackers might even started breaking into corporate networks to secretly insert data INTO their databases in order to influence the results generated by their AI models. Google has already acknowledged the potential for models like ChatGPT to replace traditional search engines. And we all know that companies are willing to pay good money to ensure their sites appear at the top of search results.
AlphaGo used AI vs AI games to train itself to top level play. There's nothing wrong with AI training on AI generated content, as long as the training data is curated for good results (in this case, good sounding music).
This is amazing stuff and I think it just adds to our ability to be more creative. To me, it's still slightly off and doesn't sound quite right, almost like music in a dream or nightmare.
yes, because the best way to teach some creativity is to give them access to a machine that will literally do everything for them. It's why our teachers teach our students by doing all of their work for them isn't it?
I think it'll be really cool as a tool for making new sounds and really weird shit, but that most people will still prefer royalty free human stuff because it'll be a bit more reliable (and is already soulless).
@@johncasey9544 So true, lol. From what I've seen following AI stuff for the past few years, I think it'll be a long while before we see AI music reach the heights that image generation is at... It's so hard for them to legally get huge amounts of training data (music) because of the huge labels that have an iron grip on stuff they own That said, I also feel like the nuances of human-created music are going to be hard for an AI to capture
This is truly fascinating. I'd use it as an idea generator - and I'm not even making music. The ability to create music in a specific style u have in ur mind when creating videos is very nice.
Dude...the humming thing...I've been imagining some kind of gear or software that could pull this off but I didn't think it was possible...I come up with ideas when alone without instruments a lot and this is just an amazing solution....I didn't catch whether this is open to the public and/or free to use....I must learn to wield this powerful sorcery!
Just a heads up that audio to MIDI has been available for many years now using software like Melodyne or Ableton Live. You might want to check those out!
5:21 Story mode would have use in games. To accentuate what is happening, a game of today plays from a bank of soundtracks, and I suppose uses some heuristics to mix in tracks from that bank. Story mode could probably produce nearly infinite, seamless, adaptive music
Games mostly use wwise, which is a middleware that let's developers provide the changing context (in combat, new area, dungeon, etc) and the musician can create stems (snippets or layers of music) that take that context into account and will be combined together on the fly. Games have been doing essentially this since LucasArt's iMuse system, used in Monkey Island and Dark Forces.
not on this training data it wont and the thing about music that's different to visual art which is actually rendered as data in the form of pixels is the training data at present they have access to is comps, not stems, not to mention they can only deal with comps that have already been written or melody lines and harmony's, they cannot come up with new ones because every individual is different and makes different choices, can probably write a pop song eventually about the limits of it.
@@123Andersonev I was mostly exaggerating on the time-frame, and I don’t imagine this specific AI picking off like you said. But I will disagree when it comes to the ceiling of AI music. I do think it’ll get to a point where you really have to break down the music in order to tell if its AI or not. Let’s be honest, AI is already getting insanely impressive, music isn’t going to be its hurdle. Even if it takes a decade to get to human level. But just like AI art, a talented human touch will always stand out from something that’s a product of previous work
@@RealityRogue that's not the problem, the problem is in how AI works, it can never be original, it can only work on what it's been taught, it cannot go beyond the realms of convention if that makes sense, so it's actually artless, it only knows how to replicate what already exists because people affirm the output as what they expect, in order for it to go outside convention you need someone sat there telling it no.
I foresee a future where each human lives in their own AI generated audiovisual bubble that creates just the optimal moods & thought patterns for manking to keep the servers going those algorithms run on.
Counterpoint: how do u even know what kind of media ur into without exposing urself to it? What abt genres that grow on u with time? Ud just get bored otherwise
@@nicf1555 Using feedback from a brainscan? Ai is already able to take brain scans of people that are seeing certain pictures, and then accurately guessing what objects the pictures contain. It's absolutely possible that an AI could estimate how much you like a song, (and also what emotions and mental imagery comes along with it) and then generate music according to that input. You could have it make music that makes you sad or angry, energizer you or make you calm down. The possibilities are endless.
@@TheCALMInstitute The awful part about the music industry is its robotic repetitiveness and regurgitation of whatever gets streams. Having an algorithm that's literally programmed to do just that would only make the situation worse. What we need is more creativity.
These musical snippits remind me of AI Art a year ago, not quite there yet but showing dramatic improvement. And now with Dalle2 and Stable-diffusion, human artists are reduced to this th-cam.com/video/zx3ROK9nOYE/w-d-xo.html. If you are in the music industry, this is your 1 year warning to prepare accordingly and get out while you still can!
The scary thing about this is I am not able to tell the difference if its human-generated to AI-generated music, especially when it makes some random lo-fi relaxation type of beats like the one shown in 3:40, at least with mid-journey or stable diffusion I can tell that this was created by an AI.
I'm impressed at the quality of the stuff this produces, and can see a lot of potential for it as a companion tool for songwriters and producers, but, jeeze, it's already hard enough getting music composition gigs because of all of the royalty-free loops out there that people think are Good Enough for their games and short films. With this I don't see why anyone would ever hire a soundtrack musician ever again.
I'm still fairly optimistic that not much would be worse from today's industry, partly since I still believe human artistry will always be superior. Similar to the digital art community, in the end they will still have their following. People follow artists for their art and no amount of 'good quality' AI works will change that. If anything, there will be less tedious souless corporate music jobs since they'll just use AI for their corporate music. This will probably apply to Top 40 stuff as well. On the bright side, maybe we'll see a rise of a counterculture dedicated to subvert 'good AI music', and the whole world of AI art by extension. Kinda' like the Dadaists.
The "best" or most successful musicians are those that harness the latest technologies, whether it's Bach with the harpsichord, the Beatles with the electric guitar and the four-track recorder, or some rapper or other with Autotune in 2010. The general public (and old people that grew up with older technology) won't like the "new music" ("It's just noise", they'll say) but technology is completely unstoppable. Just as the folk musicians couldn't stop Dylan going electric, and the "Keep Music Live" members of the Musicians Union in the 1980s couldn't hold back the tide of synths, drum machines, and samplers, no one around today will be able to stop AI. The future of music belongs to the kids that will use and love this new technology, even if it means that every human musician goes extinct.
@@AutPen38 AI is gonna be your new idea buddy, just like a little creative spark to get your musical juices flowing. With its help, you can bring your musical dreams to reality in no time. No more struggling with certain parts of the production process, AI's got your back. It's gonna level the playing field and let everyone share their ideas. But let's not forget the important role of the music producer. They still have turn those generated ideas into a killer song, add their own touch with effects like reverb, delays, and chorus, and make it sound exactly how they want it to with mixing and mastering. In the end, music producers are selling their ideas and their own personal style, that's what makes their music unique
They won't. And in 15 years, nearly every single "creative" or "artistic" endeavor will be monetized via AI. Photography, painting, drawing, singing, sculpting, etc... All of these will be replaced by AI (ironically, more than likely by the same creative-types that enjoy these passions, LOL). Heck, we're about the same time away from most labor being replaced too, IMO. The only reason it hasn't yet is due to the "old guard" that literally cannot process that it is cheaper to buy a maintain a machine than to keep hiring humans, or are threatened by tech they don't understand... And they aren't going to be around much longer, LOL. So... Yeah. :P
This is better than riffusion, but similar to a lot of other AI stuff right now, it’s pretty much just “stuff that seems like music, but doesn’t make any sense when listened to closely”. It’s been interesting to see that music is the item valued the least by AI devs - likely because we already have too much music and it’s basically free.
FYI Some of these tasks are not completely new and don't require AI. For example tools like ableton have the ability to create midi data from melodies since a coupe of years. It used to be a manual step for us to apply them to a new instrument. Same goes for the generation of melodies. There are useful websites to create melodies out of simple paraments with the UI. So a lot of the things this tool does is applying these existing technologies by beeing able to match a task to a text command. The really interesting part is how it comes all together and the interpretation of the AI.
I fell in love with the Starry Night piece. It's beautiful in a very perplexing, horrifying kinda' way. Maybe those of us without synesthesia can finally *listen* to pictures. I'm curious what the future will bring.
Some people really just do not understand how incredible AI is, like yes, this is probably not something we will listen to but just wait a year or 2. Why does everyone expect it to be perfect instantly?That yall weren't there to see the slow and stead progress of text and image generation is a big issue.
@@tomasviane3844 Isn't every computer soulless? So much of music is already made and automated by computers, why not finish the job and let the computer do the entire thing? Or at least a lot more. If it means more music I see that as a win.
as a music maker, it’s very impressive and i can’t wait for the next iteration of improvement. i see similarities to how techno was created by people trying to make something that sounded mechanical and artificial, i think there is likely to be a genre in the future which embraces ai generated sounds in the same way, but it’s possible ai generated music would advance fast enough to have this characteristic lofi sound for much less time, making is less likely for people to want to do it intentionally after a while
@@Tijaxtolan no that’s not what i meant, i don’t think ai will replace anyone doing something because they enjoy it, but it will probably replace a lot of generic music
@@morgan0 but that crappy music is still a job, i hate reggeaton but i don't wish replacement even for them, even if they make generic music, and if we put the bar higher it will be more and more difficult for begginers to climb their carreers or learning curve, if you pay attention you'll see that many musicians first sound generic and a mockery of their masters, then they achieve their real potential
Yeah true but I think at least in Italy the people on the top make generic music and barely can play any instrument , let's say we are at 50÷ to be generous. So i would not mind a replacement of those. The live music than would be left for professionist. In reality what I expect is this shitty musician/ singer getting the job done and being even more lazy, but hopefully with better music
@@morgan0 There needs to be a balance between machine mechanical objects and programs and natural human made creations...If we focus too much on the machine side, then we loose thought and presence of what made human creations so valuable. In other words, if we focus too much on AI, we'll never be able to recover the more human emotional, creation, and natural side of life.
Maybe, again, with AI, there needs to be a balance of how much power it can use until it gets into privacy and morality issues with machine learning Artificial programs versus human rights.
It all sounds a bit like muzak. There's no real life in it, but.. that makes it perfect for background music in a youtube or instructional video. Now , if only they would share it.
@@spydaboiii You mean you think that in time the music generated will be more complex and full of life? I hope that's true but I feel like , just with generated imagery, its easy to get the details but much harder for these machines to get the overall concept down just right. With any artform, the entire thing tells a story. That overarching theme is hard, if not impossible for this kind of AI to accomplish because it requires a higher level of understanding. perhaps they can combine these generators with another AI who's purpose is a higher level of concept?
I'd really like to see a Music generator that could take a preliminary input recording, re-imagine it as a classical symphonic piece, and create all the sheet music necessary for a full live orchestra to play it.
This stuff will be improved a lot in a few years. This is like an advanced plugin. I think all musicians would like to use this in someways, to experiment. But i find this program not scary as an ai image
Unlike others, I'm beyond impressed. I think this will FULLY replace real music someday. This being the first iteration of tech of it's kind, it will only improve expenentially.
I’ve been trying to find this for a couple years and it didn’t exist back then, when I used weed for like a year and my brain started making these crazy Medly-mashup things that were stuck in my head for a year So thanks for making this video so I’d find out. This sounds exactly like the inside of my head, nonsense vocals, vagueness but catchy
I think it'll be great for music producers as they can create a sound or melody right away without needing session musicians. Musicians might lose their jobs a bit to ai as itll be so much quicker and cheaper.
The thing that's missing from these AI models is any concept of musicality. It's just noise that vaguely resembles music. You need a different kind of AI entirely to make this work.
Well, compare current art AI like Midjourney to the ones we were seeing a couple years ago. The difference is absolutely night-and-day. Sure, they're still not perfect, and often lack a bunch of creativity, but if the same thing happens to AI music, it might soon be nearly indistinguishable from "real" music. Especially mass produced stuff like pop, techno, and mid-level orchestral soundtracks
@@darksunrise957 for image it works. Music not really. Until we can get midi file then we change it from there on. Otherwise is useless. Can’t use it in production and this technique won’t get improved. Is similar to limits of machine learning
@@youtubedeletedmyaccountlma2263 Audio to Midi exists since a decade, so this Audio Prompting Samples is huge, Endless new ways to make music, just needs the right human to assemble it together.
@@user-bs8es6nd7f Most of the audio to midi are just pure dogshit. Edit: Wait actually is literally all. Unless you find me that there's a usable one. Literally start from a song and give complete accurate arrangment of all musical instrument. Usually even with simple drum, the audio to midi programs all give out unusable nonsense
Man this is what i wanted!! I could make great compositions in my head but don't have the skills to play instruments so now I believe I can actually materialize the songs in my head!
It's so sad for those of us who have been learning to do these things our entire lives, but now very few people will be hiring us or needing our skills at all. AI is making us artists with decades of dedication and effort totally obsolete.
@@inthefade yep, I feel ya. This technology is threatening so many of the high-skilled, creative, and/or knowledge careers that still exist. Which really sucks because a lot of these are jobs that people actually enjoy doing and are passionate about. And it seems very likely that this technology is going to reduce the number of jobs in those fields, which will make it harder to find work and even drag down the pay of the people who do still have steady jobs in these fields. The engineers and scientists who spent the past decade creating this technology are dicks. There, I said it. All y'all are a bunch of dicks.
and how will you materialize it? You will say "computer, materialize song from my head?". Really, I'm not mean, just courious, how you want to make composition from your head alive? Without playing it note by note it will be simply not possible - and you can do it even today, without any instrument skills.
Если тренировать нейросеть с аккордами и блоками формы, то результат будет на порядок лучше. Само собой что база аудиоматериалов должна быть тоже большая. Нужно сделать нейросеть, которая качественно выделяет аккорды с midi -партиями из треков и сделать базу аккордов, мелодий и блоков формы (вариаций) с таймингами. А потом тренировать генеративную сеть уже на основе и волнового аудиоматериала, и теоретического.
2:57 The main lack of faithfulness I noticed was the total lack of an electric guitar in the response to the "arcade game" prompt that specifically asked for one. There actually WAS a male vocal "rapping" in it's response to that prompt, though the more interesting question to me is what language he, or the female singers, were using. I have a suspicion that the answer is not really any, though it's clearly inspired by English.
it will obviously get miles better in not much time at all. then what. start rioting. this is enough to be terrified of, assuming ANY rate of improvement. we're entering a horrific future & there seems to be no stopping it
Musicians seem to be the next target for job "democratization". On the other hand, it would be interesting to have a system that can generate personalized music based on the user context (following Dirk Clark's famous quote "Music is the soundtrack of your life.").
This is fascinating and scary all at once... I want to stay positive... I have worked in the arts for over two decades and I feel this will open a lot of opportunities but also decrease development in the human psyche that will now escape the journey that is learning a craft and deep diving into rabbit holes of imagination... I guess the future will tell... We can either embrace technological advancement or become obsolete...
These musical snippits are remind me of AI Art a year ago, not quite there yet but showing dramatic improvement. And now with Dalle2 and Stable-diffusion, human artists are reduced to this th-cam.com/video/zx3ROK9nOYE/w-d-xo.html. If you are in the music industry, this is your 1 year warning to prepare accordingly and get out while you still can!
I can't wait to feed an A.I. my unfinished short story from middle school and see how it ends. And then turns it into a full fledged Disney Channel movie, 2000's style.
As a music producer I find this interesting and maybe useful. I already gave up on a part, where I would be sad from ai ruining my 7 years practicing, learning and dreaming
FINALLY I am only 1 day late to AI news cuz I spent the last 10 hours making this video so please like and sub ♡
Suggestion: place your avatar in thumbnails - its very recognisable and catch attention
Okay that's the greatest thing I've ever seen in my entire life.
you got mine and mine respect
You know Google never releases there ai to puplic pretty sure this music one is no different than those other ai
i dunno bro, i wrote the first lyrical song generating engine in '94, but this software works like crud, and everything sounds like your examples. this is an important thing - if you're going to give something the possibility to generate music infinitely, you want to train on the mills brothers singing smoke rings, not dubstep.
there are good music generators, but it sounds like someone has no interest in music making this one. they don't even know that when you enter a duration, the numbers should start at the little end, so your song is 30 minutes long instead of 3. that's just crap development.
Training a text-to-image AI to generate spectograms was really clever
It's a Galaxy brain idea
I typed "Farts and burps" into riffusion with the negative prompt "Guitar" and those hellish sounds will haunt me for weeks.. also try "Monkey screams" if you need some nightmare fuel
@@legendaryra3590 I did, thanks... I will not sleep tonight
@@legendaryra3590 I also typed "Dog screams" and got some crazy weird vocals.
Is there a way to convert pre-existing audio files into the spectrograms SR uses?
The vocals remind me of human dreams where you hear a melody or see that you're reading something, but if you try to focus on it, you can not comprehend what has actually been said.
Totally. Like trying to read a book or even just a clock in your dreams. Superficially represents some information, but it is meaningless.
Aping art. Aping Intelligence. AI.
Sounds like K-pop to me lol
Am I the only one who feels it sounds like chinese? Like that the training sample had too many chinese music in it and it all ended up sounding chinese.
@@videodaniel8945 yeah to me the vocals sound pretty authentic like a human could be singing it. i don’t get any creepy or dream like feeling from it tbh, it just sounds like someone singing in a foreign language to me.
They could totally transform this into an AI audio upscaler, to make low quality music sound crystal clear! I've been wanting something like that for years
Fix some of those crappy mp3s we burnt from cds back in the day!
This already exist, people even remastered Half-Life audio.
Not sure how this video’s content relates to that, everything in this video has sounded like so-so vinyl or a somewhat worn Tape…so unless you’ve got some **really** bad MP3s idk where you got that idea
@@The_CGA yeah but these sound like that because all the ‘vocals’ are robot made. It would be much clearer over actual vocals
audio upscaler... throw it through a fourier transform, take the mag bins as a series and extend that series. haven't done it but i figure a cepstral transform may be used to "advance" the fft frame and double the bandwidth.
As a listener, i'm not impressed... As a composer and producer, i'm impressed that this even works.
Perfect comment. My thoughts exactly.
SAME
Stan(ley) Pine's dad: I'm not impressed and impressed at the same time.
a lot of people aren't even impressed by marvel movies so.. it's a scale. we saw how good ai images got in
Warning. I would not dismiss this technology based on your personal biases. AI learns at an exponential rate and only gets better over time. It seems like AI developers are hell-bent on cutting people out of the art and music they consume.
I like the conversion of voice expression to guitar. That is an actual useful tool in human composition.
agreed, this would be a fantastic tool!
There are already some audio tools that do something similar - with audio to MIDI convertors - but that example was strikingly good and serves as an example of something that will become commonplace. It will be unnecessary to have guitar lessons when you can just hum a tune and then get a computer to play it back and sound realistic.
Melodyne is great for this
That has been in Logic DAW for a while. Audio to midi
Theres a vst plugin called Vochlea that does this already, although I imagine AI would do it much better
Crazy that the model learned this from just ~5K labeled songs. If I didn't know that, I'd guess the dataset had to be hundreds of thousands of such datapoints at least.
no, it's less.
sincerely,
Scary
wtf
WAIT WHAT? Just 5k songs?
Actually apparently (copy-pasted from comment on Fireship's video on the topic): the 5.5k music samples are only the *evaluation* dataset, which they've published along with the paper, but the paper states that "the semantic and acoustic modeling stages are trained on a dataset containing five million audio clips, amounting to 280k hours of music"
As a music producer, I'm really impressed by the track at 2:01 !
It's coherent, stays on track and is catchy as well. It's better than some people who have made musics for a while
Some of the tracks are straight out of tune or just use wrong accord that makes it unmelodic sometimes
@@joemama-bu5ue What's a bit ironic with that comment is that these are often the songs that sticks the most. It's often tracks that what feels off on the first couple sets of listening that get addictive and can be listened to over and over again.
As a meddling guitar player, playing in key with scales is easy; it's what you learn when you start off. Not saying the AI is genius-level composer, but don't dismiss such creations so fast, what seems worse to you at first might be something you're just not 'getting' on the first couple of plays.
@@joemama-bu5ue Or just take jazz...... But yeah I don't mean that your comment is wrong, I'm sure there's going to be a lot of garbage generated, if it's like image AI, what is it, maybe one out of 10 is usable? 50? 100? You'd expect music to be similar... Can't all be homeruns all the time! But a bit of off-key off-scale might be a great thing; not a bad thing.
The jazz examples are an interesting contrast. They're completely incoherent even though the superficial elements that might make something "sound jazzy" are present.
Jazz is often called "musician's music" because of its technical complexity and the level of music theory involved. Maybe that makes it particularly challenging to recreate convincingly.
If you're not into jazz, the best way I can describe it is it kind of sounds like a non-musician with a really superficial understanding tried to imagine some jazz in their head. The overall vibe's right, there's drums and electric piano playing chords and a solo guitarist noodling away. But the harmony and changes are confused, the melodic lines are incoherent random notes, there's no groove to speak of and no real structure or idiomatic jazz vocabulary.
what? that sounds like trash :D i would say any newbe can do it better in one week, with the right teacher in one day
I wonder if part of the reason for creating this was the aggressive music industry copyright claims on TH-cam. Now future videos can have custom music without getting their earnings taken by some random rights management company.
I think the opposite may happen: lots of false flagging.
New technologies always threaten the hegemony, so it will be interesting to see how the Big 3 record companies react to the destabilisation of the industry that AI can cause. I could imagine the big record companies doing things like threatening to pull their songs off Spotify or TH-cam if the streaming services allowed people to upload AI-created "fakes" of real artists. Any "democratic" technology that puts creative power in the hands of consumers scares the hell out of existing rights holders. I wonder whether Universal and Sony will see AI as an enabler that it can harness for its own profit, or as a potential destroyer of the industry. (Years ago, the record industry of America basically prevented minidiscs taking off as a recording medium, and it also tried to shut down online piracy via napster, so it will be interesting to see how it reacts to random people being able to produce and share "good" music without paying a record company for it).
I doubt that was the sole reason, but likely a considered use case. A good idea none the less.
Nah man there's enough people who make music and enough free music on the internet to not be flagged
*Good. Those companies need to finally burn in hell.*
There's something a little uncanny about an AI trying to imitate human speech.
Eventually it will imitate so well that you can't tell the difference.
@@Miranox2 That's already the case. There are a lot of AI TTS papers where you can't tell.
@@laupoke There's a big difference between a research paper and a commercial product. TTS software still has a ways to go.
@@Miranox2 wym bro? there's literally multiple voice gen AIs lol
You say that now because you're human still.
It kinda has an eerie, surreal kind of liminal lostwave analog horror quality to it.
Whoa, that story mode is incredible, so many ideas you can generate alone just from being able to transition music from style to another.
»The book of the generation of Jesus Christ, the son of David, the son of Abraham.
Abraham begat Isaac; and Isaac begat Jacob; and Jacob begat Judas and his brethren;
And Judas begat Phares and Zara of Thamar; and Phares begat Esrom; and Esrom begat Aram;
And Aram begat Aminadab; and Aminadab begat Naasson; and Naasson begat Salmon;
And Salmon begat Booz of Rachab; and Booz begat Obed of Ruth; and Obed begat Jesse;
And Jesse begat David the king; and David the king begat Solomon of her that had been the wife of Urias;
And Solomon begat Roboam; and Roboam begat Abia; and Abia begat Asa;
And Asa begat Josaphat; and Josaphat begat Joram; and Joram begat Ozias;
And Ozias begat Joatham; and Joatham begat Achaz; and Achaz begat Ezekias;
And Ezekias begat Manasses; and Manasses begat Amon; and Amon begat Josias;
And Josias begat Jechonias and his brethren, about the time they were carried away to Babylon:
And after they were brought to Babylon, Jechonias begat Salathiel; and Salathiel begat Zorobabel;
And Zorobabel begat Abiud; and Abiud begat Eliakim; and Eliakim begat Azor;
And Azor begat Sadoc; and Sadoc begat Achim; and Achim begat Eliud;
And Eliud begat Eleazar; and Eleazar begat Matthan; and Matthan begat Jacob;
And Jacob begat Joseph the husband of Mary, of whom was born Jesus, who is called Christ.
So all the generations from Abraham to David are fourteen generations; and from David until the carrying away into Babylon are fourteen generations; and from the carrying away into Babylon unto Christ are fourteen generations.
Now the birth of Jesus Christ was on this wise: When as his mother Mary was espoused to Joseph, before they came together, she was found with child of The Holy Ghost.
Then Joseph her husband, being a just man, and not willing to make her a publick example, was minded to put her away privily.
But while he thought on these things, behold, the angel of the Lord appeared unto him in a dream, saying, Joseph, thou son of David, fear not to take unto thee Mary thy wife: for that which is conceived in her is of The Holy Ghost.
And she shall bring forth a son, and thou shalt call his name JESUS: for he shall save his people from their sins.
Now all this was done, that it might be fulfilled which was spoken of the Lord by the prophet, saying,
Behold, a virgin shall be with child, and shall bring forth a son, and they shall call his name Emmanuel, which being interpreted is, God with us.
Then Joseph being raised from sleep did as the angel of the Lord had bidden him, and took unto him his wife:
And knew her not till she had brought forth her firstborn son: and he called his name JESUS.«
The Gospel of Jesus Christ - Matthew - Chapter 1
I was waiting for you to make a video about this, you're great at always keeping us informed.
It would be quite interesting to transform this type of generated music into actual sheet music (or midi in a DAW), and analyze what it did from a "traditional" music theory perspective.
Also could be interesting to compare how those tracks would sound if played by a real band.
This! At that point we're basically reverse-reverse-engineering music, which is sort of meaningless by itself, but still interesting.
That sort of thing was done at least thirty years ago (and possibly earlier) when primitive AIs were fed the manuscripts of Beethoven and Mozart and prompted to take bits and create new tunes. According to taste, some of the pieces were "quite good", but lacked the emotional qualities of the human-created pieces (and in parts sounded "weird") when played by orchestral musicians. With today's deep learning, the bots can create music that would be indistinguishable from the work of the all-time greats, but humans just don't want to accept it. They will though. The market will dictate that "Beethoven" should release a new symphony. The AI that creates the best one will be top of the classical charts.
You could just make the AI run the midi through a plugin like Kontakt where it'll play real sounding violins, bass, piano etc etc it'll sound much more convincing that what we have so far. Something like classical music or Jazz will sound very convincing soon.
@@LAFELIXMUSIC @AutPen38 This is a different technology, it does not work with "notes", this thing analyzes spectrograms and does not work with melodies and harmony in any comparable way to that. It does not generate midi nor can it be fed with manuscripts. It goes from finished sound wave to finished new sound wave, without ever going trough writing parts or anything like that.
@@koraamis5568 ableton can already do audio to midi. if its one simple instrument it converts it very well and accurately. soo all you need to do is take the audio generated and convert to midi then run that midi through a good sounding plug in.
This AI music revolution will truly be having an impact, not when a company makes a breakthrough (like this), but when they make that breakthrough public and usable by everyone.
It won't change much.
When tools are available, they are available for everyone, music producers as well.
Who do you think is going to make the best out of it? The same people who are making the best out of the billions tolls that already exist today.
The AI revolution will have a huge impact, that's true, but I would be much more concerned if I was delivering stuff in a van, moving box in a warehouse on answering the phone for a company.
Musicians will be fine, you won't go to a concert to see an empty stage with a laptop in the middle, randomizing notes.
@@ChristianIce Disagree. There are a lot of talented people who just dont have the equipment to do it. That AI provides this opens it up to numerous people. Same for AI pictures. Its more accessible and removes a lot of barriers. It will change a lot.
@@TheH1st0ry
You must be unaware of the infinity of tools available to make music for free.
Anybody with a computer and a microphone can, for decades.
Yes, this tool will add to that, but you could produce loops and songs without knowing anything about music since the Amiga Tracker.
To think that the effect would be the same in music industry compared to visual art means not knowing either.
@@ChristianIce but it is going to place where you even do not need microphone - soon AI will sing for you with your voice, and after that it will be able to create any song in any style, e.g. "play me new Elvis song" - you still think it will not change anything and there is no danger for artists who will lost their audience?
@@bzdr lol
i guess the truth is that people not only listen to the music of other people but also they follow the personality, how them act, look etc. AI will make a revolution but only in production making, like any already existing VST. you’re not gonna listen to robot lol because it can do billions of GoOd songs and blah blah blah the perception of art will change
For sampling this amazing, for listening not so much, but for samples this is endless.
This is miles ahead of other music generating programs.
No wonder
And it is still lightyears behind real music, and it will never ever catch up
@@isaacnewtech nah
@@Milk_Delivery yes, because you're unable to discern what is missing, unlike millions of people who can.
@@isaacnewtech you won't be able to distinguish ai music from real music in 5-10 years mark my word
LOL that '' BELLA CIAO'' italian song, on E-guitar, holy moly this is amazing
Whenever I was listening to that Drake- Greece song, I always suspected it being AI generated from beat to vocals.
Sounds very computerized. I think these labels have been doing this kind of stuff for a minute now.
There are several music creation tools that already use AI, but nothing quite as comprehensive as this. There was a lot of hate for "Autotuned" vocals, but the record sales spoke for themselves. (Young) people like hearing new sounds and they don't care if a computer was used. In Korea they have pop bands that exist in virtual avatar form, and there are "vocaloid" plugin instruments that are virtual singers that you can program to sing your own lyrics. There's bound to me more and more of this over time, as listeners want to hear exciting new sounds and the best way to get them is to use the latest technology.
I can already imagine generating an infinite piece of music that exactly adhere to one's own music taste.
Have it generate music based on your heart rate, body temp, and brainwave frequency. Spend your life living in 'the zone' of perfectly appropriate music set to every life experience you have.
And you’ll get so tiered and bored of it
@@vmusatov I don't mean just the same beat and melody, but like everchanging, possibly with mood or whatever. I doubt it'll ever get boring if done correct.
@@astrovation3281 Itll def get boring
@@snakeyeslp How though? The entire point of this imaginary thing is to make an indefinitely ongoing song (maybe based on some you already like?) that keeps it interesting. Neither you or me know what will be possible, but I know that if it becomes a commercial product there will be a lot of effort to make it not boring.
I need to get my hands on this software! Really wanna try the humming feature 😂
did you manage? i'm trying to do the same
@@NPJGlobal never tried
Google is doing what Google does best, make something great and then bury it.
@@GamingDad lol, I remember calling Google about 10 years ago trying to see if we could pay them to license use of Google Maps and got what sounded like an intern who said wow, we aren’t really set up for that, you see we’ve got this advertising model and we don’t have a mechanism to sell things, so when our license with Delorme ended, we switched to Microsoft maps, then Google finally woke up and figured out how to sell things and we switched to them, was such a big deal the CFO got them to throw in a Google Glass kit. We gave it to a colleague as a going away present when he left to work for Indeed.
Damn Google has some sick beat
Just saw the video o the same research paper by another youtuber last night, but man this video feels something so "natural" (don't know if it's the right word,maybe soothing¿) and I don't know how to put it but "lofi like", yeah.
Bycloud, your videos have a hard to define soothing feel to it, can't put a finger on what it is.
I had a lot of struggle getting it to make anything I asked it to, it always defaults to some modern EDM crap with my requests, but surely it will get better. Pretty impressive.
Unless you're a researcher working on this model, as far as I know no-one else has access to this. Were you using some other AI-Music-Generating site? There's a whole bunch of them now, but none of them seem to work the way MusicLM (THIS video's AI) does. A lot of them generate midi songs from building blocks it's given; not making songs from scratch. They both have their pros and cons, but the ones available to the public are often much simpler and same-y sounding.
I don't get why not train AI on music notation like midi or sheet music, and then use synth/samples to actually play the sounds?
You would be able to put the notation output into a music program and have total control over tweaking it, instead of getting a raw audio output which is basically set in stone...
the same reason image ai's haven't been trained to make use of separate layers for cohesion/motion that would work well with an adobe/gimp/krita-based workflow
answer: that tech leans closer to proceduralism & well-tuned algorithms, the sort of boring yet effective work that won't get the same "ai" marketing label
That workflow is quicker using midi packs, you don't need AI to reinvent the wheel.
But I agree that in this way you can just use some loop, which again, you can make for yourself quicker through libraries.
Let's get back to it next year and see if they managed to double the bitrate, at least :)
for starters musical notation completely fails to capture what's necessary to recreate the complexity of the human voice or rap flow. but also popular music is increasingly moving away from things that can be expressed in western musical notation and the creative focus is more on tone/timbre and subtle unquantizeable rhythms/grooves. so it'd be completely inadequate for that as well as international/ethnic music that never used it in the first place.
Well that's actually been done by openAI a few years ago, it is called MuseNet
This is exactly what the company AIVA did. Imo as a musician/ producer, it was the most compelling one of all the music generation AI that I've looked into. Some genres come out terrible but a lot of them are pretty impressive. This Google MusicLM is one of the first ones that piqued my interest since AIVA tho. Way more impressive than Riffusion.
Hehehe! 'Hold on to your 2 Minute Papers'! That's probably how this ended up on my Home page recommendations!
Interesting stuff! That hum conversion really impressed me!
Besides the 24kHz sample rate it actually sound quite good to me. Humans are really going to have to start thinking about things like intent vs technical skill.
At last,.. Those were the clearest, most understandable hip hop lyrics I've ever heard!
What a time to be alive!
What all these Algorithmic generators have seemingly neglected is AI self-consumption, that at some point the models will break down as AI generated content gets into datasets. What we have currently will be the purest sets from here on out. Soon the boundaries will blur until the AIs start to break and not function so well.
Yes this exactly this.
I especially worry about the implications of using these models in the medical industry, manufacturing, and to write software, for exactly the reason you are describing. They accuracy is bad as it is. I expect it will get a lot better over the next couple of years, but then they will have exhausted the relevant/usable datasets and AI-generated content will start to feed into the training data, at which point the results will likely start degrading. I just hope society hasn't become too dependent on the technology by the time that happens!
Training data will become unbelievably critical and valuable. All of the massive public repositories of information and art will become locked behind paywalls at best, if not privatized and locked down entirely. Hackers might even started breaking into corporate networks to secretly insert data INTO their databases in order to influence the results generated by their AI models. Google has already acknowledged the potential for models like ChatGPT to replace traditional search engines. And we all know that companies are willing to pay good money to ensure their sites appear at the top of search results.
Interesting! Never heard about that.
Been thinking about that for a while now. Glad others have, too.
AlphaGo used AI vs AI games to train itself to top level play. There's nothing wrong with AI training on AI generated content, as long as the training data is curated for good results (in this case, good sounding music).
this was a really good explainer on how audiolm works! thx for making this!
This is amazing stuff and I think it just adds to our ability to be more creative. To me, it's still slightly off and doesn't sound quite right, almost like music in a dream or nightmare.
I think at this stage it can be a great way to get ideas for tracks
@@TracksWithDax I think that's how it will be used. Quickly iterating on ideas.
yes, because the best way to teach some creativity is to give them access to a machine that will literally do everything for them. It's why our teachers teach our students by doing all of their work for them isn't it?
I think it'll be really cool as a tool for making new sounds and really weird shit, but that most people will still prefer royalty free human stuff because it'll be a bit more reliable (and is already soulless).
@@johncasey9544 So true, lol.
From what I've seen following AI stuff for the past few years, I think it'll be a long while before we see AI music reach the heights that image generation is at... It's so hard for them to legally get huge amounts of training data (music) because of the huge labels that have an iron grip on stuff they own
That said, I also feel like the nuances of human-created music are going to be hard for an AI to capture
Okay the humming demo was actually insane.
Yeah that shit was hard
This is truly fascinating. I'd use it as an idea generator - and I'm not even making music.
The ability to create music in a specific style u have in ur mind when creating videos is very nice.
U could just learn music too
Dude...the humming thing...I've been imagining some kind of gear or software that could pull this off but I didn't think it was possible...I come up with ideas when alone without instruments a lot and this is just an amazing solution....I didn't catch whether this is open to the public and/or free to use....I must learn to wield this powerful sorcery!
Just a heads up that audio to MIDI has been available for many years now using software like Melodyne or Ableton Live. You might want to check those out!
What really would be helpful is an AI, that can generate a note sheet from a song. Sometimes it's really hard to figure out what's going on.
Check frettable
just learn how to do it yourself jesus christ
5:21 Story mode would have use in games. To accentuate what is happening, a game of today plays from a bank of soundtracks, and I suppose uses some heuristics to mix in tracks from that bank. Story mode could probably produce nearly infinite, seamless, adaptive music
I like how you said that
They tried something similar with Doom IIRC, but this could probably do it even better.
Games mostly use wwise, which is a middleware that let's developers provide the changing context (in combat, new area, dungeon, etc) and the musician can create stems (snippets or layers of music) that take that context into account and will be combined together on the fly.
Games have been doing essentially this since LucasArt's iMuse system, used in Monkey Island and Dark Forces.
@@SimonBuchanNz Woah
This reminds me of when art Ai started to get good.
Let’s wait a year and the music will be indistinguishable from actual music
not on this training data it wont and the thing about music that's different to visual art which is actually rendered as data in the form of pixels is the training data at present they have access to is comps, not stems, not to mention they can only deal with comps that have already been written or melody lines and harmony's, they cannot come up with new ones because every individual is different and makes different choices, can probably write a pop song eventually about the limits of it.
@@123Andersonev I was mostly exaggerating on the time-frame, and I don’t imagine this specific AI picking off like you said.
But I will disagree when it comes to the ceiling of AI music. I do think it’ll get to a point where you really have to break down the music in order to tell if its AI or not. Let’s be honest, AI is already getting insanely impressive, music isn’t going to be its hurdle. Even if it takes a decade to get to human level.
But just like AI art, a talented human touch will always stand out from something that’s a product of previous work
@@RealityRogue that's not the problem, the problem is in how AI works, it can never be original, it can only work on what it's been taught, it cannot go beyond the realms of convention if that makes sense, so it's actually artless, it only knows how to replicate what already exists because people affirm the output as what they expect, in order for it to go outside convention you need someone sat there telling it no.
Great video! I’ve been checking out the various music AI’s and this one never popped up in my searches. Thanks!
I foresee a future where each human lives in their own AI generated audiovisual bubble that creates just the optimal moods & thought patterns for manking to keep the servers going those algorithms run on.
Matrix was more than an artistic metaphor.
Humans will be equivalent to hamsters in wheels, but with headphones on.
Counterpoint: how do u even know what kind of media ur into without exposing urself to it? What abt genres that grow on u with time? Ud just get bored otherwise
how do you know thats not what you're already experience just on a much much larger scale?
@@nicf1555 Using feedback from a brainscan? Ai is already able to take brain scans of people that are seeing certain pictures, and then accurately guessing what objects the pictures contain.
It's absolutely possible that an AI could estimate how much you like a song, (and also what emotions and mental imagery comes along with it) and then generate music according to that input.
You could have it make music that makes you sad or angry, energizer you or make you calm down.
The possibilities are endless.
AI is becoming more and more scary and mind-blowing at the same time..
That fact that it already sounds like staticky music from a radio scares the daylights out of me as an aspiring music producer.
I dunno, the music industry is so awful, this won’t be the worst thing you see!
@@TheCALMInstitute The awful part about the music industry is its robotic repetitiveness and regurgitation of whatever gets streams. Having an algorithm that's literally programmed to do just that would only make the situation worse. What we need is more creativity.
@@TheCALMInstitute Though you're right, that is not the only awful part of the music industry, of course. 😅
These musical snippits remind me of AI Art a year ago, not quite there yet but showing dramatic improvement. And now with Dalle2 and Stable-diffusion, human artists are reduced to this th-cam.com/video/zx3ROK9nOYE/w-d-xo.html. If you are in the music industry, this is your 1 year warning to prepare accordingly and get out while you still can!
The scary thing about this is I am not able to tell the difference if its human-generated to AI-generated music, especially when it makes some random lo-fi relaxation type of beats like the one shown in 3:40, at least with mid-journey or stable diffusion I can tell that this was created by an AI.
You can tell the recordings sound like digital versions of Vinyl and phonographs, almost as if it was recorded on tinny ribbon microphones.
that humming to instrument functionality needs to become available, please just release it so that we can start using it....
I require it
I'm impressed at the quality of the stuff this produces, and can see a lot of potential for it as a companion tool for songwriters and producers, but, jeeze, it's already hard enough getting music composition gigs because of all of the royalty-free loops out there that people think are Good Enough for their games and short films. With this I don't see why anyone would ever hire a soundtrack musician ever again.
I'm still fairly optimistic that not much would be worse from today's industry, partly since I still believe human artistry will always be superior. Similar to the digital art community, in the end they will still have their following. People follow artists for their art and no amount of 'good quality' AI works will change that.
If anything, there will be less tedious souless corporate music jobs since they'll just use AI for their corporate music. This will probably apply to Top 40 stuff as well.
On the bright side, maybe we'll see a rise of a counterculture dedicated to subvert 'good AI music', and the whole world of AI art by extension. Kinda' like the Dadaists.
The "best" or most successful musicians are those that harness the latest technologies, whether it's Bach with the harpsichord, the Beatles with the electric guitar and the four-track recorder, or some rapper or other with Autotune in 2010. The general public (and old people that grew up with older technology) won't like the "new music" ("It's just noise", they'll say) but technology is completely unstoppable. Just as the folk musicians couldn't stop Dylan going electric, and the "Keep Music Live" members of the Musicians Union in the 1980s couldn't hold back the tide of synths, drum machines, and samplers, no one around today will be able to stop AI. The future of music belongs to the kids that will use and love this new technology, even if it means that every human musician goes extinct.
@@AutPen38 AI is gonna be your new idea buddy, just like a little creative spark to get your musical juices flowing. With its help, you can bring your musical dreams to reality in no time. No more struggling with certain parts of the production process, AI's got your back. It's gonna level the playing field and let everyone share their ideas. But let's not forget the important role of the music producer. They still have turn those generated ideas into a killer song, add their own touch with effects like reverb, delays, and chorus, and make it sound exactly how they want it to with mixing and mastering. In the end, music producers are selling their ideas and their own personal style, that's what makes their music unique
Low budget indie games will likely be using this tool, but the ones with a bigger audience/revenue will still keep paying musicians to do better.
They won't. And in 15 years, nearly every single "creative" or "artistic" endeavor will be monetized via AI.
Photography, painting, drawing, singing, sculpting, etc...
All of these will be replaced by AI (ironically, more than likely by the same creative-types that enjoy these passions, LOL).
Heck, we're about the same time away from most labor being replaced too, IMO. The only reason it hasn't yet is due to the "old guard" that literally cannot process that it is cheaper to buy a maintain a machine than to keep hiring humans, or are threatened by tech they don't understand... And they aren't going to be around much longer, LOL.
So... Yeah.
:P
2023 is a key year
Wow, a lot of professional music producers came to comment! Crazy!
This is better than riffusion, but similar to a lot of other AI stuff right now, it’s pretty much just “stuff that seems like music, but doesn’t make any sense when listened to closely”.
It’s been interesting to see that music is the item valued the least by AI devs - likely because we already have too much music and it’s basically free.
Why do you say that music is the thing AI devs value the least?
FYI Some of these tasks are not completely new and don't require AI. For example tools like ableton have the ability to create midi data from melodies since a coupe of years. It used to be a manual step for us to apply them to a new instrument. Same goes for the generation of melodies. There are useful websites to create melodies out of simple paraments with the UI. So a lot of the things this tool does is applying these existing technologies by beeing able to match a task to a text command. The really interesting part is how it comes all together and the interpretation of the AI.
Pretty much my thoughts as well.
Instead of breaking up Google. Let’s open google up to the public. They made all these algorithms and AI tech using our data anyways.
What people say: I don't want algorithms and AI dictating how to live my life.
What the data says: Oh yes you do, you predictable humans.
9:43 this is literally the essence of 2000's music and i don't know how to feel
I fell in love with the Starry Night piece. It's beautiful in a very perplexing, horrifying kinda' way.
Maybe those of us without synesthesia can finally *listen* to pictures. I'm curious what the future will bring.
Wow. Imagine the possibilities of this technology. Producers can now create custom sample packs - specific to a project idea or style.
The vocals feel like one of those "name 1 thing in this picture" but as audio
Incredible how far along that's already come!
Some people really just do not understand how incredible AI is, like yes, this is probably not something we will listen to but just wait a year or 2. Why does everyone expect it to be perfect instantly?That yall weren't there to see the slow and stead progress of text and image generation is a big issue.
I've seen so many people bash on this technology which i feel is a little undeserved imo.
That almost sounds like it's a thing to look forward to... Do you want your music to be made by a soulless computer?
@@tomasviane3844 Isn't every computer soulless?
So much of music is already made and automated by computers, why not finish the job and let the computer do the entire thing? Or at least a lot more. If it means more music I see that as a win.
Anyone wants to see listen to this? This is worse enough for music, surprised so many people want it to improve
@@astrovation3281 respectively, you must not have made music before, at least real music, not sounds/audio
as a music maker, it’s very impressive and i can’t wait for the next iteration of improvement. i see similarities to how techno was created by people trying to make something that sounded mechanical and artificial, i think there is likely to be a genre in the future which embraces ai generated sounds in the same way, but it’s possible ai generated music would advance fast enough to have this characteristic lofi sound for much less time, making is less likely for people to want to do it intentionally after a while
You mean you can’t wait to get replaced and go into oblivion? No matter how good your ideas are?
@@Tijaxtolan no that’s not what i meant, i don’t think ai will replace anyone doing something because they enjoy it, but it will probably replace a lot of generic music
@@morgan0 but that crappy music is still a job, i hate reggeaton but i don't wish replacement even for them, even if they make generic music, and if we put the bar higher it will be more and more difficult for begginers to climb their carreers or learning curve, if you pay attention you'll see that many musicians first sound generic and a mockery of their masters, then they achieve their real potential
Yeah true but I think at least in Italy the people on the top make generic music and barely can play any instrument , let's say we are at 50÷ to be generous. So i would not mind a replacement of those. The live music than would be left for professionist. In reality what I expect is this shitty musician/ singer getting the job done and being even more lazy, but hopefully with better music
@@morgan0 There needs to be a balance between machine mechanical objects and programs and natural human made creations...If we focus too much on the machine side, then we loose thought and presence of what made human creations so valuable. In other words, if we focus too much on AI, we'll never be able to recover the more human emotional, creation, and natural side of life.
I see this as analogous to version 1 of midjourney. What I see here is massive potential, and I think they're finally on the right track.
> I think they're finally on the right track.
Badumtss.
Maybe, again, with AI, there needs to be a balance of how much power it can use until it gets into privacy and morality issues with machine learning Artificial programs versus human rights.
8:00 is from DooM 2 "Nobody Told Me About id"
the hummin part blew my mind! Wow! great video @bycloud 👍
It all sounds a bit like muzak. There's no real life in it, but.. that makes it perfect for background music in a youtube or instructional video. Now , if only they would share it.
For now
@@spydaboiii You mean you think that in time the music generated will be more complex and full of life? I hope that's true but I feel like , just with generated imagery, its easy to get the details but much harder for these machines to get the overall concept down just right. With any artform, the entire thing tells a story. That overarching theme is hard, if not impossible for this kind of AI to accomplish because it requires a higher level of understanding. perhaps they can combine these generators with another AI who's purpose is a higher level of concept?
You are the first (ive heard of ai music from) thanks for the spectrograph talk up front. Makes so much sense off images idea
I really want this to be released, it’s help my channel so much lol been having writers block for months.
cmon make some original content not AI generated :/
It will soon make musicians obsolete :(
I'd really like to see a Music generator that could take a preliminary input recording, re-imagine it as a classical symphonic piece, and create all the sheet music necessary for a full live orchestra to play it.
If Lyric cohesion was there this would be amazing
4:37 I've watched practically every TMP videos and squeezed my papers before but this is the first time my jaw has dropped!!
The video is just soooo high quality man.
You had my sub at the "Hold on to your two minute papers" reference.
What I want is to be able to play a rough song idea on a guitar, and have the AI produce a full production, that would be fun I guess.
Like a drummer that follow me even in not perfectly in tempo
So, I could perform in electric kazoo, then turn it to every instrument?
That's amazing!
What about generating music directly from an image without the text description of the image. Any AI formats out there that do this?
2:34 That's the male rapping tho. It started off with what's written in the third line, but then it switched. Idk how you missed it
Just imagine what it will be a couples more papers down the line !! What a time to be alive !!
Are these "original" creations its creating? Or aggregating things that exist hybrid kind of way like how does the copyright works?
I want this for my table top games. My players would freak if I had a theme for characters/npcs. I could make music from different regions... 🤯
This stuff will be improved a lot in a few years. This is like an advanced plugin. I think all musicians would like to use this in someways, to experiment.
But i find this program not scary as an ai image
Unlike others, I'm beyond impressed. I think this will FULLY replace real music someday. This being the first iteration of tech of it's kind, it will only improve expenentially.
If it can fully replace real music one day, we should also worry about AI replacing human period.
@@greenbefore2828 Maybe 🤔 Sh*t
I’ve been trying to find this for a couple years and it didn’t exist back then, when I used weed for like a year and my brain started making these crazy Medly-mashup things that were stuck in my head for a year
So thanks for making this video so I’d find out.
This sounds exactly like the inside of my head, nonsense vocals, vagueness but catchy
It's scary how good AI is becoming.
it would be good if it could be used
@Andrew Wingstrong Uncreatives will think along these lines
Not really
@@TheLemonKiller I agree. I'm not creative anymore.
@@gRIO904 18 people agreed with me. Nobody upvoted your comment.
There were quieter male vocals after the female vocals in the first MusicLM prompt you showed. Did you not hear them?
I hope people don't use this as a music generator but more so as a tool for musicians to make their compositions come to life
People will just claim they are musicians
It will be used for both, guaranteed
@@edwinvargas7969 as much as I dislike the idea, I agree
I think it'll be great for music producers as they can create a sound or melody right away without needing session musicians. Musicians might lose their jobs a bit to ai as itll be so much quicker and cheaper.
It’ll be treated just like art was,just gonna be used by people to pretend they’re something they’ll never be dedicated enough to actually become
2:03 The arcade one is the most epic thing that i've heard for a while, and it outstands all my songs that i've made so far...
The thing that's missing from these AI models is any concept of musicality. It's just noise that vaguely resembles music. You need a different kind of AI entirely to make this work.
Well, compare current art AI like Midjourney to the ones we were seeing a couple years ago. The difference is absolutely night-and-day. Sure, they're still not perfect, and often lack a bunch of creativity, but if the same thing happens to AI music, it might soon be nearly indistinguishable from "real" music. Especially mass produced stuff like pop, techno, and mid-level orchestral soundtracks
@@darksunrise957 for image it works. Music not really. Until we can get midi file then we change it from there on. Otherwise is useless. Can’t use it in production and this technique won’t get improved. Is similar to limits of machine learning
@@youtubedeletedmyaccountlma2263good idea, to start generate a midi and then use more AI to build it out. Not just blind spectographs
@@youtubedeletedmyaccountlma2263 Audio to Midi exists since a decade, so this Audio Prompting Samples is huge, Endless new ways to make music, just needs the right human to assemble it together.
@@user-bs8es6nd7f Most of the audio to midi are just pure dogshit.
Edit: Wait actually is literally all. Unless you find me that there's a usable one. Literally start from a song and give complete accurate arrangment of all musical instrument. Usually even with simple drum, the audio to midi programs all give out unusable nonsense
This is gonna be insane for independent composition and production... Damn!
Man this is what i wanted!!
I could make great compositions in my head but don't have the skills to play instruments so now I believe I can actually materialize the songs in my head!
It's so sad for those of us who have been learning to do these things our entire lives, but now very few people will be hiring us or needing our skills at all. AI is making us artists with decades of dedication and effort totally obsolete.
If you don’t have the skills then you should be learning those skills 🤷🏻♂️
@@inthefade yep, I feel ya. This technology is threatening so many of the high-skilled, creative, and/or knowledge careers that still exist. Which really sucks because a lot of these are jobs that people actually enjoy doing and are passionate about. And it seems very likely that this technology is going to reduce the number of jobs in those fields, which will make it harder to find work and even drag down the pay of the people who do still have steady jobs in these fields.
The engineers and scientists who spent the past decade creating this technology are dicks. There, I said it. All y'all are a bunch of dicks.
and how will you materialize it? You will say "computer, materialize song from my head?". Really, I'm not mean, just courious, how you want to make composition from your head alive? Without playing it note by note it will be simply not possible - and you can do it even today, without any instrument skills.
The transition from Heavy Metal to Indian Rap was crazy
Если тренировать нейросеть с аккордами и блоками формы, то результат будет на порядок лучше. Само собой что база аудиоматериалов должна быть тоже большая.
Нужно сделать нейросеть, которая качественно выделяет аккорды с midi -партиями из треков и сделать базу аккордов, мелодий и блоков формы (вариаций) с таймингами. А потом тренировать генеративную сеть уже на основе и волнового аудиоматериала, и теоретического.
2:57 The main lack of faithfulness I noticed was the total lack of an electric guitar in the response to the "arcade game" prompt that specifically asked for one. There actually WAS a male vocal "rapping" in it's response to that prompt, though the more interesting question to me is what language he, or the female singers, were using. I have a suspicion that the answer is not really any, though it's clearly inspired by English.
As a producer and composer, I'm terrified 😬
same
🤯Story Mode! Wow! Thank you for this. 👍
It sounds like a statistically probably mash up of all the musical examples that it has been fed. Exactly as expected.
it will obviously get miles better in not much time at all. then what.
start rioting. this is enough to be terrified of, assuming ANY rate of improvement.
we're entering a horrific future & there seems to be no stopping it
I really appreciate all the time and effort you put into this video! 🙌🏻
feels like Ai Dj's are only a year away.
Musicians seem to be the next target for job "democratization". On the other hand, it would be interesting to have a system that can generate personalized music based on the user context (following Dirk Clark's famous quote "Music is the soundtrack of your life.").
This is amazing to be honest, great if you need some inspiration whenever in a pinch
GUYS PLEASE, What the music à 4:40, I don't know what is it since years now...
This is fascinating and scary all at once... I want to stay positive... I have worked in the arts for over two decades and I feel this will open a lot of opportunities but also decrease development in the human psyche that will now escape the journey that is learning a craft and deep diving into rabbit holes of imagination... I guess the future will tell... We can either embrace technological advancement or become obsolete...
These musical snippits are remind me of AI Art a year ago, not quite there yet but showing dramatic improvement. And now with Dalle2 and Stable-diffusion, human artists are reduced to this th-cam.com/video/zx3ROK9nOYE/w-d-xo.html. If you are in the music industry, this is your 1 year warning to prepare accordingly and get out while you still can!
It's possible that embracing some forms of technological advancement will make us obselete.
I can't wait to feed an A.I. my unfinished short story from middle school and see how it ends.
And then turns it into a full fledged Disney Channel movie, 2000's style.
If you think AI will "help" musicians and not "replace" them, you are obviously missing the point
As a music producer I find this interesting and maybe useful. I already gave up on a part, where I would be sad from ai ruining my 7 years practicing, learning and dreaming