MARTSM_N /// martsman
MARTSM_N /// martsman
  • 27
  • 22 549
Transposing pitch to delay line resonant frequencies with Glitztune component in Pure Data
This video is a showcase of Glitztune component I've written in Pure Data. It transposes a given MIDI note pitch to resonant frequencies of very short delay lines and applies these to an incoming audio signal.
To demonstrate this effect, I'm using pitch detection with sigmund~ on a vocal sample in the first part of the video and a simple melody line generated with my Scaleculator 2 component in the second part. Both are applied to an Amen break as audio source.
---
All components used in this video are available on my PD components repository on GitHub: github.com/devstermarts/PD-components/tree/main/various
มุมมอง: 220

วีดีโอ

Neural Audio Funk - Generating Drum & Bass patterns with RAVE model in Pure Data
มุมมอง 266หลายเดือนก่อน
In this video, I'm showing the prototype of a patch setup that forces a RAVE model to generate Drum & Bass patterns using a sample based metro object in "the right" speed along with a sequencer component that retriggers noise seeds and signal generators in changing rhythmical patterns. Additional drones come from a vschaos2 model that reacts on speed and patterns from the general setup. The mod...
Realtime neural audio latent seeding w/ RAVE custom model trained on Martsman material in Pure Data
มุมมอง 2592 หลายเดือนก่อน
This video is mainly to showcase an updated version of my "Saatgut" component. It generates an array full of randomized values to use as seed bank in realtime inference and latent space tampering for nn~ compatible models. Cycling through this seeds is done via signal value input, e.g. simple oscillators. In the shown setup, I'm achieving repeatable sound sequences by resetting the oscillator p...
Neural audio latent sequencing with RAVE in Pure Data (Martsman Black Plastics Series custom model)
มุมมอง 4082 หลายเดือนก่อน
In this video, I'm presenting another technique to stabilize the output of RAVE neural audio models to an exact loop by generating repeatable sequences of signal information/ pseudo embeddings. This is done via a compact prototype component for latent sequencing, "arseq", that I've written in Pure Data (Source: github.com/devstermarts/PD-components/tree/main/various). It generates arrays of val...
Automated chord progressions with Progressor abstraction (Pure Data)
มุมมอง 1.6K3 หลายเดือนก่อน
In this video, I'm showcasing Progressor, a chord progression abstraction written in Pure Data. It works with an extendable list of progressions that can be used to generate whole tune sections based on a single key note input. The source code to Progressor (as well as the other components used in this video can be found on GitHub: github.com/devstermarts/PD-components
Generating melodic patterns with Scaleculator 2 (Pure Data)
มุมมอง 6234 หลายเดือนก่อน
Scaleculator 2 calculates note values of 50 different melodic scales based on a given key note value (Source: en.wikipedia.org/wiki/List_of_musical_scales_and_modes) In the video, i’m using Scaleculator 2 to control three arpeggios and a bassline that play along in the same tune and scale. The source code to Scaleculator 2 (as well as the other components used in this video can be found on GitH...
Latent Jamming with RAVE: Neural audio realtime intervention in latent space using nn~ in Pure Data
มุมมอง 6376 หลายเดือนก่อน
In this video I'm presenting a practice I call Latent Jamming in which I simulate latent embeddings by employing signal generators and translate them into audio information using a RAVE model's decoder. Latent Jamming is an explorative technique which includes finding interesting seeds, playing around with signal range spread and offsets each per latent dimension as provided by model and nn~ as...
Neural audio with RAVE: Methods forward & encode+decode
มุมมอง 2206 หลายเดือนก่อน
In this video, a RAVE model's capability of transferring learned representation is being shown by using methods forward and the equivalent encode decode. The model has been trained on a royalty free sample pack of processed amen breaks. RAVE is "A variational autoencoder for fast and high-quality neural audio synthesis” created by Antoine Caillon and Philippe Esling of Artificial Creative Intel...
Neural audio with RAVE: Methods encode & decode
มุมมอง 1216 หลายเดือนก่อน
In this video I'm showing how the embedding values/ latent vectors created by a RAVE encoder can be visualized and conserved. In the second part, I'm generating "fake" embeddings by using a noise~ object and have a RAVE decoder translate this into the audio domain. The model has been trained on a royalty free sample pack of processed amen breaks. RAVE is "A variational autoencoder for fast and ...
Neural audio with RAVE: Prior & MSPrior
มุมมอง 3036 หลายเดือนก่อน
Prior and MSPrior are two model architectures that work in conjunction with RAVE by creating plausible embedding predictions based on the RAVE model's encoding tactics. The upper patch shows a Prior model from the RAVE legacy version. The other patch shows two MSPrior models - the left has been trained using ALiBi transformer (decoder_only configuration), the right one with Gated Recurrent Unit...
Employing RAVE and vschaos2 neural audio models in larger compositions + seed conservation in arrays
มุมมอง 1.1K7 หลายเดือนก่อน
In this video, I'm showcasing the patch and configuration that led to track "Tritt Nochmal Zu" from my Saatgut Proxy/ Saatgut Proxy Reflux release in late 2023/ early 2024. The patch contains a setup of vschaos2 and RAVE model decoder layers and predefined seeds for mocking latent embeddings plus a randomized harmonic pad and a minimal kick drum synthesizer. The models have been trained on a da...
More AI drones with vschaos2 neural audio synthesizer (Kerner model)
มุมมอง 5818 หลายเดือนก่อน
In this video I'm showcasing a generative setup in Pure Data using neural audio synthesis in the form of a vschaos2 model trained on my 2018 album "Kerner" and simple probability settings that alter signal values of latent vectors. vschaos2 is a vintage-flavoured neural audio synthesis package by Axel Chemla Romeu Santos. It is based on unsupervised/ (semi-)supervised training of spectral infor...
Forcing RAVE models to generate audio loops using noise~ with seeding in Pure Data
มุมมอง 5779 หลายเดือนก่อน
In this video I'm focusing on a technique to force RAVE models' to generate loops. The decoder/ generator unit of a RAVE model translates latent encodings into audio signals. Latent encodings are basically multidimensional vectors of signal values which in an Autoencoder architecture are created by the encoder's translation of an input signal. In the patch shown in this video the latent encodin...
Generating drones with neural audio synth vschaos2 trained on Martsman tracks using nn~ in Pure Data
มุมมอง 3699 หลายเดือนก่อน
In this video I'm generating drones feeding low frequency oscillators and signal generators into a vschaos2 model which I trained on a selection of my own tracks. vschaos2 is a vintage-flavoured neural audio synthesis package by Axel Chemla Romeu Santos. It is based on unsupervised/ (semi-)supervised training of spectral information using variational auto-encoders. vschaos2 on GitHub: github.co...
MSPrior & RAVE: mixing unconditional and conditional latent encodings (Anthone dataset)
มุมมอง 34210 หลายเดือนก่อน
For this video, I've set up a constellation of an MSPrior model along with encoder and decoder units of a RAVE model. While the MSPrior model generates latent encodings unconditionally, the encoder is being fed a synthesized kick drum, that's being triggered regularly, and forwards its encodings to add to the altered signal of MSPrior. The dataset both the prior and the RAVE model have been tra...
Neural audio: tickling a RAVE model with sig~ and noise~ in Pure Data using nn~ (Martsman dataset)
มุมมอง 78210 หลายเดือนก่อน
Neural audio: tickling a RAVE model with sig~ and noise~ in Pure Data using nn~ (Martsman dataset)
Neural audio with unconditional autoregressive MSPrior model via nn~ in PD (Martsman dataset)
มุมมอง 44411 หลายเดือนก่อน
Neural audio with unconditional autoregressive MSPrior model via nn~ in PD (Martsman dataset)
Real time style transfer and latent manipulation with breaks: neural audio with RAVE and nn~ in PD
มุมมอง 49011 หลายเดือนก่อน
Real time style transfer and latent manipulation with breaks: neural audio with RAVE and nn~ in PD
Generating algorithmic Jungle music based on the Fibonacci Sequence in Pure Data
มุมมอง 7K11 หลายเดือนก่อน
Generating algorithmic Jungle music based on the Fibonacci Sequence in Pure Data
Calculating chords loading data via the text object in Pure Data - Chordculator showcase
มุมมอง 40111 หลายเดือนก่อน
Calculating chords loading data via the text object in Pure Data - Chordculator showcase
AI Dub: Latent jamming with neural audio model RAVE and nn~ in a semi generative environment in PD
มุมมอง 60511 หลายเดือนก่อน
AI Dub: Latent jamming with neural audio model RAVE and nn~ in a semi generative environment in PD
Neural audio jamming with MSPrior for RAVE using nn~ (Martsman dataset)
มุมมอง 612ปีที่แล้ว
Neural audio jamming with MSPrior for RAVE using nn~ (Martsman dataset)
Melodic patterns with classical musical modes: Scaleculator abstraction showcase (Pure Data)
มุมมอง 342ปีที่แล้ว
Melodic patterns with classical musical modes: Scaleculator abstraction showcase (Pure Data)
Neural audio music style transfer with RAVE using nn~ in PD (Martsman dataset)
มุมมอง 687ปีที่แล้ว
Neural audio music style transfer with RAVE using nn~ in PD (Martsman dataset)
Neural audio timbre transfer with RAVE trained on Amen breaks using nn~ in Pure Data
มุมมอง 2Kปีที่แล้ว
Neural audio timbre transfer with RAVE trained on Amen breaks using nn~ in Pure Data
Algorithmic breakbeat slicing: Splicer abstraction showcase (Pure Data)
มุมมอง 737ปีที่แล้ว
Algorithmic breakbeat slicing: Splicer abstraction showcase (Pure Data)
Martsman - RED014 // Teaser
มุมมอง 3938 ปีที่แล้ว
Martsman - RED014 // Teaser

ความคิดเห็น

  • @Digital__rb
    @Digital__rb 5 ชั่วโมงที่ผ่านมา

    Your work with this is so good.. how long have you been using PD for? Did you have prior knowledge that you think helped you learn it?

  • @Digital__rb
    @Digital__rb 5 ชั่วโมงที่ผ่านมา

    This is absolutely fucking insane. A big part of me wishes i learned puredata instead of modular synths. The learning curve on pure data seems crazy

  • @speedwolf
    @speedwolf 20 วันที่ผ่านมา

    Would you mind if I built this into a performance ready instrument?

    • @martsm_n
      @martsm_n 15 วันที่ผ่านมา

      I don't. The repo's license is MIT. github.com/devstermarts/PD-components/tree/main?tab=MIT-1-ov-file#readme I'd appreciate a shout out and a link to your instrument when you're done, though

  • @брухэтодарки
    @брухэтодарки 20 วันที่ผ่านมา

    amazing dude

  • @TripTilt
    @TripTilt หลายเดือนก่อน

    very cool!

  • @mann-vom-mond
    @mann-vom-mond หลายเดือนก่อน

    Cool

  • @torotumbo
    @torotumbo 2 หลายเดือนก่อน

    This sounds like a party for dishwashers and i love to be in it.

  • @torotumbo
    @torotumbo 2 หลายเดือนก่อน

    nice

  • @Envelyy
    @Envelyy 2 หลายเดือนก่อน

    where am i

  • @SteamFi
    @SteamFi 2 หลายเดือนก่อน

    Neat

  • @steinnkaribrekason9945
    @steinnkaribrekason9945 2 หลายเดือนก่อน

    Cool!

  • @musicaremcasa7628
    @musicaremcasa7628 2 หลายเดือนก่อน

    this is nice!

  • @user-gy1it9cq3u
    @user-gy1it9cq3u 3 หลายเดือนก่อน

    great🌿

  • @davidevitturini5837
    @davidevitturini5837 3 หลายเดือนก่อน

    Is the model available anywhere?

    • @martsm_n
      @martsm_n 3 หลายเดือนก่อน

      No. I didn't release it. You can train it quite easily yourself, however.

    • @davidevitturini5837
      @davidevitturini5837 3 หลายเดือนก่อน

      @@martsm_n did you used V1 or V2? On V2 and around 6hrs dataset with 2 3060 12gb it's taking around 1 week for 1 M epochs (I'm using a batch size of 36) [do you have a discord for discussing?]

    • @martsm_n
      @martsm_n 3 หลายเดือนก่อน

      @@davidevitturini5837 This one is a V2 model with default regularization. Another V1 model has been used in this video: th-cam.com/video/LgNmYUJaSi4/w-d-xo.html

    • @davidevitturini5837
      @davidevitturini5837 3 หลายเดือนก่อน

      @@martsm_n thanks, really enjoyed your creativity

  • @MFRSIAM
    @MFRSIAM 4 หลายเดือนก่อน

    how are u doing latent space manipulation ? i see u are using a resnet but am not sure whats going on here ?

    • @martsm_n
      @martsm_n 4 หลายเดือนก่อน

      This is actually footage from very early and cautious interventions. As far as I can recall, it's not much more than altering the spread of latents per dimension given - in PD that'd be amplitude modulation of a signal value. You can find more videos and techniques of latent intervention in my Neural Audio playlist: th-cam.com/play/PLmNfQif1XJQ0aNdrqk8HxvwUxq90xc9al.html

  • @Kaiseo1022
    @Kaiseo1022 5 หลายเดือนก่อน

    Dope as hell

  • @steveflato
    @steveflato 6 หลายเดือนก่อน

    insane. wow.

  • @draconianrhythms
    @draconianrhythms 6 หลายเดือนก่อน

    Insane ❤

  • @neuralboop
    @neuralboop 6 หลายเดือนก่อน

    siick

  • @beatsbykabuki
    @beatsbykabuki 6 หลายเดือนก่อน

    Your patches are starting to produce results that are truly unimaginable 😮

    • @martsm_n
      @martsm_n 6 หลายเดือนก่อน

      Thank you. That feedback means a lot ❤

  • @ShihWeiChieh
    @ShihWeiChieh 6 หลายเดือนก่อน

    can you explain what is plausible outputs more? does it mean it has more variational range? thanks!

    • @martsm_n
      @martsm_n 6 หลายเดือนก่อน

      In my understanding, it's rather "sequentially plausible" in the model's understanding of what embedding should follow the one currently generated in sequence.

  • @ShihWeiChieh
    @ShihWeiChieh 6 หลายเดือนก่อน

    why max is so much more CPU draining than PD...?

    • @martsm_n
      @martsm_n 6 หลายเดือนก่อน

      Can't answer that, sorry🤔

  • @ShihWeiChieh
    @ShihWeiChieh 6 หลายเดือนก่อน

    I have my model trained successfully finally! running it in max with nn~ just fine, few latency ~50ms. May I know how long is your dataset and what is the content? i didn't separate my tracks into single instrument so it sounds experimental.... thanks!

    • @martsm_n
      @martsm_n 6 หลายเดือนก่อน

      The model used in this video is trained on music I wrote in the past 20 years under various aliases and augmented from 7-8h into somewhat around 2-3 days of audio material.

    • @ShihWeiChieh
      @ShihWeiChieh 6 หลายเดือนก่อน

      so cool! 7-8 hours thats crazy long! on google GPU? I heard 2 hours is sufficient. Do you separate your songs to multitracks like bass drums, drones, snares etc to optimal the result?@@martsm_n

    • @ShihWeiChieh
      @ShihWeiChieh 6 หลายเดือนก่อน

      @@martsm_n sorry I didn’t understand “under various aliases and augumented”…

    • @martsm_n
      @martsm_n 6 หลายเดือนก่อน

      It just means I'm using all the music I have ever written under different artist names. Also, simply put, I'm sometimes augmenting the audio datasets to become bigger.@@ShihWeiChieh

  • @federicoinzerillo5333
    @federicoinzerillo5333 6 หลายเดือนก่อน

    I tried Colab, but it has a too low limit on the usage of units, and the prices seem a bit prohibitive to me. Instead, I tried using Kaggle, but I can't find a way to export it. Is there a small tutorial about it?

    • @martsm_n
      @martsm_n 6 หลายเดือนก่อน

      My Kaggle notebook comes with instructions on training and export - maybe that helps for starters? github.com/devstermarts/Notebooks/blob/main/RAVE_Training_Template--Kaggle.ipynb I'll see to create some kind of walkthrough video at some point.

    • @federicoinzerillo5333
      @federicoinzerillo5333 6 หลายเดือนก่อน

      @@martsm_n thank you for this and for all the knowledge you're spreading :)

    • @martsm_n
      @martsm_n 6 หลายเดือนก่อน

      @@federicoinzerillo5333 I can recommend joining the RAVE discord channel for discussing this kind of detail. The short (and potentially unsatisfying answer) is: it all depends. Personally, I find V1 the most robust for my interest and use case. Also I achieve satisfying results with training lengths well below the recommended amount of training steps and datasets that would be considered small (1-2h). You can override potentially every setting defined in the .gin configuration files in /configs. Hope that helps.

  • @alchemist.D
    @alchemist.D 6 หลายเดือนก่อน

    nice patch 😊

  • @dishop
    @dishop 6 หลายเดือนก่อน

    Круто!

  • @NathanRichan
    @NathanRichan 6 หลายเดือนก่อน

    I love your stuff. The patches always sound great and everything's so well organised

  • @fermiLiquidDrinker
    @fermiLiquidDrinker 6 หลายเดือนก่อน

    This sounds fucking sick

  • @2mrw730
    @2mrw730 7 หลายเดือนก่อน

    There are no words. We can only listen to it and enjoy it.

  • @dumafuji
    @dumafuji 7 หลายเดือนก่อน

    This is cool as heck

  • @marvbordello6047
    @marvbordello6047 7 หลายเดือนก่อน

    dope

  • @stephandesme123
    @stephandesme123 7 หลายเดือนก่อน

    So awesome! I just started learning Pd, how do you get those visual graphs, like for the sampler for example? How’d you get the sequencer into a tidy little box like that? Are those all external libraries? Sounds incredible! (:

    • @martsm_n
      @martsm_n 7 หลายเดือนก่อน

      Thank you! And no, this is all PD vanilla without additional libraries. What you're referring to are mainly arrays and graphs. You'll certainly stumble upon how these work when you dive into some PD tutorials on your way :)

    • @stephandesme123
      @stephandesme123 7 หลายเดือนก่อน

      @@martsm_n I’ll definitely look into it thank you!!

  • @ShihWeiChieh
    @ShihWeiChieh 7 หลายเดือนก่อน

    hey, i really wanna install nn~ in max and learn about how to train my model with giga byte data. do you use google collab? or is there a way to not to? thank you so much and super envy!

    • @ShihWeiChieh
      @ShihWeiChieh 7 หลายเดือนก่อน

      i got some helps from the discord and i have my first .ts file trained. I guess I have to train my prior model next for max/msp and pd usage (because the first .ts file is not compatible with my nn~ object). So i started the MSPRIOR notebook of yours then i stuck with the training cell: "ValueError: cannot reshape array of size 131072 into shape ()". Can you help me with this?

    • @martsm_n
      @martsm_n 7 หลายเดือนก่อน

      ​@@ShihWeiChiehI'm not sure if here's the right place for support requests :) I'd suggest you raise this as an issue on the MSPrior GitHub repo github.com/caillonantoine/msprior

    • @ShihWeiChieh
      @ShihWeiChieh 7 หลายเดือนก่อน

      @@martsm_n ah ok, I will go there!

  • @samsface
    @samsface 7 หลายเดือนก่อน

    So good.

  • @devinandrews2346
    @devinandrews2346 7 หลายเดือนก่อน

    very tasty. well done <3

  • @filetransferprotocoldoctor
    @filetransferprotocoldoctor 8 หลายเดือนก่อน

    Crazy stuff

  • @BurkhardusX
    @BurkhardusX 8 หลายเดือนก่อน

    Where could I download this patch?

    • @BurkhardusX
      @BurkhardusX 8 หลายเดือนก่อน

      @@martsm_ncould you make this point here and maybe today or so? Please post a link here for Christmas. Thank you.

  • @inbredfam
    @inbredfam 10 หลายเดือนก่อน

    Would it be possible to take, say, a little audio clip of me burping, and have it style transfered to a lion growling or something like that?

  • @lefka2523
    @lefka2523 10 หลายเดือนก่อน

    Incredible

  • @ilyazdd.i.4386
    @ilyazdd.i.4386 10 หลายเดือนก่อน

    sick!!

  • @tsdoihasdoihasdoih2493
    @tsdoihasdoihasdoih2493 10 หลายเดือนก่อน

    v1 config proves to be killer quality once again

    • @martsm_n
      @martsm_n 10 หลายเดือนก่อน

      Yeah, they are definitely robust. I'd very much like so see/hear people's experiments with V2 and V3, however. There certainly must be advantages that simply do not fit my use case/ practices. @antoinecaillon2531 maybe?

  • @LaurinkoSattumaa
    @LaurinkoSattumaa 10 หลายเดือนก่อน

    Great stuff! Thanks for giving a view into the innerworkings of your workflow/production!

  • @Karkwaa
    @Karkwaa 10 หลายเดือนก่อน

    Wonderful work, thanks a lot ! What is the role of your 'sequencex' and specifically the 'giver' inside ?

  • @apheadair
    @apheadair 10 หลายเดือนก่อน

    Stunning! How do you find the best value range to manipulate the latent space? I see that in this case, you are between -5 and +3. Is it just by trying or is there a rule of thumb?

    • @martsm_n
      @martsm_n 10 หลายเดือนก่อน

      The exact values and constellations differs from model to model. In V1 models, the first 3-4 dimensions seem to be the most sensitive and values in the +/- one-digit range lead to nice results whereas in the higher dimensions usually you need to go higher into the two-digit range to notice differences in the timbre. V2 and V3 models behave differently most probably due to the regularization equalling out things between the dimensions - I usually have a limited range +/- 2 that I can skip through without bleeding ears.

  • @beatsbykabuki
    @beatsbykabuki 10 หลายเดือนก่อน

    I could listen to this for hours! I noticed you use an augmentation script in your Kaggle notebook, but not in the Google Collab notebook. Is there a specific reason for this?

  • @a_ghoul
    @a_ghoul 11 หลายเดือนก่อน

    so sick

  • @beatsbykabuki
    @beatsbykabuki 11 หลายเดือนก่อน

    How much audio material in terms of duration did you use for the training?

  • @LaurinkoSattumaa
    @LaurinkoSattumaa 11 หลายเดือนก่อน

    ACE!

  • @keiwee95
    @keiwee95 11 หลายเดือนก่อน

    anywhere to start learning how to do this? craziest thing ive ever seen..