Torin Blankensmith
Torin Blankensmith
  • 48
  • 362 716
Text-to-Speech ElevenLabs Plugin for TouchDesigner + Whisper, ChatGPT, and MediaPipe Integration
In this tutorial, you'll learn how to use the new ElevenLabs text-to-speech plugin for TouchDesigner. We'll also walk through a project creating a voice-activated 3D talking character by integrating Whisper for speech-to-text, ChatGPT for generating dialogue, ElevenLabs for text-to-speech, and MediaPipe for motion tracking to create an interactive experience.
Link to project file + Plugin on Patreon: www.patreon.com/tblankensmith
Previous Tutorial on ChatGPT and Whisper: th-cam.com/video/34s2p9gzWhs/w-d-xo.html
Noto the Talking Ball's Tutorial: th-cam.com/video/va6cxORsFMw/w-d-xo.html
MediaPipe Plugin Tutorial: th-cam.com/video/Cx4Ellaj6kk/w-d-xo.html
0:00 Overview
3:11 ElevenLabs Plugin Overview
7:22 Connect ChatGPT to ElevenLabs
10:11 Configure Whisper with ElevenLabs
17:40 Add MediaPipe Face Tracking
มุมมอง: 1 138

วีดีโอ

Hand Tracking in TouchDesigner with MediaPipe - TouchDesigner Beginner Tutorial
มุมมอง 4.2K4 หลายเดือนก่อน
This tutorial is inspired from the Genuary (generative art month in January) prompt from day 3 to create something in the style of a Droste effect / a Picture in a Picture. I use the MediaPipe hand tracking plugin created by Dom Scott and myself to controll the motion. This also features a bonus Reaction Diffusion effect. Link to Genuary: genuary.art/ Link to the MediaPipe plugin: github.com/to...
Face, Hand, Pose Tracking & More in TouchDesigner with @MediaPipe GPU Plugin
มุมมอง 31K8 หลายเดือนก่อน
Hey! In this tutorial, we'll go over the new free open-sourced MediaPipe plugin for TouchDesigner that's GPU accelerated and works on Mac and PC with no instillation. The plugin supports face, hand, pose and object tracking with multi-person face detection, object detection, and image segmentation / background removal. Link to the GitHub to download the Plugin: github.com/torinmb/mediapipe-touc...
Tutorial 2/3 - Audio Reactive & Motion Controlled Visuals in TouchDesigner
มุมมอง 9K10 หลายเดือนก่อน
Tutorial 2/3 - Audio Reactive & Motion Controlled Visuals in TouchDesigner
Tutorial 1/3 - New Body Tracking Plugin for TouchDesigner - No Kinect Needed
มุมมอง 17K10 หลายเดือนก่อน
Tutorial 1/3 - New Body Tracking Plugin for TouchDesigner - No Kinect Needed
GPT-4, ChatGPT, and Whisper(speech-to-text) Plugins for TouchDesigner
มุมมอง 12Kปีที่แล้ว
GPT-4, ChatGPT, and Whisper(speech-to-text) Plugins for TouchDesigner
Video to Video AI Style Transfer with Stable Diffusion and Keyframing in TouchDesigner - Tutorial
มุมมอง 10Kปีที่แล้ว
Video to Video AI Style Transfer with Stable Diffusion and Keyframing in TouchDesigner - Tutorial
Create Audio Reactive Shaders With Three.js and Shader Park
มุมมอง 11Kปีที่แล้ว
Create Audio Reactive Shaders With Three.js and Shader Park
Audio Reactive Animations with Stable Diffusion and TouchDesigner - Tutorial 3
มุมมอง 34Kปีที่แล้ว
Audio Reactive Animations with Stable Diffusion and TouchDesigner - Tutorial 3
Web GUI for TouchDesigner - Component & Beginner Tutorial
มุมมอง 8Kปีที่แล้ว
Web GUI for TouchDesigner - Component & Beginner Tutorial
Generate AI Images with Stable Diffusion + Audio Reactive Particle Effects - TouchDesigner Tutorial
มุมมอง 19Kปีที่แล้ว
Generate AI Images with Stable Diffusion Audio Reactive Particle Effects - TouchDesigner Tutorial
TouchDesigner Tutorial - Generate AI Images with Stable Diffusion using Image-to-Image Generation
มุมมอง 43Kปีที่แล้ว
TouchDesigner Tutorial - Generate AI Images with Stable Diffusion using Image-to-Image Generation
Tutorial 2 - Shader Park Beginner Course : Space Modifiers, Noise & Reusable Custom Geometries
มุมมอง 3.2Kปีที่แล้ว
Tutorial 2 - Shader Park Beginner Course : Space Modifiers, Noise & Reusable Custom Geometries
Body Tracking with No Kinect in TouchDesigner 2022
มุมมอง 16Kปีที่แล้ว
Body Tracking with No Kinect in TouchDesigner 2022
Tutorial 5.1 BONUS - Use the Table DAT with Python. Control TD with a Website using WebSockets
มุมมอง 1.7Kปีที่แล้ว
Tutorial 5.1 BONUS - Use the Table DAT with Python. Control TD with a Website using WebSockets
Tutorial 6 - Send & Receive Messages from a Website. Control TD with a Website using WebSockets
มุมมอง 2.6Kปีที่แล้ว
Tutorial 6 - Send & Receive Messages from a Website. Control TD with a Website using WebSockets
Tutorial 5 - Receiving a WebSocket Message in TD. Control TD with a Website using WebSockets
มุมมอง 2.3Kปีที่แล้ว
Tutorial 5 - Receiving a WebSocket Message in TD. Control TD with a Website using WebSockets
Tutorial 4 - Send Data from TD to a Website. Control TouchDesigner with a Website using WebSockets
มุมมอง 3.7Kปีที่แล้ว
Tutorial 4 - Send Data from TD to a Website. Control TouchDesigner with a Website using WebSockets
Tutorial 3 - Testing with WebSocket King. Control TouchDesigner with a Website using WebSockets
มุมมอง 3.7Kปีที่แล้ว
Tutorial 3 - Testing with WebSocket King. Control TouchDesigner with a Website using WebSockets
Tutorial 2 - Host a Free WebSocket Server on Heroku. Control TD with a Website using WebSockets
มุมมอง 5Kปีที่แล้ว
Tutorial 2 - Host a Free WebSocket Server on Heroku. Control TD with a Website using WebSockets
Tutorial 1 - Intro & Overview of WebSockets. Control TouchDesigner with a Website using WebSockets
มุมมอง 7Kปีที่แล้ว
Tutorial 1 - Intro & Overview of WebSockets. Control TouchDesigner with a Website using WebSockets
Shader Park Plugin for TouchDesigner & A Rendering Workflow
มุมมอง 6K2 ปีที่แล้ว
Shader Park Plugin for TouchDesigner & A Rendering Workflow
Tutorial 1 - Shader Park Beginner Course : Core Primitives & Coloring
มุมมอง 4.1K2 ปีที่แล้ว
Tutorial 1 - Shader Park Beginner Course : Core Primitives & Coloring
Background Removal and Custom Webcam Effects: TouchDesigner Tutorial. Mac + PC
มุมมอง 7K2 ปีที่แล้ว
Background Removal and Custom Webcam Effects: TouchDesigner Tutorial. Mac PC
Audio-Reactive Visuals in TouchDesigner
มุมมอง 62K2 ปีที่แล้ว
Audio-Reactive Visuals in TouchDesigner
Link up Web Midi Instruments to your DAW of Choice
มุมมอง 5275 ปีที่แล้ว
Link up Web Midi Instruments to your DAW of Choice

ความคิดเห็น

  • @ricardcantm
    @ricardcantm 3 วันที่ผ่านมา

    thank youuuuu

  • @QISUN-ms5rn
    @QISUN-ms5rn 9 วันที่ผ่านมา

    太谢谢惹!!!!!好棒好棒的组件! 😁

  • @irenearias9411
    @irenearias9411 10 วันที่ผ่านมา

    thank you sososososooooo much. you are a literal angel and saved my life and i swear i'll join your patreon when i'm not a broke college student 💗💗💗💗

  • @BossCODINGplease
    @BossCODINGplease 11 วันที่ผ่านมา

    This is truely amazing, thanks for all the hardwork

  • @JannatShafiq-fr4lr
    @JannatShafiq-fr4lr 12 วันที่ผ่านมา

    Thanku so much for this amazing video and also for the link of file. My API component is not working can u plzz tell me is it due to version difference of touch designer if it is so plzz tell me which version u used for this.

    • @blankensmithing
      @blankensmithing 12 วันที่ผ่านมา

      It works fine for me on the latest TD version. Just make sure you create an API key on computerender.com/ and swap out your key in the project

  • @nightshining5662
    @nightshining5662 15 วันที่ผ่านมา

    Thanks for the plugin! For anyone having issues with the MediaPipe tox loading forever, the last version of 2022 (.35320) works for me on an M1 mac. I just avoided having to upgrade my commercial license for another $300 over this.

  • @QianqianZhou-hj8im
    @QianqianZhou-hj8im 15 วันที่ผ่านมา

    Hi Torin! Thanks for your plugin. But do you know why the error keep running "You exceeded your current quota". I upgraded the plan and using ChatGPT 4.

  • @phule5966
    @phule5966 16 วันที่ผ่านมา

    extremely underrated channel. Thank you!!!

  • @yoli5222
    @yoli5222 17 วันที่ผ่านมา

    i would be really grateful to understand the spoutcam for windows in depth for the mediapipeline plugin. Thank you

  • @user-rz4re7io6k
    @user-rz4re7io6k 18 วันที่ผ่านมา

    Hi, I'm trying to change some parameters in your text-to-speech plugin to suit my project, but it won't work after I unlock script2, I wonder what's the point of locking script2?

    • @blankensmithing
      @blankensmithing 18 วันที่ผ่านมา

      Locking the script2 CHOP allows you to programmatically write data into the ScriptCHOP. The audio data gets streamed in as chunks of data and I wanted to have the lowest latency so that you can start playing the audio the second it starts streaming instead of waiting until the whole thing finishes downloading.

    • @user-rz4re7io6k
      @user-rz4re7io6k 17 วันที่ผ่านมา

      ​@@blankensmithing Oooh, thank you very much for your answer, I kind of understand that you wrote a script that reads elevenlabs and streams back the audio and put it in scriptchop, and when scriptchop locks, the script freezes in TD,Like bg1TOP locked?

  • @neokortexproductions3311
    @neokortexproductions3311 19 วันที่ผ่านมา

    We appreciate you!

  • @annathomas5708
    @annathomas5708 21 วันที่ผ่านมา

    Hi Torin, Ive been using the MediaPipe plugin for my research project for over a month now. Ive had some questions and there aren't enough tutorials online for mw to figure out how to go about this. My project is exploring narrative animation driven by gestures. The animations are made on blender and I use the media pipe plugin to play these animations, so for example if you wave at the screen (open palm) the character in the animation waves back at you. Anyway I dont want to go into much detail but if this something that interests you I would love if I got some help, although this plugin id already helping me realise my enquiry. thanks so much, Anna

  • @marcomelilli166
    @marcomelilli166 22 วันที่ผ่านมา

    thank you very much for sharing this project, I tried to generate images and everything worked correctly until yesterday. Despite I added credits on computerender, since today, when I generate an image I get an error back “ {”status“: ‘error’, ‘message’: ‘Could not find \’complete\” state within timeout. You may try again.", ‘errorType’: ‘INTERNAL_ERROR’, ‘jobId’: ‘60f87997ecc64cba35e185a74ed0a4498c1b89e5’} ‘ ’, could you tell me how to fix it? thank you very much, i love your channel!

    • @bardoof
      @bardoof 9 วันที่ผ่านมา

      I've the very same problem. It worked correctly until few hours ago...

  • @justpassingbylearning
    @justpassingbylearning 22 วันที่ผ่านมา

    This is insane

  • @hreyhani4775
    @hreyhani4775 22 วันที่ผ่านมา

    This is amazing!!

  • @RetropunkAI
    @RetropunkAI 23 วันที่ผ่านมา

    omg this so so good LOL! Great use case... I assume the sour cream is under the hat?

  • @dvdgoldberg
    @dvdgoldberg 24 วันที่ผ่านมา

    yea ! you made it :) great work

  • @manaaloomerbhoy5264
    @manaaloomerbhoy5264 หลายเดือนก่อน

    so cool, thank you!

  • @roberty9057
    @roberty9057 หลายเดือนก่อน

    More than amazing ! Thank you so much

  • @bentheremedia3011
    @bentheremedia3011 หลายเดือนก่อน

    Is it possible to use this while also seeing the video feed from the background TOP of the container? I've built an app for an upcoming show and I was really hoping I would be able to just access and interact with the component from my iPad but it appears to be not as straight forward as I had originally thought.

    • @blankensmithing
      @blankensmithing หลายเดือนก่อน

      Yeah, at the end of the video th-cam.com/video/H8ePMpF3s_c/w-d-xo.htmlsi=7fW34_3FB1JGW_CQ&t=1636 I show how to use VDO Ninja to share a live feed of TD to the browser

    • @bentheremedia3011
      @bentheremedia3011 หลายเดือนก่อน

      @@blankensmithing Thank you! Will I be able to click buttons as well? Or will this just display the video feed? I was hoping to retain the ability to interact with buttons as I've built out a simple app that can navigate between different pages.

    • @blankensmithing
      @blankensmithing หลายเดือนก่อน

      @@bentheremedia3011 yeah both work :)

    • @bentheremedia3011
      @bentheremedia3011 12 วันที่ผ่านมา

      @@blankensmithing I've not been able to figure out how to click buttons through the live feed. It's my understanding that VDO Ninja is broadcasting a video feed but this does not give me interactivity with Touch Designer similar to how remote desktop would function. Are you aware of any ways to have an iPad acting as an interactive monitor for TD in Perform mode? That's basically what I'm after. I have a photobooth application that has buttons that need to be pushed in order to navigate between the different pages and I was hoping to be able to run it off of an iPad but I haven't been able to find a straightforward way of doing that.

  • @zablok69
    @zablok69 หลายเดือนก่อน

    Hey guys! Anyone knows if there is a workaround to use Mediapipe TD for vertical video / stream?

  • @yotube....
    @yotube.... หลายเดือนก่อน

    genio <3

  • @federicaintelisano5639
    @federicaintelisano5639 หลายเดือนก่อน

    Can you connect cameras like arri?

  • @shshsh-zy5qq
    @shshsh-zy5qq หลายเดือนก่อน

    I just heard someone mentioning about "touchDesign Audio Reactive" and I searched it on TH-cam out of curiosity. I am so glad I did. I found you and gonna try this tutorial. so amazing!! subscribed!!

  • @frequensea_experience
    @frequensea_experience หลายเดือนก่อน

    I really appreciate this tutorial. Not only the subject matter but the way you actually explain each part. Many of the Touchdesigner tutorials come off mathematical with no simple explanation of why a certain OP is used. Thank you

  • @IndorilTheGreat
    @IndorilTheGreat หลายเดือนก่อน

    I'll echo what everyone else is saying - this is incredibly helpful! Exactly what I was looking for in terms of a tutorial. Thank you so much!

  • @ngbnxtphase
    @ngbnxtphase หลายเดือนก่อน

    Just downloaded MediaPipe and I'm super excited to use it but the loading seems to never end for me I also submitted the issue on Github with more details Hope I get to make it work soon Appreciate your fantastic work and your help in advance

  • @1saac.mp4
    @1saac.mp4 หลายเดือนก่อน

    This is so awesome! I'm %100 gonna be hitting up your patreon for the project files, this is exactly the kind of stuff I've been trying to learn more about. I would love to implement this into a project. How many people is this able to track at once?

  • @tanguypaschal6657
    @tanguypaschal6657 หลายเดือนก่อน

    Dope

  • @FabioGopfert
    @FabioGopfert หลายเดือนก่อน

    is it possible to do it locally?

  • @bitface11
    @bitface11 หลายเดือนก่อน

    very good thank you!

  • @ajk_
    @ajk_ 2 หลายเดือนก่อน

    I have a lot of love and respect for the work you are doing. Thank you for sharing ❤

  • @ajk_
    @ajk_ 2 หลายเดือนก่อน

    You can’t make a tutorial on taste 🤌

  • @Zenosart
    @Zenosart 2 หลายเดือนก่อน

    Amazing stuff. Not sure I will ever needed but very nice. Bravo!

  • @larslangenbach4256
    @larslangenbach4256 2 หลายเดือนก่อน

    So amazing!

  • @larslangenbach4256
    @larslangenbach4256 2 หลายเดือนก่อน

    Soooo cool!

  • @larslangenbach4256
    @larslangenbach4256 2 หลายเดือนก่อน

    So amazing! Fascinated by your work. Very inspiring :)

  • @feelzlab
    @feelzlab 2 หลายเดือนก่อน

    Hi Torin, I'm slowly getting to understand more about the plugin. I tried following this tutorial with the Media Pipe plugin, everything worked really great, just needed to adjust the lagging for easing but what I can't seem to find on that one is the lagging features in the recent version of the plugin. Does it contain the lagging features? if so do they have another name or where can I find it?

    • @blankensmithing
      @blankensmithing 2 หลายเดือนก่อน

      hey, yeah you can use a Lag CHOP to smooth out the motion. Alternatively, you can use a Filter CHOP, but turn down the filter width

    • @feelzlab
      @feelzlab 2 หลายเดือนก่อน

      thank you!@@blankensmithing

  • @spindlewhorls9764
    @spindlewhorls9764 2 หลายเดือนก่อน

    thank you for your work it's real fun to use this and a lot of possibilities. I have one question: would it be possible to use the mediapipe plugin in td with an Infrared webcam?

  • @HeLevi
    @HeLevi 2 หลายเดือนก่อน

    I am doing a face-tracking vfx project right now, your Mediapipe is an actual life saver, but there is an issue that the Mediapipe container node keeps showing the loading animation, how could I fix this? Thanks in advance!!!!

  • @jonas8732
    @jonas8732 2 หลายเดือนก่อน

    awesome stuff!

  • @jonas8732
    @jonas8732 2 หลายเดือนก่อน

    algorithm food

  • @mariobnyctube
    @mariobnyctube 2 หลายเดือนก่อน

    Awesome!!

  • @user-kx7lv8ms7t
    @user-kx7lv8ms7t 2 หลายเดือนก่อน

    Hey,Torin.This video is very very good, but after I modified the keywords in TD, it kept showing loading and nothing happened. My version number is 2022.32660. What should I do?

  • @lotfibouhedjeur
    @lotfibouhedjeur 2 หลายเดือนก่อน

    Huh!?

  • @Ashman333
    @Ashman333 2 หลายเดือนก่อน

    Video changed my life lad fuck yeah

  • @zachmix3993
    @zachmix3993 2 หลายเดือนก่อน

    Underrated video

  • @RetropunkAI
    @RetropunkAI 2 หลายเดือนก่อน

    DUDE! Ha! That’s so cool. Super clever. A+

  • @maximevidal7932
    @maximevidal7932 2 หลายเดือนก่อน

    Ahaha Rival Consoles is such a gem for audioreactive effect

  • @Anckg
    @Anckg 2 หลายเดือนก่อน

    Should we implement the websocket DAT on a new project or should we implement it on an existing project?

    • @blankensmithing
      @blankensmithing 2 หลายเดือนก่อน

      you can do it in either