วีดีโอ

No, runs completely local

Hit up the code link in the description and run the script. If you’re missing torch with CUDA, check out the main project README for setup. Got issues? Join my Discord and I’ll help you out: discord.com/invite/f556hqRjpv

Sorry, I can't share or make it public, as it's trained on voice data with unclear rights

Nope, the dataset I used to train the classification model only has English sentences.

Repo is here: github.com/KoljaB/WhoSpeaks

Good observations. At 0:38 it seems like the sentence got detected too quickly. I needs some deeper analysis for those edge cases to figure out why the algorithm cuts into the sentence. You can fine-tune Whisper to handle new words and then use the fine-tuned models directly with RealtimeSTT: huggingface.co/blog/fine-tune-whisper For 2:25, this looks like a Whisper quirk. Whisper tends to remove filler words like "ah" or repetitions. It might improve if I tweak the initial_prompt behavior to use it only for the real-time model but removing it from the final transcription. Whisper tends to lose some accuracy when prompted, but in this case, it’s necessary for real-time processing. Unfortunately, there’s no simple parameter to make Whisper more accurate regarding filler words. If it’s critical it's needed switch to a different ASR model, there are some that deliver precise transcriptions even on filler words, though that brings its own set of pros and cons. Regarding speaker handling, Whisper doesn’t support speaker diarization natively. It always assumes a single speaker. Real-time speaker diarization is a whole other rabbit hole. Not impossible, but very complex to pull off.

You're right, the title's a bit misleading. Edge-TTS itself isn't new, but what's new is the real-time aspect. You can now use LLM outputs and get instant TTS results, or load massive texts and hear the output in just a second.

I know better docs and tutorials are needed. The thing is, I love building stuff, but I hate documenting. Plus, I’m a bit of a perfectionist, and making a proper tutorial that covers everything RealtimeTTS can do would take me weeks. :)

Latency way better. Linguflex core is growing nicely. Home assistant module hasn’t seen much progress. Honestly, I could really use help (PRs welcome). Got way too many projects, so some stuff is a bit stuck in "proof of concept" mode for now.

This code is already outdated. Please use the main installation (pip install realtimestt) from the master branch. Then start the client with the `stt` cli command. If you still run into problems I suggest we discuss that on my discord in the RealtimeSTT section discord.com/invite/f556hqRjpv or per mail (kolja.beigel@web.de), happy to assist

Good question. It likely depends on the Edge service availability and your internet speed. My system is pretty fast, but some imports required for other engines might affect performance on slower CPUs.

Hey! I get why a video would help, but it's tricky for me since my system is already fully set up for AI. On my setup it’s just `pip install RealtimeSTT` because everything (Python, CUDA, etc.) is already installed. Also installation is very different for Linux and Windows. For example on Linux, Python is pre-installed and you can handle the installation process just via CLI. On Windows, you'd need to install Python, CUDA, and the toolkit software first. I suggest join my Discord at discord.com/invite/f556hqRjpv for help (we can voicechat there, also there's a RealtimeSTT section) or mail me at kolja.beigel@web.de!

yes please. I'm lookign to do this myself.

I don't think it can run without an Nvidia GPU. The assistant relies on several models, like Whisper, Silero, and WebRTC VAD for ASR, as well as XTTS for TTS and some RVC models for TTS post-processing. These typically require CUDA for optimal performance, and I'm not sure if they’re fully compatible with ROCm for AMD GPUs.

Use input_device_index property of AudioToTextRecorder class (hope that's what you mean)

@@Linguflex it works!! thank you...btw I want to make a question answering ai base on your RealtimeSTT transcribed. I am open for suggestion...

@@krosx278 Please look at advanced_talk.py or openai_voice_interface.py in RealtimeSTT test folder or at my LocalAIVoiceChat project

@@Linguflex I want to use llama and fine-tune it... Is that possible?

@@krosx278 Yes, of course

github.com/KoljaB/RealtimeSTT/blob/master/tests/realtimestt_test_stereomix.py This script records from system audio, you might need to change LOOPBACK_DEVICE_NAME to your stereomix device name

Will try tomorrow on ubuntu

Had similar issues on Ubuntu. I upgraded RealtimeSTT to v0.3.5 so with "pip install RealtimeSTT==0.3.5" these problems should be solved now.

how did you implement that? can you share your code?

@@krosx278 implement what exactly? to have the STT detect browser audio? For that, you need an audio mixer or virtual cable.

@@kritikusi-666 i already saw the latest code from the author its quiet i need... check for the update.

Code is here: github.com/KoljaB/RealtimeTTS/blob/master/tests/pyqt6_speed_test_chinese.py

faster_whisper for speech to text, Silero VAD + webrtcvad for voice activity detection, llama 3.1 8b for sentence end verification

@@Linguflex llama api?

@@parthpatwari3174 LMStudio

work in progress: github.com/KoljaB/RealtimeSTT/blob/dev/tests/realtimestt_speechendpoint.py

Thanks for the amazing work . Will it support automatic multi language detection as well? Pls advise

​@@SaddamBinSyedWouldn't be hard to implement. faster_whisper is currently buggy when no language is set

Voice is mashed together in Elevenlabs, a bit complicated process involving training a voice from different voice sources

Yes sure. It's MIT license so you can use it for whatever you like.

Yes, github.com/KoljaB. No code for this one up yet tho, bcs it's too early work state

What about now? Is the github for this available? ​@@Linguflex

@@Linguflex Hi, is there still github yet for this?

@@AustinKang-wk8cl I checked in the code for memory in Linguflex github. It's not perfect, but it's a start...

Sorry, can't help. I have nothing to do with MoneyPrinterTurbo. This is a translation example as showcase for my TurnVoice GitHub project.

You gotta put the pexels API key in quotes like this - """ pexels_api_keys = [ "nE113EOVlRVbpWvRE0yZFuy6KmM9WAqvelyadayadayada",] """.