วีดีโอ

github.com/Mayurji/Explore-Libraries/tree/main/MLOps

Noted. 👍

Langchain is free, and pinecone is not free. But you can use any vectordb like chroma or faiss for the experiment.

@mlworks thankyou again

Please find the doc here: github.com/Mayurji/Explore-Libraries/tree/main/MLOps/Ray/ray-mlops

import dask import dask.dataframe as dd import pandas as pd from dask import delayed # Step 1: Create a function that uses pandas' json_normalize def normalize_json(data): return pd.json_normalize(data) # Step 2: Wrap the function with dask.delayed delayed_normalize = delayed(normalize_json) # Example JSON data data = [ {'name': 'John', 'info': {'age': 30, 'city': 'New York'}}, {'name': 'Jane', 'info': {'age': 25, 'city': 'Los Angeles'}} ] # Step 3: Use delayed to process the data # Create a delayed object delayed_data = delayed_normalize(data) # Step 4: Compute the result result = delayed_data.compute() print(result) Explanation: Function Definition: The normalize_json function takes a list of dictionaries (the JSON data) and applies pd.json_normalize. Dask Delay: The function is wrapped in delayed, allowing it to be executed lazily. Data Processing: When you call delayed_normalize(data), it creates a delayed object. The computation is triggered only when you call .compute(). Benefits: Parallelism: You can encapsulate multiple calls to json_normalize using Dask's task graph, allowing Dask to optimize execution. Flexibility: This approach allows you to leverage Pandas functions while still benefiting from Dask's parallel processing capabilities. Source: ChatGPT. Let me know if it works as expected.

Notebook Link: github.com/Mayurji/Explore-Libraries/blob/main/Deep-Learning/Audio-Clf/audio-classification.ipynb

@@mlworks thanks alot brother ❤️

You can follow this link huggingface.co/blog/fine-tune-wav2vec2-english

You can change the TH-cam setting Quality while watching.

Can you be clear?

@@mlworks dask uses lazy evaluation so in order to calcul the output u have to finish your line by compute()

You can transform the wav file or audio into floating point representation using librosa or any other audio processing library.

Sample: stackoverflow.com/questions/54482346/reading-a-wav-file-with-scipy-and-librosa-in-python

Thank you

huggingface.co/blog/fine-tune-wav2vec2-english

@@mlworks thank you so much

That you can get using the requests library.

Good question....idk lol.

Yes.

Which LLM provider are you using?

@@mlworks Using the HuggingFace Pipeline with the same model you used. I did different queries and their responses were in Chinese

@@DKBOsei00 can you share the prompt?

@@mlworks Question: What is South Park? Answer: 表演

github.com/Mayurji/Explore-Libraries/blob/main/LLMs/phoenix-llm-observability-checkpoint.ipynb

Sure

Yes.

Yes, you can use the ada embedding model for generating an embedding vector and store it in a vector store. If you have large documents, you can chunk them based on various chunking techniques that are available in many RAG applications, ensuring the embedding model doesn't truncate your embedding randomly.

how would you do the search? What Tool do you recommend me? Im very new to the topic. Thanks in advance@s

can you tell me about the chunking techniques and the RAG Application? How could I start? :)@@mlworks

@@Z4rnii For chunking, follow this notebook. github.com/FullStackRetrieval-com/RetrievalTutorials/blob/main/5_Levels_Of_Text_Splitting.ipynb

@@Z4rnii Simple RAG - huggingface.co/learn/cookbook/en/rag_zephyr_langchain

page_content_column is the column that represents, the context column of the dataset. Checkout the context column here huggingface.co/datasets/databricks/databricks-dolly-15k What is the task you're going to implement with the dataset? QA I assume

@@mlworks yes a QA, and I'm suppose to use LLM model to make it more interactive and that's when I got confused because when I passed the data my bot is responding Grammatically incorrect sentences

@@rps6949 RAG helps LLMs to provide accurate and precise information based on the knowledge base. Are you using pretrained LLM or a custom LLM which you've got after fine tuning llm for your dataset, that has learnt the mapping between QA. If yes, then you can generate embeddings of your answers & store in vector store and ask the query to the system.

@@rps6949 if you mean interactive as in chatbot, you can set the system prompt for the LLM and this act as conversation agent.

@@mlworks i'm using a pretrained model , as in your code you are using datasets and then huggingface loader to load , what i have to do is ui from where user is uploading a csv file so i used CSVLoader but just like context column i have a text column but i'm not able to provide that column for embedding i just loader data from loader.load(), what i can do here?

I've stopped that in the last few videos.