See accessibleai.dev/post/ml_net_2_0_text_classification/ for this content in written form and github.com/IntegerMan/MattEland.AI.MLNet.TextClassificationTurtles for the relevant code.
Hello! Im facing problem "System.MissingMethodException: 'Method not found: 'Tensor functional.softmax(Tensor, Int64)'.'" have no idea how to fix that. Have you any ideas?
Thanks so much for sharing this! The problem with some of this stuff is that it can change so frequently and I've been busy recently with a new class of students.
torch sharp cuda on update causes the load exception for method call not implemented...ITransformer model = pipeline.Fit(trainData); TorchSharp-cuda-windows .99.1 works
Hi Matt, Thank you for the great information! The sentence 'Heroes in a half shell' gave me the correct match with intent 'Ninjitsu' (with a score of 3,19) I created a new Intents file (using your sentences) by asking chatGPT to do the classification for the 500 sentences 😄 This new input file gave me the following confusion table as result: Confusion table ||======================================== PREDICTED || 0 | 1 | 2 | 3 | 4 | Recall TRUTH ||======================================== 0 || 23 | 5 | 0 | 0 | 0 | 0,8214 1 || 0 | 18 | 0 | 0 | 0 | 1,0000 2 || 0 | 0 | 17 | 0 | 0 | 1,0000 3 || 0 | 0 | 0 | 19 | 0 | 1,0000 4 || 0 | 0 | 2 | 0 | 20 | 0,9091 ||======================================== Precision ||1,0000 |0,7826 |0,8947 |1,0000 |1,0000 |
How do you match the score with the predicted label? I am writing some software involving image classification and want to provide a percentage score for the label the model has predicted. I can't work out what order the score array is in so I can't match the predicted label with the score it was given. For example if my classes are fish, horse, cow, eagle, cat and I input an image of a fish, how can I get the program to output: fish 86%. Thanks for the informative video!
The predictions for multi class classification come in an array ordered by the order the training process encounters each class, by default. That way you get the % chance something is each category, not just the highest likely category. I suspect there's a way to fix the order of the classes, but I believe the default is just the order encountered in training data.
Hi I have an understanding issue: If I trained the model via Fit method I will save the model. In the next step I can load this model again to predict something. What about retraining? How can I add new data to the model so that the predictions will be more exact over time? Is it the predict method itself, and I just have to save the model again? Do you have an example how to handle this? kind regards, Christian
To retrain a model, typically you run the fit process again with an updated data set that includes your new data points. This way your code to train your model is re-used, but the source data changes and the new model replaces the old one. In this manner, what you do as a data scientist is more of creating a pipeline that can repeatably train, evaluate, and even deploy or save the resulting models for usage. This is typically what is done with model training and replacement though specialized models may be exceptions to this rule.
Matt, I need an advice. I plan to learn some ML in upcoming holidays. I have zero knowledge of python and ML but long experience with C#. Should I learn pytorch bc more popular or ML. Net can be OK? Should I buy a computer with lot of ram and gpu? Final advice should I buy the ebook "introducing ML" by Ms press or it's outaded now? (pub 2020) thanks. Your voice is clear I can understand from a phone and I am not native English
Thanks for your comment! I would recommend scikit-learn as a first ML library for those familiar with Python and ML .NET if your comfort with C# is larger. Introducing ML is a great book, but it gets pretty advanced pretty quickly. You may have better luck with some of the white cover O'Reilly books. As far as your computer goes, as long as you're working with small datasets your computer doesn't matter too much. When you start getting into the bigger tasks and datasets, you may want to use cloud ML on Azure or Spark clusters or similar - areas where your computer doesn't matter either.
Hi. Loved your video, I am very new to ml topic , could you please guide me how can I generate a model text to c# suggestion , like openai does. Thanks.
I'm not fully sure I understand what you're describing. If you're talking about taking some text prompt and generating C# code from it, your best bet would be to use the OpenAI service on Azure or direct with OpenAI at the moment.
This would have been a pretty good video, but the speaker mumbles when he speaks. Just about every sentence is mumble, mumble, mumble. I find this often when viewing these kinds of TH-cam videos. People are so eager to share their knowledge, but they don't stop to consider that they need to present it well, which includes speaking clearly, emphasizing key words when appropriate, slowing down your cadence when a point being made might be misunderstood, adding some redundancies to reinforce a concept, and so forth. Don't just rattle off stuff using your mumble voice.
Not only do I disagree with this comment, but his clear enunciation changed my life. Even as I sit here in awe, my own data models, which are written in C# btw, are telling me how many pepperonis are worth going on an optimal pizza. While the feedback is still valuable I am sure, I would suggest to the original commenter that they turn on captions or try to watch the video again but this time with a heart that is open regardless of what pizza toppings are on their pie while learning about open source ML libraries. I mean sure pineapple is fine on a pizza, but are pepperonis? P.S. after years of training my data models, there are overwhelming results suggesting 0 pepperonis are the optimal number to put on a pizza.
See accessibleai.dev/post/ml_net_2_0_text_classification/ for this content in written form and github.com/IntegerMan/MattEland.AI.MLNet.TextClassificationTurtles for the relevant code.
Really well explained. Thanks
Thank you! I had fun with that one. I plan to explore ML.NET more in the new year.
Enjoyed this - thanks for this. Working on this topic as well, trying to find optimal tokenization and engine settings
Let me know what tips you have after you've gotten there.
Hello!
Im facing problem "System.MissingMethodException: 'Method not found: 'Tensor functional.softmax(Tensor, Int64)'.'" have no idea how to fix that. Have you any ideas?
was in my csproj. I needed to change it to 0.99.1. Looks like someone removed this method :)
Thanks so much for sharing this! The problem with some of this stuff is that it can change so frequently and I've been busy recently with a new class of students.
torch sharp cuda on update causes the load exception for method call not implemented...ITransformer model = pipeline.Fit(trainData); TorchSharp-cuda-windows .99.1 works
Hi Matt, Thank you for the great information!
The sentence 'Heroes in a half shell' gave me the correct match with intent 'Ninjitsu' (with a score of 3,19)
I created a new Intents file (using your sentences) by asking chatGPT to do the classification for the 500 sentences 😄
This new input file gave me the following confusion table as result:
Confusion table
||========================================
PREDICTED || 0 | 1 | 2 | 3 | 4 | Recall
TRUTH ||========================================
0 || 23 | 5 | 0 | 0 | 0 | 0,8214
1 || 0 | 18 | 0 | 0 | 0 | 1,0000
2 || 0 | 0 | 17 | 0 | 0 | 1,0000
3 || 0 | 0 | 0 | 19 | 0 | 1,0000
4 || 0 | 0 | 2 | 0 | 20 | 0,9091
||========================================
Precision ||1,0000 |0,7826 |0,8947 |1,0000 |1,0000 |
That's super cool!
How do you match the score with the predicted label? I am writing some software involving image classification and want to provide a percentage score for the label the model has predicted. I can't work out what order the score array is in so I can't match the predicted label with the score it was given. For example if my classes are fish, horse, cow, eagle, cat and I input an image of a fish, how can I get the program to output: fish 86%. Thanks for the informative video!
The predictions for multi class classification come in an array ordered by the order the training process encounters each class, by default. That way you get the % chance something is each category, not just the highest likely category. I suspect there's a way to fix the order of the classes, but I believe the default is just the order encountered in training data.
@@MattEland Amazing thank you so much!
Hi
I have an understanding issue:
If I trained the model via Fit method I will save the model. In the next step I can load this model again to predict something. What about retraining? How can I add new data to the model so that the predictions will be more exact over time? Is it the predict method itself, and I just have to save the model again? Do you have an example how to handle this?
kind regards,
Christian
To retrain a model, typically you run the fit process again with an updated data set that includes your new data points. This way your code to train your model is re-used, but the source data changes and the new model replaces the old one. In this manner, what you do as a data scientist is more of creating a pipeline that can repeatably train, evaluate, and even deploy or save the resulting models for usage. This is typically what is done with model training and replacement though specialized models may be exceptions to this rule.
I received an error: Unable to load DLL 'LibTorchSharp' ... I didn't see anything about needing this lib in the docs or how to install it.
Did you find How to resolve it?
Apologies on the delay. What operating system are you running on?
@@MattEland i don't know he, but i had the same error and i was using windows 11
@@lucasbianchezzioliveira7664 hey bro did you find any solution for this error?
@@mhrezash no, i guess it's a problem in LibtTorchSharp's library, maybe compative problems. I already gave up
Matt, I need an advice. I plan to learn some ML in upcoming holidays. I have zero knowledge of python and ML but long experience with C#. Should I learn pytorch bc more popular or ML. Net can be OK? Should I buy a computer with lot of ram and gpu? Final advice should I buy the ebook "introducing ML" by Ms press or it's outaded now? (pub 2020) thanks. Your voice is clear I can understand from a phone and I am not native English
Thanks for your comment! I would recommend scikit-learn as a first ML library for those familiar with Python and ML .NET if your comfort with C# is larger.
Introducing ML is a great book, but it gets pretty advanced pretty quickly. You may have better luck with some of the white cover O'Reilly books.
As far as your computer goes, as long as you're working with small datasets your computer doesn't matter too much. When you start getting into the bigger tasks and datasets, you may want to use cloud ML on Azure or Spark clusters or similar - areas where your computer doesn't matter either.
Great vid thanks. How do you export the model to file and then load again? Would love to bust these out into train, evaluate, and use classes.
can I use it to extract data from text instead ? like give it sentences and it extract emails, or phones etc .. ?
Could you make same code with different example other then turtle ? Like health care or e-commerce
I'm making a few examples around Football for the book I'm writing, but will consider e-commerce or health care for the course I'm creating
Hi. Loved your video, I am very new to ml topic , could you please guide me how can I generate a model text to c# suggestion , like openai does. Thanks.
I'm not fully sure I understand what you're describing. If you're talking about taking some text prompt and generating C# code from it, your best bet would be to use the OpenAI service on Azure or direct with OpenAI at the moment.
This would have been a pretty good video, but the speaker mumbles when he speaks. Just about every sentence is mumble, mumble, mumble. I find this often when viewing these kinds of TH-cam videos. People are so eager to share their knowledge, but they don't stop to consider that they need to present it well, which includes speaking clearly, emphasizing key words when appropriate, slowing down your cadence when a point being made might be misunderstood, adding some redundancies to reinforce a concept, and so forth. Don't just rattle off stuff using your mumble voice.
Thanks for your valuable feedback! Have a great week!
Bro turn on captions
Not only do I disagree with this comment, but his clear enunciation changed my life.
Even as I sit here in awe, my own data models, which are written in C# btw, are telling me how many pepperonis are worth going on an optimal pizza.
While the feedback is still valuable I am sure, I would suggest to the original commenter that they turn on captions or try to watch the video again but this time with a heart that is open regardless of what pizza toppings are on their pie while learning about open source ML libraries. I mean sure pineapple is fine on a pizza, but are pepperonis?
P.S. after years of training my data models, there are overwhelming results suggesting 0 pepperonis are the optimal number to put on a pizza.