Why I Don't Like Machine Learning
ฝัง
- เผยแพร่เมื่อ 24 มี.ค. 2019
- In this video I discuss why I have no interested in machine learning.
----
If you like cooking, checkout my side project: www.mysaffronapp.com/
----
Join the Discord: / discord
----
Patreon: / benawad
----
Follow Me Online Here:
Twitch: / benawad
GitHub: github.com/benawad
LinkedIn: / benawad
Instagram: / benawad97
Twitter: / benawad
#benawad
TikTok: / benawad
----
Follow me online: voidpet.com/benawad
#benawad - วิทยาศาสตร์และเทคโนโลยี
![BOWKYLION - เจ้านายคะ (work-life imbalance) [Official MV]](http://i.ytimg.com/vi/6bFj8G2pOi4/mqdefault.jpg)
![[LIVE] : ONE ลุมพินี 72 วันนี้!! คู่เอก "ก้องศึก vs โจอาคิม"](http://i.ytimg.com/vi/hT1EOSsRPac/mqdefault.jpg)
![โกหกเธอทั้งนั้น (Pinocchio) - SERIOUS BACON [Official MV]](http://i.ytimg.com/vi/P1HhTYwrR5A/mqdefault.jpg)
People who are passionate about machine learning either have a domain specific problem they are trying to solve or care about inventing novel learning algorithms.
Most underrated comment here.
Lol wtf
novel learning algorithms are fun tho
For me, the interesting part of ML is not training/tweaking models on a dataset, it's learning about the algorithms that power the models. I agree that just spending hours tweaking a model's paramteres is not that fun, but building your own neural network, decision tree, support vector machine, etc from scratch and getting it to work on a variety of datasets can be a fun challenge and is way more engaging then just training a model on a dataset. True, it doesn't have as much real world application, but it definitely gives provides you with an interesting challenge as a developer.
It can have critical real-world applications if you are at a research level making better algorithms. Having meaningful impact at the world is a hard mission that tends to take time. You have to do the basics and then go deeper on a problem until you are doing something unique.
where can i learn how to build a neural network from scratch
@ neuralnetworksanddeeplearning.com/
Well but in the great majority of works in this area you won't be able to build your own decision tree or whatever lol, you will in fact, tranining/tweaking models on dataset's for solve problems.The market share for researchers of this type is relatively small, and you may be left with just becoming an academic. I don't understand the students' hype about ML, I guarantee you that for those who don't know ML, it looks a lot more interesting than it really is xD (and this comes from a person that loves math and physics aswell)
@@Lucas-of6ou I find your point of view hard to relate to as I am graduating from my undergrad this semester and DL has basically changed my entire cs career. I went to a state school for CS not really knowing what kind of software I wanted to engineer. After being taught about neural nets in my junior year, pretty much everything has been rapidly getting better for me. I became fairly obsessed with deep learning, I write code daily, I have been flooded with interviews, I've written two research papers along with PhDs from more prestigious universities and I love the research I do. I'm actually not great at math and have been working very hard to improve my understanding of advanced ML concepts like hessian matrix approximation and bayesian hyperparameters search but most importantly the research I do fascinates me. And all of my research is application driven: Vehicular LiFi, bacteria colony classification, space situational awareness. All of this research has completely different challenges but all eventually boil down to a neural network making the hardest parts possible. That's just amazing to me and I hope other young ML enthusiasts here are able to find similar experiences to mine. Also I'd like to note that ML encouraged me to start learning web development, big data programming, and cloud technology so I could get my models in "deployment" behind a great website. It doesnt have to be all boring theory I promise 😁
The problem with this approach is that you don’t learn the math. ML is great if you like math.
there are plenty of other topics in computer science that are great as well if you like math. in fact, I would argue that machine learning probably isn't that great if you like pure math, but rather enjoy learning how to solve math problems. For me, the math involved with machine learning is just not my type of "flavor" of math that I prefer, I guess. I much prefer learning about more theoretical, discrete math, hence my interest in programming language theory and implementation. Super interesting stuff if you enjoy the cross between the theory and practice of computer science.
Sadly, it was not the case for me though. The math behind ML is mostly from the analysis and probability theories, isn't it? Surely I'm not a deeply trained mathematician so somewhat afraid of saying this aloud, but what I could say is that not every mathematicians would like ML, who would enjoy algebra and geometry, or even analysis. Too much notations but not fancy results which I could discover from the books like Hatcher or Conway.
@@Baconator1368 I agree
@@sungjuyea4627 to me ML seemed mainly like calculus with a bit of statistics added in, but I've only looked as far as how a basic forward/backward propitiation algorithm works.
I like math but don't like ml
I concur. I've been a data scientist for 6 months, hated it, went back into software development.
Machine Learning (or inductive programming, more correctly) is only stimulating if A) you're purely working on the math behind the models or B) you're making an entire application with the models you're building or C) you're programming a model for which no implementation exists in a mainstream framework. This last case leads you to learn a lot about low level programming and parallel computing, and honestly I find it really fascinating.
Our thoughts match a lot, I also tried ML a year ago with full dedication because of the hype but eventually realized I enjoy the process of building a project a lot more than tweaking models and analysing data, so I switched back to being a software developer and have been better at it since then.
Same here. I also fell for the hype created around ML, AI and DS. But now I have realized I love creating and building projects than working on models.
as a webdev currently learning ML, all of this is true. i like that you own it.
As a Data Scientist currently learning Web Dev, all of this is true. i like that you own it.
@@jean4j_ quick comeback 🤣
Jean-Loïc De Jaeger I HATE WEB DEV LMFAO
DC - TLC alright bro
DC - TLC you got me!
This is a common problem: software dude gets in to ML thinking all he needs is to learn code and libraries and doesn't end up understanding what's going on cause it's all mathematics and statistics. Kaggle is not the best, but the worst way to start ML. The best way is cal+linear algebra and then a ML course which explains the math.
It’s crazy math, you need to know Taylor series expansion in order to understand simple logistic regression in some instances.
@@tyrantula767 Taylor series expansion is highschool level math
Vroom bloody hell 🤦🏿♂️
If you’re from the UK, you need A level maths and further maths knowledge to understand linear/logistic regression, using matrices & vectors, and differentiation for gradient descent or the normal method of optimising the cost function.
Billy I’m learning data engineering now with Python and SQL; I want to pick up the math and transfer over to data science / machine learning.
I’ve sort of felt a similar thing with my own work with deep learning. Though Fortunately for me as I’m working on a research project where I’m in charge of a lot of the technical aspects and their implementation, I also work on developing the mobile app for it, the data pipeline, the UI etc as well. So luckily for me I get to mix things up a bit and that ends up making the deep learning parts very interesting. Plus I also get to look for different algorithms/solutions myself and learn a ton, and I don’t have what algorithms/approaches get used dictated to me by the higher ups so much. It could just be that you are more suited to working for a smaller scale project/startup rather then developing some random model that’s a small piece in a large company.
Nice content. I love how your channel is not just about tutorials, but also opinion and personal experience. Subscribed!
thanks, welcome :)
I made a hunter/gatherer simulation in Unity with their ML tools and this was pretty much my experience too. At least in version 0.3b I would have to wait for over an hour to see if the changes I made to my training setup was any good.
As an ex-data scientist who has transitioned into backend software development, this resonates with me on multiple levels! Thanks for putting this out so clearly.
can u recommend a few sources to learn software development? like which courses, books, projects should i do to get into it
I love SKY Kit Learn.
I also like to program in Arrrrrrr
I feel like I’d lose interest in ML too if I was using sky-kit and not sci-kit
😂😂😂
SKY rocketing your models with Sky-kit since 2010. lol
I love psychic learn
There’s specific algorithms that tune all the hyper-parameters like Bayesian optimization so you don’t have to just randomly tune hyper-parameters. The real work is not adjusting values by 5% to improve the accuracy, that’s a perfect menial job for a computer.
-be rich
-grid search
I'm with you on this one. I too did some ML projects when I was in college and it was fun at first then it just started getting boring, waiting for the model to finish training and tuning all these parameters again and again until your loss function is low was just not for me. But the thing is I do like doing data science stuff if I had the opportunities but not ML.
And like you said doing system design and backend engineering is more of my thing
I agree. When I was doing AI stuff I found the possibilities pretty interesting, but the actual practice of doing everything was kind of boring and also not really capable of doing exactly what I wanted to and to the degree that I wanted. I personally love building web apps, or even mobile apps, because I like interacting with people and building something that can solve their problems and improve aspects of their lives. Also it's more tangible and for me that always helps
The AI Hype Train is amusing. I really enjoy listening to or reading non-Techies pontificating. They act like the cure for all diseases is a few months away.
Great video. Figured it was a boring field.
PS - maybe too honest admitting to telling people you interned at BoA - in your age bracket that could get you spit on.
I'm somewhat opposite lol. I like the mathy, scientific computation. I would say that I don't enjoy working on front-end stuff for other people's products. However for my own entrepreneurial pursuits, I like that too.
Mark S I would agree with you currently, but you can’t just write AI off... especially since you don’t know what new innovations are going to happen in the future. Also the current methods aren’t even close to what a true AI system is supposed to be so I don’t see how you can write it off so soon. Everything takes time. It might take a decades, but at some point AI will solve many problems. But I don’t think anybody was thinking that it would be advanced enough to cure diseases in a few months anyways.
@@JoeARedHawk275 Please point out where I said I was 'writing off' AI - just saying it's overhyped. Specifically it is currently extremely limited and those using it - like TH-cam here - aren't very good. Why does TH-cam recommend videos I've already watched or from channels I already said 'don't suggest anymore'? I'm sure you and other programmers know why - because it only does what it's told to.
Even ML - it's only as good as the humans correcting it. Did you know there are rooms full of people in Africa and Asia that check the ML to see if it was right or wrong? If they screw up the ML can think a cat is a dog.
@@WestCoastAce27 I agree with you somewhat. It's overhyped, yes, but I believe it's primarily stuff like deep learning and "AI"/ reinforcement learning, and not everyone realizes it's largely FAANG/IBM/Nvidia doing some insane stuff while most other places can use simple solutions (not necessarily easy - you'd still need people who understand the stats) to solve real problems.
Machine learning is the glorified name of applied statistics
And applying statistical models is just another word for profiling
It's more math than computer science.
Yes and no. For some problems, it’s more powerful to have higher abstraction mental models that describe the emergent phenomenon. The relationship between statistics and emergent machine learning is very much the relationship between physics and emergent chemistry.
(And for perspective, statistics and physics are also emergent, some thinking is even basic charges like the charge of the electron are due to emergent phenomenon from simple rules at scale)
@Demon King The stuff you explained is still math, and yes I'm well aware of heat transfer, finite element method and etc.
Computer science to me is more about algorithms, data structures, compilers, computer architecture and stuff like that.
Only math students can relate to this precious comment
The way I learned it and came to enjoy it was by researching the theory first, but then when I came to its application, I didn't use python (since I knew too little of it) and I didn't use any library whatsoever. I went pure raw c++ (what I knew best) and as I said, I didn't use any library, I just coded the entire thing myself. I had to re-learn derivatives, so I picked math books, and went to math videos explaining the whole thing from scratch. It was actually fun, I always thought that no matter how complicated something is, when you do it yourself, when every single step is done by yourself, no matter how frustrating some moments get, it's infinitely more rewarding. Only after learning that, and after getting competent at it, I started learning python and picking some libraries, slowly, one by one. But if you are just copying and pasting code, you might get results faster, but in my opinion, you are doing it the wrong way as you will have no idea of what you are doing for half the process, and at least for me, that's not enjoyable.
After working as a software dev for a few years, I recently took interest in ML. Now, I have similar thoughts as you.
Learning the math behind ML and the model building process you mentioned is really taxing.
true
Yes, I work with ML projects, and you are right but it's supply and demand. And you get to build user interfaces in ML projects as well.
Did you use ML for your recipe site to extract ingredients, steps, etc..? Has your opinion changed since this video?
Yeah... I fully understand you, I am a full stack developer, and I would say that I also feel boring with ML :(
But it depends on the project, some projects really cool to include some ML in it.
As a mathematician who has paid too many side courses in ML and after 4 years of professional experience in data analysis/statistics and machine learning I am looking forward a change in my career. All you said is true, but I'd extend your critics even more.
ML is so hype that even serious researchers tend to include buzzwords in academic papers. Many algorithms are based in quite old mathematics, which is totally fine but many times this old tools are renamed in a very fancy way so they get more acceptance... Which is intellectually dishonest.
The thing get worse in industry. Many times data hasn't enough quality to be analyse nor use in any ML model, but the manager who is extremely hiped just want the model to work so the company can say that they are a "data driven company". At the end you end up with an extremely silly model biased towards what the final user wants to hear.
Not talking about the quality of the software develop by data scientists... Which extremely poor most of the time (including me, but at least I try to get better). Notebooks every where!
Nice insight. Thanks!!
So overrated that we also got incredible technologies like GPT and Midjourney which are making a real impact right now. It's not overrated at all! People just used it to things they shouldn't have used for!
This aged well. 💀
Yep it's aged like wine soybots were hyping AI thinking it's gonna give them AI waifu.
@@GIGADEV690 It won't???? 😔😔😔😫😫😫
I ventured into ML a bit a few months ago. It definitely seemed tedious to get a model working. I backed off when I realized I had plenty of other stuff to learn and didn't need to get deep into machine learning at that point in time.
Hello ive been watching a lot videos from this channel cuz your pronunciation its very smooth and it helps me to improove my english skills. Thanks
serious question, if all you do to change a machine learning model is tweaking parameters, can't you automatise that? i'm not saying automatise everything, but part of it.
for example, if you don't know how many layers your model should have, you could try every depth from 2 to 10 by training 9 different models and comparing results.
Regardless of whether he had any basics or not, I think it is very important to recognize that the role of an ML Engineer or a data scientist is very different to that one of a backend or software dev. It is important to recognize that if you already enjoy your work in one of those roles, finding an opportunity that allows you to experiment with doing data analysis or even a deep neural net and keep your common responsibilities in another web dev role is near impossible.
To learn ML or DL and apply it correctly you will most likely end up doing a career swap.
I actually come from a math background. I have a MS in applied math. A lot of grad students' first thing they do after they graduate is go into "data science" because that's what is hyped. I tried it but didn't find it that interesting. I find web dev front end stuff more fun. I like having an idea first and then implementing it learning what I need to know as I go along. Whereas what I see a lot in the ML world is people learning the techniques first and then trying to find something to apply them to. I have trouble bring motivated to do that.
If I see a use for an ML model in something I actually find interesting, I would have no problem doing it.
what is the best sources to get good math background ??
hichem Benchikh I’m using Khan Academy, brilliant.org, and some for dummies books
@@hichembenchikh930 uni
Thank you very much for your point of view. It has been very useful for me :)
The most difficult part of the process is having to wait for the model to train and evaluate. The major drawback many ML enthusiasts have is not having sufficient compute power to work on models. Although, there are free trails available but that doesn't add up to quality time. Unless one has access to good GPU, it would be really boring if not frustrating at times.
This is exactly what I had in mind. Thanks for telling it loud
If you have large dataset. You could probably take small enough sample to train and test a model quickly before using that model to be trained over the entire dataset(‘s 70-80%).
Also their are some online (free and paid) resources available to do the processing faster but in general the process is iterative.
So it could come down to how, during the exploration process, when you analyse the data, what kind of model do you think would suit the data.
For example if you plot a dataset and see a linear trend with low spread you could for linear regression, if you see an exponential trend still you could use linear regression by transforming the dataset by taking its logarithm.
Going head first into problem can lead to multiple iteration and the pros are that you gain valuable in depth knowledge of what doesn’t work.
Alternatively, you could formally do a course that that demonstrates implementation of models suited for varied datasets, so that for any new project you know what model would suit the dataset aka give best predictions. Additionally, instead of straight away diving into its implementation and then tinkering the model parameters or datafields one could revise the mathematics of that model for some time.
I done my BSc+MSc in AI (yes full complete AI programs in the Netherlands have existed for over 15 years now).
I don't like the ML community as it currently is with everyone hyping it up, and using the standard buzz words for everything. Everyone tends to use DL for everything and new folks to the field are directly introduced into it after they seen linear regression. It is like all you do is give them a hammer (or rather a sledgehammer or a shotgun) and teach them everything is a nail. There is no true understanding, there is just applying and not even the good type.
Every introducee to the field is hyped and every project they do results into the same: "I tried very advanced stuff for weeks, code is disorganized jupyter books and the output/deployment sucks". They don't learn that the first question in ML you should ask is always "Should you?" and the second question is "Can you?". It is just like all the other CompSci stuff where you need to engineer smth that fits the problem well. There is absolutely no point in wasting countless hours on setting up complex models such as DL for some simple 1k rows dataset where in its deployment setting it doesn't have the necessary data for prediction.
Good enough, manageable, (low/ un)biased, deployable, understandable non-cheating ML is what one should aim for.
A first and second degree in AI right off the bat? Something has to be said about grasping fundamental engineering skills before diving into something very niche as AI. Once you do that, learn to write proper code. Once you do that then there might be a chance of working on something niche, that is if you have time left.
Hey can you guide me on how to do that? Any books, any resources, where I can learn this thing fundamentally?
I agree! ML/ data science is so much more than just coding complex stuff. Understanding what you're doing and understanding the data itself is super important and imo an important first step
ANIRUDH TD Imperial London College launched a course of mathematics for Machine Learning. After that you can do the Machine Learning course from Andrew Ng. I think those two set a good basis for truly understanding ML.
I think that the main problem is that all that work from those PhD geniuses making life easier through very complex code reduced into a simple class or functions has really take a toll on newcomers ML aspirants. People don’t realize the probability theory behind all those functions so it’s blind for them. Nowadays people just sit down, load the data and run the machine learning function. They don’t take the time of checking the distribution of the data, the standard deviation and all that important concepts that are fundamental for a good prediction.
Totally my thoughts man. Currently learning neural networks, just want to make a game AI and then I don't know if I will stick to it.
What kind of game AI do u need? Chances are is that NN is not the way to go. Lots of games of AI based on just simulating movement or emergent behaviour (look up how F.E.A.R. does it with states). So much easier and lots more quick results and u can actually inspect+control.
I went through the same thing man, I was planing to read about every neural net architecture & publish an article about it, so far I went through the perceptron, I got how does a multilayer & CNN work but I seriously got stuck on the RNN LSTM as it requires lot of maths and I didn't really enjoy using those libraries like tensorflow or sklearn because I end up just copying and pasting code which I didn't enjoy so I had to stop.
Yeah it takes a lot of study to understand how it all works
@Shah Bhuiyan who will fund me while I study that?
dude, you work in academia? If you do, just find another job.
I love the results of ML (next to it being a scary black box you don't know what it bases its guesses on), but I can imagine this process being rather tedious. Thank you for your insights!
I had the exact same experience but this was 8/9 years ago. I didn't like the slow feedback loop but curious if there are better ways to train the model thesedays
I feel the same way! The end of DL may be interesting but the process is quite boring and is hard for non-methematics backgroud people. But I kind of have to learn DL for my research :(
Still waiting for next trend in data analysis field!
Probably you can try to get into the math of ML and try to make sense of why some set of parameters work and why others don't, to make it less boring. The mathematical intuition will probably help you in the longer run in your future projects.
yeah if you want to get into ML, learning the math behind it is a good idea
um.. no. It's either the math or the application. The industry forces you to do so.
@@bawad No. It should be later not first. These Indians are employed by giant Outsourcing firms and they have hidden agenda to make entry to ML harder.
@@SR-er6hx ????? LMAO
@@ScribbleDribble Seriously
Thanks for this video!!
I basically use ML whenever I want to recommend something to a user from a database filled with user generated content. similar to how youtube wants to recommend videos to you, without knowing beforehand which videos will be available. I think that kind of reinforced learning is actually the most useful and also pretty easy to implement compared to a lot problems that people try to tackle that don't have a real world application (yet)
I wrote a web app which utilised ML for my dissertation in my last degree. I understand what you are saying about about the repetitive process involved in tuning the model and the unexpected caveats (e.g. the model being too specifically trained to the data used and resulting in it being less effective at generalising on unseen data).
For me, however, the interest came in the effects of introducing less obvious features into the mix. Those features that seem at first unrelatable, where others would not think to put them together, and correlation at first not obvious. Seeing an increase in prediction because you thought of using them to aid prediction interested me a lot. A sort of immediate mechanism to validate a connection between something you had a hunch on, even if it couldn't be explained.
I have a similar experience with ML due to lack of any kind of visualisation. What else specialisation should I go for now (that I don't wanna continue with ML & I have already learnt some web application development stuff both front and back end)???? Please help!
Did you like web dev? You can specialize in frontend or backend
What I got from all this is that ML is basically like hammering a puzzle piece until it fits.
We had to train a model at work once, and it was really exciting at first because we were given a few months to learn about Machine leaning. After the fun learning Phase we got to tuning Parameters over and over and it got boring tho, i just made a script that trains the model a few times with every Combination of Parameters possible, let that run for two weeks, and we got a nice model lmao
At the end machine learning is always just Approximation and I dont really like that
In machine learning models can you use any if then statements or does it all have to be only numerical values and equations? How about a mixture of both hard instructions
And number training??
You can re-write if-statements as calculations and vice versa. To what degree your model operates on numerical values depends on the data that is to be learned from.
haven't done ml-related projects before but is it possible to automate the tuning?
Honestly, part of the reason I never got into ML was that I had a hunch that what you say in this video would be the case. Glad to have my suspicions confirmed! Thank you :)
Totally, I always thought it would be like this. But just like he said, I would like to use ML libraries in my project, that I am cool with.
Definitely agree with all of this. I’m a PhD Student focusing on ML applications for biomechanics. I think the coolest part is finding unique applications for your models and integrating them into a software application. You’re definitely right. It’s not for everyone, because it is extremely iterative and maybe even monotonous. But when you understand the math and the specific hyperparameters the iterative process can be expedited. It definitely can be boring at times but I guess it does depend on the person
mind inception: make a machine learning algorithm to tune your other machine learning algorithm
Totaly agree with you. I did a Business Intelligence/ Data Science major during my studies. But I'm not that interested in it anymore, because I find Full Stack Web development much more appealing and fun. At the moment, I'm specializing in using Spring Boot and Angular in combination. It's just so much fun to create applications for real-world problems. Keep up with your nice videos, you just earned a sub!
You beautifully explained and summarized my exact experience with ML. Moved back to building web apps.
Enjoy your shit pay. ML engineer 400K salary reporting in.
@@j22n3s how did you got your first job?
@@j22n3s Who cares about the money when you dislike the job?
"mine's going to be linear regression" you got me there for a second.
İ didnt understand can you explain
@@merhabamerhaba9823 no, I can't lecture you for hours in machine learning so you can understand the joke.
@@SoulReaver jdksnxkdnskcnddn
@@SoulReaver Hahahahaha
I am between the two world's (ML and WebDev) I see your point. In "traditional" Software development you can quite fast build a somehow working solution. In ML you can come up with a baseline model within an hour, but to have something usable (for example to provide as a service) you need this time-consuming iterations. Make them as short as possible and setup some metrics which helps to decide what your next step should be. There you can face lack of tooling or experience. Scrum, GIt, CI Tools and all this stuff is often not primarly developed for this type of work. But I see how different tools get better month by month..
I’m currently a web dev and not looking to switch but interested in the data science side , so I have been studying that.. so far getting Famiar
with python
For me it gets interesting when I’m looking at new _applications_ of machine learning. I agree that the process of building models, in itself, isn’t very interesting
Alright can someone clear this up for me, I used Kaggle and DataCamp to learn machine learning fundamentals. Idk bout kaggle being the best but is datacamp a good way of learning ML concepts?
prod.KashKari I’m using dataquest.io currently. I think they give a brief overview and show you how to implement in code, but they don’t seem to go into the mathematical concepts as deep as needed. I think brilliant.org’s ML course it very well done and teaches the mathematical concepts needed.
Phil r u doing supervised or unsupervised machine learning
prod.KashKari it teach basic regression, decision trees, naive bayes, k means clustering, svms, and neural networks.
Try online course by caltech "Learning from data".
I have tried learning ML. Totally agree with you. I would create something visual, that could be used right away, rather than creating model, adjust, wait wait, test, adjust, wait, test, adjust, wait. It's like a lot of waiting for something that you are guessing on, rather than creating real deals :/
I think a lot of people would prefer to use an existing library in their application. The problem is, though, that there isn't always a good pre-trained model for your use case
thats is true. Also additional things to add from me as a working data scientist:
in kaggle its all about accuracy, accuracy, and accuracy. but in real world most of the time it doesn't matter. What matter is make a model that is good enough that you can explain to your non technical client or manager. And most of the time you will just use logistic regression or similar linear model because it is pretty much the only model that non technical people can understand. It is also the least black-box model when you can easily explain your parameters to non-technical people. And most of the time, you don't even need machine learning, just standard statistical tests are enough. And yes, 80% of your work is data cleaning. And you maybe think "ah thats because you dont work for cool technology like image recognition AI robotic etc". That maybe true, but unless you are really smart and has phd and released paper or working for FANG, you will also just use models made by other people, and your work will also just tuning the model. Maybe you will just connecting API made by Google, Apple n others, because they provide APIs for image recognition etc.
I'm not saying data science job sucks, but it is too glorified.I still like my job and in my opinion it is still better compares to my previous engineering job. But if you think you will make all the cool shits all the time, you are wrong. Unless you work in R&D department in FANG company I guess (but tell me, how many percent of the world's data science jobs are like that? Its very very small proportions, most data science jobs in real world are for banks, consulting firm, insurance, e-commerce, etc).
It is pretty much like any other jobs, where 80% of your jobs are boring tedious routine.
I reivew lots of CVs that come to the company where I work. Some of those are from fresh grads. I see it all the time that there are candidates with interesting universtiy projects around system programming or in distributed systems, but they feel almost obligated say something about ML & AI in their CV. When you talk to them turns out they dont fully understand what it is or just say that don't enjoy ML. I usualy appreciate honesty. It reminds me how 5 years ago we had the same conversation about "big data".
I think it is ok to finaly say it. I dont enjoy working on AI :) and let someone, who knows what they are doing, do it.
Which company? I'm job-hunting...
You basically read my mind. I've been working on an ML related project for a while now with my team and just like you I've been more interested in using the ML libraries/models in my applications rather than implementing the models myself. I just found it really boring and kinda like grunt work. Great video though.
I think one of the hardest things I had to learn about ML was that ML won't always give you the best solution... especially with very classical kinds of data like time series. With time series, I always had better success with Exponential Smoothing or Season Autoregressive Integrated Moving Average models than with something like a Long Short-Term Memory/Recurrent Neural Network model even if I spent a ton of time tuning hyperparameters, dropout rates inside the cells, etc. Couldn't get it anywhere close to the classic methods in terms of lowering the average error rate of guessing the next step. In real life, it's rare that you have enough data for the model to actually get better results than those other two methods. There are a few free papers online comparing ML to classical methods for time series data if anyone reading this is interested. Definitely consider other options! Just because the phrases "Machine Learning" and "Neural Network" get the higher-ups excited when they hear it doesn't mean it's the only choice you have!
Coming from a Java background and programming Enterprise systems, I completely feel you
interesting, i did learn a lot about ML, I am 53, true, it feels better building something, plus not many applications yet and feel in the future their will be graphical tools and much easier to use software
Im a web developer and i feel similarly. I love thinking of the model to use and how to make novel concepts work in ML but i am not sure i will enjoy the loop as much. I dont mind cleaning up data but waiting for the models results just so i can tweak it slightly seems pretty annoying.
I will give an ML job a try though since a lot of new tools are being developed so its possible a lot of the boring stuff will be automated away
Really helpful summary, thanks
When you don’t take linear algebra and can’t figure out back prop:
Back prop? Is that a short for something?
@@LanPodder backpropagation
Didn't take linear algebra but I understand the gist. Basically go back and redo the training with the successful dataset and adjusted bias.
@@nazarm6215 strongly recommend you 3blue1brown's videos on deep learning, they'll give you a good intuition of what neural networks and backprop are.
You don't really need linear algebra for backprop
You need good calculus
Also most frameworks don't require you to compute backpropagation
newbie question: what do you mean by "train that model"? is it like run some executable in the terminal or ide or something?
It could be a simple executable. However in Python, most of the time, you are fiddling interactively in an interpreter (usually Jupyter Notebook to be able to plot graphics).
At the end of the day, It is an instruction of your program.
For example in pseudo-code:
// assumes a function CreateModel() that does the heavy-lifting of initializing the program that will "learn", called a "model".
// It could be a perceptron, a support vector machine, a random forest, a neural network, etc.
model = CreateModel()
model.train(train_dataset) // this is the "train that model" part. takes a few minutes/hours/days depending of the model & the size of the train dataset
results = model.test(test_dataset)
Finally, you analyze the results. If the trained model is good enough, maybe save it in a file to be able to reuse it later.
I hope you can get the gist of it from that explanation!
@@NaviSly Very well explained! Now I get it. Thank you
I'm a first year uni student who was bumped up into some third year classes that "don't have big prerequisites" (I feel like that's a red flag for subpar stuff..). One of those classes is ML. It has been interesting learning how this stuff worked (from the surface level sklearn) but at least from my personal experience I still don't really understand much about how it actually works mathematically, and this surface level stuff I could have learned on my own. Literally all I needed was the name of the book we were working from, and some structure in going through it for practicing (e.g. labs n Kaggle and tests). I came to uni to learn low-level stuff so this frustrates me. This year seems to be 100% surface level bs - one of my 200 level classes is literally just teaching yourself how to use two software. I'm now needing to reign in my hopes of what uni will be over the next 3 years... On the bright side of ML, we do have a great professor :)
I think front-end development will be replaced by ML models that generate front-end code. What do you think?
Your reasons sound convinving however I'm terrified that regular software development WITHOUT any kind of machine learning or AI will drop in popularity dramatically. Just like people don't really make static websites anymore like they used to, I feel like people aren't going to want "dumb" crud apps that can't take meaningful action independently.
I would think that would largely depend on what's needed in the market. If AI becomes good enough that it can write software itself then yeah sure, but until devs will always be needed to build complex web apps, mobile apps, games, server side stuff, etc.
I see ML being incorporated with more apps, but there will be tools that make it easy to integrate without every software developer needed to develop his own model
@@bawad I agree with that too. There's already a good amount of that
Don't worry, there will be plenty of jobs for us non-ML devs. It will over time become a regular implementation. Just like you don't really know need to know anything about complex hashing algorithms to be able to securely hash data
Agree w @Pasquale below.
And read up on 'the Hype Cycle' - ML will drop in buzz soon enough. It's a very promising tech - for some use cases. Not a solution to everything. As Ben states the time (thus cost) to ramp it up to being 'accurate enough' can be huge - won't be viable in many situations.
I’m totally agree with you. I prefer to use the models someone else did to put it in an application 😊
lol, why you not say I'm prefer using the app someelse create instead of creating my own (suppose you are a mobile developer)
@@tuananhdo1870 That's an awesome point of view. And this can be responded easily: in a team, I prefer to create the UI, meanwhile (in that team) there is someone who prefer to construct the Backend, another the Architecture, another the ML Models, etc, etc, etc.
Lazy and proud.
@@Newtube_Channel You sure, I'm proud to have a team and to be happy with that. I can notice you're proud to be you and your comments, that's great 😊👍
Great video. Good insight
well one major thing is sentiment analysis seems boring to me...maybe i am wrong but i myself did in an intern for a month and got really bored. The thing is that there is not much to visualize/analyze(or maybe i was bad at it) but image recognition, recommender systems are one of the most fun stuff i have ever done, so it think it all comes to the kind of project u r doing
I have been coding for about 6 years now. So, why am I subscribed to you?
First of all, I still learn things from you but most importantly I mostly agree with you on these things and I know most of my colleagues do too. Thus I shared your channel with them.
What is a site name to learn python which Ben mentioned? It's hard for me to understand when he pronounce it
i think he said "kaggle"
@@hazimvl1226 , yep indeed. Many thanx)
I'm currently working on a project that requires img recognition to recognise subtle difference between colours in real life (i.e. light blue and very blue). I"ve clicked >>20 k images as part of the data preparation..
When I attended an AI meetup (my first, and so far last), I was quite surprised how the programmers talked about the code: "Hopefully it is picking it [in this case a sound frequency] up now." Let's see if that will yield any results." Things like that.
Only 2 days before that I attended a "functional programming in javascript" meetup, where each and everyone was so happy about the core message of FN (as it were): " you get **exactly** what you want, and you get it always!"
What a contradiction that was - "maybe it works" vs "it will always work as expected" - - - - at least emotionally...
Exactly, and if you're lucky, and everything turns perfect, you can expect something like 65-70% correctness, depending on the problem. I won't trust my life to something that vague.
Imagine if your car's brakes worked only 80% of the time. Not good.
I'm someone who switched from learning ML to Full-stack development. One of the (several) reasons for that was, there was no real sense of achievement, or completion, or CERTAINTY. Like something you did in ML could always be improved upon, unlike normal coding. Like, there was no end to solving a problem, no concrete finishing line, and reward (I'm talking of feeling a sense of reward).
I realized that I was not really feeling fulfilled and that the reason I got into it was simply because of the hype, and the lure of stable future.
I mean you can program a learning system with a language of any paradigm...
@@DoubleM55
Brakes worked much worse before they were optimized. And you wouldn't want to get into the plane with a pilot, who hasn't yet learned how to fly.
@@PRATIK1900 I mean... there are plenty of programming problems that are computationally intractable and require greedy solutions, all of which can in principle be improved on.
Feel the same like you, it takes too long to train models over and over again, and as I am not very familiar with the models , I have to choose switch between models again and again
I'd say im a fan for now..been doing this for a couple of years now.. mainly for ecosystem processes forecasting
Can genetic algo help in boring parts
You just forgot the most important and the interesting part i.e Feature Engineering where we analyse the data and brainstorm over data to build new features which could benefit the ML model.
They also say that almost 70-80% of your time should be devoted into cleaning data, understanding data, and feature engineering and only 10-20% of your time into selecting and tuning your model.
I guess what turned you off was the parameter tweaking and and time taken for any particular algorithm which is mostly done by Kagglers just to get an edge at the end of competition.
Yes! ML isn't just about coding and much more about data/stats/intuition and even the code is just a means to an end that is to use that derived meaning in what often happens to be another area.
Stanford NLP is ancient. You should get back into ML and look at more modern methods. The way the math works behind it is absolutely mind blowing. The interesting part is making the model scalable and deploying via cloud.
There's a lot of stuff I would like to do, like Machine learning, IOT, AI, Game development, Systems development, but I'm struggling with Web development and that's what I find the easiest.
Dude, i've been watching your videos i realized that there are so many things that you don't like man.
its good he actually has opinions on things
lol .
@@captainwalter that's exactly what I wanted to say
He just uses the hater personality as his YT identity to seem more interesting. Similar to Tech Lead.
Sentiment Analysis. It's interesting you don't like ML, as I am the opposite and feel like web development is something that I will probably struggle with, as I begin my journey.
I really like math and modeling though, and currently lack a lot of software engineering skills.
However, I think you would need to dig a little deeper to appreciate (or gasp at the lack of intelligence of) ML a little more. The algorithms are pretty crazy and it's pretty cool how different concepts of math are brought together.
I feel like I'm the only data scientist who switches from ML to Web Dev lol
Although tuning your Machine Learning model to obtain better metrics is a big part of ML, I think you are missing where the beauty lies with ML/AI. The thing that attracts people to the field of ML is more about leveraging the data you have and being able to extract valuable and meaningful information from it. What I believe many people in the field consider to be the exciting part is not necessarily limited to the points you discussed, but rather using their deep understanding of the math and algorithms behind many machine learning techniques and understanding which algorithms to use for the particular data and use case. This process of analyzing what features from a dataset and what mathematical techniques, such as distance metric optimization, feature reduction, etc to apply to your data to get better result with the paired algorithm being used is what attracts people to ML. Personally, my favorite part about ML is the process of model selection, applying my knowledge of different ML algorithms paired with analyzing the meaning behind the data at hand to create the best model I can. The analogy that comes to mind is, this is like me saying “I dislike web dev because all I do is make buttons look nice, some pages, and some API endpoints.”. However, the people that love web dev can understand the complexity that goes into it and have an appreciation for it. Just like ML, if you gain a deeper understanding, you will gain a deep appreciation for it.
What is the name of the website......cargill
Did you try reinforcement learning? I think it is funnier than other ml topics :)
I also don’t really enjoy machine learning, although for different reasons than you said. One of the things I do enjoy learning is the math, as I just enjoy math in general although a lot of the math is on the more applied side rather than pure mathematics side, but I still enjoy it.. Particularly I don’t really understand why we are training machines to many times mimic human behavior as human behavior is very often irrational or illogical. It begs the question, why are we so focused on making these logical machines illogical? And yes, I have seen applications that are useful. I’m not saying it’s not useful, it’s just not something that greatly interests me or seems like it is headed in the right direction. Just some thoughts.
It's definitely more fun if you have a strong maths background. It gives you the tools to approach a machime learning problem almost philosophically. While from a coding perspective there's really not much to it..
The only way for a beginners to get interested in Machine Learning is to never give up and deep dive on to understanding the concept.
Machine Learning is a different way of programming from traditional way of what we are accustomed with, once you are familiar with the concept you are ready to rock and roll everyday...
A piece of advice for engineer to begin their journey in ML by building some of the neural network from scratch or debug the tensorflow/pytorch library you will get lot of idea about the algorithms and the feedback loop will be faster while learning.
Bruh, the Namescheap shirt. So silky soft. Got it from the LA Hacks hackathon back in 2013 and it was my favorite shirt for a long while.
Totally agree. Took a course on linear algebra in college and it bored me to no end. Maybe I just don't like math/statistics. I'll just stick to app dev, cause it's fun to me, and I can do it for hours on end.
I feel like non-ML/DS software engineering will still continue to exist for a long time, so there's nothing to worry about. I see some software engineers rushing to learn ML/DS because it's the new hot tech, despite them not even enjoying the subject matter. That seems like a good way to be miserable at your job. For what? Clout?
Additionally, ML/DS will eventually get super high-level and libraries like tensorflow will get refined to a point where it's basically plug and play with a low learning curve. Hell, it may already be close to that state, I haven't looked. At that point, the remainder of ML/DS jobs will probably be for PhDs that do research into developing new algos.
I'm glad there are more sub-categories of software engineering popping up, whether it be an intersection b/t CS and Stats (ML) , CS and Finance (algo trading), CS and Physics (game engine development), etc. That just gives us more options to specialize on what interests us the most.
I see your point, I used to be a Data Analyst in a Social Listening company (Audience Analysis, Sentiment Analysis and Topic Categorization). At first I did repeat a process similar to yours. Scrap data from Social Networks, then boolean search for a topic, pay some indian dudes on amazon turk to get a somewhat decent dataset (labeled), clean it up and start trying out models. At first it was thrilling (I had to come up with the process). Then it became real hard for me to show up at work, it was boring AF. Truth is ML is not that challenging, just repetitive. You can set up your whole suite in a couple months and then just plug an play with data, fix your visualizations and integrations and your done. Level up to data streams realtime (or cloud based ML) and that's it. From time to time you try out new libraries or tweak your algorithms but that's it. Clients were contempt with the product, then my served accounts multiplied tenfold, several google marketing departments (B2B, Waymo, Small Business and several others). Then boredom multiplied tenfold. In all honesty, scrapping was the funniest part of this whole process, imagine.
BTW fuck Cambridge Analytica, they forced the Zuck to make life so much harder for humble data farmers like myself. Luckily I can still access mobile.facebook.com/
The algo never picked this video up since the like/dislike video is so high 😅
Thanks for the feedback off your work
Same here, for me creating or architecting software is much more fun. I do like to include machine learning models if it fits in the product but building machine learning models just doesn't excite me.
Anyone got a TLDR, or a time stamp ?
At the end of the day (5? 10 years?), you should be able to build a web app by simply invoking Alexa, Siri or one of their distant cousins. We will get to a stage where the most typical use cases can be addressed through software that uses ML.
ML, by itself, will also benefit from ML. If you pick Salesforce Einstein as an example, Einstein decides the models, applies them and surfaces intelligence. Well, in theory anyway - but eventually that should give us a taste of what will come ahead.
None of that will sound the end for developers, who will find some other interesting (& hopefully money-making) things to do. But, I can take a safe bet that ML will be an integral part of that story.
yeah we are entering a world full of ML