Reality behind data science, machine learning jobs
ฝัง
- เผยแพร่เมื่อ 15 ก.พ. 2022
- Reality behind data science jobs. Is machine learning really cool?
Do you want to learn technology from me? Check codebasics.io/ for my affordable video courses.
🌎 My Website For Video Courses: codebasics.io/?...
Need help building software or data analytics and AI solutions? My company www.atliq.com/ can help. Click on the Contact button on that website.
🎥 Codebasics Hindi channel: / @codebasicshindi
#️⃣ Social Media #️⃣
🔗 Discord: / discord
📸 Dhaval's Personal Instagram: / dhavalsays
📸 Instagram: / codebasicshub
🔊 Facebook: / codebasicshub
📱 Twitter: / codebasicshub
📝 Linkedin (Personal): / dhavalsays
📝 Linkedin (Codebasics): / codebasics
🔗 Patreon: www.patreon.com/codebasics?fa...
❗❗ DISCLAIMER: All opinions expressed in this video are of my own and not that of my employers'.
Do you want to learn technology from me? Here are my affordable video courses: codebasics.io/?
Both concerns are necessary the process as well as the outcome. Love the process but also be concerned for what you are doing all these dirty things!
iwant to do diploma in machine learning
" Love the process , not the final outcome "
Fantastic line ❣️ .
Ye to Bhagwat Geeta me bhi hai
Karmanyevadhikaraste shloka
@@TheHariPutraOfficially toh kya me karu?
@@RakeshSingh-qd5ih mc
Tere baghwan k bare me kuch nh bola usne bosdk
@@mdmd-un2gd gaand de de bhai 😆🤣
@powerfulpower9 ye yajur ved mai bhi hai jaha se Buddha ne Buddhism banaya
Improve systems, not goals
Atomic habits by James Clear
U have to quote a book to emphasize on an age old philosophy quote on “love the process but not the end goal”
Apparently someone is not loving the process
@@jingyitay6179 bro your system decide your future if you are grinding now you will live a butter like life in future but if you living in your comfort zone right bow you will grind your while fucking life so improve system and you get result damn sexy results and most important Be patience
He said it very clear
@@jingyitay6179 ur too quite too judge bro... Love process doesn't necessarily mean improve system... U can be loving an inefficient process...so what he said is entirely different than the age old love process not goal quote
I am about to write the name of book but saw this comment.
A favorite quote I've heard about data: "The data are always dirtier than they say it is and usually dirtier than they think it is."
What you want to say? Please say it clear
Dirty work as in "manual data cleaning" is definitely a part of DS work, however, it looks good in its initial state. for example, when you're training with 500-1k data points and making your first working model then yea, at that time dirty work is justified but when you reach >20k data points(a rough estimate ) then "manual verification/cleaning" simply shows the immaturity of leadership .... that's why every ML project needs a small explainability setup ready along with original model development.
That just means the company should've tasked the data engineer with data cleaning. But ofc, nowadays they try to make data scientist do the whole thing aka data engineer, data scientist, machine learning engineer lol
@@johnwig285 Yes, a data engineer should be cleaning up the data. But in my experience:
1. Many “data” engineers are “process” engineers with little understanding of the data. I.e., if their cleanup process runs, then the data is good.
2. Many senior managers and data customers think that the data presented to them is in roughly the same for as the source data. I.e., they don’t realize that many datasets are the combination of multiple source sub-datasets. All of which need to be clean.
3. Data engineers need to understand the business meaning of the data that they are scrubbing. Many companies have different businesses with different categories of customers THAT DO NOT MIX. Understanding those categories, identifying the parameters of each category, and accurately categorizing each customer into the correct category is CRITICAL. Without understanding the business meaning of their data, how is a data engineer going to clean the data?
4. In the real world outside of a lab, all data needs to be scrubbed and not just a sample. And this data needs to be live data, not manually created data for modeling. To be sure, some data elements need to be secured for privacy. But the semantics canNOT be corrupted. Uniqueness,cardinality, nullability, domain, etc. must be maintained.
In one project, the compass position of a customer was a critical identifier. Unfortunately a database engineer determined that NULLS would not be allowed in the database. I.e., it was a “dirty” condition. The rule was that all NULLS must be converted to a 0. So they bounced a a dataset that I had scrubbed back to me, screaming
“COVERT NULLS TO ZERO”!
I fought that but lost. It turns out that the data was poor, and customer compass position was NULL for something like 90% of customers. After replacing all those NULLS with zero, the database engineers were happy and my dataset was deemed “good”. That until the end users saw the data. The went ballistic saying, “how can 90% of customers be True North?!?” (0 degrees on the compass).
So instead of identifying an issue that could be addressed, the data was corrupted because there was no way of separating the plugged 0’s from the rows where the customer WAS due North.
This is a true story, not a fictional story to make a point. Unfortunately it’s an example of what the real world is like. Arrogance, incompetence, politics, and ignorance is out there. Technical competence is not good enough. A good data engineer has to know a lot more
That's Bhagavad Gita, very well said sir, Thank you !!
Yes. And people says wow, not knowing this is from bhagvadgita
Where it is written like that
It is written
@@not_your_chiruकर्मण्येवाधिकारस्ते मा फलेषु कदाचन्। मा कर्मफलहेतुर्भूर्मा ते संगोस्त्वकर्मणि।। BG chapter 2 verse 47
@@rolexsirrgreat job yar❤.
Hindu religion need people like you.
true hare krishna
Love the process, not the final outcome. - S' Dhaval
Words of wisdom, Sir. You're right... Sometimes, it's tempting to compare my progress to other people and think of them as light-years behind my abilities. But thinking about SUCCESS is not gonna produce SUCCESS - it's the hard work, tears, and pain that no one sees. This video clip made me shed a tear.
Thanks for your encouragement!!! I'll keep going. :)
In civil Engineering even u love the process of work ur not satisfied with output .ur not paid wat u deserve..
@@mr.aauspicious9229 yes.
Its not s dhaval but shree krishna the first one for this qoute
Very true
U need to explore and find ur interest
Trust me finding your interest and what u love can take lot of time but don't get anxious or worried because once you find it you will thank your past self
"Love the process"...
That's the key to happiness ❤❤
Argh! My life everyday…frustrating but eventually you love the chaos.
Hey bro
bro u ai engineer?
@Engineering Study Zone hi, I'm 10 th pass student and want to be data scientist, can u pls tell me the roadmap of it because I'm really very confused!!
@ITV_sportsshorts9101 I am 12th passed out student. Now about to join in B tech ai & ds course.. is it good ? Will it have scope or after pursuing this degree can we get a job in IT company ? Pls reply me 😢😢😢 I am very confused..
@Scripted_TV9101bro i m confused between data science and cyber security also can u suggest which should i choose ( i m average in maths not soo good not soo bad just normal )
So true. Recently finished a MS in Statistics, and there is no way a person could complete that program and work in this field unless they were BOTH a) smart and b) willing to work very hard at very difficult and often tedious things. There will never be a surplus of true data scientists because there aren't actually very many people like this.
Thank you for saying this!
कर्मण्येवाधिकारस्ते मा फलेषु कदाचन ।
मा कर्मफलहेतुर्भुर्मा ते संगोऽस्त्वकर्मणि ॥
अर्थ: तेरा कर्म करने में ही अधिकार है, उसके फलों में कभी नहीं। इसलिए तू कर्मों के फल हेतु मत हो तथा तेरी कर्म न करने में भी आसक्ति न हो.
😂
Hare krshn❤
Yeh sab doglapan hai., 😂
Can you explain more about dirty work, what is dirty data cleaning works means
Yeah exactly. I didn't understand what he said.
I think he wanted to tell the audience about complex data cleaning process
Generally people think that a data scientist would only build ML/DL models but in reality a lot of effort goes in data preparation,cleaning,transformation,treatment etc and this is sometimes called as "dirty work", not exactly dirtly but not considered as shiny as developing a model but it is a very critical step :)
He meant boring work, like for a backend developer writing documentation, test cases, debugging are boring work while others thinks they are writing code all day..
he means cleaning the data and transforming it into something that your ML models can actually use.
Love the process,,, Absolutely! After 30 years as a Data Engineer/Scientist/Analyst/whatever, I found that all data is “bad”. I.e dirty, incorrect, wrong cardinality, inadequate, limited, and/or not in a format that is useful.
The biggest problem is that virtually all consumers of data do not understand how truly horrible the source data is. And how difficult it is to cleanse and manipulate the data to into a format that is useful. (That includes most other technical people too.)
Most people thought that I was a software developer because I programmed in something like 8-9 procedural and set-based languages. But, when asked, I told them that I was a “data guy”. Most did not understand what I meant. They only saw the outside of what I did and not that all the tools and techniques were the means to get good data and information.
The final result was maybe 5% of the effort. The grunt work was 95%. But that was OK to me. It was satisfying grunt work.
Dan
30 years? As a Data professional? Whoa
Explained very well! Thank you.
Exactly what I needed to hear !
Can you explain what do you actually mean it's like people talk good things about ds but in reality it's something else
@@ankitlakshya450its been 2 yrs after your comment..did you pursued ds ?
Just wanted to know
@@anamikaaami3175 ever evolving field there are jobs no doubt every year syllabus keeps increasing so if you are good with maths go ahead
The secret of sucess is ... Dirty work
Thank You So Much for Motivate Me Man 😌😌😌🙏🙏
Nice one sir!Really inspiring
Enjoy the journey rather than wait for destination, this has been motto of my life.
It's really a great line !
Thank you sir! Behind the reality of Data scientist work..❤️❤️
Sir, Aap bhand the kya thode ?
Reallity is.. Hark work + interest= Success🏆💪🏆💪
The biggest issue I've found at every level is nepotism.
that doesn't work in data science
@Fabiya gaming Yes it does, I've seen people skip Python and sql coding tests at big companies. All so a mate of a senior engineer, analyst, or scientist can get in easily. How else do you think undergraduates who can barely write hello world or a select statement get senior jobs often requiring high level industry specific software skills.
Change your mindset: LOVE THE PROCESS
Every software developer does go through the same process . It's not really a difficult job once you get used to it . Not a tough grind.
coming from someone whose probably never worked in the role a day in his life
excellent, love the process,
Love the process!!
The market is lacking any junior roles. It is impossible to get a job if you didnt secure one beforehands..
Sometimes you’re only provided with the dataset but not the data dictionary. Will be difficult to understand each columns specially those which contains abbreviations. You won’t always get the meaning of each columns just by googling or trying to get more information from the source of the dataset. Sometimes there won’t be anyone to guide and you’ve to assume certain statements.
I am beginner in data science journey, thank you for giving advise. I hope, I remember this all time untill I place.
Bro is there really a very difficult maths in DS? beacause I dont like Differentiation and Integration aur simplification but I like somewhat Statistics and calculations.
Great motivation 😉
U r correct. I dont understand why people creating their life soo.. Complicated.
Well said. It's not always sunshine and rainbows not matter what job is it
I love getting tensed
what i needed to hear!!!
thanks sir... i've joined the journey of data science..i'm a bit afraid but i'll make it
will you please guide me about it!!! How you started your journey commerce or science please explain me
@@hinnashah9675 I have computer studies as background since I did BCA and Msc.Information systems. But I don't wanna impress you with my cv😅😅 , I guess, no matter Your background You can still make it to data science just start watching sir's tutos on python, then You watch panda's tuto and You can start machine learning tutos at the end . good luck
@@arnolddumba7949 thank you so muchh
@@saiprasadganji codebasics channel is the only online platform i'm learning data science from
Bhai apki placement lagi?
Rightly said , data cleaning , brainstorm and understanding the business problems are the things to be handled. 😊
I like the idea of dirty working as in manual intervention using the brain cells
Very true and genuine advice!!
Well said brother man
reading all the comments and don't understand, what was so amazing he said!
I mean yes, a very good quote was pasted in the last.
But working as a Data Scientist or Analyst you have to deal with the data, that is your fukin job, that's what you do, and that's why you choose Data Scientist, you have to deal with the process and I believe that Data Scientist must know and love their work
and no job is fully satisfying, you are not getting satisfaction all the time.
Please guid us regarding data analyst job. Because now a days it become so much difficult to get job,not much vacancies in data analyst for a person doing transition,trying from 4 months
Depends on whether you consider "Maths" as dirty work..
Great looks like I'm gonna have to SPAND time on learning ds in order to get a JOBA...
LOVE THE PROCESS NOT THE FINAL OUTCOME
Many describe this as the 80/20 rule. Data scientists spend only 20 percent of their time on building models and the other 80 percent gathering, analysing, cleaning, and reorganising data. Dirty data is the most time-consuming aspect of the typical data scientist’s work.
As someone who doesn't have a background in maths and statistics I can only imagine how many hurdles I have to over come
"Love the process, not the final outcome" The group of words I needed to continue. Thank you so much Sir
Trust the process and hold the vision 🧭
Very well explained sir.
Thanks sir for mind opening advice 🙏
Well said bro
Thank You! TRUTH!
Very true data cleaning is the factor
Love the process ❤️❤️
Love a job😢
Thank you ♥️♥️
EXACTLY! ♥️
These Little tips really help for someone trying to switch into data science
I am not a computer science or data science student so I am not able to understand what does it mean bt dirty data and dirty things , can somebody explain???
th-cam.com/video/Uf_FlOOjOpA/w-d-xo.html
Best advice ❤️
Most of the work I've done in Machine Learning has been collecting and recollecting data
After hustling for 4 years, now I exactly understand what he is saying.
Hello sir can I choose data science course after 12th PCB?
Thanks Alot 👍
Well said
Perfect
Love the process, not the final outcome
"Love the process not the final outcome" Bravo...
Hello, I want to apply for Junior NLP Developer position. Could you please advice me projects for my portfolio?
😄😄finally it’s always a dirty work when it comes to IT jobs
Thanks for your opinion or explanation 😊
Well Said
I am bad at maths ,programming and thereby coding.
and my astrologer says You're compatible in technical 9-5 jobs Only
meanwhile i love creating 3D art , modelling, Concepting
Love the line ❤️❤️
'Love the process' - the best advice for youngsters who want to earn big and see results instantly. When you enjoy the process, the results inevitably turn out great and passion shows.
True...
Very useful piece of advice
Glad it was helpful!
Yes the floor is made of floor...
You always motivate me sir ❤
You solve my problem
You have to tell me most important thing its need me thank sir
Sir ... Did You live in Vyalikaval , Bangalore for a time ? Bicycle Days ?
Excellent 👌
Thanks for the reminder to Love the Process Not the Outcome.
I like how u mentioned at the end love the process not the final outcome ❤❤❤
Without working,is it possible to achieve your goals ?
Ofcourse NO
Well I might sound kind of off the topic but I have literally experienced the generation gap growing/changing incredibly fast
in my master thesis in sociology I built a multi level regression model/ random coefficient regression model. 99% of the coding time in R was about data wrangling 😂 the regression model itself took me 1 day.
building the model is cool, but recoding variables and so on is really boring and annoying. but I listened to techno background music😀
Very amazing and informative video
Plz share the link from where you bought a shirt
In process, dirty work (data cleaning, math, statistics) and brain storming.
what's the meaning of dirty work ,its all about cleaning the data ?
Sir, could you please suggest me the courses available.
I have done the bsc in statistics in 2018, and currently m working in textile industry. Now I am getting confuse that what course should have done. I m thinking that even I have capabilities to do something better but not giving my best, just doing 9-6 jon. I am not dissatisfied with my work.
How can I get rid out of it.
DSA round is necessary for data scientist in FAANG or other product based companies?
Thanks for the explanation. Now this field is out of my goal.
I hope you understand sarcasm and the real message behind this video. Data industry is growing fast and data scientist is a great career role promising and amazing future. I wanted to highlight that it requires hard work and the attitude of loving a process rather than attaching to a final outcome or too much focus on building the fancy ML model!
*This is the ultimate success rule that - Love the process, process, process not the final outcome,apne sir mera mind change kar diya*
👍😊
No work is dirty! Thats the secret of success
YESS
Sir,I liked your coaching very much which made me craze about Data science I need to change my career as Data scientist kindly guide me to achieve it.
You are right sir
I like videos sir. Thanks for helping me through your videos.