- 13
- 76 270
skill bits
United States
เข้าร่วมเมื่อ 21 เม.ย. 2020
skill bits is a channel dedicated to innovations, programming and essentials ideas. Our videos are or will be among others about machine learning, data science and IT basics. We want to share our knowledge and interests for your joy and growth.
channel's icon is from www.flaticon.com/free-icon/duck_789432?term=duck&page=1&position=28&page=1&position=28&related_id=789432&origin=search
channel's icon is from www.flaticon.com/free-icon/duck_789432?term=duck&page=1&position=28&page=1&position=28&related_id=789432&origin=search
You have to learn python data processing NOW! | rabbitmq grafana twitter
Hi! Let's create data engineering pipeline using data from Twitter API. We'll use Grafana to visualise them and RabbitMQ for sending. All necessary data processing is done with Python. Let us know if you would like to have more such videos in the future.
Twitter Tutorial Part 1 - register: th-cam.com/video/nYcuZDItwa0/w-d-xo.html
GitHub repo: github.com/Skill-Bits/realtime-twitter-analysis
Grafana dashboard tutorial: th-cam.com/video/4qpI4T6_bUw/w-d-xo.html
InfluxDb docs: influxdb-python.readthedocs.io/en/latest/examples.html
Chapters:
00:00:00 Intro
00:00:45 Requirements
00:01:21 Architecture
00:01:46 Repo cloning
00:02:07 Preparation
00:02:55 Running docker
00:03:42 Rabbitmq - dashboard
00:04:11 Grafana demo
00:05:15 How the pipeline works
00:06:19 Grafana Dashboard edit
00:07:35 Grafana - adding widgets
00:09:39 Python Script
00:11:53 Python - Analysis
00:13:01 Python - Persistence
Twitter Tutorial Part 1 - register: th-cam.com/video/nYcuZDItwa0/w-d-xo.html
GitHub repo: github.com/Skill-Bits/realtime-twitter-analysis
Grafana dashboard tutorial: th-cam.com/video/4qpI4T6_bUw/w-d-xo.html
InfluxDb docs: influxdb-python.readthedocs.io/en/latest/examples.html
Chapters:
00:00:00 Intro
00:00:45 Requirements
00:01:21 Architecture
00:01:46 Repo cloning
00:02:07 Preparation
00:02:55 Running docker
00:03:42 Rabbitmq - dashboard
00:04:11 Grafana demo
00:05:15 How the pipeline works
00:06:19 Grafana Dashboard edit
00:07:35 Grafana - adding widgets
00:09:39 Python Script
00:11:53 Python - Analysis
00:13:01 Python - Persistence
มุมมอง: 1 929
วีดีโอ
How to install Python library from GITHUB with PIP
มุมมอง 8K3 ปีที่แล้ว
In this video you will get to know how to easily install with python pip command libraries from github. Two options are shown, first that allows you to install the latest changes in the library and the second that installs chosen tag or commit from the history. Timestamps: 1:45 install with pip latest commit from a library 3:09 install with pip chosen tag or commit from a library
How to connect to Twitter API in 2021 | #3 SAVE DATA
มุมมอง 3.3K3 ปีที่แล้ว
In this video you'll learn how to easily get and save data from Twitter API to csv. CODE: github.com/Skill-Bits/twitter-api SUBSCRIBE to see our development and new materials 🐦 link to part2 with getting data: th-cam.com/video/6HJ1MLbZPTw/w-d-xo.html timestamps 🐦: 0:20 get python code and prepare env 2:00 dotenv library and authentication explained 2:30 get and filter tweets 3:50 save tweets to...
Host MLflow with Docker in 15 minutes! | Docker beginner
มุมมอง 24K4 ปีที่แล้ว
Lets set up the MLFLow Tracking Server for Machine Learning experts teamin one docker-compose command on your Virtual Machine Links: Docker installation docs.docker.com/engine/install/ubuntu/ Docker-compose installation docs.docker.com/compose/install/ Github MLFlow-docker github.com/Toumash/mlflow-docker Attributions All of the images used in this clip are from unsplash.com Music freesound.org...
Python for Data Analysis *ABSOLUTE BEGINNER*
มุมมอง 1274 ปีที่แล้ว
Do you wish to start your first data analysis project? Here is the introduction of 10 most often used commands which are useful when working with data. Code in Python is provided 🐍 and references to most essential Python libraries. Links to libraries: pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.head....
Lemmatization vs Stemming in NLP
มุมมอง 5K4 ปีที่แล้ว
Quick dive into the topic of lemmatization and stemming in NLP using Python. 🖋️ Useful resources: towardsdatascience.com/all-you-need-to-know-about-text-preprocessing-for-nlp-and-machine-learning-bc1c5765ff67 gdcoder.com/all-you-need-to-know-about-nlp-text-preprocessing/ timestamps 🖋️: 0:11 problem with text complexity 1:01 stemming method 1:27 lemmatization method 1:45 examples in Python Icon ...
Tokenization in Natural Language Processing
มุมมอง 2.2K4 ปีที่แล้ว
This is a very, very short introduction into what tokenization in NLP is; with an example in Python. SUBSCRIBE for more materials on NLP NLTK library installation: www.nltk.org/install.html Icons used in the video are under CC BY 3.0, links to the authors: www.iconfinder.com/webalys music: FreeBackgroundMusic, Alston & Ozone - LSDesigns th-cam.com/video/XR9T0ITF6PQ/w-d-xo.html
Natural Language Processing Explained | 2 minutes
มุมมอง 2674 ปีที่แล้ว
This is a short introduction into natural language processing. It's in a way an encouragement to get more interest into this field of AI. SUBSCRIBE for more materials on NLP timestamps: 0:10 where NLP is used 0:28 what natural language is 0:50 what natural language processing is 1:04 what the aim of NLP is 1:13 basic NLP tasks 1:56 key terms in NLP Icons used in the video are under CC BY 3.0, ...
How to connect to Twitter API 2021 | #2 GET DATA
มุมมอง 25K4 ปีที่แล้ว
In this video you'll learn how to easily access twitter API to get data using Python. CODE: github.com/Skill-Bits/twitter-api SUBSCRIBE to see our development and new materials 🐦 link to part1 with registration: th-cam.com/video/nYcuZDItwa0/w-d-xo.html music: FreeBackgroundMusic, Alston & Ozone - LSDesigns th-cam.com/video/XR9T0ITF6PQ/w-d-xo.html timestamps: 0:30 create app in developer twitter...
How to connect to Twitter API 2021 | #1 REGISTER
มุมมอง 5K4 ปีที่แล้ว
In this video you'll learn how to register to get access to twitter API to collect data of your choice. SUBSCRIBE to see our development and new materials 🐦 music: FreeBackgroundMusic, Alston & Ozone - LSDesigns th-cam.com/video/XR9T0ITF6PQ/w-d-xo.html
Linear Regression Simply Explained | 5 minutes
มุมมอง 4054 ปีที่แล้ว
In this material you'll learn basics of linear regression and how this machine learning algorithm may be applied to your data. Simple example in python using jupyter notebook is also provided. SUBSCRIBE to see our development and new materials 🚀 code: github.com/Skill-Bits/linear-regression/blob/master/linear regression.ipynb music: FreeBackgroundMusic, Alston & Ozone - LSDesigns th-cam.com/vid...
Handle missing data with Python: part 2 | interpolate, fillna
มุมมอง 2024 ปีที่แล้ว
Fixing missing values with interpolation and forward/backward fill SUBSCRIBE ! and see our development and new materials :) code: github.com/Skill-Bits/missing_values/blob/master/missing_data_part2.ipynb dataset: www.kaggle.com/se18m502/bee-hive-metrics?select=temperature_wurzburg.csv music: FreeBackgroundMusic, Alston & Ozone - LSDesigns th-cam.com/video/XR9T0ITF6PQ/w-d-xo.html
Handle missing data with Python: part 1 | dropna, fillna
มุมมอง 5054 ปีที่แล้ว
Missing data? In this tutorial we'll learn how to handle them with Python and Jupyter Notebook. Basic functions to identify missing values and drop or fill them will be presented. SUBSCRIBE to see our tutorials development and new materials :) link to code: github.com/Skill-Bits/missing_values/blob/master/missing_data_part1.ipynb link to dataset: www.kaggle.com/rtatman/animal-bites Timestamps: ...
bro, can you same project but using web app not virtual machine . I have teried bu my mlflow does not show all experiments
Thanks for the interesting content! 😍 Just a small off-topic question: 😅 I only have these words 🤔. (behave today finger ski upon boy assault summer exhaust beauty stereo over). I don't know what they are. What should I do with them? 🤷♀️
What is the command to install the library from Github? The font is so small, I cannot see.
This is awsome, thanks for sharing this video.
Thank you so much for your video! I had issue with Clearbit installation and was not aware of installation procedure using the github-repo.
Many thanks for you work. But I have 2 questions: 1) Can you add script for remove deleted experiments (from mlflow trash) by cron? 2) I try use you actual version but after 3 months usage i have an error: BAD_REQUEST: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'mlflow.runName-a171d72f207b4cb38afb5d918a778c9e' for key 'PRIMARY'") Do you have any idea how to fix this?
Hi, i get an error. Version in "./docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.
That was super helpful. Thank you.
Any plan to update this for mlflow 2.0?
Could you please make a video on how can we change the twitter link into image in JavaScript?
Hey, great video, thanks
How would you be able to send a message to your twitter from c#, or connect different social network apis
Amazing explanation !!!
Awesome
woow this was very helpful thanks for the great video... have just created my developer acc. will meet again wen i get the confirmation.
Can you make a video on how to find and save followers of any user?
Any particular idea how to use this data? We could possible visualize it in a graph or something, with followers of followers. What do you think?
@@skillbits5941 I was thinking of exporting it to CSV and run data analysis on it. This could be pretty interesting if we generate a followers list of a handle and when was each handle created, how many followers they have (just number), how many tweets each follower has made. Then maybe I can take it further and analyze a tweet, who have liked it or tweeted it. This lays the foundation for many other uses. It will be very useful.
Running the script I get the following error: "TypeError: user_timeline() takes 1 positional argument but 2 were given " Any idea what the issue is? did the function parameters change?
I see AWS specific configuration in the env file... This doesn't appear to be compatible with Google Cloud or Azure hosted Docker servers.
Its not AWS. Minio is a drop in replacement for the S3 protocol (storage protocol/api) that AWS/Amazon created, hence the name of the variable. Its compatible with all environments that can run docker. For more infomation about minio and S3 min.io/
please do this on aws by running it on fargate
Theres an in-depth tutorial here: towardsdatascience.com/deploying-a-docker-container-with-ecs-and-fargate-7b0cbc9cd608 Magic of docker is that if you are able to create a container, you can deploy it anywhere you like - whatever environment that supports docker (and networking in this case so the service is available from the internet)
Great video thanks for the concise information.
Excellent. Finally found a simple great video explaining stemming and lemmatization. Thank you
th-cam.com/video/E59UuAXKdeg/w-d-xo.html
Is there a way to enable a password protected login to the mflow web interface?
Yes, you can try the nginx basic auth newbedev.com/how-to-run-authentication-on-a-mlflow-server, but it will depend on your needs
Hi. Thank you very much for such an amazing demonstration. I wanted to know if there is any way to dockerize the python scripts as well so that they can run on startup using the docker-compose file.
Of course you can heres the ready script. I would encourage you to add requirements.txt to the dockerfile to install dependencies from that file
Mmh, the IDE is telling me this, while refusing to execute the twitter-search.py: "Traceback (most recent call last): File "/twitter-api/3-search/twitter-search.py", line 38, in <module> with open('elon_tweets.csv', mode='w', encoding='utf-8', newline='') as csv_file: TypeError: file() takes at most 3 arguments (4 given)
Then probably your IDE is set to parse the python with version 2 instead of 3+ - just as discussed here stackoverflow.com/a/42182014/3711660
This is great! I always like seeing content that is not the same basic tutorial repeated times and times again!
More to come!
I think a big intention of setting up a remote tracking server is to run your code locally (not from the VM), and then be able to view the results in the remote mlflow ui. If you ran your code locally, or had multiple users submitting runs in python on there local machine, can they type the url and view it locally?
Yes, thats definetely the case. You dont get the url in the local run, but when you go to the web interface you can share a link with others ;)
Today we've updated the github repo (it wasnt working because of some breaking changes in minio). Check this out github.com/Toumash/mlflow-docker/commit/b6ecfe7d0c0b8c3d3674b1d5a1a63c4d6f6249df
This is great, thanks! Quick question however. Is there a reason why you don't include the installation of Minio inside your docker file?
Directly because that is a separated system so for that reason minio lives in the docker-compose file (so "docker compose up" starts it up. Not sure if i got your question right :) Surely you can run minio and mlflow on the same container, but its not the way Docker was designed to be used. If the containers are separate, then you can easily update one independently of another one. Here you have an app, database, and storage completely independent, so you can use minio for other projects too
Please help with model serve
Whats the problem there? I would suggest you provide a link to a question of yours created for/on stackoverflow.com
i did not understand the AWS part ? why did we gave AWS ? (just as a dummy ?)
Where do you have any "AWS" part in the video? There are 2 places when you have an option: hosting provider (could be azure, digitalocean, aws, whatever) and s3 bucket creation - you can do it using native aws cli or by using the script just as in the video
How could I do it on an AWS EC2 window instance? and My application doesn't have any parameters. It captures the image every 10 seconds and it also uses a flask to create a server. Please guide me on this.
You should install docker and then create a docker-compose.yml file just like we did with your application. Heres the tutorial with flask docs.docker.com/compose/gettingstarted/
ERROR: Could not find a version that satisfies the requirement python ERROR: No matching distribution found for python WARNING: You are using pip version 20.3.3; however, version 21.1.1 is available. You should consider upgrading via the 'C:\Users\Win\Anaconda3\python.exe -m pip install --upgrade pip' command.
Tried installing but still it shows error. How to solve the problem?
Are you sure you are installing the python-dotenv as in the tutorial, or you dont have python on your computer?
I have a problem on dotenv part. I cannot import dotenv and even install it.. Please I need help :(
Whats the problem there? Did you post some error logs somewhere like stackoverflow.com when we, as a community, could help you?
you fucking know how to make a tutorial ty bro
well played bro ;)
Can you explain for autistic people like me because I didnt even understand what is the first windowe you open please
Do you mean the github website or terminal in PyCharm?
@@skillbits5941 The terminal
late reply but might help other people, Go to "View" -> "Tool Windows" -> "Terminal"
Timestamps: 🐦 00:00:00 Intro 00:00:45 Requirements 00:01:21 Architecture 00:01:46 Repo cloning 00:02:07 Preparation 00:02:55 Running docker 00:03:42 Rabbitmq - dashboard 00:04:11 Grafana demo 00:05:15 How the pipeline works 00:06:19 Grafana Dashboard edit 00:07:35 Grafana - adding widgets 00:09:39 Python Script 00:11:53 Python - Analysis 00:13:01 Python - Persistence
How long it takes to accept the application?
Approximately should take 1-2 weeks
Thanks, this is helpful 👍
In my Twitter app there isn't consumer key, so how to find consumer key ?
Check out this screenshot by us imgur.com/a/PkpocKq
I requested the archive, it arrived, but the problem is that I have an archive of direct messages for two years, from which I only received a small part. What is the solution to get the complete messages and thank you for your cooperation
No idea, but you can try contacting the twitter directly. Let us know if they are happy to cooperate on this one!
Hi! There's a way to get more than 20 statuses?
You can try to increase count value but it may not help as Twitter API search allows getting tweets for approximately past 7 days. If only 20 tweets appeared in the past 7 days then you will get only them. developer.twitter.com/en/docs/twitter-api/v1/tweets/search/overview . You could also delete the line "exclude_replies=True" if you want to have replies as well, then you will have more results.
How can we do this for GCP?
Follow this tutorial to create VM th-cam.com/video/A9Ulrn98DPs/w-d-xo.html then this to open up the ports docs.aws.amazon.com/AWSEC2/latest/UserGuide/authorizing-access-to-an-instance.html then continue with this channel video :)
timestamps 🐦: 0:20 get python code and prepare env 2:00 dotenv library and authentication explained 2:30 get and filter tweets 3:50 save tweets to csv file 5:00 open tweets data in excel
Check out our new video on saving tweets to csv file! 🐦 th-cam.com/video/rfqEJpJtCzw/w-d-xo.html
@Mustafa Dholkawala Did you set up all the environment variables? AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, MLFLOW_S3_ENDPOINT_URL, MLFLOW_TRACKING_URI on the machine you're executing?
Check out our new video on saving tweets to csv file! 🐦 th-cam.com/video/rfqEJpJtCzw/w-d-xo.html
Check out our new video on saving tweets to csv file! 🐦 th-cam.com/video/rfqEJpJtCzw/w-d-xo.html
It is a pity that this tutorial series are not uploading anymore.
Thank you for your interest! We'll be back soon
In case you are still interested, we have just added a new video on saving data from Twitter API to csv file th-cam.com/video/rfqEJpJtCzw/w-d-xo.html
Following up
Hi I am interested in twitter api part 3 please
In case you are still interested, we have just added a new video on saving data from Twitter API to csv file th-cam.com/video/rfqEJpJtCzw/w-d-xo.html