Cheers mate! This really helped a lot, didn't know about virtual environments before this week... My Python setups were pretty much configured in all the ways you should NOT set up Python and in hindsight they were really quite hideous hah. With this knowledge things are miles better now, I managed to use the knowledge gained in this video to overcome an issue that has been causing me so much grief for the last 3 days. Really appreciate the help.
Cool, for someone relatively new to Python's virtual env, this sort of helped. Good to know that the virtualenv, has some advantages over the newer venv. So, it appears that knowing/using both would be good and not just abandoned virtualenv altogether. I appreciate your taking the time to make these videos and I'm sure you want to do the best with them. So, don't take this the wrong way, but, it would be helpful if you could slow down or meter your speech more. I struggled understanding some of your descriptions. I found CC did too.
By the way, you can also use Poetry to manage your environments. It works great and I think is has a relatively fast dependancy manager. And if you want you project to becoma a Pip package, Poetry makes that process extremely easy.
my 2c is that poetry is unnecessary. it doesn't add anything for libraries and it's merely a convenience for applications. it's just as much if not more work to distribute something to pypi with poetry. and maybe it's changed but last time I tried poetry I encountered 5 bugs in the first 5 minutes using it which definitely did not instill confidence
Great video, so helpful to just have all of this laid out in a straightforward manner. Also in general recently discovered your channel and twitch, can't tell you how useful it's been! Two questions I quickly had: 1. Do you have any practices around storing / keeping track of virtual environments? 2. For python versions, does is virtualenv natively ship with all python versions? Or do you need to download each python that you want to create virtual environments with?
good questions! glad you enjoy the content :) (1) on my system I ~usually follow this: - I have one at ~/opt/venv where I store tools I use (more details in these videos: th-cam.com/video/OXmYKh0eTQ8/w-d-xo.html th-cam.com/video/O390_abzo08/w-d-xo.html ) -- I then symlink those tools into ~/bin so they're "globally" available (things like my text editor, pre-commit, and actually virtualenv too!) - then I make virtualenvs per-project at ./venv (sometimes ./venv2 or ./venv39 if I need to have ones versioned based on the python version) (2) virtualenv needs the pythons available _somewhere_ to work. I use deadsnakes (launchpad.net/~deadsnakes/+archive/ubuntu/ppa which I maintain!) to install pythons on ubuntu. on other platforms you'd probably use either the python.org installers or something like pyenv (though pyenv is very difficult to get correct in my opinion)
@anthonywritescode 9:54 It's actually possible to create virtualenvs in other Python versions using Venv only: In windows you type in cmd: python -{version you want to create the virtualenv in} -m vevn {name of your virtualenv} for example you can say: python -3.5 -m -venv -THIS_IS_A_VIRTUALENV_IN_PYTHON35 and it'll create a virtualenv in python 3.5 (you have to add python 3.5 pip to Env Variables before) idk if that works with python 2 also, I don't use it either
I mean, I intentionally did not give a decision but left you to decide based on the tradeoffs -- I use virtualenv myself but there's reasons to use venv as well
I'm actually quite confused. Are you to install a virtualenv for every program you write? What do you do if you want your apache server to run the script? It is not going to be using virtualenv. How do you get one program written in one virtualenv to work with another program needing a different virtualenv? How does docker integrate with it all?
yeah I generally use a separate virtualenv for every project that I set up. I install all the tools I need for that project into the project's virtualenv. As for the apache usecase, at least with mod-wsgi the recommended setup is to use `WSGIPythonHome` or `python-home` setting to point at your virtualenv. you usually don't need to talk across-virtualenvs -- usually you'd install all the tools you need for the project. docker doesn't really play into this, though I still recommend using a virtualenv in docker: th-cam.com/video/O390_abzo08/w-d-xo.html
@@anthonywritescode i was working hard learning python. It was all going well and I was happy thinking I'd be finished by now. Then things went south due to python interdependencies. So I built a new PC, installed a new os. Now always running with virtualbox. Cloning regularly. Still I'm scared of python going bad on me again. So i want everything in virtualenvs. But in the future I suspect I'll need virtualenvs talking to others. Do you have any videos showing how to package them up and talk to each other?
Great videos, thank you! I caught this video a number of years ago and recalled you briefly mentioning conda without elaboration. This week, was watching other YT videos on conda vs pip/venv, caused me to want to find this video of yours again, specifically the part at 3:05 into the video where you briefly mention conda. I was curious not only what your current thoughts were on package management, but if you've warmed to conda... and I was wondering what about conda you did not like back then/now. I could not find a conda specific video of yours but if you discuss somewhere, a link or title to vid would be great. Anyway, this is a casual curious question so response required if your cycles are trim... thanks again, great content!
it doesn't solve problems I have and gets in my way (and is way slower to use than to not). but then again I don't ever install numpy and friends so that probably puts it in perspective
@@anthonywritescode okay... so it has overhead for little gain given your use cases... I've had that happen in other contexts so that makes great sense. I tend to like pip/venv since they're built-in... I've not hit the caveat of venv not being available but largely use standard distributions of Python on Windows/Linux, maybe building it on Linux. To me, pip/venv is a "minimalist" approach that works. I myself don't like to add momentum-reducing layers without justification. 😊 Thank you for clarifying!
Great video! At the end when you created the environment with a particular python version, I am assuming that you had downloaded and installed the particular version first, correct? I.e., virtualenv does not download it and install it when you run the "virtualenv venv39 -p python3.9" command, correct? If so, is there any precautions or steps that I need to take to simply download and install different python versions for future use with virtualenv? If not, I assume that I can simply download and install versions 3.9 and 2.7 with "apt" in my debian bookworm distro, then create their respective environments. But, this sounds too easy, lol, so I feel like my logic is flawed. Any insights are greatly appreciated, thanks Anthony!
@@anthonywritescode, really, okay. I'm going to try your approach then because last night I tried to use pyenv but it didnt work for me. Well, it probably did, but its a little complex so I most likely messed something up. In any case, thanks for the reply!
it's very convenient for installing complicated sciency things, but breaks a lot of expectations for how python usually works -- especially on windows. it's also very very heavy
Excellent! I know it's years later, but this was a very good succinct explanation! I do have some additional questions. If I built a Django project using system python packages instead of in a virtual environment, how do I move my Django project into a virtual environment to isolate it? Is it as simple as copying or moving the Django project directory into the same directory that contains the virtual env folder?
you should be able to make (and activate) a virtualenv anywhere! -- I usually work on code in a ~/workspace/(repo) directory and put my virtualenv for the particular project inside the repository directory
this tutorial became a bit confusing because you are explaining the difference between two different tools that kind of do the same thing, and where the thing that they create is referenced by the same name as one of those tools
i`ve been seeing a lot of videos, could you recommend me what should i choose, a ide like jupyter, or a text editor like vscode? im starting to study seriously data science and this is an important step for me haha
pyenv is completely unrelated, and has wasted thousands of hours of my time. I do not recommend it as it's difficult to build python from source and it does not help you enough to build quality executables
but how do you use conda with 6 different versions of python at once? how does no one mention this? conda can't handle different versions of python! why doesn't it work with this!
Hi. I guess you made a wrong statement at 9:28. You probably wanted to say. The advantage of virtualenv to venv is it works with python 2. It confuses because you started arguing about benefits of venv over virtualenv. Thx anyway for this comparison.👍
??? this is what's at that timestamp in the video: "the other advantage, well we saw the speed advantage before, the other advantage to virtualenv is it works with python 2"
@@anthonywritescode OK. I guess it was my bad. The english grammar of "advantage to virtualenv" sounded wrong to me and let me misunderstood you. English prepositions are confusing me. I would have used "[...] advantage of virtualenv [...]", but I am a non-native speaker. Sorry!
Hi, thanks for this clear video. I must be missing something about venv anyway as I still can't import google.cloud module. To install python3.10, I did: adding sudo add-apt-repository ppa:deadsnakes/ppa sudo apt update sudo apt install python3.10 I installed venv: python3.10 -m venv venv Then in directory containing venv: . venv/bin/activate and once activated , I ran python3.10 -m pip install google.cloud and also python3.10 -m pip install google Installation of google.cloud semt ok Nevertheless when I run the following script ( venv still activated ) #!/usr/bin/python3.10 from google.cloud import bigquery # Construct a BigQuery client object. client = bigquery.Client() I get the following error message: Traceback (most recent call last): File "/home/ced/bigquery-gcloud/./survival_frequency", line 2, in from google.cloud import bigquery ModuleNotFoundError: No module named 'google' Also changed the shebang to #!/usr/bin/env python, but it doesn't solve the issue. Thanks for your help
actually, looking directly in env/lib/python3.10/site-packages/google_cloud-0.34.0.dist-info/METADATA, it is said that this package doesn't install anything after june 2018 and that it is necessary to get packages directly from other sources. On pypi, the google.cloud package supports python version
it might still work if you install it -- iirc the gcp packages were split up so you have to install the specific apis you need for whatever specific service
@@anthonywritescode Indeed, it's necessary to install separately the needed APIs, and in fact, google-cloud-bigquery 2.16.0, the latest one, supports python versions from 3.6 to 3.9 included and not python 3.10 as stated on pypi.org . Do you know other way to install it in simple way using python3.10 ?
looks like they set `python_requires` to be unnecessarily restrictive -- I'd suggest making an issue on their repository asking them to relax that (the best practice is to only have `>=` and not `
always wondered whats the difference and thank you so much for this video, very informative
Cheers mate! This really helped a lot, didn't know about virtual environments before this week... My Python setups were pretty much configured in all the ways you should NOT set up Python and in hindsight they were really quite hideous hah. With this knowledge things are miles better now, I managed to use the knowledge gained in this video to overcome an issue that has been causing me so much grief for the last 3 days. Really appreciate the help.
Cheers Anthony for the clarification - virtualenv & venv.
Cool, for someone relatively new to Python's virtual env, this sort of helped. Good to know that the virtualenv, has some advantages over the newer venv. So, it appears that knowing/using both would be good and not just abandoned virtualenv altogether. I appreciate your taking the time to make these videos and I'm sure you want to do the best with them. So, don't take this the wrong way, but, it would be helpful if you could slow down or meter your speech more. I struggled understanding some of your descriptions. I found CC did too.
you can always watch in .75 if you need it slower!
DUDE!! I WAS SOO CONFUSED BEFORE WATCHING THIS!! THANK YOU MY MAN!! YOU GOT A SUB!
Awesome video ....Thank you very much for posting
You have a really nice persona, and thank you for your explanation. I shall look for other explanations from you :)
By the way, you can also use Poetry to manage your environments. It works great and I think is has a relatively fast dependancy manager.
And if you want you project to becoma a Pip package, Poetry makes that process extremely easy.
my 2c is that poetry is unnecessary. it doesn't add anything for libraries and it's merely a convenience for applications. it's just as much if not more work to distribute something to pypi with poetry. and maybe it's changed but last time I tried poetry I encountered 5 bugs in the first 5 minutes using it which definitely did not instill confidence
Perfect explanation man. Thanks!
Great video, so helpful to just have all of this laid out in a straightforward manner. Also in general recently discovered your channel and twitch, can't tell you how useful it's been!
Two questions I quickly had:
1. Do you have any practices around storing / keeping track of virtual environments?
2. For python versions, does is virtualenv natively ship with all python versions? Or do you need to download each python that you want to create virtual environments with?
good questions! glad you enjoy the content :)
(1)
on my system I ~usually follow this:
- I have one at ~/opt/venv where I store tools I use (more details in these videos: th-cam.com/video/OXmYKh0eTQ8/w-d-xo.html th-cam.com/video/O390_abzo08/w-d-xo.html ) -- I then symlink those tools into ~/bin so they're "globally" available (things like my text editor, pre-commit, and actually virtualenv too!)
- then I make virtualenvs per-project at ./venv (sometimes ./venv2 or ./venv39 if I need to have ones versioned based on the python version)
(2) virtualenv needs the pythons available _somewhere_ to work. I use deadsnakes (launchpad.net/~deadsnakes/+archive/ubuntu/ppa which I maintain!) to install pythons on ubuntu. on other platforms you'd probably use either the python.org installers or something like pyenv (though pyenv is very difficult to get correct in my opinion)
@@anthonywritescode Thank you so much for this info - I'm realizing how important it is to get to know the file structure on your OS :)
switching to virtualenv!
@anthonywritescode
9:54 It's actually possible to create virtualenvs in other Python versions using Venv only:
In windows you type in cmd: python -{version you want to create the virtualenv in} -m vevn {name of your virtualenv}
for example you can say: python -3.5 -m -venv -THIS_IS_A_VIRTUALENV_IN_PYTHON35
and it'll create a virtualenv in python 3.5 (you have to add python 3.5 pip to Env Variables before)
idk if that works with python 2 also, I don't use it either
that's not actually cross version, py -3.5 *is* python3.5 (and there's no venv module in python 2)
@@anthonywritescode Yes, Virtualenv is still more uselful, I just wanted to say that you can use different 3.x python versions with venv
you can't though, you need the venv module available in every python version
Thanks for this, Anthony... another great video.
Thank you so much
Nice. Always wondered about this.
Really clean explanation! Thanks for making the video!
This video left me more confused on which one is recommended to be used.
I mean, I intentionally did not give a decision but left you to decide based on the tradeoffs -- I use virtualenv myself but there's reasons to use venv as well
I'm actually quite confused. Are you to install a virtualenv for every program you write? What do you do if you want your apache server to run the script? It is not going to be using virtualenv. How do you get one program written in one virtualenv to work with another program needing a different virtualenv? How does docker integrate with it all?
yeah I generally use a separate virtualenv for every project that I set up. I install all the tools I need for that project into the project's virtualenv. As for the apache usecase, at least with mod-wsgi the recommended setup is to use `WSGIPythonHome` or `python-home` setting to point at your virtualenv. you usually don't need to talk across-virtualenvs -- usually you'd install all the tools you need for the project. docker doesn't really play into this, though I still recommend using a virtualenv in docker: th-cam.com/video/O390_abzo08/w-d-xo.html
@@anthonywritescode i was working hard learning python. It was all going well and I was happy thinking I'd be finished by now. Then things went south due to python interdependencies. So I built a new PC, installed a new os. Now always running with virtualbox. Cloning regularly. Still I'm scared of python going bad on me again. So i want everything in virtualenvs. But in the future I suspect I'll need virtualenvs talking to others. Do you have any videos showing how to package them up and talk to each other?
I've never had a use case for talking across virtualenvs so I don't even know what I'd talk about!
very helpful, thanks 👏👌
Great videos, thank you! I caught this video a number of years ago and recalled you briefly mentioning conda without elaboration. This week, was watching other YT videos on conda vs pip/venv, caused me to want to find this video of yours again, specifically the part at 3:05 into the video where you briefly mention conda. I was curious not only what your current thoughts were on package management, but if you've warmed to conda... and I was wondering what about conda you did not like back then/now. I could not find a conda specific video of yours but if you discuss somewhere, a link or title to vid would be great. Anyway, this is a casual curious question so response required if your cycles are trim... thanks again, great content!
it doesn't solve problems I have and gets in my way (and is way slower to use than to not). but then again I don't ever install numpy and friends so that probably puts it in perspective
@@anthonywritescode okay... so it has overhead for little gain given your use cases... I've had that happen in other contexts so that makes great sense. I tend to like pip/venv since they're built-in... I've not hit the caveat of venv not being available but largely use standard distributions of Python on Windows/Linux, maybe building it on Linux. To me, pip/venv is a "minimalist" approach that works. I myself don't like to add momentum-reducing layers without justification. 😊 Thank you for clarifying!
You can have several venv activated at the same time on different folders right ?
you can have multiple on PATH but usually activating a venv by its activate script will deactivate the others
Great explanation!
Great video! At the end when you created the environment with a particular python version, I am assuming that you had downloaded and installed the particular version first, correct? I.e., virtualenv does not download it and install it when you run the "virtualenv venv39 -p python3.9" command, correct? If so, is there any precautions or steps that I need to take to simply download and install different python versions for future use with virtualenv? If not, I assume that I can simply download and install versions 3.9 and 2.7 with "apt" in my debian bookworm distro, then create their respective environments. But, this sounds too easy, lol, so I feel like my logic is flawed. Any insights are greatly appreciated, thanks Anthony!
correct, I've got a bunch installed via deadsnakes
@@anthonywritescode, really, okay. I'm going to try your approach then because last night I tried to use pyenv but it didnt work for me. Well, it probably did, but its a little complex so I most likely messed something up. In any case, thanks for the reply!
why don't you like conda?
it's very convenient for installing complicated sciency things, but breaks a lot of expectations for how python usually works -- especially on windows. it's also very very heavy
Can they work in background as a service or a normal application? And probably run using cronjob?
Excellent! I know it's years later, but this was a very good succinct explanation!
I do have some additional questions.
If I built a Django project using system python packages instead of in a virtual environment, how do I move my Django project into a virtual environment to isolate it? Is it as simple as copying or moving the Django project directory into the same directory that contains the virtual env folder?
you should be able to make (and activate) a virtualenv anywhere! -- I usually work on code in a ~/workspace/(repo) directory and put my virtualenv for the particular project inside the repository directory
How do you access windows cmd directly from Ubuntu ?
eheh -- it's a virtual machine: th-cam.com/video/8KdAqlESQJo/w-d-xo.html
much appreciated
this tutorial became a bit confusing because you are explaining the difference between two different tools that kind of do the same thing, and where the thing that they create is referenced by the same name as one of those tools
it's just a directory name -- and they do do the same thing
Nice video!
Does it make multiple copies of the packages in the directory? Is there a way to minimize that?
who cares, disk is functionally free in 2022
How did you set your directory and prompt in two different lines? It's really cool!!!
I go over my PS1 in this video: th-cam.com/video/ngLwml9XI-I/w-d-xo.html
Having 2 packages managers with the very same name is soooo confusing from a beginner point of view
i`ve been seeing a lot of videos, could you recommend me what should i choose, a ide like jupyter, or a text editor like vscode? im starting to study seriously data science and this is an important step for me haha
whatever you're most comfortable with! they both have trade offs (and I actually don't use either of them so I can't really make a recommendation)
@@anthonywritescode what you use then?
I use my own text editor that I wrote -- though I wouldn't really recommend it hahah th-cam.com/video/WyR1hAGmR3g/w-d-xo.html
😎 cool topic
can u upload a video about pyenv?
I've talked a few times about pyenv on stream -- I don't recommend it personally because it is really easy to miscompile python
@@anthonywritescodewhat about pipenv?
I don't use it on principle and don't recommend others use it as well
@@anthonywritescode why
it's slow, clunky, and I won't use software by the original author unless absolutely necessary
Are you running windows 11 with ubuntu subsytem?
I'm using a virtual machine -- there's a video on my channel of setting up virtualbox and another in the faq playlist of why I do it this way!
Though on pipenv?
it's slow and I think it's unnecessary -- and I avoid software written by the original author on principle
A year+ later do you still prefer virtualenv over venv?
yep!
@@anthonywritescode thx, was curious as I delve into this side of Python.
How about pyenv?
pyenv is completely unrelated, and has wasted thousands of hours of my time. I do not recommend it as it's difficult to build python from source and it does not help you enough to build quality executables
Tl,DR: just use virtualenv, not venv.
but how do you use conda with 6 different versions of python at once?
how does no one mention this? conda can't handle different versions of python! why doesn't it work with this!
easy: I don't use conda
Hi. I guess you made a wrong statement at 9:28. You probably wanted to say. The advantage of virtualenv to venv is it works with python 2. It confuses because you started arguing about benefits of venv over virtualenv. Thx anyway for this comparison.👍
??? this is what's at that timestamp in the video:
"the other advantage, well we saw the speed advantage before, the other advantage to virtualenv is it works with python 2"
@@anthonywritescode OK. I guess it was my bad. The english grammar of "advantage to virtualenv" sounded wrong to me and let me misunderstood you. English prepositions are confusing me. I would have used "[...] advantage of virtualenv [...]", but I am a non-native speaker. Sorry!
anything on microcrash is annoying
Hi, thanks for this clear video.
I must be missing something about venv anyway as I still can't import google.cloud module.
To install python3.10, I did: adding sudo add-apt-repository ppa:deadsnakes/ppa sudo apt update sudo apt install python3.10
I installed venv: python3.10 -m venv venv
Then in directory containing venv:
. venv/bin/activate
and once activated , I ran python3.10 -m pip install google.cloud and also python3.10 -m pip install google
Installation of google.cloud semt ok
Nevertheless when I run the following script ( venv still activated )
#!/usr/bin/python3.10
from google.cloud import bigquery
# Construct a BigQuery client object.
client = bigquery.Client()
I get the following error message:
Traceback (most recent call last): File "/home/ced/bigquery-gcloud/./survival_frequency", line 2, in from google.cloud import bigquery ModuleNotFoundError: No module named 'google'
Also changed the shebang to #!/usr/bin/env python, but it doesn't solve the issue.
Thanks for your help
actually, looking directly in env/lib/python3.10/site-packages/google_cloud-0.34.0.dist-info/METADATA, it is said that this package doesn't install anything after june 2018 and that it is necessary to get packages directly from other sources. On pypi, the google.cloud package supports python version
it might still work if you install it -- iirc the gcp packages were split up so you have to install the specific apis you need for whatever specific service
@@anthonywritescode Indeed, it's necessary to install separately the needed APIs, and in fact, google-cloud-bigquery 2.16.0, the latest one, supports python versions from 3.6 to 3.9 included and not python 3.10 as stated on pypi.org . Do you know other way to install it in simple way using python3.10 ?
looks like they set `python_requires` to be unnecessarily restrictive -- I'd suggest making an issue on their repository asking them to relax that (the best practice is to only have `>=` and not `