Yay, a code roast! XD There is one big thing that I think I would still do: turn this into a package. Currently, you just have a collection of modules, but you have a proper package. It’s super easy to do with poetry. Why? • Better separation between the library code (in src/my_package) and the main script (main.py lives outside). • But instead of a script, you can also create a simple CLI, and for that, having a package will greatly help too • Right now, running the main application and the tests ONLY WORKS because you run them from the root folder. But imports in the tests would fail if you were to run them from inside the test folder. And you cannot move the main.py elsewhere. Having a package removes those limitations, as you would import from the package in the tests and the scripts. The imports will always work because you have installed the package in your environment, so python knows where to look. • And as a bonus, this would FIX your red-lines issues in your IDE! I guess what happens is that Pylint is having trouble locations the modules you import. Usually imports work a lot better with packages. BTW, if you have a poetry environment, you have to tell VSCode to use it (cmd palette > Select Python Environment) I hope I convinced you 😊 Also, one more thing FYI: you have OUTPUT_TYPE = “visualizer” # “visualizer” | ”console”. Right now, the type of this variable would be inferred as string, but if you want to more precisely define it, you can do it like this: “OUTPUT_TYPE: Literal[“visualizer”] | Literal[“console”] = “visualizer”, and Literal imported from typing. However, I’m not sure I really recommend this approach, I wonder if a StrEnum wouldn’t be better… Okay, thanks for reading if you made it here 😊 And thank you again for the video 😊
I have been learning so much from you. You Not just python but how to gain job ready skills. I used to be afraid that I do not know how to use any of the python I learned. You bring this down to earth with clear succinct and byte sized lessons. THANK YOU! 🙏
Great video! A good follow up would be good to configure the package with `setup.cfg` or the `.toml` file. The community it's trying to move away from `setup.py` because of security concerns about arbitrary code execution 👍
Absolutely! In fact, I just recorded a Poetry video that I’ll publish in a few weeks. I’m also going to move all the code examples I publish to use a toml file from now on.
Very useful video. I've been using old methods of building my Python packages, but banged on "deprecated" messages this morning... You video really helped me upgrading my vision of distributing Python code .
Thanks you very much for your video, it's very hard to find a clear tutorial about building a python package. Now I'm ready to build a project for server-side transaction verification for mobile 😁
@11:00 I like to install the package like this: pip install -e . The e flag makes it "editable" so while you continue developing any changes you make will be picked up.
During the installation part is usefull to use pip install -e . so you don't need to reinstall you package for every modification you do in the project.
That video was superb. I was so intersting that I forgot to like it in the first place. full of useful information and pro-tips. I just returned to like and leave a comment. Thanks a million.
Excellent tutorial as always. Do you have any plans for a tutorial to distribute a python application for win/mac/linux (not just a single package)? Ideally through some kind of automated github pipeline.
Thank you for the assistance. I found it incredibly helpful. Suppose I have developed an application that is functioning perfectly on my machine. Now, I would like to obtain comprehensive information about the versions of all the requirements and ensure their accuracy. To achieve this, I used the "pip freeze" command to retrieve the versions of the libraries I utilized. I would appreciate your confirmation on whether this is considered a correct practice or if there are any alternative methods you would recommend.
Excellent video! I think there is way too few videos on the topic of packaging python files. So I wouldn't be surprised if this video will generate quite some views...
With the test folder being on the same level as your src folder, does that mean you have to build and install your package every time you test? I can't get relative imports to work...
The tendency nowadays is to avoid packaging as code. People get really imaginative when having the possibility to code and start adding logic in `setup.py`, making the building process not so strait forward. The first approach was to do all the same but in `setup.cfg`, an INI-like metadata file format that setuptools understands. Severing the possibility to write complex functions that run during the packaging process. Finally, the consensus is to use a standard build system, that may use setuptools as back-end as well, and define the building rules as metadata too, but in a `pyproject.toml` file. I think that last approach is the most accurate way to teach Python Packaging today. A follow up video perhaps, just to show the current recommended way to package? 😅😛
In 2023 you talk about the old setup? I thought it just got removed and we now have toml parser in core. Nevertheless: please more packaging videos. Poetry? Flit?
Hi Arjan, thanks for the video. After I run the build commands I'm not being able to import it. It says that the module was not found. Do you know whyt is it happening? Thanks!
PEP 632 is about distutils-> setuptools migrations, which is probably interesting only to people who have been packaging python software for at least 10 years and do not need tutorials?
Hmm, I didn't realize that import supported a tuple as the import targets. Not sure how I feel about that, stylistically! Great video though. I wish I'd seen it last year when I was detangling some package creation for a legacy project. Maybe I should try "import time_travel" and help Past-Me....
Hi Arjan, again a great video that covers a topic from begin to end. Although I have made quite some packages within the company where I work, I've never taken the threshold to actually publish a package (to a company internal Pypi server that is). I join suggestions from other commenters: could you also do a video on packaging with conda and one with poetry? And what the 'deeper thoughts' behind these alternatives are? So as to understand why they were created anyway? And, if possible, explain why you would favour one over the other? I realize, quite some questions but I sincerely think many struggle with this.
One thing I struggle with python packaging is that most of the programs i develop require additional programs that are not python packages. How do we make it so that we can make thise requirements explicit and stop a package from instaling and declare the tools missing?
I believed that the "best practice" way to build a wheel is the "pip wheel" command and requirements should be in the usual `requirements.txt` file(s) and be loaded through `pkg_resources.parse_requirements`
Are you spying on me? Everytime you release a video it is relevant to what I am currently working on. xD But is there a reason why you do not use the folder structure they use on docs.python when they talk about file/folder structure? (A src folder with the package folder inside)
Thx for the video, but take into account that according to setuptools docs running python setup.py directly as a script on recent versions is considered deprecated
Any suggestions how to build sip-4.16.5 ? Should MS Visual Studio Express 2010 or mingw/gcc be used? For example, Nuitka reports: Python: 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 19:28:18) [MSC v.1600 32 bit (Intel)] Version C compiler: C:\Users\Temp\AppData\Local\Nuitka\Nuitka\Cache\downloads\gcc\x86\11.3.0-14.0.3-10.0.0-msvcrt-r3\mingw32\bin\gcc.exe (gcc). Which seems like both Microsoft Visual Studio 2010 and gcc... so I'd like to build using that gcc rather than microsoft studio since that may mess up Nuitka?
Sir i am having a problem i uploaded the file but when i load the file by pip install none of the function work it only imports nothing works idk why i imported all the modules in the __init__ file can you please help em
Next level: Try to compile a Python package written in C with manylinux tools and publish it via an automated GitHub workflow 😂😂😂 I mean the way you publish doesn't really require PyPI. You'd probably be better off with a Git submodule, tbh. Just clone the repo, cd into it and run "pip install .", basically does the same but without PyPI in your way 😉
5:30 - `python setup.py bdist_wheel` apparently does nothing. Did you forget to show how to prepare setup.py? Even after going ahead and preparing `setup.py` and `__init__.py`, after installation, none of the projects, nor any python CLI can see anything inside the package. Can't import anything, can't use any classes, nothing.
Can you read my mind Arjan cos I was thinking of building a package that incorporates Twitter api with chat gpt. By the way I need frontend volunteers, comment below if you feel it’s a suitable project to be part of.
👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
Man, you are taking Python learning to the next level. I am a big fan of your videos.
Thank you so much!
Yay, a code roast! XD
There is one big thing that I think I would still do: turn this into a package. Currently, you just have a collection of modules, but you have a proper package. It’s super easy to do with poetry. Why?
• Better separation between the library code (in src/my_package) and the main script (main.py lives outside).
• But instead of a script, you can also create a simple CLI, and for that, having a package will greatly help too
• Right now, running the main application and the tests ONLY WORKS because you run them from the root folder. But imports in the tests would fail if you were to run them from inside the test folder. And you cannot move the main.py elsewhere. Having a package removes those limitations, as you would import from the package in the tests and the scripts. The imports will always work because you have installed the package in your environment, so python knows where to look.
• And as a bonus, this would FIX your red-lines issues in your IDE! I guess what happens is that Pylint is having trouble locations the modules you import. Usually imports work a lot better with packages. BTW, if you have a poetry environment, you have to tell VSCode to use it (cmd palette > Select Python Environment)
I hope I convinced you 😊
Also, one more thing FYI: you have OUTPUT_TYPE = “visualizer” # “visualizer” | ”console”. Right now, the type of this variable would be inferred as string, but if you want to more precisely define it, you can do it like this: “OUTPUT_TYPE: Literal[“visualizer”] | Literal[“console”] = “visualizer”, and Literal imported from typing. However, I’m not sure I really recommend this approach, I wonder if a StrEnum wouldn’t be better…
Okay, thanks for reading if you made it here 😊 And thank you again for the video 😊
I have been learning so much from you. You Not just python but how to gain job ready skills. I used to be afraid that I do not know how to use any of the python I learned. You bring this down to earth with clear succinct and byte sized lessons. THANK YOU! 🙏
Thank you Arjan! It's so valuable to have all of this concentrated into a single video.
Glad it was helpful!
Packaging is by far the most challenging aspect of Python for me. This helped quite a lot though, thanks
Finally, a simple and straightforward approach to do it. Thanks for sharing!
Glad you found it helpful!
Great video! A good follow up would be good to configure the package with `setup.cfg` or the `.toml` file. The community it's trying to move away from `setup.py` because of security concerns about arbitrary code execution 👍
Sorry for the typos, TH-cam won't let me edit the comment 😥
Absolutely! In fact, I just recorded a Poetry video that I’ll publish in a few weeks. I’m also going to move all the code examples I publish to use a toml file from now on.
@@ArjanCodes any plans for (Ana)Conda?
I personally love poetry :) Never learnt the other ways of code packing - so this is neat to know! Thanks.
Very useful video. I've been using old methods of building my Python packages, but banged on "deprecated" messages this morning... You video really helped me upgrading my vision of distributing Python code .
Glad it helped!
Great outline. I'm building some common modules at work and this will allow me to easily share with my colleagues
Thanks you very much for your video, it's very hard to find a clear tutorial about building a python package. Now I'm ready to build a project for server-side transaction verification for mobile 😁
Thanks for the quality video. Thanks to you I have become a much more structured dev.
Glad the videos have been helpful!
Rien a dire si ce n'est MERCI.Tres satisfaisant vos contenus.👍
thanks so much Arjan! Stars aligned, I had put knowledge about Python packaging on my to-do list this week haha. All the best!
Wow, glad I could help!
@arjancodes please make a video on poetry with typer. Making a CLI and packaging it please
Excellent video, Man. keep it up!
I'm eager to watch a video you explain about meta class and meta-programming!
Noted!
Thanks for the complete guide Arjan!
@11:00 I like to install the package like this: pip install -e .
The e flag makes it "editable" so while you continue developing any changes you make will be picked up.
great tutorial, thx
With poetry, you can just do "poetry new your_project_name", "poetry build", "poetry publish" and you're done.
During the installation part is usefull to use pip install -e . so you don't need to reinstall you package for every modification you do in the project.
That video was superb. I was so intersting that I forgot to like it in the first place. full of useful information and pro-tips. I just returned to like and leave a comment. Thanks a million.
Thank you and I'm so glad you enjoyed it!
dude. you literally have a video for every python obstacle I encounter! lol
Haha, happy to be of service 😁
Excellent tutorial as always. Do you have any plans for a tutorial to distribute a python application for win/mac/linux (not just a single package)? Ideally through some kind of automated github pipeline.
Quite good. Thank you!
Glad you liked it, David!
Great video, Arjan. Could you please make a video on Poetry?
Video on Poetry is coming soon! Stay tuned :)
@@ArjanCodes Eagerly waiting for it :)
Thank you for the assistance. I found it incredibly helpful. Suppose I have developed an application that is functioning perfectly on my machine. Now, I would like to obtain comprehensive information about the versions of all the requirements and ensure their accuracy. To achieve this, I used the "pip freeze" command to retrieve the versions of the libraries I utilized. I would appreciate your confirmation on whether this is considered a correct practice or if there are any alternative methods you would recommend.
Thanks. just what I needed. I am a relative newbie. I am curious about Wade Curry's questions
This is amazing information. I believe this will be a great addition( having a course in details) to one of your amazing course library.
very helpful video, thank you!
Excellent video!
I think there is way too few videos on the topic of packaging python files. So I wouldn't be surprised if this video will generate quite some views...
Wanna learn framework building process. Thanks for this awesome tutorial ❤️
With the test folder being on the same level as your src folder, does that mean you have to build and install your package every time you test? I can't get relative imports to work...
Thanks Arjan! Have you ever used declarative metadata? What are their (dis)advantages compared to setuptools?
The tendency nowadays is to avoid packaging as code. People get really imaginative when having the possibility to code and start adding logic in `setup.py`, making the building process not so strait forward.
The first approach was to do all the same but in `setup.cfg`, an INI-like metadata file format that setuptools understands. Severing the possibility to write complex functions that run during the packaging process.
Finally, the consensus is to use a standard build system, that may use setuptools as back-end as well, and define the building rules as metadata too, but in a `pyproject.toml` file.
I think that last approach is the most accurate way to teach Python Packaging today.
A follow up video perhaps, just to show the current recommended way to package? 😅😛
Coming soon! I'm also going to include a pyproject.toml file from now on in my code examples on GitHub :).
now we want a video about licenses!
Noted!
Locally install 📦 - 10:56
A M A Z I N G !!
:D
Excellent tutorial, would you be interested in doing the same but with the poetry library?
from my perspective the community is moving away from setup tools. I'd probably go over a framework like poetry or flit these days
In 2023 you talk about the old setup? I thought it just got removed and we now have toml parser in core. Nevertheless: please more packaging videos. Poetry? Flit?
Video on Poetry is coming soon! Stay tuned :)
Amazing information 👍👍🙏
Great video talking about the built in abilities of Python packaging! conda is a neat option too!
Thank you!
Wish you explain package folder structure in more detail.
Thank you so much
You're most welcome
Woah! Holy crap you look like John Green! Great Videos btw.
Glad you liked the content!
Hi Arjan, thanks for the video. After I run the build commands I'm not being able to import it. It says that the module was not found. Do you know whyt is it happening? Thanks!
You simply goated
Thank you!
Can we have an update (if applicable) with regards of python 3.12 PEP 632?
PEP 632 is about distutils-> setuptools migrations, which is probably interesting only to people who have been packaging python software for at least 10 years and do not need tutorials?
Hmm, I didn't realize that import supported a tuple as the import targets. Not sure how I feel about that, stylistically! Great video though. I wish I'd seen it last year when I was detangling some package creation for a legacy project. Maybe I should try "import time_travel" and help Past-Me....
S-Teir content Thanks man.
Hi Arjan, again a great video that covers a topic from begin to end.
Although I have made quite some packages within the company where I work, I've never taken the threshold to actually publish a package (to a company internal Pypi server that is).
I join suggestions from other commenters: could you also do a video on packaging with conda and one with poetry? And what the 'deeper thoughts' behind these alternatives are? So as to understand why they were created anyway? And, if possible, explain why you would favour one over the other? I realize, quite some questions but I sincerely think many struggle with this.
Thanks for sharing! Video on Poetry is coming soon! Stay tuned :)
what about using poetry?
Video on Poetry is coming soon! Stay tuned :)
Can you do build package with poetry too
thanks man😊
Welcome 😊
What pypi server do you use for private packages?
One thing I struggle with python packaging is that most of the programs i develop require additional programs that are not python packages.
How do we make it so that we can make thise requirements explicit and stop a package from instaling and declare the tools missing?
I believed that the "best practice" way to build a wheel is the "pip wheel" command and requirements should be in the usual `requirements.txt` file(s) and be loaded through `pkg_resources.parse_requirements`
Are you spying on me?
Everytime you release a video it is relevant to what I am currently working on. xD
But is there a reason why you do not use the folder structure they use on docs.python when they talk about file/folder structure?
(A src folder with the package folder inside)
The same thing happens with me. One day I was searching for pydantic models in python and the same video came in 2 days.
I would like to know too.
Why have the app directory? is this the same thing as a src directory?
Thx for the video, but take into account that according to setuptools docs running python setup.py directly as a script on recent versions is considered deprecated
where is the version of the package stored?
amazing topic and video. want to see more on this.
Thank you! Will do!
your file system here is confusing me, pls try showing the desktop. It will be way better
How do you choose the name of the package?
can you give us a video to make a deployment file so that people can access it
could not do on linkedin - will do on facebook though
wow. it is overcomplicated.
have you published it yet
Not all punctuation are allowed, how select the desired ones
Any suggestions how to build sip-4.16.5 ? Should MS Visual Studio Express 2010 or mingw/gcc be used?
For example, Nuitka reports:
Python: 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 19:28:18) [MSC v.1600 32 bit (Intel)]
Version C compiler: C:\Users\Temp\AppData\Local\Nuitka\Nuitka\Cache\downloads\gcc\x86\11.3.0-14.0.3-10.0.0-msvcrt-r3\mingw32\bin\gcc.exe (gcc).
Which seems like both Microsoft Visual Studio 2010 and gcc... so I'd like to build using that gcc rather than microsoft studio since that may mess up Nuitka?
Sir i am having a problem i uploaded the file but when i load the file by pip install none of the function work it only imports nothing works idk why i imported all the modules in the __init__ file can you please help em
Next level: Try to compile a Python package written in C with manylinux tools and publish it via an automated GitHub workflow 😂😂😂
I mean the way you publish doesn't really require PyPI. You'd probably be better off with a Git submodule, tbh. Just clone the repo, cd into it and run "pip install .", basically does the same but without PyPI in your way 😉
I could watch another 12 movies on this topic. It’s the most confusing least intuitive part of python
give me file which should work as same as the build file works in nodejs
5:30 - `python setup.py bdist_wheel` apparently does nothing. Did you forget to show how to prepare setup.py?
Even after going ahead and preparing `setup.py` and `__init__.py`, after installation, none of the projects, nor any python CLI can see anything inside the package. Can't import anything, can't use any classes, nothing.
C'mmon where's poetry?
Coming soon!
Can you read my mind Arjan cos I was thinking of building a package that incorporates Twitter api with chat gpt. By the way I need frontend volunteers, comment below if you feel it’s a suitable project to be part of.
Not a frontend expert, but would like to be a part of this.
Awesome! Thanks for sharing this.
Checkout pdm
Do you also cover Julia language?
Nope, the answer is poetry
I just recorded a video about Poetry, so I agree now :).
The actual answer is pdm! Another example of the „There are 14 competing standards“ xkcd^^ I like poetry but imo pdm is even better.
@@joelbeck1822 totally agree on PDM. poetry capped versions are far more troublesome than useful.
Pipenv is good enough!
I like pbr.
is he dutch? he looks dutch
He is as Dutch as they come.