Python Sucks And I LOVE It | Prime Reacts
ฝัง
- เผยแพร่เมื่อ 26 ก.ย. 2024
- Recorded live on twitch, GET IN
/ theprimeagen
Article: blog.cameron.r...
Author: github.com/wzid
MY MAIN YT CHANNEL: Has well edited engineering videos
/ theprimeagen
Discord
/ discord
Have something for me to read or react to?: / theprimeagenreact
Python is the second best language for every software problem
Oh no you didn't
that actually sums it up pretty well
What was the best, C?
You know what's sucks, when programming an ai that uses too much resource overheat the gpu 😂.
@@SimGuntherwhat they meant was, the best language depends on the project, but Python is the second best choice for every project
LOL. Just stumbled on this.
I wrote a C extension for Python that eventually became the CSV module in Python.
I did it because i was working on a fixed price project dealing with 100's of megabytes of CSV files exported from SQL Server via Excel (don't ask... Finance people are strange). Each iteration of my code during development took 45 minutes using a pure Python parser I wrote to strictly implement Excel semantics. It was literally costing me money that Python was so slow.
I figured I could develop the parser in C and finish the project in less time than just continuing in pure Python. Turned out I was right.
I submitted the pretty raw code to the Python project and over a couple of months, with the help of two other people, made it a lot nicer to use.
Piplup needs HUGS
You literally wrote the module that I used to write my first commercial project 😍 thanks man xx
@@jamesbedwell4715that is awesome.
I figured back in the day that if I was struggling with reliability parsing massive amounts of CSV, then some time soon, someone else would be.
Just curious if Pandas uses your CSV extension in the read_csv() function? i've used it to read csv's in many programs i've written. Lately though I've moved away from using CSV's at all though unless the person wants data stored in a csv file. Have switched to using sqlite3 a lot.
@@resresres1I doubt it. Pandas probably loads data into arrays of native values. The CSV parser creates Python objects.
As an embedded dev, who usually develops for embedded Linux systems:
- I'm all in on C/C++, whenever there is time to build something.
- I'm all in on Python3, if management wants it by yesterday!
I rather spend my evenings working on my own coding projects, versus working unpaid overtime in C/C++ due to management being incapable of planning ahead!
Same here, also an embedded dev, agree 100% with this!
I'm really enjoying my embedded classes in engineering at uni... what sort of companies hire doing your sort of work? What kind of boards are you mostly working with?
@@poogle9368 I've done a ton of different projects over the years.
I started out with embedded systems, for museum and attraction park equipment. After that I went to work at a university for some years, helping post-docs with automating their scientific research. I then went on working for a small engineering firm, which develops medical and aerospace equipment for larger corporations and startups.
All above jobs where very interesting, but came with tons of red tape and bureaucracy. So I changed job once more, and now work in a medium sized company that develops access and control systems.
Most boards we use are custom made, usually made with Altium or KiCad. They contain a broad range of controllers (imx8, RPI-CM4, ESP32, STM's, etc ...)
@@poogle9368 Most boards we use, are custom made. But they contain a plethora of chips, eg imx8, RPI-CM4, STM32, ESP32, etc ...
For what companies, there are some larger firms and a plethora of smaller startups.
The smaller startups are often fast-phased and not for everyone. I however prefer them, as there isn't as much bureaucratic overhead.
The larger corporations are extremely well structured, however everything moves at a snails phase which frustrates me massively...
@@poogle9368 Most boards we use, are custom made. But they contain a plethora of chips, eg imx8, RPI-CM4, STM32, ESP32, etc ...
The company recommended for you, usually depends on age, interests and personality.
The larger firms are extremely well structured, usually respect the 9-5 workhours, with better pay, etc ... Those larger firms are however very bureaucratic, which makes everything progress at a snails phase. I can't stand it, so I avoid these firms.
The smaller startups are usually very chaotic, with less respect for the 9-5 workhours, and less pay... The projects are however fast-phased, which means you get a new technical challange about every other month.
The only thing I always recommend, is to work for a company that direclty profits of your work. I mean with this, make sure the product you develop is meant to be sold! Avoid companies that hire you for internal service or development work, they usually deem your function as an unavoidlable overhead.
The thing I like about Python is that it's really good for getting basic things out of the way quickly. As long as you have a library that does the thing you need to do, you'll probably have good performance, because that'll be written in a more performant language.
It's as simple as running hack.exe and saying "I'm in."
You hope that it's written effective and not uses another 20 packages just to do simple stuff that can be written in a few lines of fast code xD
@@melonenlord2723 Pretty much, yes. That's one of the reasons why I check the dependencies of anything that I am planning to use long-term.
Edit: Originally wrote it differently, but the phrasing was a very ugly double negative.
Correct. Let the C/C++ autists struggle. I wanna have fun programming 😂
Mostly... Those libraries suck, mostly. You would be surprised what can be done by the standard library.
I used to be vehemently against Python for superficial reasons until I actually learned it. Sure, it's overly opinionated about shit that doesn't matter (indentation) and it has a few truly gross quirks (loop variables are available out the loop itself, rather than being scoped to it). But it's just a really fantastic scripting language. If you're trying to write something quick and dirty to solve an immediate problem, it's really hard to go wrong with Python. It reads and writes like a perfect, formalized version of pseudocode. It's got great built-in libraries, and its popularity means that it has shit-tons of amazing third-party libraries covering pretty much any use case you can imagine. And honestly, gripes about performance are completely overblown. Nobody is trying to write bare-metal systems code in Python, and if they are, they've simply chosen the wrong tool. That doesn't make Python a bad language.
But also all the stuff that needs to be quick has something like a C backend it interops with anyway.
take:
Pandas
Tensorflow
Pytorch
Numpy
etc
they are all interoping with something faster behind the scenes to do thing like malloc contiguous memory ect
python is just the application logic and they are fast enough 99.9% of the time.
Pure python is slow, sure, but if you are using python is probs because of a libary it has.
Past that though, it is pretty awful.
And apps made with it get REALLY stupid to work with.
JavaScript does better performance, easy scripting, and less foot guns. A few less anyway.
I hate python's variable declaration and scoping. It's way too easy to write unreadable spaghetti.
> It reads and writes like a perfect, formalized version of pseudocode
Pretty good description of Python
@@lunafoxfire And get side effects
the problem is that every small program eventually will become a massive enterprise mess
Yes, I usually like starting my messes in bash
@@Derian_De_Greyyeah, and at the end there is no difference AT ALL in time spent, in comparison with just all the way mastering C from the start, for example.
Then go use C, its been the same forever and is the grandfather of all.
@@Derian_De_Grey idk i guess it exists as spotify and instagram are running django
No every enterprise program started small but most small problems stay that way. You just never have to think about them again if their pure one offs lol
I relate with this guy. I had a few years of experience writing assembly, C, and C++ code when I first came across Google's Python training videos. I could code but I didn't like coding. 2 days with Python totally changed that. I remember writing a script in 30 minutes that scraped transit timetables from a website and helped me finish a data analysis project that would have typically taken me a long time to write. I've loved Python ever since and though I spent years writing Java, I finally managed to switch to Python professionally and love it. New releases of Java, especially with Streams have made Java a lot easier to like, but I still prefer to use Python for nearly all of my personal projects. The new proposal to remove the GIL from Python should make it run a lot faster soon
Yes there will be a official no-GIL version of python but just like all the other attempts in history it will make single threaded performance worse.
@@burarum1 we could have the two and like use one rather than the other as soon as we multi thread
relatable
I very much enjoy your videos on the whole. Your mix of bawdy humor and blunt CS commentary has me constantly swinging between learning something fantastically valuable and laughing away the annoyances of my career.
This one especially spoke to me. I work as a Data Scientist. I will out-Excel 99% of the world, I learned VBA (shudder) as my first language, I love R for my EDA and I will script any random thing that I expect to run twice (once when I think I know what's going on, again when I realize I f'ed up and it's more complex than I thought) in Python. My employer is constantly pushing for more speed and that has me deep in Go and and exploring Rust. These are all fantastic tools, *for purpose*. I'll never write a Go program to graph correlation from a sample data set, and I'll never write R to run a publicly exposed web server. People who shit on a certain tool don't understand the power of context and the fact that you, the Crustious Crustacean TH-camr I know of, will flat out say "Python can be a quick W; when you just need to get something done, get it done" makes me smile. Sometimes, it's just the right tool.
The main reason I learned Python was because of my hobby in higher mathematics. The open source software SageMath is wonderful and is built on slightly modified Python.
If you hobby is mathematics try Lisp or Clojure
I remember trying Sage like 15 years ago bc I didn’t want to buy Mathematica or matlab. I’m guessing it’s way more useable now
I can believe the LISP thing, once you've done enough CL/Scheme/Clojure the parenthesis madness disappears like the code of the matrix and you just see the tree of code, it's like learning Vim motions. Takes a *while* to get there tho ngl 🤣
Yeah, Lisp is actually a very productive language. With a proper programming environment, the parentheses are a non-issue. You can write extremely elegant solutions to problems.
@@macaroni_italic (are you(sure(about that)))
Lisp is so good that it is likely the reason for it's "failure". Guys can write programs on their own, don't care about documentation and making other people understand their code. Also, they don't want to teach anyone, since it's their secret sauce, or because they could write more programs in the time they waste on teaching people about how it works writing documentation.
@@macaroni_italic Man I want me some LISP jobs, I like you can resume after errors. LISP is less standardized than Python in its libraries though. You can use macros in Python by using sexp Python like Hy btw.
@@NoidoDev Dude with Lisp/Smalltalk/Python you can write introspective tools to help you trace what programs are doing thanks to their dynamicity. You can use static type inference as well. Like in LISP you can run code in your own custom eval that produces a trace history for all the variables of a function call and then customize Emacs to show you it (hint: replace apply, let with your own version of apply/let that keeps a history). Another thing I saw people do in Python is to get a __getattr__ proxy objects that log what methods are used on them and find all callables in sys.modules (all your symbols) and pass in your proxy objects to generate a "dynamic" type, and this kind of technique can be used for Smalltalk and LISP as well.
I am just starting my coding journey. Your videos have this incredible way at jabbing at the little idiosyncratic nuisances that even a beginner like myself encounters. Really entertaining channel man I have laughed out loud several times. Quite informative too honestly, since sometimes making fun of a certain issue makes it more relatable to our own experiences, which makes it easier for us to understand why that certain issue exists in the first place. Holy run on sentence.
"Lisp is used to tell other people you do Lisp." - Rust guy who does Rust
Lisp actually takes very little time to develop because code = data. So you design your data structs and algos follow naturally. I wrote some Clojure code and it was a pleasure to write in
and if you use common lisp, your dynamic code will run faster than any python code, and even faster than clojure. all you have to do is use SBCL and CLOS, for the javascript part you use parenscript a compiler of a subset of cl to js
I still have no idea how lisp programmer can be confident when using a function in lisp without knowing the types of the input.
@@balenol1209 LISP is the OG REPL-icious language. Just turn on the REPL and call the function
@@vikramkrishnan6414 that's not a good argument. Using REPL is cool to test a function, but it wouldn't catch the requirement of an input's type holistically.
@@balenol1209 Typed scheme exists, you can implement a type checker using macros. But mostly it is indeed the OG dynamically typed language. Some dialects have first class contracts as the idiomatic way to enforce constraints, including type constraints
how does he get this kind of energy, feels like he is always on crack to get this energy in him, and his video edits are hilarious
Too much time in front of a screen doing software engineering causes changes in the brain that alter how reality is perceived, akin to consumption of psychodelic drugs or achieving nirvana by self starvation. I am working hard to reach that state of consciousness that he has achieved.
@@miguelarribas9990 yeah me too
the ultra [coding] instinct.
Pythonista - An anagram of “A Hypnotist”
dios mio you are right
I end-up in python for obvious ml reasons and I agree with the article. It is a joy to script in python exactly because you can get stuffs up and running pretty fast. For me is like executing pseudo-code!!!
When I was young we had something called Basic, it was similar to python in many ways. For example it was easier to learn, simpler to write and ran slower than C or ASM. And just like today, the "real" programmers that were seeing you using this automatically had to look at you with disgust... lol
Plus ça change, plus c'est pareil!
On the point about type script being "enterprise" and allowing you to build these huges messes
python has inbuilt type hints with generics and union types now
as someone who uses mypy and type annotations like it's the bible, not really
type annotations is the greatest python feature in recent memory, intellisense, code docs and reading/writing code is so much more ergonomic with it
however, it's just that - annotations
you still need the external mypy tool to type check your code and python will still be python, lazily typed and runtime type error prone, it really just helps to double check your code rather than to be a better language
plus python already is kinda enterprise already tbh
@@calliioa we run a python backend for our platform with flask and we use python type annotations like if it was typescript. I love it and makes everything more ergonomic to work with
Python's built in type hints are epically shit. They are orders of magnitude worse than TypeScript types.
@@smallfox8623 Hard disagree, particularly considering recent progress, and it's only getting better.
Making it more of a mess. Why not
In the past 8 years I don't think I've come across situations where python wasn't performant enough with room to spare even when I was doing things that needed to be as fast as possible (DNS server oddly enough). More than 8 years ago I did a bunch of image processing that python couldn't do in a timely manner. That was doing a visual diff between documents thousands of pages long.
This article really captured my thoughts on Python well. It's not the most performant language out there in terms of raw execution speed (you can improve it with Cython or PyPi, but still, it's never gonna be C or Go or Rust), but the speed and ease at which you're able to just solve a problem with a usually reasonable level of performance just feels good. I've started using it a lot for smaller automation tasks and alerts for technologies that don't have good tools for alerting. And Flask legit is just my favorite micro web framework (despite all the problems with microservice architectures). The productivity is just so nice, so long as you keep in mind that you need to use the right tool for the right problem (a Netflix-scale web API probably needs something more performant).
Python devs have FastAPI. Why would you use Flask instead of it? I'm not a pythonista myself to prove next statement but this microframework looks like a game changer for python in web)
@@whu.9163 what are the major differences?
Using Flask is like saying a roof ripped away by a tornado a minor inconvenience. It's not completely wrong, but massively understates the complexity of the situation, and depends on the focus of the observer.
FastAPI very easily allows mixing async and sync handlers. Integration with Pydantic is non-frictional. Writing something small using decorated endpoint handlers is as easy as creating more class-based endpoint handlers (unlike Flask). The dependency injection works and is welcome for anything slightly more complex than the trivial tutorial examples in Flask. There are some annoyances, mainly the default error handling mechanism is absolute ass, and just about requires complete replacement for even the most trivial customization, just like adding CORS support (either the default is okay for you, or you have to configure _everything_). Also the main documentation is largely okay, however severely lacks a reference section.
Although it must be mentioned, a great deal of FastAPI features are actually just Starlette features, the framework FastAPI is based on. And often there's better information in the Starlette docs than there is in the FastAPI docs, something where the FastAPI developers could make more effort to at least _link to_.
So you're saying you would use FastAPI Any day over Flask ?@@FrederikSchumacher
@@conan_der_barbar FastAPI is asyncronis, by just adding the "async" keyword if front of functions. FastAPI also provides a complete "docs" page that automaticly writes all the API documentation. Also, FastAPI is the new rookie kid on the block. So the only real reasons to use Flask are, that it is grandfathered in, or if you want to make a Rapid Development Website for a limited number of users inside a Subnet.
Python is what I use at work, and it's so easy to do simple things like getting data from APIs and doing ETLs and other data manipulation. It's like bash in that it's super easy to automate and finish something quickly that works "fast enough" for most things that most people need.
Python and data processing are a match made in heaven
And with readability that bash scripts can only dream of. Generally, I find that you have to be actively malign to conceal your intent in Python. It can be ugly, inefficient, babbling, but rarely do I stare at a Python function thinking "what the f* is this even doing?" which happens a lot in some other languages
I'd like to know how many hours of oncall and lost engineer productivity could be traced back directly to a lack of typing and the shitty patterns in Python. I see it all the time in frontend work, the JS code always has random bugs that we have to fix, but the correctly-typed TS code almost never has issues unless something in the backend changed.
@@thorbergson You can also generate type hints by using static or dynamic type inference in Python.
Fun fact: they can easly make Python much much faster, but they don't want lost all compatibility with tools, librares and frameworks.
I mean they were fine with losing compatibility when making python 3 lol
@@oderchannel426 no, they were shocked by the number of tools written in Python. Check what Guido van Rossum thinking about this.
@@oderchannel426yeah, and look how much of a problem that was. There's shit out there that still won't get off 2.7.
@@oderchannel426Python 3 was such a huge mistake and mess that CPython coredevs are saying that there will never be a Python 4.
The best program is the one that does its job. A program cannot do its job unless it is written. If Python lets you actually write the program where another language would have been a blocker, it was the best language for the job. If you need to consider changing the implementation language later, you’ve won.
Just like [x for x in list] you can use parenthesis instead of square brackets like (x for x in list) to use generator comprehension instead of list comprehension, which is lazily generated
If you pass it as the only argument to a function, you don't even need the parenthesis. foo(x for x in lst)
@@mattmess1221 this is also true but usually looks kinda ugly for more than 1 arguments
You can do this for other types:
tuple(x for x in some_list)
[{x:y} for x, y in some_list]
@@kubre for more than one argument, then parentheses are mandatory.
As a data scientist, I can tell you that’s why I love Polars as a baby of mama RUST and Python. It so fast to process and analyze data, both development ease/time and performance
You can use the compile option unter Linux with Pytorch but it won't be as fast as C++ at least not until everything is compiled., I'm just a guy playing around but I just recreated my Python NN with all its function in Pytroch C++ Frontend, It went from ca 40-60 minutes down to 4 minutes and I'm still in Debug . When you run this in an professional environment this makes huge difference in money.
In what aspects is polars faster? It’s just parallelizing the processes automatically under the hood from what i’ve seen. Is there something else that makes it more efficient? I tested polars code and pandas split up in a multiprocessing pool and the time was the same. just curious as i’m a data science intern atm.
@@Fiercesoulking Yeah the big deep learning frameworks are all catching up with performance. It does make huge differences~
@@allah9896 I mean query optimization is pretty great, and also the syntax just make the coding part so much more efficient and easy to follow
@@allah9896 How big is your dataset? If it is 10,000 items, you probably won't notice any difference between them. But if it is 60,000,000, like a dataset I'm working on at the moment, Polars is very fast whereas Pandas just falls over.
Apparently C is easier to work with then Java, and also runs faster, so my takeaway is I'll be using C.
It takes forever to get anything done, but the simplicity makes the footguns more obvious (also the low level makes the footguns more like foot dynamite). C++ is particularly guilty of making the footguns non-obvious, and sometimes UB takes millions of iterations to surface, good luck finding them in your test suite :-/
@@cheaterman49 I think this is true, but C is not C++. I feel C is very clean. There are not a lot of concepts you have to hold in your head. If you know how the memory works it’s very clear. C++ just adds a lot of abstractions that make things less obvious.
Yeah that was also where I stopped taking that graph seriously
@@MaryTheTankGirl " C++ just adds a lot of abstractions that make things less obvious."
Yeah but those abstractions when designed well are incredibly helpful. There's no universe in which mallocing and freeing everything explicitly is better than using vector or unique_ptr. There simply isn't.
@@isodoubIet Absolutely 💯. I didn’t mean to imply that these abstractions are bad. I just think it’s more complex, because there are more features.
I have been using Python to make reference implementations of stuff I found in math and computer science papers for a while now. Great language for doing that. For example, I recently created a genetic algorithm library; I started in Python, then reimplemented in Go; Go was more performant by a wide margin even before I refactored to use memory pools (which resulted in a 99.94% reduction in benchmark times for simple use cases).
That's a very nice roadmap - you must have learned tons from it!
@@tywky I have, yes. Very fun way to get back into the groove of things after being sick and useless for 3 weeks. Prior to the illness, I was implementing the Practical Isometric Embedding protocol, but I've put it on the back burner for now. A couple months ago, I implemented a scripting language based on Bitcoin script for use as embedded ACL in distributed systems; last year, I made a bunch of advanced cryptography proofs-of-concept (e.g. taproot and ed25519 signature adapters); the year before that, I implemented MuSig. Python is my go-to language for figuring stuff out because all the building blocks for anything I want to do are available.
@@k98killerwow how long you been programming? What was your first language? What all projects you did at the beginning?
@@mallukittens177 I started as a kid writing batch/command scripts in Windows XP; within a few years, I had learned c/old c++ on Linux, a Windows scripting language based on Basic called AutoIt, and bash scripting. In highschool, I learned JavaScript, ActionScript+Flash, HTML+CSS, and PHP.
From the start, my interests were largely in networked/telecom programs. One early project was using the built-in CLI network tools on Windows to make a simple chat program. One of my first C programs was a UDP chat program on Linux. I basically try to make chat apps in every language I learn using whichever methods/strategies i think are worth exploring, e.g. I explored multicast UDP a few months ago.
I am a python developer. It was the first language i have learnt and i just kept getting jobs. It is pretty good for externally bottlenecked, server gluecode projects, but you can shot yourself in the foot with it quite easily. while "anyone can write python", and be super productive, you still want to hire developers who know what they are doing, and give them enough time write it properly, if your project is larger than a few thousand lines. a comprehensive automated test suite is essential. that way it IS faster. but there is a prisoners dilemma type problem with python: you can churn out code that seems to work way faster than it would take to write it in a maintainable way. But since the pitch to the suits was that it would get done fast, it is against your personal interest to do a proper job, because "delivering results" is advantageous to your career. The fact that it will end up taking much longer this way will not lead back to you, because it will just cause the other devs work slower, and incrementally grind the whole project across the board.
I'm a computational chemist. When I want to do a huge set of calculations or build a simulation I'll use something like rust or C++ but most of my coding involves generating input files and pulling info from some esoteric data files and plotting it and python serves me so well in just a few lines. It's not fast, but it's faster than me, and I can get a plot done 15 min before a meeting.
You missed a great opportunity to finish with "The name is the pythogen"
I don't dislike python, dislike how everyone seems to go directly for python just because a random guy on the internet said it is "easy", "can do anything" and "you can get a job in 2 months"
it is even harder to distribute python than TS.
A lot of python modules rely on OS level packages, you can install something with pip and it will still not work.
This is as true as it is false. There's many reasons for this, least of which is that libraries are badly written or only supported on one platform.
@@edwardcullen1739 from my personal bias exp js/ts tend to be more plat agnostic as authors tend to not step outside of node/ js/wasm sandbox.
with wasix i think this will skew it even more in the future as some of the c libs could be recompiled.
That's why you use venvs and setuptools or Poetry.
@@eUnkn0wn The fact that you need something like venvs to make sense of the dependency hell is as much an indictment of python as anything.
@@isodoubIetYou use node_modules..
I like python for short little stand-alone scripts, and rough drafts.
It's a great language for dashing something together, to address a lot of the questions and uncertainty and preferences you want to handle -- so you don't have to compile every tweak and change, one at a time, into the main project's C++ version.
The more time you spend in an interpreted language, the less time you spend writing and compiling a C-lang.
As a Pythonista myself, I love this article
You can add type hints in python. Your IDE will give you warnings when you don't follow those.
Laughs and cries in Ruby
common lisp is faster than python in both runtime execution and productivity. This is why Rich Hickey who was a C++ expert with 15 years of experience chose Lisp over C++
Hickey was making a software for a customer he first made it in common lisp, took him 6 months then the customer was impressed of how fast that was but had an additional requirement it had to be in C++ and Hickey with having the problem figured out, rewriting it in C++ took him 18 months.
Common Lisp is faster to develop for custom solutions that don't require many standard libraries since you can rerun code after you get an error, however, since its a LISP there is no standard library ecosystem. Every LISPER creates their own eDSL to solve their own problems, and other people may not understand their code. Also why not use custom compilers to convert code from LISP to C++, and write a LISP eval function in C++ to handle certain difficult cases.
1:50
CPython (what you get if you apt-get python3) does not have a JIT. pypy is a jitted interpreter of python. It's alright. Don't recall if they support the entire language or what was going on but I've used it and it works alright. It's no V8 though.
in the icpc (international collegiate programming contest), the python is in fact more fast to write, if you has the best solution in python and other language, python will be more easy to easy to write, all those built in methods helps a lot
C++ is king for competetive programming
The elephant in the room for Python: package management. It's terrible and there are no good options. Conda takes forever to build environments. Pip may not give a valid environment at all.
this right here is why i shift my attention as a scientist to Julia (I know it is not a general purpose as python especially cause static compilation is not yet a thing) and Rust because their Package managers are superior. Julia built from the begining with both a package manager and environment manager. And the greatness that is cargo in Rust.
Mamba
5:40 was so good, you had me already laughing and you just kept going, well done :)
Love your content, you give me some Dr. Disrespect vibes but for engineers.
TypeScript wasn't a mistake. JS on the back-end might be a mistake, but on the front-end - where JS just isn't avoidable - the added stability wheels are a godsend, even if they're phantom wheels.
If only there was an language agnostic execution environment in the browser to allow more languages being used... Oh wait there is, but it just happens to lack crucial features by design. Or those features requiring debilitating workarounds. Ah well, just the typical W3C nonsense.
actually python type annotations, static type checkers and a data modeling and validation library like pydantic make developing large scale projects a pleasure.
That is a terrible way to develop large python projects as it slows down your development speed. You should be using microservice based architecture instead.
mypy is terrible, its type checker is actually kind of faulty, I've caught it allowing me to do stuff that should obviously violate the type system. pydantic requires a rust runtime to know the types, so you have a small performance hit on top of the already slow CPython interpreter/virtual machine.
"Python is Slow"
> Discord bot
tbh the difference in speed between python and rust, is not significant for applications that do a lot of async calls like just waiting for user input and stuff. unless of course your bot is used in a lot of servers or if it does heavy computation or something.
the author probably knows this anyway :P
lets be serious, sometimes speed doesn't matter, you dont need c, c++, rust, you need to get the project up faster
Especially when creating a discord bot.
plus why chain yourself to one language be a chad and learn em all
Python has typehints and mypy. It has generics and can emulate higher-kinded-types using mypy plugins. You can make a completely statically typed program.
What I can't understand is the reason why these "performance conscious" developers start worrying about performance even before measuring the performance of the program and identify the bottlenecks.
For many cases, especially when developing backend systems, bottlenecks tend to live in IO or data storage layer so using "fast" language like Rust, C++ doesn't help much to improve the performances of the system anyway.
right, like these people are making a discord bot, the bottleneck is discord's API, which takes about a second per request regardless of any other factors, so your best bet is to send off as many requests as possible at the start of your command and then do all your calculations while discord is slowly doing its thing, and that's about as performant as it can get no matter your language of choice
Premature optimization is a problem, but there is validity to speeding up even when bottlenecked by IO. And it is simply to reduce resource usage. Not saying this is major part or important part in early development, but it should not be ignored entirely
having IO or network bottlenecks is not a valid excuse in most cases. you shouldn't prematurely optimize but "pessimizing" your code isn't good either.
@@marcs9451... in most cases this falls off that cliff.
@@marcs9451 Profile then optimize. If your code happens to be a bottleneck, first find where/why, then fix it. Prematurely writing everything in the fastest possible way is a recipe for disaster - and that applies to Rust too: as Prime already said, your first implementation of pretty much anything in Rust should be very ugly, very slow, making copies everywhere etc - and if that's fast enough to ship, then no need to waste expensive engineer hours optimizing it further. Python is just taking this approach to an extreme where machine time is considered largely irrelevant in cost compared to human time, and that seems to be true for most companies with most (medium sized) user-bases. It only makes sense to consider machine time when your hosting is within an order of magnitude of an engineer's salary ; if you're paying a $5 VPS each time you do a new client project because there's gonna be less than 1k users, meanwhile the engineer responsible for optimizing said backend costs $5000/mo to the company, the Python approach is the obvious correct one.
The first 4.5 minutes are exactly why I code in Nim. It reads and writes like Python, so I'm able to get projects developed relatively quick, and it also has the performance of C++.
i learned python because it's the scripting language of so many softwares i need to make plugins for.
this points out the point I always make.
Python can easily make a reasonably good calculator. If you want to know 50 * 50 you just type 50 * 50.
In Java you would need to do:
public class program {
public static void main(string[] args) {
System.out.printf("%d", 50 * 50);
}
}
which is significantly longer and easier to mess up. This obv has it's values, but sometimes the performance doesn't matter and you may as well just use python.
if you know ahead of time that performance is going to matter, there are ways to write python to be fast e.g. numpy, pypy, numba, or cython.
You don't need the public static and the string[] args bit from Java 21. It has been streamlined considerably.
with type annotations and type linting it's actually quite a pleasure to use and debug python. where performance is needed there are either specialized libraries or you can pretty easily bind/interop to C++ and C code
I used to hate python but then I had to get actual work done and it gets stuff done fast.
I used to like Python and then I had to do actual work and it is the worst experience ever.
@@smallfox8623 You either haven't used Java or you love Java.
The thing about python is that it is really easy to shoot yourself in the foot by assigning a variable the wrong type or by initializing a variable twice, and with no static checks, the only way to prevent this is to write tests, which I don't like doing
This just happened to me but pycharm easily caught that. Now I'm on astrovim so idk.
If you edit your program with the REPL connected to your running program, can you can just run your tiny pure functions to see if they give the output you want.
I think it's very clear: people love shitting on forced indentation because they're the guilty ones, the ones with abysmal indentation practices. And Python makes that a syntax error... **giggles**
Exactly, I consider it a good feature
I just let the editor take care of that and don’t think about it too much in any language. The only exception is in YAML, which sometimes confuses me and the editor.
*giggles* should be a syntax error, that was painful to read.
Python is always a hassle in production! New version of the interpreter that gets install by your OS and your code may break.
Update of modules and your code may break.
The mortal sin started with breaking compatibility with 2.7 (idiots).
Libraries that the module use (especially anything OpenSSL) and your code breaks. *SSL libraries never a dull time (in any language)
You want a statically compiled binary on production that you know your OS or Docker can change and your code will still work. Python also npm (had that last week at the bank) suddenly doesn’t work because a new version of ADF ARM deployment required a new version of NodeJS. Ugghhhh I hate that! Exact same build function call and yet “version mismatch” wtf?! If the interface doesn’t change your code shouldn’t break! Especially for doing the exact same thing!
So for just a tool that doesn’t need to run production grade software use what you want. For production grade software I stand with compiled languages with string typing only! And the binary needs to be statically compiled so is independent from system updates.
But sane people who write Python don’t operate in a mission critical environment (I hope).
Python lambdas in AWS have lightning fast cold-start speed compared to everything else (except Go lol). If you’re writing basic crud apps using micro-services, Python should be at the top of your considerations. Powerful stuff.
C/C++ should have a very fast cold start time
Exactly. I chose Python for AWS due to first class support on AWS and some specific API libraries I needed. I have used Lambda with Step Functions using Map and also Fargate. Great stable performance.
@@mihailmojsoski4202 C and Nim have single digit cold starts in AWS Lambda with custom runtime. Very nice.
In a full stack course I learned JavaScript (the horror) and they did a small segment about TypeScript. they showed us how it's just JavaScript in disguise.
It is a mistake.
I think being able to immediately solve something in a new language is extremely important, much more than performance. That's why I would chose go over python every day. It doesn't hurt that it is also more performant, has better tooling, actual type safety and allows to define data layouts for a low memory footprint.
So, how fast can u make REST project with Golang? or can u make Instagram or Github with Go in 3 days?
@@merlin7411bruh
@@merlin7411 😂
Even as a card carrying python hater I can't agree with this. Go wants me to code like it's 1970. No thanks.
in Python loops do not create a scope inside, you declare a variable inside a loop it will be visible outside
Perl rules, all the problems are already solved and there was a module for them before you found out the problem existed ;-)
To be honest Python is the best everywhere if you don't care on performance. Most of webistes don't count prime numbers, it's just pages with some buissnes logic, and Python here can be awesome.
agree, u should try Ruby, man you code lines will be less then it was
I disagree, when the business logic gets too complex types are there to help.
In my opinion python shines where the complexity of the task is extremely low.
AttributeError: 'dict' object has no attribute 'creation_date'
>when the business logic gets too complex types are there to help.
Types are just crutch, test codes are the ultimate tool to deal with the complexity imo.
@@CamaradaArdi it's not true - there are types in Python and you can execute them. I was developing complexity website in Python so I don't get this point. Also it's obvious that dict has no method like that so again I don't understand the point, every python junior knows it.
@@ANONAAAAAAAAA very true
At one point I thought I'd learn Powershell. I gave myself a basic, useful script as a goal. 5 hours later I gave up and tried using Python instead. Finished it in less than 10 minutes. 10 minutes if you count the polish.
i just tried that and got more stackoverflow stuff than my actual questions .
man helped me get stuff done
Python is a cute toy that accidentally became a loaded gun
Large projects are just fine in Python :-) you use the gained productivity and saved time to implement robust tests, maybe type-hint absolutely everything, maybe have 100% coverage (I do the latter because type-hints aren't necessarily available for all the libs I use, but I'm also writing those on my spare time because I want everything typed, it helps) ; either way you can iterate on the product faster, meaning you also get better feedback, meaning you build a better product and everyone's happy - huge W as you noted!
Note that you have to take 'type hints' by word. These are hints and optional. These do nothing for safety of your running code. Still with hints ...an int can be a string can be a float can be a char can be a chameleon...
If you're using the saved time to implement all the other stuff, then you probably would have been better off with basic java and enjoy a faster application.
There are static analysis tools to make sure your types are sound. It's not perfect but it does help quite a bit.
Yea but then you have to write Java… I fully admit that Python is kinda a really slow crap language but… I would pick Python over Java anyday.
@@scythazz LOL. Agreed. Java is awful.
I am building my no code tool MVP in TypeScript. Later I’ll build the complete thing in Go. And at the end probably in Rust. Seems like a natural evolution. And what I’ve build so far works great!
I think the complete opposite. I've made a Discord bot before. In fact, I wrote the same Discord bot 4 times: the first and second time in JS, the third time in CoffeeScript, the fourth time in TS.
I can tell you, the third rewrite was by far the worst to work with. There were random bugs occurring due to the lack of static typing, and they took hours upon hours of debugging to find. All these bugs vanished when I rewrote it in Typescript. And it didnt even take more time to make it than when I wrote the coffeescript version. It was just better from the get go.
Thats why I cannot see myself loving the experience of making an entire product in a dynamically typed programming language. I'll just write code slower because I'm constantly thinking of types I have to now hold in my head instead.
For me, Rust being fast is a bonus. The *real* benefit is its extremely helpful type system.
Python is a strongly typed language. You can treat it as a statically typed language and have a type checker (called mypy) running while you program. The effect is like Typescript. @Speykious
"typescript gives you illusion you have types"... so does C++ and every other language.
CPython doesn't have a JIT. There is an implementation of Python (PyPy) that does. But PyPy has compatibility hurdles with Python modules written in C.
I used to do everything in c++ but I haven't touched it in like 3 months after I started doing a lot of small projects in python, I can easily develop anything in a small amount of time
this i got bored one day and and started bangin out like 3 big ass text adventure games want to do a text rpg that incorporates random math for dice roll style combat but i digress my love of python is its clean syntax its simple and clean and effective
The last thing I want to care when writing in Python is "performance". Really. Because I know, it may suck. But the language and pip ecosystem is so good, you don't give a f.
As we say in Russian: Хуяк, хуяк и в продакшен. Which means like rough and ready, but it works in production. It's the main purpose of Python.
I don't want to waste time f-ing around types, low-level stuff and compiling. I don't have a time for it, I better spend it elsewhere.
But if my project suddenly shoots out in a wild and I become a star, there're ways to optimise. Like using async, memory caching or rewriting entirely or heavy lifting only in Rust, Go or C, or other... Like pydantic v2 did.
Python is by far my favourite language. Writting code on it just feels pleasing. I'm not using it everywhere just becsuse it poor performance and outdated environment.
If someone would create a go variant with python syntax instead of this bulky mess, it would instantly become my number uno language
My issue is the nightmare with dependencies, python versions etc the fact your basically forced to use conda or something to reign things in is insane to me… upgrading python even at minor versions breaks programs sometimes
Really? I've had worse luck with NPM. PIP rarely gives me any issues.
@@redpillsatori3020 oh npm has its issues as well, but node rarely has an issue where going to a newer minor release breaks backwards compatibility that’s def happened with python
You're not supposed to upgrade the minor version (3.x) without being familiar of the changes, because it's likely for things to break
If you develop with 3.8 then stick with that version, although you could update to 3.8.10 for example
I never use conda, for me it's virtualenv and pip
npm is 999999 times worst than pip
Woahhh, that's a mad enlightening take on typescript
The meaningful stuff does C, C++ or Rust in the background. And you can easily jit compile functions with Numba. Of course ideally you'd just use a fast language without all this need to optimise.
Still much slower than native
@@isodoubIet Not sure about the "much" part. Have you done any benchmarks? The benchmarks I've seen are using native Python to do operations on large arrays, which isn't really done with any serious software.
For some of us, a rewrite in Rust would take years and it's simply not feasible. It's cheaper to simply buy compute power in most instances.
@@incremental_failure "The benchmarks I've seen are using native Python to do operations on large arrays, which isn't really done with any serious software."
Where people go wrong is when they measure the python overhead of their numpy solution and think they're close to native. The overhead of python may be small, but they're not accounting for the overhead of writing a solution nobody would've written natively. For example, if you write something like, a = b * c + d, numpy will create temporaries, whereas a native solution could make use of fused multiply-adds. Imagine doing this in a loop -- you could easily dwarf the cost of the numerical operations just with allocations.
"Not sure about the "much" part."
At my company we beat sklearn by like 10x without even trying. I think that qualifies as "much" faster.
"For some of us, a rewrite in Rust w"
There's no reason to reach for Rust when writing numerical software. C++ is a much better tool for that job and significantly easier to write.
Python has one advantage that is GREATLY overlooked. By being arguably the fastest language to develop in and the slowest language to run. It FORCES you to become extremely crafty with your code optimizations and due to ease of use. lets you trial and error to that point quickly.
I used python to make a brokerage simulator.
first run was so slow It couldn't be used. The current one is so fast I had to add implicit delays to not break rate limits.
Skilled engineers trump the inherent weakness of a language by being better.
good point
Reminds me when I was first learning Python. I would read excel files and manually loop through all the rows/columns with while loops and adding data to lists, which was slow. Then, I learned about Pandas and reading them to a dataframe, but i was still just while looping through the dataframe (still somewhat faster though). Learned about for loops around this time, instead of while loops. Then, I learned I didn't need to do loops at all, I could use pandas comprehensions on the dataframes and I also learned about vectorization methods on those dataframes. This sped things up MASSIVELY.
I love python. Never cared what other people would say... I get a lot done when using it.
I don’t love it myself, but you hit the nail on the head “I get a lot done using it” if you do that & manage to not create a mess in the process, it’s the perfect language for the job
I'm learning F# and I think there is no need for me to learn python. It is succinct, static typed with the feeling of being dynamic and it is fast. This is my third language after PHP and Js and I love it much more.
I'm not a very good programmer. I've mainly only used python, but have fiddled around with javascript and C a little. Although the other day I wanted to try something and I was thinking to myself while programming, "Python is like a sketchpad that I can just make doodles in". That ease of use and flexibility really gives even a dumb shit like me the ability to learn and accomplish things.
I love types now, but I used to do a lot of Ruby on Rails, and sometimes miss it...I could bang out a fully functional app (front end, back end, db, authentication, multiple data models, robust data validation and error handling) in a day or two.
I spent most of yesterday debugging generic function signatures in TS 😶
Same with django. Django is not easy to deploy tho
@@rozennrd4802 Yeah I had a good experience with Django too, the built-in admin console was a great idea IMO. I've been meaning to try fastAPI, looks like maybe a happy medium between type safety + efficiency
I mainly write Python. What I like about the type hinting in Python is that you can *choose* whether to sprinkle type annotations on your code and the common interpreter (CPython) does not enforce it, which means adding type annotation is just completely for your own good and readability. I am also a "typed" guy and this really boost my productivity. Of course if you like to get completely restricted like in TS you can use static type checker like mypy.
On the other hand if you want to write TS and not be totally restricted, you can use a lenient tsconfig.
Type _Annotations_ are such a good feature. There's zero value in writing adapters and converters just because "foo.bar.Person" is technically a separate type to "bar.foo.Person". Technically that's correct, but practically it's irrelevant (in the sense of a running program fulfilling business processes). Annotations provide all the good stuff, the IDE support, the documenation, without all that adapt/convert BS. But, just in case you like that sort of pain, there are ways to actually use type-enforcers and this kind of choice is great!
Python v3 with type annotations makes it way easier. I'm no Python or TypeScript fan, but I enjoy earning an income.
my short experience with python for nontrivial projects is that indeed it allows you to move quickly.
The biggest flaw for me is the fact it has exceptions, in combination with poor documentation.
Program can fail at any point due to some exception raised by a 3rd party component or even std modules that do not specify exactly which conditions generate which exception.
There are many other problems, but this is the worst for me.
Now back to writing python code..
I hate range in Python. It makes ZERO sense. Why would you write range like this. "I need 0 till 5 excluding 5, so I ACTUALLY need 0 till 4. Let's pretend like this makes sense."
exclusive range?? skill issue
How to out yourself as having never written a C style for loop in your life
5:17 - 5:36 is by far the best content PrimeDaddy has ever put out there. Ive been dying for 5 minutes and its not my ring tone.
python is my fav language tbh
C, Rust and GO are my next 3 lol
I feel one important aspect is missing in the video and the discussion. It is cross-platform. It takes some effort to do that with C/C++. Maybe not relevant for a Discord bot.
If a Python program seems slow, most of the time it is because of the way the developer thought it had to be done.
For instance, multiple levels of loops inside each other and looping over strings and searching for a keyword or adding it individually to another string which you would maybe do in C.
It takes some time to get an intuition for what things are slow in Python but then you can avoid them and use appropriate built-in types like linked lists and hashmaps which are way better to work with in Python than buffers pointers and structs or classes.
Python does not suck. Python is one of my favorite programming languages/environments. I have yet to find a perfect language.
Try nim
name me reason it does not suck
here's my reasons why it sucks:
- terrible performance
- types can unwieldy the longer your program gets and the type checkers are very faulty at times
- the GIL makes python play pretend that it's actually running on multiple threads but that actually never happens
- optimizing python means reducing the amount of time your program is python, so you just end up installing and using a bunch of C libraries and if u do any sort of loop in python it slows your program down, this also means you have to spend time with a profiler checking where stuff is running on python land
- bc of the way python is designed, if you copy a piece of code and u miss a single space your whole program will have a logic error and depending on where that space is that error might not even cause a easy to spot error so you might unknowingly be running a buggy program for a while until you get burnt by it
- python abuses decorators and class inheritance to achieve some basic functionality available in other languages such as ordered dictionaries, enums, static methods, etc etc
- package versioning is hell, you have to use the hacky (aka retrofitted) virtual environments and often times some other package manager like poetry cause pip just installs the library for the user or globally so you can break your whole system by updating a single package, which means a lot of distros end up shipping their own python libraries separately to avoid the user breaking their system
- you simply cannot have ur program bundled from one system to another, there are "options" but they're all either dependent on you having pip installed (wheel) and will break if your python interpreter is the wrong version or you have to use something like pyinstaller or equivalent which result in big binaries that also take a while to load and might result in bad performance overall
- expressions and list comprehensions are not optimal, the `x if x % 2 for x in range(100)` or whatever is just painful, if you need a complex set of expressions they're basically useless as they hurt readability, you might as well just use regular statements by then
- lambdas are very lacking, lambdas in python are very limited, their inclusion changed very little in python, unless you're doing very simple maps, reduce, etc they're not very good since to write a more complex lambda you'd have to use the very sucky expressions and list comprehensions
@@FlanPoirot There are tradeoffs with each programming language. If you use it properly for the use case it is designed for, then the language is good. Every language sucks at some point. You just listing some negative points does not mean the language sucks. It just means it is the wrong tool for you. I use Python since over a decade and it does not suck (but it does at some specific points).
- performance is not terrible, it depends what you do with it
- types is a weak point of the language, but a programming language does not consist only of types
- threads, argument about GIL does not matte if it provides real benefits
- optimizing Python does not mean reducing being Python, there are many ways to optimize, not only the one way you describe
- buggy code is not the fault of Python, but your lack of testing
- that Python uses decorators and inheritance to achieve basic functionality of other programs is a bad thing?
- package versioning is a problem with almost every programming language, especially those of interpreted ones, but that is a problem if you are clueless and careless
- list comprehensions are good and expressions might not be optimal, it's not a problem
- lambdas are special cases, not a reason to avoid the language
Looks like you are a Python hater. That's fine. Some of your points are nonsensical. I recommend to watch the video, so you understand why Python does not suck as people tell. I could write an essay about every language why it sucks. Python is a fantastic programming language with some weakness, but provides a lot of good stuff too. I have yet to find a programming language that is perfect and have no weak points.
If you use Python for every possible use case, then its not the fault of the language, but its your fault to use the wrong tool. Rust, Haskell, Go, Zig, C, C++, each of these language suck. So we should stop using them? Which language does not suck in your opinion?
There's the concept of absolute speed and relative speed. And by relative I mean not relative to other languages but to fast enough performance. You can have projects where Python is comfortably fast enough and anything faster than that makes sense only if you can develop it faster. You might have projects where Python is not fast enough.
Python all by itself is a nice and easy language to play with that lacks both the LISPy flexibility and the accidental chaos of JS.
Python in professional practice, on the other hand, is an entirely different beast: it is an extremely productive runtime and build environment for blazingly fast modules written in C that would be horrible to use and combine inside C itself. It brings high-leven sanity to projects built around lots of C code. It thus delivers what C++ has once promised but never achieved.
Oh, I wholeheartedly agree that perl is winning in terms of how long it takes to solve a problem. It's just that the solution becomes a problem in of itself after an approximate 3 days of not having looked at the code, so while perl is fast at solving problems, it is by no means not capable of exacerbating the throughput of problems you need to solve just by its own nature.
Python is so easy to develop in. It took me a month to build a partially working game_engine. It’s taken maybe about 6 months to write a similar project in c++
Python is beautiful.
Python is like an ultra high tech futuristic mega rail gun, mounted on a lego block base. The language itself is hot garbage, the libraries is what makes it powerful
Cython has both optional static typing and faster performance than Java and C++. You can achieve up to 10x faster runtime with it compared to Java and C++. The comparisons depend on what you're doing of course, but I've measured it first-hand.
There's even JIT versions like Numba, PyPy and there's Taichi for CPU/GPU-specificity and parallel computation.
PyPy got its data science libraries covered recently, so it has very good coverage now compared to in 2019.
Even vanilla Python is making improvements; 3.11 was 35% faster than 3.10. And 3.12 was 4% faster than 3.11.
I still keep C++, C and Golang in my toolbelt for enterprise software, but there's no beating Python's development speed, prototyping, low verbosity and ease of use for unskilled teams. And for data science, there's no better language imo (R is too inflexible, less libraries and even slower AFAIK).
If Python was to take Cython as its standard distro, simplify that syntax a little bit, and add static typing - no other language would be able to parry.
Could you explain what you mean by, "simplify that syntax"?
@@DylanBeaudry Make wrappers for the C variable declarations, so it's less verbose and more Pythony.
I did not know Prime speaks Parseltongue. 5:20
Had a job interview with a company from the tourism industry that was planing the build a flightsearch backend with Python. They asked me about the Global Interpretator Lock in Python... never heared again of them
I love python for standing up solutions very quickly for a temporary problem. If you have to maintain a python code base for a long time it starts to be a bummer.
13:00 What in the runic inscription imbued enchanting bullshit is that code???
1. Certain implementations of Python have JIT. The mostly-used implementation, CPython, does not. PyPy does. There is also Cython for ahead-of-time compilation
2. Python has optional typing. Because my company doesn't use TS, I get more static type safety in Python than I do in JavaScript 😂I maintain a handful or python projects, and all of them are >95% statically-typed.
3. Polars, a Rust dataframe library with Python bindings, is really cool -- blazing fast (rust) and succinct (python)
we use typescript because the editor we use is not intelligent enough to give us what type of variable it is when we are using a variable declared on another file