Dude, I am SO happy I found this channel a couple of days ago. These are exactly the kind of in depth discussions I've been searching for. It's the education I need at my skill level as a professional programmer. Thank you so much for doing these videos!
@ArjanCodes I need a book to study all these nice coding principles and unwritten rules of efficient python programming. Is there any reference out there?
Literally the best channel about Python Software Engineering! Right place to gain theoretical knowledge, proper mind-set, and last but not least, the experienced developer's line of reasoning. Thanks, I firmly appreciate your work and contribution!
this will be at position #1 on Google for protocol vs abc. No matter how many searches I have done, nothing explained so clearly like this video. Thank you, Arjan!
The one advantage I see for protocols is cases where you might have overlapping function needs in multiple places which may be implemented by the same class. An example might be a tune up shop class that can perform services on anything with an engine that has a vehicle operator (car, motorcycle, tank, etc). Then, you have an entirely different class that is a tire shop class which can perform service on anything with tires that has a vehicle operator (car, motorcycle, bicycle, etc). In this case, a car can be serviced by the tune up shop or the tire shop but inheriting from both ABCs would cause a collision on the vehicle operator functions. On top of that, we can't have one combined ABC. Doing so would force the bicycle to implement all functions to meet the tune up shops needs and force the tank to implement all functions to support the tire shops needs, which wouldn't make sense since the bicycle can't go to the tune up shop. With protocols, these issues wouldn't arise because each shop would just require the things it needs in order to perform service.
Thanks very much for the clear explanation! I was searching for the difference between ABC and Protocols but was unable to find such a clear explanation!
One thing I find is often overlooked during the discussion of ABC vs Protocol, is that a Protocol is implemented as an ABC. (Technically, it's an extension of ABCMeta and ABC is just a helper class to simplify and keep code looking cleaner). Nothing prevents you from creating a class that extends Protocol and defining abstract methods as you would on an ABC - you will get the same behaviour and in fact is how the default protocols are defined within the typing module itself. This means depending on the exact use-case you can use either and/or both techniques within a single class definition.
this makes a lot of sense and it's actually what I was imagining from the point of view of implementing something similar to protocols in a language like C++: you could do it by redefining the same data structure in the various compilation units and then use this redefinition as a normal abstract base class. This also fits nicely with python's "same structure" concept - of course paying attention to padding if you're redefining it just partially or if you have more levels of inheritance. Maybe the parallelism is not quite correct, for example I feel that what @Vivek said about type traits is more technically precise (I'm not expert on traits), but it's a way to see things I guess :D
Protocols are exactly the sort of solution that is needed for proper interface declarations in large systems with complex interactions. Fantastic. Thanks!
Edit: I wrote my comment before playing the video up to the end, where Arjan mentions both ISP and Uncle Bob. But I decided to leave the comment as is. Anyway, good job Arjan! What happens here when Device abstract base class was replaced by two protocols seems like "I." from "S.O.L.I.D." principles. Namely "ISP = Interface Segregation Principle". And hey, Python allows doing that without too much strict typing, but rather with "duck typing"! Depending on the case this may be handy. And sometimes when more info is necessary when implementing a new class (to avoid mistakes ofc.) - would be to stick to ABC. Anyway, thanks for informative video 👍
Protocols seem to be a great way to achieve the Interface Segregation Principle of SOLID. Probably I'm going to use it more instead of only relying on ABCs. Thanks for the amazing video!
@@ArjanCodesI need a book to study all these nice coding principles and unwritten rules of efficient python programming. Is there any reference out there?
The more I use mypy and typing in python the more I like it! It really helps me when writing functions or methods to know in advance what parameters I am expecting and what I do have to return. And, of course, sometimes catches logical errors, too, before running, which is great.
I'm still not convinced protocols are better than ABC's when used in the way shown in the video. I will address the 2 main points: 1) Removing inheritance and imports: This removes the explicit relationship between the parts of code, and turns it implicit. You say it reduces coupling; yes in the actual program, but it only moves the coupling into the brain of the programmer, as they have to know about which protocols the class should implement. I find this bad because then you are relying on the programmer to ensure the protocol is correctly followed, and computers are better at catching errors than people. 2) Splitting up the protocol and multiple inheritance: Splitting up the interface can be done with ABC's, but this is supposed to be bad because multiple inheritance is then needed. I don't see a problem with multiple inheritance though. Sure it can get messy quick if the defined ABC's are bad, but the same is also true for protocols. Consider functions for a second; should we opt for global variables if we need more than one input variable? It "reduces coupling" because when it is called you don't need to know anything about the function or the input variables. I feel that many people would disagree with using functions like this. I will say however, that the points made from 20:59 are good use cases for protocols, and are the most compelling reasons I've heard for using them
I strongly disagree with (1), the programmer doesn't need to remember the interface, it is done by the type checker. With (2) I disagree just a bit, because you could use functools.partial and closures to remove the need for global variables.
@@sebastianrodriguezcolina634 yop, the type checker will do its job, but imagine you'll work in team of several people (some will quit the job, some new will arrive). When using ABCs it's always clear what "interface" the class implements. When using Protocol it's not so clear at first glance. The type checker or python interpreter gaves you hint eventually, but otherwise you must keep track of the interface implementations somehow (in your head when working a alone, in some shared documentation when working in the team)
I think of Protocols and ABCs as changing the ownership of the definition. ABCs are owned by the implementation, the person creating the class has to know, at the time the class is created, that they want to implement that ABC - it reduces the complexity of implementing consumers as you can be quite rigid in your ABC. A Protocol on the other hand is a declaration by the consumer of what they're able to consume. A Printer class may be able to consume any object which has a `print` method which returns an `str`, and therefore it defines a `Printable` protocol. The authors of the classes which the Print class consumes didn't have to realise that they were implementing the protocol. In both cases the method implementation has to do the right thing, and no typing mechanism is going to help assert functionality beyond the correct parameter and return types.
This video brought attention to the letter board for this new subscribers. Now we need to go back to find your hard work! Thank you for the great content. Take care
Great explanation of the use cases of these two Python features. Really appreciate your channel and the clear way you go through the logic of your code and programme design.
Hi Arjan! Thank you for this excellent video and example. I think that you need multiple protocols in this example is a symptom of violating the single responsibility principle in the device class in the first place. The device class clearly has more than one responsibility. It actually reminds me of Uncle Bob's modem example where he shows how single responsibility violation looks like. The modem has the functionality "connect" and "disconnect" (responsibility 1) and "send" and "receive" data (responsibility 2) if I remember correctly. I think this example can be found in his book about agile software development. Anyway, great job!
Great video as always, Arjan! Although I would argue for a stronger conclusion: I feel that abc IS obsolete. First, you totally can explicitly use protocols as a base class (class Animal(Protocol): ... ; class Cat(Animal): ...). And second, you can even have a default method implementation in a protocol (although, I strongly advise against it). With that, abc doesn't seem to have any advantages in practice. I switched to protocols completely and don't regret it (more details if you google “sinavski abc vs protocols”).
Nice article, but relies a bit too much on external resources for context, rather than providing the context directly. External resources are prone to link rot, robbing the reader of your article of the required context. They also require a context switch: the reader needs to open the link and read the (often lengthy) article, which probably covers far more than the limited coverage that understanding of yours requires. Hopefully it doesn't also reference external resources of its own, or the reader will have absorbed dozens of overlapping articles before they're ready to return to yours. And then the reader needs to come back to your article, offload the discussion from the external article, and pick up the train of thought that was interrupted when they were sent off elsewhere. I think it's better if information is self-contained. Referencing external resources for a more in-depth treatment of the context is OK, but the parts that are necessary for a proper understanding of the primary article should be _in_ the primary article, even if in shortened form and lacking otherwise important nuance that just isn't relevant in the context of the primary article. Also, links where the anchor text is just "here" are useless once the link target disappears. The reader has no idea what the title was, who the author was, and will be unable to retrieve the information even if it did survive elsewhere, as they have zero clues on what to search for. And that's something I just realised I should also start applying to my own writings, as I tend to do the same. 😂 Anyway, don't take this as a "you suck" criticism; it is meant in a constructive way. If I thought your blog post was not worth my time, or if I didn't appreciate your effort, I wouldn't have bothered responding. I love it when people freely share their knowledge, moreso when they're knowledgeable about the subject and cover things that go beyond the introductory matter. Your blog post was certainly appreciated.
A truly excellent video, as always, you've gone way above and beyond simply explaining the concept, and outlined excellent use cases and way of thinking to know when to pick which one. thanks again, Arjan :)
I think Protocol is meant to define an interface and use it only as type hinting for arguments. Similar to Iterable, if an object has a couple of dunder methods, it means it adheres to the Protocols ABC should be used to define relationships between superclasses and subclasses as mentioned in the of the video
I agree, and it goes beyond that IMHO ; sometimes it is impossible to accurately define a function with pure type hinting (say when a method takes a coroutine as argument), and you can instead implement a Protocol's `__call__` method to accurately describe it.
I was literally debating with myself which of the two will be more suitable for a project I have at work when I opened TH-cam. Impeccable timing and excellent video 👌🏽
Very interesting, this is exactly a paradigm for a driver collection/service that I hacked very poorly together to test a piece of hardware with the intention of wrapping that driver up in a connection management service. This a most fascinating look at a much better way of handling my idea. Thank you very much for the work, not just because it hit on a topic that was directly interesting to me but because you've done such a great job of making videos on non-trivial topics in general
Another extremely informative video, thank you. Did not expect being able to split up Protocols like that, but it makes total sense. Will definitely be incorporating them more now.
Protocol seems more suited for distinguishing _traits_ with _trait objects_ . For example, you might define an interface that's really just an extension for objects like BeautyPrint with one method beauty_print() that just beautifully prints the object. Now, this isn't really meant to establish a hierarchy of beautifully printed objects so much as it gives an extension methods to those who need it. I don't think anything useful comes out of defining a function that takes a BeautyPrint. That is, BeautyPrint might be a _trait_ (interface), but it's not a _trait object_ (doesn't represent a bunch of polymorphic types). Protocol might help prevent it from being a _trait object_ if you so choose.
It looks like there is a lot of benefit to using Protocols over ABC, however, I still _really_ want to see Python supporting some way of explicitly defining that a class implements a Protocol, similar to TypeScript's Interfaces (I believe you've mentioned this in a previous video; it is still just as valid). Also, I'd (personally) like to see type checkers raise warnings about missing methods/attributes from ABCs/Protocols at a class' definition instead of at the class' usage elsewhere in the code. If I forget to return a value in a statically-typed method, I expect to get a warning in the method definition, not when I attempt to call the method elsewhere in the code. There may be implementation issues for classes using methods from multiple files (which I've only ever read about), but I'm sure that's its own can of worms when it comes to best practices.
Interesting video - Protocols seem like a good way to formalise duck typing and effectively introduce interfaces into Python by the back door. It might even get me using typing more! The only thing I don't like about them is the ... notation - this joins things like dataclasses (when is something a class variable and when is it an instance variable?) and properties in stretching the syntax of classes in ways that get hard to explain to beginners. I've even read the Protocol PEP before but it didn't click with me that ... was actual syntax rather than just shorthand in their examples for code they hadn't written!
... and after posting that I've now read several articles about ... - I didn't realise it had such a long Python history (introduced as the singleton Ellipsis in 3.0). Perhaps because I don't use numpy, which seems to be the only place it was used before typing and protocols. I like the anecdote that it was introduced as a constant because they thought it looked 'cute'!
Love this video. Thanks for touching on the differences between the two and showing examples. :) Love your Python videos. I code primarily in Python and in Golang. Wish there was a teacher like you for golang! It's an untapped market imo.
Protocols are helpful in adhering to the Interface-Segregation principle by allowing us to define multiple interfaces to be used, without using multiple inheritance. ABCs can also allow us to create multiple finely-defined interfaces but we will have to use multiple inheritance to make it happen.
ABCs allow for type-checking at the time of object instantiation due to inheritance whereas Protocols allow for type-checking at the time of calling the object's method. Note that Protocols can also be inherited.
Use ABCs when you need some general method implementations in the base class that can be inherited and used in the sub-classes. Use Protocols when you need pure interfaces with only abstract methods without any implementation.
i have to admit i hadn't noticed the letterboard.. i'm sorry! love your content. it's really helped me stay motivated. And helped my coding, details and structre. thank you so much.
Protocols are mirrored by Type Classes in functionsl programing Type classes are a powerful feature in functional programming that can help solve the expression problem of adding new types to which existing operations apply. In functional programming, the expression problem refers to the challenge of adding new functionality to existing code without modifying that code. This is particularly difficult when dealing with types, as adding a new type often requires modifying existing code to support it. Type classes provide a solution to this problem by allowing you to define a set of operations that can be applied to any type that implements the type class. This means that you can add new types to your code without modifying existing code, as long as those types implement the required operations. For example, let's say you have a type class called `Printable` that defines a single operation called `print`. Any type that implements the `Printable` type class can be printed using the `print` operation. Now, if you want to add a new type to your code that can be printed, you simply need to implement the `Printable` type class for that type. You don't need to modify any existing code that uses the `print` operation.
It's nice to see your vim usage getting better over time. Should be able to `da(` without actually having the paren selected, btw. Then you could hit `.` to repeat the command. And if you used / to find the open parent, you could then `;.;.;.;.` to repeatedly find and delete the class inheritance syntax.
Hello Arjan. Thank you for the quality of your videos and all the knowledge that you dissipate. This video is from a year ago but you look younger (I prefer you old [😎])
There's nothing wrong with multiple inheritance and this is precisely the kind of situation where it's useful. The decision between structural and nominal typing should be one of interoperability: do you want classes that implement the required interface to work with your code, or would you like to keep things controlled and open access only to those that explicitly opt in? You may want different things in different cases.
whoa... i thought protocals were just a fancy way to annotate the type of a variable or parameter containing a function... by typing the __call__ method of the protocal you can do some things that the typing.Callable[...] syntax can't (or maybe thats fixed now?)... This is a much more sane use of protocals. Thankyou for the excellent video as always, you are amazing at explaining clearly!
Arjan... I really enjoy your videos. Thank you for sharing your knowledge! Would you consider doing a video about Mixins? I've seen it is largely used in Django framework, and I would like to know more about best practices about Mixins.
Hey all cool, thank you so much for teaching us more than happy to give you the views, main focus is programming so hey thank you Arjaan I love the Netherlands btw tiptop thumbs up from a Norwegian
Hi David, I also use Pylance, but you have to make sure that your typeCheckingMode in VSCode's settings is set to 'strict'. See also: code.visualstudio.com/docs/python/settings-reference.
Is there a way to declare that your class satisfies a Protocol, for the sake of future developer groking the design? I guess it would require an import of the Protocol class / schema. I guess that may just be where naming convention or documentation comes into play?
Finally this is what I've been waiting to fully understand the use and importance of abstract and protocol. What error highlighting you are using? I'm using pylint but it's not showing like that specially on unused imports
I’m using the Python extension in VS Code (Pylance). To get the type checking messages, make sure that typeCheckingMode in your settings is set to ‘strict’. See also: code.visualstudio.com/docs/python/settings-reference.
i have interface created in python using protocol but when i click to any method in vs code it does not open main method but it opens method in protocol / interface class, how can i click to open main implementation instead of interface ?
I took me a couple of minutes but I got it. Protocols are like interfaces in Java. Except in Java you have to explicitly implement an interface on an object. In python if the class has certain methods it can fulfill a protocol aka implements an interface but nothing tells the poor developer that a class actually realizes a protocol. It looks like a bunch of unrelated objects which somehow work together. The correct way would be IMO in this example to define two ABCs, Device and DiagnosticSource and a specific device should inherit from both. That way the connection would remain clear between the classes yet anything could inherit from one or the other. As I see with Protocol the code could be simpler at first glance but in reality it become more complicated and much more harder to read. Other problem is if a class has a certain method it fulfills a protocol when that class is never meant to do that. In the example, lets say a message queue adapter easily can pass as a device cause it probably has connect, disconnect and send_message methods. Protocol is a shady stuff!
You should split the intro chapter into 2 chapters - 1 for talking about letter-boards and one for the actual intro - not that it was a bad segment to include, but people should be able to skip it IF they dont care about it or if they already heard about it because they are coming back to this video for a second time
Love your videos, this clarified a lot of things I didn't understand about protocols. They seem to be great for splitting up coupling and being sure each part of the program only knows exactly what it needs. I have a question, though: Do protocols provide things other than type hinting? Can they raise errors or prevent vulnerabilities in other ways? I was surprised that Python didn't raise an error when you run register_device, for instance, until it actually ran into that missing connect method. Though the type hinting is very helpful in an IDE. Thanks!
Hi @arjanCodes, I'm curious to know what course you provide includes the amazing video, Protocol Or ABC In Python - When to use which one?, I just watched above. Is this part of "The Software Designer Mindset" course?
Great content, Arjan. I'm learning a lot from your videos. I'd like to ask a question though. What should I do if I want to support different device objects from different external libraries? Let's say, for example, that they all implement a connection method but with different names, like "connection()" or "connect_to()".
One place where Protocols have been making me a happy programer is by helping typehints be more meaningful in my programs. I used to have some ugly unions that seemed arbitrary. Now my unions are meaningful! Ow, about the letter board... good call! They were cool, but far from what makes these videos stand out, imo.
Hi thanks for the vedio. May I know how do you do the following via shortcut which is quite useful? Thanks! 1. Replace pass with 3 dots ... 2. delete (device) in function.
I suspect that Arian is using a ‘vi’ type plugin. In vi, to change a word, you simply put the cursor at the start of the word and type ‘cw’ and the new word
Great video. But i have a question about HueLight as protocol how is it different from any other class? I wonder if the inheritance from Device was correct if we do not expect the abstractmethods from Device to be mandatory for it.
Great video! very insightful I have a question: is there any problem if the same method have different arguments on different subclasses of a protocol? or should I be sticked to use the same arguments?
Thanks! Regarding your question: I would definitely avoid that, otherwise what would be the purpose of defining types if the subclass doesn't adhere to it?
@@ArjanCodes it is a “player” class (the protocol one) and for a “human” (subclass) being one method uses the UI to handle human input but for the “bot” subclass that exact method requires some data that passess through the method but not the UI… I’m just learning about OOP and I’m designing simple games onto which I want to try all those new things.
Do you use some kind of plugin or extension for the highlighting? I cloned your repo and commented out the "connect"-method in Huelight as you do but I do not get any notice at all that the HueLight is not following the expected Protocol. I only get an error when running saying: "AttributeError: 'HueLightDevice' object has no attribute 'connect'"🤔
In the protocol code, if I implement the device class as a normal colon 'class Device:' instead of 'class Device(protocol)'. The code can still run without any issue. Besides the IDE don't do static typing check before running. Then why we use the 'from typing import Protocol' library?
From experience. It seems protocols are for objects you do not control and ABCs are for objects you do control. Example. how do you type hint an sql connection? It can be SQLite or Postgres or MYSQL. libraries. You can type the the connection as a protocol of the methods you use instead of the concrete connection
All other issues aside, are there any performance issues between the two methods. Are the method arguments checked each time the methods are called? Thanks
Content was so useful for me. but I have one question. As you mention the protocol is the natural way in python. Why we can not have interface for each of instance of each protocol. In your Device abstraction class some how we can say that you violate interface segregation in SOLID principle. Somehow in Protocol some of the dependencies are hide ?
I watched this video now a few times and it is really great. Now I have a question relying on the Interfaces: How would I visualize the optional diagnostic Interface in the UML (for example in mermaid)? I am on my way to understand design patterns and principles but this would be good to know for me. I also ask myself how to keep track of the different protocols in large projects (in my job we have a Django application with many….many apps)? If the question is too complex to answer maybe an idea for an advanced mermaid tutorial? ❤ Thank you for your great videos!!!
Get my FREE 7-step guide to help you consistently design great software here: arjancodes.com/designguide.
Great content, Arjan. I've been trying downloading the design guide that you mentioned, but sadly the link from the email doesn't work.
Today, accidentally, I was searching "what is the difference between protocol and ABC" in google. And now I have this video! What a lucky day.
I posted it just in time ;)
Big brother is watching you …
So Google does use our data to improve our lives!
How do you aciddentaly search something?? 🧐
Dude, I am SO happy I found this channel a couple of days ago. These are exactly the kind of in depth discussions I've been searching for. It's the education I need at my skill level as a professional programmer. Thank you so much for doing these videos!
feels strange that I cannot find these info using just google.
@ArjanCodes I need a book to study all these nice coding principles and unwritten rules of efficient python programming. Is there any reference out there?
Literally the best channel about Python Software Engineering! Right place to gain theoretical knowledge, proper mind-set, and last but not least, the experienced developer's line of reasoning. Thanks, I firmly appreciate your work and contribution!
Thanks so much Mateusz, glad it was helpful!
Yes
Totally agree.
this will be at position #1 on Google for protocol vs abc. No matter how many searches I have done, nothing explained so clearly like this video. Thank you, Arjan!
The one advantage I see for protocols is cases where you might have overlapping function needs in multiple places which may be implemented by the same class. An example might be a tune up shop class that can perform services on anything with an engine that has a vehicle operator (car, motorcycle, tank, etc). Then, you have an entirely different class that is a tire shop class which can perform service on anything with tires that has a vehicle operator (car, motorcycle, bicycle, etc). In this case, a car can be serviced by the tune up shop or the tire shop but inheriting from both ABCs would cause a collision on the vehicle operator functions. On top of that, we can't have one combined ABC. Doing so would force the bicycle to implement all functions to meet the tune up shops needs and force the tank to implement all functions to support the tire shops needs, which wouldn't make sense since the bicycle can't go to the tune up shop. With protocols, these issues wouldn't arise because each shop would just require the things it needs in order to perform service.
Thanks very much for the clear explanation! I was searching for the difference between ABC and Protocols but was unable to find such a clear explanation!
Glad it was helpful!
One thing I find is often overlooked during the discussion of ABC vs Protocol, is that a Protocol is implemented as an ABC. (Technically, it's an extension of ABCMeta and ABC is just a helper class to simplify and keep code looking cleaner).
Nothing prevents you from creating a class that extends Protocol and defining abstract methods as you would on an ABC - you will get the same behaviour and in fact is how the default protocols are defined within the typing module itself.
This means depending on the exact use-case you can use either and/or both techniques within a single class definition.
Good point, Stephen!
this makes a lot of sense and it's actually what I was imagining from the point of view of implementing something similar to protocols in a language like C++: you could do it by redefining the same data structure in the various compilation units and then use this redefinition as a normal abstract base class. This also fits nicely with python's "same structure" concept - of course paying attention to padding if you're redefining it just partially or if you have more levels of inheritance.
Maybe the parallelism is not quite correct, for example I feel that what @Vivek said about type traits is more technically precise (I'm not expert on traits), but it's a way to see things I guess :D
@@ArjanCodes would you make a video on this?
Protocols are exactly the sort of solution that is needed for proper interface declarations in large systems with complex interactions. Fantastic. Thanks!
Edit: I wrote my comment before playing the video up to the end, where Arjan mentions both ISP and Uncle Bob. But I decided to leave the comment as is. Anyway, good job Arjan!
What happens here when Device abstract base class was replaced by two protocols seems like "I." from "S.O.L.I.D." principles. Namely "ISP = Interface Segregation Principle".
And hey, Python allows doing that without too much strict typing, but rather with "duck typing"!
Depending on the case this may be handy. And sometimes when more info is necessary when implementing a new class (to avoid mistakes ofc.) - would be to stick to ABC. Anyway, thanks for informative video 👍
Protocols seem to be a great way to achieve the Interface Segregation Principle of SOLID. Probably I'm going to use it more instead of only relying on ABCs. Thanks for the amazing video!
Thanks Paulo, happy you’re enjoying the content!
@@ArjanCodesI need a book to study all these nice coding principles and unwritten rules of efficient python programming. Is there any reference out there?
This channel became my go-to channel when I need some deep discussion. Excellent 🔥🔥🔥🔥
The more I use mypy and typing in python the more I like it! It really helps me when writing functions or methods to know in advance what parameters I am expecting and what I do have to return. And, of course, sometimes catches logical errors, too, before running, which is great.
Dear Arjan, great thanks for your videos. That is one of the few channels that really help me upskill as a hobbyst python dev
Thank you so much!
I'm still not convinced protocols are better than ABC's when used in the way shown in the video. I will address the 2 main points:
1) Removing inheritance and imports: This removes the explicit relationship between the parts of code, and turns it implicit. You say it reduces coupling; yes in the actual program, but it only moves the coupling into the brain of the programmer, as they have to know about which protocols the class should implement. I find this bad because then you are relying on the programmer to ensure the protocol is correctly followed, and computers are better at catching errors than people.
2) Splitting up the protocol and multiple inheritance: Splitting up the interface can be done with ABC's, but this is supposed to be bad because multiple inheritance is then needed. I don't see a problem with multiple inheritance though. Sure it can get messy quick if the defined ABC's are bad, but the same is also true for protocols. Consider functions for a second; should we opt for global variables if we need more than one input variable? It "reduces coupling" because when it is called you don't need to know anything about the function or the input variables. I feel that many people would disagree with using functions like this.
I will say however, that the points made from 20:59 are good use cases for protocols, and are the most compelling reasons I've heard for using them
Exactly my thoughts!
Good points!
I strongly disagree with (1), the programmer doesn't need to remember the interface, it is done by the type checker. With (2) I disagree just a bit, because you could use functools.partial and closures to remove the need for global variables.
@@sebastianrodriguezcolina634 yop, the type checker will do its job, but imagine you'll work in team of several people (some will quit the job, some new will arrive). When using ABCs it's always clear what "interface" the class implements. When using Protocol it's not so clear at first glance. The type checker or python interpreter gaves you hint eventually, but otherwise you must keep track of the interface implementations somehow (in your head when working a alone, in some shared documentation when working in the team)
I think of Protocols and ABCs as changing the ownership of the definition. ABCs are owned by the implementation, the person creating the class has to know, at the time the class is created, that they want to implement that ABC - it reduces the complexity of implementing consumers as you can be quite rigid in your ABC.
A Protocol on the other hand is a declaration by the consumer of what they're able to consume. A Printer class may be able to consume any object which has a `print` method which returns an `str`, and therefore it defines a `Printable` protocol. The authors of the classes which the Print class consumes didn't have to realise that they were implementing the protocol.
In both cases the method implementation has to do the right thing, and no typing mechanism is going to help assert functionality beyond the correct parameter and return types.
listened to this again for lunch and already applied protocols succesfully for the first time:)). thanks again Arjan.
Great to hear!
Your video finally made it clear how these two approaches differ, and why protocols are often the better choice. Thanks!
You’re welcome, glad it was helpful!
This video brought attention to the letter board for this new subscribers. Now we need to go back to find your hard work! Thank you for the great content. Take care
Thank you, glad you like the videos!
most of Python tutorials are basics... your channel is very pleasant discovery ;-)
best channel with the best content available with the best coder
Great explanation of the use cases of these two Python features. Really appreciate your channel and the clear way you go through the logic of your code and programme design.
Thank you Joel, happy to hear you like the channel!
Hi Arjan! Thank you for this excellent video and example. I think that you need multiple protocols in this example is a symptom of violating the single responsibility principle in the device class in the first place. The device class clearly has more than one responsibility. It actually reminds me of Uncle Bob's modem example where he shows how single responsibility violation looks like. The modem has the functionality "connect" and "disconnect" (responsibility 1) and "send" and "receive" data (responsibility 2) if I remember correctly. I think this example can be found in his book about agile software development. Anyway, great job!
Great video as always, Arjan! Although I would argue for a stronger conclusion: I feel that abc IS obsolete. First, you totally can explicitly use protocols as a base class (class Animal(Protocol): ... ; class Cat(Animal): ...). And second, you can even have a default method implementation in a protocol (although, I strongly advise against it). With that, abc doesn't seem to have any advantages in practice. I switched to protocols completely and don't regret it (more details if you google “sinavski abc vs protocols”).
Nice article, but relies a bit too much on external resources for context, rather than providing the context directly. External resources are prone to link rot, robbing the reader of your article of the required context. They also require a context switch: the reader needs to open the link and read the (often lengthy) article, which probably covers far more than the limited coverage that understanding of yours requires. Hopefully it doesn't also reference external resources of its own, or the reader will have absorbed dozens of overlapping articles before they're ready to return to yours. And then the reader needs to come back to your article, offload the discussion from the external article, and pick up the train of thought that was interrupted when they were sent off elsewhere.
I think it's better if information is self-contained. Referencing external resources for a more in-depth treatment of the context is OK, but the parts that are necessary for a proper understanding of the primary article should be _in_ the primary article, even if in shortened form and lacking otherwise important nuance that just isn't relevant in the context of the primary article. Also, links where the anchor text is just "here" are useless once the link target disappears. The reader has no idea what the title was, who the author was, and will be unable to retrieve the information even if it did survive elsewhere, as they have zero clues on what to search for. And that's something I just realised I should also start applying to my own writings, as I tend to do the same. 😂
Anyway, don't take this as a "you suck" criticism; it is meant in a constructive way. If I thought your blog post was not worth my time, or if I didn't appreciate your effort, I wouldn't have bothered responding. I love it when people freely share their knowledge, moreso when they're knowledgeable about the subject and cover things that go beyond the introductory matter. Your blog post was certainly appreciated.
The best channel for Python design principles.
Thank you Michael!
A truly excellent video, as always, you've gone way above and beyond simply explaining the concept, and outlined excellent use cases and way of thinking to know when to pick which one. thanks again, Arjan :)
I'm really glad my video was helpful!
I think Protocol is meant to define an interface and use it only as type hinting for arguments.
Similar to Iterable, if an object has a couple of dunder methods, it means it adheres to the Protocols
ABC should be used to define relationships between superclasses and subclasses as mentioned in the of the video
I agree, and it goes beyond that IMHO ; sometimes it is impossible to accurately define a function with pure type hinting (say when a method takes a coroutine as argument), and you can instead implement a Protocol's `__call__` method to accurately describe it.
@@cheaterman49
>(say when a method takes a coroutine as argument)
Just use typing.Coroutine (can be parameterized too)?
I love duck typing filosofy/pattern. It keeps the things so simple and organic.
I'll have to watch the video a couple more times, but it's brilliant stuff. You explained it so well I can't thank you enough.
I was literally debating with myself which of the two will be more suitable for a project I have at work when I opened TH-cam. Impeccable timing and excellent video 👌🏽
Happy to serve 😊
If your videos are anything like your code irl, it must be thumbs up.
Very interesting, this is exactly a paradigm for a driver collection/service that I hacked very poorly together to test a piece of hardware with the intention of wrapping that driver up in a connection management service. This a most fascinating look at a much better way of handling my idea. Thank you very much for the work, not just because it hit on a topic that was directly interesting to me but because you've done such a great job of making videos on non-trivial topics in general
Thanks for creating and sharing this video. This is a topic that confused me for quite some time.Your explanation is very clear. Love it.
Glad to hear it helped, Andy!
Thank you for presenting this topic! Last week I spent more time than 23:45 trying to figure this out.
You’re welcome Joe, glad you liked it! I also spent way more time than 23:45 to figure it out before I recorded the video, so I totally get it 😊.
תודה!
Thank you!
Another extremely informative video, thank you. Did not expect being able to split up Protocols like that, but it makes total sense. Will definitely be incorporating them more now.
Protocol seems more suited for distinguishing _traits_ with _trait objects_ . For example, you might define an interface that's really just an extension for objects like BeautyPrint with one method beauty_print() that just beautifully prints the object. Now, this isn't really meant to establish a hierarchy of beautifully printed objects so much as it gives an extension methods to those who need it. I don't think anything useful comes out of defining a function that takes a BeautyPrint.
That is, BeautyPrint might be a _trait_ (interface), but it's not a _trait object_ (doesn't represent a bunch of polymorphic types). Protocol might help prevent it from being a _trait object_ if you so choose.
Other newer languages have this feature like Rust
This is an excellent example! Without foos and bars 🤩
Thank you so much @bioman!
Arjan, I love your channel. thank you so much for the high quality. I have no doubt your channel will be recommended in CS courses within a year
Thank you, glad you like it!
The part I found the most impressing was how you used your IDE to edit the code, mostly with shortcuts.
Glad you liked it!
Thanks
Thank you for the support! Glad you enjoyed the content :)
It looks like there is a lot of benefit to using Protocols over ABC, however, I still _really_ want to see Python supporting some way of explicitly defining that a class implements a Protocol, similar to TypeScript's Interfaces (I believe you've mentioned this in a previous video; it is still just as valid).
Also, I'd (personally) like to see type checkers raise warnings about missing methods/attributes from ABCs/Protocols at a class' definition instead of at the class' usage elsewhere in the code. If I forget to return a value in a statically-typed method, I expect to get a warning in the method definition, not when I attempt to call the method elsewhere in the code. There may be implementation issues for classes using methods from multiple files (which I've only ever read about), but I'm sure that's its own can of worms when it comes to best practices.
I wanted to write the same thing
Thanks again for your very helpful video which explains the differences between protocols and ABCs.
Glad the content was helpful!
Hi Arjan, glad to find this lecture, love it. Ty!
Always enjoy your videos. Would love a video on how you personally set up each of your python projects within VS Code.
@ArjanCodes
Interesting video - Protocols seem like a good way to formalise duck typing and effectively introduce interfaces into Python by the back door. It might even get me using typing more!
The only thing I don't like about them is the ... notation - this joins things like dataclasses (when is something a class variable and when is it an instance variable?) and properties in stretching the syntax of classes in ways that get hard to explain to beginners. I've even read the Protocol PEP before but it didn't click with me that ... was actual syntax rather than just shorthand in their examples for code they hadn't written!
... and after posting that I've now read several articles about ... - I didn't realise it had such a long Python history (introduced as the singleton Ellipsis in 3.0). Perhaps because I don't use numpy, which seems to be the only place it was used before typing and protocols.
I like the anecdote that it was introduced as a constant because they thought it looked 'cute'!
Love this video. Thanks for touching on the differences between the two and showing examples. :)
Love your Python videos. I code primarily in Python and in Golang. Wish there was a teacher like you for golang! It's an untapped market imo.
Thanks Andrew, glad you like the videos! I’m not a Go expert (yet ;) ), but I’m intrigued by it.
thanks, i've been using these ones a lot lately!
You’re welcome!
Your videos never disappoint!
Thanks!
Really good explained. It's given me some good piece of advice for an app I'm refactoring.
Great explanation. Very good work! Congrats!!
What an amazing way of explanation
Thanks so much, glad you liked it!
Protocols are helpful in adhering to the Interface-Segregation principle by allowing us to define multiple interfaces to be used, without using multiple inheritance. ABCs can also allow us to create multiple finely-defined interfaces but we will have to use multiple inheritance to make it happen.
ABCs allow for type-checking at the time of object instantiation due to inheritance whereas Protocols allow for type-checking at the time of calling the object's method. Note that Protocols can also be inherited.
Use ABCs when you need some general method implementations in the base class that can be inherited and used in the sub-classes.
Use Protocols when you need pure interfaces with only abstract methods without any implementation.
i have to admit i hadn't noticed the letterboard.. i'm sorry! love your content. it's really helped me stay motivated. And helped my coding, details and structre. thank you so much.
Thanks for the video!
For me python's dynamic typing seems as a major design flaw, and ABC and protocols look like convoluted afterthoughts.
Protocols are mirrored by Type Classes in functionsl programing
Type classes are a powerful feature in functional programming that can help solve the expression problem of adding new types to which existing operations apply.
In functional programming, the expression problem refers to the challenge of adding new functionality to existing code without modifying that code. This is particularly difficult when dealing with types, as adding a new type often requires modifying existing code to support it.
Type classes provide a solution to this problem by allowing you to define a set of operations that can be applied to any type that implements the type class. This means that you can add new types to your code without modifying
existing code, as long as those types implement the required operations.
For example, let's say you have a type class called `Printable` that defines a single operation called `print`. Any type that implements the `Printable` type class can be printed using the `print` operation. Now, if you want to
add a new type to your code that can be printed, you simply need to implement the `Printable` type class for that type. You don't need to modify any existing code that uses the `print` operation.
It's nice to see your vim usage getting better over time. Should be able to `da(` without actually having the paren selected, btw. Then you could hit `.` to repeat the command. And if you used / to find the open parent, you could then `;.;.;.;.` to repeatedly find and delete the class inheritance syntax.
Hello Arjan. Thank you for the quality of your videos and all the knowledge that you dissipate.
This video is from a year ago but you look younger (I prefer you old [😎])
Thank you very much Aarjan! This is something college never teaches me 🤩
Excellent overview of ABCs vs Protocols. I miss the white board though-- it was one of my favourite parts of your videos.
Thanks! Who knows, perhaps the letterboard makes a return in the future 😎
@@ArjanCodes When the one-liners come easy you should slip it back in just to see if we're paying attention. :)
It is funny to see how strongly Python claim to be a pure OOP language, and being slow at enhancing its OOP features. Great content and great channel!
Thank you so much! The explanation is so clear and the content super useful.
Glad you liked it, Matias!
There's nothing wrong with multiple inheritance and this is precisely the kind of situation where it's useful. The decision between structural and nominal typing should be one of interoperability: do you want classes that implement the required interface to work with your code, or would you like to keep things controlled and open access only to those that explicitly opt in? You may want different things in different cases.
crystal clear, thanks for the video!
Next video suggestion: Abstract vs Metaclasses
Metaclasses is a good suggestion, thanks!
Great explanation, you saved the day
Thank you Alejando, glad you liked the video!
Another great tutorial. Thanks again
Great explication ! Thanks
Best as always. Super Pro
Thanks so much Fernando, glad you liked it!
whoa... i thought protocals were just a fancy way to annotate the type of a variable or parameter containing a function... by typing the __call__ method of the protocal you can do some things that the typing.Callable[...] syntax can't (or maybe thats fixed now?)... This is a much more sane use of protocals. Thankyou for the excellent video as always, you are amazing at explaining clearly!
Arjan... I really enjoy your videos. Thank you for sharing your knowledge! Would you consider doing a video about Mixins? I've seen it is largely used in Django framework, and I would like to know more about best practices about Mixins.
He says in a different video to not use them
Another very interesting video, thanks!
Glad you liked it!
So good 😊, such clear education
Glad it was helpful!
Thank you for this video once again.
You’re welcome!
Hey all cool, thank you so much for teaching us more than happy to give you the views, main focus is programming so hey thank you Arjaan I love the Netherlands btw tiptop thumbs up from a Norwegian
Hi Daniel, thank you and glad you like the videos! Haven’t been to Norway, but it’s a place I’d love to visit one day.
Thanks for the awesome explanation!
You’re welcome Carlos!
What type checker are you using in vs code? I am using PyLance, but yours seems to be more thorough. Great vids, much appreciated!
Hi David, I also use Pylance, but you have to make sure that your typeCheckingMode in VSCode's settings is set to 'strict'. See also: code.visualstudio.com/docs/python/settings-reference.
@@ArjanCodes awesome, thank you!
Is there a way to declare that your class satisfies a Protocol, for the sake of future developer groking the design? I guess it would require an import of the Protocol class / schema. I guess that may just be where naming convention or documentation comes into play?
Finally this is what I've been waiting to fully understand the use and importance of abstract and protocol.
What error highlighting you are using? I'm using pylint but it's not showing like that specially on unused imports
I’m using the Python extension in VS Code (Pylance). To get the type checking messages, make sure that typeCheckingMode in your settings is set to ‘strict’. See also: code.visualstudio.com/docs/python/settings-reference.
@@ArjanCodes Oh thank you so much.
@@ArjanCodes Thank you it's now working.
Great tutorial. Thank you very much
Glad you like it!
i have interface created in python using protocol but when i click to any method in vs code it does not open main method but it opens method in protocol / interface class, how can i click to open main implementation instead of interface ?
I took me a couple of minutes but I got it. Protocols are like interfaces in Java. Except in Java you have to explicitly implement an interface on an object. In python if the class has certain methods it can fulfill a protocol aka implements an interface but nothing tells the poor developer that a class actually realizes a protocol. It looks like a bunch of unrelated objects which somehow work together. The correct way would be IMO in this example to define two ABCs, Device and DiagnosticSource and a specific device should inherit from both. That way the connection would remain clear between the classes yet anything could inherit from one or the other. As I see with Protocol the code could be simpler at first glance but in reality it become more complicated and much more harder to read.
Other problem is if a class has a certain method it fulfills a protocol when that class is never meant to do that. In the example, lets say a message queue adapter easily can pass as a device cause it probably has connect, disconnect and send_message methods.
Protocol is a shady stuff!
You should split the intro chapter into 2 chapters - 1 for talking about letter-boards and one for the actual intro - not that it was a bad segment to include, but people should be able to skip it IF they dont care about it or if they already heard about it because they are coming back to this video for a second time
Great tutorial, quick question what font are you using on your IDE
This one's really helpful
Thanks Ari, Glad that the content it helpful
Love your videos, this clarified a lot of things I didn't understand about protocols. They seem to be great for splitting up coupling and being sure each part of the program only knows exactly what it needs. I have a question, though: Do protocols provide things other than type hinting? Can they raise errors or prevent vulnerabilities in other ways? I was surprised that Python didn't raise an error when you run register_device, for instance, until it actually ran into that missing connect method. Though the type hinting is very helpful in an IDE. Thanks!
Type hints are glorified comments, the interpreter won't check any of them.
Thanks, great video! What about Sphinx dealing with Protocol?
Hi @arjanCodes, I'm curious to know what course you provide includes the amazing video, Protocol Or ABC In Python - When to use which one?, I just watched above. Is this part of "The Software Designer Mindset" course?
Hi! it's covered in The Software Designer Mindset. :)
Great content, Arjan. I'm learning a lot from your videos. I'd like to ask a question though. What should I do if I want to support different device objects from different external libraries? Let's say, for example, that they all implement a connection method but with different names, like "connection()" or "connect_to()".
One place where Protocols have been making me a happy programer is by helping typehints be more meaningful in my programs. I used to have some ugly unions that seemed arbitrary. Now my unions are meaningful!
Ow, about the letter board... good call! They were cool, but far from what makes these videos stand out, imo.
Nice videos! What was the shortcut called which you used to directly remove the inheritance but not the : at around 12:00? 😉
Hi thanks for the vedio. May I know how do you do the following via shortcut which is quite useful? Thanks!
1. Replace pass with 3 dots ...
2. delete (device) in function.
I suspect that Arian is using a ‘vi’ type plugin. In vi, to change a word, you simply put the cursor at the start of the word and type ‘cw’ and the new word
Great video. But i have a question about HueLight as protocol how is it different from any other class? I wonder if the inheritance from Device was correct if we do not expect the abstractmethods from Device to be mandatory for it.
Great video! very insightful
I have a question: is there any problem if the same method have different arguments on different subclasses of a protocol? or should I be sticked to use the same arguments?
Thanks!
Regarding your question: I would definitely avoid that, otherwise what would be the purpose of defining types if the subclass doesn't adhere to it?
@@ArjanCodes it is a “player” class (the protocol one) and for a “human” (subclass) being one method uses the UI to handle human input but for the “bot” subclass that exact method requires some data that passess through the method but not the UI…
I’m just learning about OOP and I’m designing simple games onto which I want to try all those new things.
Do you use some kind of plugin or extension for the highlighting? I cloned your repo and commented out the "connect"-method in Huelight as you do but I do not get any notice at all that the HueLight is not following the expected Protocol. I only get an error when running saying: "AttributeError: 'HueLightDevice' object has no attribute 'connect'"🤔
In the protocol code, if I implement the device class as a normal colon 'class Device:' instead of 'class Device(protocol)'. The code can still run without any issue. Besides the IDE don't do static typing check before running. Then why we use the 'from typing import Protocol' library?
From experience.
It seems protocols are for objects you do not control and ABCs are for objects you do control.
Example.
how do you type hint an sql connection?
It can be SQLite or Postgres or MYSQL. libraries.
You can type the the connection as a protocol of the methods you use instead of the concrete connection
All other issues aside, are there any performance issues between the two methods. Are the method arguments checked each time the methods are called? Thanks
Is there any award for this guy? Asking for a friend?.
Content was so useful for me. but I have one question. As you mention the protocol is the natural way in python. Why we can not have interface for each of instance of each protocol. In your Device abstraction class some how we can say that you violate interface segregation in SOLID principle. Somehow in Protocol some of the dependencies are hide ?
I watched this video now a few times and it is really great. Now I have a question relying on the Interfaces: How would I visualize the optional diagnostic Interface in the UML (for example in mermaid)? I am on my way to understand design patterns and principles but this would be good to know for me. I also ask myself how to keep track of the different protocols in large projects (in my job we have a Django application with many….many apps)? If the question is too complex to answer maybe an idea for an advanced mermaid tutorial? ❤
Thank you for your great videos!!!