You can also have right vs left associativity. Like doing 4+7+3 will execute left to right (left associative). 3^5^6 will be right associative, so it will execute like this: 3^(5^6)
There's always got to be ONE functional programmer that's got to pop up and say "well actually..." and then fill the blackboard with equations whilst the rest of us quietly back towards the door. ;)
@@edgeeffect i wish i was a functional programmer. I am a mathematician trying to be a better programmer at my shitty data hore job hahahahaha Knowing the difference between associativity and commutativity is the most i can contribute in this domain besides yelling "Do yOu kNoW abOUt YonEdas LeMma?" Hahahahah
Finally someone who actually came close to explaining monads in a way that normal people can understand! I've only been waiting over a decade since Brian Beckman made the 'definitive' explanation that ended up confusing everybody.
In general, when thinking about programming languages in terms of category theory, functions in the PL are considered morphisms, and types are the objects. Functors map one category into another category. Therefore, unless you're somehow departing the language altogether, you're only ever going to touch endofunctors. To be more precise a functor, is any type that is: 1) defined in terms of another type, 2) preserves functions between types (this is what "map" gives you), and 3) preserves identity functions. The initializer is NOT required for it to be a functor. That is, any functor F does not automatically give you a function T -> F for any type T. The initializer is better thought of as a natural transformation from the identity functor to the functor in question.
This is a fantastic video, I have tried time and time again to understand monads, but this was the first video that really clicked. I like how you broke down the "monoid in the category of endofunctors" so thoroughly, that was a huge help.
Happy to see some more advanced content! After being lost for about 18 minutes, it suddenly clicked how I could have used something like this instead of regular error handling on a project a couple of months back. Cool to be reminded that there are alternative approaches out there! Cool to be reminded in mind that there are more ways to do stuff!
If we don't care where the error occurred and the debugging is simple, than I could see monads replacing explicit error handling. At 17:00, there's a nice example of handing errors at the end of the track instead of handling errors in the middle of each bind.
Thanks, Arjan. As a Scala refugee, this was very useful to me. If someone in the Golang community can come up with an implementation, it'll be funny to see what they called it.
haha, this one was magical and I didn't see it coming! Following along I was like "feels like this weird concept which I haven't fully understood when i was trying to get into Rust" - turns out it was and I finally got it now! so I've watched a python video about a thing I shouldn't do in python, learnt about rust and ended up a better programmer than i was before. That's why you're once of a kind on youtube! have my thumps up :)
Thanks for picking up this important topic! Those new to functional programming might find the explanations involving monads, functors, and monoids within category theory, where their origins lie, to be intimidating. In my view, it is more crucial to first present the practical elegance and simplicity of these concepts, as demonstrated in your video. Just a small hint: i think you are sometimes confusing associativity with commutativity. commutativity: (a + b) == (b + a) => order of operands does not affect the result associativity: (f ・ (g ・ h))(x) == ((f ・ g) ・ h)(x) => grouping of operands does not affect the result
Monads fill in the shortcomings of functional programming languages. They allow to sequence function calls while attaching a context, which is something that exists natively in python, so monads should stay in the FP world as python has better native ways to do the same things. But they're nice, the maths behind it is a bit harsh just to understand that it's merely a context passed down the chain of recursive calls to emulate a chain of sequential calls.
Thanks for showing this. Best video on monads by far and great to see better pattern matching usecases. Lately I’ve been writing tools that return tuples (eg (ok:bool, val: int, reason:str) returned from tryParseInt ). This is a great use case for pattern matching logic in the caller.
Not necessarily a criticism, more of an observation but this seems leaps and bounds ahead in complexity with respect to most of the subjects of your other videos.
@ArjanCodes Very heady topic presented in an extraordinarily approachable way. I really liked seeing you turn maybe into a decorator. That gives me lots of ideas for my own home-grown functions. Thanks for another great tutorial.
Thank so much Arjan! I tried to understand Monads each time I got hyped on functional programming, but only this time I think I understood the idea of a monad and usecases. And actually it seems that I have already been using something like that for error handling with ts-result library that brings in this railway approach that you mention in the video
good and informative video, i have a small nitpick, Maybe isn't really the typical exception handling mechanism, its more the null or none handling mechanism, and to this extent the Result in rust would be the try except equivilant and Option would be Maybe (synonymous) monadic handling can be very useful even in languages its not implemented, it would come down to developer preference and style, i myself prefer monadic handling
I've watched like 5 videos on this really esoteric topic, and this is one that finally made sense to my programmer brain by typing up code examples rather than like Venn diagrams or whatever else functional thinking people do.
Any chance of a video on Generics? I have used them a bit in the past but i don't understand why you made the Monad depend on T at 8:49. I've never seen a good video on this topic.
Thanks a bunch for your video, very informative. Regarding the usability of monad - I've spent quiet a lot of time getting rid of code that relies extensively on results library (specifically, it's error handling way). At some point of codebase growth it became virtually impossible to tell, what behavior and return types you should expect from methods (and code overall), therefore, we decided to move to Python's built-in exception handling and typing mechanisms
Thanks Arjan, I tried to understand what monads were some time ago, but every video I started was just too complex for my understanding. You've explained it in a way that was very easy to understand!
I had this discussion more than once in companies. Not about implementing a whole monad ecosystem in python, but implementing the Maybe modad. The argument revolved a lot about the "Billion Dollar Mistake" that has functions of any kind possibly returning Null. But, in Python, None is the sole instance of the NoneType type. That means that if you are using type annotations and your function can return None, it needs to be explicitly stated. The Maybe/Response monad can provide convenience -- like the NullObject pattern. But in python it's counter intuitive and requires unnatural boilerplate. I am diving more in this subject to apply it in Haskell
I'm glad I completely agree with your conclusion. If it's in the language, sure, why not? But I won't step out of my way to use that when it's not the norm. It just makes reading the code more confusing if and when someone else has to dive in.
I've known monads from using functional reactive programming paradigms like RxJava KotlinFlow and Swift Combine. They're very useful on multithreaded the front systems like mobile apps where you want to safely react to user input and do thread safe concurrent operations with standardised error handling. Functional reactive is a combination of monads and the observer pattern
Hi Arjan, thanks for this video. I'm actually considering the railroad oriented programming approach in a data project. With a colleague we are about to migrate some processes running in an ETL tool that is actually already working in a functional way and which already has a way of chaining operations along happy/sorry paths. Besides functional programming makes parallelization easier, and in data processing this is important. So yes, in conjunction with the itertools, generator expressions and multiprocessing, I think we will give monads a chance under Python 😉
This is a big step forward in describing monads. I feel closer to understanding them but still not there yet. Perhaps need more examples, we kind a drifted into talking about maybe and railroad etc. Other examples of monad usage would be needed to help differentiate.
It's description could be simplified much more: A monad is an object with a value that can be run through a pipeline(a chain of functions) where in each of the function applications in the chain it does some additional proccessing like checking for None or waiting for an operation in Async to finish. It's useful for automating repetitive task, by appying this additional operation each time a function is called. Monads that are actually used in mainstream languages are Result, Option, Async and technically also List. Bind is sometimes referred to as and_then or flatmap in some languages. So in essence: A chain in where each step it does something extra in addition to applying the function.
I think for a 20-ish minute video you're doing a good job in explaining what monads are. However there are a few things that are missing, unclear or slightly wrong IMO: * After describing monads at 11:13 I find the explanation of how they can capture side effects very unclear and hand-wavy (both literally and figuratively). I don't think it's obvious how from the structure of monads it follows that they can capture side effects. Adding an example of such monad would be helpful. * The Maybe monad is meant to be used as a replacement for None or null values. Using it to represent errors is akin to using None to represent errors: it could work in some simple cases, but is not the general solution. * In rust, the Maybe monad is called Opion. The Result monad is somewhat similarly looking to Maybe/Option, but instead of the None case, it has the error case, where it contains the error that happened. So it's a better mapping for exceptions. I agree that monads are most useful when they are part of the language from the get-go. In ML languages like Haskell, Ocaml and Rust you don't have the concept of null. Instead the standard library and basically the rest of the code is using the Maybe/Option monad. There are no exceptions either, so they are using the Either/Result monad. The compilers also check if the code handles all cases, which makes the use of monads there much better. Their usefulness in languages like Python or Java is limited. Even so, it's good to expand one's knowledge.
Thanks for adding some useful info and pointing out where info is lacking-I also found the point about side effects went by 10x too fast-If that’s the point, I’d really like to understand that better :D I can kinda understand though-based on my understanding that monads are used to create things like… string.strip().lowercase() In these cases, bind() is hidden with syntactic sugar. Explaining this would also make this video more valuable.
I've been curious about funcotrs/monoids for so long and finally a video explaining this in practical terms! I also love that you explain why it wouldn't be useful yet in python. Personally, I see this as doing a bunch of unnecessary computation. If i divide by zero, there is no reason why I should continue down the Maybe(None) rail, why not just short circuit actually handle the error. On the other had I do see how this allows you to move error handelling to a high level and not worrry about it when there are other things going on
The thing that I was immediately missing was any information about where and why in my chain of Maybe Monads the error occurred that led to a None-Monad. My guess is that you might be able to have a monad value that encapsulates the kind of error that occurred, but at that point, that seems like the same thing as throwing a specific error in Python.
Thanks a lot!! Really cool and clear explanation. Just have a question. Can we do early return when there is an exception in the bind chain? like in this example Maybe(input_str).bind(parse_int).bind(is_positive).bind(lambda n: Maybe(double(n))) Thanks!
Looks like something that you can find in code that you inherit from developer who suddenly left the company and noone is familiar with. Great content. Thanks.
The problem with explaining monads is that all monads are different, and sometimes completely different from each other. A list and an Async task does not have much in common except that they are monads. It's sort of like a spaceship and a bicycle are both transportation devices, but that's where the similarities end.
I'm just learning about monads and functors in python. Cool stuff. Might be wrong but looking at the example you gave with the `add_one` method been mapped to the Functor(5) which returns Functor(5 + 1). It think a functor is a programming implementation of functions in math. I feel that given that a functor returns its value when called, we can say: Functor(5) = f(5) -> 5 Say '->' means returns Hence; Functor(x) = f(x) -> x If add_one(x) -> x + 1 Then; f(x).map(add_one) -> f(x + 1) If we let; g(x) = f(x).map(add_one) = f(x + 1) g(x) = f(x + 1) -> x + 1 But f(x) -> x, Therefore g(x) -> f(x) + 1 Mehh. I'm not actually sure if this is right but if it is, then i think understand better using this analogy.
thank you, that was very good explanation, it could be usefull if you have lot of possible types and validations, so you don't have to write special cases for everything, but define it in generic level
Great video, very good explanation! One thing, is that in Rust the equivalent of Maybe is actually the Option enum, the Result enum is another sum type that is specialized to represent an error.
Great video!! This explanation would be absolutely top-notch with a comparison against a functional language (Elixir?) or in a language with functional capabilities (R?, Julia?).
Great vid! First class functions - a good thing. Functional programming - meh not so much. And I write a lot of Rust... For real-world applications functional programming will give you poor performance, or great performance right up until COOOOOOOOWtf? - clunky error handling and readability problems past a certain scale. For raw data processing it's fine. Feels like unix pipes. For anything bigger, anything complicated nah. AAA game? Image editor? Web browser? Not a prayer. I've never seen a functional body of code that didn't come with a huge accompanying document trying to explain the big picture because the abstractions start to fall apart as things get bigger. Error handling is either ignored or becomes a nightmare that relies on a bunch of macros. Haskell is 30+ years old and still not widely used. Fun ideas, but 99.999% of all code is some form of imperative - even most real world Rust is often heavily imperative, unless you're playing code golf. Keep up the good work!
Best way to understand monads is to forget all those great mathematical tomes and just write a bit of JavaScript using promises then you'll just start to wonder what all the fuss was about.
You know, its been a long time since we had a code roast, and that kind of videos is my favorite, i would be happy for another episode, maybe even something that uses openai api or django...
It's easier to understand functors and monads using diagrams. A functor is a mapping between categories. A category is a collection of objects with arrows between the objects that can be composed. An endofunctor F in the category of Python types maps the type a to the type F[a] and a function from a to b to a function from F[a] to F[b]. The simplest functor in Python is the list constructor []. List comprehension is a clear example of mapping a function between types to a function between lists. In the category whose objects are functors, a monad is a triple, consisting of a functor and two natural transformations (that is the name of the mapping between two arrows from functor to functor). One acts as the identity under composition and the other composes two arrows. That is the monoidal structure. A list is also a monad, with double list comprehension acting as the composition. Since the result is a list and not a list of lists this operation is called flatMap in some functional languages. Scala and Haskell allow using comprehensions with any monad. Since this allows sequencing operations, it is commonly called a glorified semicolon. Monads allow reifying computations as data structures. Which means that instead of calling an API immediately one obtains instead a description of the API call. This is the reason why it is said that in pure functional programming there are no side effects. All functions return data. In Haskell only the main function can execute side effects. They are called side effects because in mathematics there is no function for sending emails. The idea of functional programming is to make writing software akin to writing a book in Word instead of with a typewriter. It is not for academics. It is what engineering always does: it gives us more powerful and sophisticated tools that allow us to go farther than our ancestors. The Joy of Abstraction is a quite accessible introduction to category theory. Also check out CQL and Ballerina. Programming with diagrams is not old-fashioned, it is only getting started. Thanks for making this video Arjan! 🎉
best description i've seen so far, But.. how many maybe's do you have to create in your code. One for null values, one for oversized strings, one for car number plate formsts, one for sanity checked order values..?
Alas, by devoting his life to the way of the serpent he has forfeit his right to preach the good word. If only the high priests of Rust were more benign in their judgement...
Typescript is also not a functional programming language. Even though with fp-ts you can do it very comfortably. It provides you all the monads you need - and I find the "exception handling" in fp superior to the "object oriented"
Great channel, subbed. Reddit sent me here. I am still working through a bunch of kaggle courses to learn python all the way up to AI coding. Even me having a barely basic understanding of the syntax, this was extremely easy to follow. 😊
You mean to ask whether it is an endofunctor? If you limit it to only having a map function that acts on the data in the dataframe, then I guess so... (The chaining of other functions/methods would be a bad example, I guess, because they process the value directly in some way, they don't map a user-supplied function necessarily.) But that doesn't make it a monad yet, does it? That also requires an associative binary operation with an identity element. What would that be? (Btw, I'm not an expert; just following the logic from the video.)
No, data frames in themselves are not monads, but some operations on data frames are monoids, e.g. data frame addition. Moreover addition is a commutative monoid, and the nice thing about commutative monoids is that you can use them in a reduce operation, e.g. in Apache Spark, to easily parallelize addition.
I think Monads have a usecase when dealing with computationally expensive processes, e.g. fitting multivariate functions. It would avoid spending a lot of time on values that you know would throw an error in later parts of the algorithm.
You are correct that operating on monads shouldn't require any boilerplate. In fact, it requires neither that nor close integration. A monad is not a runtime representation or syntax feature, but an abstract category: a relation formed by morphisms defined over structures, with coherence properties. You never arbitrarily define a composite class or modify the language to exploit the monoid formed by integer addition, the functor formed by a list, or the monad formed by async actions. Property based testing, perhaps a runtime protocol, would make more sense. Generally speaking, the separation of behaviors from representations is advised in the functional style, so rather than considering a monad as a tightly coupled object to be converted to, consider it as operations which may be defined over potentially existing objects, if they haven't already. It would be fairly trivial to define the bind operator as a regular function decorated with singledispatch, for instance. In fact, type classes in Haskell are implemented similarly with an implicit parameter gathered by the type system. To make this concrete, consider the list monad: let unit(x) = [x] bind(xs, f) = [y for x in xs for y in f(x)] where fail = [] Extensibility without modification is a powerful principle of software architecture closely related to the Expression Problem. Otherwise, excellent content, as always!
I believe the concept of the monad can be very useful. I noticed a lot of people using the exception framework to communicate business validation errors in a DDD architecture. To me this has always been kind of weird, because it is expected behaviour, and not an exception. So switching to another control path is kind of off. I am using Result bags for this, a custom solution to communicate errors, warnings, etc. Monads could be nice for this as well as it doesn't break the main control flow and leaves the exception framework for what it is meant for.
I get what you mean but you’re a little bit wrong in the way that the reason why exceptions are called exceptions and not errors is that as their name indicate they’re not just meant to deal with errors, even though in the real world they’re mostly used to do that. One can argue that using exceptions to deal with exceptionnal behavior is by definition using the tool for its precise purpose. I happen to work with the DDD philosophy (DDD is not really a specific architecture) and for our project we created a DomainException (which is then subclassed for more specific purposes) in order to differenciate technical error handling and domain validation. This also makes it easier to setup a middleware in the webapi to bubble these appropriately towards the UI. There may be ways to do that in a more elegant fashion, however it has the virtue to keep things simple and so hold up the criminally underrated KISS principle.
What I'm missing in this video is an explanation of higher-order functions. I'm aware you explained it before, but that was a while ago. I already know the concepts in the video, but if I didn't I would wish I knew what a higher-order function was.
I'm a mathematician who works with monads on a daily basis and yet I've never managed to figure out why programmers, of all people, care about monads. I have to admit, I still have to squint to see the connection to the arcane programming terminology :p A category is just a notion of stuff and stuff that goes from stuff to stuff. Functors are stuffs that go between categories, so it has to be able to do something both to the stuff and to the stuff that goes from stuff to stuff, in some consistent way. Endofunctors go from a category to itself. In particular that means you can do it multiple times to the same stuff. A monoid is a stuff that has a way to "multiply" different copies of itself. For endofunctors that multiplication is composition, and the "way" is to compute the composition to an instance of just a single use.
I understood... at least half of that. I can definitely see the use case for them, at the very least, though I admit some of the construction logic is a bit confusing to me(though I've never really understood generics so that might be part of it).
Something isn't quite right with the example at 4:15. What do you exactly mean when you say it's not an endofunctor? I mean, in a programming languages we usually talk about only one category - category of types of the language. Which means all functors are endofunctors just by nature. And it's not even a functor, at least because map isn't polymorphic as it should, it only works with functions from str to str. Correct me if I'm wrong, maybe I just misunderstood.
@@MdImrulHassan I wouldn't agree on the rest of the video. Many details in the video aren't correct. I understand that there was a good intention to make this video easy to comprehend, but it sacrificed correctness so much that can in fact worsen your understanding instead. It's a really difficult task to make a "monad tutorial" easy to understand. But at least we should appreciate the efforts.
thanks for the vid, just a small error: 6:35 talking about the binary operation of a monoid "it should be associative so the order in which you do the binary operation shouldn't matter..." That's not associativity but conmutativity "a·b" = "b·a" and the binary operation of a monoid does not need to be conmutative. What you meant to say was "the way in which you group operation doesn't matter" that is: " (a·b)·c " = "a · (b ·c) " I guess you wanted to say this. Just to clarify.
Interesting. I could see a potential use for this in data pipelines where you have succesive transformations and don't want to put error handlers everywhere. On the other hand, it would be nice to know exactly where the error occurred so maybe not 😂
That’s what I’m thinking too, it makes it very easy for ‘lay’ people (eg. Mathematicians, hence its use in Haskell) to write code against a value that doesn’t necessarily error out, but on the other hand, you will often get garbage-in, garbage-out type situations that makes it harder to understand where things go wrong. I think this would be useful in languages that have no verbose error handling (like Haskell) where you are expected to practically write your own exception handler. And I also think that this is just one step to the hell that pure object-oriented languages are, I can see the temptation to make a spaghetticode of things that are abstracted away to that level.
How about adding an `exception` field to Maybe to save whatever got raised. It could allow better user feedback, and also better debugging because the programmer could see the actual exception.
The good thing about Monads is that NOONE, not even a single developer in the world, even ones that ONLY use funcional programming, needs to know what it is.
in most languages it does not make sense to directly implement monads as a single entity. it can be beneficial only in languages that have function-based dispatching (not class-based) and that can reliably inline higher order functions. errors as values are totally superior to exceptions outside of scripting. i wish there was a way to switch between those on per-session/per-build basis (theoretically, this is possible).
You said associativity but explained it like commutativity. Didn’t even notice until I was confused by your code that tests for associativity of the monad when I wondered „Huh, where is he switching the order?“. Would be nice if you corrected that. Otherwise nice video.
I understand classes, Objekts and functions/methods in Python. So was interested what a functor is. The definition of the map Returns an Objekt or instance of the class. Thus the definition reminds me of recursive functions.
![[TH] 2024 PMGC League | Survival Stage วันที่ 1 | PUBG MOBILE Global Championship](http://i.ytimg.com/vi/FVFT-PFwU80/mqdefault.jpg)
👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
Whew, thank you for uploading this. I have a wedding to go to tonight and wasn't sure what to talk about with my fellow attendees. Now I do!
7:05 small correction: associativity means that x+(y+z) = (x+y)+z. What you've describe is commutativity (or the abelian property).
You can also have right vs left associativity. Like doing 4+7+3 will execute left to right (left associative). 3^5^6 will be right associative, so it will execute like this: 3^(5^6)
@@ArachnidAbby Right and left associativity is a thing on the notation level, rather than a mathematical concept
There's always got to be ONE functional programmer that's got to pop up and say "well actually..." and then fill the blackboard with equations whilst the rest of us quietly back towards the door. ;)
@@mtcomscxstart ah makes sense. I just remember having to deal with it while working on my compiler
@@edgeeffect i wish i was a functional programmer. I am a mathematician trying to be a better programmer at my shitty data hore job hahahahaha Knowing the difference between associativity and commutativity is the most i can contribute in this domain besides yelling "Do yOu kNoW abOUt YonEdas LeMma?" Hahahahah
Finally someone who actually came close to explaining monads in a way that normal people can understand! I've only been waiting over a decade since Brian Beckman made the 'definitive' explanation that ended up confusing everybody.
“We’re not afraid”
I got kicked in the Monads once, boy was that painful!
@@ what the heck @@ lol :))
Is Roshambo a Functor?
No monads after that
In general, when thinking about programming languages in terms of category theory, functions in the PL are considered morphisms, and types are the objects. Functors map one category into another category. Therefore, unless you're somehow departing the language altogether, you're only ever going to touch endofunctors. To be more precise a functor, is any type that is: 1) defined in terms of another type, 2) preserves functions between types (this is what "map" gives you), and 3) preserves identity functions. The initializer is NOT required for it to be a functor. That is, any functor F does not automatically give you a function T -> F for any type T. The initializer is better thought of as a natural transformation from the identity functor to the functor in question.
This is a fantastic video, I have tried time and time again to understand monads, but this was the first video that really clicked. I like how you broke down the "monoid in the category of endofunctors" so thoroughly, that was a huge help.
Happy to see some more advanced content!
After being lost for about 18 minutes, it suddenly clicked how I could have used something like this instead of regular error handling on a project a couple of months back.
Cool to be reminded that there are alternative approaches out there!
Cool to be reminded in mind that there are more ways to do stuff!
If we don't care where the error occurred and the debugging is simple, than I could see monads replacing explicit error handling.
At 17:00, there's a nice example of handing errors at the end of the track instead of handling errors in the middle of each bind.
Thanks, Arjan. As a Scala refugee, this was very useful to me. If someone in the Golang community can come up with an implementation, it'll be funny to see what they called it.
Gona-
nevermind...
My thoughts exactly! :)
😁
In imperative programming there's no need for monads lol
@@chudchadanstud ok. But what's your point?
Video 3 on monads. Still working on the comprehension! Very well explained.
Glad it was helpful!
haha, this one was magical and I didn't see it coming! Following along I was like "feels like this weird concept which I haven't fully understood when i was trying to get into Rust" - turns out it was and I finally got it now! so I've watched a python video about a thing I shouldn't do in python, learnt about rust and ended up a better programmer than i was before. That's why you're once of a kind on youtube! have my thumps up :)
Very nice to have you discussing an advanced topic again! Are you currently working in languages where monads do make sense?
Thanks for picking up this important topic! Those new to functional programming might find the explanations involving monads, functors, and monoids within category theory, where their origins lie, to be intimidating. In my view, it is more crucial to first present the practical elegance and simplicity of these concepts, as demonstrated in your video.
Just a small hint: i think you are sometimes confusing associativity with commutativity.
commutativity: (a + b) == (b + a) => order of operands does not affect the result
associativity: (f ・ (g ・ h))(x) == ((f ・ g) ・ h)(x) => grouping of operands does not affect the result
Seems like after all Arjan tests the associativity but describes it incorrectly.
Monads fill in the shortcomings of functional programming languages. They allow to sequence function calls while attaching a context, which is something that exists natively in python, so monads should stay in the FP world as python has better native ways to do the same things.
But they're nice, the maths behind it is a bit harsh just to understand that it's merely a context passed down the chain of recursive calls to emulate a chain of sequential calls.
Thanks for showing this. Best video on monads by far and great to see better pattern matching usecases. Lately I’ve been writing tools that return tuples (eg (ok:bool, val: int, reason:str) returned from tryParseInt ). This is a great use case for pattern matching logic in the caller.
Not necessarily a criticism, more of an observation but this seems leaps and bounds ahead in complexity with respect to most of the subjects of your other videos.
Maybe u think that way because you are not used to this way of coding. The examples itself are not that complex
Yes, finally, we're so back
@ArjanCodes Very heady topic presented in an extraordinarily approachable way. I really liked seeing you turn maybe into a decorator. That gives me lots of ideas for my own home-grown functions. Thanks for another great tutorial.
I'm very glad the video inspired you, Matt!
Thank so much Arjan! I tried to understand Monads each time I got hyped on functional programming, but only this time I think I understood the idea of a monad and usecases. And actually it seems that I have already been using something like that for error handling with ts-result library that brings in this railway approach that you mention in the video
Awesome you back on advanced stuff! You should do metaprogramming topics
good and informative video, i have a small nitpick, Maybe isn't really the typical exception handling mechanism, its more the null or none handling mechanism, and to this extent the Result in rust would be the try except equivilant and Option would be Maybe (synonymous)
monadic handling can be very useful even in languages its not implemented, it would come down to developer preference and style, i myself prefer monadic handling
Very nice explanation about Functors and Monads. Seems like you have broken the curse of the Monad!
Thank you ahaha! Let's hope so :)
I've watched like 5 videos on this really esoteric topic, and this is one that finally made sense to my programmer brain by typing up code examples rather than like Venn diagrams or whatever else functional thinking people do.
Any chance of a video on Generics? I have used them a bit in the past but i don't understand why you made the Monad depend on T at 8:49. I've never seen a good video on this topic.
That would be nice, especially with the new 3.12 generic type hinting syntax.
Thanks a bunch for your video, very informative. Regarding the usability of monad - I've spent quiet a lot of time getting rid of code that relies extensively on results library (specifically, it's error handling way). At some point of codebase growth it became virtually impossible to tell, what behavior and return types you should expect from methods (and code overall), therefore, we decided to move to Python's built-in exception handling and typing mechanisms
Thanks Arjan, I tried to understand what monads were some time ago, but every video I started was just too complex for my understanding. You've explained it in a way that was very easy to understand!
I'm happy to hear it helped you, Andrea!
I had this discussion more than once in companies. Not about implementing a whole monad ecosystem in python, but implementing the Maybe modad. The argument revolved a lot about the "Billion Dollar Mistake" that has functions of any kind possibly returning Null. But, in Python, None is the sole instance of the NoneType type. That means that if you are using type annotations and your function can return None, it needs to be explicitly stated. The Maybe/Response monad can provide convenience -- like the NullObject pattern. But in python it's counter intuitive and requires unnatural boilerplate. I am diving more in this subject to apply it in Haskell
Dang, missed opportunity to title the video "What the func are monads?!"
I'm glad I completely agree with your conclusion. If it's in the language, sure, why not? But I won't step out of my way to use that when it's not the norm. It just makes reading the code more confusing if and when someone else has to dive in.
I've known monads from using functional reactive programming paradigms like RxJava KotlinFlow and Swift Combine. They're very useful on multithreaded the front systems like mobile apps where you want to safely react to user input and do thread safe concurrent operations with standardised error handling. Functional reactive is a combination of monads and the observer pattern
Hi Arjan, thanks for this video. I'm actually considering the railroad oriented programming approach in a data project. With a colleague we are about to migrate some processes running in an ETL tool that is actually already working in a functional way and which already has a way of chaining operations along happy/sorry paths. Besides functional programming makes parallelization easier, and in data processing this is important. So yes, in conjunction with the itertools, generator expressions and multiprocessing, I think we will give monads a chance under Python 😉
This is a big step forward in describing monads. I feel closer to understanding them but still not there yet. Perhaps need more examples, we kind a drifted into talking about maybe and railroad etc. Other examples of monad usage would be needed to help differentiate.
It's description could be simplified much more: A monad is an object with a value that can be run through a pipeline(a chain of functions) where in each of the function applications in the chain it does some additional proccessing like checking for None or waiting for an operation in Async to finish. It's useful for automating repetitive task, by appying this additional operation each time a function is called. Monads that are actually used in mainstream languages are Result, Option, Async and technically also List. Bind is sometimes referred to as and_then or flatmap in some languages. So in essence: A chain in where each step it does something extra in addition to applying the function.
Thank you, an actual explanation of what the attributes of monad mean in practice
Glad it was helpful!
I think for a 20-ish minute video you're doing a good job in explaining what monads are. However there are a few things that are missing, unclear or slightly wrong IMO:
* After describing monads at 11:13 I find the explanation of how they can capture side effects very unclear and hand-wavy (both literally and figuratively). I don't think it's obvious how from the structure of monads it follows that they can capture side effects. Adding an example of such monad would be helpful.
* The Maybe monad is meant to be used as a replacement for None or null values. Using it to represent errors is akin to using None to represent errors: it could work in some simple cases, but is not the general solution.
* In rust, the Maybe monad is called Opion. The Result monad is somewhat similarly looking to Maybe/Option, but instead of the None case, it has the error case, where it contains the error that happened. So it's a better mapping for exceptions.
I agree that monads are most useful when they are part of the language from the get-go. In ML languages like Haskell, Ocaml and Rust you don't have the concept of null. Instead the standard library and basically the rest of the code is using the Maybe/Option monad. There are no exceptions either, so they are using the Either/Result monad. The compilers also check if the code handles all cases, which makes the use of monads there much better. Their usefulness in languages like Python or Java is limited. Even so, it's good to expand one's knowledge.
Thanks for adding some useful info and pointing out where info is lacking-I also found the point about side effects went by 10x too fast-If that’s the point, I’d really like to understand that better :D
I can kinda understand though-based on my understanding that monads are used to create things like… string.strip().lowercase()
In these cases, bind() is hidden with syntactic sugar. Explaining this would also make this video more valuable.
I've been curious about funcotrs/monoids for so long and finally a video explaining this in practical terms! I also love that you explain why it wouldn't be useful yet in python.
Personally, I see this as doing a bunch of unnecessary computation. If i divide by zero, there is no reason why I should continue down the Maybe(None) rail, why not just short circuit actually handle the error. On the other had I do see how this allows you to move error handelling to a high level and not worrry about it when there are other things going on
The thing that I was immediately missing was any information about where and why in my chain of Maybe Monads the error occurred that led to a None-Monad. My guess is that you might be able to have a monad value that encapsulates the kind of error that occurred, but at that point, that seems like the same thing as throwing a specific error in Python.
Thanks a lot!! Really cool and clear explanation.
Just have a question.
Can we do early return when there is an exception in the bind chain?
like in this example
Maybe(input_str).bind(parse_int).bind(is_positive).bind(lambda n: Maybe(double(n)))
Thanks!
Looks like something that you can find in code that you inherit from developer who suddenly left the company and noone is familiar with. Great content. Thanks.
Glad you enjoyed the content!
The problem with explaining monads is that all monads are different, and sometimes completely different from each other. A list and an Async task does not have much in common except that they are monads. It's sort of like a spaceship and a bicycle are both transportation devices, but that's where the similarities end.
I'm just learning about monads and functors in python. Cool stuff.
Might be wrong but looking at the example you gave with the `add_one` method been mapped to the Functor(5) which returns Functor(5 + 1). It think a functor is a programming implementation of functions in math.
I feel that given that a functor returns its value when called, we can say:
Functor(5) = f(5) -> 5
Say '->' means returns
Hence; Functor(x) = f(x) -> x
If add_one(x) -> x + 1
Then;
f(x).map(add_one) -> f(x + 1)
If we let;
g(x) = f(x).map(add_one) = f(x + 1)
g(x) = f(x + 1) -> x + 1
But f(x) -> x,
Therefore g(x) -> f(x) + 1
Mehh. I'm not actually sure if this is right but if it is, then i think understand better using this analogy.
thank you, that was very good explanation, it could be usefull if you have lot of possible types and validations, so you don't have to write special cases for everything, but define it in generic level
Great video, very good explanation!
One thing, is that in Rust the equivalent of Maybe is actually the Option enum, the Result enum is another sum type that is specialized to represent an error.
we need more videos like this!
Great video!! This explanation would be absolutely top-notch with a comparison against a functional language (Elixir?) or in a language with functional capabilities (R?, Julia?).
Would totally love more Julia content from @ArjanCodes!
Great vid! First class functions - a good thing. Functional programming - meh not so much. And I write a lot of Rust...
For real-world applications functional programming will give you poor performance, or great performance right up until COOOOOOOOWtf? - clunky error handling and readability problems past a certain scale. For raw data processing it's fine. Feels like unix pipes. For anything bigger, anything complicated nah. AAA game? Image editor? Web browser? Not a prayer. I've never seen a functional body of code that didn't come with a huge accompanying document trying to explain the big picture because the abstractions start to fall apart as things get bigger. Error handling is either ignored or becomes a nightmare that relies on a bunch of macros. Haskell is 30+ years old and still not widely used. Fun ideas, but 99.999% of all code is some form of imperative - even most real world Rust is often heavily imperative, unless you're playing code golf. Keep up the good work!
I once try to understand monads years ago and found that Leibniz wrote a humongous book about that theory called Modanology
Best way to understand monads is to forget all those great mathematical tomes and just write a bit of JavaScript using promises then you'll just start to wonder what all the fuss was about.
Leibnitz's "Monadology" has nothing to do with monads in modern sense
And the book is rather short btw
You know, its been a long time since we had a code roast, and that kind of videos is my favorite, i would be happy for another episode, maybe even something that uses openai api or django...
I really wish this video existed in 22 years ago when I had to sit through my first computer science class.
It's easier to understand functors and monads using diagrams. A functor is a mapping between categories. A category is a collection of objects with arrows between the objects that can be composed. An endofunctor F in the category of Python types maps the type a to the type F[a] and a function from a to b to a function from F[a] to F[b]. The simplest functor in Python is the list constructor []. List comprehension is a clear example of mapping a function between types to a function between lists. In the category whose objects are functors, a monad is a triple, consisting of a functor and two natural transformations (that is the name of the mapping between two arrows from functor to functor). One acts as the identity under composition and the other composes two arrows. That is the monoidal structure. A list is also a monad, with double list comprehension acting as the composition. Since the result is a list and not a list of lists this operation is called flatMap in some functional languages. Scala and Haskell allow using comprehensions with any monad. Since this allows sequencing operations, it is commonly called a glorified semicolon. Monads allow reifying computations as data structures. Which means that instead of calling an API immediately one obtains instead a description of the API call. This is the reason why it is said that in pure functional programming there are no side effects. All functions return data. In Haskell only the main function can execute side effects. They are called side effects because in mathematics there is no function for sending emails. The idea of functional programming is to make writing software akin to writing a book in Word instead of with a typewriter. It is not for academics. It is what engineering always does: it gives us more powerful and sophisticated tools that allow us to go farther than our ancestors. The Joy of Abstraction is a quite accessible introduction to category theory. Also check out CQL and Ballerina. Programming with diagrams is not old-fashioned, it is only getting started. Thanks for making this video Arjan! 🎉
Edit: just saw the bit about “there is not function for sending emails” hahahaa
Nice video !! I took haskell class at collage but I never imagine I could bring this to python.
Thank you! I'm glad you enjoyed the content.
best description i've seen so far, But.. how many maybe's do you have to create in your code. One for null values, one for oversized strings, one for car number plate formsts, one for sanity checked order values..?
I love this kind of videos! Do you have any plan to create a video or a serie talking about rust? 🤔🤔
Alas, by devoting his life to the way of the serpent he has forfeit his right to preach the good word. If only the high priests of Rust were more benign in their judgement...
Typescript is also not a functional programming language. Even though with fp-ts you can do it very comfortably. It provides you all the monads you need - and I find the "exception handling" in fp superior to the "object oriented"
Excellent post, thank you.
Thank you for the kindness, Greg!
Great channel, subbed. Reddit sent me here. I am still working through a bunch of kaggle courses to learn python all the way up to AI coding. Even me having a barely basic understanding of the syntax, this was extremely easy to follow. 😊
Have to see this a few more times… 🤓
So, would a dataframe be a monad? Since you can chain functions on a dataframe together and they return the DF
You mean to ask whether it is an endofunctor?
If you limit it to only having a map function that acts on the data in the dataframe, then I guess so... (The chaining of other functions/methods would be a bad example, I guess, because they process the value directly in some way, they don't map a user-supplied function necessarily.)
But that doesn't make it a monad yet, does it? That also requires an associative binary operation with an identity element. What would that be?
(Btw, I'm not an expert; just following the logic from the video.)
No, data frames in themselves are not monads, but some operations on data frames are monoids, e.g. data frame addition. Moreover addition is a commutative monoid, and the nice thing about commutative monoids is that you can use them in a reduce operation, e.g. in Apache Spark, to easily parallelize addition.
Best explanation of monads I've seen.
Glad you enjoyed the content!
I think Monads have a usecase when dealing with computationally expensive processes, e.g. fitting multivariate functions. It would avoid spending a lot of time on values that you know would throw an error in later parts of the algorithm.
You are correct that operating on monads shouldn't require any boilerplate. In fact, it requires neither that nor close integration. A monad is not a runtime representation or syntax feature, but an abstract category: a relation formed by morphisms defined over structures, with coherence properties. You never arbitrarily define a composite class or modify the language to exploit the monoid formed by integer addition, the functor formed by a list, or the monad formed by async actions. Property based testing, perhaps a runtime protocol, would make more sense.
Generally speaking, the separation of behaviors from representations is advised in the functional style, so rather than considering a monad as a tightly coupled object to be converted to, consider it as operations which may be defined over potentially existing objects, if they haven't already. It would be fairly trivial to define the bind operator as a regular function decorated with singledispatch, for instance. In fact, type classes in Haskell are implemented similarly with an implicit parameter gathered by the type system.
To make this concrete, consider the list monad:
let unit(x) = [x]
bind(xs, f) = [y for x in xs for y in f(x)]
where fail = []
Extensibility without modification is a powerful principle of software architecture closely related to the Expression Problem. Otherwise, excellent content, as always!
What makes an object a Functor?
Map
What makes an object a Monoid?
Reduce
What makes a Functor a Monad?
FlatMap/Bind and because they are wrappers.
I believe the concept of the monad can be very useful. I noticed a lot of people using the exception framework to communicate business validation errors in a DDD architecture. To me this has always been kind of weird, because it is expected behaviour, and not an exception. So switching to another control path is kind of off.
I am using Result bags for this, a custom solution to communicate errors, warnings, etc. Monads could be nice for this as well as it doesn't break the main control flow and leaves the exception framework for what it is meant for.
I get what you mean but you’re a little bit wrong in the way that the reason why exceptions are called exceptions and not errors is that as their name indicate they’re not just meant to deal with errors, even though in the real world they’re mostly used to do that. One can argue that using exceptions to deal with exceptionnal behavior is by definition using the tool for its precise purpose. I happen to work with the DDD philosophy (DDD is not really a specific architecture) and for our project we created a DomainException (which is then subclassed for more specific purposes) in order to differenciate technical error handling and domain validation. This also makes it easier to setup a middleware in the webapi to bubble these appropriately towards the UI.
There may be ways to do that in a more elegant fashion, however it has the virtue to keep things simple and so hold up the criminally underrated KISS principle.
What I'm missing in this video is an explanation of higher-order functions. I'm aware you explained it before, but that was a while ago.
I already know the concepts in the video, but if I didn't I would wish I knew what a higher-order function was.
very nice, thank you, complex topics but I think I´ve learned something!
I'm happy to hear! Glad you enjoyed the video :)
I'm a mathematician who works with monads on a daily basis and yet I've never managed to figure out why programmers, of all people, care about monads.
I have to admit, I still have to squint to see the connection to the arcane programming terminology :p
A category is just a notion of stuff and stuff that goes from stuff to stuff.
Functors are stuffs that go between categories, so it has to be able to do something both to the stuff and to the stuff that goes from stuff to stuff, in some consistent way.
Endofunctors go from a category to itself. In particular that means you can do it multiple times to the same stuff.
A monoid is a stuff that has a way to "multiply" different copies of itself. For endofunctors that multiplication is composition, and the "way" is to compute the composition to an instance of just a single use.
Thanks for the explanation. I agree. I love the predictability of modads, but at this time using them in most python projects is *overkill*.
I understood... at least half of that. I can definitely see the use case for them, at the very least, though I admit some of the construction logic is a bit confusing to me(though I've never really understood generics so that might be part of it).
This is great!!! Thank you so much for making this
Thank you for the kind words!
Monads!
Love the videos, Arjan
Glad you're enjoying the content! :)
Something isn't quite right with the example at 4:15. What do you exactly mean when you say it's not an endofunctor? I mean, in a programming languages we usually talk about only one category - category of types of the language. Which means all functors are endofunctors just by nature. And it's not even a functor, at least because map isn't polymorphic as it should, it only works with functions from str to str. Correct me if I'm wrong, maybe I just misunderstood.
+1. I also think the definition of EndoFunctor was not technically correct. The rest of the video is still very good though.
@@MdImrulHassan I wouldn't agree on the rest of the video. Many details in the video aren't correct. I understand that there was a good intention to make this video easy to comprehend, but it sacrificed correctness so much that can in fact worsen your understanding instead. It's a really difficult task to make a "monad tutorial" easy to understand. But at least we should appreciate the efforts.
Finally a good explanation.
I agree with your assessment: it doesn't seem to integrate well with existing "pythonic" code.
Love the t-shirt, where'd you get it?
thanks for the vid, just a small error:
6:35 talking about the binary operation of a monoid
"it should be associative so the order in which you do the binary operation shouldn't matter..."
That's not associativity but conmutativity "a·b" = "b·a" and the binary operation of a monoid does not need to be conmutative.
What you meant to say was "the way in which you group operation doesn't matter" that is:
" (a·b)·c " = "a · (b ·c) "
I guess you wanted to say this. Just to clarify.
спасибо, Эржан
Good Stuff....... Thank you..
Glad you enjoyed it!
Small correction on your Maybe monad. Bind always returns `Maybe[U]`. The return type of `Maybe[T] | Maybe[U]` is incorrect.
Thanks. I am studying FP and Thinking how to apply some concepts in Python. You rocks.
I'm happy to hear it was helpful, Lucas!
Fantastic video
Thank you!
Great use of decorators!
Thank you, Jay!
Is the method `map` in the class `Functor` the same as `>>=` in Haskell?
Interesting. I could see a potential use for this in data pipelines where you have succesive transformations and don't want to put error handlers everywhere. On the other hand, it would be nice to know exactly where the error occurred so maybe not 😂
yeah that's always the main argument it seems like.
That’s what I’m thinking too, it makes it very easy for ‘lay’ people (eg. Mathematicians, hence its use in Haskell) to write code against a value that doesn’t necessarily error out, but on the other hand, you will often get garbage-in, garbage-out type situations that makes it harder to understand where things go wrong. I think this would be useful in languages that have no verbose error handling (like Haskell) where you are expected to practically write your own exception handler. And I also think that this is just one step to the hell that pure object-oriented languages are, I can see the temptation to make a spaghetticode of things that are abstracted away to that level.
Gods, this will be so useful for my work.
Glad to hear it was helpful, Victoria!
Amazing!
Thank you!
Thanks!
Glad you enjoyed the content!
Now I will be able to answer what a monad is when a relative asks me in a family funeral.
How about adding an `exception` field to Maybe to save whatever got raised. It could allow better user feedback, and also better debugging because the programmer could see the actual exception.
I watched this and realised i will never be using this and dont want to hear about it again
Oh these are GREAT
Thank you for the kind words!
please explain the Monad using Typescript!
Libraries: returns, expression - any other ?; Railway programming - usefull for pipelines, etl process
The good thing about Monads is that NOONE, not even a single developer in the world, even ones that ONLY use funcional programming, needs to know what it is.
I don't use monads in my normal python development, but i enjoyed the theory here regardless.
90 seconds in and he said the thing!
in most languages it does not make sense to directly implement monads as a single entity. it can be beneficial only in languages that have function-based dispatching (not class-based) and that can reliably inline higher order functions.
errors as values are totally superior to exceptions outside of scripting. i wish there was a way to switch between those on per-session/per-build basis (theoretically, this is possible).
You said associativity but explained it like commutativity. Didn’t even notice until I was confused by your code that tests for associativity of the monad when I wondered „Huh, where is he switching the order?“.
Would be nice if you corrected that. Otherwise nice video.
Option is probably more like maybe thab result is.
Monads the best thing OOP has to offer
Books there? Clean Code, Clean Architecture, Programmer's Brain. The last one?
Lovely!
Oh shit I think I kinda almost understand what an Endofunctor is…. One step closer to:)
I've become a better programmer watching this channel.
Glad you've been enjoying the content!
I understand classes, Objekts and functions/methods in Python. So was interested what a functor is. The definition of the map Returns an Objekt or instance of the class. Thus the definition reminds me of recursive functions.
But honestly, I didn't understand all the Rest of your Video. No Definition of func(self.value) sent me into compile error Status.