the timing of it is insane, I have a program that receives user input and it also polls updates from some API. I kept having random issues with starting a loading screen but something erroring and it never leaving the loading screen, or it stopping the polling but never starting it it then occurred to me that context managers are a thing and even if I don't need clean-up like closing a database connection or closing files, that I can use it for re-turning on the polling and for changing the screen away from the loading screen I then asked myself what happens if you do try: finally: and return in the finally block (I expected it to run the finally, clean-up but then prioritize the first return) this video appearing and these things being covered here 1 day after I implement them in a real project is quite the timing
This video is brilliant, I've found that there isn't much information online which clearly and simply explains context managers in a way like this. Appreciate it.
There was a lot of stuff in this that I didn't know I didn't know. I thought I had a pretty good understanding of try/finally and context managers, but it turns out there is quite a bit more. Especially that finally pre-empts a return and variations thereof.
I'd like to provide a (somewhat hacky) use case for suppressing exceptions in __exit__: conditional context managers. In my case, I had an external API with a notion of "tasks" where the code should only run if the task is not done, and once the code runs successfully it should mark the task as done (so it doesn't do it again, and so other systems [or humans] can view which tasks have been completed). In the first line of the with block, I can check if the task is done and raise a custom exception if it is; then, in __exit__, I can simply suppress this exception and exit from the with block. If any other exception was raised it can propagate like normal, and if no exception is raised it marks the task as done.
I've got another one for ya! A while ago, I made a Transaction context manager that would run a stack of undoable actions. An exception while performing the actions would tell the Transaction to undo all of those actions, then suppress the error. The gui running the Transaction would just check the transaction's result and decide what to do with that exception. It was super handy for performing file operations.
With the 'return print(a)' example, we can think of return print("hello") as shorthand for x = print("hello") return x In this case, one can think of the close being called after the print but before the return.
Very informative video as usual. I would add that disassemblling a minimal example like from dis import dis def test(cm): with cm: pass dis(test) was pretty surprising
I used to be scared of your videos. Now they're a walk in the park. Thank you, and I'm glad advanced python isn't as scary anymore. Your explanations are great
Regarding 7:12 where you explain the semantics of with statements, is the finally necessary at all? It seems to me the code would be functionally equivalent if the finally label were removed and you dedented the last two lines
It would not be the same; if you moved the last two lines out of finally, they wouldn't be executed if except is raised. But you are right that you don't need finally. Sometimes we want to ignore the exception, and sometimes we want to perpetuate it. We need a way to keep track of that. I store the error in `error` and check if it is undefined at the end of the try/except block to see if any errors were throw that we do care about: ``` throw = random.choice(["value","other"]) error = None fp = open("test.txt", "w") try: if throw == "value": raise ValueError("bad") else: raise Exception except ValueError: print("There was a value error") except Exception as err: print("Something else went wrong") error = err fp.close() if error != None: raise error print("Keep working") ``` Finally is useful as shorthand to do the same thing. Finally perpetuates errors raised in exception blocks: ``` fp = open("test.txt", "w") try: if throw == "value": raise ValueError("bad") else: raise Exception except ValueError: print("There was a value error") except Exception as err: print("Something else went wrong") raise err finally: fp.close() print("Keep working") ``` Finally does not block/ignore exceptions, it puts them off until it's done working.
14:30 I would say this is quite useful if you have a complicated enter/exit logic with a lot of local variables used before and after "yield" (e.g. interfacing with a 3rd party library in a specific way) where also wrapping in a class would be overkill because that would be the only use of the class.
Hey, Excelente video throuought! Wanted to add also a sugestion I think that a vertical split would be a nicer look for both the terminal and the file!
Context manager would be great for a machine learning program that you need to terminate early. Write the exit as a summary/save function and you're good to cancel it whenever.
The reason I prefer the contextmanager decorator is just cuz it’s pretty straightforward to define a function vs. designing a class that needs to be instantiated at some point within the program.
I wish that the yield statement RETURNED the exception instead of raising it, in the contextmanager decorator. I.e: exc_type, exc_val, exc_tb = yield val
Dissapointed to hear you say that about the contextmanager decorator. It is by far the superior way to do simple context managers in a clean way, even with the try-finally. However, the only reason that you need the try-finally in the generator is that the decorator will re-raise the exception inside the generator at the yield statement for you to catch it and do something with it. In the example shown, as well as most other uses of context managers, you don't care about the exception. You just care about making sure the resource is released/closed. There could be a different decorator (or a parameter to the decorator) that disables the re-raising of the exception inside the generator so that an exception happening inside the with block won't prevent the closing code inside the generator from running. Functions will always look cleaner to me than peppering classes everywhere, so maybe I'm a bit biased.
I just made a cleaner decorator. Instead of raising the exception, the yield statement just returns it. And you can suppress it by returning True. It's published on PyPI under ez-context
just as it appears to me as a noob but try finally seems kinda useless. usually we want to catch our errors and say something, so it seems to make more sense to do try: pass except (Error(s)' types) as E: pass [then you could just write what you want to do regardless on a non-indented line] whats the point
Ah, there is a bug in the π code. The pi() function is taken from the documentation of the decimal module I noticed, and contains the same bug. If you compare the term t to zero, then the loop terminates as expected once the additional terms become zero within the precision. The current test can never become False.
The output shown in the video is from running the code shown in the video, so it does terminate. I don't follow the logic you stated in your comment, could you elaborate and give code that exhibits this infinite loop behavior you mentioned?
@@mCoding My bad. I missed the lasts = s at the beginning of the loop. Without that you have to check if t ≠ 0. Multiple ways to skin a cat I guess. Not I still have to figure out which sequence is used. n follows the odd powers ((2*i-1)²), d follows ((4*i+1)² -1), for i ∈ {1, 2, 3, …}. But I haven't found the sequence yet.
I think it's shicking how "easy" it is to break python programs that use context managers, as shown with your example at the end. Definitely an important security issue IMO.
06:08 is the black magic I'm always excited to learn about in Python.
or in Java (yes, you can do all these black magic in Java's `finally` too 😂
2/3 python channels I watch in one space! I wonder if arjancodes will be here 😂
Man, never knew of return overriding on finally. Such simple, deep, examples.
the timing of it is insane, I have a program that receives user input and it also polls updates from some API. I kept having random issues with starting a loading screen but something erroring and it never leaving the loading screen, or it stopping the polling but never starting it
it then occurred to me that context managers are a thing and even if I don't need clean-up like closing a database connection or closing files, that I can use it for re-turning on the polling and for changing the screen away from the loading screen
I then asked myself what happens if you do try: finally: and return in the finally block (I expected it to run the finally, clean-up but then prioritize the first return)
this video appearing and these things being covered here 1 day after I implement them in a real project is quite the timing
finally teaching instead of showing off which lots of channels are suffering with
This video is brilliant, I've found that there isn't much information online which clearly and simply explains context managers in a way like this. Appreciate it.
I like how functionality explained using a bunch of prints
There was a lot of stuff in this that I didn't know I didn't know. I thought I had a pretty good understanding of try/finally and context managers, but it turns out there is quite a bit more. Especially that finally pre-empts a return and variations thereof.
Thank you for the great video James! Clear and straight to the point as always
I'd like to provide a (somewhat hacky) use case for suppressing exceptions in __exit__: conditional context managers. In my case, I had an external API with a notion of "tasks" where the code should only run if the task is not done, and once the code runs successfully it should mark the task as done (so it doesn't do it again, and so other systems [or humans] can view which tasks have been completed). In the first line of the with block, I can check if the task is done and raise a custom exception if it is; then, in __exit__, I can simply suppress this exception and exit from the with block. If any other exception was raised it can propagate like normal, and if no exception is raised it marks the task as done.
I've got another one for ya! A while ago, I made a Transaction context manager that would run a stack of undoable actions. An exception while performing the actions would tell the Transaction to undo all of those actions, then suppress the error. The gui running the Transaction would just check the transaction's result and decide what to do with that exception. It was super handy for performing file operations.
@@nicolasmaclean2895Nice!
With the 'return print(a)' example, we can think of
return print("hello")
as shorthand for
x = print("hello")
return x
In this case, one can think of the close being called after the print but before the return.
Very informative video as usual.
I would add that disassemblling a minimal example like
from dis import dis
def test(cm):
with cm:
pass
dis(test)
was pretty surprising
I used to be scared of your videos. Now they're a walk in the park. Thank you, and I'm glad advanced python isn't as scary anymore. Your explanations are great
Really great advice as usual, James.
Regarding 7:12 where you explain the semantics of with statements, is the finally necessary at all? It seems to me the code would be functionally equivalent if the finally label were removed and you dedented the last two lines
It would not be the same; if you moved the last two lines out of finally, they wouldn't be executed if except is raised. But you are right that you don't need finally.
Sometimes we want to ignore the exception, and sometimes we want to perpetuate it. We need a way to keep track of that. I store the error in `error` and check if it is undefined at the end of the try/except block to see if any errors were throw that we do care about:
```
throw = random.choice(["value","other"])
error = None
fp = open("test.txt", "w")
try:
if throw == "value":
raise ValueError("bad")
else:
raise Exception
except ValueError:
print("There was a value error")
except Exception as err:
print("Something else went wrong")
error = err
fp.close()
if error != None:
raise error
print("Keep working")
```
Finally is useful as shorthand to do the same thing. Finally perpetuates errors raised in exception blocks:
```
fp = open("test.txt", "w")
try:
if throw == "value":
raise ValueError("bad")
else:
raise Exception
except ValueError:
print("There was a value error")
except Exception as err:
print("Something else went wrong")
raise err
finally:
fp.close()
print("Keep working")
```
Finally does not block/ignore exceptions, it puts them off until it's done working.
I see, thanks!
14:30 I would say this is quite useful if you have a complicated enter/exit logic with a lot of local variables used before and after "yield" (e.g. interfacing with a 3rd party library in a specific way) where also wrapping in a class would be overkill because that would be the only use of the class.
While I now prefer to deal with absolute file paths, sometimes a context handler for changing the current working directory is very useful.
crazy timing as I was just trying to figure out how to do the `with...as` thing with a selenium driver.
woah I did not know one with statment could have multiple things in it that is great!
Hey, Excelente video throuought!
Wanted to add also a sugestion
I think that a vertical split would be a nicer look for both the terminal and the file!
I love context managers
Context manager would be great for a machine learning program that you need to terminate early. Write the exit as a summary/save function and you're good to cancel it whenever.
Im ml you can use callbacks to save every batch. I think there is even already built in options for popular libraries
The reason I prefer the contextmanager decorator is just cuz it’s pretty straightforward to define a function vs. designing a class that needs to be instantiated at some point within the program.
I just wish the yield expression returned the exception, instead of raising it.
@@marckiezeendertry creating your own decorator then :)
@@omgwtfafterparty Ok. Here it is: pip install ez-context.
16:09 The code uses the proposition that pi = 3 + 6 * sum_{n=0}^inf ((2n+1)!!)^2 / (4n+6)!!. I don't know how to prove this.
Thanks for the video!
I wish that the yield statement RETURNED the exception instead of raising it, in the contextmanager decorator. I.e: exc_type, exc_val, exc_tb = yield val
Copy the implementation and make the changes you want to see!
@@mCoding I did it! And I published it on PyPI under ez_context
Super useful!
Dissapointed to hear you say that about the contextmanager decorator. It is by far the superior way to do simple context managers in a clean way, even with the try-finally. However, the only reason that you need the try-finally in the generator is that the decorator will re-raise the exception inside the generator at the yield statement for you to catch it and do something with it. In the example shown, as well as most other uses of context managers, you don't care about the exception. You just care about making sure the resource is released/closed. There could be a different decorator (or a parameter to the decorator) that disables the re-raising of the exception inside the generator so that an exception happening inside the with block won't prevent the closing code inside the generator from running.
Functions will always look cleaner to me than peppering classes everywhere, so maybe I'm a bit biased.
I just made a cleaner decorator. Instead of raising the exception, the yield statement just returns it. And you can suppress it by returning True.
It's published on PyPI under ez-context
"People often say that Python is easy to learn."
more like, easy to write
context managers/try-finaly
defer
destructors
basically the same thing
This should have been posted yesterday, on Pi Day.
just as it appears to me as a noob but try finally seems kinda useless. usually we want to catch our errors and say something, so it seems to make more sense to do
try:
pass
except (Error(s)' types) as E:
pass
[then you could just write what you want to do regardless on a non-indented line]
whats the point
Python 3.10 here; use with decimal.localcontext(decimal.Context(prec=10000)): instead.
nice, & thank you
Discord server gang 🪃
Much appreciated!
I feel like more languages should have a `with` construct.
Great video
just as "super considered super":
finally means finally.
Wuhu!
Maybe return shouldn't have been allowed in finally.
Ah, there is a bug in the π code. The pi() function is taken from the documentation of the decimal module I noticed, and contains the same bug. If you compare the term t to zero, then the loop terminates as expected once the additional terms become zero within the precision. The current test can never become False.
The output shown in the video is from running the code shown in the video, so it does terminate. I don't follow the logic you stated in your comment, could you elaborate and give code that exhibits this infinite loop behavior you mentioned?
@@mCoding My bad. I missed the lasts = s at the beginning of the loop. Without that you have to check if t ≠ 0. Multiple ways to skin a cat I guess. Not I still have to figure out which sequence is used. n follows the odd powers ((2*i-1)²), d follows ((4*i+1)² -1), for i ∈ {1, 2, 3, …}. But I haven't found the sequence yet.
Rust
Por que amigo?
I think it's shicking how "easy" it is to break python programs that use context managers, as shown with your example at the end. Definitely an important security issue IMO.