Debugging 101: Replace print() with icecream ic()
ฝัง
- เผยแพร่เมื่อ 17 พ.ค. 2024
- Today we learn about the Python package called icecream, which you should use instead of print when debugging your code.
◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾
📚 Programming Books & Merch 📚
🐍 The Python Bible Book: www.neuralnine.com/books/
💻 The Algorithm Bible Book: www.neuralnine.com/books/
👕 Programming Merch: www.neuralnine.com/shop
💼 Services 💼
💻 Freelancing & Tutoring: www.neuralnine.com/services
🌐 Social Media & Contact 🌐
📱 Website: www.neuralnine.com/
📷 Instagram: / neuralnine
🐦 Twitter: / neuralnine
🤵 LinkedIn: / neuralnine
📁 GitHub: github.com/NeuralNine
🎙 Discord: / discord - วิทยาศาสตร์และเทคโนโลยี
![Supernova - aespa エスパ 에스파 [Music Bank] | KBS WORLD TV 240517](http://i.ytimg.com/vi/bepqfR-pxY0/mqdefault.jpg)
You could do this all much quicker by just inserting the built in python debugger with the line "import pdb; pdb.set_trace()" where you would normally put your print statement. It will then start the debugger in your terminal and you can view and even change all variables during execution. It also helps finding exactly where your code is breaking as you're stepping through the code line by line. I can't recommend it enough.
And recall in Python >=3.7 you can simply use `breakpoint()`
Ah! I didn't know that, thanks🎉🎉
Thank you. I have been debugging code for like 2 weeks using print and exit()
Don't get too cozy with that. ><
Professional environments often lack the ability to run debuggers. You are only guaranteed print logging. Debuggers come later. Ice cream as a better print starts to look real good in cutting edge environments.
They may(or may not) have caught up now, but during my time, cloud/embedded systems lacked debug support. Odds are, the next cutting edge thing will too.
@@gpeschkeWhat kind of environment doesn't have access to the terminal? The python debugger is a standard library included with python, and it starts in the same place where you would see your print statements.
You know, from 3.10 onward you can essentially do this without packages by doing print(f"{variable=}").
Putting an = after the thing you are formating makes it so that it tells you the name/calculation together with the value.
He covered this at the beginning
@@shubhamshrivastava5812 well he only hinted outputting with f function, but not the = suffix, which is interesting information.
I think the key advantages of icecream here are - being able to store the returned value, automatic context (line number, function name), ability to write to files rather than stdout, and the ability to disable all ic logging with a single command as opposed to tracking down every print statement you added while debugging.
I would compare icecream to the `logging` package in python's standard library and not to `print` with f-strings.
ok wow. I'm using string formatting for so long but never knew this. Awesome. But the ic looking like less writing required than that;
Thanks for this tip 🙂
Learnt something useful today. Thanks, this is something I will probably use in my classes as debugging is what I do 99% of the time.
Use a debugger
@@Djellowmanas a viewer of bad code, I approve this message.
@@Djellowman vs-code debuger is still not very good with Django... So this ic seems a good option to me
With Django, I failed to use even the PyCharm's debugger 🤷♂️
(I haven't googled yet, though.)
I made a custom logger once with a decorator style function, similar to this and called "log_it", Tired of debugging step-by-step and wanted a complete debug during a full run.
Was like a swissknife that I could simple plug @ on any unknow function that somebody had written to "log_it" and check what was done. Printed the functions names that were being executed, the *args, **kwargs, variables values / types, total elapsed time, memory comsuption, functions results, and log errors aswell. Had a option to print only or save all this data every run in rootfolder /logs.json. There were functions in module to retrieve all the log history back to check what was done.
No complications, just put the module in root folder of any project, decorated the functions with @log_it and the magic happened
I'm not the IT guy (i'm financist) but damn, I felt like a python god when I was learning and wrote those 50-100 lines lol
One of the first things I do in any new language is create a debugging system. Helps to get you confortable with the syntax and environment, and gives you the tools up front to progress more efficiently.
This is nice, too.
Sounds like a good project, thanks
When you say that you create a debugging system, what does this entail for you?
@@JanB1605 Honestly, probably way more than I need. Usually, a sepatate console window that runs with the program, or if fullscreen, a separate screen I switch to that shows the state of current elements. Recently, I've been programming quite a bit directly on my phone in Pygame with Pydroid3, and it doesn't have a way to show debug print() statements in a console like you would normally have in an IDE, so I use pygame to draw the debug info direcrly to the game as it runs. I'll write my debug info to a txt file for later review while also displaying it in the program. The particulars depend on the application, but I usually get the variable name, its value, variable type, line number, filename, possibly a traceback, etc. The normal output you would need to swuash bugs.
Got any video on good debugging systems?
@@emmaaberg3566 There are ton of TH-cam videos on debugging in VSCode. If they don't satisfy you, check out debugging in Visual Studio (similar). There are also a ton of TH-cam videos on debugging in Pycharm if that's your thing. And of course, some for Sublime and others.
Very useful. Don't worry about "professional" way. Log files and print statements are the way to debug large runtime programs.
@RobertLugg, Log files are very useful.
Not going to lie, I use A LOT of print statement to debug. This is very cool , learn something new and will start trying TODAY!
Thanks so much, I'm constantly learning new things from your videos and it's like this video was made for me. I was feeling specifically targeted when you were talking about using print(a) and print(type(a)), etc. 😄
I used at least print(f'(where): {var=}').
Or logging.debug(...)
(Yes, I know, it is not effective, and it is discouraged by the linters.)
I would suggest skipping ice cream and going straight to python logging. It give much more control over how/what logging in emitted. I have found on many projects that starting from day 1 with using logging, instead of print is a great idea. The only thing I see that ic does that other don't is the quick ability to wrap a lines of code in ic and get a decent print that can help quickly debug issues. That said a debugger could also solve that use case.
Same 😊😊
I was going to post this. Using logging levels feels much cleaner and flexible.
Yes I agree
Great insight. Always love your videos!
Only after hearing it a dozen times did I realize that the function ic() is supposed to be a pun "I see" lol
Interesting. Will check it out. The 'timing' of when the icecream output happens looks relevant and you didn't touch on the behaviour there. But a nice introduction. Thank you.
People will move mountains so they don't have to use the logging package
Or use an actual debugger. It's so much better than anything else.
That’s me. I have been putting off learning how to use it
Proper Logging and a real debugger are the most valuable tools for debugging
Great! Thanks for all your work and great videos! 👍
this is good to take debugging step by step, it reaches out to be complicated at the end, so I think it would be better to show the real level at the end, which is the built in debugger, where you can do all the debugging techniques ( breakpoints, tracing code line by line, viewing all variables..etc )
I just found out about ice cream a couple of days ago. It was just what I needed when I needed it. Right now, I am developing in Flask, which doesn't print to the console, unless you flush. Ice cream takes care on that.
I don't think there's anything more 'professional' about using a debugger.
And I'm really happy to learn about this new tool. It looks awesome. I sort of wonder if the enable and/or disable functionality can be used as a context manager.
@@skulver - I don't like them. They encourage the creation of code that can only be understood by stepping through it with a debugger, and they often don't actually work, or they alter the environment enough that the bug doesn't occur when you're running in the debugging environment. They're also often a huge pain to attach to things that are doing things like handling web requests.
As for IDEs, I only sometimes use a debugger in the context of one. Perhaps, more often than not. But I do frequently use a debugger from the command line, when I use a debugger at all.
Sometimes, they're a good tool, and I will use them if I think I can solve my problem more efficiently by using one. But, most of the time, a print statement is all I need.
@@Omnifarious0 I think debuggers truly shine when there's an external dependency. For example, in an ETL script, you will get the data from multiple external sources and no matter how well you frame your code, there's bound to be unexpected behaviour since the data itself might be misformatted. Adding print statements in this case can quickly become tedious especially if the parts of the data are correlated with each other.
Also, the problem with the prints is that it requires due diligence to clean them all up once you're done.
Another important use case of a debugger is the flexibility to poke further into the other instances based on the behaviour during runtime. Let say if i got some value as X instead of Y, I can quickly examine why that's the case when the program is suspended at the breakpoint. In case of a print statement, I'd have to rewrite them at a different level and rerun the program. And this would be a huge headache if you code requires a lot of time to setup up things.
@@HT79 - Of course, the flip side of the fact that print statements require diligence to remove is that sometimes the print statements become an excellent guide to what should be logged. :-) Especially at the DEBUG or INFO levels.
@@Omnifarious0 Debuggers are great when they work as intended, Pycharm has an excellent debugger in my experience. However you do run into situations where they become nearly impossible to use solely due to a dependency out of your control. For over a year straight there was a bug in Kivy that had no negative consequence for my application whatsoever but would instantly crash the debugger anytime I run my app, so the debugger was essentially unusable for me for that project for an entire year. I didn't force a solution to the issue, because I'm perfectly comfortable debugging without a debugger, but it just highlights how fragile they can be sometimes.
This is very cool! Didn't know about it, but this looks super helpful. Thanks!
Thanks for sharing! Looks great.
This is a nice showcase of icecream IC. Looks nice for some smaller projects. I do think, however, that there are better options to replace a good old print() statement, some of which are named in other comments. As for the pretty printing, if you add a bit more depth to some of those dictionarie, it could become bothersome to find what you are looking for. It is nice to be able to write the output to a file. Alternative for this includes the logging module which allows you to log on certain levels. Using logging together with some of the other builtin features might give you a nice solution that depends on standard library only.
It is convenient, however, to have everything with a simple function call.
Sounds like I'm calling goodbye() on print()! Thanks for the video mate, super handy
Thanks for sharing this python package - icecream. nice tutorial
5:22 Why the hell does the print execute before the ic? or is the ic running in like a thread seperate from the main thread and just takes longer?
or does it only print normally when you assignt it to a variable and the ic only gets printed when printing the variable later on? not on pc so cant test it rn but I think what I said above makes way more sense
It uses stderr by default, look into the source it is pretty simple library
@@TheRealChicken it uses stderr
Very useful! I will be using it from now on changing my 25 year old habit of using print()
Thanks so much. As a Python Data Engineer, this is incredibly useful
I think you should try logging
Excellent video. highly recommended.
very nice library, thanks for sharing. Exactly what i am looking for!! thanks again!
Omg I just finished a project with lots of loops to manage some big dictionnaires and 2d array. Debugging with print and debugger was such a pain in the ass.
I wish TH-cam recommended me this video 2 weeks ago. It could have saved me from lots of troubles
Should have used pandas. You basically reinvented the wheel, but as a cube...
@Zorcky-2C, when dealing with embedded dictionaries and lists like you would get from a non-trivial YAML file, I find the pprint library to be invaluable.
While I can appreciate the tool (I'd written myself helpers that do similar things, so good to know there's a fully developed, feature-rich package for it), I feel that yous examples here made me go "akshually, that's what proper unit tests should do".
Thank you. It is really useful even for beginners.
Lovely! I will use this in the project i'm working on currently.
Nice tip, I tried it and enjoy the results. I believe it will become my tool of choice.
Very good video …. Thanks for letting us know about ice cream …. I feel like the actual ice cream , this will be one liked by all 😀 …. Thank you so very much for such a great video ❤
Thanks - great library !
Brilliant! Thank you for a really useful tip.
The best part is that it returns the value from a function. Makes it easier to throw in print statements.
rich inspect and rich logging handler are a little bit more sophisticated and better, but icecream is also nice.
Why the output for result 30 appears/printed before the output for IC? @5:22
[EDITED] I just saw that this happens on every run, the IC output always comes after the program runs. I'm curious to know why.
Nice package, didn't know about it. But I expected more...
The introduction with the 3 prints (a+b, a and type(a)) made me expect that it would have a simple way to print the type, maybe some statistics about the number of times some line of the code was reached and other more fanciful common debugging strategies.
Also, I didn't like that it returns back what the function would return. This just makes it easier for some `ic` call to go into production by mistake.
I am using the same method in order to reg logs but without icecream, just adding what is needed using strings to a txt file among with the date stamp. Wondering which version is more memory effective.
For stylized output I used "rich" library, usually with "rich.inspect", otherwise "rich.print"
I just quickly checked "ic", and find one big downside, it doesn't work with None values at all.
Usually for this sort of debugging I need to check if value is set and not None, I do this kind of check all the time, but because ic behave differently, it does not work. And those kind of checks would not work at all in REPL. A short example:
>>> ic({"a": None}["a"])
ic| Error: Failed to access the underlying source code for analysis. Was ic() invoked in a REPL (e.g. from the command line), a frozen application (e.g. packaged with PyInstaller), or did the underlying source code change during execution?
let's just say that dict is some variable that I'm working with and it have optional value, with None as result it do a different thing.
this video changed everything for me. thank you 🍦
When I started programming back in 1980/90s we didn't have debuggers. So never really got out of the habit of print statement debugging :D
Can you please teach us how to use keyboard shortcuts in the editor like how you were doing?
Great tour of Icecream. Thanks!
I found this video earlier today. Tried icecream out. Love it. Bye bye print(). An idiom is to use ic();ic(foo,bar) so as to print out the line at the same time. It would have been nice to have some way to tell ic() to always print out the line number. It shouldn't be hard to hack it in, so I'll poke around. I'm actually quite curious how it works.
I think with the ic.configureOutput, just put the prefix as ic() so it'll give the line number every time? Didn't try but could work
@@shu7bh Thanks, I'd missed that.
ic.configureOutput(includeContext=True)
Great alternative without any imports is f-strings with Python 3.8 and up.
For example:
```
def is_odd(a: int) -> int:
return i % 2
num: int = 123
print(f"{is_odd(num) = ") # prints: "is_odd(123) = 1"
mylist: list[str] = ["a", "b", "c"]
print(f"{mylist=") # prints: 'mylist=["a", "b", "c"]'
```
seems like a good solution when using terminal editor, it’s kinda annoying to download debugger
This is amazing! Is there something like this for C++?Regardless, this would be helpful for a couple projects I'm struggling with in python.
In C++ fmt library can print lots of things by default - saves your time to avoid manual looping to print a vector for example. You can extend it too, maybe there are more advanced printing libraries idk. Fmt will be used in regular print in C++23.
I personally prefer to debug but it’s not often possible in multithreaded or other kinds of environments. Ideally you unit test and keep your code clear so you don’t have to debug 😅 but it always comes down to it anyway.
I noticed at around 8:30, you have the ic() statement before the return value, but in the console output, the return value comes before the ic() output. Is there a reason for this?
print prints to stdout, ic prints to stderr, and they can get interleaved due to buffering.
@@vytah Thank you
I tend to do a lot more poking around at the >>> prompt, copying and pasting between an editor and the python prompt while trying things out. You don't lose state that way, files and sockets stay open, etc. which makes it easier to test more complicated things. And the objects are all there if you want to see some details. Depends how comfortable you are with IDEs, but I try not to rely on them much because a lot of the time I'm logged in to a system through 3 successive ssh sessions and just won't have many tools available. I can see icecream becoming part of a workflow once it's installed by default. Most systems I can't even run pip on since they lack access to the Internet.
If you use the -i flag when running a script (python -i main.py) the script won't exit when it is finished, and you can still access variables etc
An interesting alternative would be to use the built in `breakpoint()` function which will set you up using the inbuilt debugger (pdb). You can think of it as a supercharged >>> prompt as it also will allow you step through functions!
@@amminamaromind elaborating for a debugging novice?
Thank you for sharing this very useful tool.
The functionality described at 4:45-4:55 could be achieved by print by using the walrus to assign the variable in question
Which ide is this?,
Also is there a plugin which auto populate argument keys?
This is way better than logger, nice content
Good to know.
Right now, I do most stuff like that:
```Python
pt = print
def print(msg):
if sys.gettrace() is not None:
pt(msg)
```
But I also use BP a lot, since VS starts python in debugging anyway. And if you set a BP, you can then use F11, to step through the code (or start it with F11).
try logging library. Its... 10x better than any print.
Dumb question - @4:05 or slightly after.. you do a keyboard mash to select just that word (print in this case) and replace.. what keyboard combo are you hitting to do that so fast?
I'm familiar with double clicking with the mouse to select just the word but that's finicky and doesn't seem to always work as expected with selecting.
I assume you're also hitting arrow keys for going to next line and it's going to the same location because the lines are, at that place, the same...
But how are you rapidly selecting that word with the keyboard to the paste (or type) IC?
Ctrl-A would select too much and things like Ctrl+shift+end would grab the whole line..
I'm not familiar with a 'just select this word' combo
Appreciate the feedback!
also this IC thing is slick but do people have a rapid way to pull it back out, and leave code functional, when you're done debugging? Curious how much bloat or overhead leaving this in source code would create vs taking it out (at a bigger scale of course). As well as - the end product may just want print statements in a lot of those places but not necessarily all; so how do you rapidly replace all or a select few or comment out every debugging occurrence? (Similar to my above question as I find it tedious to swap words and comment debugging lines in/out.
Those are Vim bindings. Check out my videos on Vim or NeoVim. It is worth it :)
Always with something amazing thanks
does it work with pandas dataframes?
Cool.
Pro print for debug :)
Waiting for a django tutorial from you.
What is better, ic or logging?
In Julia you have the built-in macro @show to do that. I love Julia :) and I wish it was more widespread.
The interesting question is, how does this work. If you are familiar with python, you know, that if you see a line like func1(func2(value)), the order of execution is this: evaluate 'value', call 'func2' on 'value', call 'func2' on the result of the func1 call. So func2 will never see that it's argument was generated with func1. So how can the function ic examine it's arguments BEFORE they get evaluated? The answer in case of icecream, it uses a package named 'executing' which let's you inspect the current frame and the related AST. I would like to see a video on that topic.
¡Excelente video! muy buena idea para no tener que llenar de print todo el código
very-very useful. What IDE is this that you are using?
PyCharm
PyCharm
I checked the recent video, it's pycharm
i hope every new language can have something like this builtin
Thankyou. this is really useful to me.
Cool. Seems like it would be smoother for the 1st example to just decorate the add function with ic. Could that work?
how did you do what you did in timestamp 4:11? that's pretty cool.
not a bad place to start, was just curious on an idea that hit me earlier for a print-based debugging library just as a little project idea and wondering if one's been built / how well it works. even if nothing more than a small build for the fun of it, it would be kind of cool to have a whole suite of functions uniquely designed for capturing, tagging, storing, parsing, and comparing variables at common intervals. haven't thought nearly enough about how it would be done optimally (if at all) but it would be nice to have any given value stored alongside different "distilled" states that you set, so that an algo can detect "this is how this data value should transform under conditions A, B, C, and D within these states" and flag + auto-track deviations over time. if the library was robust and clever enough it might be able to catch your own fuck-ups well before you do in those cases where breakpoints and typical debugging isn't ideal.
As a side-note, you *can* capture a function's return data while simultaneously calling it within print, but you'd have to use a walrus operator.
print(result := add(10,20))
print(result)
would print 30 twice.
I think I will start using this module!
Is there a way to configure ic so that it is virtually removed, so that you can leave them in the source code but not only does it not ouput anything, but it even doesn't slow it down?
Of course py is mainly scripts, but I have an issue with the way icecream works - you sort of have to build it into your program. Just because you call ic.disable you're still adding a built in overhead to your entire program, and stripping it can become an exponentially hard task if you rely on it too much. Not to say that it's not useful, but there is definitely a trade-off between development time and performance overhead and you need to consider.
I don't think you should be leaving prints in your code anyways. If you need to hardcore some kind of feedback that's what logging is for.
I Agree with Willman. Icecream is for developing debug, not for production logging. There are other libraries for that.
You can have a conditional import that replaces ic with a no-op if you don't want to have it as a dependency. That's probably what I would do; import ic if it's available, and replace it with a simple print or no-op otherwise.
@@nio804 This is not what OP's talking about.
Personally, I think f-strings are almost equivalent and prevent me from adding another dependency to my project. For example,
print(f'{add(30, 70) = }')
will result in printing: add(30, 70) = 100
This is essentially the same thing that printing with ic will do, just using an '=' instead of a ':'.
I think the nice dictionary printing is a good feature, but I usually just use pretty print (pprint) for this. E.g.
import pprint
pprint.pprint(data)
This does still need to be imported, but only if you need to be printing dictionaries this way AND it's a built-in module so still no extra dependency.
Do you really need it as a project dependency? Have you added pprint to your project's dependencies? Can you use it as a development tool only?
+
Hi, could you tell me.. Why it is better than logging or logger
One hour after deciding to install icecream, I finally manage to reinstall python in a way where pip actually works.
I don't know if someone else ever told you. But I have seen plenty of your videos and they are awesome.
Me along with so many people watch would really appreciate if you could zoom in the recording a little bit. The text is kinda small, it could potentially harm our eyes to watch videos for hours. I am sure you don't want your viewers and subscribers to get eye glasses 🤓 ❤
How do you automiatcally get all the argument names added when calling a function?
Really good info. Like the little niceties ice cream adds
great video learnt something new thanks ✌✌
wow! So nice and really useful! Thanks bro!
Thank you bro this video is so informative
What is that IDE? Doesn't look like PyCharm CE.
My bad, it is PyCharm CE with IdeaVIM plugin.
Logging is great, but dropping in a "breakpoint()" with pdb++ will still be my go to for actual debugging.
Liked. Will need to try ic.
Does this package work in AWS Lambda functions?
Very nice library! ❤
Maybe start looking into adding test coverage if you are actually doing this on a daily basis
Excellent
Can we use this in azure databricks notebook?
Time saver library
usefull info 😀😀thank you bro...
I like icecream but I have seen a bug when using it to print a lot of data. In VS Code the output is truncated. It works fine if I run it without an ide. Can you replicate this error?
Is it there a django compatibility as well?
Very useful. Thanks
a better solution would be adding a simple fstring into the print statement like print(f'{adding(10, 30)=}') and thats do the same, except for the return thing, but i found id more usefull
Ah, this is partially how it would normally work in at least some variant of Lisp I know: (setq result (print "foo")) prints and sets the result variable (called symbol) to "foo" but also still returns "foo".
thanks you bro
This is really helpful, because I'm trying to come up to speed with Python. I used to use Basic, but I see Python is the future. So, I subbed, and please keep making this great content.
Thanks for a great tutorial. Can you also explain how to reset outputFunction to sys.stderr? ic.configureOutput(outputFunction = sys.stderr) errs.