I am a masters student with major in data science. Though I am familiar with python(mostly) but your insights are amazing and the way you articulate. Really appreciate your content Tim :) Thanks so much.
One tip that I don't see a lot is a method for taking over iteration of another iterable. Let's say you have an iterable out of which you would like to make a generator, such as in the example at 7:31. Instead if manually looping over the iterable and yielding each value individually, you could simply write the following: def gen(n): yield from range(n) It's way more concise. Hopefully that's helpful.
Another way of making a generator is by using () other then [] in the list comprehension: a = (i**2 for i in range (100)) print(a) # < generator at...> for i in a: print(i)
Aint this only under the assumption that data in the variable a is unchangeable afterwards. Both a list[] and tuple() should work indifferently as example
@@jeverydk I think I didn't get what you said In this example, a is not a tuple(), it's a generator, that is, it's gonna generate the values lazily. If you try that with range(10**9) in a list() or a tuple() you sure gonna run out of memory, whereas using the generator as shown you wouldn't have the memory issue
I enjoy your videos. Not only do you explain how, but you also explain why. You could explain to me how something was built. However, explaining why this method was chosen over a different method, is a whole different level of understanding. 10/10
Great video, as per usual. Tim is really a programming prodigy and a great teacher, making complicated topics seem simple and easy-going. Thank you! As a bit of trivia, a useful method for keeping track of large numbers in code is to use the " _ " character for visual delimitation: >>> nr = 1_00000_000_000 >>> print(nr) Output: 100000000000.
Hi Tim. I'm a software tester working in automation, and having a go at intermediate level certification in Python. This is what I needed has complement to Python Institute's course. Thank you for sharing your knowledge! Greetings from Portugal.
At 8:28 you create a generator object 'g' that you then use in a for loop. If you are going to use a generator in a loop, you can write for i in gen(1000000): and miss out creating g.
Consider using what is called generator expression much like list comprehension, the difference being you get a generator object instead of, well, a list. Just wrap the expression with parentheses instead of square brackets. gen=(x**2 for x in range(10000000)), print(gen.next())
Keep up the series man! I'm a college student as well and most of the time, you are one of the channel which has helped me to continue learning python and self explore projects on the side to build my portfolio!
You can also create asynchronous programs using generators because you can pause a function with yield and then use the send method of generators to continue execution. Not that this is insanely useful but perhaps nice to know ;)
Very helpful. Thanks for sharing. Some follow up questions: 1. Would I be correct in thinking that underlying all of this is Python utilizing some sort of linked list data structure to know where it currently is within the iterable? Is that how/why the __next__ method is applied? 2. I noticed when I ran the test on size using the sys.getsizeof() method for the same size list, I got 87624 for the list and 120 for the generator. Why the difference? Does it have anything to do with 32 bit vs 64 bit systems?
What happens if you need to collect all the values? Do you still need the list comprehension, or can you output the generator to a list, and does that save any memory compared to the list comprehension? Thanks
@1980legend if you need all of the values at the same time, then you cannot save any memory using a generator. But if for example you only need to remember the last 100 values or the largest 100 values of the values seen so far, then you could feed the generator into a queue or heap and reduce your overall memory usage.
@@loganrowe1727 thanks. So just so I have this straight. If you iterate through an array or list but after you need access to all the resulting values, you don’t gain anything by using a generator? If that’s so what’s the point of a generator? To pipe a number through a function in-line for collection later?
@@1980legend As soon as you make an array to hold the N values a generator would create, then a generator serves no purpose because O(N) memory would be used. However when you only need one or a few of the N values, then you can use a generator to reduce the total memory. i.e.: for i in range(N): range(N) is a generator so instead of storing N values in memory you only need store one value at a time so it is O(1) but for i in list(range(N)): creates a an array that stores N values so it requires O(N) memory.
Then, I guess, range() also uses yield keyword, right? I used it a lot in C# with many Reflection APIs: the function actually don't run when it is called, but only when the returned value's iterator is requested, just until the next yield keyword at which time it is paused...
so let me rephrase it in a very simple way: return vs yeild in a function or method be like: return: i am doing barbell curls in a gym and also counting my reps, so my focus is on muscles as well on counting. yeild: i am doing barbell curls but i ask my friend whos name is "generator" to count for me while i am doing the exercise with a free mind. This will consume less memory of our brain practically.
Hello Tim. I am learning your pygame tutorial series.I have a question.I downloaded the sprite sheet of Itachi and i was able to move him left and right. but i wanted to do a power attack just by pressing a key. How can i make him to do that animation by loading images . Please reply
As always, amazing tutorials. Looking at the metaprograming tutorial made me question, is it possible to bring the DateTime helpers from ruby onrails to python? you can do very nice things with those, mainly for time calculation like 10.minutes.from_now or 2.hours + 20.minutew + 15.seconds
You can totally do that, yes. The built-in pathlib library does a similar thing with file paths: Path objects implement __div__ to allow /-chaining of paths. Your datetime idea might be a great little project to practice how to use dunder methods in Python :)
that's because you have a length 1 array of generators, not of the generator''s In [17]: x1 = array(i**2 for i in range(10)) In [18]: x1 Out[18]: array(, dtype=object) In [19]: x2 = array([i**2 for i in range(10)]) In [20]: x2 Out[20]: array([ 0, 1, 4, 9, 16, 25, 36, 49, 64, 81])
Totally amazing, I´m going through all of your videos bro, you de best. I´d like to know if you know about Cython, it´s off topic but I´d like to do faster programs with Cython and I was wondering if you may know. Anyways, keep your hard work, I´m following.
I've seen a substantial speed up by using generators over lists, but this really depends on how you implement both approaches. It mostly comes down to memory management overhead that can be reduced by using generators.
I am a masters student with major in data science. Though I am familiar with python(mostly) but your insights are amazing and the way you articulate. Really appreciate your content Tim :)
Thanks so much.
def fibb_gen(n):
a,b=1,1
yield 1
while a
One tip that I don't see a lot is a method for taking over iteration of another iterable. Let's say you have an iterable out of which you would like to make a generator, such as in the example at 7:31. Instead if manually looping over the iterable and yielding each value individually, you could simply write the following:
def gen(n):
yield from range(n)
It's way more concise. Hopefully that's helpful.
Another way of making a generator is by using () other then [] in the list comprehension:
a = (i**2 for i in range (100))
print(a) # < generator at...>
for i in a:
print(i)
I was just about to say the same)
Aint this only under the assumption that data in the variable a is unchangeable afterwards.
Both a list[] and tuple() should work indifferently as example
@@jeverydk I think I didn't get what you said
In this example, a is not a tuple(), it's a generator, that is, it's gonna generate the values lazily. If you try that with range(10**9) in a list() or a tuple() you sure gonna run out of memory, whereas using the generator as shown you wouldn't have the memory issue
@@adryelbarros3250 i might have misunderstood your example. My bad
I enjoy your videos. Not only do you explain how, but you also explain why. You could explain to me how something was built. However, explaining why this method was chosen over a different method, is a whole different level of understanding. 10/10
Great video, as per usual. Tim is really a programming prodigy and a great teacher, making complicated topics seem simple and easy-going. Thank you!
As a bit of trivia, a useful method for keeping track of large numbers in code is to use the " _ " character for visual delimitation:
>>> nr = 1_00000_000_000
>>> print(nr)
Output: 100000000000.
Hi Tim. I'm a software tester working in automation, and having a go at intermediate level certification in Python. This is what I needed has complement to Python Institute's course. Thank you for sharing your knowledge! Greetings from Portugal.
At 8:28 you create a generator object 'g' that you then use in a for loop. If you are going to use a generator in a loop, you can write for i in gen(1000000): and miss out creating g.
Probably the best explanation out there.
Best series ever :D thanks Tim!
This is such a good series. Keep it up Tim!!
Consider using what is called generator expression much like list comprehension, the difference being you get a generator object instead of, well, a list. Just wrap the expression with parentheses instead of square brackets. gen=(x**2 for x in range(10000000)), print(gen.next())
Thank you! This series is great!!
Holy smokes, I copied out the code you did in the first two minutes and ran it, my computer made a noise I've never heard before...
xDDDD
Keep up the series man! I'm a college student as well and most of the time, you are one of the channel which has helped me to continue learning python and self explore projects on the side to build my portfolio!
wow! now i understand why yield and next are so useful. that wass greatly explained
You can also create asynchronous programs using generators because you can pause a function with yield and then use the send method of generators to continue execution. Not that this is insanely useful but perhaps nice to know ;)
you mean instead of using events?
Hi, does anyone know why program prints twice more ram used for the same code and same given n as it does at 12:55
Thanks! Question: is there a reason to separate next() and __next__()?
I think not. Put all the code in dunder next() and call the instance with the built-in next(instance) function, which executes instance.__next__().
Very useful, thanks!👌👌
exceptional video
This make so much sense and thank you for your video
Great video! Is there a way to nest generators within other generators? Like nested for loops that create lists, but with generators instead?
Amazing series, I hope there are more!
Very helpful. Thanks for sharing. Some follow up questions:
1. Would I be correct in thinking that underlying all of this is Python utilizing some sort of linked list data structure to know where it currently is within the iterable? Is that how/why the __next__ method is applied?
2. I noticed when I ran the test on size using the sys.getsizeof() method for the same size list, I got 87624 for the list and 120 for the generator. Why the difference? Does it have anything to do with 32 bit vs 64 bit systems?
I got 85176 for the list and 112 for the generator. I wonder why.
the list is a list, meanwhile the generator is a function capable of computing the list.
Great video.
This isn't really related to generators, but why do you define a separate next method rather than just writing the code inside the dunder next method?
What happens if you need to collect all the values? Do you still need the list comprehension, or can you output the generator to a list, and does that save any memory compared to the list comprehension? Thanks
@1980legend if you need all of the values at the same time, then you cannot save any memory using a generator. But if for example you only need to remember the last 100 values or the largest 100 values of the values seen so far, then you could feed the generator into a queue or heap and reduce your overall memory usage.
@@loganrowe1727 thanks. So just so I have this straight. If you iterate through an array or list but after you need access to all the resulting values, you don’t gain anything by using a generator? If that’s so what’s the point of a generator? To pipe a number through a function in-line for collection later?
@@1980legend As soon as you make an array to hold the N values a generator would create, then a generator serves no purpose because O(N) memory would be used. However when you only need one or a few of the N values, then you can use a generator to reduce the total memory.
i.e.: for i in range(N): range(N) is a generator so instead of storing N values in memory you only need store one value at a time so it is O(1) but for i in list(range(N)): creates a an array that stores N values so it requires O(N) memory.
@@loganrowe1727 cool thanks man
Then, I guess, range() also uses yield keyword, right? I used it a lot in C# with many Reflection APIs: the function actually don't run when it is called, but only when the returned value's iterator is requested, just until the next yield keyword at which time it is paused...
I have the exact code with Tim, I try to get the generator size, and it prints 112, does memory consumption different in machines?
great vid dude
Thanks!
hi tim, i by no means am following this tutorial but i would like to know how to download pygame onto python 3.8 on IDLE.
What is a good software to edit video in Linux?
Tim , have you use python for programming problems and data structures?
Did you accidentally post this on private I can’t see it on your channel can only get here by link?
Nope, think it's just loading still. Literally just uploaded it
so let me rephrase it in a very simple way:
return vs yeild in a function or method be like:
return: i am doing barbell curls in a gym and also counting my reps, so my focus is on muscles as well on counting.
yeild: i am doing barbell curls but i ask my friend whos name is "generator" to count for me while i am doing the exercise with a free mind.
This will consume less memory of our brain practically.
Hello Tim.
I am learning your pygame tutorial series.I have a question.I downloaded
the sprite sheet of Itachi and i was able to move him left and right.
but i wanted to do a power attack just by pressing a key.
How can i make him to do that animation by loading images .
Please reply
Heyy Tim! Nice vid,
(It’s Shane btw)
Why not use e notation for a large number?
def gen(n)
for i in range(n):
yield i**2
g=gen(10000)
thank you for your python tutorials now i know how to zip a file that the user input
Great video as always, my question is this, is there any way to reset the generator, so you can iterate greater than once an item??
no, but you can pre-make a copy with itertools.tee
It helps me a lot thanks
As always, amazing tutorials.
Looking at the metaprograming tutorial made me question, is it possible to bring the DateTime helpers from ruby onrails to python? you can do very nice things with those, mainly for time calculation like 10.minutes.from_now or 2.hours + 20.minutew + 15.seconds
You can totally do that, yes. The built-in pathlib library does a similar thing with file paths: Path objects implement __div__ to allow /-chaining of paths.
Your datetime idea might be a great little project to practice how to use dunder methods in Python :)
Hi Tim
Hi
Kite incompatible with Spyder 3.3
Good stuff. Could you do a video on lambdas, and also coroutines?
Thanks a lot
Does anyone know why I am getting twice the memory usage he gets? I am on a workstation using ECC ram - could that be it?
Is extending Python code with C/C++ programs considered expert Python or expert C/C++?
Nothing, you can copy pasta from net.
I mean it is all about what you want to do.
Please teach some of the most used bulitin fuctions like filter map reduce zip izip nd many more
x1 = np.array(i**2 for i in range(10000))
print(sys.getsizeof(x1))
44 bytes
that's because you have a length 1 array of generators, not of the generator''s In
[17]: x1 = array(i**2 for i in range(10))
In [18]: x1
Out[18]: array(, dtype=object)
In [19]: x2 = array([i**2 for i in range(10)])
In [20]: x2
Out[20]: array([ 0, 1, 4, 9, 16, 25, 36, 49, 64, 81])
I luv u bro. ♥
i was curious if i can fill 96Gb ram, this code did it, but it took ages !
wow, how do you remember all these functions when your are teaching, i don't see any edit outs
doesn't the class need an __iter__() method ??
my brain just exploded.
Oh yeah i just learned yield from niel patel books and here comes the explanation, thanks bro i enjoyed the video
Totally amazing, I´m going through all of your videos bro, you de best. I´d like to know if you know about Cython, it´s off topic but I´d like to do faster programs with Cython and I was wondering if you may know. Anyways, keep your hard work, I´m following.
nice vid man
I think the most important and the most hard theme it is recursion.
Tim is super cute!
😉
How much time do you spend reading each day?
Ty
Thanks.
lmaoo watching 1 hour before my exam
Gaanerator
So, generators speed up the process for Python.
It’s not necessarily speed, it’s just optimizing memory
@@TechWithTim
I understand.
That feature is still useful.
💯💯💯💯💯💯
I've seen a substantial speed up by using generators over lists, but this really depends on how you implement both approaches. It mostly comes down to memory management overhead that can be reduced by using generators.
@@nmertsch8725
I understand what you mean.
Bro just use (i**2 for i in range(1000000000))
No that’s a generator comprehension
@@TechWithTim good video Tim, show us this next time.
Too much kite
If you call it an Expert tutorial, you should have gone through all the other examples like complex computations, closing/stopping, inputs, etc.
🤓👆