For a newbie in the field of parallel computing and gpu accelerated computing like myself, thanks for taking the time to lay a strong foundation on how parallel computing works. It helped me correct a wrong assumption I had about GPUs.
you probably dont care but does anybody know a way to log back into an Instagram account..? I stupidly forgot my password. I would love any tricks you can give me.
@Trenton Lian thanks for your reply. I got to the site through google and im waiting for the hacking stuff now. Takes quite some time so I will reply here later with my results.
This was a great video to me, I have very limited C++ experience and was looking for an explanation of CUDA. Another video like this could easily have been 70-80% over my head. This one was only about 15% whoosh. And now I actually find C++ interesting again!
Yes. I wrote an articel about it. towardsdatascience.com/fibonacci-linear-recurrences-and-eigendecomposition-in-python-54a909990c7c?source=friends_link&sk=6dbc6bcaf2b9551e108da037963aea33
Nice talk, but your history is wrong -- the first graphics-on-a-chip was done by Silicon Graphics, Inc in about 1985, with the "geometry engine" chip. SGI dominated the industry in the late '80s and the '90s, much like Apple and Google do now. NVidia was a later spin-off in the late '90s formed mostly by people from SGI.
Evans & Sutherland was doing hardware-accelerated, rasterized graphics in the '60s and '70s. SGI did pioneer the bringing of the technology to the general workstation markets in the '80s, though.
@@NUCLEARARMAMENT "Raster graphics" on a chip may have been done earlier than SGI, but not fully 3D polygonal rasterization, which is a the basis of modern 3D computer graphics.
@@paulhansen5053 Also, the CT5 simulator from 1981 may not count as being from the '70s or '60s, but from what I understand, the CT5 was capable of realtime, rasterized, 3D polygonal rendering and was $20 million at the time. It used gouraud shading, if memory serves. There were several other CT (continuous tone) simulators developed by E&S in the '70s that did something similar or of much lower capability than the CT5 of '81. There was also the Digistar planeteriums that date back to the early '80s, and the Picture System goes back to at least the early '80s. Might be vector or raster, not entirely sure myself, though.
I was just starting to learn OpenGL (with the Superbible) to get a feel for how gpus work. So far it's been very fun. I had always assumed it was waaay more complicated to make anything work in a gpu, but it doesn't seem to be the case.
Lmao, I thought I had to write some sort of gpu assembly. But they don't even publish it, in practice, the gpu driver is what exposes what you can do with it... by what I read with a quick google search.
Hey Siraj, I love watching your videos because of the way you tell the story. Great graphics mate. Love the reference to rocket man too... lol keep up the good work.
Siraj, thanks for taking time to create these videos. It is unfortunate that people view your videos and then feel inspired to complain about a free gift. Folks could just keep it moving or add helpful insights.
You can calculate the nth fibonacci number without calculating the previous fibonacci numbers with its closed form. ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-042j-mathematics-for-computer-science-fall-2005/readings/ln11.pdf (page 9)
Did you use an eGPU for your Mac? How did you manage to write CUDA program using IOS? P.S. Thanks for your introduction to GPU. I think this is one of your best videos. Keep on the good work.
Here's an idea. You can speed up Fibonacci number calculation by using the closed form formula. You distribute the multiplication of the powers over multiple threads. To assure accuracy, instead of using floats, you can write a class to represent numbers of the form a + b * sqrt(5), where a and b are rational numbers.
Thread and blocks size they are not infinite. and of which of which I know it is more profitable to perform in a loop than to create large thread and blocks meshes.
Hi siraj, can you guide me what to install inorder to execute CUDA code? I'm scratching my head since a long time for the successful execution of the program on windows using Visual Studio.
In reference to 8:00, what happens if you specify a number of GPU threads/cores that is greater than that of the GPU itself? Is there a GET function that retrieves these values so as to prevent exceptions?
There must be some : min(no_of_cores, input_cores) inbuilt , so if you pass the input cores more than the no of cores , it'll take the no of cores else it'll take your input
Hey Siraj, I am a massive fan of yours and just started drinking coffee to be capable of keeping up with your speed of talking .. just kidding :D, but to the point. I am now starting my third year of Software Engineering degree and would like to create a text summarizer for my Final Year Project. I decided to use the seq2seq that Tensorflow provides, because I recently read a paper saying that a Seq2Seq + Attention + Bidirectional model would outperform every other model for the moment. At the end the paper stated that you can improve its performance by training the model on more and more data. So my question is - Can I use Tensorflow's model and then improve its accuracy by training it with one dataset, then pickling it, then training it further with another dataset and so on... I am stuck for advice and would love it if you answer. All the best
you could also use faster asymptotic algorithm variant, if you want only to compute a single term of the sequence. It suffices to use a square and multiply on a certain matrix. It doesn’t make it parallel though :(
Do we need the learn Cuda if we plan on using Tensorflow? I know Cuda allows us to process on many cores, but I heard something along the lines that Tensorflow already has this idea built in. Is this true, or do we need to pair Cuda and Tensorflow in our deep learning programs?
What machine do you use for your GPU programming? Can you tell us how to port what you are doing from your on-premises computer to a Cloud provider? And which Cloud providers support this? That would be interesting to know.
Easier way to start is a cloud instance, eg. on AWS p2.xlarge (aws.amazon.com/ec2/instance-types/p2/). If you do a lot of computation and want to save expenses for cloud or train faster, you can build you own computer (eg. timdettmers.com/2015/03/09/deep-learning-hardware-guide/).
Errata from a GPU programmer: hardware T&L was not invented by nvidia, they were only the first in the PC space there is a non-recursive formula for the Fibonacci sequence, and the algorithm itself is parallelizable as well ( it just wont be very efficient) stackoverflow.com/questions/16464498/parallelize-fibonacci-sequence-generator all it requires is just the unrolling of recursion to the desired level of parallelism
I understand that this video is meant to be an introduction but can you make a video or two where you go into more detail and do some coding of this? I like your format of short videos with memes as introduction and longer videos for the coding but I can’t find the longer coding video for using CUDA or GPUs
You can also skip the middle man and create directly a python extension which uses CUDA. In my submission you can find a very minimal implementation of it: github.com/tterava/Mandelbrot It has all the necessary functions to allow your your C (or CUDA) code to be called from python code directly. All you need to do is to use the nvcc compiler to build the extension and python knows how to use it automatically.
There are some performance trade-offs when using Unified Virtual Memory (UVM). Memory pages are migrated between the host and the GPU on-demand (plus some nifty prefetching heuristics) but frequent page faults often stall the system. I suggest using explicit memcpy semantics between the host and the GPU.
Thank you Daniel, that was the kind of trade-off explanation I was looking for. Kevin, I'd like to read more about that. New GPUs seem to be _meant_ for that kind of stuff, don't they? I'd be surprised if it would really kill your GPU.
Could you help me, I need the best GPU for science purposes not more expensive than 2500$ (it should work good with double): could you tell me on what characteristics should I look and what is the best variant?
I spend several hours yesterday trying to install tensorflow-GPU in Anaconda so I could use my GPU to speed up training of our neural network. It's an incredible pain in the ass as there are 7565 dependencies that all need to be installed in specific way and you're forced to register for the Nvidia Developer Spam Program. I didn't manage to do it in the end. Anyone have any tip for painless installing of tensorflow-GPU? I was really suprised and annoyed by how difficult it was.
Mac doesn't support CUDA I think?, Maybe if you use windows 10 (Or even better Ununtu) on the Macbook through bootcamp with a 3.0 minimum level GPU card through thunderbolt might work
@@existentialchild698 yeah know, but is a shame having to buy one when you already have one in your sistem. anyways i just make it work. compiled tensorflow with it and now i have enable gpu in my mid 2012 macbook pro
@@MygenteTV Okay. There's also a service made by Google called "Google Colab" that allows you to run your python code on an Nvidia Tesla K80. (That's a really good GPU that would otherwise cost you over $2000 depending on where you buy it).
When ROCm (AMD's new Open Source Deep Learning libs) is stable for Tensorflow, please do a video on it as well. Here is their github: rocm.github.io/dl.html Caffe is finished, while Tensorflow and Caffe2 are currently in development with working builds already. :)
For those who have not tried it out yet and have an AMD chip don't bother...it gives segmentation fault and doesn't run at all and a/c to otoy company they claim that they can do it it's not released yet so sadly go back and do it for open cl and Thanks Siraj, you have great videos. EDIT: Don't even install that nvidia-cuda-toolkit that it will recommend you if you have amd gpu... it will ruin your graphic drivers and you will be unable to boot into the system
That was very well explained. I have only have taken one course, and you made it clearer than my professor or fellow students ever did.
"cuda" in polish means miracles (we pronounce it little diffrent though) so it's kind of funny because gpu seems like little miracle
"Tsoo-dah", no doubt!
This was by far one of the most enlightening videos you have put up on your channel. Thanks and keep up the good work!!
For a newbie in the field of parallel computing and gpu accelerated computing like myself, thanks for taking the time to lay a strong foundation on how parallel computing works. It helped me correct a wrong assumption I had about GPUs.
you probably dont care but does anybody know a way to log back into an Instagram account..?
I stupidly forgot my password. I would love any tricks you can give me.
@Trenton Lian thanks for your reply. I got to the site through google and im waiting for the hacking stuff now.
Takes quite some time so I will reply here later with my results.
@Trenton Lian It did the trick and I now got access to my account again. Im so happy:D
Thanks so much you saved my account :D
@Liam Christian You are welcome =)
LOL. Loved the graphic at 6:23! Brought tears to my eyes.
Thanks for actually showing the speed increase with more threads, haven't seen anyone do this.
Typing 1
can you explain please? I'm not too familiar with c++
@@1kounter '
people who do this have 99% chance of using the ternary operator whenever possible
@@jeffreylebowski4927 So it's not exactly a million elements? This is misleading.
@@pcrizz Yeah i agree, there was no need to make things more complicated and then not even be precise about it. - Its bad to do this as a teacher.
This was a great video to me, I have very limited C++ experience and was looking for an explanation of CUDA. Another video like this could easily have been 70-80% over my head. This one was only about 15% whoosh. And now I actually find C++ interesting again!
OK those animations + your relevant narration is by far the best combination for learning. Loved it!
3:08 actually the n'th fibonacci number can be found in O(log n) using matrix exponentiation.
Yes. I wrote an articel about it.
towardsdatascience.com/fibonacci-linear-recurrences-and-eigendecomposition-in-python-54a909990c7c?source=friends_link&sk=6dbc6bcaf2b9551e108da037963aea33
And you actually do not need matrices, only the diagonal values from eigendecomposition.
It can be found in O(1). As far as I remember the formula is derived using LDU decomposition or Diagonalising a matrix, for matrix exponentiation.
The Knowledge of Siraj knows no bounds.
Video quality is awesome ! Great job
Thank you so much for this video. It has helped me massively to prepare for my computer science exam.
Love your videos. Please don't stop!
Oh Siraj, your tutorials are incredible and inspiring....
I have no idea what kind of videos i am watching ... but i sure will learn
Nice talk, but your history is wrong -- the first graphics-on-a-chip was done by Silicon Graphics, Inc in about 1985, with the "geometry engine" chip. SGI dominated the industry in the late '80s and the '90s, much like Apple and Google do now. NVidia was a later spin-off in the late '90s formed mostly by people from SGI.
Evans & Sutherland was doing hardware-accelerated, rasterized graphics in the '60s and '70s. SGI did pioneer the bringing of the technology to the general workstation markets in the '80s, though.
@@NUCLEARARMAMENT I kinda don't think so, actually don't have time to check it out. Pretty sure that E&S in '60s and '70s was all vector graphics
@@NUCLEARARMAMENT "Raster graphics" on a chip may have been done earlier than SGI, but not fully 3D polygonal rasterization, which is a the basis of modern 3D computer graphics.
@@paulhansen5053 Also, the CT5 simulator from 1981 may not count as being from the '70s or '60s, but from what I understand, the CT5 was capable of realtime, rasterized, 3D polygonal rendering and was $20 million at the time. It used gouraud shading, if memory serves. There were several other CT (continuous tone) simulators developed by E&S in the '70s that did something similar or of much lower capability than the CT5 of '81. There was also the Digistar planeteriums that date back to the early '80s, and the Picture System goes back to at least the early '80s. Might be vector or raster, not entirely sure myself, though.
CUDAs to you, for covering... CUDA!
Excellent presentation! Thanks for sharing this vital knowledge!
I saw this in my recommended. The most I knew about gpu was that u need that for the cool games. Now imagine me watching this video.
Is 'Introduction to CUDA GPU Programming' a new series that is going to come up ?
Swanand Kulkarni I hope so
i wasn't planning on it but this vid seems to be doing well so may consider
these memes increase production quality by 10x
I was just starting to learn OpenGL (with the Superbible) to get a feel for how gpus work. So far it's been very fun. I had always assumed it was waaay more complicated to make anything work in a gpu, but it doesn't seem to be the case.
Lmao, I thought I had to write some sort of gpu assembly. But they don't even publish it, in practice, the gpu driver is what exposes what you can do with it... by what I read with a quick google search.
Hey Siraj, I love watching your videos because of the way you tell the story. Great graphics mate. Love the reference to rocket man too... lol keep up the good work.
Say what you want about Siraj's bad choices recently, the dude is a master teacher.
Freaking
Awesome.
I regret I have but one up-thumb to give.
Siraj, thanks for taking time to create these videos. It is unfortunate that people view your videos and then feel inspired to complain about a free gift. Folks could just keep it moving or add helpful insights.
Another Awesome Video, Loved it
thanks!
Good stuff Siraj!
You can calculate the nth fibonacci number without calculating the previous fibonacci numbers with its closed form.
ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-042j-mathematics-for-computer-science-fall-2005/readings/ln11.pdf (page 9)
Nice read, thanks.
siraj is noob
You beat me to it...
Fib(n) = ( 1.6180339..^n - (-0.6180339..)^n ) / 2.236067977..
it's only an approximation
see the top answer here codereview.stackexchange.com/questions/163354/using-2-threads-to-compute-the-nth-fibonacci-number-with-memoization
Thank you for this video
This was really, really helpful to me. Thank you.
Very helpful video. Thank you :) !
Love your videos bro! Time to put down that redbull though lol just kidding happy holidays!
Just got a Jetson Nano - looking forward to learning more!!
Awesome video, as usual. You should do a video on what makes a good feature and how to select good features (feature engineering). Have a good one!
Love your style... Steel the same Siraj, subject very interesting!!
i like how you pointed at the vertex shader image when you said "cpu"
Absolutely lovely visuals!!!
what a passionate tutorial! I wish you were my professor for my parallel programming course. Well done!
I didn't know TES V was available for the NES platform. I wonder if it has mod support.
Your videos are awesome , thanks a lot for this quality content :)
What do I need to know to create my own deep learning framework like PyTorch? What are the books and courses to get knowledge for this?
Did you use an eGPU for your Mac? How did you manage to write CUDA program using IOS?
P.S. Thanks for your introduction to GPU. I think this is one of your best videos. Keep on the good work.
Here's an idea. You can speed up Fibonacci number calculation by using the closed form formula. You distribute the multiplication of the powers over multiple threads. To assure accuracy, instead of using floats, you can write a class to represent numbers of the form a + b * sqrt(5), where a and b are rational numbers.
You don't need a "for loop". Each thread computes sum of a[blockIdx.x * blockDim.x + threadIdx.x] + b[blockIdx.x * blockDim.x + threadIdx.x].
Thread and blocks size they are not infinite. and of which of which I know it is more profitable to perform in a loop than to create large thread and blocks meshes.
Very useful video, thanks a lot, Go ahead
Holy cow gpu progrmaming is complicated
Hi siraj, can you guide me what to install inorder to execute CUDA code? I'm scratching my head since a long time for the successful execution of the program on windows using Visual Studio.
I have a CS degree but this shit entire level above my understanding
your my new favorite youtuber. LOLs.
It's like missiles of knowledge coming through his brain.😂😂😂😂
In reference to 8:00, what happens if you specify a number of GPU threads/cores that is greater than that of the GPU itself? Is there a GET function that retrieves these values so as to prevent exceptions?
There must be some : min(no_of_cores, input_cores) inbuilt , so if you pass the input cores more than the no of cores , it'll take the no of cores else it'll take your input
thanks for the clear explaination
What's the link for the GitHub at the end of the video?
OK, found it: github.com/alberduris/SirajsCodingChallenges/tree/master/Stock%20Market%20Prediction
Thanks! ^^
thanks bro!
Awesome!
just updated, can't believe i forgot it, won't happen again
Awesome tutorial, thanks.
Did you check the results of the addition? I have run the code and the values of 'y' before and after the add function remained ... what is wrong?
Hey Siraj, I am a massive fan of yours and just started drinking coffee to be capable of keeping up with your speed of talking .. just kidding :D, but to the point.
I am now starting my third year of Software Engineering degree and would like to create a text summarizer for my Final Year Project. I decided to use the seq2seq that Tensorflow provides, because I recently read a paper saying that a Seq2Seq + Attention + Bidirectional model would outperform every other model for the moment. At the end the paper stated that you can improve its performance by training the model on more and more data.
So my question is - Can I use Tensorflow's model and then improve its accuracy by training it with one dataset, then pickling it, then training it further with another dataset and so on...
I am stuck for advice and would love it if you answer.
All the best
For fibonacci you can also just use a memoizer and store the previous values calculated on a cache to make your algorithm faster xd just a trick 👍
you could also use faster asymptotic algorithm variant, if you want only to compute a single term of the sequence. It suffices to use a square and multiply on a certain matrix. It doesn’t make it parallel though :(
Great tutorial man!
Siraj I ll be waiting for your reply...
Can you please make a video on how to use Google collab notebook to train our machine learning project
the cpu is the powerhouse of the pc
Hey Siraj, 2007 called and they want their memes back.
+Tanbir Sohail haha oh shit noted thx
the calling and wanting things back meme is older than that.
Do we need the learn Cuda if we plan on using Tensorflow? I know Cuda allows us to process on many cores, but I heard something along the lines that Tensorflow already has this idea built in. Is this true, or do we need to pair Cuda and Tensorflow in our deep learning programs?
I hope khrono group goes through with its plan of making vulkan a replacement for open cl
Vulkan is the successor of OpenGL.
Dave Axiom but as it got rid of most of that api's( open gl ) baggage it also makes sense to use it for computation
Dave Axiom www.phoronix.com/scan.php?page=news_item&px=IWOCL-2017-OpenCL
Dave Axiom www.phoronix.com/scan.php?page=news_item&px=clspv-OpenCL-To-Vulkan
thanks for this siraj
What machine do you use for your GPU programming? Can you tell us how to port what you are doing from your on-premises computer to a Cloud provider? And which Cloud providers support this? That would be interesting to know.
Easier way to start is a cloud instance, eg. on AWS p2.xlarge (aws.amazon.com/ec2/instance-types/p2/). If you do a lot of computation and want to save expenses for cloud or train faster, you can build you own computer (eg. timdettmers.com/2015/03/09/deep-learning-hardware-guide/).
th-cam.com/video/Bgwujw-yom8/w-d-xo.html
Thanks.
Great video, thank you.
good work, helped me a lot
Errata from a GPU programmer:
hardware T&L was not invented by nvidia, they were only the first in the PC space
there is a non-recursive formula for the Fibonacci sequence, and the algorithm itself is parallelizable as well ( it just wont be very efficient)
stackoverflow.com/questions/16464498/parallelize-fibonacci-sequence-generator
all it requires is just the unrolling of recursion to the desired level of parallelism
Damn you are such a great teacher dude.
OpenCL please
ill put that in the queue
That would be awesome ! Keep it open guys ! (Even is CUDA is pretty awesome...)
THIS!!
We should not be promoting proprietary tech
@@SirajRaval What do I need to know to create my own deep learning framework? What are the books and courses to get knowledge for this?
I understand that this video is meant to be an introduction but can you make a video or two where you go into more detail and do some coding of this? I like your format of short videos with memes as introduction and longer videos for the coding but I can’t find the longer coding video for using CUDA or GPUs
Yeah I need to do a long cuda video good idea
finally u'r coding in C, :) :) :) :) good video man
Step 1: do NOT install GPU driver updates until you're dead certain that CUDA is supported on the given driver version.
You just opened my eyes to parallel programming. Thanks for the quick overview.
So If I was to make my own gpu would I have to learn C++ or something else?
Your videos are awesome .
Cuda was named after a car originally, but then the marketing team though that was lame so they changed the meaning
Thank you for this gift :D
Can this be doe with Python ?
Victor Gallagher yes you can usw pygpu or my recommendation numba
Cool! Thanks for replying, I only starting learning Python 3 months ago and just discovered numba this week. Regards
yes mathema.tician.de/software/pycuda/
You can also skip the middle man and create directly a python extension which uses CUDA.
In my submission you can find a very minimal implementation of it:
github.com/tterava/Mandelbrot
It has all the necessary functions to allow your your C (or CUDA) code to be called from python code directly. All you need to do is to use the nvcc compiler to build the extension and python knows how to use it automatically.
as stated you can use numba. just decorate your function with "vectorize"
Seems like CUDA tries to make it as easy as possible to access the memory from both GPU and CPU, but what's the tradeoff?
Can be very wearing on a GPU reducing its lifespan
Source?
There are some performance trade-offs when using Unified Virtual Memory (UVM). Memory pages are migrated between the host and the GPU on-demand (plus some nifty prefetching heuristics) but frequent page faults often stall the system. I suggest using explicit memcpy semantics between the host and the GPU.
Thank you Daniel, that was the kind of trade-off explanation I was looking for.
Kevin, I'd like to read more about that. New GPUs seem to be _meant_ for that kind of stuff, don't they? I'd be surprised if it would really kill your GPU.
The benchmark for using 256 threads completed 171 times faster. Is that typical of parallelized operations in cuda?
Could you help me, I need the best GPU for science purposes not more expensive than 2500$ (it should work good with double): could you tell me on what characteristics should I look and what is the best variant?
2:22-wasnt 3Dfx around before nvidia?
Since you are adding the same numbers in the array, won't path prediction affect the speed of addition?
Hip can now be used for cuda. Break the monopoly.
I spend several hours yesterday trying to install tensorflow-GPU in Anaconda so I could use my GPU to speed up training of our neural network. It's an incredible pain in the ass as there are 7565 dependencies that all need to be installed in specific way and you're forced to register for the Nvidia Developer Spam Program. I didn't manage to do it in the end.
Anyone have any tip for painless installing of tensorflow-GPU? I was really suprised and annoyed by how difficult it was.
Make a video regarding "Reinforcement Learning". Thanks in advance.
Looks cool. Wish I understood.
Omg u r an angel thank you so much 😭
np
will i be able to use CUDA if i use an external nvidia gpu with my macbookpro? is this recommended?
Andrew what gpu
Mac doesn't support CUDA I think?, Maybe if you use windows 10 (Or even better Ununtu) on the Macbook through bootcamp with a 3.0 minimum level GPU card through thunderbolt might work
macOS supports cuda.
metal 2 shading language from Apple is Best for Apple hardware.
I havent bought the external gpu yet, but looking at the 1080. just wanted to know if its feasible before getting it
hey Siraj, do you write your own scripts? who funds all this?
At the end how is kernel parsing entire array if 'Stride' variable will be size of block. Some elements of array will be skipped?
the real stuff here is how are you using GPU in a mac? can you make a tutorial on how to make it work? as many know nvidia and apple are at war.
There are external GPUs that are compatible with MacBooks. Favorite of mine, the Razer Core X. Almost every eGPU is compatible with Nvidia.
@@existentialchild698 yeah know, but is a shame having to buy one when you already have one in your sistem. anyways i just make it work. compiled tensorflow with it and now i have enable gpu in my mid 2012 macbook pro
@@MygenteTV Okay. There's also a service made by Google called "Google Colab" that allows you to run your python code on an Nvidia Tesla K80. (That's a really good GPU that would otherwise cost you over $2000 depending on where you buy it).
@@existentialchild698 i see thank you.
@@MygenteTV no problem, thankful to help! 😊
When ROCm (AMD's new Open Source Deep Learning libs) is stable for Tensorflow, please do a video on it as well. Here is their github: rocm.github.io/dl.html
Caffe is finished, while Tensorflow and Caffe2 are currently in development with working builds already. :)
Patrick B Oh interesting, which amd Card corresponds to gtx1080? And has this rocm included?
thanks for the link will look into it
For those who have not tried it out yet and have an AMD chip don't bother...it gives segmentation fault and doesn't run at all
and a/c to otoy company they claim that they can do it it's not released yet so sadly go back and do it for open cl
and Thanks Siraj, you have great videos.
EDIT: Don't even install that nvidia-cuda-toolkit that it will recommend you if you have amd gpu... it will ruin your graphic drivers and you will be unable to boot into the system
oh and nvidia didn't release the first graphics processing unit
You know you have seen it all when you find a random ass man in a youtube video kiss kim jong un.
I thought you were going to write a NCHW convolution function.
Nice intro. But I wish those 4 min of fillers were more explanations about the complicated stuff presented really really briefly.