this is such great content. I appreciate your efforts If it's not too much work, can you demo how to work with images in cuda -- this is the most common use of cuda, right?
I don't know if it's true that CUDA is primarily used to work with images (people like running ML workloads on lots of things other than images). I can definitely do some work with images though!
Around 8:30, you mention that "we can do the same padding trick as last time". Which previous video are you talking about? Is this video part of some series? Btw, thanks a lot for your content, they're really helpful.
The easiest way to do it (with a matrix width known at compile-time) is with a quick typedef. In general you should prefer the linearized way of indexing (in many cases you don't know the dimension of the matrix at compile-time, and the solution to that is rather messy, and incurs unnecessary performance penalties). I've re-written the example from this video in the link that follows to use 2d indexing. Hope this helps! github.com/CoffeeBeforeArch/from_scratch/blob/master/2d_array/matrix_mul.cu
Hey iam getting a assertion failed error if i take number of threads = 64 can anyone tell what maybe the reason for that and how to fix it?? Please help me!!
If you're using 64 for threads (and the code from the video), that is giving you a thread block w/ 4096 (64 * 64) threads, which greater than the maximum 1024 threads per thread block allowed by hardware.
I’m trying to figure out the logic behind cudamallocmanaged(&a, bytes) needing the ampersand sign when a is an array , surely it’s not wanting the address of the pointer ?
If you look at the api, the input to cudaMallocManaged is a void** devptr. The api call returns in *devptr a pointer to the allocated memory. By passing in the address of the pointer, the api call just overwrites the pointer with one pointing to the new memory.
I'm doing an experiment with this code (everything same except I initialized matrices A and B on host memory and then copied it to device) using cudaMemcpy. I performed verification on matrix sizes 2x2... 8x8... till 2048 x 2048. But when I changed int N = 1
You might just be running into a memory error (depending on your GPU) since each 4kx4k matrix is 64MB. CUDA runtime API calls may fail, but that doesn't necessarily crash your program. I'd check to see if you're kernel is even running on the device (or if indeed your other API calls are completing successfully).
@@CoffeeBeforeArch Thanks, we obtained execution time for GPU operation and CPU verification operation. They differ drastically as we climb up the matrix sizes, so we think the kernel is being executed on the GPU. We changed the BLOCKS = (N + threads -1)/threads to BLOCKS = 256 directly and it worked. We are not sure why!
This video helped me infinitely to finish my college assingment- GREAT CONTENT MAN!
Thanks! I'm glad you found it helpful!
this is such great content. I appreciate your efforts
If it's not too much work, can you demo how to work with images in cuda -- this is the most common use of cuda, right?
I don't know if it's true that CUDA is primarily used to work with images (people like running ML workloads on lots of things other than images). I can definitely do some work with images though!
Is it global memory version of matrix multiplication?
Thank you! Very clear tutorial
Glad you liked it! :^)
Around 8:30, you mention that "we can do the same padding trick as last time".
Which previous video are you talking about? Is this video part of some series?
Btw, thanks a lot for your content, they're really helpful.
I’m following the same code with you but ended with Segmentation fault (core dumped). Any solutions?
Hi - great video. Quick question... How would you Implement parallel blocked GEMM using CUDA?
Hello
how to do remove_postfix with different starting points in cuda
Is Linearizing the 2d array necessary...or we can use the original array?
You're more than welcome to is a 2d array (e.g., a[N][N]) if you wish. Whatever works for you!
@@CoffeeBeforeArch thanks for replying to my question.. Can you suggest me some resources where I can learn how to do it without changing the array?
The easiest way to do it (with a matrix width known at compile-time) is with a quick typedef. In general you should prefer the linearized way of indexing (in many cases you don't know the dimension of the matrix at compile-time, and the solution to that is rather messy, and incurs unnecessary performance penalties). I've re-written the example from this video in the link that follows to use 2d indexing. Hope this helps! github.com/CoffeeBeforeArch/from_scratch/blob/master/2d_array/matrix_mul.cu
Hey iam getting a assertion failed error if i take number of threads = 64 can anyone tell what maybe the reason for that and how to fix it??
Please help me!!
If you're using 64 for threads (and the code from the video), that is giving you a thread block w/ 4096 (64 * 64) threads, which greater than the maximum 1024 threads per thread block allowed by hardware.
Hi, can you please explain me why i have this error " no kernel image is available for execution on the device". Thank you!
It's likely you're using a CUDA version that is unsupported by your GPU
@@CoffeeBeforeArch Thanks for your reply, how can i know what version of CUDA that is supported by my GPU?
You didn't use memcpy in this video.. Is it not needed ?
No, not if you're using cudaMallocManaged (the CUDA runtime manages the transfer of memory back and forth for you)
I’m trying to figure out the logic behind cudamallocmanaged(&a, bytes) needing the ampersand sign when a is an array , surely it’s not wanting the address of the pointer ?
If you look at the api, the input to cudaMallocManaged is a void** devptr. The api call returns in *devptr a pointer to the allocated memory. By passing in the address of the pointer, the api call just overwrites the pointer with one pointing to the new memory.
CoffeeBeforeArch Thankyou for the detailed response. That makes sense now i.e. it is indeed the address of the pointer it needs
I'm doing an experiment with this code (everything same except I initialized matrices A and B on host memory and then copied it to device) using cudaMemcpy. I performed verification on matrix sizes 2x2... 8x8... till 2048 x 2048. But when I changed int N = 1
You might just be running into a memory error (depending on your GPU) since each 4kx4k matrix is 64MB. CUDA runtime API calls may fail, but that doesn't necessarily crash your program. I'd check to see if you're kernel is even running on the device (or if indeed your other API calls are completing successfully).
@@CoffeeBeforeArch Thanks, we obtained execution time for GPU operation and CPU verification operation. They differ drastically as we climb up the matrix sizes, so we think the kernel is being executed on the GPU. We changed the BLOCKS = (N + threads -1)/threads to BLOCKS = 256 directly and it worked. We are not sure why!
tysm
The audio is soooo low. Like you dont want someone in the next room to hear you.