I have been following your videos for a while. Your explanations on different image processing techniques and deep learning approaches are great. Thank you so much for the videos. I have request. Can you please make a video on patch based CNN for classification? Keep up the excellent work.
Great advice, thanks! But what if not all objects are annotated within a patch, but you have an extent of annotated part? Is it possible to crop / turn into "no data" the rest of the patch?
Excellent video! As Vasily asked in a previous comment, how do we handle cases where the patch boundary splits a mask such that the annotated object and mask exists across 2 (or more) adjacent patches? Should we choose a different patch size? Should we rerun patchify but with overlapping patches? What's the best practice? Thanks Sreeni
Thank you sir ,I just discovered your tutorials, they are brilliant. I am working with whole slide images segmentation, I do patch sampling and end up with three types of patches: completely normal patches, completely tumor patches and patches with a combination of both. My question is doesn't showing the model with completely white or black masks will trick the model and cause a loss in performance? due to the large size of WSI, there will be large areas (larger than the patch size) with all tumor regions?
Hi Sreeni, I have images and test sites polygons. do I need to convert the polygons to different formate before creating the subimages or patches for training with U-Net? Thank you
hi, thanks for your wonderful trainings , I ussed this method for a rgb image , and my patch dimention is (1 , 256 , 256 , 3) I dont know how should I save it as image . can you help please ?
Hi Sreeni, hope you are having a good dayCan you tell how to automatically mask all these objects in a large image. I am trying to replicate your code for random forest for anemia detection but I want a mask image with all RBC and sickle cells?
Sir, could you please make a video about SGAN to improve the high resolution of images, it is really important to improve result of segmentation, while we apply these images for SGAN then apply U-net
Hi sir How about Binarized Statistical Image features? BSIF work based on learnt from a training set of natural image patches by maximizing the statistical independence of the filter responses. I hope you can make video about it. Thankyou
This is not augmentation as you are not transforming data to create new data. This is also not increasing the dataset size as you are working with the same data. This is just data pre-processing where you are cutting down the size of each numpy array to ensure they fit your memory. Please remember that for each batch of images you are extracting many filtered images and store in memory. This means you cannot just work with a bunch of 1kx1k images. Example calculation for a batch size 16 images with float32 values. 128x128x3 image - 3MB Same image with 64 filers: 64MB with 128 filters: 128MB In reality, you have many such filters in a deep learning network. Now, let us look at a batch (16) of 1024x1024x3 images - 192MB Same image with 64 filers: 4GB with 128 filters: 8GB How about 1024x1024x3 image with a batch size of 1: 1 image - 12MB 64 filters: 256MB 128 filters: 0.5GB We do not have a large amount of memory to handle these large images during training. Therefore, you need to crop them into smaller sizes that we can handle. PS: I used this image size calculator. svi.nl/ImageSizeCalculator
How to read large image (pixel is high and image size is 500mb), cv2 is giving error validate input image size. I have to do same work you done in this video but i am getting error in first step. I am getting error when i ma reading the image in opencv. Please help me
Hello Sreeni, I was trying to patchify my images (shape 128,128,3) [my original image is 1024, 1024, 3] and then after doing some operation, I tried unpatchify the image. But when unpathify the image, the boarder of the patched images are visible in the unpatched image. Could you please suggest how can I get rid of this artefact?
My images cover the polygons of marginal lands and non-marginal lands. What I want to do is to use U-Net to classify and identify marginal lands from non-marginal lands. Thank you
Thank you, Sir, for this video, Sir I have a question can you please help me in this regard. I don't have any masks for my dataset. How do I generate masks for my problem?
Hi Sreeni, thanks for this video. But what if we have coloured image having 3 channels. Then its giving me error except if i give patches = patchify(np.array(img2), (256, 256,1), step=256) and then the shape becomes (16, 32, 1, 256, 256, 3) , how to handle it. Thanks
They are just like RGB images except in RGB images you have 3 channels (bands) and in multispectral images you have multiple. So instead of using (256 x 256) you will be using (256 x 256 x n) where n is the number of bands.
Not sure what you mean by having different image objects in 3rd dimension. If the dataset represents a 3D volume you can define a 3D patch to extract sub volumes. If the dataset represents a 2D stack where you have a stack of different 2D images you need to extract 2D patches. Same rule applies for multispectral images.
Hi, Thank you for your video. I have a question: if i open the image using the PIL library, patchify give me an error. it seems it works with the image.shape which is not defined in PIL library. what should I do if I want to open an image with PIL rather than cv2. and I want to know how can we use these patches in pytorch.
When you use PIL to open an image, it opens it as an object. You need to convert it to a numpy array. from PIL import Image import numpy as np # Open the image using PIL image = Image.open("image_file.jpg") # Convert the image to a NumPy array image_array = np.array(image)
For multichannel or multidimensional images, your patch size needs to be defined accordingly. For color images, patch size would be something like 64x64x3, instead of just 64x64 that we use to define greyscale images. Similarly, for 3D images, patch size can be 64x64x64. Same rules apply when you are unpatchifying.
Sir i am new to image processing.. I am following your videos, I want to do image segmentation on lungs CXR so I download some image from kaggle but I doesn't contain mask image, so can sir help me on how to extract mask for the images I get.. I really need help sir
Mask is the ground truth and for that you need expert annotation. If you are an expert in the field you can draw your own annotations using software that is designed for deep learning annotations. I use www.apeer.com because this is what my team develops. If you are not an expert in the field, you need to find datasets with annotation.
I am not sure what you mean by the resolution getting affected drastically. Patchify breaks your large numpy array into smaller patches. The resolution (pixel values) should not change at all unless you are doing some binning operation. The quality of patches may get affected depending on the format into which you are saving your images. For example, you will lose quality if you save them as JPG. I recommend saving as PNG or tiff.
Why not? The method from this video does not care about your application, it just takes your images (numpy arrays) and divides them into patches. If your image is 1024x1024 and you want to divide it into patches of 256x256, this process helps. But if your image is 1024x1024 and you are trying to divide into patches of 2048x2048, it does not make sense as you are defining a patch larger than your image itself. This is the error you are getting.
Yifei Pei don't we have to generate segmented masks for training Unet ? I have the images in my hand. And I know the location of ROI I need to obtain a segmented mask for . So what should I do?
@@abdulrahimshihabuddin1119 No, U-Net is just a CNN architecture. If your project is not object detection, you don't need the masks. If your project is related to object detection, you have to use some softwares to label the images and save the masks.
I have a question. I want to detect pores in concrete. The image is taken, processed and presence of pores is detected. How do you go about that? Much appreciated. Thanks in advance. I'm really looking for a nice, fruitful reply.
![[TH] VCT Pacific Stage 2 - Playoffs Day 1 // GEN vs RRQ | TLN vs TS](http://i.ytimg.com/vi/JhkzKOhvOps/mqdefault.jpg)
I've recently stumbled upon your channel and it is absolutely excellent, thank you for all your great work!
Welcome!
Love the way you explain everything man, thank you!
I have been following your videos for a while. Your explanations on different image processing techniques and deep learning approaches are great. Thank you so much for the videos. I have request. Can you please make a video on patch based CNN for classification? Keep up the excellent work.
You should know how much of a blessing this is. Thank you very much.
what an easy way to do it! really struggled to get to this video. wish I found it sooner
Really helpful, thanks for sharing your knowledge....
that's very helpful, thank you. i didn't know about this patchify library.
Great advice, thanks! But what if not all objects are annotated within a patch, but you have an extent of annotated part? Is it possible to crop / turn into "no data" the rest of the patch?
Great always the best 💖
Just what I needed, thank you !
You are so welcome!
Thank you for sharing new one...
Excellent video! As Vasily asked in a previous comment, how do we handle cases where the patch boundary splits a mask such that the annotated object and mask exists across 2 (or more) adjacent patches?
Should we choose a different patch size?
Should we rerun patchify but with overlapping patches?
What's the best practice?
Thanks Sreeni
Thank you sir ,I just discovered your tutorials, they are brilliant.
I am working with whole slide images segmentation, I do patch sampling and end up with three types of patches: completely normal patches, completely tumor patches and patches with a combination of both. My question is doesn't showing the model with completely white or black masks will trick the model and cause a loss in performance? due to the large size of WSI, there will be large areas (larger than the patch size) with all tumor regions?
Thank you very much for nice and concise video
You are welcome!
Hi Sreeni, I have images and test sites polygons. do I need to convert the polygons to different formate before creating the subimages or patches for training with U-Net? Thank you
Very interesting!! could you show satellite image classification using python please
does patchify pads image if the image size is not a multiple of patch size?
hi, thanks for your wonderful trainings , I ussed this method for a rgb image , and my patch dimention is (1 , 256 , 256 , 3)
I dont know how should I save it as image .
can you help please ?
Thank you sir
Hi Sreeni, could you please share your tutorial on semantic segmentation using LITS dataset?
Hi Sreeni, hope you are having a good dayCan you tell how to automatically mask all these objects in a large image. I am trying to replicate your code for random forest for anemia detection but I want a mask image with all RBC and sickle cells?
Sir, could you please make a video about SGAN to improve the high resolution of images, it is really important to improve result of segmentation, while we apply these images for SGAN then apply U-net
Sir when are we going to work with audios, advance GANs and Auto-encoder models ?
please sir is it possible to join all the blocks together again?
Hi sir
How about Binarized Statistical Image features?
BSIF work based on learnt from a training set of natural image patches by maximizing the statistical independence of the filter responses.
I hope you can make video about it.
Thankyou
How do we deal with patchify when we have different numbers of bands for images(for eg, 10 bands image) and labels (for eg, 1 band mask file)?
Thanks 🌷, but I have question, is this considered as augmentation and increasing the dataset size?
Or just for make manipulation easy?
Thanks again 🌸
This is not augmentation as you are not transforming data to create new data. This is also not increasing the dataset size as you are working with the same data. This is just data pre-processing where you are cutting down the size of each numpy array to ensure they fit your memory. Please remember that for each batch of images you are extracting many filtered images and store in memory. This means you cannot just work with a bunch of 1kx1k images.
Example calculation for a batch size 16 images with float32 values.
128x128x3 image - 3MB
Same image with 64 filers: 64MB
with 128 filters: 128MB
In reality, you have many such filters in a deep learning network.
Now, let us look at a batch (16) of 1024x1024x3 images - 192MB
Same image with 64 filers: 4GB
with 128 filters: 8GB
How about 1024x1024x3 image with a batch size of 1:
1 image - 12MB
64 filters: 256MB
128 filters: 0.5GB
We do not have a large amount of memory to handle these large images during training. Therefore, you need to crop them into smaller sizes that we can handle.
PS: I used this image size calculator. svi.nl/ImageSizeCalculator
@@DigitalSreeni Awesome, very clear explanation… thanks.
No problem Ayad.
I have a whole slide image. I want to get small patches from the WSI. Can you please share some details on how to do it?
sir can we use for classification also
Is that possible to train an unet without generating its corresponding mask ?
how can I merge the patches after that
Sreeni do you accept research collaborations?
Does anyone knows how to deal with '.svs' file for the similar task done in this video?
hi sir opening a tif file called this error !!!
"""TiffPage 0: requires the 'imagecodecs' package"""
How to read large image (pixel is high and image size is 500mb), cv2 is giving error validate input image size. I have to do same work you done in this video but i am getting error in first step. I am getting error when i ma reading the image in opencv.
Please help me
Hello Sreeni, I was trying to patchify my images (shape 128,128,3) [my original image is 1024, 1024, 3] and then after doing some operation, I tried unpatchify the image. But when unpathify the image, the boarder of the patched images are visible in the unpatched image. Could you please suggest how can I get rid of this artefact?
My images cover the polygons of marginal lands and non-marginal lands. What I want to do is to use U-Net to classify and identify marginal lands from non-marginal lands. Thank you
Thank you, Sir, for this video, Sir I have a question can you please help me in this regard. I don't have any masks for my dataset. How do I generate masks for my problem?
Please check my videos on the topic of image annotation. I recently published them and I believe I named them, 'Labeling images using....'.
Hi Sreeni, thanks for this video. But what if we have coloured image having 3 channels. Then its giving me error except if i give patches = patchify(np.array(img2), (256, 256,1), step=256)
and then the shape becomes (16, 32, 1, 256, 256, 3) , how to handle it. Thanks
did u solve this ? i'm trying to use this patchify library to cut some RGB images
@@matancadeporco am too ,did u solve it or did u have a another code to do this
@@marwanabdou1644 did you find a solution to this?
Sir if you have multi spectral image with have more than 8 bands then what to do
They are just like RGB images except in RGB images you have 3 channels (bands) and in multispectral images you have multiple. So instead of using (256 x 256) you will be using (256 x 256 x n) where n is the number of bands.
Great.. But sir what about third dimension.. If we have different image slices for a 3d object. Then what would be the method..
Not sure what you mean by having different image objects in 3rd dimension. If the dataset represents a 3D volume you can define a 3D patch to extract sub volumes. If the dataset represents a 2D stack where you have a stack of different 2D images you need to extract 2D patches. Same rule applies for multispectral images.
I waiting for a long time thanks to you sir. small request I not able to find the code file sir.
When I try to execute this code it shows the error "Window-Shape is too large " . How to deal with this error??
when i try to read and RGB image, i get this error, i'm trying to divide RGB images into patches
Hi, Thank you for your video. I have a question: if i open the image using the PIL library, patchify give me an error. it seems it works with the image.shape which is not defined in PIL library. what should I do if I want to open an image with PIL rather than cv2.
and I want to know how can we use these patches in pytorch.
When you use PIL to open an image, it opens it as an object. You need to convert it to a numpy array.
from PIL import Image
import numpy as np
# Open the image using PIL
image = Image.open("image_file.jpg")
# Convert the image to a NumPy array
image_array = np.array(image)
When I run this code to my landsat data with 6 bands the results showed an 'window_shape' is too large.. sir, can you please tell me a solution
sir how to patch using opencv
Hi, how should I use patchify for colored images 256*256*3 images needs to merged to for larger image.
For multichannel or multidimensional images, your patch size needs to be defined accordingly. For color images, patch size would be something like 64x64x3, instead of just 64x64 that we use to define greyscale images. Similarly, for 3D images, patch size can be 64x64x64. Same rules apply when you are unpatchifying.
@@DigitalSreeni thanks for the response shreeni, I am currently working on this in my company on orthophotos.I am planning to use this.
Hi Professor, the coding returns me error ValueError: `window_shape` is too large
Which value to be modified?
Hope to hear back, all the best
It says window shape is too large, please make sure it is not larger than your image itself.
when i try to divide RGB patches, i get this error... did u solve it?
You need to check the image size. If the image size is less than window shape, it will throw this error.
@@yifeipei3049 how decrease window shape
How would you do this with RGB images? Thanks
No difference, same process, except your patch size would be 256x256x3 instead of 256x256
Sir i am new to image processing.. I am following your videos, I want to do image segmentation on lungs CXR so I download some image from kaggle but I doesn't contain mask image, so can sir help me on how to extract mask for the images I get.. I really need help sir
Mask is the ground truth and for that you need expert annotation. If you are an expert in the field you can draw your own annotations using software that is designed for deep learning annotations. I use www.apeer.com because this is what my team develops. If you are not an expert in the field, you need to find datasets with annotation.
@@DigitalSreeni okay.. Thank you sir for the suggestion.. But will it be possible to mask using code
While extracting image patches from RGB images, the resolution is affected drastically. What do you think can be done? Kindly guide
I am not sure what you mean by the resolution getting affected drastically. Patchify breaks your large numpy array into smaller patches. The resolution (pixel values) should not change at all unless you are doing some binning operation. The quality of patches may get affected depending on the format into which you are saving your images. For example, you will lose quality if you save them as JPG. I recommend saving as PNG or tiff.
@@DigitalSreeni Ok, Sir. Will try to change the format and check again. Thank you so much.
I am getting an error on "patches_image" stating: "ValueError: `window_shape` is too large". By any chance can you help me fix this Error?
Sounds like you are trying to divide your image into a patch size that is larger than your image itself.
@@DigitalSreeni I was trying to Prepare Patch of my Satellite Imagery. Seems like this code would not be feasible for Satellite Imagery?
Why not? The method from this video does not care about your application, it just takes your images (numpy arrays) and divides them into patches. If your image is 1024x1024 and you want to divide it into patches of 256x256, this process helps. But if your image is 1024x1024 and you are trying to divide into patches of 2048x2048, it does not make sense as you are defining a patch larger than your image itself. This is the error you are getting.
hi sir, how can i do this for RGB images?
Just add another dimension for the patchify box, e.g., (256, 256, 3).
@@yifeipei3049 I don't have any masks for my dataset. How do I generate masks for my problem?
@@abdulrahimshihabuddin1119 What do you mean "mask"?
Yifei Pei don't we have to generate segmented masks for training Unet ? I have the images in my hand. And I know the location of ROI I need to obtain a segmented mask for . So what should I do?
@@abdulrahimshihabuddin1119 No, U-Net is just a CNN architecture. If your project is not object detection, you don't need the masks. If your project is related to object detection, you have to use some softwares to label the images and save the masks.
Thank you sir, can you please share the code?
All my code can be found on my GitHub account. Please look at the video description for the link.
This doesn't work for RGB images.
Yes, it does. I have used it in many of my tutorials, for example this one: th-cam.com/video/jvZm8REF2KY/w-d-xo.html
I have a question.
I want to detect pores in concrete. The image is taken, processed and presence of pores is detected.
How do you go about that?
Much appreciated.
Thanks in advance.
I'm really looking for a nice, fruitful reply.
thank you sir