Thanks for that great job professor Wu. I was left with a question, how can I create my own sample dataset so that it can be used for classification in ee-python-api. I ask why in the Javacript code editor it is very simple, just create samples and merge it before submitting it to the classifier, however in ee-python-api I have not yet been able to reproduce the same thing.
You can use the drawing tools to create training samples and then use Map.user_rois to retrieve the drawn geometries and put them into a FeatureCollection.
Thanks for the answer @Qiusheng Wu . I did a test using this, but I found it a bit of a pain when compared to the experience in the JS code editor. I didn't find a practical way to change the classes' Property "label", and as the number of samples and classes increases things seem to get messy.
Thank you so much for the great videos. Can you do a video on linear regression or classifiers with regression output and also show how you can make a prediction after getting coefficients?
Hello, I have run into an issue in my code and was hoping you could help. In my project I have a labelled dataset imported as a feature collection, beacuse it had mutliple dates. Then i converted it to an image using the reducer.first() method. Now I want to extract the outside boundary geometry of this labelled image so that I can use it in defining the "region" in the points object like in 9:33 of this video. However I am getting an error saying "Provide 'geometry' parameter when aggregating over an unbounded image". Because of this I am seaprately tracing out the outline of the labelled dataset so that I can use its full scope for sampling the points. Any suggestions on how I can specify the "region" attribute directly with the labelled dataset geometry. p.s labelled_dataset.geometry() is also not helping
Dr. Qiusheng Wu. I have tired NLCD land cover in Asian region like Laos and Thai but it doesn't classify land cover classification. Can you suggest me where can I get this data to classify land cover specificly in Laos?
Thank you for the great video Dr. Wu! I am trying to upload a NAIP imagery from my local computer from jupyter notebook to perform supervised classification. I am using gee.Image(file location) but it is throwing me the error. Which function should we use for the upload?
The entire NAIP archive is already available in the GEE data catalog. There is no need to upload NAIP imagery. developers.google.com/earth-engine/datasets/catalog/USDA_NAIP_DOQQ You can upload other data to your GEE account if needed. developers.google.com/earth-engine/guides/image_upload
Hello Dr. Wu, I have noticed one issue in GEE regarding accuracy while preparing for LULC map. If we shuffle the order of bands ,for eg. var bands=['SR_B1', 'SR_B2', 'SR_B3' ] to var bands=['SR_B3', 'SR_B2', 'SR_B1' ] , accuracy gets changing around +/- 5% . What is the reason behind it. I don't think this line of code should be affecting overall accuracy but it has been encountered. What might be the possible solution to it such that for every order of bands we get same accuracy.
Dr Wu, really great work! appreciated! I wonder how many training samples can we use at most? I am asking it because i met the timing out problem when using ＞20000 training points. I am using random forest and want to include all the training samples
@@giswqs yes. but I am using RF to explain the response with some explaining variables, not really make predictions or classification. I'd like to use all of them (~25000). Are there any other ways than using part of them?
Hello.. 20,000 points ?? What's your ROI size ?.. I'm trying with RF classification method (supervised) but ain't getting the results with it.. I'm using ~300 training pts. should I increase ?
Dear professor, thank you so much for the great video! I have a question that can we choose the training data from different images? In other words, is it possible for us to get many training samples from different images, and using this training set to classify an image collection?
Yes, you can. Use image.sample() to generate as many training points as you need from any images and then combine the points as one single FeatureCollection.
It's very helpful! How to import local file of land cover? Now I have a shapefile and want to use 1 of the columns as labels (land cover), but when I import it, there are only some polygons but no labels and colors
Dr. Wu, How can I include NDVI as one of the parameters/bands for classification as you have done in this section # Use these bands for prediction. bands = ['B1', 'B2', 'B3', 'B4', 'B5', 'B7'] Can I input my NDVI calculation in this bands list as one of the elements. The output of my NDVI calculation is image type. Regards
@@giswqs I am not able to do it I tried it as below # Use these bands for prediction. bands = ['B1', 'B2', 'B3', 'B4', 'B5', 'B7', ndvi] I got error message since as such Image.select: Unrecognized type for band selector. Only strings and numbers are allowed.
I tried using the training sample tool on Geemap but keep getting an error about "NameError: name 'draw_control' is not defined" any idea what I am doing wrong?
Hi Mr. Wu, could you provide some material on how to work with the .tif file of the image classification once we have it in QGIS? I'm to that point and want to publish a pretty map but I'm having some problems
@@ericklarsen6608 Then you might want to look for some QGIS tutorials. Once the dataset is exported from GEE, it no longer has anything to do with GEE. Making maps lies within QGIS.
Dr thank you for the very informative video which helps a lot to those who are new to programming. I want to ask that if the Landsat Images having less percentage of cloud cover is not available in the region, what process we can do prior to the supervised or unsupervised classification to counter the cloud cover, So that the clouds are not shown in the Land-use classification file
Hi, My interactive map doesn't display unless I use "import folium import geemap.foliumap as geemap" Then step 2.3 isn't working. The satellite imagery isn't displaying. What could I be doing wrong? Kindly help out
Thank you for nice video professor, I have a question about clipping NLCD image. I will follow your tutorial with another image provided by Korean government. I found that NLCD image data has ['cpg', 'dbf', 'prj', 'sbn', 'sbx', 'shp', 'shx'] data format. However my image data has only ['dbf', 'prj', 'shp', 'shx']. Then, can't I follow the above function to create training data? Should I make all of those format file in Arcmap or QGIS manually before training? (If there is no possible way, I'm also thinking of method - train with NLCD and test in Korea) Thanks :)
Is there a reason why we don't have to remap() the landcover you are training on? This is non continuous for this case, so it was my thought that we would have to map this out into new integer values.
such as good tutorial how can we use LSTM model for supersised land cover classification and how many samples do I used for classification with 23200 km2 study area thank so much
Hello Professor, I am having a problem while downloading the code, it is saving as 'txt' file. I would like to request you to give instructions what to do.
Hello Dr. Wu, Though I have been able to perform the classification analysis, I have a problem. I am using a landcover map (raster) of my study area. However, when I upload this raster, GEE cannot read the information of its attribute table. When I check the raster's information, the Properties section (GEE) is empty. Do you know why GEE cannot read the complete information of the raster (attribute table)? I know that GEE has the option of editing properties. But, in this case, need to add values to the same property, but I cannot do that in GEE. I would appreciate it if you or someone here in the Comment Section could help me.
GEE can't read metadata when importing raster datasets. You will have to manually enter the properties. A raster does not really have an attribute table. It is simple composed simply pixel values. Not all software packages can read raster attribute table.
Dear professor I have a question. Wich technique is better for large area classification for example for the whole country. Supervised or un supervised. If supervised then witch algorithm of supervised is better for a large area. Looking forward.
I have a problem when importing the shapefiles to be my training data. I want to set the properties for separated collections from 0 to 6 as I have 7 classes. Here is one of the class: var tame = ee.FeatureCollection('users/thuydoan30121994/FinalSurvey_3103_tame_value1'); tame = tame.set('landcover', 1); However, after printing the collection, I found no property named landcover in the properties of all the points. Could you please help me to figure this out? Thanks a lot.
I got this error: "EEException: Property 'landcover' of feature '0_0' is missing." I´ve searched StackOverflow for a way to solve this problem. And come up with this link 'stackoverflow.com/questions/44066749/supervised-classification-with-the-google-earth-engine'. But the forum answer was not clear to me. How can i solve this error?
The error is in this part of the notebook. ' # Classify the image with the same bands used for training. result = image.select(bands).classify(trained) # # Display the clusters with random colors. Map.addLayer(result.randomVisualizer(), {}, 'classfied') Map '
Hi, I like your work and appreciate your ability to share. I would like to know if you have an example of image classification or image collection (with loop or map) using points or polygons imported from google earth engine? For example, points or polygons are created for me and saved in my gee directory. It's possible? Thanks.
Hi professor, thanks very much for your video, it really helps me a lot!! I have one question that how can I update my own polygon areas into Python if I want to use my own research area for supervised classification? Thanks in advance!
When I am running geemap.Map() it is giving me this error: " Authentication() got an unexpected keyword argument 'auth_mode' " How should I fix this? Thank you!
I was able to fix this by ee.Authenticate(). But now it is asking for an authentication code. and the link provided says The version of the app you're using doesn't include the latest security features to keep you protected. Please make sure to download from a trusted source and update to the latest, most secure version.
It seems most of your packages are very outdated. I highly recommend you create a new conda env to install geemap. For this specific error, you need to update the ee package using “pip install -U earthengine-api”
@@giswqs I generated an authentication token which was successfully accepted. But now I am getting this error: RefreshError: ('invalid_scope: Bad Request', {'error': 'invalid_scope', 'error_description': 'Bad Request'}) Also, every time time I run it, I am asked to enter an authentication code. Is there a way around it? (Also I am running everything on a brand new laptop which I got 2 days ago and had everything freshly set up. But for some reason, when you asked me to update, it works. Any thought why that might be the case?)
GREAT VIDEO BY ALL STANDARD. I HAVE A PERCULIAR PROBLEM AT HAND THAT I WANT TO HAVE. I HAVE TWO RASTER FILES TO PROCESS. I WANTO TO USE ONE OF THE RASTERS AS AN INDEPENDENT CATEGORY AND THE OTHER RASTER AS AN EVIDENCE CATEGORY. FOR EACH PIXEL VALUE IN RASTER 1, I WANT TO OBTAIN ITS CORRESPONDING PIXEL VALUE IN RASTER 2 AND DO SAME ACROSS THE ENTIRE RASTER. AT THE END OF THE DAY, MY INDEPENDENT FACTORS (PIXEL CLASS) WILL COME FROM RASTER 1 AND THEIR CORRESPONDING PIXEL CLASS WILL ALSO COME FROM THE EVIDENCE MAP (RASTER 2) SO I CAN USE THEM IN TRAINING THE MODEL. PLEASE DO YOU HAVE ANY IDEA ABOUT HOW THIS IS DONE USING PYTHON OR ANY OTHER SOFTWARE?.PLEASE KNIDLY RESPOND
Professor, this classification is based on the pixel and the result was patchy. classification process does not consider the spatial characteristics of the data. I wonder whether GEE can complete the convolution classification？look forward .
That's why it is called pixel-based classification. It will always have the salt and pepper effect. For object-based classification, see this video th-cam.com/video/2R0aTaMtYTY/w-d-xo.html
This a Four and Half STAR Tutorial, perfect and detailed Thanks Mr. Wu. I want toLearn how to work with SVG and how to save GeeMap Output to SVG(Vector) files in python environment, Can younplease help me Out Mr. Wu. Eg: Saving the Maps and your project from Colob into SVG so you can latter work on it.
I was looking for remote sensing satellite imagery segmentation and I found this.
thanks
You are the best !!! Thanks for such a great tutorial.
very detailed, great work!
great tutorial professor!
我来给 吴老师 支持
Thanks for your sharing. It is very detail. Best wish to you!
very good job, i'm waiting for next tutoriel
But how to take sample points at any other places if we want
Hello, Thanks for providing excellent tutorials. I am curious about that are you going to implement a deep learning model with geemap?
Very clear and awesome tutor. thank you
Thank you so much! Amazing videos!
Thank you so much for your contributions!. Could you do a video with the Tensor Flow repositories?
Yes, it is on my to-do list.
Thanks for that great job professor Wu. I was left with a question, how can I create my own sample dataset so that it can be used for classification in ee-python-api. I ask why in the Javacript code editor it is very simple, just create samples and merge it before submitting it to the classifier, however in ee-python-api I have not yet been able to reproduce the same thing.
You can use the drawing tools to create training samples and then use Map.user_rois to retrieve the drawn geometries and put them into a FeatureCollection.
Thanks for the answer @Qiusheng Wu . I did a test using this, but I found it a bit of a pain when compared to the experience in the JS code editor. I didn't find a practical way to change the classes' Property "label", and as the number of samples and classes increases things seem to get messy.
You can also save the samples you created in GEE Code Editor to your personal assets and load it with Python.
Thank you so much for the great videos. Can you do a video on linear regression or classifiers with regression output and also show how you can make a prediction after getting coefficients?
Thanks for the suggestion. I will make a regression video in the future.
Hello, I have run into an issue in my code and was hoping you could help. In my project I have a labelled dataset imported as a feature collection, beacuse it had mutliple dates. Then i converted it to an image using the reducer.first() method. Now I want to extract the outside boundary geometry of this labelled image so that I can use it in defining the "region" in the points object like in 9:33 of this video. However I am getting an error saying "Provide 'geometry' parameter when aggregating over an unbounded image". Because of this I am seaprately tracing out the outline of the labelled dataset so that I can use its full scope for sampling the points. Any suggestions on how I can specify the "region" attribute directly with the labelled dataset geometry.
p.s labelled_dataset.geometry() is also not helping
Dr. Qiusheng Wu. I have tired NLCD land cover in Asian region like Laos and Thai but it doesn't classify land cover classification. Can you suggest me where can I get this data to classify land cover specificly in Laos?
Great tutorial! 👍🏽
It's possible important the image?
How you divide these classes what ranges you have use for classification
Thank you for the great video Dr. Wu! I am trying to upload a NAIP imagery from my local computer from jupyter notebook to perform supervised classification. I am using gee.Image(file location) but it is throwing me the error. Which function should we use for the upload?
The entire NAIP archive is already available in the GEE data catalog. There is no need to upload NAIP imagery.
developers.google.com/earth-engine/datasets/catalog/USDA_NAIP_DOQQ
You can upload other data to your GEE account if needed.
developers.google.com/earth-engine/guides/image_upload
@@giswqs Thank you Dr. Wu!
Hello Dr. Wu, I have noticed one issue in GEE regarding accuracy while preparing for LULC map. If we shuffle the order of bands ,for eg. var bands=['SR_B1', 'SR_B2', 'SR_B3' ] to var bands=['SR_B3', 'SR_B2', 'SR_B1' ] , accuracy gets changing around +/- 5% . What is the reason behind it. I don't think this line of code should be affecting overall accuracy but it has been encountered. What might be the possible solution to it such that for every order of bands we get same accuracy.
Dr Wu, really great work! appreciated! I wonder how many training samples can we use at most? I am asking it because i met the timing out problem when using ＞20000 training points. I am using random forest and want to include all the training samples
do you have suggestions how to avoid the timing out problem while dealing with large amount of samples? thanks a lot
Try a smaller sample size. The performance might not improve much with a large sample size.
@@giswqs yes. but I am using RF to explain the response with some explaining variables, not really make predictions or classification. I'd like to use all of them (~25000). Are there any other ways than using part of them?
Hello.. 20,000 points ?? What's your ROI size ?.. I'm trying with RF classification method (supervised) but ain't getting the results with it.. I'm using ~300 training pts. should I increase ?
Great video, thank you so much!
Can you please tell me is it possible to do supervised classification using, for example, NDVI time series?
Yes, use image=collection.toBands()
How can I visualize the results of classes against number of pixels in the form of histogram?
Dear professor, thank you so much for the great video! I have a question that can we choose the training data from different images? In other words, is it possible for us to get many training samples from different images, and using this training set to classify an image collection?
Yes, you can. Use image.sample() to generate as many training points as you need from any images and then combine the points as one single FeatureCollection.
It's very helpful! How to import local file of land cover? Now I have a shapefile and want to use 1 of the columns as labels (land cover), but when I import it, there are only some polygons but no labels and colors
See th-cam.com/video/OlNlqfj4uHo/w-d-xo.html
Dr. Wu,
How can I include NDVI as one of the parameters/bands for classification as you have done in this section
# Use these bands for prediction.
bands = ['B1', 'B2', 'B3', 'B4', 'B5', 'B7']
Can I input my NDVI calculation in this bands list as one of the elements. The output of my NDVI calculation is image type.
Regards
yes, you can calculate NDVI and put the NDVI band name in the list
@@giswqs I am not able to do it I tried it as below
# Use these bands for prediction.
bands = ['B1', 'B2', 'B3', 'B4', 'B5', 'B7', ndvi]
I got error message since as such
Image.select: Unrecognized type for band selector. Only strings and numbers are allowed.
I tried using the training sample tool on Geemap but keep getting an error about "NameError: name 'draw_control' is not defined"
any idea what I am doing wrong?
Hi Mr. Wu, could you provide some material on how to work with the .tif file of the image classification once we have it in QGIS? I'm to that point and want to publish a pretty map but I'm having some problems
Not sure what you meant. Is the tif file exported from GEE? You want to make a pretty map based on the exported data from GEE?
@@giswqs yes, the idea is to make a pretty map in qgis with the classification data from gee
@@ericklarsen6608 Then you might want to look for some QGIS tutorials. Once the dataset is exported from GEE, it no longer has anything to do with GEE. Making maps lies within QGIS.
Many thanks for the suggestions. Much appreciated.
@james chisholm Thanks James, some good tips in there!
Dr thank you for the very informative video which helps a lot to those who are new to programming.
I want to ask that if the Landsat Images having less percentage of cloud cover is not available in the region, what process we can do prior to the supervised or unsupervised classification to counter the cloud cover, So that the clouds are not shown in the Land-use classification file
See th-cam.com/video/RsYJJG5HHgA/w-d-xo.html
Hi,
My interactive map doesn't display unless I use
"import folium
import geemap.foliumap as geemap"
Then step 2.3 isn't working. The satellite imagery isn't displaying.
What could I be doing wrong? Kindly help out
Thank you for nice video professor, I have a question about clipping NLCD image. I will follow your tutorial with another image provided by Korean government. I found that NLCD image data has ['cpg', 'dbf', 'prj', 'sbn', 'sbx', 'shp', 'shx'] data format. However my image data has only ['dbf', 'prj', 'shp', 'shx']. Then, can't I follow the above function to create training data? Should I make all of those format file in Arcmap or QGIS manually before training? (If there is no possible way, I'm also thinking of method - train with NLCD and test in Korea) Thanks :)
See th-cam.com/video/OlNlqfj4uHo/w-d-xo.html
@@giswqs Thank you so much!
Can you tell us, how to calculate the volume from raster image of successive
how can we add DEM(srtm) data along with bands for training the image for better classification in the hilly region?
developers.google.com/earth-engine/apidocs/ee-image-addbands
@@giswqs thanks sir it worked
Is there a reason why we don't have to remap() the landcover you are training on? This is non continuous for this case, so it was my thought that we would have to map this out into new integer values.
example: www.cartoscience.com/supervised-classification
such as good tutorial how can we use LSTM model for supersised land cover classification and how many samples do I used for classification with 23200 km2 study area thank so much
Super work
Hi, Is it possible to import the SciKit and use some of the classification method from the package?
Not possible at this moment, but there some works in progress. See github.com/giswqs/geemap/issues/124
Hello Professor,
I am having a problem while downloading the code, it is saving as 'txt' file. I would like to request you to give instructions what to do.
Hello Dr. Wu,
Though I have been able to perform the classification analysis, I have a problem. I am using a landcover map (raster) of my study area. However, when I upload this raster, GEE cannot read the information of its attribute table. When I check the raster's information, the Properties section (GEE) is empty. Do you know why GEE cannot read the complete information of the raster (attribute table)? I know that GEE has the option of editing properties. But, in this case, need to add values to the same property, but I cannot do that in GEE.
I would appreciate it if you or someone here in the Comment Section could help me.
GEE can't read metadata when importing raster datasets. You will have to manually enter the properties. A raster does not really have an attribute table. It is simple composed simply pixel values. Not all software packages can read raster attribute table.
@@giswqs Thanks for your prompt response!
Dear professor I have a question. Wich technique is better for large area classification for example for the whole country. Supervised or un supervised. If supervised then witch algorithm of supervised is better for a large area. Looking forward.
Please do not post the same question on every video!
I have a problem when importing the shapefiles to be my training data. I want to set the properties for separated collections from 0 to 6 as I have 7 classes. Here is one of the class:
var tame = ee.FeatureCollection('users/thuydoan30121994/FinalSurvey_3103_tame_value1');
tame = tame.set('landcover', 1);
However, after printing the collection, I found no property named landcover in the properties of all the points. Could you please help me to figure this out? Thanks a lot.
See developers.google.com/earth-engine/guides/feature_collection_mapping
Thank you so much for these great videos. Would like to know Is there any way to voerlay LULC and slope in Google Earth Engine?
I got this error:
"EEException: Property 'landcover' of feature '0_0' is missing."
I´ve searched StackOverflow for a way to solve this problem. And come up with this link 'stackoverflow.com/questions/44066749/supervised-classification-with-the-google-earth-engine'. But the forum answer was not clear to me.
How can i solve this error?
The error is in this part of the notebook.
'
# Classify the image with the same bands used for training.
result = image.select(bands).classify(trained)
# # Display the clusters with random colors.
Map.addLayer(result.randomVisualizer(), {}, 'classfied')
Map
'
Hi, I like your work and appreciate your ability to share. I would like to know if you have an example of image classification or image collection (with loop or map) using points or polygons imported from google earth engine? For example, points or polygons are created for me and saved in my gee directory. It's possible?
Thanks.
Yes, you can use your own FeatureCollection or load a shapefile geemap.org/notebooks/10_shapefiles
My study area is outside of USA so I can't use NLCD data for training. What should I do now?
Try ESA WorldCover
@@giswqs Thank You
l faced reducer error, is relate with version?
AttributeError: module 'ee' has no attribute 'Reducer'
Your VPN problem
Hi professor, thanks very much for your video, it really helps me a lot!!
I have one question that how can I update my own polygon areas into Python if I want to use my own research area for supervised classification? Thanks in advance!
See th-cam.com/video/OlNlqfj4uHo/w-d-xo.html
Please can it be applied to radar data (sentinel 1)if yes can you please send me the link .thanks
When I am running geemap.Map() it is giving me this error:
" Authentication() got an unexpected keyword argument 'auth_mode' "
How should I fix this?
Thank you!
I was able to fix this by ee.Authenticate(). But now it is asking for an authentication code. and the link provided says
The version of the app you're using doesn't include the latest security features to keep you protected. Please make sure to download from a trusted source and update to the latest, most secure version.
It seems most of your packages are very outdated. I highly recommend you create a new conda env to install geemap. For this specific error, you need to update the ee package using “pip install -U earthengine-api”
@@giswqs I generated an authentication token which was successfully accepted. But now I am getting this error:
RefreshError: ('invalid_scope: Bad Request', {'error': 'invalid_scope', 'error_description': 'Bad Request'})
Also, every time time I run it, I am asked to enter an authentication code. Is there a way around it?
(Also I am running everything on a brand new laptop which I got 2 days ago and had everything freshly set up. But for some reason, when you asked me to update, it works. Any thought why that might be the case?)
Make sure you check the two boxes on the authentication interface
Thanks Prof. very nice work.. can i know how to export GEE split panel map images..Thanks.
Exporting split-panel map is not supported.
GREAT VIDEO BY ALL STANDARD. I HAVE A PERCULIAR PROBLEM AT HAND THAT I WANT TO HAVE. I HAVE TWO RASTER FILES TO PROCESS. I WANTO TO USE ONE OF THE RASTERS AS AN INDEPENDENT CATEGORY AND THE OTHER RASTER AS AN EVIDENCE CATEGORY. FOR EACH PIXEL VALUE IN RASTER 1, I WANT TO OBTAIN ITS CORRESPONDING PIXEL VALUE IN RASTER 2 AND DO SAME ACROSS THE ENTIRE RASTER. AT THE END OF THE DAY, MY INDEPENDENT FACTORS (PIXEL CLASS) WILL COME FROM RASTER 1 AND THEIR CORRESPONDING PIXEL CLASS WILL ALSO COME FROM THE EVIDENCE MAP (RASTER 2) SO I CAN USE THEM IN TRAINING THE MODEL. PLEASE DO YOU HAVE ANY IDEA ABOUT HOW THIS IS DONE USING PYTHON OR ANY OTHER SOFTWARE?.PLEASE KNIDLY RESPOND
Are the exported images georeferenced?
Yes.
Professor, this classification is based on the pixel and the result was patchy. classification process does not consider the spatial characteristics of the data. I wonder whether GEE can complete the convolution classification？look forward .
That's why it is called pixel-based classification. It will always have the salt and pepper effect. For object-based classification, see this video th-cam.com/video/2R0aTaMtYTY/w-d-xo.html
@@giswqs thanks a lot
This a Four and Half STAR Tutorial, perfect and detailed Thanks Mr. Wu. I want toLearn how to work with SVG and how to save GeeMap Output to SVG(Vector) files in python environment, Can younplease help me Out Mr. Wu. Eg: Saving the Maps and your project from Colob into SVG so you can latter work on it.
Unfortunately, GEE does not support export SVG format. If you figure it out, let me know. I would be happy to implement it in geemap.