Amazon Athena and AWS Glue for S3 Data Query

===================================================================
1. SUBSCRIBE FOR MORE LEARNING :
/ @cloudquicklabs
===================================================================
2. CLOUD QUICK LABS - CHANNEL MEMBERSHIP FOR MORE BENEFITS :
/ @cloudquicklabs
===================================================================
3. BUY ME A COFFEE AS A TOKEN OF APPRECIATION :
www.buymeacoffee.com/cloudqui...
===================================================================
This video shows how to set up AWS glue crawlers to extract the raw data from AWS S3 bucket and put it across in AWS Athena service to query the raw data.
It has step-by-step explanation of all set up that has to be put in place to extract the raw data from AWS S3 bucket ( of format .csv) and put in Amazon Athena so that Business analytics can be performed there.
#awsglue #awsathena #query #s3 #aws #cloudquickpocs

[@thewisearchitect]

Great tutorial. Thanks.

[@cloudquicklabs]

Thank you for watching my videos.
Thank you so much for the encouragements. Keep watching and keep learning , Happy learning.

[@nivas143srinivas]

Excellent work sir , followed all the instructions but I'm getting 0 records when I query the table from Aetna even though there is data in my S3 bucket files. Any suggestions would be appreciated.

[@cloudquicklabs]

Thank you for watching my videos.
This could be happening because of Glue crawlers has imported the successful. Please validate the crawler configuration again.
And also don't forget to create new work group at Athens Query.

[@jettsalvador8688]

You may also try to check the policy configured in the iam role used by the glue crawler, you might need to add the resource arn of the S3 bucket that you want to be crawled.

[@ndipsamuel475]

@@jettsalvador8688 Thank you. i was blocked for two days and didn't know why. This solved my problem.

[@riyajain6713]

what if i applied a schedule to it and i wanted only the new data in athena to perform logics

[@cloudquicklabs]

Thank you for watching my videos.
In this case you need to configure from S3 bucket and AWS Glue side to extract only the required data by archiving not required data.

[@danieljust295]

The most important part is missing. This is how the crawler assigns the type to the columns and how to modify the type of the column when clawer finished its job.

[@cloudquicklabs]

Thank you for watching my videos.
Could you please elaborate the question here, is it 'how to modify the type of the column when crawler finishes its job?'

[@danieljust295]

@@cloudquicklabs Notice, that Glue reads the file with data and decides the type for each column (string, integer, float etc.) - this is also crawler’s job and this is why crawler can create schema. This step should be always verified because we don’t know if crawler mapped data types properly to each column from data file. What’s more - I guess - Glue’s crawler uses ML algorithm to determine the type of data found the column.

[@cloudquicklabs]

Thank you very much coming back on this. You have made very good observations, may be it should the same way that you guessed. AWS Glue crawlers should be Intelligent enough to indetify the data type while extracting data from source

[@user-rv5dz9ko1b]

crawler runs and stops but table is not created in my case

[@cloudquicklabs]

Thank you for watching my videos.
Did you if your IAM role has required permissions, and please check error message if you can.