#Cloud4DevOps AWS Storage options - - Difference between AWS storage - Use-cases and differences between S3, EBS & EFS - Where to use and where not to?
AWS EFS is one of the contender for Big data Analytics as its durable , decoupled, and secure file system accessible from all nodes in the cluster. So as the functionality on the storage side is to grow from TBs to PBs without any downtime makes its better to be used for big data platform. And the big data concepts itself describe by below:- Volume [ranges from TBs to PBs] Variety [wide range from logs, media interaction and transactions] Velocity [analysis within short window] Since EFS is capable of providing support to above from AWS side , would be one of the better option here to select around AWS storage. Also not being a Big Data expert i would like you to go through other considerable option using Data Lakes where we have S3. aws.amazon.com/big-data/datalakes-and-analytics/
Hi Shashank, consider: if a collaborator gives access to their data on S3 (they make up a user name, access key and access ID), but I have my own S3 account where I want the data to go, how can I either 1) transfer that data directly to my S3, or 2) provide a way for them to upload the data directly to my bucket? The goal is to not download and upload the data due to size. Thanks!
If you are looking to upload data to S3 from EC2 then with the help of IAM role you access S3 and schedule job to upload data , in case you dont want to use access and secret keys. another way is you can use S3 browsers or tnt drive to map the S3 with your instance and schedule the upload.. One more way of doing is. use Storage Gateway with S3 as nfs mount point to schedule data upload. in AWS regardless of who is owner , accessing one service from another you need IAM permission. Hope that helps..
Thanks Jack for updating me.. Yes I did a mistake on last slide of my presentation. I explained the concept correctly , made a mistake on last slide on difference between AWS storage.. And yes EFS is file system. Highly appreciate the focus..
![[LIVE] : ONE ลุมพินี 69 | คู่เอก "กุหลาบดำ vs นาบิล"](http://i.ytimg.com/vi/PNM6U8e_HMo/mqdefault.jpg)
![[TH] VCT Pacific Stage 2 - Week 4 Day 1 // PRX vs RRQ | GE vs GEN](http://i.ytimg.com/vi/iIi2_04jGr0/mqdefault.jpg)
Thank you. Very clear. Can you pls elaborate how EFS can be used for big data
AWS EFS is one of the contender for Big data Analytics as its durable , decoupled, and secure file system accessible from all nodes in the cluster. So as the functionality on the storage side is to grow from TBs to PBs without any downtime makes its better to be used for big data platform.
And the big data concepts itself describe by below:-
Volume [ranges from TBs to PBs]
Variety [wide range from logs, media interaction and transactions]
Velocity [analysis within short window]
Since EFS is capable of providing support to above from AWS side , would be one of the better option here to select around AWS storage.
Also not being a Big Data expert i would like you to go through other considerable option using Data Lakes where we have S3.
aws.amazon.com/big-data/datalakes-and-analytics/
Good job man!
thanks..please share the channel among circle so to get more technical minds..
Thanks for sharing ..
really superb.....
Hi Shashank, consider: if a collaborator gives access to their data on S3 (they make up a user name, access key and access ID), but I have my own S3 account where I want the data to go, how can I either 1) transfer that data directly to my S3, or 2) provide a way for them to upload the data directly to my bucket? The goal is to not download and upload the data due to size. Thanks!
If you are looking to upload data to S3 from EC2 then with the help of IAM role you access S3 and schedule job to upload data , in case you dont want to use access and secret keys. another way is you can use S3 browsers or tnt drive to map the S3 with your instance and schedule the upload.. One more way of doing is. use Storage Gateway with S3 as nfs mount point to schedule data upload.
in AWS regardless of who is owner , accessing one service from another you need IAM permission. Hope that helps..
thank so much!
Glad it helped!
Thanks for sharing! I think you may have made an error on EFS. It is a file-level storage, not object storage. Please confirm?
Thanks Jack for updating me.. Yes I did a mistake on last slide of my presentation. I explained the concept correctly , made a mistake on last slide on difference between AWS storage.. And yes EFS is file system. Highly appreciate the focus..
Thanks for your reply! Best appreciated.
@@Cloud4DevOps maybe you should add a comment on the video itself.
Which storage is applicable for my eCommerce WordPress website?
If usage is less then go for EBS , if you have high usage then EFS is good option..
EFS is file storage
yes EFS is file system.. I made minute mistake and its been discussed in one of comment section..
@@Cloud4DevOps thank you