Hello Sir, I have gone through the entire snowflake playlist, your all videos are outstanding . But there is one important concept is missing in your snowflake playlist i.e. SCD type 1,2,3 data loading. If possible, then please make a video in this , it will help millon of learners. Thanks & Regards, Gunjan
Hi Janardhan, Thank you for this video. Got a good idea on the data loading automation. in my requirement the data comes every month in s3 in parquet format and saved under folder a folder e.g 20240101 and the entire folder structure looks like mainfolder/monthly/tablename/data/20240101/file.parquet. I have 2 questions 1. if i give the path of the main s3 location will it process all the files inside it? 2. every month i want to load only the new month data and not the old ones. Kindly help me with this query
1. if i give the path of the main s3 location will it process all the files inside it? No, you have to specify exact path. If you want to process files present in multiple folders, you have to use multiple copy commands. 2. every month i want to load only the new month data and not the old ones. Snowflake automatically consider the unprocessed files unless until you specify FORCE=TRUE option.
Thank you. I have more than 200 tables as folders and inside each table folder I have monthly data folder that hold the Parquet files. Does that mean I have to mention the 3xact path of the monthly folder for all the 200 folders also when a new month folder comes how can I dynamically mention that path in the sbowflake query.
That thing you have to handle from your storage account side, in general when new files are coming they should come as files not folders, you have to make that arrangement
Sir present how the calls for snowflake, because i absorbed there are no calls for snowflake in job portals, so is it worth to learn in present situation?? But I have interest... please reply me sir.
Sir can you please make video like loading the data from mysql or oracle or any other database to snowflake. Because most of companies following data migration so that it would be helpful if you make video on data migrating from oracle or mysql to snowflake
Hi Sir, I learned snowflake by following your channel and got a job, thank you very much for the channel!! I request you to teach us any ETL tool which fits with snowflake!!
Hi sir, will it be possible to modify table from ADF in snowflakes ....as my requirement is to copy data from adf to snowflakes, where the tables in adf will be modified with adding new columns.....will it be possible to modify tables from adf?....as snowflakes requires table schema created prior only...
In general we don't execute DDL commands from ADF or any other ETL tool, they are for data loading and unloadings. So I am not sure. If the user/role that you are using with ADF has table alter permissions on the target table why don't you try and check.
@@mrjana520 actually, I am able to copy data from adf to snowflakes .....but the issue iam getting is if any new coloumn is added in the source then...it should be reflect in snowflakes automatically...without altering the table in snowflakes....How to do so ....i am not getting that
Please get it from Udemy, it will cost you just 500 rs with below link, I will be uploading all future scripts/programs in Udemy only. www.udemy.com/course/snowflake-complete-course-for-clearing-interviews/?couponCode=0D096876AA7A5A91FBCD
Sir present how the calls for snowflake because i absorbed there are no calls in naukari ,so in present situation is it worth to learn ?? But I have interest to learn, please reply sir.
Hi, no one can pass snow pro core certification with one video. You have to study all concepts properly, practice some sample papers and dumps(you can get these questions in Google or from Udemy you can purchase)
a great video addressing a very common usecase. I am sure it will be a gold for many users like me who needed answer to this problem
Very helpful to know more about the store procedure. Excellent video and clear-cut explanation.
Glad it was helpful!
awsome generic solution for multiple loads
Your explanation in the videos is very good
Hi Janardan
Can you make a video for loading data from raw layer to il layer by creating sp and mention update logic
Very helpful, and explained clearly
Excellent
Super
That is generous sharing on YT
Thanks for this video sir
Hello Sir,
I have gone through the entire snowflake playlist, your all videos are outstanding .
But there is one important concept is missing in your snowflake playlist i.e. SCD type 1,2,3 data loading.
If possible, then please make a video in this , it will help millon of learners.
Thanks & Regards,
Gunjan
will do in near future
Amazing video sir really helpfull to us thnk u so much 👌👍❤️❤️
Thanks and welcome
Thanks you!
thank you bro.
Hi Jana, I have a doubt. Exactly this how we work in real time industry projects ?
This is just 1 example, the real-time implementation is completely dependent on the tools and tech stack used by that company and project
Hi sir can you explain same sp or any other sp using javascript. What are the changes we need to make.
I have already posted a video on that, please check my snowflake playlist
Hi Janardhan, Thank you for this video. Got a good idea on the data loading automation.
in my requirement the data comes every month in s3 in parquet format and saved under folder a folder e.g 20240101 and the entire folder structure looks like mainfolder/monthly/tablename/data/20240101/file.parquet.
I have 2 questions
1. if i give the path of the main s3 location will it process all the files inside it?
2. every month i want to load only the new month data and not the old ones.
Kindly help me with this query
1. if i give the path of the main s3 location will it process all the files inside it?
No, you have to specify exact path. If you want to process files present in multiple folders, you have to use multiple copy commands.
2. every month i want to load only the new month data and not the old ones.
Snowflake automatically consider the unprocessed files unless until you specify FORCE=TRUE option.
Thank you. I have more than 200 tables as folders and inside each table folder I have monthly data folder that hold the Parquet files. Does that mean I have to mention the 3xact path of the monthly folder for all the 200 folders also when a new month folder comes how can I dynamically mention that path in the sbowflake query.
That thing you have to handle from your storage account side, in general when new files are coming they should come as files not folders, you have to make that arrangement
Thank you for your valuable inputs
Sir present how the calls for snowflake, because i absorbed there are no calls for snowflake in job portals, so is it worth to learn in present situation?? But I have interest... please reply me sir.
If you think like that you can't learn any technology now, not only snowflake there are no calls for any technology due to current it market situation
Sir can you please make video like loading the data from mysql or oracle or any other database to snowflake. Because most of companies following data migration so that it would be helpful if you make video on data migrating from oracle or mysql to snowflake
Even I want to do, but I don't have required equipment or technology to do that migration..
@@mrjana520 try to do sir because it helps a lot to many people. Because no video is clear as your explanation there in youtube regarding in migration
Hi Sir,
I learned snowflake by following your channel and got a job, thank you very much for the channel!!
I request you to teach us any ETL tool which fits with snowflake!!
Hi sir, will it be possible to modify table from ADF in snowflakes ....as my requirement is to copy data from adf to snowflakes, where the tables in adf will be modified with adding new columns.....will it be possible to modify tables from adf?....as snowflakes requires table schema created prior only...
In general we don't execute DDL commands from ADF or any other ETL tool, they are for data loading and unloadings. So I am not sure. If the user/role that you are using with ADF has table alter permissions on the target table why don't you try and check.
@@mrjana520 actually, I am able to copy data from adf to snowflakes .....but the issue iam getting is if any new coloumn is added in the source then...it should be reflect in snowflakes automatically...without altering the table in snowflakes....How to do so ....i am not getting that
That is impossible
Bro primary key constraint works in snowflake
We can define, but they do not enforced, but we can use them as join keys like that
Great Janrdan..can we have the proc script.?
Please get it from Udemy, it will cost you just 500 rs with below link, I will be uploading all future scripts/programs in Udemy only.
www.udemy.com/course/snowflake-complete-course-for-clearing-interviews/?couponCode=0D096876AA7A5A91FBCD
Sir, Can you please attach the procedure which you have created
Get it from Udemy please, lengthy one, not able to put all code in description
Sir telugu lo snowflake videos cheyyandi sir please
sorry brother, don't have time. but my udemy course is in basic english, anyone can understand it easily
where can i get the above code??
You can get it from my Udemy course
is this working with snowflake?
Yes, watch the full video for execution
Sir, snowflake ki future vunda sir,
Coding ela sir,e tool next 10 years ala vuntada sir, coding level ela sir
reach me on jana.snowflake2@gmail.com
Sir present how the calls for snowflake because i absorbed there are no calls in naukari ,so in present situation is it worth to learn ?? But I have interest to learn, please reply sir.
That is bcoz of current market situation, less calls for any technology
Hi Sir,
Can you please make a video on Preparation to crack Snow Pro Certification.
Hi, no one can pass snow pro core certification with one video. You have to study all concepts properly, practice some sample papers and dumps(you can get these questions in Google or from Udemy you can purchase)
Excellent