วีดีโอ

Hi, i create a dynamic table and have task that validates data on tables (standard tables) that are used in the dynamic table. The thing is when i finish the validations i need, i put an ALTER DYNAMIC TABLE IF EXISTS xxx REFRESH; inside the task, and when the task is executed the refresh is not starting. If i execute manually the command it works but is like it doesn't when is inside the task, is this a limitation right now??

how do we do this in snowpark python

Failed to create table Failed to create table, caused by: SnowflakeSQLException: SQL execution error: Creating table on shared database 'COVID19' is not allowed. HTTP code: 500, type: com.dataiku.dip.exceptions.DataStoreIOException this is an eeror what can i do

is the ref function referecning the country codes seed csv file or is it referencing the country codes table? Since there are both, how do you know which one is being referenced?

Greetings Adam. I passed core with your Udemy course. Now I need to take an advance cert, do you think about course will help/what else would I need. I have limited time to complete it. Thanks in advance.

Clear and interactive intro

Do you have any voucher code to attempt for the snow pro certification?

Has anyone used their guides and successfully passed?

Hey Adam, Audio is coming on on channel only. Hard to listen to on headphones

Hi Adam, great presentation!

good video. Thankyou

Looks nice ! Is that using chatgpt?

what to do if in an environment, I am not able to see "models" folder only. How to create it

hi Adam, can we use Microsoft Authenticator for Snowflake MFA instead of the DUO mobile app?

Thanks a lot. I understood it well. Very well explained!

pls give the access to enroll

i want also

🎯 Key points for quick navigation: - Introduces Alex Lee, a Snowflake senior engineer [1]. - Mentions that Dynamic Tables are in private preview but Snowpipe Streaming is in public preview [1,2]. - Explains that Snowpipe is used for batch data ingestion [2]. - Discusses the limitations of Snowpipe including needing a staging area [4]. - Introduces Snowpipe Streaming which is designed for real-time data ingestion [5]. - Explains that Snowpipe Streaming removes the need for a staging area [7]. - Shows a demo of Snowpipe Streaming ingesting data from a Kafka topic [10]. - Demonstrates how Dynamic Tables are used to merge data from multiple sources [11]. - Discusses the benefit of Snowpipe Streaming and Dynamic Tables being able to handle Json data types [17]. - Mentions that a good use case for Snowpipe Streaming is when dealing with streaming data [18]. - Alex Lee offers to answer questions about Dynamic Tables and Snowpipe Streaming [19]. - Discusses Snowpipe's ability to ingest various data formats including structured, semi-structured (JSON, XML, Parquet), and unstructured data [2,3]. - Mentions that Snowpipe Streaming and Dynamic Tables will be used in the demo [3]. - Explains that Snowpipe Streaming and Dynamic Tables simplify data architecture by combining streaming and batch data pipelines into one platform [3]. - Highlights that Snowpipe Streaming enables serverless ingestion and automated transformations using joins [3,4]. - Discusses the benefit of expanding the data ecosystem within the Snowflake Data Cloud [4]. - Discusses the complexity and cost associated with managing orchestration and data frequency for traditional batch data pipelines [4]. - Highlights the benefit of using Snowflake's native platform and security features to simplify data management [4]. - Introduces Snowpipe, a file-based data ingestion method designed for batch data [5]. - Explains that Snowpipe Streaming is a new technology designed for ingesting real-time data streams (e.g., Kafka topics) [5]. - Clarifies that Snowpipe Streaming complements Snowpipe, and the choice depends on the data source and workload [5,6]. - Discusses the benefit of Snowpipe Streaming eliminating the need for a staging area for data ingestion [6]. - Explains that Snowpipe Streaming allows for loading data directly into the final table, streamlining the process [6,7]. - Highlights that Snowpipe Streaming reduces maintenance overhead and simplifies data architecture [7]. - Provides a brief overview of the upcoming demo [7]. - Explains that the demo will showcase merging two tables using Dynamic Tables and Snowpipe Streaming [7]. - Briefly describes the use case of an ad campaign where data will be streamed and merged [8]. - Provides an introduction to the upcoming demo [8]. - Explains that the demo will showcase merging data from a Kafka topic (ad campaign information) with a pre-created mapping table using Dynamic Tables [8,9]. - Briefly explains the purpose of the Dynamic Table and its configuration (set to update every minute) [10]. - Explains the concept of Dynamic Tables and their syntax [10]. - Highlights that Dynamic Tables automatically refresh data based on a specified lag time (minimum 1 minute) [10,11]. - Clarifies that Snowflake manages the compute warehouse resources for Dynamic Tables [11]. - Discusses how Dynamic Tables enable transforming data on the fly using native SQL [12]. - Draws an analogy between Dynamic Tables and DBT models, highlighting the use of SQL for defining the target table [12,13]. - Discusses the benefit of Dynamic Tables in simplifying data transformation workflows [13]. - Explains that Dynamic Tables allow writing transformational logic directly in the table definition, eliminating intermediate steps [13,14]. - Showcases a use case where a Dynamic Table is used to automatically ingest and transform JSON data from a Kafka topic [14,15]. - Demonstrates a Dynamic Table in action, merging data from two source tables [15]. - Highlights that the Dynamic Table behaves like a regular table and refreshes automatically based on the lag time [15,16]. - Showcases how the Dynamic Table handles JSON data transformation within the select statement [16,17]. Demonstrates Snowflake's data analysis capabilities [17:41]. - Highlights automatic data refresh based on lag time [17,41]. - Showcases features for data analysis including: ability to view data analysis without writing code [17,55], ability to create charts directly from the data [18,23], ability to monitor data lineage [18,37]. Made with HARPA AI

Hi Adam, How do you handle schema change in the source with the incremental loading? Thanks in advance. Your videos are awesome. Kr, Yahya

SSL done and Pls share more videos...Thanks you so much for sharing knowledge ...

This was very helpful!

Nice

HI Adam, So I am trying to create an SCD from a table that already contains changes say for example you have on row with the order status as shipped for order AAA and another for the same order where the status is delivered. When I use your method, on the second run, I get the error that I can only update one row per time, how can I create an SCD from this type of table using dbt.

How long did you prepare for the exam, Adam? 🤔

Hi what if I have to accounts in snowflake like Dev invironment and UAT invironment? I already registered my phone number in Dev and now I dont why there is an error registering my phone number in UAT

Nice video sir. Can you please make a video on unit testing with dbt core.

Can you try the document AI feature ?

Now I know to call all the three parallel lines in applications hamburger lines. THANKS! Also, good tutorial so far.

Can anyone provide the SQL queries, the link is not properly opening for me.

Good morning, Adam, I've been reading your materials and want to take your Snowflake Core Exam class. My LinkedIn is not working. Can you please share a new discount code? Also, how are the questions broken out? I saw three different sections. Thanks for all your knowledge, hard work, and sharing your expertise!

Great explanation! you won a suscriber, thanks for the content!

You have excellent Teaching attributes. Thanks for this.

The contents of this video is very clear to understand and very in-depth informative. Thanks for your efforts for this video.

Is a stored procedure the only way to execute multiple SQL statements?

helpful !!

Awesome, Thanks a lot! Is there any your course (DBT + Snowflake end to end) available in udemy? please let me know.👍

Can someone send me the link to the dbt course that is being referenced in this video?

🤭 'promo sm'

Could you please create a video on managing credentials in Matillion for the Azure connection

the main thing i came here for was to see how to pass in a parameter into a SP by calling it. IT would be great if you could show this.

Thanks for the video. How can Snowflake, in real time when the event happens, push logs out to an API? Other old school DBs have a function that exports DB logs, in real time when events happen to a SYSLOG format on the file server for SIEM /security analysis. Snowflake has no such thing ;-(

Does Snowflake utilise micro partitions in views? If I were to create a view that captured a year but wanted to take a slice of a day, each day, for example, would Snowflake create temporary micro partitions each time and have to scan the entire view each time?

How does dbt address data summarization? As you know you can summarize data in dyn tables for increased performance

if we are writing tests for incremental models which are already existing the model output is getting compared to old tables . not sure how to switch over schemas between them

That wrapper is generic your SP is dynamic based on the 2 character state ? Thanks man I’m an analyst not a developer, just trying to understand the process E2E

Hi Adam, very nice. Could you please explain what is the primary difference between Dynamic Tables vs Views vs Material Views?

I"m confused is DBT just a query builder? If I use raw SQL in my webapp instead of an ORM. Where would DBT fit in? Would DBT be used to switch from raw SQL queries when fetching my data variables e.g. sql_data_python_object = raw_sql_query(against existing DB). So instead of raw_sql_query I would now write a dbt_query? But the benefit is that this query works on more DBs than say a raw pg query. So is DBT a query builder which works like an ORM in the sense that you can use the query on any DB? If not I don't see the purpose for it when I could just use a raw SQL query on the datawarehouse after the pipeline has finished filling up the datawarehouse.

I got the requirement create event table for snowpipe logs , can you please make the video

Tnx for sharing

The value that you are providing is just ridiculous. Thanks for the straightforward explanation Adam