Would love to see some deep-dive detail about using both serverless and dedicated sql pools to handle different layers within a data lake when building a data platform.
Great stuff . I am trying synapse and other Azure offerings .. These short videos are great .. Can you also do a video so we can understand which is the efficient way to do ETL and load to PBI ..
Usually great stuff but this video seemed like a pure marketing stuff from the channel. Of course dedicated gonna be much more powerful because of strict schema and way more compute available. But it’s also very expensive.
It will be nice if we have some cost and capability comparison on premise, serverless and dedicated sql. Is the principal question a manager or a customer asks when offering this kind of solution.
Ya this is definitely a good introduction to understand the difference between Dedicated vs serverless SQL pool. You should have added about Spark SQL pool also... what is that and how is that different?
What I found is that External Tables on serverless sql pool are like views. So you can't perform any DML operations in serverless sql pool which defeats the basic requirements of a DB
Note: with columns that take on values that are very large (e.g. json bits stored in col values), the simple syntax demo'd here won't work. This is because the inferred data type is varchar(8000). To bring back larger strings, you'll need to schematize the data in the query via the WITH construct, setting such cols to varchar(max) data type. Why varchar(max) isn't the default inference is beyond me.
Our issue with Serverless is its wildly inconsistent. 2m row table 30 seconds to load and then 5 mins later that same 2m row dataset now takes 7 mins. Any way we can address this?
Great Video, I would like to know if we can expose the serveless External Tables or View Via API for web application to consume? is Data API builder an option?
Hi Please help me. I have created an external table in the synapse lake database, Now I like to load the records from the external table into the dedicated SQL pool table. Please advice on the procedure.
As always, wonderful, I like this channel the most this video is very wonderful, very helpful, very informative again you were great Guys (in the cube :) )
Dedicated pools are just too expensive for any serious DWH stuff. Also its just Parallel Data Warehousing in the cloud, which is a horrible application to begin with. For me, I never use dedicated pools, I either use serverless or a Azure SQL DB. Now the downside with the Azure SQL DB is that microsoft enforces a recovery model of FULL, instead of SIMPLE like we had on onpremise. But that is on purpose, or simply no one would ever use dedicated pools as the difference would be too small. The whole MPP never sold me and it always performed like shit or could be done WAY better with some custom code. There some many improvements left for Microsoft to do in Azure, but some will be against their own business interest. Despite all the negativity in this post, I do love serverless SQL pools and its a huge step in the right direction. P.s. Also the presenter is DEAD wrong on dedicated pools. You dont choose this because of performance. You choose this as it fits your scenario. It is NOT faster at all, its faster at CERTAIN workloads. To make this comparison in with the serverless and dedicated Power BI load is very misleading.
Fully agree! Microsoft needs to figure out the future for dedicated sql pool! It might fit some specific scenarios however so far building the next gen enterprise data platform during recent years, I’ve not seen one case yet that we have to leverage the dedicated sql pool. In my opinion, Microsoft probably eventually have to sunset that!
Reserved 3 year is 300 dollars per 100DWU per month. Pay as you go is 1000+ Dollars So I guess that is a huge difference. Dedicated makes sense when you reserve it for 3 years. Sure... you don't get the option to turn it off but the workloads suited for this kind of service is intended for high availability just like direct query reports. But that means you do your data modeling in SQL and keep it minimal in PBI
Please dont breath on mic man, It feels like you are brething in my ears 🥹🥹🥹 I am just hating it 😡😡 Please do something, I can't concentrate on what the guest is saying
Would love to see some deep-dive detail about using both serverless and dedicated sql pools to handle different layers within a data lake when building a data platform.
Great stuff . I am trying synapse and other Azure offerings .. These short videos are great .. Can you also do a video so we can understand which is the efficient way to do ETL and load to PBI ..
Keep them coming. I am learning synapse and this was helpful
Hey, I am just checking same concept on Internet and got your notification in TH-cam. Its awesome.
Usually great stuff but this video seemed like a pure marketing stuff from the channel. Of course dedicated gonna be much more powerful because of strict schema and way more compute available. But it’s also very expensive.
It will be nice if we have some cost and capability comparison on premise, serverless and dedicated sql. Is the principal question a manager or a customer asks when offering this kind of solution.
This looks great for data virtualization for raw sourcing into the data warehouse.
That video was very informative! Great great video!
Ya this is definitely a good introduction to understand the difference between Dedicated vs serverless SQL pool. You should have added about Spark SQL pool also... what is that and how is that different?
Can you talk about the pricing for the Serverless Sql Pool, thanks!
What I found is that External Tables on serverless sql pool are like views. So you can't perform any DML operations in serverless sql pool which defeats the basic requirements of a DB
Note: with columns that take on values that are very large (e.g. json bits stored in col values), the simple syntax demo'd here won't work. This is because the inferred data type is varchar(8000). To bring back larger strings, you'll need to schematize the data in the query via the WITH construct, setting such cols to varchar(max) data type. Why varchar(max) isn't the default inference is beyond me.
Q: how can I automate the pausing of the dedicated pool? Does it come with scheduling options?
Our issue with Serverless is its wildly inconsistent.
2m row table 30 seconds to load and then 5 mins later that same 2m row dataset now takes 7 mins.
Any way we can address this?
Nope this is the downside is sharing resources with other users.....
Great Video, I would like to know if we can expose the serveless External Tables or View Via API for web application to consume? is Data API builder an option?
Is there a possibility to connect Synapse SQL Dedicated Pool from Powerbi using service principal?
Why was dedicated 2 seconds vs. Serverless at 30 seconds? Is the hardware not the same on the back end? What’s the difference?
Very useful. Thank you.
Hi Please help me. I have created an external table in the synapse lake database, Now I like to load the records from the external table into the dedicated SQL pool table. Please advice on the procedure.
awesome video, i had no idea autopause was a thing!
As always, wonderful, I like this channel the most
this video is very wonderful, very helpful, very informative
again you were great Guys (in the cube :) )
Assuming you can meet SLAs with each. I guess cost is the biggest contributing factor
Lot of heavy breathing on that mic bro. Haha
Again!
Dedicated pools are just too expensive for any serious DWH stuff. Also its just Parallel Data Warehousing in the cloud, which is a horrible application to begin with.
For me, I never use dedicated pools, I either use serverless or a Azure SQL DB. Now the downside with the Azure SQL DB is that microsoft enforces a recovery model of FULL, instead of SIMPLE like we had on onpremise. But that is on purpose, or simply no one would ever use dedicated pools as the difference would be too small. The whole MPP never sold me and it always performed like shit or could be done WAY better with some custom code.
There some many improvements left for Microsoft to do in Azure, but some will be against their own business interest. Despite all the negativity in this post, I do love serverless SQL pools and its a huge step in the right direction.
P.s. Also the presenter is DEAD wrong on dedicated pools. You dont choose this because of performance. You choose this as it fits your scenario. It is NOT faster at all, its faster at CERTAIN workloads. To make this comparison in with the serverless and dedicated Power BI load is very misleading.
Thanks so much for sharing this point of view.
😀
Fully agree! Microsoft needs to figure out the future for dedicated sql pool! It might fit some specific scenarios however so far building the next gen enterprise data platform during recent years, I’ve not seen one case yet that we have to leverage the dedicated sql pool. In my opinion, Microsoft probably eventually have to sunset that!
Have you used incremental refresh in PQ on an ASA Serverless SQL pool?
Reserved 3 year is 300 dollars per 100DWU per month. Pay as you go is 1000+ Dollars So I guess that is a huge difference. Dedicated makes sense when you reserve it for 3 years. Sure... you don't get the option to turn it off but the workloads suited for this kind of service is intended for high availability just like direct query reports. But that means you do your data modeling in SQL and keep it minimal in PBI
Amazing content! 🔥
Great video
The heavy breathing though; gotta go see a doctor man
this whole virtual warehouse, serverless pool is useless and doesn't work in production scenarios....
Please dont breath on mic man, It feels like you are brething in my ears 🥹🥹🥹
I am just hating it 😡😡
Please do something, I can't concentrate on what the guest is saying