Looks very promising and solves a big problem of connecting reports into delta lake HOWEVER I agree with the comments below that the table does not appear in Synapse Studio Serverless Sql pool and also does not support stored procedures. The table does appear in SSMS under the serverless pool but once again does not support stored procedures. Why does the video make out that stored procedures and functions are supported? It would also be nice if the video explained the benefit of using data lake pool over sql serverless pool: Cost? Performance? Features? Please we need more detail.
I am facing performance Issue in Azure Synapse Notebooks and using 4vcore 28gb spark configuration. How to improve the performance of the azure synapse notebook. The Synapse Notebooks running fine for sequential execution but when the notebook runs in parallel , experiencing the performance and taking longer time execution in the process, We wanted to ensure the process should run quickly as a behavior for both parallel and sequential.
Hello, im my case after enabling the link between D365 and Azure synapse i am trying to develop a pipeline that will query the lake database everty night, first problem i am facing is that i cannot develop stored procedures or functions insie the lake database, second i dont know what component i should use in order to query the lake database inside the pipeline, can you please tell me what's the right approach to do so. thank you
This really doesn't work for me. I am doing the exact same thing, I can also query the damn table in the notebook but when trying to use SQL Serverless on it, I get invalid object. And Yes I am including the [dbo] in the name. SSMS doesn't see the table either.
Hi Please help me. I have created an external table in the synapse lake database, Now I like to load the records from the external table into the dedicated SQL pool table. Please advice on the procedure.
If your external table is stored in a compatible format such as CSV, Parquet, or ORC, you can use PolyBase to load data into a dedicated SQL pool table. PolyBase allows you to create an external table that references the data stored in external files, and then you can use a T-SQL INSERT INTO...SELECT statement to load data from the external table into a SQL pool table. INSERT INTO TargetTable (Column1, Column2, Column3) SELECT Column1, Column2, Column3 FROM ExternalTable;
Our by Notebook with pySpark created Delta Tables are also available as a Lake Database as shown in the Video, but we are wondering why the Synapse Studio Designer is not supporting them and it looks like that there are small issues in the GUI. See Screenshots in here: learn.microsoft.com/en-us/answers/questions/1048309/synapse-studio-lake-database-designer-and-delta.html Currently we have decided to NOT use the Designer, as if we create the Database first and then run our Notebook then no Tables appear in Synapse Studio. And if you recreate them by the Designer you can't select Delta. Little bit nasty is that Columns are not available in Synapse Studio, so you can't create a Select Statement with all Columns, but with SSMS it is working. Not sure if we will mith something else in future if we can not use the Designer?
![[Full Episode] The Restaurant War Thailand ศึกพ่อค้าซ่าแม่ค้าแซ่บ Episode 4 | 13 ต.ค. 67](http://i.ytimg.com/vi/h8N9tY46P-M/mqdefault.jpg)
Very nice overview of integrating Lake Database with Serverless pool in Synapse.
Looks very promising and solves a big problem of connecting reports into delta lake HOWEVER I agree with the comments below that the table does not appear in Synapse Studio Serverless Sql pool and also does not support stored procedures. The table does appear in SSMS under the serverless pool but once again does not support stored procedures.
Why does the video make out that stored procedures and functions are supported?
It would also be nice if the video explained the benefit of using data lake pool over sql serverless pool: Cost? Performance? Features?
Please we need more detail.
Great video. Thanks.
I am facing performance Issue in Azure Synapse Notebooks and using 4vcore 28gb spark configuration. How to improve the performance of the azure synapse notebook. The Synapse Notebooks running fine for sequential execution but when the notebook runs in parallel , experiencing the performance and taking longer time execution in the process, We wanted to ensure the process should run quickly as a behavior for both parallel and sequential.
Hello, im my case after enabling the link between D365 and Azure synapse i am trying to develop a pipeline that will query the lake database everty night, first problem i am facing is that i cannot develop stored procedures or functions insie the lake database, second i dont know what component i should use in order to query the lake database inside the pipeline, can you please tell me what's the right approach to do so. thank you
This really doesn't work for me. I am doing the exact same thing, I can also query the damn table in the notebook but when trying to use SQL Serverless on it, I get invalid object. And Yes I am including the [dbo] in the name. SSMS doesn't see the table either.
Hi Please help me. I have created an external table in the synapse lake database, Now I like to load the records from the external table into the dedicated SQL pool table. Please advice on the procedure.
If your external table is stored in a compatible format such as CSV, Parquet, or ORC, you can use PolyBase to load data into a dedicated SQL pool table. PolyBase allows you to create an external table that references the data stored in external files, and then you can use a T-SQL INSERT INTO...SELECT statement to load data from the external table into a SQL pool table.
INSERT INTO TargetTable (Column1, Column2, Column3)
SELECT Column1, Column2, Column3
FROM ExternalTable;
Our by Notebook with pySpark created Delta Tables are also available as a Lake Database as shown in the Video, but we are wondering why the Synapse Studio Designer is not supporting them and it looks like that there are small issues in the GUI.
See Screenshots in here: learn.microsoft.com/en-us/answers/questions/1048309/synapse-studio-lake-database-designer-and-delta.html
Currently we have decided to NOT use the Designer, as if we create the Database first and then run our Notebook then no Tables appear in Synapse Studio. And if you recreate them by the Designer you can't select Delta.
Little bit nasty is that Columns are not available in Synapse Studio, so you can't create a Select Statement with all Columns, but with SSMS it is working.
Not sure if we will mith something else in future if we can not use the Designer?