This is amazing. I am actively looking for something to replace our current approach to simulate real-time data from on-perm SQL database to a storage account. This is a true solution in my opinion. Can’t wait to test it out. Thank yoi
Hi @NicolasPappasA per Bradley, "I'm breaking building this report a bit more in my next video. For the max size I used a measure to make the most recent bubble bigger than the others. I put that measure in the size option for the ArcGIS Map. The colors were a conditional statement I did not cover, but I used the values to make the trailing dots yellow and the most current dot Red. I will try to expand on this the next time I'm putting a video together on the ISS. I'm not done, hopefully, I've got some fun stuff planned for this!"
@@Tales-from-the-Field amazing... looking forward to it...I created a measure too with a max latitude but couldn't get the way you did...sounds like I am missing the conditional statement, going to try that...Thanks again for replying to this
Thanks for the Demo. I investigated mirroring several months ago and saw disclaimers not to use this on production databases. Do you know if it is safe at this point?
Hi @Thirdwaveltech per Bradley "Hello! This is still in Public Preview, so just a little bit on what that means. At this point our CSS teams are trained, there is a PG that can work on feature enhancements and bugs, we have an internal SLA like any prod Azure product, however.... HOWEVER, the product is not yet covered by a financially backed SLA. So what does that mean? It means our goal is to support this and have an uptime support as if this is in production and to exceed our targeted SLA. But if the service went down, there is not a financially backed SLA meaning you are not entitled to a financial credit if there is an issue. Now I don't fully understand what all of this means, to the nth degree other than that statement. Other than that the limitations that are currently in place, and there are still some like regional availability, 500 table limitation, views, tables without primary key (coming but not here yet), some of those things are still limitations. I hope this helps!"
Hi @mathiassaint per Bradley, "Sure thing. So for this demo what I wanted to showcase was that no matter what format the data is stored in with Microsoft Fabric we can access it using any Engine we would like. So why did I use KQL DB instead of the SQL Endpoint? I could do that. However, this data set is very linear in nature and I'm using Direct Query. The KQL Engine works really good with time series data, I could create T-SQL stored procedures to accomplish something similar but KQL's query capabilities for a specific fraction of time that grows as I write more data to the database allow me to get this quickly in a way that will not deteriorate over time with volume and remains lightning fast. Could be nothing more than preference on my part, but if I were building this for custmer use that's the engine I would recommend for this workload. Thanks for the questioni!"
Great video, thanks, Bradley. based on my understanding, mirroring doesn't allow cross-tenant connection. I have a trial fabric account, but can't add any other Azure services, like ASQL there (pay as you go option is disabled for some reason). Any ideas on how to overcome that problem?
Hi @fazizov per Bradley, "Hello sir! If I'm correct in my understanding you have a Fabric / Power BI Tenant and the Azure Account is not associated with that through the M365 Tenant. The Azure Account that you want to Mirror must be associated witht he M365 account (not an M365 expert, but this is what I've been told). If that is the case the Mirroring should not be limited to Trial capacity, you should be able to Mirror into it. The Trial capacity is equivalent to an F64 so you should have the functionality of an F64 other than those items that have been called out as not included in the trial capacity like private endpoints and managed identity. If your Azure Account is associated with the M365 account your tenant is in please let me know, Mirroring should not be facing any other restriction."
Hi @gopalannasundaram9989 per Bradley, "Great question, this functionality is on the road map but it is not currently in Azure SQL DB Mirroring in Microsoft Fabric. Right now there are some limitations with regards to data masking or encrypting the fields and it being brought over into Microsoft Fabric. That functionally is coming but I'm not sure how soon it will be here."
I think you went by the connection too fast. Spending more time on a proper authenticated method using identity should have been covered because your viewers are working with production workloads and not just setting up a demo. Otherwise, very useful content. Thanks for taking the time.
I encountered the following error: "The database cannot be mirrored to Fabric due to below error: User does not have permission to mirror the database." How can resolve this issue?
Hi @totnguyen3308 per Bradley, "In the Fabric Workspace for the Mirror click on the three dots ... and click on Manage Permissions, then make sure than the Managed Identity for the Azure SQL Database is listed with Read, Write permissions. If it is not there you need a workspace admin to add it, but that will prevent Mirroring for working. You can also run, EXEC sp_help_change_feed against the mirrored database. There is a column named state. You want the state column to equal 4 otherwise, mirroring is not successfully set up."
@@Tales-from-the-Field Could you provide a detailed video tutorial on the setup process? I am currently using the trial version and have an admin role in the workspace. I suspect the issue might be with the Azure SQL Database configuration, but I'm not exactly sure. Thank you for your support.
@@totnguyen3308 Working on this for next Wednesday! It has been a while since we released a video with all the hurricanes and family impacted by them, so stay tuned this may happen sooner!
Great demo Bradley!
Thank you Will!! Always appreciate your support my friend!
This is amazing. I am actively looking for something to replace our current approach to simulate real-time data from on-perm SQL database to a storage account. This is a true solution in my opinion. Can’t wait to test it out. Thank yoi
Great demo and thanks for adding Power BI into picture. KQL Database so far is not demoed very often, great to see this example as well!
Thank you @pawewrona9749! We are BIG fans of the KQL DB over here. Agreed, it doesn't get nearly as much love as it should!
Amazing Demo...Thank you Bradley.
Bradley how did you get the max bubble in your Power BI working like that?
Hi @NicolasPappasA per Bradley, "I'm breaking building this report a bit more in my next video. For the max size I used a measure to make the most recent bubble bigger than the others. I put that measure in the size option for the ArcGIS Map. The colors were a conditional statement I did not cover, but I used the values to make the trailing dots yellow and the most current dot Red. I will try to expand on this the next time I'm putting a video together on the ISS. I'm not done, hopefully, I've got some fun stuff planned for this!"
@@Tales-from-the-Field amazing... looking forward to it...I created a measure too with a max latitude but couldn't get the way you did...sounds like I am missing the conditional statement, going to try that...Thanks again for replying to this
Thanks for the Demo. I investigated mirroring several months ago and saw disclaimers not to use this on production databases. Do you know if it is safe at this point?
Hi @Thirdwaveltech per Bradley "Hello! This is still in Public Preview, so just a little bit on what that means. At this point our CSS teams are trained, there is a PG that can work on feature enhancements and bugs, we have an internal SLA like any prod Azure product, however.... HOWEVER, the product is not yet covered by a financially backed SLA. So what does that mean? It means our goal is to support this and have an uptime support as if this is in production and to exceed our targeted SLA. But if the service went down, there is not a financially backed SLA meaning you are not entitled to a financial credit if there is an issue. Now I don't fully understand what all of this means, to the nth degree other than that statement.
Other than that the limitations that are currently in place, and there are still some like regional availability, 500 table limitation, views, tables without primary key (coming but not here yet), some of those things are still limitations. I hope this helps!"
Awesome demo! Thanks for sharing!
Thank you @crax2000! Really appriceate you watching!
Thanks for a great demo.
Can u explain why ur not able to use the SQL endpoint directly into PBI?
Hi @mathiassaint per Bradley, "Sure thing. So for this demo what I wanted to showcase was that no matter what format the data is stored in with Microsoft Fabric we can access it using any Engine we would like. So why did I use KQL DB instead of the SQL Endpoint? I could do that. However, this data set is very linear in nature and I'm using Direct Query. The KQL Engine works really good with time series data, I could create T-SQL stored procedures to accomplish something similar but KQL's query capabilities for a specific fraction of time that grows as I write more data to the database allow me to get this quickly in a way that will not deteriorate over time with volume and remains lightning fast.
Could be nothing more than preference on my part, but if I were building this for custmer use that's the engine I would recommend for this workload.
Thanks for the questioni!"
Great video, thanks, Bradley. based on my understanding, mirroring doesn't allow cross-tenant connection. I have a trial fabric account, but can't add any other Azure services, like ASQL there (pay as you go option is disabled for some reason). Any ideas on how to overcome that problem?
Hi @fazizov per Bradley, "Hello sir! If I'm correct in my understanding you have a Fabric / Power BI Tenant and the Azure Account is not associated with that through the M365 Tenant. The Azure Account that you want to Mirror must be associated witht he M365 account (not an M365 expert, but this is what I've been told). If that is the case the Mirroring should not be limited to Trial capacity, you should be able to Mirror into it. The Trial capacity is equivalent to an F64 so you should have the functionality of an F64 other than those items that have been called out as not included in the trial capacity like private endpoints and managed identity.
If your Azure Account is associated with the M365 account your tenant is in please let me know, Mirroring should not be facing any other restriction."
Thank you very much for detailed explanation,I will definitely try that advice!
Great demo! Can we anonymize certain PII fields or select subset of columns in a table while setting up mirroring?
Hi @gopalannasundaram9989 per Bradley, "Great question, this functionality is on the road map but it is not currently in Azure SQL DB Mirroring in Microsoft Fabric. Right now there are some limitations with regards to data masking or encrypting the fields and it being brought over into Microsoft Fabric. That functionally is coming but I'm not sure how soon it will be here."
I think you went by the connection too fast. Spending more time on a proper authenticated method using identity should have been covered because your viewers are working with production workloads and not just setting up a demo. Otherwise, very useful content. Thanks for taking the time.
I encountered the following error: "The database cannot be mirrored to Fabric due to below error: User does not have permission to mirror the database." How can resolve this issue?
Hi @totnguyen3308 per Bradley, "In the Fabric Workspace for the Mirror click on the three dots ... and click on Manage Permissions, then make sure than the Managed Identity for the Azure SQL Database is listed with Read, Write permissions. If it is not there you need a workspace admin to add it, but that will prevent Mirroring for working.
You can also run, EXEC sp_help_change_feed against the mirrored database. There is a column named state. You want the state column to equal 4 otherwise, mirroring is not successfully set up."
@@Tales-from-the-Field Could you provide a detailed video tutorial on the setup process? I am currently using the trial version and have an admin role in the workspace. I suspect the issue might be with the Azure SQL Database configuration, but I'm not exactly sure. Thank you for your support.
@@totnguyen3308 Working on this for next Wednesday! It has been a while since we released a video with all the hurricanes and family impacted by them, so stay tuned this may happen sooner!