Use Source Control with your Power BI Report in Microsoft Fabric!
ฝัง
- เผยแพร่เมื่อ 30 ม.ค. 2024
- People have been asking for source control with Power BI reports for a long time. It's here in Microsoft Fabric! Adam explores how to set this up.
Introduction to Git integration (preview)
learn.microsoft.com/fabric/ci...
Manage a workspace with Git (preview)
learn.microsoft.com/fabric/ci...
Power BI Desktop projects (PREVIEW)
learn.microsoft.com/power-bi/...
Git
git-scm.com/
📢 Become a member: guyinacu.be/membership
*******************
Want to take your Power BI skills to the next level? We have training courses available to help you with your journey.
🎓 Guy in a Cube courses: guyinacu.be/courses
*******************
LET'S CONNECT!
*******************
-- / guyinacube
-- / awsaxton
-- / patrickdba
-- / guyinacube
-- / guyinacube
-- guyinacube.com
**Gear**
🛠 Check out my Tools page - guyinacube.com/tools/
#PowerBI #SourceControl #GuyInACube - วิทยาศาสตร์และเทคโนโลยี
Be sure to register for the first ever Microsoft Fabric Community Conference 2024 and save $100 using code CUBE100! fabricconf.com
The MOST important Power BI video in years. Bookmark this video.
Would love a follow up video to this showing pipeline automation and recommended setup for full CI/CD.
that's actually better because by moving through pipeline, other rules can be applied. In my set up I have dataset in a different pipeline so by moving them through matching stages, it pointd to the right dataset stages.
This!
What I usually prefer is having just one development workspace hooked up with Git and using deployment pipelines to move from development to testing to production.
Agree that seems like a much simpler workflow
This makes sense to me. Since Dataflows are not yet supported in Git and the deployment pipelines automate pointing DF connections in the same workspace.
Hey! Great video! I think you should go a step further with Git + Deployment Pipelines integration. I found this was the best option for me and works smoothly. Also, if you could comment what kind of licence you need to implement each case would be great!
Watch your videos all the time! You are amazing!
Plus ,
1. The method shown here is a bit different from normal gitflow as far what I have seen, which starts from dev branch and creating a feature branch from there. Can it be done here also? Or we always need this initial file mode.
2. How will in a big organization the access control of those branches happen?
Thanks Adam! You're the REAL MVP!! This is EXACTLY what we have been waiting for to manage our Power BI projects.
Thanks for sharing Adam! this is exactly how I was working and it's great to see you guys doing it to compare best practices for repo integration. I guess we just need to wait for an automatic way of pulling the changes to the workspace and update the App with Rest API and we could finally build an amazing version controlled by environment CICD.
I can't stop thinking that deployment pipelines will be a different approach but not necessary if we keep this amazing practices keeping developers working with repos and DataOps with Fabric.
Super helpful, thanks Adam!
Wow 🤩🤩🤩 That was AMAZING !!!
What an awesome video - just what I needed. Agree with comments - how does this work with Deployment Pipelines?
Happy to see heavy command line Git. It's the best way to learn.
Just what I’m used to. It’s rare for me to see someone go command line.
@@GuyInACubeThanks for the video! Could you tell us how did you get the PowerShell color scheme (blue for the folder, yellow for the branch ...) ? Thanks a lot!
I think it's a git zsh theme. Could you share it? It's nice! Thanks!
Q: What if two developers are working on the same dashboard at the same time?
Q: What if two reports are connected to the same model and we create new measures in both the reports then how will it work?
Q: Will it work when we use a live connection to develop a new report?
Just some quick questions. Great video!!!!
1. Yes it's possible there is a tool called ALM
2.create the measures in the siematic model instead of report
3.yes it works
well this is cool and long awaited! Thank you for the intro!
@GuyInACube how do you make the terminal to display the branch in the path?! that is awesome
That's cool!! Can you also do a follow up video on how to roll back to any of the previous commits no one have explained this any of the videos or blogs that would be really helpful
Ohhhh that’s a good one. Will do. My assumption is you need to do that from a Git perspective.
@@GuyInACube Actually in Azure DevOps is possible if you go to Repos->Pull requests, you can revert that one specific pull request you made in the past. I am still in kind of testing how this behaves wrt PBI so I might be wrong, but since this worked in the past for some "Descriptions" in the model, I assume this should work
This video is awesome. How about configuration of parameters between dev and prod and change that automatically with the deployment from dev to prod?
Agree, would love to see how to rollback to a previous commit. Even better if this was a selection option within the workspace (submitting to idea place).
Also, as others have mentioned, would like to see a good practice incorporating the deployment pipeline.
This is a perfect video, thanks. I would like to understand how does this concept connect with Pipelines. Is there a Video on this, please?
Should we use deployment pipelines in this process? For example, only push via git to the dev workspace and then use deployment pipeline to push up to the higher lifecycles? Is this not advised?
So pull requests deploys the changes directly to the prod ws in power bi ? Pipelines is not necessary ? Thank you for a great video!
Cannot wait to set up this for the team once they support more than Azure DevOps repositories. 😅
Q: nice one! How does it work together with Fabric deployment pipelines?
So many questions.
-can we report on the commit messages like we can with GitHub?
-is this functionality ideal for report source control?
-I’ve been using Tabular Editor + TMDL + GitHub for semantic model source control - and I’m loving it. Could you walk through this process for a large dataset that is managed solely via XMLA endpoint?
@GuyInACube I'm a little bit confused by the different options to manage deployments at this point. Should I use the paradigm of git branches or PowerBI development pipelines?
Can this be set up to work with deployment pipelines in any meaningful way?
Any idea when there is support for Direct Query datasets?
This is great!! Should organizations start using this at a global level while this is still in preview or should they wait for a general availability release?
Is there a way of linking to a devops account on a different tennant? Also be great if this functionality was directly within the desktop app
Same problem in our organization
Big Question: yay or nay for mapping data flows in Data Factory (FABRIC) this year?
Q: where do you guys learn all this stuff? I feel kind of lost
What a great and useful watch! Thank you so much for this.
I've got a question: given all the prerequisites, is it also possible for a multi-user collaboration work in PowerBI Web with Git? A report is in a shared workspace and the structure is as follows: each git branch is an isolated dev's work and they merge changes into the master to obtain the end report for prod. No matter what each dev is working at - a page, semantic layer, etc.
Hey Adam,
first off, great video! Really helpful stuff. My question, is version control also possible in the Power BI service? We comment a lot in the reports as plain text, but overwriting each other's work is a problem for us.
Thanks!
How do you get colors for the branches and other stuff?
I am currently using CICD on Azure DevOps with pbix and PBI API.
I love that MS finally have PBIP so I don't have to use the pbix (binary file) on git. But the deployment method is kind of sub-par to me. As this vid shows, the roll back method is to add another commit. If you don't remember what you have changed, you have to go through all of the commits.
I will stick with PBIX until PBI API supports publishing reports with PBI or MS allow to build PBIX from PBIP in a CLI fashion.
Really useful material. Does it require premium workspace?
It is important to know the limitations of Git in Fabric, too. Like size of files etc. Also, Premium pipelines have own limitations. Yes, it is good that Fabric has got those features but it is not enterprise-ready at the moment given the limitations.
This is great and all but how do I convince my admin to turn it on for me? Also, will it work with pro accounts?
What about resolving conflicts during pull request?
I bet it will be just like the bad old days of visual studio - relevant semantic changes drowning in giant noisy random incomprehensible XML diffs for no good reason. Programming is (still) terrible...
@@trumpetpunk42i do not believe this to be a problem with the new tmdl syntax, where each table is saved as a separate file.
Pull request conflicts can be solved in your local git, as one would solve other pull request conflicts when doing other type of development. If you have a feature branch and you get conflict on PR to main, rebase feature branch onto main branch (on the local git), resolve merge conflict , and then create PR again.
How does this all integrate with deployment pipelines (if at all)? Would this be something that effectively replaces deployment pipelines, or do they work together somehow?
Would love to see more videos on this and Microsoft's/your recommended best practices for how to best handle CI/CD in PBI/Fabric!
I think we use Git to push all changes into DEV but then use deployment pipeline to sync between DEV and PROD. It seems a lot of push and pulls syncing between DEV and PROD using GIT. I think can be simplified with a deployment pipeline between the 2 workspaces. My 2 cents.
@@nafiulislam4214 Thanks, that was the conclusion I came to as well, Git integration basically up to prod, then leveraging deployment pipelines.
It's all great, but you skipped over one of the most important parts of all this - reviewing the actual pull requests. What I've found so far is that is not great, a bunch of random files that I don't necessarily understand. Thoughts on that?
I saw other video with some guys from MS saying that they are working on making project files more human readable, especially report layout but don't know when that is coming
yes, that sucks and make it almost un-reviewable. What I implement in my teams is to have the developing report published to a playground workspace for reviewing and light testing. Still wish the file is more readable but I doubt it is possible in the near future. Changes in dataset is somehow readable though, not easy but its acceptable.
The new TMDL format solves this issue! 🎉
Q: I am very new to Git. What is that interface you are using ?
Does it work with metrics too? For example: you have a team that will develop dashboards in Power BI, but each of them is using a specific branch and they will make different pages in Power BI, each creating their specific metrics.
does this have limitation for reports using live connect ? my pbip file, which uses Live Connect to an on-premises SQL Server Analysis Services (SSAS) server, won't be synchronized through Git integration, it raise error that Failed to discover dependencies
Does it need to be a Premium Workspace to enable Git capabilties?
When you use Git+DevOps, why would you use Deployment Pipelines?
Nice! Does anyone know what tool is used in the GIT cmd of the video?
He also did a TH-cam video:
th-cam.com/video/VT2L1SXFq9U/w-d-xo.html&pp=ygVXc2NvdHQgaGFuc2VsbWFuIE15IFVsdGltYXRlIFBvd2VyU2hlbGwgcHJvbXB0IHdpdGggT2ggTXkgUG9zaCBhbmQgdGhlIFdpbmRvd3MgVGVybWluYWwg
I figured it out, its oh-my-posh
Q: @adam - I tried using this, we have an issue with sync, the error message says datasetdefinition: required artifact missing. When I checked the required fes needed in the microsoft blog, the name says as definition.pbidataset but when I save it as a .pbip, it stores the datset under semanticModel with just “definition” folder name. Does this naming convention cause an issue while trying sync and throw an error saying required dataset artifact missing or I’m missing something all together. Please answer this question, it would save ton of time and effort.
Would be interesting to understand how would GIT work if you have for example 5 different reports connected to a single semantic model where all artifacts are within a same workspace and would that create any problem while doing commits / pulls.
It would not be any problem, since each report would be saved as a separate pbip project and thus separate folder. Ideally, the semantic model would also be saved as a separate pbip so that you can more easily apply the golden dataset rule. So in the Git repo you would have 6 folders, one for the dataset and one for each of the 5 reports that go into the workspace.
How does the difference looks like between two commits? is it properly readable? Till Date I am using Sharepoint Check in & Check out method for version control of a pbix.
It's readable. Similar to what ALM toolkit shows you
Can multiple people work on the same file?
@adam what if i just want to upload the report in the repo in dev branch and raise a PR instead of using got commands on console. Is that feasible. I think it is feasible but what file we need to upload or replace the pbip or everything.
there are many UI applications for git. I used SourceTree personally but so many good options out there
Nice video. 👊 What's the point of deployment pipeline? This seems like a better option just not as user friendly as the pipeline flow and more manual steps to check for differences. Also is the developer working on the pbip file and not pbix file? Is there any risk there
I couldn't agree more. Once they add automation like pulling the change automatically at the workspace or update Power Bi App... I won't see the point for deploment pipelines... This would be way better to keep control version of each environment.
Any ideas how this would work when a dev workspace pulls from a dev data warehouse? I can't see how this would work with git, but it seems integrated in deployment pipelines
Hey, Adam, 90s called. she wants her Emoticons back. 😂😂
How do u do this using vs code? Or do u need to do this in azure devops?
For the PBIP on its own, you can use whatever you want. For the integration in the Fabric portal, it has to be Azure DevOps as of today.
How about gitlab?
Why does a merge conflict occur if main branch and dev branch create a new report page (which have nothing in common and different names). As soon as I rebase to combine both i get a merge conflict. That does not make any sense to me.
You should have a policy on the main branch so that direct commits to the branch are not allowed, only PR.
Hi, Guys. Can anyone please explain what is the benefit of doing all this?
Q: I'm confused because we can use just prod and Dev workspace as it is and for version control we can use OneDrive. Personally I think this is little bit lengthy process. Please correct me if I'm wrong.
BTW @GuyInACube, I'm big fan.
Collaboration during development. Two developers working on different changes to the same file. That's where git excels
It would be great if you could announce in the video (near the start) or in the thumbnail if the content requires a premium or PPU sku so that us PRO scrubs needn't bother watching. Thanks
Let me know when you have a visual interface for GIT in your visual interface product. Nobody on our PBI teams uses GIT or has needed to. Going back to DOS land to manage PBI files.. blarg.
If the *common practice for doing things* is rocket surgery, then it's a system ripe for disruption.
Using Visual Studio Code instead of powershell is an option but still a little too "techie" for an average user and requires multiple tools. I hope it gets easier and simpler.
Adam demoed the visual interface in the video at about 10:45 (th-cam.com/video/zvyr2qYCQNo/w-d-xo.htmlsi=9ZTCEECNYadgke-f&t=645). Is this not exactly what you're asking for?
git has the best command line interface in that it gives reminds of the commands you might use.
@@KNP-BIIt's great IF you're working in Fabric. Fabric still isn't approved for our workflow because our DevOps folks are leery of MS dumping support for whatever flavor of hotness of the month is.
Meanwhile, as demonstrated in the video there's no integration in the Desktop App, so we're still having to manually play around with PBIP solution folders on the network storage at which point Git is little more than a extra work to document why we have a dozen solution folders for 'branches'.
FWIW, we're using GIT in VS. But there's no editor in VS for PBI, so it's just a glorified Synch tool.
It would be nice to have REST API that allow to publish .pbip the same way as .pbix. On our project we have fully automated deployments from git repository and without REST API still need to have a pbix copy of report
This is now available
B A N A N A S !
is this a premium only feature?
Starting from PPU.
@@rimantsvinups-sakars9151 looks like it can be enabled if you have a fabric capacity so an F2 sku allows the git integration selection on workspace
Thanks. I love using the Git integration. Source control in VS Code makes it a BIT easier, but still a bit heavy for a non business user. A small, well meant challenge: Try do a few videos leaving out the word “Perspective”
love it, but waaaaaay to many bugs. Guys, it is BETA, so don't go all in to fast.
I don't want to refresh my entire model (it's close to 1 billion rows) for minor changes to the tabels.
ALM toolkit is still being used to bypass the bugs. Otherwise thumbs up
Disappointed. I thought pbix merge would come with this. That's the big deal we're waiting for. Without it, multiple people cannot work on the same report simultaneously. We need more granularity and fragmentation of pbix files.
Need premium. RIP
Thanks for the video. This is burdensome. Don't tell me we can't do this with native DevOps instead of Git? I mean, just Microsoft DevOps alone. Pull requests and all can be done through the browser instead of doing stuff through code in PowerShell or whatever.
Could this be any more complicated? Pretty much unusable for mortals. Why is there not just an add to source control button?
it's a total of 5 commands.. git pull, git checkout -b "XXX", git add. , git commit -m "XXX", git push
Write it down and repeat, it's pretty much a standard.
Or just pay for PowerBi cloud and you don't have to go through all this BS...