Great to see this. I'm a bit of an obsessive over tidying up my transforms into as few steps as possible. I didn't realise it had such a bearing on performance so I'm glad I have an additional, practical reason to so heavily optimise my transforms now!
I noticed this too. Thank you. If you could expand on this topic, that will be great. For example, could you go back to a step and add more actions to itself. Is there a tool that would "look into your" power Query and "Optimize" it for you?. You could explore what actions are more time-consuming than others. For example, merging, vs filtering, vs replacing values.
The biggest issue i have in PowerQuery regarding flagged memory errors are when i need to use 'group by' for mass amounts of data and then un-'group by' to get the underlying line details back. Still have not been able to solve for this other than maybe order some more RAM memory to hopefully address this.
hi and thanks. i am wondering what to do about "Add Index" step that i use 2 times after one another, it takes too much time according to query diagnostics?
Great! I have one question Paul : in therms of memory addressed , are all those steps treated by PowerBI as a sort of new variable each that never gets erased till the end of the steps? just asking would you suggest to lower the ram used by forcing reusing the same 2 or 3 'names' of variables/stepnames to load the dataset output of each step in fewer ram (just in case of many programs opened and less write to disk operation which may improve overall speed of execution?). Have you ever tested or had experience of improved performance as such? thank you
Not only that, but it looks much cleaner and professional and is easier to read the flow of steps when you don't have a lot of duplicate steps!!
Great to see this. I'm a bit of an obsessive over tidying up my transforms into as few steps as possible. I didn't realise it had such a bearing on performance so I'm glad I have an additional, practical reason to so heavily optimise my transforms now!
thanks for this! I always group together similar M steps by instinct out of sheer borderline OCD. Apparently it's the way to go (or be)!
I noticed this too. Thank you. If you could expand on this topic, that will be great. For example, could you go back to a step and add more actions to itself. Is there a tool that would "look into your" power Query and "Optimize" it for you?. You could explore what actions are more time-consuming than others. For example, merging, vs filtering, vs replacing values.
Hi Paul. I'm running Office 365 and do not see the Tools menu at the top of the Power Query editor. Do you have any guidance, thanks.
Great learned useful Diagnostic test to see the process time for each steps of the query
Very nice video! Thanks
The biggest issue i have in PowerQuery regarding flagged memory errors are when i need to use 'group by' for mass amounts of data and then un-'group by' to get the underlying line details back. Still have not been able to solve for this other than maybe order some more RAM memory to hopefully address this.
I have the same problem. Did you solve this yet?
hi and thanks. i am wondering what to do about "Add Index" step that i use 2 times after one another, it takes too much time according to query diagnostics?
It's really a Great Video!
Thanks. I'd like to have more tutorials like these.
Great! I have one question Paul : in therms of memory addressed , are all those steps treated by PowerBI as a sort of new variable each that never gets erased till the end of the steps? just asking would you suggest to lower the ram used by forcing reusing the same 2 or 3 'names' of variables/stepnames to load the dataset output of each step in fewer ram (just in case of many programs opened and less write to disk operation which may improve overall speed of execution?). Have you ever tested or had experience of improved performance as such? thank you
Simple, but very helpful. Thanks
Cool I gotta redo all of my queries now
Really helpful and well explained, thanks!
Thank you, great video. Already subscribed
Thanks for details
Great tip thanks so much
Thank you!
9mins just to tell us to consolidate our steps. I think you need to optimise your videos...