The advice to stich together collections to load large data sets is probably the most shared (bad) advice in the community. Thank you for making such an excellent video that highlights this.
@@RezaDorrani I think it is because delegation is sometimes explained as the maximum number of rows PowerApps is able to work with, which is not true at all! A better explanation would be: It is the maximum number of rows it SHOULD work with, but LOCALLY and not server side. The example where you scrolled to the second page of your dataset showed this so beautifully together with the row count. It showed that when you respect delegation, PowerApps will actually help you paginate your dataset and ensure you still get the correct result from your queries. I mean how nice is that?! You don't even have to do anything; Power Apps just handles it for you. I stitched together collections in the beginning because I read bad advice and the worst thing about it, is that it kinda works; it performs badly, but it gets the job done. It was only later until I learned about delegation from gurus like you and Shane, that I understood my approach was severely flawed, and that by understanding delegation I realized that it makes my life much easier. I simply didn't understand that PowerApps could filter 1.000.000 rows server side trough delegable queries, but still be limited to only show 500/2000 rows locally. But it makes sense when you think about as you said, that the whole system is in place to urge you to minimize the amount of traffic between the app and the server. I still have many apps where the delegation is set to 2000 rows, but when you think about, how often do you really need to present more than 500 records to the user at the same time? Probably never. And how many apps have tables with more than 500 records? Probably all of them. Delegation works brilliantly when you understand it, and that's why I wanted to show my appreciation for your work Reza, because your video can improve the whole Power Apps community in a place where it is most needed.
A really good refresher course on this all important topic. Hopefully, a few more functions will become delegable soon. Thanks for sharing. Appreciated.
Awesome video Reza. In regards to collection, one of my client keeps insisting that we load 6k to 8k into a collection and I have to use concurrent function. Takes about 30 secs and the app's performance is slightly worse, not App breaking but not as nice at all.
if the function is non delegable then the intellisense should be showing the delegation warning,..... surprised to know the issue with ' with ' function ... thanks for this amazing video sir............
"With" not having a warning is expected in the example shown at ~10:40, it would be the same as if you did 1) ClearCollect(colStudentData, Students), 2) Filter(colStudentData, Region.Value "South") (no delegation)
@@MuFeR. I agree. Delegation warnings should only be for functions that is expected to work in the data source. With is a purely Power Apps function that can not be delegated. So a warning is misleading.
Dude you are amazing. I learn best from examples like you show. Thank you so much. The Superbowl is on right now and I'm learning from your video. I'm learning here to advance my personal power, otherwise I would be wasting time. Thanks :)
1) Don't trust the warnings for delegation (either that they do or don't show as delegable) 2) Delegation ONLY becomes an issue for your apps if the dataset gets large (>500 or >2000 once you change the defaults). 3) You can test to see if you have some delegation issues by setting an abnormally low query limit for your app (down from 500 to a smaller #) 4) All Delegation issues can be solved through better queries, better data organization, using Power Automate & ParseJSON, etc. (which all mean Dataverse isn't required to solve delegation issues in most scenarios) 5) Microsoft is likely to push better delegation features within Dataverse first (but that doesn't mean you should use Dataverse unless your app requires it or your organization is focused on supporting Dataverse more widely)
It's been great Reza sir if you post on your experienced basis,scenario based interview questions its been really helpful for us You are Gem 💎💎 for us Regards
Reza, thank you for the video. I built an app connected to a SQL server that provides data that changes minute by minute. What is the best way to provide a count of this data based on different criteria like total number of calls, high priority calls, and pending calls? I do get the delegation warnings but the number of records displayed at one time is around 200 or less. This app also provides the users the ability to do a history check based on either an address or phone number. Is there a way to ensure that the users are getting the full list of records if this search does exceed the delegation limit?
Count functions are non delegable with sql. You could call stored procedures to get counts. I have no video references on sql to help direct you to. I will recommend checking or posting your query on forums in case someone has done something similar powerusers.microsoft.com
Very well explained Reza sir! Thanks for creating this video. I have seen Power Apps slowing down with 1000 rows in SP list. What is the best or recommended source/backend for any Power App?
Hi Reza, Thanks for this video. Really Helpful. Quick Question: I am working with around 0.5 Million rows SQL table. On this table we need to perform a group by operation and create landing page and for slicers we need to perform a distinct on column and get all different values from that column. The challenge here now is Group by & Distinct are non delegable Power Fx functions when connected to a SQL connector. There is no way that I can found to do this with other delegable functions. How do we approach this problem now?
@@adarshmc8158 @RezaDorrani - Thanks for the idea, I can create a sql view for group by post which also I will end up with a table greater than 2000 rows, On this table if I am using distinct its still an issue as it picks up the values from only first 2000 rows and I end up not having all unique values from my column. Anny idea on how do I tackle this after creating Views. Thanks in Advance
hi reza, thanks for the great video!! i do have a question is about, i heard there's actually a 5000 items limitation for sharepoint.. i'm just worry about, if my sharepoint list has more than 10,000 items, when i using filter() function, will the data be impacted by that 5000 sharepoint limitation even filter is a delegate function?
Great video as always! I saw in documentation the 'in' Fonction is delegable in dataverse but I try to filter a gallery on a multi choices column and get delegation warning. Does a workaround exists? I try addcolumn(concat to filter on string but performance is not great because the app needs to collect the whole table.
Hello Reza, It's again a nice video. I have question regarding delegation. I set delegation limit to 2000 (that is max limit) in this case if my search result crosses the limit i would get only 2000 records, how do i get rest data which couildn't come beacuase of limit.
Good morning Reza, I have a sharepoint table with 2 columns, the first: ID_PARENT is numeric and the second PROJECT is a text and contains 8500 records. If I use the Filter(StartsWith(PROJECT,"PROJECT001")) function I get all the records of PROJECT001, if instead I use the Filter(StartsWith(ID_PARENT, 10)) function I get only the data that are in the first 2000 records. Maybe the problem is that you can't use StartsWith with numeric fields? Thank you.
Thanks for this great video. Very Helpful!! May we use Power Automate to bring more than 2000 items to Power Apps and keep them into a collection? Using Get Items action?
To avoid the delegation issue. I understand we should not work with large local collections but, sometimes, it can help when you need to use non delegable queries as CountRows or Distinct and the number of records are, e.g., between 1 and 10000 (no more).
@@AlbertoAllegueVarela I would not recommend doing that as it will have performance implications. Thats why one has to use better data sources like Dataverse that offers max flexibility in terms of delegation and more.
This is fine when using a Sharepoint list, but with SQL server, there's a limit too surely. Do you know what it is? I tried your Filter() method with my SQL data source and it brought back only 500 records - tops! No more records beyond my filter could be brought back, even though I know there's thousands of more records in sql. So how can we control delegation for sql server databases? What's best practice?
I have a really large list with about 90K records. Filter and StartsWith does not show any records. I don't see any delegation warning but no results either. To be precise, it is very errant in showing results. Sometimes I see results and sometimes I don't even when I know there are records that start with the term provided.
Depends upon the query you have framed. I have not come across any issues with lists > 500k records as well. I recommend posting your issue/query with screenshots on the forums at powerusers.microsoft.com
So it’s not a good idea to load sharpoint data in collections and use collections as data source? Should we connect to SharePoint data source directly if your records are more than 2k?
Consider I'm collecting a table from a backend custom connector API which has 10000 data rows in that case how can i optimise the gallery and show it. Please help reza. Is it possible we can achieve the same like when scrolling it loads?
So what power app will do if anyone is using a custom connector and retrieving data The power app itself is not suggesting that ? So we have to use sql, dataverse, SharePoint?
Hey Reza, can you pleaseeee let me know how to get the attachments through ms forms. Setting is disabled for external users, but i need it anyhow to put the documents as attachments in the saem list item created by form submitted. Please Help!!!
I have done videos on it. Ms forms anonymous users - there is no option for attachments. If setting is disabled - there is no other option that Im aware of.
Thanks for swift reply. But can you pleasee let me know any other options but ms forms to get the responses as well as attachments from external organisations into our sharepoint.
How to upload multiple files to dataverse record and save it to share point doc library.And send notification email to user in that particular dataverse record with multiple documents attached in the email
I do not have a video reference on this scenario and would have to try it out to provide guidance. I recommend checking or posting your query on forums in case someone has done something similar powerusers.microsoft.com
@@malikhan0123 Try it out and see. It will load as many as you need. I dont see any user doing that though as scrolling to get data to find is not ideal. Better to have good filters on screen.
Basically, you need to become very creative at the moment of writing the query of your choice. Which heavily relies on the data source used, being SharePoint one of the most common, and at the same time one of the most limited.
why does everyone show samples with a share point list instead of a MS SQL data source. on one in there right mind in a large organization is going to use a sharepoint list.
Because setting up a SQL data source requires resources + cost implications + licensing needs. Because everyone has access to SharePoint and is easy to get started with.
@@RezaDorrani On a text propert of a label, I write " CountRows(Filter(Temp_Item_Selection, Dealer_Code = "Cust0001")) " this code. And it shows delegation warning. Warning massage : The highlighted part of this formula might not work correctly on large data sets. The “CountRows” operation is not supported by this connector. We use sql server as connector
@@RezaDorrani I write a query like " CountRows(Filter(TestDB,VendId = "Vend001")) " on a text property of a label. But i found this warning massage. The highlighted part of this formula might not work correctly on large data sets. The “CountRows” operation is not supported by this connector. We use sql server as connector
Yet another awesome video Reza. Thank you so much!
You’re most welcome
The advice to stich together collections to load large data sets is probably the most shared (bad) advice in the community. Thank you for making such an excellent video that highlights this.
I couldn't agree more. What surprises me is that folks are using those techniques to avoid delegation.
@@RezaDorrani I think it is because delegation is sometimes explained as the maximum number of rows PowerApps is able to work with, which is not true at all!
A better explanation would be: It is the maximum number of rows it SHOULD work with, but LOCALLY and not server side.
The example where you scrolled to the second page of your dataset showed this so beautifully together with the row count. It showed that when you respect delegation, PowerApps will actually help you paginate your dataset and ensure you still get the correct result from your queries. I mean how nice is that?! You don't even have to do anything; Power Apps just handles it for you.
I stitched together collections in the beginning because I read bad advice and the worst thing about it, is that it kinda works; it performs badly, but it gets the job done.
It was only later until I learned about delegation from gurus like you and Shane, that I understood my approach was severely flawed, and that by understanding delegation I realized that it makes my life much easier.
I simply didn't understand that PowerApps could filter 1.000.000 rows server side trough delegable queries, but still be limited to only show 500/2000 rows locally. But it makes sense when you think about as you said, that the whole system is in place to urge you to minimize the amount of traffic between the app and the server.
I still have many apps where the delegation is set to 2000 rows, but when you think about, how often do you really need to present more than 500 records to the user at the same time? Probably never. And how many apps have tables with more than 500 records? Probably all of them.
Delegation works brilliantly when you understand it, and that's why I wanted to show my appreciation for your work Reza, because your video can improve the whole Power Apps community in a place where it is most needed.
That testing tip for setting return query to "1" row was awesome!
👍
Made multiple PowerApps 🎉 Super thankful to your videos Reza! More power to you 🎉🎉
Fantastic!
A really good refresher course on this all important topic. Hopefully, a few more functions will become delegable soon. Thanks for sharing. Appreciated.
Glad it was helpful!
Awesome video Reza. In regards to collection, one of my client keeps insisting that we load 6k to 8k into a collection and I have to use concurrent function. Takes about 30 secs and the app's performance is slightly worse, not App breaking but not as nice at all.
Loading > 2k records is not advisable. Performance will take a hit.
Thanks for this great overview of Delegation!
Most welcome
if the function is non delegable then the intellisense should be showing the delegation warning,..... surprised to know the issue with ' with ' function ... thanks for this amazing video sir............
I agree that the warning must be shown. I will take that feedback to the team.
"With" not having a warning is expected in the example shown at ~10:40, it would be the same as if you did
1) ClearCollect(colStudentData, Students),
2) Filter(colStudentData, Region.Value "South") (no delegation)
@@MuFeR. I agree. Delegation warnings should only be for functions that is expected to work in the data source. With is a purely Power Apps function that can not be delegated. So a warning is misleading.
Dude you are amazing. I learn best from examples like you show. Thank you so much.
The Superbowl is on right now and I'm learning from your video. I'm learning here to
advance my personal power, otherwise I would be wasting time. Thanks :)
BTW what version are your currently using?
Thanks so much!
Enjoy the super bowl.
version of?
Thank you @Reza, for another excellent and informative video . Tons of thanks. 👍
Thank you for this very good explanation! 🙂
Most welcome!
1) Don't trust the warnings for delegation (either that they do or don't show as delegable)
2) Delegation ONLY becomes an issue for your apps if the dataset gets large (>500 or >2000 once you change the defaults).
3) You can test to see if you have some delegation issues by setting an abnormally low query limit for your app (down from 500 to a smaller #)
4) All Delegation issues can be solved through better queries, better data organization, using Power Automate & ParseJSON, etc. (which all mean Dataverse isn't required to solve delegation issues in most scenarios)
5) Microsoft is likely to push better delegation features within Dataverse first (but that doesn't mean you should use Dataverse unless your app requires it or your organization is focused on supporting Dataverse more widely)
Thanks for sharing
informative and well explained with example...
Thanks
It's been great Reza sir if you post on your experienced basis,scenario based interview questions its been really helpful for us
You are Gem 💎💎 for us
Regards
That is not an area I am well aware of.
Reza, thank you for the video. I built an app connected to a SQL server that provides data that changes minute by minute. What is the best way to provide a count of this data based on different criteria like total number of calls, high priority calls, and pending calls? I do get the delegation warnings but the number of records displayed at one time is around 200 or less. This app also provides the users the ability to do a history check based on either an address or phone number. Is there a way to ensure that the users are getting the full list of records if this search does exceed the delegation limit?
Count functions are non delegable with sql.
You could call stored procedures to get counts.
I have no video references on sql to help direct you to. I will recommend checking or posting your query on forums in case someone has done something similar powerusers.microsoft.com
Great video as usual, very informative!
Very well explained Reza sir! Thanks for creating this video. I have seen Power Apps slowing down with 1000 rows in SP list. What is the best or recommended source/backend for any Power App?
Thanks!
There is no single answer to best data source. Answer is "it depends". I deep dived into this concept during the LIVE training workshops.
@@RezaDorrani Okay..thanks
Well explained Reza. This really helps.
Glad to hear that
Great Video as usual. Thanks for the Info
Thank you for a refresher!!!
Welcome!!!
Thank you another amazing video, Reza
My pleasure!
Thank you for this useful reminder
Always welcome
Hi Reza,
Thanks for this video. Really Helpful. Quick Question:
I am working with around 0.5 Million rows SQL table. On this table we need to perform a group by operation and create landing page and for slicers we need to perform a distinct on column and get all different values from that column. The challenge here now is Group by & Distinct are non delegable Power Fx functions when connected to a SQL connector. There is no way that I can found to do this with other delegable functions.
How do we approach this problem now?
Create sql views
Table shaping functions are not delegable.
Sql views may be an option
@@adarshmc8158 @RezaDorrani - Thanks for the idea, I can create a sql view for group by post which also I will end up with a table greater than 2000 rows, On this table if I am using distinct its still an issue as it picks up the values from only first 2000 rows and I end up not having all unique values from my column. Anny idea on how do I tackle this after creating Views. Thanks in Advance
hi reza, thanks for the great video!! i do have a question is about, i heard there's actually a 5000 items limitation for sharepoint.. i'm just worry about, if my sharepoint list has more than 10,000 items, when i using filter() function, will the data be impacted by that 5000 sharepoint limitation even filter is a delegate function?
SharePoint list can store upto 20M records. The 5000 is the list view limit. Check documentation on index columns for SharePoint
@@RezaDorrani noted with great thanks reza!
Great video as always! I saw in documentation the 'in' Fonction is delegable in dataverse but I try to filter a gallery on a multi choices column and get delegation warning. Does a workaround exists? I try addcolumn(concat to filter on string but performance is not great because the app needs to collect the whole table.
in is delegable for certain data types not all. Check documentation.
Hello Reza, It's again a nice video. I have question regarding delegation. I set delegation limit to 2000 (that is max limit) in this case if my search result crosses the limit i would get only 2000 records, how do i get rest data which couildn't come beacuase of limit.
Welcome!
Thats the limitation if your query is not delegable. Better not to use search function and use StartsWith in that scenario.
Good morning Reza,
I have a sharepoint table with 2 columns, the first: ID_PARENT is numeric and the second PROJECT is a text and contains 8500 records.
If I use the Filter(StartsWith(PROJECT,"PROJECT001")) function I get all the records of PROJECT001, if instead I use the Filter(StartsWith(ID_PARENT, 10)) function I get only the data that are in the first 2000 records. Maybe the problem is that you can't use StartsWith with numeric fields? Thank you.
you can't use StartsWith with numeric fields
@@RezaDorrani Thank you.
Amazing tips as always ❤❤ Thanks Reza
Glad you like them!
Great videos! Do you know how to turn the Comments off when using the Edit mode in PowerApps?
I am not aware as I have not come across anyone wanting to turn it off. Check documentation and see if there is something provided for it.
Thanks for this great video. Very Helpful!! May we use Power Automate to bring more than 2000 items to Power Apps and keep them into a collection? Using Get Items action?
Welcome!
Delegation over anything. Why do you need to load 2000 records in collection via flow?
To avoid the delegation issue. I understand we should not work with large local collections but, sometimes, it can help when you need to use non delegable queries as CountRows or Distinct and the number of records are, e.g., between 1 and 10000 (no more).
@@AlbertoAllegueVarela I would not recommend doing that as it will have performance implications. Thats why one has to use better data sources like Dataverse that offers max flexibility in terms of delegation and more.
@@RezaDorrani How can we load more than 2k record in our gallery if I am using datasource is Dataverse .
This is fine when using a Sharepoint list, but with SQL server, there's a limit too surely. Do you know what it is? I tried your Filter() method with my SQL data source and it brought back only 500 records - tops! No more records beyond my filter could be brought back, even though I know there's thousands of more records in sql. So how can we control delegation for sql server databases? What's best practice?
I will recommend checking the documentation for delegation to see SQL limits.
I have a really large list with about 90K records. Filter and StartsWith does not show any records. I don't see any delegation warning but no results either. To be precise, it is very errant in showing results. Sometimes I see results and sometimes I don't even when I know there are records that start with the term provided.
Depends upon the query you have framed. I have not come across any issues with lists > 500k records as well. I recommend posting your issue/query with screenshots on the forums at powerusers.microsoft.com
So it’s not a good idea to load sharpoint data in collections and use collections as data source? Should we connect to SharePoint data source directly if your records are more than 2k?
Collection is not a data source. It is only a space to temporarily load data for the users session. Best to connect directly to data sources.
If somehow we solve delegation . Powerapps can we used to make enterprise level application
Delegation is not a problem. It is an inbuilt feature that's a benefit to app performance.
Power Apps is used to make enterprise level applications.
Consider I'm collecting a table from a backend custom connector API which has 10000 data rows in that case how can i optimise the gallery and show it. Please help reza. Is it possible we can achieve the same like when scrolling it loads?
If everything is in a collection, then there is little scope for optimization as you have already loaded data in memory in the power app.
@@RezaDorrani I couldn't understand 😢
What if i had 1M data then loading it in collection is not a best practice
So what power app will do if anyone is using a custom connector and retrieving data
The power app itself is not suggesting that ?
So we have to use sql, dataverse, SharePoint?
Please reply when you are seeing it reza😊
I'm a fresher working in a startup 😢
Love from India
1:13 Very Good
Hi Reza, If I want to make an app that can connect Dataverse and Web Apps, Can we make a connection between them?
Whats web apps?
Power apps can connect to multiple data sources.
@@RezaDorrani Can we input data from a web (HTML, CSS, JavaScript) to a dataverse?
@@annisamegarestya8800 Not that I am aware of
Hey Reza, can you pleaseeee let me know how to get the attachments through ms forms. Setting is disabled for external users, but i need it anyhow to put the documents as attachments in the saem list item created by form submitted. Please Help!!!
I have done videos on it.
Ms forms anonymous users - there is no option for attachments.
If setting is disabled - there is no other option that Im aware of.
Thanks for swift reply. But can you pleasee let me know any other options but ms forms to get the responses as well as attachments from external organisations into our sharepoint.
@@StoneCold9898 Im not aware of other options.
VITAL! Thanks Reza!
Most welcome!
Thanks!
Welcome!
I have a SQL (Synapse) datasource with about 60,000 records. What's the best way to work with this?
I have no clue about that
How to upload multiple files to dataverse record and save it to share point doc library.And send notification email to user in that particular dataverse record with multiple documents attached in the email
I do not have a video reference on this scenario and would have to try it out to provide guidance. I recommend checking or posting your query on forums in case someone has done something similar powerusers.microsoft.com
CAN we show more than 2k line items in gallery, gallery is delegable or not
Thats what the video explains. If query is delegable and you keep scrolling down, gallery will keep loading all the data.
@@RezaDorrani i mean can it load after 2k line items..it would be good if you shown that in video
@@malikhan0123 Try it out and see. It will load as many as you need.
I dont see any user doing that though as scrolling to get data to find is not ideal. Better to have good filters on screen.
Basically, you need to become very creative at the moment of writing the query of your choice. Which heavily relies on the data source used, being SharePoint one of the most common, and at the same time one of the most limited.
Absolutely!
From a delegation flexibility standpoint:
Dataverse > SQL > SharePoint
why does everyone show samples with a share point list instead of a MS SQL data source. on one in there right mind in a large organization is going to use a sharepoint list.
Because setting up a SQL data source requires resources + cost implications + licensing needs.
Because everyone has access to SharePoint and is easy to get started with.
how to handle countrows function under a filter query?
like:
CountRows(Filter(datasource,logic)) > 10
Countrows is not a delegate function unless you use dataverse (limits are documented)
@@RezaDorrani On a text propert of a label, I write " CountRows(Filter(Temp_Item_Selection, Dealer_Code = "Cust0001")) " this code.
And it shows delegation warning.
Warning massage : The highlighted part of this formula might not work correctly on large data sets. The “CountRows” operation is not supported by this connector.
We use sql server as connector
@@RezaDorrani I write a query like " CountRows(Filter(TestDB,VendId = "Vend001")) " on a text property of a label.
But i found this warning massage.
The highlighted part of this formula might not work correctly on large data sets. The “CountRows” operation is not supported by this connector.
We use sql server as connector
@@NimbusBI CountRows is not delegable with sql either. Check documentation to see what’s supported for delegation for your data source.
Hi Reza, can you please help me to load more than 6000 records in powerapps
If you keep scrolling in gallery it will load as many as you need
how to retrieve more than 2000 item in gallery without delegation warning?
That’s the whole idea of delegation. It loads more than 2000 items as the user scrolls down. One should NOT be loading all that data in one go.
V। ❤❤❤❤❤😂😂
?
Ko
?
😢😢😢😢😢
Good or bad?