Thanks for the rundown! Didn't see any reason to not partition either. Good to have it confirmed. Question though, when you look at the Google analytics demo data they have the drop down. Am I correct that this is an old way of how they used to group tables based on a table suffix? And that it's now simply being done 'under the hood'? Combining the schema vs having a schema per table.
Yes you're right Michel - that was the old way (basically a date-sharded table), and now partitioning can be based off of any column and is done 'under the hood' as you say without modifying the table suffix.
I'm new to bigquery and looking to see if it can work for my problem. I have been following you from the sheetcon conference. Could you help me in figuring out if bigquery can work for my case. I have lots of data that is in different google sheets for different projects. They all have the same structure in terms of column naming. My questions are.... 1. If i connect sheets to bigquery will it be uploading in realtime? 2. If someone deletes or updates data in the sheets will it also be deleted or updated in bigquery? 3. Can i be able to combine all the sheets into one table in bigquery so that i can visualize all of them from datastudio? Thanks for the awesome videos following from 🇰🇪 😎
Hey Paul, yes you can use BigQuery's native 'create table from google sheet' functionality, that will perform both #1 and 2 for you (it's basically a real-time view of whatever's in your Sheet). And you can use a 'view' query, that runs a UNION ALL on those tables you've set up, to roll them up into one table for Data Studio usage.
Hi! Thanks for the video. One question i have is, does it matter of we partition/cluster if we do select * from table? My point is that it only makes sense if we filter it by where clause or use incremental models, no?
thank you for the explanation. it was very clear.
also, I really like that shirt man!
Kindly make a vedio on practical guide example of a selection of partitioning table
Thanks for the rundown! Didn't see any reason to not partition either. Good to have it confirmed.
Question though, when you look at the Google analytics demo data they have the drop down. Am I correct that this is an old way of how they used to group tables based on a table suffix? And that it's now simply being done 'under the hood'? Combining the schema vs having a schema per table.
Yes you're right Michel - that was the old way (basically a date-sharded table), and now partitioning can be based off of any column and is done 'under the hood' as you say without modifying the table suffix.
I'm new to bigquery and looking to see if it can work for my problem. I have been following you from the sheetcon conference. Could you help me in figuring out if bigquery can work for my case.
I have lots of data that is in different google sheets for different projects. They all have the same structure in terms of column naming. My questions are....
1. If i connect sheets to bigquery will it be uploading in realtime?
2. If someone deletes or updates data in the sheets will it also be deleted or updated in bigquery?
3. Can i be able to combine all the sheets into one table in bigquery so that i can visualize all of them from datastudio?
Thanks for the awesome videos following from 🇰🇪 😎
Hey Paul, yes you can use BigQuery's native 'create table from google sheet' functionality, that will perform both #1 and 2 for you (it's basically a real-time view of whatever's in your Sheet).
And you can use a 'view' query, that runs a UNION ALL on those tables you've set up, to roll them up into one table for Data Studio usage.
Hi! Thanks for the video.
One question i have is, does it matter of we partition/cluster if we do select * from table?
My point is that it only makes sense if we filter it by where clause or use incremental models, no?
Hi, why 10 years of data cant deal with daily partitioning?
isn't it just 3650-ish partition?
BigQuery has limits on the number of partitions on a table (to my understanding)