Thanks, solid rant. Data issues can only get worse as the information age ages. Older systems, with each new decade a new generation discovering that data entry issues go back ever further in their data sets.
We must work at the same place! 🤣. I spend a lot of time gathering and cleaning data as well and I catch grieve because it takes so long. One of my important data streams is an SQL server table with table and variable names in German. 🤬. I often have to read in spreadsheets people use to log data - crazy headers, joined cells, typos, don't get me started! Good stuff Mark!
Thanks, solid rant. Data issues can only get worse as the information age ages. Older systems, with each new decade a new generation discovering that data entry issues go back ever further in their data sets.
That would be awesome to look at a case analysis of what kinds of questions can be answered as well as applied.
We must work at the same place! 🤣. I spend a lot of time gathering and cleaning data as well and I catch grieve because it takes so long. One of my important data streams is an SQL server table with table and variable names in German. 🤬. I often have to read in spreadsheets people use to log data - crazy headers, joined cells, typos, don't get me started! Good stuff Mark!
I love it when they change the names of columns without letting downstream know. Ha.
Going through this very situation right now. Perfectly stated.
Never ending.
Very well said Mark! It takes a lot of time to clean data.