Is Data Being Used Effectively?
COVID-19 has deeply impacted the way we work, learn, teach, parent, play, connect … the way we live. Although this is not breaking news, we are all going through it and it can be sad, costly, difficult, rewarding, confusing… insert emotion. While reflecting on the impacts of the pandemic on our daily lives, I started to think (can’t help it) about the impacts of these changes on the data that companies rely on to personalize our experiences, plan inventory or services, support their customers; basically interact with the broader ‘us.’
A Realization as Data Transforms:
Data is simply a collection of recorded dots generated by our interactions with the physical and digital world around us. An analyst can then connect the dots and see patterns, behaviors, ‘signals’ contained within the data. These actionable data insights are used to inform everything from how much of an item to place in a warehouse to what ad we will see next time we are browsing cat videos on Youtube (or so I’ve heard from a friend who enjoys cat videos).
Now imagine, all of the signals change at the same time.
We all stay at home for months. Instead of buying from the store down the road, we shop from Amazon or a retailer who was quick enough to improve on their in-store pick up process. We no longer commute but now work from home and share Zoom meeting memes. Our kids are home and we balance it all while the world changes around us. Our behaviors are deeply impacted and the result is that past data no longer indicates what the future might bring.
Change Happens All the Time
Yes, change is constant. But the previous changes that we have experienced since we began collecting data at scale share a similarity. The change at play had enough time to interact with the world as it was and created actionable data insights to match. These interactions shifted the data we collected, providing us with a gradual view of the changes in the signal presented within our data. Even with the financial crisis of 2008, considered to be a black swan, there were signals that were apparent to some really smart people (queue “The Big Short” – a really great movie).
Think about a company that introduces a new process, such as a new loyalty program. Their customers continue to interact with the company and gradually sign up for the program. Over time, behavior changes. The key is time and the interaction of the current and future states that get recorded in the data.
The COVID-19 pandemic represents an external change that business has no data connected to. One moment online revenues accounted for 20% of revenues, the next they were practically all of it. One day one-third of your products originated in China, the next China was closed for business. The amount of time was short, shifts were dramatic, and there was no interaction between current and immediate future state. Just a stop and start.
Wait, Everything is Going Back to Normal
While the known is comfortable, we have embarked into uncharted territory, and uncertainty has become the new norm. Chances are, behaviors will change. We do not know how yet and we cannot reliably predict it.
Organizations that fail to recognize and embrace the uncertainty will likely cease to exist. Some pre-COVID-19 behaviors will remain while new loyalties will emerge to companies that met customer needs in innovative ways during the pandemic. The notion of going to a place that is ‘packed’ may not be as appealing.
When discussing the shifts that occurred in the early days of quarantine, one of our customers described how they went from 5 percent to 97 percent of their employees working remotely within the span of a week. The expectation is that 50 percent will work remotely going forward.
There are constants and stable events that organizations will continue to experience. The key is to identify these stable events, optimize, automate where possible (read autopilot). The shifts will be new, sudden, and unexpected. An approach different from the traditional must be deployed to get our arms around new opportunities, understand them, and effect change.
Measure the Shifts and “Hello New Data”
So, now we have new dots and we can start analyzing the signals within. You may ask, “What about the old data?” I am not suggesting you do a hard delete on your historical data, but I am suggesting you start looking at it with a renewed level of skepticism. How can we run predictions when our data does not represent the future? How can we adjust our models when we don’t know what the future conditions might bring?
Shift statistics is a starting point. Simple shift statistics relay on a constant k with which we lift an entire distribution by that constant. Yet, the changes in behavior are not uniform and a shift of mean and median by k would fall short. Instead, we can deploy a more robust approach[i] that looks at two groups, across multiple points grounded on a specific behavior
Let’s call Group 1 a sample of our customers before COVID-19 and Group 2 the same folks, but after COVID-19. Mapping their behavior across simple deciles we can observe a shift. While the median may remain constant between pre and post COVID-19 their behavior has indeed changed.
Fig1. Pre and Post COVID-19 Behavior Score
Now that we see these changes, let’s figure out what changed. Consider a quantile distribution and analyze the difference between pre and post COVID-19 behavior scores.
Fig 2. Behavior score variance pre and post COVID-19
Fig 3. Shifts by quantile pre and post COVID-19
Armed with this information we can start to test variations of our predictions. What ad should we show? How much product to stock? What will our foot traffic be next Sunday?
While a good start, shift statistics may provide some insights into the changes, but what about new data? New data will be generated as business conditions change, new business models emerge and new companies are created. We have no data on these, hence we will need to deploy a series of adaptive models to compensate for the lack of history. Adaptive Machine Learning (AML)[i] models are deployed in a controlled explore and exploit manner, where multiple propositions are presented, and based on immediate feedback on success and failure rates the models continue to learn and adjust. None of this is new in business but it does open up discussions regarding explainability, visibility, new ways of collecting, and leveraging data that require broader discussion.
Data In Action
Don’t throw away your data. The previous observations will be useful in understanding the shifts your business is experiencing. Question models based on ‘old data’ that have not been adjusted based on shifts brought on by COVID-19. The constant events in your business are a prime opportunity for optimization.
Challenge traditional approaches. Explore new ways of accessing data. Implementing AML capabilities can futureproof your organization and inform your decision-making process. Intuition is key when accompanied by robust experimentation and a rebaselining approach. The way to deal with the uncertainty of times to come is by chipping away at the unknown, reorienting constantly, and adjusting as you go.
In the wake of the pandemic, how are you challenging traditional approaches and exploring your new data? Contact us to discuss how our experts can help you with incorporate your new and old data to gain insights and develop actionable intelligence.
[i] Bershad, N.J. & Ibnkahla, Mohamed & Castanie, Francis. (1997). Statistical analysis of a two-layer backpropagation algorithm used for modeling nonlinear memoryless channels: The single neuron case. Signal Processing, IEEE Transactions on. 45. 747 – 756. 10.1109/78.558493.
[i] Beyond differences in means: robust graphical methods to compare two groups in neuroscience Guillaume A. Rousselet, Cyril R. Pernet, Rand R. Wilcox bioRxiv 121079; doi: https://doi.org/10.1101/121079