In his blog post entitled It’s About Time to Rethink Forecasting, Greg Girard, Program Director of Merchandise Strategies at IDC, discusses the future of retail forecasting in the context of cloud, Big Data, social business, and mobile technologies. In contrast to what’s now possible thanks to recent technological advances, he points out that retailers still largely rely on outmoded forecasting tools that were the result of the technological limitations of yesteryear.

Retailers have made due with time series forecasting tools that came of age when technology was expensive and difficult to use, data were scarce and latent, and growth and profitability were not as dependent as now on fast and accurate “granular” forecasts. Under these circumstances, advances in statistical science and tools were slow. Precision and flexibility were compromised to suit technology limitations. 

Greg’s primary argument is that cheap computing resources and high-performance analytics software are enabling accelerating innovation in predictive and optimization analytics, and this is bolstered by the advent of cognitive systems and machine learning. At the same time, the dramatic rise of omni-channel is necessitating a move beyond the limited time-series forecasting approaches broadly in use today. Along with many more “degrees of freedom” for all parties involved, omni-channel means a whole new level of complexity for retailers.

A new approach that’s gaining traction is structured data creation through digital experimentation. Rather than attempting to mine large volumes of data for signal as with traditional forecasting techniques, this approach is based on “manufacturing” task-specific signal through deliberate, controlled experimentation:

• 1. High-precision experiments enabled by the granular targeting capabilities of online
  2. Ability to methodically look for signal in particular areas of keen interest
  3. Cost-effective means to execute testing at scale, allowing for very tight confidence intervals

In contrast to data mining, digital experimentation of this sort allows for very high levels of signal to noise.

This is largely made possible by the recent explosive growth of digital shopper engagement, including e-commerce, digital coupons, social media, and mobile. Shoppers now regularly frequent social websites, purchase products online, print digital coupons, and use mobile apps before and even during their shopping trips. It should come as no surprise that – according to research by Deloitte – 64% of in-store purchases are now influenced by digital.

Another set of enabling factors centers around advances in multivariate testing techniques. This includes more efficient experimentation through fractionated experimental designs and intelligent algorithms to help guide the design and analytic decision-making process. With the help of machine learning, it’s possible to combine evolutionary and reinforcement learning methods that allow for faster realization of optimal promotional creatives, while understanding parameter effects and minimizing the cost of experimentation. This is done by continually “pruning” the test set, and converging on a smaller, optimal set of marketing creatives.

Taken together, this presents a real opportunity for traditional brick-and-mortar retail to garner a new competitive advantage as omni-channel continues to transform the landscape. With the growth in the volume and complexity of data, the need for structured, targeted experimentation will only increase. At the same time, traditional retail experimentation has historically been hampered by the need to do expensive, burdensome in-store pilots. These recent advances in digital and experimentation design allow brands and retailers to quickly and methodically design more effective in-store promotions by testing thousands of different offer ideas with real shoppers online – something only e-commerce players could do in the past.

It’s not only time to rethink forecasting – it’s time to think beyond forecasting!

To learn more, visit What is Offer Innovation.