What industry do you know that is primarily a business of decision making? This business doesn’t build products … it doesn’t ship hard goods … it doesn’t provide a service that is consumed by its clients. But it does provide a highly valuable service when called upon. And what business do you know that has arguably more historical data about past decisions than other industries?
Right, Insurance. What a perfect opportunity to “mine” that historical data and learn from it, so future decisions are better than past decisions.
Enter predictive analytics and predictive modeling.
My first exposure to predictive analytics was in 1981, though I didn’t know it at the time. I was taking Modern Algebra my senior year in college, studying vector spaces and wondering what the heck this stuff could possibly be used for in the real world. Little did I know at the time that I was studying the foundation for
Support Vector Machines (SVM), one of the many predictive modeling algorithms.
Fast-forward 20 years, and SVMs are one of the algorithms used in the suite of algorithms for the first predictive models I was associated with as a member of the Valen Technologies team. Fast-forward another 10 years, and everybody is doing predictive modeling.
And why not? It is a perfect science for the insurance business; a business where a multitude of decisions are made every day, at every level of the organization, where some turn out to be good decisions and some not so good, and any one bad decision could cost a company millions of dollars, or tens of millions of dollars.
So if we build models to help us make better decisions, everything will be better, right? Wrong.
What if the model we build works great on the test data but doesn’t work in real life? Worse yet, what if we don’t know it? Scary? It happens—and probably more often than people care to admit.
I don’t mean to scare you, but I do mean to warn you. The tools for building models have become so easy to use, we forget the additional factors needed to make them truly effective. Why, there is even an Excel plug-in that will build you a model. Mark Gorman, in a paper on predictive modeling, quoted an interviewee as saying, “With these new tools, my grandmother could build a GLM model.” But building models that work, well that’s a different story.
What are the keys to building models that work? There are a number of things, and the tool itself is but one of them—and it is not high on the critical scale. Obviously, data is vital, but the preparation of that data for the modeling process is extremely important.
The choice of modeling algorithms is important too. Some techniques are better suited to particular problems. Also, the rigor with which that modeling technique is applied is crucial. Simple things like how the data is split into training and test sets can make all the difference as to whether the model really works or simply appears to work. The education, talent, and experience of the modelers, and the rigor they apply to the process, are all really important keys to successful models.
Another key to success, and the subject of another blog, is organizational change management. The science works, but if people in the organization don’t accept it, then success will be limited (or worse).
Predictive analytics is all the rage, and I am a believer. But it drives me nuts when others trivialize predictive model development. It is science and art; and if your models work, you will be rewarded. If they don’t work, and you don’t know it, you will be making business decisions based on incorrect predictions—a truly destructive path.