One of the first biases a data scientist can introduce into a data project is model selection.  Therefore it has become standard to evaluate data using a variety of modeling approaches.  A recent HBR article, Why “Many-Model Thinkers” Make Better Decisions by Scott E. Page, explores the benefits of using many models to evaluate data. 

Intuitively it makes sense that to get the full picture, a different angle can be more insightful than increased focus.  There are many ways to improve a single model to the extent that data analysts may be tempted to over-fit, designing models that are so specific that they tend to perform poorly when put into use. 

In order to apply different models, the question asked of the data may need to change.  For example, credit card data is frequently used to forecast quarterly revenues for a company.  In addition to making a numerical forecast, a different model can be applied to evaluate whether a company is likely to beat or miss consensus revenue estimates.  Yet another model can be used to generate a probability curve for a range revenue estimates, perhaps validating an analyst’s financial model.

Not only does a multiple model approach reduce bias, it also provides a level of confidence in predictions.  Data which is more uniform will tend to provide similar results when analyzed using a variety of techniques. The opposite is generally true of data which is more quirky.  Multiple approaches to data examination may bring attention to data characteristics that should be taken into consideration, perhaps by building yet another model.  The increased understanding that comes from applying multiple models to a dataset is obviously missed when analysis is not done internally.  When an asset management firm purchases a prediction from a third-party research provider, they have no idea to what extent that data has been processed and analyzed.  An exception is a firm such YipitData, which provides predictions but also provides access to the underlying data and consultations with analysts who have worked with the data. Building internal data analytics capabilities may be difficult, but it is part of the fiduciary responsibility that managers are entrusted with.