Big data projects face heightened risk of failure


11 October 2015

Analytics projects that utilise big data or advanced analytics are increasingly popular but present a heightened risk of failure, according to Gartner. The IT research group predicts that, by 2018, 50 per cent of business ethics violations will occur through improper use of big data analytics.

"Although big data and advanced analytics projects risk many of the same pitfalls as traditional projects, in most cases, these risks are accentuated due to the volume and variety of data, or the sophistication of advanced analytics capabilities," said Alexander Linden, research director at Gartner. "Most pitfalls will not result in an obvious technical or analytic failure. Rather they will result in a failure to deliver business value."

According to Gartner, failure to properly understand and mitigate the risks can have a number of unintended and highly impactful consequences. Those can include loss of reputation, limitations in business operations, losing out to competitors, inefficient or wasted use of resources, and even legal sanctions.

The research group has recommended key best practices which will help analytics leaders to improve the likelihood of success:

Linking analytics to business outcomes through benefits mapping

According to Gartner, linking analytic outputs to traceable outcomes using a formal benefits-management and mapping process can help the analytics team navigate the complexities of the business environment, and keep analytic efforts both relevant and justifiable.

Investing in advanced analytics with caution

Gartner said it is the complexity of the analytical question to be addressed that drives the need for advanced analytic tools, and in many cases desired outcomes can be achieved without resorting to more sophisticated analysis.

Balancing analytic insight with the ability of the organisation to make use of the analysis

Because analytics can only be beneficial in organisations that are willing to embrace change, Gartner said it makes sense to limit investment in analytics to a level that matches the organisation's ability to use the resulting insights. Analytics, it said, may not be the most suitable approach

  • if pertinent data is absent,
  • when there are high levels of ambiguity,
  • where there are entrenched opposing points of view, and
  • in highly innovative or novel scenarios.

In these cases, Gartner said scenario planning, options-based strategies, and critical thinking should also be incorporated into analytical approaches to better support the organisation's ability to take action.

Prioritising incremental improvements over business transformation

According to the research firm, using big data and advanced analytics to improve existing analyses, or to incrementally update and extend an existing business process, is easier than using them to deliver business transformation, because there are fewer dependencies to overcome to ensure success.

However, Gartner warned that in some cases, deep reform of the business strategy may still be necessary — for instance, when a new disruptive vendor enters a market, when technology innovation changes the business model, or when an organisation has become dysfunctional.

Considering alternative approaches to reaching the same goal

Big data and advanced analytics help validate proposed hypotheses and open an even wider range of potential approaches to addressing corporate priorities. Investment, Gartner suggested, may be better targeted on human factors, re-education or reframing the problem.

More detailed analysis is available in Gartner's report "Seven Best Practices for Your Big Data Analytics Projects."