Grasping the Data Opportunity in Utilities

Sebastian Fox, Sector Manager at The Future of Utilities talked to Greg Hanson, VP EMEA & LATAM at Informatica, about his key takeaways from the Future of Utilities and Informatica June 2020 roundtable on the role of data in a fast-changing energy and water market.
Greg Hanson, Informatica

Greg Hanson

Our June 2020 roundtable connected data analytics and digital experts from across energy and water companies for a lively discussion about data and its critical importance to driving transformation. There was common agreement among the participants that the industry now has access to more data than ever but that this bounty brings problems of its own.

“The sector is data rich but information poor,” said Greg Hanson of Informatica, summing up the mood of the industry. “The potential is huge but it’s currently going unrealised because so many organisations are struggling with issues around data quality.”

Those struggles have been well documented. According to one survey, 55 per cent of all data collected by companies is so-called “dark data” which they either have captured but can’t use or data they are not sure they have[1]. Studies suggest that many utilities companies are struggling with the quality and reliability of their existing data sets, which may include records that are inaccurate, incomplete or even analogue[2].  And across all sectors, data scientists and analysts waste 50 to 80 per cent of their time on so-called “data janitor” work, trying to find, collect and clean-up unruly data, while as many as eight out of ten AI and machine learning projects have stalled due to poor data quality and 96 per cent have run into problems with data quality and the data labelling required to train AI[3].

For energy suppliers, new models enabled by advanced data analytics also pose an existential threat: advances such as AI-powered auto-switching and blockchain-enabled energy trading among micro-grids mean there may come a point where increasing numbers of customers no longer need an energy supplier. In the nearer term, there’s also the ongoing competitive threat for utilities incumbents from digital-first brands that use data to generate innovative propositions that add real value to customers, not to mention the looming risk of platform businesses that could yet drive further margin-eroding disruption.

“There’s growing recognition that incumbents’ struggles with data need to be resolved if the industry is to capture real value from opportunities big data and advanced analytics present and, in so doing, respond effectively to a fast-changing world,” said Hanson.

[1] The State of Dark Data, Splunk

[2] The Digital Utility, McKinsey, 2018 “Most operators tell us their data is scarce, patchy, or not even digitized.”


Meeting rising customer expectations

All this comes at a time when legally binding Net Zero targets require huge capital spending programmes and customers have diminishing tolerance for sub-par customer experience. 

Indeed, one leading survey of utilities leaders found that four out of five think that within ten years utilities that fail to match the level of personalisation and convenience that customers receive from Amazon, Google and Netflix will find it extremely difficult to satisfy customer demands, while 91 per cent believe it’s vital that energy and water companies provide value-added services in order to avoid being further commoditised[4].

Delivering those best-in-class customer experiences and identifying value-added services requires data. Increased granularity would allow utilities companies to gain a much more holistic view of their customers. They could identify important lifestyle moments – the arrival of a baby meaning more water usage, say, or recovering from chemotherapy requiring more heating – as well as early indicators of vulnerability, enabling earlier interventions. And with better understanding comes the ability to configure smarter tariffs and help customers take control of costs and emissions.

“Utilities companies have a privileged position of access to our homes and how we live our lives,” said Informatica’s Hanson. “Rather than this being a purely transactional relationship, there’s a chance to use this privilege for good, to get closer to customers and help them lead smarter, safer and more sustainable lives. Given the thin margins in the retail business, this is a chance to seek out higher margins – but it won’t work without smarter data strategies.”

[4] The Future of Utilities, 2019, Marketforce

Drowning in a data deluge

For many incumbents, the problem is one of too much data, much of it affected by data drift – changes to data format and semantics – over years of organisational change and most of it held in silos that make it hard to build a holistic view of the customer. Challenger utilities brands clearly have an advantage as they have started with a clean data slate and can capture, catalogue and manipulate data using common coding standards and a well-architected platform.

But even digital-first brands have data challenges. “Everyone has to deal with messy data but you should only do it once,” said Hanson. “Once it comes in, you deal with it, clean it and then you know you can trust your data.”

Importantly, solving the problems around poor data shouldn’t be seen as an IT problem. “This is a business problem and it takes a pan-organisation approach to fix it,” said Hanson.

A helping hand

Many incumbents, still struggling with the volume and quality of data, are reliant on third-party providers to help them make sense of their data.  This can make data seem less readily accessible but at the same time often brings fresh thinking.

“It’s important to remember that many utilities are, by nature, highly risk-averse and conservative,” Greg Hanson of Informatica said. “Sometimes a third-party provider brings in that cognitive diversity that can help prevent groupthink and stimulate the search for innovative solutions.”

Hanson also pointed out that there are many aspects of data strategy, such as finding, integrating, labelling and cataloguing data that can be commoditised. “This is not core to a utilities company,” he said. “Third-party providers that have developed AI, machine learning and NLP tools for data management and cataloguing which have had millions of man-years of effort built into them. Why would you do that yourself? Far better to focus on how that data can be used to make better decisions and improve outcomes for customers.”

A purposeful and multidisciplinary approach

Rather than being background noise, data increasingly needs to be front and centre of the innovation agenda. Whether developing smart home kit or a new app, it’s essential that right from the outset there’s due consideration to what data it should be capturing and how. “You don’t have to collect everything, just everything useful,” said Hanson. “It’s about having purpose in what you’re doing.”

Making those decisions requires a multi-disciplinary approach, the roundtable participants agreed. Indeed, Gartner predicts that 50 per cent of organisations will see increased collaboration between business and IT teams by 2022. According to the roundtable participants, the advanced mathematical modelling skills of the data scientist need to be augmented by other specialists, including data engineers and software development, as well as input from the business side to make sure the outputs are answering real business needs.  

Without this clear alignment to a business case, there’s a risk that data remains a murky, little-understood asset. When a project goes wrong, poor data often finds itself in the searchlight as the main culprit but when things go right, it’s hard to clearly identify that data was at the heart of the success.

“Organisations like to say that data is an asset but too many don’t understand its value,” said Hanson. “We need to start putting pounds and pence to the value of what we do and talk the language of the business.”

Translating data into business advantage – telling the story

A key role is that of the business translator, someone who can act as narrator so that everyone across the business can understand the value of the data and what the business needs. And this works both ways: business departments often think they need a fancy app or dashboard but it’s only when the IT people understand what they’re trying to achieve that the right solution can be found – and often it’s something very different.

“The easiest parts of the data journey to narrate are those that involve putting data to work in pursuit of cost-savings, typically as part of an automation drive or optimisation of asset maintenance,” said Hanson. “Yet the real upside lies in using data to drive customer lifetime value models, which can be used to generate insights that reduce churn, build engagement and increase margins.”

The good news is that technology now makes it easier than ever to democratise the once mysterious world of data and analytics. By getting the right data to the right people at the right time, it’s possible for utilities to configure compelling value propositions, transform customer experience and spot previously hidden opportunities.