The energy system is changing, with an increasing number of distributed energy resources and changes in usage and demand. To handle these evolving complexities and support the transition to Net Zero it will be necessary to harness data and digital technologies to manage the system most effectively.

Energy Systems Catapult partnered with National Grid Electricity Distribution (NGED) (formerly Western Power Distribution) on the Presumed Open Data challenge: a series of data science challenges to engage the energy community, demonstrate the value of open data, encourage open science, and solve some of the key problems facing distribution network operators.

The Background

Energy Systems Catapult led both the Energy Data Taskforce  and subsequent Energy Digitalisation Taskforce, commissioned by the UK Government, Ofgem and Innovate UK to develop an integrated data and digital strategy to helps unlock the opportunities of a modern, decarbonised and decentralised energy system for the benefit of consumers.

Described as “game-changing” by the Energy Networks Association, the taskforces engage with hundreds of organisations and kick-started over 50 digital and data projects within two years of beginning, including the Presumed Open Data project.

The Challenge

A lack of open data, especially within the distribution network system has limited the speed of advancements within the energy data science research community.

Robust and scientific advancements in machine learning and algorithms are highly dependent on data being shared and for reproducibility and comparisons.

Currently experimental results are restricted to single experiments from project teams using non-shareable data, or to experiments on some of the few open data sets which are available. In the latter, these datasets are often overused, and the results are no longer relevant and biased due to overfamiliarity.

Further to this, most of these projects and their outputs occur in silos within academia with little comparison to common benchmarks or industrial approaches.

Data science competitions help to open science up to the wider community and help develop the benchmarks that can drive innovation.

The Solution

Energy Systems Catapults worked with NGED to set up a series of data science challenges to help define some of the problems facing  current and future network operators and show what could be achieved by openly sharing high value datasets.

The project aimed to improve internal data processes, release more data, increase engagement with newly published data, identify some key challenges that could be solved with data and engaged the innovator community through data challenge competitions.

Framing these challenges within a competition environment, combined with sharing the necessary and relevant data to solve them, provided an opportunity to crowdsource a diversity of knowledge and expertise from a passionate community of data science enthusiasts, and help develop knowledge and benchmarks which can be the foundation for future innovations and support further research.

The Presumed Open Data challenges demonstrated the value that network data can bring to innovators and the wider community, including:

• Highlighting many data driven problems being faced by network operators.
• Releasing unique and valuable datasets to the research community.
• Demonstrating state-of-the-art methods and techniques via our wrap-up webinars.
• Creating new outputs including peer-reviewed academic publications and open access code for advanced algorithms.
• Producing a growing list of energy data science enthusiasts and a network of skilled individuals who can help drive the digitalisation of the energy system.

The Outcomes

Outcomes of the Presumed Open Data challenge included:

• 365 Registrations for the kick-off event with individuals come from 72 different organisations/institutions, 15 different countries and 30 universities
• 55 teams –  a total of 142 individuals – participated in a least one round
• By the 5th round of tasks, 37 teams survived and and of those 65% outscored the benchmark with the Top 5 teams participating in a special “Value in Energy Data Seminar”
• A number participants publishing their work in peer reviewed academic journals and highlighting our data challenge approach as a useful way to link academic talent with industry need.

The Impact 

Impact of the Presumed Open Data challenge included:

• New advanced solutions and benchmarks for major problems facing distribution network operators
• Strengthening industrial and academic links
• Better cross-party working between industry, academia, innovators and regulators
• Increased data insights and datasets that can inform future energy system design
• Over 2000 views and 1000 downloads of the released datasets
• Over 750 views of our kick-off events
• Hundreds of global experts engaged across at least 72 organisations. 15 countries and 30 universities
• Over 370 members of our data science LinkedIn group.
• The launch of NGED’s Connected Data Portal, which now hosts almost 100 open datasets about National Grid’s network infrastructure and operations, including a number of live data sources.

Next steps 

After the Presumed Open Dat project, National Grid Electricity Distribution are now:

• Exploring new data challenges and considering new ways to tackle the biggest problems facing the energy sector more generally.
• Engaging the data science networks created through these challenges to understand data, modelling and coding needs.
• Bridging the gap between academia and industry helping to transfer the valuable research insights and help support the much needed data skills across the sector.
• Investigating the data ethics challenges that will inevitably be facing the energy community as more sensitive data becomes available and new services create new interventions for consumers.