Smart meter data is essential to the operation of the future smart energy system. It helps the energy industry observe patterns in consumer energy use, so it can plan ahead to ensure everyone has the energy they need. It can show us how consumers are responding to changing situations, like the cost-of-living crisis, or energy reduction initiatives like the Demand Flexibility Service experiments. But to truly understand consumer behaviours and needs, we need to observe not only what consumers are doing, but also why they are doing it.
At Energy Systems Catapult, we use our Living Lab to collect smart meter and sensor information and pair it with in-depth consumer insights to test new clean energy products, services, and policies. This article illustrates how our data scientists and user researchers collaborate to understand energy consumption patterns, such as daily and seasonal variations, and the impact of changing tariffs, by integrating both quantitative and qualitative data analysis.
Energy Systems Catapult’s Living Lab helps clean energy innovators test new products, services, and policies with real people, in real homes.
We have over 2,300 homes in the Lab. Many of our participants (‘Labbers’) share data with us about their energy usage. This includes energy consumption from smart meters, heating usage from smart heating controllers, and electric vehicle (EV) chargers. We also have an array of other sensors we can put in homes, to measure data points like temperature, humidity, and the energy consumption of individual appliances.
When a customer wants to test a new energy saving product or service in the Lab, we collect this sensor data, as well as usage data from the product. Our data scientists analyse this to understand what people are doing with the product, and how well it works.
But one of the core strengths of the Lab is our ability to combine this quantitative data with qualitative insights. Our user researchers speak to our Labbers to understand why they’re behaving as they are, and what it’s like to use the product.
To illustrate how we do this, we conducted a deep-dive analysis of a Labber’s energy consumption. We selected one of our Labbers, analysed her smart meter data, and created graphs to show her. We then interviewed her about those graphs, to understand what was really going on with her energy usage.
Through this, we identified some interesting patterns and behaviours, including the impact of changing tariff on energy usage in the home.
Our Labber, who we’ll call Anna, shares smart meter data with the Living Lab, which we collect at half-hourly intervals. (Anna’s smart meter currently sends us data at one minute intervals using a Consumer Access Device, but she only installed this fairly recently, so some of her historical data was half-hourly. So we aggregated all of her data to 30 min intervals for consistency. 30 minute intervals were enough for us to see the fluctuations and variability in the electricity consumed across a day).
We then cleaned the data in preparation for the analysis. This involved:
We used this data to generate several graphs showing Anna’s energy consumption. We spotted some interesting patterns in her data, and we had some hunches about what was causing those.
We invited Anna to an interview, to understand how her daily life shapes her energy usage.
Anna is a professional who lives with her partner in a detached house in Scotland with gas heating. She works from home most days. She owns a plug-in hybrid car, which she mainly uses at weekends, and is currently on an agile energy tariff. (This is a variable tariff where the price of energy can change half hourly, based on wholesale prices). Anna and her partner usually cook dinner around 6 pm each day. They try to use the washing machine and dishwasher at night, when their energy is cheaper. But they only recently discovered that the dishwasher is programmable.
We then showed Anna the graphs we’d generated,. We questioned her about the energy usage patterns they revealed, and tested our hunches.
We started by looking at Anna’s daily consumption, averaged across all days. There were some distinct peaks in usage. We figured that Anna was charging her car at night, and cooking breakfast and dinner around the same time each day.
Anna confirmed that the overnight surge in consumption during the night was caused by EV charging, and using the dishwasher and washing machine. There was another peak in the morning between 6:30 am and 9:30 am, which Anna confirmed was when she and her partner woke up and made breakfast. There was also a peak in the evening between 5:00 pm and 8:30 pm when they stopped working and cooked dinner.
“We probably start cooking at 5:30, latest at 6pm.”
On days they are home, there is a slight peak around lunchtime. Consumption never dropped to zero, even when they were both out. Like all homes, they always have some baseload consumption from the fridge/freezer and other minor power sources, such as the router.
We then looked at Anna’s average energy consumption per day. This graph provided us with more insights into Anna’s weekly habits. We could see that most significant energy peaks occurred on Sunday and Monday nights. Anna explained that this is when she usually charged her car.
Figure 1. Average electricity consumption across the day per week
We could also see that the morning breakfast peak is slightly later on weekends, although dinner time is around the same time every day. We spotted a slight reduction in energy consumption during the day on Saturdays, when Anna tells us her and her partner are often out.
We also created graphs that looked at Anna’s energy usage across the different seasons of the year. We noticed that Anna’s night-time consumption peak was higher during summer. It also occurred during a narrower time window than the equivalent peaks in winter and spring. But we weren’t sure why.
When we showed Anna this graph, she told us that she had switched her tariff from an EV tariff (with a fixed cheap period overnight) to an agile tariff, in July. The cheaper rate window varies daily with the Agile tariff, so she now manually adjusts the car’s charging accordingly. This means that the hours during which she might charge her car could start earlier or finish later when compared to the EV tariff. When she can, she also adjusts the start times of the washing machine and dishwasher to align with her agile tariff.
“We would match times to the tariff. So initially, that was like a fixed four-hour window, I think from two to six. And then it changed when we switched to Octopus, we would just we had a fixed period, I would just have it stopped when that period started. And then when we had the Agile we just check the evening before, and quite often, it would be sort of similar time anyway. But it might change by like, half an hour, one hour.”
We excluded autumn from the visualisation as the data aggregated consumption from September 2023, October 2022, and November 2022, corresponding to periods she was on two different tariffs. During October 2022, Anna’s energy consumption was also much higher, and she couldn’t remember why. To investigate this matter further, we would need to compare the data with that of other Labbers, or interview Anna’s partner.
Figure 2. Average electricity consumption across the day per TARIFF
After the interview, we generated a graph to illustrate the tariff changes Anna mentioned. This graph confirmed the variation in energy consumption associated with the tariff changes. The peak for the Agile tariff was shorter and higher than the other two EV tariffs she had been on previously. We observed that the peak for the agile tariff was shorter and higher compared to the other EV tariffs. For the two EV tariffs, Anna had a fixed cheapest tariff, for Bulb EV from 2 am to 6 am, and for Octopus EV from 0:30 am to 4 am.
We then examined Anna’s average consumption per month.
Figure 3. Average electricity consumption per month vs average of year.
Anna switched from Bulb to Octopus in April, at first going onto the EV tariff, and then changed to the agile tariff in July. We noticed a drop in consumption in July, which we first thought might have been related to the change in tariff. However, when we asked Anna about this, she told us that she was away with her partner in July, which explained why she used less energy. In August and September Anna used more energy. She told us that she had visitors staying at that time, and used her car more.
Here, speaking to Anna helped us understand the patterns we saw in the data. Without that context, we might have jumped to the wrong conclusions. With a large enough pool of homes, we can see how major interventions like changing tariffs affect people’s energy use. But more subtle patterns can be missed. And with smaller datasets, there is a greater risk of jumping to the wrong conclusions about the underlying causes of changing usage, if we don’t know the context.
From Anna’s smart meter data alone, we were able to infer quite a lot about her daily habits. But some of the patterns didn’t have a clear explanation, or led us to conclusions that weren’t quite right. By talking to Anna, we were able to understand what she was actually doing, and why. This shows how valuable it can be to combine quantitative and qualitative research for consumer energy data.
For example, we learnt how and why changing from a time of use tariff to an agile tariff impacted Anna’s energy usage, and how she felt about this. We can use these insights to try to identify similar patterns in other Labbers’ data which might indicate a change in tariff.
In the Lab, we also have other forms of sensor data, such as EV charging patterns, heating schedules, and temperature monitoring. Looking at these alongside smart meter data gives us great insight into what people are doing, and what outcomes they are getting. But speaking to people tells us what’s really going on in their lives. Put together, these are a really powerful combination.
For example, in a recent trial we looked at how people used electric heaters alongside their central heating system to keep warm in their homes. We monitored data from the heaters to understand when people used them, and the settings they used. We also used data from the home central heating systems to compare the times people used their main heating systems with the times they used the electric heaters. We also measured the impact the heater had on air quality and temperature in the home. Finally, we spoke to people to find out why they used the heaters in the way they did, what they were trying to achieve and whether they got what they wanted. This allowed us to recommend ways to improve the heaters, and propose new design features that might meet users’ needs better.
We’re also currently using this approach in our project with equiwatt. equiwatt provide a service to help balance the grid by rewarding consumers for reducing their energy consumption during special energy saving events at peak times. By observing participants’ smart meter data, we can see what impact equiwatt’s energy saving events are having on balancing the grid. But we’re also speaking to participants about the events they do and don’t take part in to understand why. What is preventing people from taking part when they don’t? How could we make it easier or more attractive for people to participate more?
Combining multiple sources of digital data with contextual data from the people behind the data is one of the great strengths of the Living Lab. We can look beyond just abstract patterns in the data – to what is really going on and why, so that we can help design a truly human-centred energy system fit for the future.
We take our participants’ privacy very seriously. Before we conducted this piece of analysis, we got permission from Anna. But it is worth noting that just as we can identify behaviour patterns such as car charging routines, other behaviours, perhaps those associated with protected characteristics, might be visible from this kind of data. For example, observation of certain religious practices.
We never share data in a form that would enable individuals to be identified. The only exception is where we explicitly ask participants for permission to share their data with a partner organisation or for a specific project.
Quick, safe and affordable. Design, market-test and launch innovative products, services and business models for Net Zero with real people in over 2,000 connected homes.
Find out moreFind out more about how Energy Systems Catapult can help you and your teams
Find out more about how Energy Systems Catapult can help you and your teams