Overview
Energy Queensland is the group of electricity distribution, retail, and energy services businesses owned by the state of Queensland. They deliver electricity across Queensland through their ‘poles and wires’ businesses Energex and Ergon Energy Network. Their retailer, Ergon Energy Retail, sells this electricity to customers throughout regional Queensland.
These essential service activities are also supported by a range of innovative products and services delivered through their Yurika brand.
Energy Queensland Limited is a 100% Government Owned Corporation, reporting to the Queensland Government. Their shareholding Ministers are the Treasurer and Minister for Investment and the Minister for Energy, Renewables and Hydrogen and Minister for Public Works and Procurement on behalf of the Queensland community.
The Challenge
Energy Queensland‘s Corporate Business Plan outlines a goal to embed a ‘Big Data’ capability to address immediate needs in forecasting, enterprise analytics and pricing and grow into an organic enterprise platform that transforms the way that Energy Queensland uses data in their business.
After a successful 4.5 week Proof of Value phase with Ignite, the team at Energy Queensland moved confidently to implementation. Timing meant that the system had to support 30- and 15-minute interval data along with new 5-minute interval data coming through the market from 1st of October 2021.
The Solution
The implementation of Data Lake and Data Science capability, leveraging AWS and Databricks, had three key deliverables:
- A Databricks data science platform that aligned with Energy Queensland’s AWS Cloud project, in particular, cloud security considering Energy Queensland’s physical asset base.
- Energy Queensland’s Meter Data Analytics Solution (MDAS) was designed to support 5-minute intervals. Changes in project timing meant that the system went live and immediately supported 5-minute intervals without skipping a beat!
- Operations upskilling to ensure Energy Queensland team could confidently support the new capability and adopt new ways of working.
Build data products to realise repeatable and sustainable business value from data.
Ignite implement an integrated data platform that enables the core building blocks for your use cases. Our Data Platform Build Service delivers:
- Implementation of your systems, components, and environments design
- Integration with your other core applications
- Security design including role design and AD integration
- Data modelling, and development, standards and guidelines
- Core ingestion and data transformation services
- Code and release management
- Test strategy, test scripts and documentation
- Deployed interfaces and base data models for critical datasets
- As-built solution documentation and component configuration
- Ongoing operations guide and operations team upskilling
- People change management plan
“It worked from day one. The platform has really enabled our team to expand their thinking and their capabilities.”
The Results
The Strategic Forecasting Team now utilise MDAS to inform network investment decision-making processes. They have several use cases to predict gross solar PV across the network, feeding into minimum and maximum demand forecasting to plan, monitor and maintain Energy Queensland’s network. Now the team can identify areas of the network that may be impacted before it happens.
Energy Queensland’s Energy and Demand Management team are using the solution to improve their strategic planning of their distribution network. Electric vehicles are increasing in Queensland and Energy Queensland is making plans to support these customers. The MDAS enables machine learning models to identify where EV charging is occurring at greater granularity through the network so that Energy Queensland is ready to serve electric vehicle owners when and where they need it.
- Access years of interval meter data along with associated master data for NMIs, tariffs and regions.
- Detect and interrogate trends.
- Fully load and process NEM12 files in accordance with regulatory requirements.
- Generate standard, accurate reports easily.
- Access the Databricks platform from a variety of interfaces (including Python and SQL).
- Scale and move data easily into other systems, as required.
- Build sophisticated pricing queries and modelling.
Where to Next?
The teams at Energy Queensland had to learn a lot of new technology. The MDAS is breaking new ground, giving the Enterprise Intelligence Team and the Strategic Forecasting Team a robust, innovative platform to build from. This project wrapped up on the 31st of January 2022. So right now, the teams at Energy Queensland are working with the platform and drawing value from it. The next step is to adopt the endorsed and proven big data architecture and deliver a backlog of business use cases.
Watch this space…
Update
Energy Queensland has been busy since we last checked in!
As of April 2024, Energy Queensland has clicked over:
- 1 million NMIs in Queensland with
- 5-minute interval data;
- registering over 600 million read values per day!
This milestone for Energy Queensland was achieved using the system we built with them two years ago. All the team had to do was add a bit of juice! Or compute power (the more formal term).
Technologies
- python
- aws
- databricks
