Last week, a small group of energy industry, data-focused leaders gathered in Melbourne to listen to Chris Smyth, ZEN Energy’s Inaugural CIO and recently announced EGM of Customer Operations, share his and ZEN’s journey to build a scalable data platform from scratch. It was a fascinating and engaging discussion, don’t worry – we took notes for you!
SIX BIG THEMES
That permeated the discussion
1
Outsource vs Insource - How do you decide?
ZEN’s journey with Ignite has been over seven years. In Chris’ own words, ‘it took that long for the kids to grow up and leave home’ – and even then, Ignite continues to help us as we ‘occasionally pop back home to get the washing done’.
ZEN did not change their mind from Outsource to Insource, rather it was about ZEN’s maturity. Maturity of the organisation, the people, the processes, and the platform.
In the early days of its business, ZEN was not ready or able to scale in-house. Instead, it chose to leverage the core competency of Ignite to help them build up their capabilities. As ZEN built up its capabilities, Ignite transferred the knowledge.
‘Good documentation and processes were key to this. Core values alignment was important when we selected Ignite – they were the right partner for us on this journey.
2
Deciding on the Centralised / Decentralised operating model
At the start when ZEN had few people, everything was largely centralised – the beauty of a small business. As ZEN has grown, Chris and the team adopted a more hub-and-spoke model, with analysts forming part of operational teams. The platform has become the lynchpin to ensure everyone is using the same data.
3
Celebrate the successes!
Chris’ biggest lesson is that they never celebrate their successes enough. In the data world, one good sign that things are going really well is when nothing bad happens – and this takes a lot of work. Work that may go unnoticed by the rest of the business. Chris reminded us all to make sure that we celebrate data success when it occurs.
4
Processes and procedures will make or break a good platform.
Chris noted that the lack of data governance can undermine the integrity of the best-designed data platforms.
He suggested that identifying ‘system owners’ rather than ‘data owners’ helps to build the robust processes and procedures required to deliver the best results.
5
Architecture and engagement allow business plans to evolve.
The Ignite/ZEN partnership first started around Ross Garnaut’s kitchen table when the idea of ZEN as a new energy company was born. Chris and Harj (CEO of Ignite) had a shared vision of taking advantage of a clean slate. Here was an opportunity to do things the way that they always wished that they could do things. No legacy systems, no incumbent processes or procedures, no data fiefdoms. But they had no idea what the organisation might evolve into so they needed to design an architecture that could grow and evolve with the business.
Since those early days, ZEN has been through a merger and demerger and switched billing engines twice and the platform has scaled and supported all these changes.
Now the platform is to acquire new data from generation assets as ZEN expands its asset portfolio. The architecture must allow for business change.
6
Data platforms as a recruitment tool?
While ZEN has a standout purpose of attracting the right people, the modern data platform is like icing on the cake. As Oscar, one of ZEN’s recent hires in the room said, it was very attractive to see the modern capabilities that would make me successful at ZEN. ZEN has actively promoted its tech stack and capabilities to prospective recruitments, and it has worked!
WANT A DEEPER DIVE? MORE DETAILS ON CHRIS AND THE TEAM’S JOURNEY
Chris, you have been around this industry for some time. What were you thinking about when you embarked on this journey with ZEN?
I started with the concept of ‘let’s not repeat the mistakes of the past’ – how can I learn from all of my experience and create something standout from scratch?
Large organisations often grow by acquisition, forcing numerous databases and platforms together, that become very hard to reconcile. There are often better platforms available for better results, but large organisations have to make what they inherit work, they have to justify their acquisition, and the resulting data platform is often a poor fit for purpose.
With ZEN, we got the chance to build a platform from scratch and we did that with the right partners, with common values and similar experiences – Harj & Kris and the team at Ignite.
We thought – if we get to start something new here, let’s do it right from the very get-go.
What was the aim?
We were trying to anticipate the data sources we were going to need to operate a successful business and work from the ground up so that we would have an integrated, consistent data platform for pretty much everything.
And today, ZEN is enjoying the benefits of not having dispersed data all over the place and has limited problems with data integrity.
Over the seven years, we had the strong support and enablement of Ignite and because of that, we were able to build up our first billing platform out of open-source software and successfully invoiced our first customers in the order of $60 million over a few months. That was a real career highlight and a tremendous start to a very fruitful journey with Ignite.
And today, with what we’ve got now, knowing that it was based on the hard work and the creativity of those early team members – is just fantastic.
COVID helped us so much!
We had to do a lot during COVID. We were going through a demerger at the time of Melbourne’s first lockdown and then, during the second, we migrated to a new billing platform!
We were deeply concerned, as we sent everyone home. But, because we had built a robust, cloud-based platform, there were no problems with that at all. We completed two monumental data tasks – because we had set the platform right from day one – flexible and scalable and able to cope with the completely unexpected!
I remember thinking to myself how are we going to do this while we are in lockdown? But actually, the lockdowns made it easier – we needed to do testing on the weekends, and no one cared! They were happy to engage and work together!
What would you do differently?
Harj had always spoken to me about celebrating success. But my view was, well, the results should speak for themself. You can see the quality of the work.
But not everyone does, and you need to consistently celebrate the successes. You’ve got people on the team who’ve done these things, and you need to highlight them and their work, otherwise, the business can take it for granted.
It was the hard work and the clever thinking of creative people who’ve got us here and celebrating their success or just acknowledging it more, I think would have put us in an even stronger position.
And another thing I’d do differently is focus on creating the sort of organisational power and strength for the data team so that the organisation recognises the work involved in creating these systems because it’s really hard work, and remarkable when you get it right.
The cliche that “retail is detail” is true. There’s a tremendous amount of work that goes into making sure data is configured in a way that’s accurate and reliable, repeatable and scalable. Data consumers don’t always see that.
Making sure governance works – sticking to processes.
I’d probably be a bit heavier-handed on the process and ensuring that people who interact with our systems have followed the process as good governance.
We’ve had a couple of initiatives where we’ve had to tidy up the mistakes of others. And I don’t think you should be particularly forgiving when processes are not adhered to on a regular basis.
Watch for the long tail.
When, you go through a transformation like that, you give more thought to the bulk of the use cases with the big results. But there is a long tail of smaller use cases that often get neglected. You want to pay attention to these too as they are what stop you from getting big tasks completed.
Reaping the benefits of the system.
Seven years on, ZEN is at the point where we’re really getting a sophisticated level of data use, we’ve got dashboards that are enabled and a frontend for numerous reports and analytic tools. We have some sophisticated risk analytics capabilities, and our risk team have done some really clever stuff.
You just open up the URL and there’s all of ZEN’s dashboards laid out for you. So, you can check forecasting accuracy, risk and uncertainty metrics and much more that the business needs. It’s there all on a dashboard of dashboards – a thing of beauty!
Our forecasting team has gone on from building an Excel spreadsheet from nothing to a much more powerful set of outcomes, ingesting new data regularly, all more or less automated. Now, we are implementing a new energy trading risk management tool and know that this will be made easier with our platform.
We have the robustness of our overall architecture to accommodate new data pipelines and use cases easily.
And the transformation from outsourced to insourced was significantly enabled because of the architecture we built. We have a complete record of the building’s history ourselves. And the support of Ignite whenever we need it.
That all started with a strong relationship.
Starting with just a whiteboard and getting to where we are today is a testament to the strength and quality of the relationship that we have with Ignite. The support we got from outsourcing, the analytics itself, their view on what was happening in the energy market, what technology was out there, defining our architecture and helping us with our data strategy was invaluable.
ZEN’s growth trajectory over these seven years has been very steep, growing rapidly from zero customers to 5.2% renewable market share today. That’s a massive increase in the scale of the business including rapid business changes, with lots of industry changes, acquiring new customers, the occasional liquidity squeeze, and changing ownership. So much we have worked through together over the years.
The analogy I like to give is that the kids are grown up and we have to make our own way into the world. And so, today we have a team of data engineers and software developers in-house.
We couldn’t justify that back in the day. And what I particularly appreciated about the relationship with Ignite is the flexibility, awareness and the support as we’ve gone through going from infancy into leaving home.
It was probably a little bit bumpy as we moved out of home but Ignite supported us as moved out sharing all the documentation, the support to enable us to move on, with the best wishes and a great relationship still intact – that is why I am here today
Related Services
Data Platform Build
Create the foundational capability that allows for economical, flexible, and scalable growth.
Ignite implement an integrated data platform that enables the core building blocks for your use cases.
Platform Manage + Grow
Monitor and grow the value of your data assets, continuously and incrementally.
Let us manage and scale your platform, ensuring its reliability, its adoption by end users, and enabling you to grow your business.
