Indian Energy Exchange sees power in partnership and a service-based architecture

Indian Energy Exchange (IEX) operates a platform where electricity generators, distribution utilities and large industrial buyers trade electricity before it is delivered to end users. It has been working with partners to modernise its infrastructure and respond to changes in the market.

No electricity flows through the exchange, just data representing offers to produce, to consume and to pay for electricity. Since that electricity must be consumed as soon as it is generated, the exchange holds regular auctions to match demand, which is not constant but at least somewhat predictable, and supply.

Trading used to take place on the day-ahead market, with one auction held each day for supply of the next day’s electricity in 15-minute blocks. Many of the book-keeping operations relied on manual processes that could take four or five hours to complete.

The increasing share of renewable sources in the electricity supply, their output varying with the weather, prompted IEX to introduce a real-time market (RTM) on June 1, 2020, with 48 auctions per day for electricity to be delivered as little as an hour later.

“The real-time market segment was created to support distribution companies, to manage their demand-supply variations. It integrates renewable energy in an efficient and effective manner and provides the mechanism to support safe and reliable grid operations,” says Sangh Gautam, CTO of IEX since August 2019.

To get to that point required a change in IEX’s skillset and development culture. Before Gautam’s arrival, he says, “We were partly or mostly doing our development in house. And I think we could clearly see that if we want to be moving to a future state-of-the-art architecture, then we had to move towards creating technology partnerships. That’s where we started working with companies like Capgemini. They gave a very comprehensive idea of power exchanges the world over, what kind of technology are they using? In what directions are they going?”

Open interfaces

That led Gautam to conclude that IEX had to replace its monolithic auction software, adopting a more modular service-based architecture connected through open APIs with its members and the National Load Dispatch Centre in charge of the electricity grid. The exchange, its members and the NLDC also had to automate manual processes in order to complete each phase of book-keeping within the 30 minutes available between auctions.

“At the software level we had to tackle many challenges because in our existing architecture all our markets used the same components… order management, risk management, back office etc. All of those components were using the same software, so many computations of other market segments were happening in the system at the same time as the RTM tasks were being done,” says Gautam.

The project required a change in pace not just of dealing, but of development too: “We could clearly see that our culture is different than how start-ups are,” he says. “We created a very comprehensive training exercise for our team, on newer technologies…. We had to move from a traditional waterfall kind of methodology to an agile one. We incorporated tools like Jira, and CI/CD pipelines, so that whenever there is a change that is happening, our team is able to adapt to that change very swiftly. We can allocate our team from one project to another in the least amount of time.”

It took around eight months to build the real-time market in the run-up to its launch, and then another three or four months to finish some of the process automation and fix a few of the problems encountered at launch, Gautam said.

IEX derived other advantages from breaking up the monolithic software and opening the APIs. It was able to integrate the trading platform with its financial system in SAP, eliminating manual elements from some back-office settlement processes and enabling it to pay market participants in minutes, not hours, he says.

Gautam has been keen to incorporate customer feedback into the development process. “We ingrain that into our culture, getting customer feedback. So even our tech team is involved in from day one. We talk to a customer, so they understand what exactly the customer thought processes are, so that culture had to be also brought into the team.”

Even so, the development team can’t always please everyone all the time — but with the new APIs, auction participants were not limited to the user interface provided by IEX: they can build their own. “A system which is which is intuitive for one customer may not be for others. With member APIs, we have given that freedom as well, that they can easily create them themselves,” says Gautam.

Adding redundancy

There have been evolutions throughout the underlying infrastructure, not just in the user interface. The exchange system is built on a virtualized environment with hardware redundancy at the machine level for high availability, and further redundancy in software and even at the task level. There are also backup machines so that even if one component in a particular RTM process fails, it can switch to the backup machine in an automated manner, within seconds. This was done to ensure 100% uptime, he says.

On big transformations such as this, Gautam says it’s important to start with the technology vision, and then start bridging the gaps between the current reality and that vision, one by one, drawing on the expertise of partners where necessary.

It’s also important to plan for when things go wrong, as they inevitably will. “When we design the system, it should be designed from day one thinking about auto-healing,” he says.

While self-healing technologies may seem far off in some domains, another of Gautam’s watchwords, monitoring, is helping bridge that gap: “All the systems that we have, from hardware to software to every process that we have, almost, we are monitoring them all the time. There are alerts set up. The moment there is an alert, our system team gets the alert, then they’re in the know.”

“We’re even thinking of how we can have those alerts integrated into a self-healing system. Some of them have been integrated, and now we are moving towards how we can do more,” he says.

It’s not always humans looking out for problems: In some cases, it’s a software bot, says Gautam. “In case something fails, our system automatically detects there is a problem and triggers an alert. There are specific instructions to the bot on how to resolve that issue, which the bot is able to execute.”