FBI, Secret Service investigating cyberattack on Florida water treatment plant

Federal law enforcement is now looking into a cyberattack at a water treatment plant in Oldsmar, FL where someone was able to remotely access systems and add a dangerous amount of chemicals to the town’s water supply.

On Monday, Pinellas County Sheriff Bob Gualtieri explained during a press conference that an employee at Oldsmar’s water treatment facility saw his mouse moving independently of him on Friday morning but thought nothing of it—it’s common for people in the field to remotely access systems through their TeamViewer software.

But it happened again later that afternoon, according to Gualtieri, and this time the person moving the mouse changed the levels of lye, or sodium hydroxide, from 100 parts per million to 11,100 parts per million. The chemical is used to help officials manage the town’s potable water PH levels. But when sodium hydroxide is added at those levels to the water, it can become dangerous for people to even touch. 

“This is obviously a significant and potentially dangerous increase. Sodium hydroxide is the main ingredient in liquid drain cleaners. It’s also used to control water acidity and remove metals from drinking water in the water treatment plants,” Gualtieri said.

SEE: Incident response policy (TechRepublic Premium)

The person at their computer watching the hacker do this immediately put the levels back to normal and called his superior, who then called the police. 

The situation has now become national news, with Florida Sen. Marco Rubio asking the FBI for assistance and writing on Twitter that this “should be treated as a matter of national security.”

Gualtieri and others working at the plant said there was never any danger because the treatment plant has multiple systems in place to make sure that a change like that could not be implemented. Even if it was implemented, it would take about 24 hours for the chemical to enter the water stream, Gualtieri noted.

Echoes of previous attacks

But the situation raised concerns and references to similar attacks that have taken place across the world. 

Many online referenced the Russian attack on a Ukrainian power grid in 2015 and another attack on at least two Israeli water treatment plants last year. 

Justin Fier, former national intelligence officer and director of cyber intelligence at cybersecurity firm Darktrace, said the attack “was a stark reminder of the risks that come from the hyper-connected world we live in.”

“Analog ICS systems have either been updated or retrofitted with remote monitoring and control systems, exasperating the great challenge that faces defenders today. Governments around the world will certainly be looking at this incident and probing their own systems to see if they are similarly vulnerable,” Fier said. He noted that the media attention around the attack may also have been a goal of the people behind it.  

“This time an amateur move of a rogue mouse cursor gave the preparators away, but we are seeing a sharp rise in sophisticated, stealthy attackers that slip under the radar unnoticed. What will happen the next time there is no flashing red light? Critical environments do not fail gracefully.” 

The rise of remote tools

Many cybersecurity experts said the move to using remote tools like TeamViewer was a consequence of the COVID-19 pandemic and a more general shift to digitizing systems. But this digitization came with downsides, as seen in this cyberattack. 

It is also becoming increasingly easy for attackers to simply pick unprotected targets out of a hat using platforms like Shodan and others. Etay Maor, senior director of security strategy at Cato Networks, said attacks like this have happened before, both in the US and abroad, but that the increasing reliance on remote access and remote administration systems was making it easier for average hackers to do damage. 

Maor shared a simple search he did today on Shodan for Remote FrameBuffer systems that don’t have a username or password enabled. There were more than 6,300 available, with about 900 in the US and about 1,500 in Sweden. 

“Look at how many results I got from this very naïve and simple search. I can run similar searches for specific protocols, software, hardware, etc. Securing these systems is no easy task, on the one hand, you want easy management and administration—think about an emergency situation,” Maor said.

“However, these systems must be properly secured, use more than just a username and password for authentication, and constantly monitored for threats and attempted breaches. Remote administration, as is remote work, is truly a tough task these days–we were rushed into this situation, one that may actually require a new way of thinking about how to connect, secure and manage all of these systems while still allowing productivity and efficiency.”

To illustrate how easy it is for cyberattackers, Maor shared a screenshot of his search on Shodan, showing how simple it is to look for and gain access to a variety of utility companies using these kinds of remote tools. 

“I am literally a click away from controlling this system, whatever it may be,” Maor noted. 

Andrea Carcano, co-founder of Nozomi Networks, echoed those comments, noting the relative lack of sophistication in the attack because the attacker didn’t conceal their visual presence to the personnel monitoring the water treatment operation.

Carcano added that the attacker also did not know that such a massive change would trigger automated systems and alerts, meaning the person did not have any background knowledge on the system. 

“Nevertheless, this incident is important because it reflects the status of too many industrial control system installations, especially those with smaller budgets and a smaller size, where security is often overlooked,” Carcano said. 

“Remote access, in particular, when not designed with security in mind, is often the beachhead used by remote attackers to infiltrate an ICS network. In this very case, the water treatment plant of Oldsmar has been using a TeamViewer instance, which apparently was accessible from the Internet.”

Seyi Fabode, CEO of water distribution system monitoring company Varuna, explained that the water system industry is losing a significant amount of expertise as a generation of experts retires, leaving many treatment facilities scrambling to find people who are able to detect any anomalous changes like the one the hacker was trying to achieve.

The need for updated technology was also bringing its own issues due to a dearth of cybersecurity talent.

“As new tech tools are being brought into the industry (IoT, new treatment methods, etc.) the industry lacks the expertise to detect these types of hacking activity,” Fabode said, adding that it was important for enterprises to understand cybersecurity and also have systems in place to detect anomalies. “They are water systems, not tech companies, and their vendor partners should be experts at tech and provide the support.”

Public utilities as targets

Some cybersecurity experts referenced a 2016 case where the small Bowman Avenue Dam in Rye Brook, NY was targeted by Iranian hackers as part of a larger plot.

Austin Berglas, the former head of FBI NY Cyber and now an executive at cybersecurity firm BlueVoyant was the lead on the investigation into the Bowman Avenue Dam case and said water supply facilities have long been targets for cyberattack from both criminal and state-sponsored entities.

“Water facilities rely on systems control and data acquisition (SCADA) systems to manage the automated process or water distribution and treatment. Many of these industrial control systems are outdated, unpatched, and available for review on the Internet, leaving them incredibly vulnerable to compromise,” Berglas said. 

“In addition, many ICS solutions were designed for non-internet facing environments and therefore did not incorporate certain basic security controls. This offers additional vulnerabilities as more and more operational technology environments are allowing access to their ICS systems from the Internet,” Berglas added, highlighting the vulnerability of certain critical infrastructure when their ICS systems are allowed to be exposed to the Internet and not isolated.

Throughout 2020, there were hundreds of cyberattacks on schools and hospitals, raising concerns about the country’s ability to protect critical enterprises. 

But many cybersecurity experts said foreign governments, including the US itself, have spent years targeting public utilities because of the damage attacks could do. 

Cerberus Sentinel VP Chris Clements explained that just last month, a utility company in Independence, MO suffered a cyberattack that took down their payment portal for a month, causing residents to receive billing for 60 days’ worth of usage at once.  

Clements and others, like Vectra CEO Hitesh Sheth, said utilities and other infrastructure had to move beyond simply throwing money at the problem and implement more stringent controls before it is too late. 

“Public utilities, including power and water systems, have been prime cyberattack targets for years. There’s a whole Russian cyber team, ‘Energetic Bear,’ focused on hacking American energy infrastructure,” said Sheth. 

“In the Oldsmar case, it’s premature to assign motive or place blame. However, we’ve seen enough breaches of the US power grid, water systems, and even nuclear plants to conclude this: protecting these critical facilities, and upgrading their cyber defenses, should be a far higher priority.”

Cybersecurity Insider Newsletter

Strengthen your organization’s IT security defenses by keeping abreast of the latest cybersecurity news, solutions, and best practices.
Delivered Tuesdays and Thursdays

Sign up today

Also see