Industry Reactions to U.S. Water Plant Hack: Feedback Friday
The U.S. government revealed this week that unknown hackers had managed to remotely access systems at a Florida city’s water plant and attempted to elevate levels of a certain chemical to a point where it would put the public at risk of being poisoned.
The attack, which targeted the water supply in Oldsmar, a small city in Florida, was discovered by staff at the plant — they noticed the mouse moving on the screen — and they rushed to take action before any damage was caused.
The attackers breached the facility via TeamViewer, which staff had been using to monitor systems remotely and respond to issues related to the water treatment process. The computers at the plant were running Windows 7 and all devices used the same password for remote access. Computers were remotely accessible from the internet and were not protected with firewalls, making it easier for the hackers to gain access.
Industry professionals have commented on various aspects of the breach, including implications and the measures organizations should take to prevent such incidents.
Daniel Kapellmann Zafra, Manager of Analysis, Mandiant Threat Intelligence:
“Since last year, Mandiant Threat Intelligence has observed an increase in cyber incidents perpetrated by low sophisticated actors seeking to access and learn about remotely accessible industrial systems. Many of the victims appear to have been selected arbitrarily, such as small critical infrastructure asset owners and operators who serve a limited population set. Through remote interaction with these systems, actors have engaged in limited-impact operations that often included manipulation of variables from physical processes. None of these cases has resulted in damage to people or infrastructure given that industrial processes are often designed and monitored by professional engineers who incorporate safety mechanisms to prevent unexpected modifications. We believe that the increasing interest of low sophisticated actors in industrial control systems is the result of the increased availability of tools and resources that allow malicious actors to learn about and interact with these systems.
While the incident does not appear to be particularly complex, it highlights the need to strengthen the cybersecurity capabilities across the water and wastewater industry similarly to other critical infrastructure sectors.”
Joe Slowik, Senior Security Researcher, Domain Tools:
“[The Stuxnet, Triton, Industroyer and the 2015 Ukraine power grid attacks emphasized] a critical barrier to adversary success: the ability to evade, influence, or outright deny operator visibility into and control over ICS environments. In all four examples, the attacks required some mechanism to hide from operators or deny their ability to correct or mitigate changes made to operating parameters.
In the case of the Oldsmar treatment plant incident, the intruder failed to attempt any such action based on information currently available. Had the unknown entity spoofed or otherwise interfered with HMI display parameters or sensor data, the operator on duty would be less likely to notice the incident as it took place, resulting in an attack moving on to engineering and process controls for potential mitigation or detection. Not only did the intruder fail to limit or manipulate process view in the environment, they executed the event during primary working hours on a weekday, almost ensuring that such activity would be quickly noticed (and mitigated).
Based on these observations and in light of past ICS incidents, we can therefore make a reasonably confident claim with available evidence that this was not an especially complex or savvy “attack”. As described in multiple sources, the intruder appears to have merely taken advantage of weakly secured, accessible remote access mechanisms to connect to plant equipment controls, followed by either deliberate or potentially inadvertent manipulation of the environment. That such an attempt occurred at all is certainly concerning, but the overwhelming evidence given event timing and execution indicate that there were only slight possibilities for this event to produce significant damage or harm.”
Ron Brash, Director of Cyber Security Insights, Verve Industrial:
“This will be a very interesting trial for any individuals caught, but it’s likely those who were managing this facility were struggling too &/or potentially negligent as they may have been aware that additional cybersecurity was needed given the importance of this infrastructure. I suspect insurance due diligence would have been problematic had a disaster occurred; it might even be revoked now…
This was not the first attack on water or utilities, and lucky there was a human in the loop to prevent disaster. The warning bell should be sounded, but CISOs (or those in charge) are lucky because they are in a very defensible position. In fact, I believe this is a call for organization’s to double down on the cybersecurity basics, assess their asset & infrastructure, and validate controls on their “crown” jewels. There are effective and feasible strategies out there to help municipalities get control of an out of control situation that will slowly gain speed.
Whether the government should step in and help, or by what means is debatable. Municipalities need help, but they also need ongoing support & commitment – not a one time grant that probably will be without a follow up.
Digitalization of water and utilities presents a problem we are seeing more and more often. New connected systems are being added, and this magnifies/multiplies the risks that may not have been applicable before. Security always degrades, and remote connectivity or any new technology.”
Grant Geyer, Chief Product Officer, Claroty:
“Water and wastewater is one of the most at-risk critical infrastructure sectors today. Industrial control system (ICS) vulnerability disclosures impacting the sector have increased significantly year-over-year. As noted in our Biannual ICS Risk & Vulnerability Report released a few days ago, the Claroty Research Team found that ICS vulnerabilities disclosed during the second half (2H) of 2020 increased by 54% from 2H 2019 and 63% from 2H 2018 in water and wastewater.
Due to the long depreciation period of equipment in critical infrastructure environments, technology obsolescence and the security accompanying security vulnerabilities is a common occurrence. Additionally, many water utilities are small entities and are under-resourced, making the challenge of developing a robust security program that much more challenging.
The solution is not as simple as eliminating remote access to such high-stakes environments. The nature of our increasingly digitized world, especially with the shift to remote work caused by the pandemic, makes remote access a requirement – even in critical infrastructure. This isn’t a “should we or shouldn’t we” discussion – it’s coming at us. The key is how remote access can be implemented securely, so that we can stop these attacks – which will inevitably continue to happen – before the damage is done.”
Andrea Carcano, Co-Founder, Nozomi Networks:
“Based on the information available at this moment, this attack seems to lack any sophistication that could trigger more profound reactions. The fact that the perpetrator didn’t conceal his visual presence to the personnel monitoring the water treatment operation is the first signal that suggests the relatively low complexity of the attack. Furthermore, according to the reports of the incident, the attacker increased the levels of sodium hydroxide by a significant amount, typically monitored by automated systems, which likely suggests that the threat actor didn’t possess a specific background knowledge of the water treatment process.
Nevertheless, this incident is important because it reflects the status of too many industrial control system (ICS) installations, especially those with smaller budgets and a smaller size, where security is often overlooked. Remote access, in particular, when not designed with security in mind, is often the beachhead used by remote attackers to infiltrate an ICS network. In this very case, the water treatment plant of Oldsmar has been using a Teamviewer instance, which apparently was accessible from the Internet. While it is not known at this stage how the attackers obtained the credentials required, this incident, like many that we’ve documented in recent years, didn’t seem to rely on sophisticated zero day exploit for its execution.”
Chris Grove, Technology Evangelist, Nozomi Networks:
“As evidenced in this cyber attack, typical cyber security activities may not have mitigated this risk, including vulnerability management, network segmentation, system hardening, identity and access management, firewalling, etc. In many cases, and especially during this pandemic, remote administration solutions have been thrown into the mix, sometimes haphazardly. In some cases the due diligence and compensating security controls haven’t been recognized. In other cases it has. Either way, facilities should stop thinking like they will prevent cyber attacks and start thinking like they’re already happening. They may not see it, so they should be in a constant state of recovery.
That said, concepts such as zero trust start making sense. Once the operator realizes that nothing is to be trusted, they move towards monitoring the process itself, and the parameters being sent from all of the devices in the control room to the equipment. If the water facility in Oldsmar had this level of cyber security, alarms would have gone off the moment the values were set to anomalous numbers.
Unfortunately most of today’s facilities are only protected a little bit by wide monitoring which doesn’t go deep into the industrial control protocols themselves. Any facility where human lives are at risk, particularly so many, should monitor the industrial control process using artificial intelligence and anomaly detection to monitor, alert and stop anomalies within the process that aren’t a part of regular operations. By doing so, the facility would mitigate many risks including malicious or negligent insiders that may accidentally type a few digits too many, as well as external attackers looking to pull off an act of terrorism.”
Austin Berglas, former head of FBI NY Cyber and Global Head of Professional Services at BlueVoyant:
“Along with energy production and manufacturing, water supply facilities are part of the United State’s critical infrastructure and have long been targets for cyber attack from both criminal and state sponsored entities. Water facilities rely on systems control and data acquisition (SCADA) systems to manage the automated process or water distribution and treatment. Many of these industrial control systems (ICS) are outdated, unpatched, and available for review on the Internet, leaving them incredibly vulnerable to compromise. In addition, many ICS solutions were designed for non-internet facing environments and therefore did not incorporate certain basic security controls – this offers additional vulnerabilities as more and more operational technology environments are allowing access to their ICS systems from the Internet.
In 2013, the FBI investigated a compromise of the Bowman Avenue Dam in Rye Brook NY and found that members of the Iranian Revolutionary Guard had gained access through Internet facing controls. Although the Dam was not functioning at the time and was most likely not the Iranian’s main target, it demonstrates the vulnerability of certain critical infrastructure when their ICS systems are allowed to be exposed to the Internet and not isolated.”
Saryu Nayyar, CEO, Gurucul:
“The cyberattack against the water supply in Oldsmar, Florida, last week should come as a wakeup call. Cybersecurity professionals have been talking about infrastructure vulnerabilities for years, detailing the potential for attacks like this, and this is a near perfect example of what we have been warning about. Though this attack was not successful, there is little doubt a skilled attacker could execute a similar infrastructure attack with more destructive results. Organizations tasked with operating and protecting critical public infrastructure must assume the worst and take more serious measures to protect their environments.”
Chloé Messdaghi, VP of Strategy, Point3 Security:
“The thing we need to understand is that you don’t have to be a highly skilled attacker to be able to successfully breach a system like this. Although alarms would’ve been triggered before any dangerous water reached anyone’s taps, this plant was very lucky that the worker noticed his mouse moving and was able to address it quickly. Water plants are not known for their security resources, and between budget cuts and COVID keeping people working remotely, they’re even more vulnerable. It’s becoming more and more easy to access systems like these by people who have hardly any experience at all.
The area this happened in has a high population of children, and it’s disturbing to think someone would attempt to do harm like this.”
Karl Sigler, Senior Security Research Manager, SpiderLabs at Trustwave:
“All systems used for critical networks like these should have very limited, if any, Internet access. User accounts and credentials used to authenticate locally on the workstation and for TeamViewer should be changed frequently and utilise multi-factor authentication. In this instance, it was lucky that the user was physically there to see the remote control and what settings had changed, but all critical activities should be audited, logged and monitored for abuse.”
Learn more about threats to industrial systems at SecurityWeek’s ICS Cyber Security Conference Series and Security Summits virtual event series