Mark Fabro successfully married tech knowledge and C-level chops to help elevate his role and SCADA security to the next level. Dan Kaplan reports.

There was a time in Mark Fabro's career when he was perfectly content avoiding the men and women in the corner offices.

After all, it was the early 2000s and Fabro was buried deep in the weeds of critical infrastructure research, focusing on challenges like threat profiling and recognition, risk analysis, intrusion testing, data collection and “grid” cracking. Coveted skills, for sure, but not something to which a power company CEO was giving much thought at the time.

“Really, the insider was one of the only things you had to worry about, other than natural or manmade system incidents or failures that could impact production,” recalls Fabro.

Then, around the middle of the decade, things started to quickly change. The systems that manage, direct and regulate utilities, like chemical plants and oil-and-gas refineries, increasingly became connected to the public internet and the corporate network, effectively opening the door for the first time to the threat of malware. All of a sudden, the prospect of a hacker shutting off the lights to millions of people shifted from a Hollywood script idea to something that could at least pass the plausibility test.

The men and women in the boardroom started perking up. And with that, Fabro's career path changed.
Suddenly came the need for people with technical talents who could also convey the threat and business challenge posed by supervisory control and data acquisition (SCADA) systems to upper management, in easy-to-understand terms.

Fabro, who now serves as president and chief scientist of Lofty Perch, was a natural. In June, as a testament to his work in this field, he was named information security professional of the year at the 2011 SC Awards Canada.

Fabro, whose company specializes in assessments, training and compliance strategies for critical infrastructure entities, says industrial control systems – which provide “national security, economic security and quality of life” – traditionally were isolated from the corporate environment. But that changed with the rise of internet connectivity.

“The business demanded they start getting connected and working together,” Fabro, 44, explains. “Now you have executives who want to instantaneously know what's going on in the control system environment. The competitive advantage lies in how fast you can get situational awareness from your control system into the corporate space to shape and meet supply and demand.”

With each passing year, Fabro's skill set has become more valuable. Back in 2007, the largest SCADA-related headline-grabber was the release of a video, produced for the U.S. Department of Homeland Security, which depicted a “hacker”-controlled turbine shaking wildly until pieces break off and plumes of smoke fill a test lab.

But it wasn't until 2010, with the emergence of Stuxnet, the first malware ever written specifically for industrial control systems, that the narrative of a ticking time bomb really took flight. Stuxnet gave life to additional research, and so far this year, the relatively nascent U.S. Industrial Control Systems Cyber Emergency Response Team, part of US-CERT, revealed a number of vulnerabilities affecting SCADA products. Not all have been particularly serious, but their evidence shows that researchers are paying close attention.

And most critical infrastructure operators, as well as product vendors, don't appear entirely ready to battle these new threats.

According to a joint study earlier this year from McAfee and the Center for Strategic and International Studies (CSIS), which surveyed 200 IT security executives working at utilities in 14 countries, 40 percent believe their sector's vulnerability to attack has increased since last year.

Meanwhile, vendors have been slow to respond to product vulnerabilities, as they are not used to dealing with critical flaws that need quick patching.

Enter Fabro

Michael Assante (right), the former chief security officer at the North American Electric Reliability Corp., which oversees U.S. electric grid operators, says he contracted Fabro a few years ago to work with the utilities so they could better understand their risks.

“One of the major contributions that Mark brought to the table is the issue of understanding cybersecurity in the context of industrial technology, control systems and SCADA,” Assante says. “Mark does an incredible job of bridging the gap between the hard, technical story and how it matters to you. He just has a lot of resonance when he speaks.”

Assante cites his reasonable and responsible approach that never relies on the fear card. Fabro also has a way of connecting – not just with business executives – but also those SCADA experts who may not be too familiar with cybersecurity.

“The other very important element is Mark can sit down with a control systems engineer and have the capability of communicating,” Assante says. “He can translate how you think about security, knowing the mission is to keep that system up and running.”

Experts preach patience as the control system community matures.

Control systems are purpose-built and designed for longevity and maximum uptime, Fabro says. Not considered during the design process were the three pillars generally applied to securing conventional IT systems – confidentiality, integrity and availability.

“Once an adversary is able to get through the vendor access channel or corporate domain, they find themselves in an environment that is sensitive to enumeration and large data bursts,” Fabro says. “The IT profile of many of these critical infrastructure systems is circa 1995.”

Rick Moy (right), president and CEO of NSS Labs, an independent network testing firm, says control systems present a unique challenge with which end-users are not familiar.

“It's hardware and software together,” Moy says. “There's a lot more moving parts than, say, patching Adobe Flash. Those processes are not really developed. We're early in the maturity cycle. We're somewhere in the early to mid-90s right now, compared to quote-unquote internet security. There really isn't much awareness yet. We just saw the beginning with Stuxnet. There aren't a lot of guidelines for folks to follow. It's a very new juncture.”

But this general lack of polish around SCADA security doesn't mean the answer is panic, says Fabro.

“With the amount of recon going on in these industrialized systems, it would be foolish to think that an adversary is not interested in compromising an element of the North American critical infrastructure,” Fabro says. “[But] you can't evangelize through fear, uncertainty and doubt.”

SC Magazine recently caught up with Fabro, who was candid about the challenges facing the control systems industry. But he was quick to point out that more is being done than most people realize – and it's a gradual process.

SC Magazine: You were trained as a technologist but you recently have taken on a more evangelical role. Talk about how you got into the control system field and how your career has evolved.

Mark Fabro: SCADA security can be unexciting at the best of times, and unless you create interest, you risk having key decision makers not get on board with the cause. Having been in cybersecurity for almost 20 years, and focused on SCADA, control systems, and critical infrastructure for about 12 of those, the diversity of cultures demands you discover creative ways to get the message across. But the technologist has not gone away, and in fact those skills are called upon almost every day to help shape the solutions and countermeasures that passion and excitement simply can't fix.

SC: What are some of the major challenges the industry faces? Help us separate the FUD from the reality.

MF: As of late, the industry seems to be having some difficulties showcasing the great work it has been doing, and there seems to be a continuous surge of naysayers that just do not want to support the fact that we are making progress. There is a very large amount of vulnerability research being released that seems to take priority and overshadow their efforts. In a post-Stuxnet world, the public seems to be more concerned with the hype surrounding each and every vulnerability rather than some of the great work being done by vendors, asset owners and information-sharing communities.

SC: Protecting the smart grid (which delivers electricity from suppliers to consumers using two-way digital technology) is obviously a focus for power companies. Is enough being done in this area?

MF: The smart grid is, of course, on almost every utility's agenda, and security is a key focus area. There is a lot of activity in the security area, but the responsibility to secure the architecture includes the asset owners and the vendor and others that provide communications (i.e., backhaul). I do believe that we as a community need to do a better job at trying to define just what the smart grid is, as it can mean a lot of things to a lot of people, and that can shape how security is approached. There are many parts to this thing. There are many different technologies involved that do a lot of different things and each one needs to be looked at from a security perspective.

SC: Aside from your work at Lofty Perch, can you describe some other projects with which you are involved?
MF: I have a few things that are ongoing, and almost all of them are in support of outreach and education. I am lucky to be involved in the Canadian Industrial Cybersecurity Council, which I chair, and our focus is to review Canadian public sector security activities or programs and ensure they take into account vital SCADA and control system assets.

I also support the Repository for Industrial Security Incidents (RISI) at securityincidents.org, perhaps the largest database available for SCADA and control systems security incidents.

Lastly, I am also working to ensure security is integrated into the engineering curriculums, so that our SCADA and control system engineers of tomorrow have a headstart in knowing how to protect vital systems, as well as build and operate them.

SC: What's your best advice for control system personnel when it comes to building a strong security program?
MF: My experience has taught me that strong SCADA/control systems security programs are created by teams – teams that are comprised of both IT security and control system engineers. Modern SCADA and control systems have matured to be very IT-based, but the uniqueness and nuances associated with industrial automation simply does not allow for IT security best practices to always be mapped directly. I recommend that collaborative teams be formed, and that the great products developed by ISA [International Society of Automation], NIST or the Department of Homeland Security Control Systems Security Program (CSSP) be used as a starting foundation for program development.

SCADA bypass: PLC versus HMI

In May, a scheduled conference talk on vulnerabilities in Siemens industrial control systems was shelved after the affected vendor was unable to develop a working fix in time and expressed concern.

Researcher Dillon Beresford, an analyst at NSS Labs, decided to pull the plug just hours before he was set to hit the stage, due to the potential of real-life harm that the presentation could have caused. (Now that a patch is in place, Beresford plans to present his findings this month at the Black Hat conference in Las Vegas).

Considering the sensitivity of SCADA products, one might expect more instances like this in the future, says Rick Moy, president and CEO of NSS. Especially if the vulnerabilities are present in programmable logic controllers, or PLCs, systems that directly connect to production instruments, such as valves and motors.

As a result, flaws in PLCs are potentially more dangerous than SCADA bugs in a human machine interface (HMI), which is software used to program PLCs, Moy says.

“If it's in a PLC, then an attacker could access that PLC directly without going through a user's workstation,” he says. “It allows an attacker to completely bypass security controls.”  – Dan Kaplan