How in the World Do You Measure Cybersecurity, Anyway? with Michael Daniel

Software supply chain attacks threaten today’s connected cars, pacemakers, and even, potentially, the elevator you’ve just entered. Amid these rising stakes, what is government’s growing role in cybersecurity regulation and how can compliance—and even an organization’s “cybersecurity-ness” level be assessed and evaluated?

In this episode of Finite State’s podcast, “IoT: The Internet of Threats,” Michael Daniel, President and CEO of Cyber Threat Alliance discusses today’s increasing calls for transparency in product and supply chain security and the role of the SBOM as a key control in this initiative.

During this 20-minute episode, Michael and Eric Greenwald, Head of Cybersecurity Policy, and General Counsel at Finite State, examine:

• The government’s evolving role in cybersecurity regulation, from the Cybersecurity Maturity Model Certification (CMMC) to Executive Order 14028
• How to measure the efficacy of cybersecurity products and practices and the pros and cons of first- and third-party certifications
• The government’s contribution to improving cybersecurity practices by encouraging the adoption and implementation of the Software Bill of Materials (SBOM)
• How SBOMs help us see inside the software we use and address a key weakness in cybersecurity right now

All episodes of Finite State’s “The Internet of Threats” podcast can be heard on Spotify, Apple Podcasts, and Google Podcasts. Listen to this episode in its entirety below:

Episode Guest: Michael Daniel, President & CEO, Cyber Threat Alliance

Bio: Prior to his role as President and CEO of Cyber Threat Alliance, Michael served as the Cybersecurity Coordinator to President Obama’s National Security Council (NSC). His work at the NSC followed a 17-year tenure as Program Examiner and later a Branch Chief for national security programs with the U.S. Government’s Office of Management and Budget.

Michael earned an MS in National Resource Planning from National Defense University and holds an MPP in National Security from Harvard Kennedy School.

Full Podcast Transcript:

Eric Greenwald: Michael Daniel is the president and CEO of the Cyber Threat Alliance. Prior to that, he spent 17 years with the Office of Management and Budget in the US government, working on the National Security budgets and very deep, dark, secret stuff that Michael knows all about. And then after that, he was the Cybersecurity Coordinator for the Obama administration, which title that is informally referred to as the Cybersecurity Czar. And, full disclosure, when Michael was the cybersecurity czar, he was my boss at the White House. Michael, welcome. And thank you so much for joining the podcast.

Michael Daniel: Oh, thank you for having me.

Eric Greenwald: You bet. So, Michael, as I mentioned, you are the president, CEO of Cyber Threat Alliance. That’s a job that you’ve had since leaving the White House, can you tell us a little bit about the Alliance and the work that it’s doing?

Michael Daniel: Sure. So, the Cyber Threat Alliance is a nonprofit that is intended to enable cybersecurity providers to share threat intelligence with each other. And to do that in both an automated and an analytic fashion. So, in both machine speed and at human speed, and we have about 35 members right now headquartered in 11 different countries around the world that share cyber threat intelligence with each other. And that’s really what we do. And the goal of CTA is to enable those member companies to do a better job protecting their customers and clients to support the disruption of the bad guys and to raise the level of cybersecurity across the digital ecosystem.

Eric Greenwald: So, we’ve talked on this podcast previously about the information sharing role that the US government plays, and the way that they’ve been leaning forward, at least if you measure over the last decade into being a sharing information, threat information with the private sector, whether in an unclassified or classified format. It seems like what you’re doing is trying to complement that maybe fill some gaps that the government can’t fill through its information sharing program.

Michael Daniel: So, I view it as really work. It’s where the private sector version of what you were just describing, right, because some information is really all about what the private sector sees, and what the private sector is aware of. And so that’s what we focus on, and enabling those private sector companies to share with each other. And, you know, I think one key thing … and one of the reasons we got it started is, if you think about it, like when I came out of the administration, right, we had a financial services ISAC for the financial sector. We had a healthcare ISAC for healthcare. We had one in energy. We had one in retail. But there wasn’t one for the cybersecurity industry, which is actually kind of interesting, because that, in some ways, is the industry that needs to be sharing the technical data the most. And there’s plenty of sharing that was going in an ad hoc and informal way. But CTA sort of formalized that and put some structure in place around it to make it sustainable and scalable over time.

Eric Greenwald: Well, I imagine a key element of that is trying to get the cybersecurity companies to cooperate and treat each other as allies in this space where, you know, it’s the rising tide lifts all boats, rather than … Oh, my threat information is sacrosanct that I won't share it with you.

Michael Daniel: No, that’s right.

Eric Greenwald: So, well, that what I wanted to talk to you a little bit more detail about today was where things are going with respect to government regulation, for cybersecurity. And we’re seeing some interesting trends, whether it’s in the nature, frequency, severity of cyberattacks, cyber threats, or the government’s reaction thereto, and the way that they seem to be leaning into the prospect of regulation, for companies with respect to their cybersecurity practices. Can you tell me a little bit about what you’re seeing high-level with respect to how government is approaching cybersecurity regulation and how that’s changing?

"Once IT systems begin to affect the physical world, it shouldn't be a surprise that the government will play a bigger role in cybersecurity." - Michael Daniel 

Michael Daniel: I certainly think that there is a different view of the government’s role in some cybersecurity regulation, compared to, say, 10,15 years ago. But at one level, I argue that that’s a natural consequence of the digitally connected world, right. Now that we have … now that we’re living in this world where your car is connected, your watch, your phone, the power plant, the water system, like everything is connected to the internet, it’s not surprising that the government is actually going to play a greater role in that, right. I mean, we used to joke about this all way back at our White House days, right? It’s one thing if your spreadsheet crashes, it’s another if your connected car does, right. It’s another if the bad guys are able to access your pacemaker, or the elevator that you’re riding in, and so not surprisingly, once IT systems begin to have an effect in the physical world, it really shouldn’t surprise anyone that the government starts playing a greater role in their security. And, so, I certainly think that the appetite and interest and frankly, capability of the government to have those conversations is much greater than it was, say, a decade ago.

Eric Greenwald: Right? Well, it seems like the lead vehicle, but by no means the only one through which government is pressing into this space is Executive Order 14028. And, you know, we're now at a place where that executive order dropped over a year ago, and the directions to the government, for the actions it was supposed to take in advancing cybersecurity regulations, again, limited to, you know, government procurement of software. They’re supposed to be quite a long ways away, in producing that those standards, the regulations, whatever they’re going to be, how do you feel the government has done in moving this ball forward from the concept that they initially rolled out a year ago, to where they are now in implementation?

Michael Daniel: You know, I think they’re actually doing a pretty good job of moving this along. And the this in this case is quite complex, right, and I’ve forgotten exactly how many taskings there were in that executive order?

Eric Greenwald: A lot.

Michael Daniel: Yeah, a lot, especially because it’s not my job to track that sort of thing anymore. But the, you know, but there were a lot of them, right. And they were all complex. And anytime you're talking about, you know, updating things like the federal acquisition regulations and policies like that, that takes considerable amount of time. But I think you can really see that they are making progress against … against the taskings in that executive order.

Eric Greenwald: Well, I’ll tell you, one of the things that I we talk about a lot on this podcast, and I want to dig into a little bit more with you is the software bill of materials, and where that’s going. But before we get there, there’s a … there’s sort of a big-ticket item that's been the subject to conversation of late. And that is, when there are requirements that come out of the executive order, whether the companies that are subject to them, if their compliance is going to be proven by an attestation by a third party, independent third party, or if they will only be required to attest themselves, that is first-party attestation. And the signal that we’re seeing right now suggests that it's going to be first-party attestation. And I wanted to get your thoughts on that on why you think the government is going in that direction. And whether that is concerning or not concerning to you.

Michael Daniel: Part of the reason for that, right, is, if you’re talking about cybersecurity and cybersecurity products, we actually have a really difficult time measuring efficacy of cybersecurity products right now. It’s actually really hard when you think about it, like, and we don't have a set of agreed measures, so that you can say, you know, this has a consumer ratings report, you know, score of 4.8, you know, or whatever, on the cybersecurity product. And, you know, there are some organizations out there that are trying to look at this issue. But when you think about that, what that means is that right now, third-party testing is very difficult to do. And it's very difficult to have it have any meaning. Which is, which is why I think the government wants it like it doesn’t want to put itself in a box where like, sure you’re doing third-party testing, but it tells me nothing. So now I’ve gone through a whole lot of, you know, effort and to get information that doesn’t actually help me. Eventually, eventually, Eric, I do think that we’re going to need third-party testing and validation, particularly of cybersecurity products. And like I said, there are some … there’s some new companies out there that are trying to move into that space, right, things like cyber ratings and, you know, organizations like that, but they’re not fully mature and, like fully robust and developed yet.

Eric Greenwald: Right. Well, and it does presuppose one question that is, as you reference, how meaningful or effective is the technical standard that any particular company is being asked to attest compliance with? And, you know, we’ve certainly seen some consternation over whether it's the level of detail, the level of transparency into these specific technical standards that are coming out of the executive order. But there’s another angle from which I can understand concern about trying to do third-party certification. And that is the example of CMMC, which if I remember correctly stands for the Cyber Cybersecurity Maturity Model that the Department of Defense implemented or I should say, attempted to implement, a number of years ago, which did require third-party certification, and has been tangled in a hopeless morass. So, I was curious to ask you, you know, what your thoughts are about, you know, a little bit about the future of CMMC, whether we’re seeing the executive order supplanted it, or, you know, also the extent to which you think maybe the folks who are implementing the executive order have learned from the troubles that the CMMC has encountered?

Michael Daniel: Yeah, I mean, I think that’s a good question, Eric. When you think about it there, again, just like we were talking about, when it comes to the, you know, cybersecurity products like measuring cybersecurity-ness of an organization, right, is still really hard. And we don’t have a lot of good performance measures to actually measure that. And again, that’s why I think when you when you’re in that situation, it's hard to do third-party certification. Right, because the, you know, what is the third-party measuring against? Right, and how does that differ from what you can get to through attestation? Right. Now, there’s some, you know, there are reasons to have third party looks like, you know, if you remember, like, if you remember the whole retirement of Windows 7 machines. Remember, we were in government? And you remember the discussions with the agencies where they would swear on a stack of Bibles that like they had, you know, they had no Windows 7 machines. And then, you know, our Director of Federal Cybersecurity would be working with OMB, and it’d be like, what about these 30,000 machines that we found? Oh, those machines?! So, there is a, you know, there is a reason, right, why you sometimes want some third-party review of, you know, what organizations are saying, but, but like, in that case, the example that I was giving you there was a clear, there was a clear thing that you were looking at, right, like, does this machine have an operating system installed? Right, what kind is it? That is a thing that you can measure, you can look at, and you can understand, whereas there’s still a lot of in the cybersecurity space, that is very difficult to do. And I think that’s why CMMC has run into trouble. Another reason I think it’s run into some trouble is like, it’s hard to account for all the variety of compensating controls. Right, like, sure. Okay, fine. So, they’re still running some vulnerable systems, but have they put in the right compensating controls to manage that risk? It’s still really complicated. I don’t think anybody should actually be shocked that the first time out of the gate, something like CMMC has had some troubles.

Eric Greenwald: Right? Well, and I think that it’s certainly in this DNA to try not to shock the private sector system, right, with the rule systems that it rolls out. And the idea that they’re, you know, dipping their toe a little bit, whether it’s in you know, the early on, we saw that they were trying to stage the scoping of the executive order so that it didn’t start out covering the waterfront, but just critical software to begin with, even though I don’t think that that ever got really well-defined. But now also the idea of starting with just first party attestation, rather than trying to jump into what is proven to be extremely complicated and third party, I can understand why they why they went that route.

Michael Daniel: Yeah. And I also think, you know, it’s easy to sort of poke fun at first party, you know, attestation on the other hand, even in accounting, right, there’s a lot of things that even with auditors, there’s a lot of things in the accounting world, in the business world, right? Where you’re attesting to something, right? And if you don’t think the CFOs of companies take that really darn seriously when they’re like, it’s their name and their reputation on the line for that, you know? So, there’s value in that attestation of having somebody formally sign up and say, no, really, we’re complying with these standards, because you can be held accountable for that later on. If it’s shown that you’re not.

Eric Greenwald: Absolutely no, I don’t want to diminish first party attestation and suggest that it is nothing. It’s not third-party certification. But as you say, there’s a lot of complexities associated with third-party attestation. I will say, though, that the thing that I focus on with first party attestation is what you alluded to earlier, and that is, does the person making the attestation actually even know what they’re talking about? And I can’t resist using that as a segue way back to the topic of the software bill of materials. Because, to me to my assessment, that is potentially the most significant contribution to improving cybersecurity practices that the executive order might be able to make. And that is, you know, promoting, at least within its limited scope, wider-spread adoption and implementation of the SBOM. And, you know, as they say, one of the key elements of design of the SBOM is so that you actually have an inventory of what’s in your software product that you’re selling to the government, or in other contexts, because the SBOM … it is used, it’s just not required, or particularly in wide adoption these days. And right now, we are seeing that CISA is moving forward with the SBOM listening sessions that it promised. Those are going to be taking place in July. I’m curious to know whether you have a sense as to the industry reaction to the fact that SBOM seems to be in certainly the topic of the day, the flavor du jour. As far as cybersecurity practices go, are you seeing strong reactions one way or the other in your interactions with private sector practitioners?

Michael Daniel: I mean, I think that in general, most of the serious practitioners recognize that you need something like SBOM, right that, like, why was Log4j such a problem? Right, for example. And part of the reason it was such a problem was it was everywhere, but you didn’t know where. Right, and having to spend a lot of time trying to figure that out, you know, slowed down the response. And so, everybody recognizes that you, we’ve got to have something like the SBOM. Now, I think there's a whole variety of reactions of people who … they would have a different way of, you know, collecting that information or whatever. So, you can see some of that kind of grousing, but I think in general, the recognition is that, yeah, it’s gonna be a pain. But we need something like the SBOM in order to really get a handle on what’s actually in all of these products.

Eric Greenwald: And, and we’re starting to see the SBOM creep into other forms of potential government regulation, whether it’s FDA recommendations for cybersecurity, or some legislate draft legislation that has been popping, that seems to be oriented around trying to legislate some of the standards and processes that FDA is recommending. The Patch Act is probably the best example where medical device manufacturers, under that law, under that bill, were it to become law, medical device manufacturers would need to provide an SBOM for their medical devices. My question to you then is the SBOM coming regardless as to what happens with the executive order and how decisively it implements, or the level of technical detail with which implements any sort of SBOM requirement?

"So much code gets used and reused. We must understand the provenance of our software and make a rational judgement about its security." - Michael Daniel

Michael Daniel: Oh, yeah, this idea that you should be able to understand the content of the software that you’re using, just in the same way that you should be able to understand the content of a physical product or device. I just think that it makes so much sense. And it’s addressing a key weakness in a lot of our cybersecurity now, and, you know, there's so much code that gets used and reused right in from different libraries and different things that you really have to understand the provenance of that of that software in order to make any sort of, you know, rational judgment about its security.

Eric Greenwald: Right. Well, I think what I’ve heard a lot it’s not the magic bullet to addressing cybersecurity supply chain issues, but we have so little transparency into the cybersecurity supply chain. And SBOM, at least if implemented well has the potential to provide a significant degree of greater transparency into the cybersecurity software supply chain.

Michael Daniel: You got to start with information, which, right now, you can’t even make good policy because of that lack of transparency, right. That, to me is a key is a key element.

Eric Greenwald: Well, Michael, I really appreciate you taking the time to talk with us. What I’d like to do is probably have you back on once we get a little bit more clarity about where things are going with the executive order, to talk through how it’s actually going to be implemented, or, you know, where we’re going to see other regulatory initiatives pop up. So, thank you very much. Appreciate it greatly.

Michael Daniel: Yeah, no, thank you for having me. I enjoyed the conversation.