The problems with vulnerability reporting

Igor Omilaev / Unsplash

Several recent incidents in the U.S. system for reporting vulnerabilities highlight the importance of accurate, comprehensive bug reports for defenders and the need for a better, more flexible system for rectifying inaccurate records in central vulnerability reporting systems.

The U.S. system for identifying, defining and cataloging publicly reported vulnerabilities is a lynchpin enabling cybersecurity defenders at all organizations to stop digital threats before they become catastrophic. Defenders across public and private organizations rely on these systems to prioritize and mitigate the thousands of vulnerabilities reported yearly to keep their digital infrastructure safe.

Several recent incidents highlight how the systems in place for reporting common vulnerabilities and exposures, or CVEs, play a crucial role in promptly identifying and prioritizing the flaws so defenders can avert compromises. These incidents also illustrate how inadequately reporting and cataloging vulnerabilities could lead to a potential cascade of missed opportunities to fix dangerous bugs.

Moreover, in some cases, the current opaque bug reporting system can lead to the irreversible overclassification of harmless software bugs as vulnerabilities, leaving software makers with little recourse for correcting these mistakes and forcing them to spend time dealing with the knock-on effects of these errors. Conversations with experts provide no easy solutions for how to fill these cracks in the vulnerability reporting systems.

How the CVE reporting system works

The CVE reporting system is based on security researchers, software providers or any other interested party submitting vulnerability reports primarily to two unbiased, widely relied-upon CVE numbering authorities (CNAs). One primary CNA is MITRE, a federally funded R&D center (FFRDC) that maintains a CVE list. The other major CNA is the National Institute of Standards and Technology (NIST), which keeps the National Vulnerability Database (NVD). Cyber defenders rely heavily on both resources to learn more about vulnerabilities that affect their systems and networks.

Each vulnerability is assigned a unique ID or number by MITRE or NIST and a severity score according to a complex rubric called the common vulnerability scoring system, or CVSS. A CVSS score can range from zero to 10, with 10 representing the most severe vulnerability. Vendors, researchers, open-source software providers, CERTs, bug bounty provider organizations and others authorized by the CVE Program can also assign IDs and scores to CVEs.

Currently, 324 CNAs (322 CNAs and 2 CNA-LRs, or CVE Numbering Authorities of Last Resort, for vulnerabilities not covered by other CNAs) from 37 countries are participating in the CVE Program. (The Cybersecurity and Infrastructure Security Agency is also a CNA and maintains a widely used catalog of known exploited vulnerabilities, or KEVs, that many federal agencies must remediate within a specified period of time.)

When a security researcher or software vendor discovers a vulnerability, they typically try to report it to the relevant software provider to produce a fix for the flaw before submitting a public report, a process that usually takes around 90 days. But in some circumstances, where the flaw is not just hypothetically exploitable but actively being exploited in the wild, reporting vulnerabilities can become frenetic.

Apple versus Google CVEs regarding the same vulnerability

One such complex situation occurred recently after researchers at Citizen Lab reported on Sept. 7 that they found an actively exploited one-click zero-day vulnerability in Apple's iOS called BLASTPASS. BLASTPASS is an exploit chain consisting of two zero-day flaws that a malicious actor leveraged to infect an iPhone of a Washington, DC-based civil society organization with NSO Group's Pegasus spyware. One component of BLASTPASS was a buffer overflow vulnerability in ImageIO, Apple's image parsing framework, which contains WebP, an image format.


Brett Jordan / Unsplash

Many software providers widely use WebP, and it was a WebP 0-day that allowed threat actors to gain remote execution capabilities through malicious images. Apple issued a security report the same day, assigning the vulnerability two IDs, CVE-2023-41064 (the WebP 0-day) and CVE-2023-41061.

Google is the primary open-source maintainer for the libwebp library that contains WebP, with most, if not all, of the people developing the WebP code employed by Google. Google uses libwep in Chrome. Five days after Apple patched its flaw and issued its report, Google published its own vulnerability report, assigning the flaw a different ID of CVE-2023-4863, which experts say was the same flaw Apple reported.

But in the process of submitting its report, Google identified the flaw in what is known as the Common Platform Enumeration (CPE) as affecting Chrome, in contrast to Apple's report, which identified the flaw in its CPE as affecting Apple's internal ImageIO framework, where the WebP-0 day was. Therefore, defenders running vulnerability scanners on their organizations initially missed many other instances where the vulnerabilities appeared because the CVEs applied to only Apple or Chrome and not the libwebp library.

NIST ultimately clarified its entry for CVE-2023-4863 to reflect the broader reach of libwebp. This confusion reportedly caused some defenders delays in patching the many other software products and applications affected by the vulnerability because they were getting incorrect read-outs from their scanning systems.

Some vulnerability experts have adopted a forgiving attitude toward the two companies, given the urgency in shipping out a patch to a billion iPhone users and then shipping one out to another billion or so Chrome users a few days later. "Perhaps Google and Apple didn't do a great job after the fact, and once the dust had settled, had to go back and correct the record," one top expert told README.

But, he said, "I think that if I were in Apple's position, I'd probably do what they did, and if I were in Google's position, I'd probably do what they did. I don't think anything from a human level has gone particularly wrong there in terms of the decision-making. It does seem like situations like this can arise if you're making decisions that are in the best interest of user security."

If anything, the NVD analysts might bear much of the responsibility for any adverse downstream effects, the expert said. "What we needed to have happened there was that the NVD analyst had to select, okay, this bug affects an open-source library called libwebp. Then once that's in the system, many other products and services can take that machine-readable description of this bug affecting this library and do all that supply chain analysis."

Complaints of 'phony' vulnerabilities

Another glitch in the vulnerability reporting system can arise when anonymous reporters outside a software organization submit a CVE to NIST or MITRE that the software developer contends is not a vulnerability. One such case occurred recently involving the open-source command-line tool called curl.

In late August, curl maintainer Daniel Stenberg complained in a blog post that, to his surprise, he recently learned that an anonymous individual filed a curl vulnerability report, CVE-2020-19909, to NIST and MITRE for a flaw with a CVSS score of 9.8, very close to the worst possible score. The CVE was dated 2020, an oddity given that most flaws are reported the year that they are submitted. (This flaw is unrelated to a recent and more serious flaw reported by curl).

The 2020 report related to an integer overflow vulnerability. In 2019, a researcher from HackerOne reported an overflow problem in curl's "--retry-delay" command line option, which Stenberg maintains was a bug but not a security vulnerability. Curl fixed the bug in 2019, a month after the researcher reported it. "The user found a bug that we fixed in Curl in 2019, and we had already discussed if it was a security problem or not," Stenberg told README. "And we concluded that it wasn't a security problem, so we just fixed it, released a new version, and went on with our lives."

Despite Stenberg's repeated efforts to remove the flaw from both organization's databases, neither NIST nor MITRE have done so, although NIST downgraded the CVSS score to 3.3. Part of Stenberg's problems in dealing with both NIST and MITRE is the lack of transparency and the inability to communicate with the two CNAs except through their anonymous reporting portals.

"There's no transparency," he told README. In terms of his efforts to rectify the misclassification with MITRE, Stenberg said, "There's just an alias that responds that says, no, no, we think it's a security problem. End of discussion, no name [of the person saying this], nothing." (README contacted both NIST and MITRE for comment through those same portals and received anonymous responses suggesting we contact their organizations' press departments. README contacted the press departments at both NIST and MITRE and received no response.)

The problem with curl's CVE staying in the NIST and MITRE systems, particularly with a score of 9.8, is that system administrators running vulnerability scans kick out curl, throwing a monkey wrench into other parts of their systems' functioning. "I've had users contact me, and they say that they have these requirements according to some contract that they've signed, that they're obliged to fix all their CVEs above a certain threshold within X number of days, so they're forced to fix them, but it's not actually a problem," Stenberg said.

"That leads to weird things. I had a problem with people finding these kinds of problems in the Windows version of curl, with users removing curl from Windows and replacing it. Then they ended up very sad because that broke Windows update," Stenberg said. "And then they contacted me with very desperate questions about how to fix the Windows update again, which I, of course, had no clue how to do."

Stenberg has decided the only way he can fix this problem is for curl to become a CNA itself so that any future vulnerability reports are routed to curl directly. "Since I started whining about this problem, the only way I have figured out that I can help myself here is to register our project ourselves as a CNA within the CVE system," he said. "It's a slow process, but that's what I'm going to do. Hopefully, this will [fix the problem] so that others cannot create future CVEs for curl without me knowing about it at least."

In another recent case of a "phony" vulnerability, the PostgreSQL Security Team was made aware of CVE-2020-21469, which someone filed without the prior knowledge of the PostgreSQL Security Team. The team claims it is not a vulnerability. According to the team, the CVE says creating a denial-of-service in PostgreSQL 12.2 is possible by sending repeated SIGHUP (or reload) signals to the primary PostgreSQL process.

The PostgreSQL Security Team requests that anyone who suspects a flaw in their software contact them directly for evaluation, adding that it works "with all reporters to determine what is a valid vulnerability and to provide transparency to our users around security issues." README contacted for comment but did not receive a response.

No clear path to solving these problems

When it comes to fixing the vulnerability reporting system to minimize these kinds of incidents, there doesn't appear to be a clear path to solving the problems. "I will put out there it's a hard and difficult problem because different people want different information from these reports," David Brumley, CEO of ForAllSecure, told README.

"I think top-most that you want to know if a vulnerability is being exploited in the wild. That's the first thing I think people want to understand. And then, if there is an exploit for it, what are the attackers able to do with that exploit?"

Brumley thinks that "CVEs and CVSS scores have kind of just become a resume game where there's a number of things out there that we know that it's important to file, and because of that, people will file them just to pad their resume."

Regarding phony CVE reports, "what sometimes breaks is CVEs get confirmed before the developers even know about it," Brumley said. "In the case of curl, these are very active, good people, and they will confirm or not confirm a vulnerability, and we kind of trust their opinion as the experts."


Hans-Peter Gauster / Unsplash

Brumley aims his sharpest criticism at MITRE. "We have this federally funded research and development center, this kind of quasi-government organization running this. It's not really the government. They're not beholden to the same rules and regulations. And for a long time, that was kind of good enough. I think what happened along the way is MITRE doesn't really feel like it has to be responsible to anyone. I feel that MITRE thinks this is just money they'll get every year to maintain it."

In the case of curl, "MITRE was basically saying you don't know your own code. You're wrong. That would be kind of insulting if you're writing the code and you're told, this isn't a vulnerability, and people are like, 'no, you're wrong.' What was funny is when MITRE said, no, you're wrong, it wasn't even signed by a human. There was no way actually to dispute this."

The same inability to dispute CVEs also applies to the NVD database, which currently contains over 200,00 CVEs. But, "the first thing you have to understand is that [what the NVD] analysts are entering into that database entry, it really is like a best effort approach," one expert told README. "They're trying to do this at scale. There are thousands, tens of thousands of CVEs being published each year, which they're trying to keep on top of and provide some sort of useful signal to everybody who's consuming the database."

The expert added, "We're not expecting the NIST analyst to be an expert to the level" of those who deal with cybersecurity vulnerabilities at the upper-most echelons in the industry. "The economics doesn't make sense just because of the scale they're working with and the amount of time that would be practically allocated for each CVE." He said, "I think that the social contract here is probably not particularly well understood."