Next, we present brief definitions of common terms used throughout the article:
接下来，我们给出文章中常用术语的简要定义：

Table 1 summarizes six related works whose survey partially overlaps ours. The first two columns of Table 1 indicate the Reference and Year. The next three columns show the three major categories in our survey. An “X” indicates that the related work includes some of the same literature as our survey for that category. We provide an overview of these categories in Section 5. Next, the Summary column provides a brief overview of the related survey. Finally, the last column explains how we expand upon and differ from the prior work.
表 1 总结了六个相关研究，其调查部分重叠于我们的研究。表 1 的前两列表示参考文献和年份。接下来的三列展示了我们调查的三个主要类别。一个“X”表示相关研究包含与我们调查该类别相同的部分文献。我们在第 5 节对这些类别进行了概述。接下来，摘要列提供了相关调查的简要概述。最后，最后一列解释了我们如何扩展和区别于先前的工作。

As seen in Table 1, the prior work with the greatest overlap to our study is by Pendleton et al. [91]. The categorization of the studies in our survey is heavily influenced by Pendleton et al., which we discuss further in Section 4.2. However, there are also many key differences between our study and theirs. More than half (49 out of 76) of the papers we surveyed were published after 2016, which is the year of publication for the Pendleton et al. study. There are differences in scope, which also reduce the overlap. Pendleton et al. focus on metrics, whereas we focus on the assessment methods used to produce scores like metrics. Pendleton et al. examine system-level metrics, whereas our primary focus is vulnerability-level assessment. Finally, Pendleton et al. focus on all security assessments, whereas we focus specifically on the topic of exploitability, a subset of security. Focusing on exploitability allows us to examine assessments in greater detail. As shown in Table 1, the remaining surveys overlap within only one category.
如表 1 所示，与我们研究重叠度最大的先前工作是 Pendleton 等人[91]的研究。我们调查中的研究分类受到 Pendleton 等人研究的强烈影响，我们将在第 4.2 节中进一步讨论。然而，我们的研究与他们的研究也存在许多关键差异。我们调查的论文中，超过一半（76 篇中的 49 篇）是在 2016 年之后发表的，而 Pendleton 等人的研究是在这一年发表的。在范围上存在差异，这也减少了重叠度。Pendleton 等人专注于指标，而我们的重点是用于产生如指标等评分的评估方法。Pendleton 等人考察系统级指标，而我们的主要重点是漏洞级评估。最后，Pendleton 等人关注所有安全评估，而我们专注于安全的一个子集——可利用性。关注可利用性使我们能够更详细地考察评估。如表 1 所示，剩余的调查仅在某一类别内重叠。

CVSS v3.1 has three metric groups: Base, Temporal, and Environmental [25]. The Base metric group includes characteristics of the vulnerability that are constant, regardless of context. The Temporal metric group captures the characteristics of a vulnerability that can change over time, such as whether an exploit has been made publicly available. The Environmental metric group includes vulnerability characteristics that may vary based on the environment, such as whether a vulnerable library is being used by an application that manages sensitive data.
CVSS v3.1 有三个度量组：基本、时间和环境[25]。基本度量组包括与漏洞相关的恒定特征，无论在何种环境下。时间度量组捕捉到随时间可能变化的漏洞特征，例如是否已公开利用。环境度量组包括可能根据环境变化的漏洞特征，例如是否存在一个易受攻击的库被管理敏感数据的应用程序使用。

In CVSS v3.1, the Base score is derived from two sub-scores, Exploitability and Impact, and adjusted based on a third Scope score. For this survey, within the Base score, we only consider the Exploitability score and its sub-metrics to be “exploitability related.” The Exploitability sub-score [25]includes four metrics: Attack Vector (AV) indicates how close an attacker must be to be able to exploit the vulnerability; Attack Complexity (AC) indicates whether exploiting the vulnerability requires conditions that the attacker does not directly control; User Interaction (UI) indicates whether a user, other than the attacker, is required to provide inputs to the application for the attacker to exploit the vulnerability; and Privileges Required (PR) indicates the level of logical access needed to exploit the vulnerability. In CVSS v3.1, the Base Exploitability sub-score is calculated as $8.22\times AV\times AC\times UI\times PR$.
在 CVSS v3.1 中，基础分数由两个子分数组成，即易损性和影响，并根据第三个范围分数进行调整。对于本次调查，在基础分数中，我们只考虑易损性分数及其子指标为“易损性相关”。易损性子分数[25]包括四个指标：攻击向量（AV）表示攻击者必须接近到何种程度才能利用漏洞；攻击复杂性（AC）表示利用漏洞是否需要攻击者无法直接控制的条件；用户交互（UI）表示是否需要除攻击者以外的用户向应用程序提供输入以使攻击者能够利用漏洞；所需权限（PR）表示利用漏洞所需的逻辑访问级别。在 CVSS v3.1 中，基础易损性子分数的计算公式为 $8.22\times AV\times AC\times UI\times PR$ 。

The Temporal group includes the Exploit Code Maturity (E) metric, which “measures the likelihood of the vulnerability being attacked” [25]. The other Temporal metrics, Remediation Level (RL) and Report Confidence (RC), are not related to exploitability.
时间组包括漏洞利用代码成熟度（E）指标，该指标“衡量漏洞被攻击的可能性” [25]。其他时间指标，修复级别（RL）和报告置信度（RC），与可利用性无关。

The Environmental metric group [25] includes modified scores of all Base metrics, including the Exploitability metrics: Modified Attack Vector (MAV), Modified Attack Complexity (MAC), Modified User Interaction (MUI), and Modified Privileges Required (MPR). The remaining Environmental metrics are based on Impact. The modified scores update the Base Metric sub-scores based on environmental factors such as using a non-default configuration setting [25].
环境指标组[25]包括所有基础指标的修改得分，包括可利用性指标：修改后的攻击向量（MAV）、修改后的攻击复杂度（MAC）、修改后的用户交互（MUI）和修改后的所需权限（MPR）。其余的环境指标基于影响。修改后的得分根据环境因素（如使用非默认配置设置[25]）更新基础指标子得分。

Many of the publications surveyed refer to CVSS v2 since CVSS v3 was not released until 2015 [23, 101] and not officially supported in the NVD until 2019 [89]. In the Base Exploitability group of CVSS v2 [23], the Attack Vector (AV) was referred to as the Access Vector (also abbreviated AV). In CVSS v2, the Attack Complexity (AC) and User Interaction (UI) metrics were combined in the Access Complexity metric (also abbreviated AC). Finally, instead of a Privileges Required (PR) metric, CVSS v2 had an Authentication (AU) metric which recorded the minimum number of times an attacker had to provide credentials to an application when exploiting the vulnerability. The Base score CVSS v2 also lacked the Scope (S) sub-metric [23]. The CVSS v2 Base Exploitability metric is also calculated $20xAVxACxAU$—that is, using a slightly lower weight than CVSS v3 (20 vs. $8.22$).
许多被调查的出版物都提到了 CVSS v2，因为 CVSS v3 直到 2015 年才发布[23, 101]，并且在 NVD 直到 2019 年才正式支持[89]。在 CVSS v2 的基漏洞利用组[23]中，攻击向量（AV）被称为访问向量（也简称 AV）。在 CVSS v2 中，攻击复杂性（AC）和用户交互（UI）指标被合并为访问复杂性指标（也简称 AC）。最后，CVSS v2 没有所需的权限（PR）指标，而是有一个认证（AU）指标，该指标记录了攻击者在利用漏洞时向应用程序提供凭证的最小次数。CVSS v2 的基分数也缺少范围（S）子指标[23]。CVSS v2 的基漏洞利用指标也是按照 $20xAVxACxAU$ 计算的——即使用略低于 CVSS v3 的权重（20 vs. $8.22$ ）。

In the Temporal metric group, the name of the metric referred to as “Exploitability” in CVSS v2 was updated to “Exploit Code Maturity” (E) in v3 to better reflect what the metric evaluates [23]. Additionally, in the Environmental Metric, CVSS v2 had two distinct sub-metrics: Collateral Damage Potential (CDP) and Target Distribution (TD), which assessed the impacts of exploiting a vulnerability. The CVSS v2 Environmental metric did not include the Modified Base scores.
在时间度量组中，CVSS v2 中被称为“可利用性”的度量名称在 v3 中更新为“利用代码成熟度”（E），以更好地反映该度量所评估的内容[23]。此外，在环境度量中，CVSS v2 有两个不同的子度量：附带损害潜力（CDP）和目标分布（TD），它们评估了利用漏洞的影响。CVSS v2 的环境度量不包括修改后的基本得分。

The Base CVSS score is widely available for all vulnerabilities in the CVE list since the NVD [89] adds a Base CVSS score for each CVE. Some vendors, such as Oracle,² also use the Base CVSS score to communicate the severity of vulnerabilities in their systems.
基础 CVSS 评分对所有 CVE 列表中的漏洞都广泛可用，因为 NVD[89]为每个 CVE 添加了基础 CVSS 评分。一些供应商，如 Oracle， ² 也使用基础 CVSS 评分来传达其系统中漏洞的严重性。

Criticisms of CVSS include the lack of environmental or temporal information in the Base CVSS score [81], and the weakness of the score’s correlation with exploit likelihood and overall risk [5, 6, 14, 136]. These have been addressed in later versions of the CVSS specification, which emphasizes that the Base score is not intended to include temporal or environmental factors, and “CVSS is designed to measure the severity of a vulnerability and should not be used alone to assess risk” [26]. Other criticisms include the unequal distribution of CVSS scores for vulnerabilities in the CVE list [80, 115], where most CVSS scores are relatively high. Several aspects of the score distribution remain unexplored, such as whether the uneven distribution is a product of the source (i.e., whether vulnerabilities in the CVE list tend to be higher severity) or of CVSS itself.
CVSS 的批评包括基线 CVSS 评分中缺乏环境或时间信息[81]，以及评分与利用可能性和整体风险的关联性较弱[5, 6, 14, 136]。这些问题在 CVSS 规范的后续版本中得到了解决，强调基线评分不包括时间或环境因素，“CVSS 旨在衡量漏洞的严重性，不应单独用于评估风险”[26]。其他批评包括 CVE 列表中漏洞的 CVSS 评分分布不均[80, 115]，其中大多数 CVSS 评分相对较高。评分分布的几个方面尚未探讨，例如不均匀分布是否是来源（即 CVE 列表中的漏洞是否倾向于更高的严重性）或 CVSS 本身的结果。

We followed a two-phase process for collecting relevant papers, involving both a Keyword Search using Active (Machine) Learning phase and Snowballing phase, based on the SYMBALS methodology [120]. Proponents of using active learning for literature surveys found that active learning alone can perform similar to human researchers, with precision and recall greater than 70% [120, 139]. Combining active learning with Snowballing can produce precision and recall greater than 90% [120].
我们采用了基于 SYMBALS 方法[120]的两阶段收集相关论文的过程，包括使用主动（机器）学习的关键词搜索阶段和滚雪球阶段。支持使用主动学习进行文献综述的学者发现，仅主动学习就能与人类研究人员的表现相似，其精确率和召回率均超过 70%[120, 139]。将主动学习与滚雪球方法相结合，可以产生精确率和召回率均超过 90%[120]。

The inclusion/exclusion criteria used in both phases of the review are discussed in Section 4.1.1. In the first phase, we use an active (machine) learning tool, FAST2 [139], to analyze the results returned from a keyword search, followed by further review and analysis by three researchers. In the original SYMBALS research by van Haastrecht et al. [120], the authors found that FAST2 consistently performed better than the alternative learning system, ASReview, in terms of recall (60%–90% compared to 40%–70%). We discuss the first phase further in Section 4.1.2. In the second phase, addressed in Section 4.1.3, we performed snowballing [120] to collect the papers citing and the papers cited by the studies returned in the first phase. An overview of the paper collection process is shown in Figure 2.
审查的两个阶段所使用的纳入/排除标准在 4.1.1 节中讨论。在第一阶段，我们使用一个主动（机器）学习工具 FAST2 [139]来分析关键词搜索返回的结果，随后由三位研究人员进行进一步审查和分析。在 van Haastrecht 等人[120]的原始 SYMBALS 研究中，作者发现，在召回率方面，FAST2 始终优于替代学习系统 ASReview（60%–90%与 40%–70%相比）。我们将在 4.1.2 节中进一步讨论第一阶段。在第二阶段，如 4.1.3 节所述，我们进行了滚雪球[120]以收集第一阶段返回的研究引用的论文和被引用的论文。论文收集过程的概述如图 2 所示。

Once we had identified the set of papers, we grouped papers that were part of the same study and should be considered together in a practical application of the exploitability assessment. We consider papers to be part of the same study if the papers (1) were from the same authors and include some of the same analysis, and (2) if later papers were building on the prior model for exploitability. We reviewed the studies to determine if prior work existed and if it would be necessary to read to understand the assessment technique proposed, but where the prior paper would not have been classified as “exploitability” related. For example, earlier work built on by Plate et al. [95] and Ponta et al. [96–98] did not refer to the tools in terms of “exploitability.” Searching for additional papers in each study added 7 papers to the survey. Our survey ultimately includes 76 papers, 72 of which are grouped into 59 studies, whereas 4 papers were proposing or presenting industry standards CVSS and the Exploit Prediction Scoring System (EPSS), written by the authors of the standards.
一旦我们确定了论文集，我们将属于同一研究的论文分组，这些论文在可利用性评估的实际应用中应被视为一组。我们认为如果论文（1）来自同一作者并包含一些相同的分析，以及（2）如果后续论文是在利用先前模型的基础上构建的，则这些论文属于同一研究。我们审查了这些研究，以确定是否存在先前的工作，以及是否需要阅读以理解所提出的评估技术，但先前论文不会被归类为“可利用性”相关。例如，Plate 等人[95]和 Ponta 等人[96-98]所基于的早期工作没有将工具称为“可利用性”。在每个研究中寻找额外的论文增加了 7 篇论文到调查中。我们的调查最终包括 76 篇论文，其中 72 篇被分为 59 项研究，而 4 篇论文是提出或展示行业标准 CVSS 和可利用性预测评分系统（EPSS），由标准作者撰写。

After the initial keyword search, an initial classification of papers was performed by two researchers and iterated upon further by the first author. We used a keywording-based approach [94] to determine the categories and characteristics useful to compare and contrast the different vulnerability assessment methods. The first author identified a set of keywords that applied to a wide range of papers, and the first and third authors then applied these keywords to a random subset of 15 papers and identified new keywords that were frequently occurring. The keywords were then grouped into categories and applied to the initial set of 38 papers. The keywords and categories were further iterated on and expanded during and after Phase 2.
在初步关键词搜索之后，两位研究人员对论文进行了初步分类，第一作者在此基础上进一步迭代。我们采用基于关键词的方法[94]来确定用于比较和对比不同漏洞评估方法的类别和特征。第一作者确定了一组适用于广泛论文的关键词，然后第一和第三作者将这些关键词应用于 15 篇随机选取的论文子集，并识别出频繁出现的新关键词。随后，这些关键词被分组归类并应用于最初的 38 篇论文集合。在第二阶段期间和之后，关键词和类别进一步迭代和扩展。

Our categories aligned closely with three categories for “Metrics for Measuring Severity” of software vulnerabilities from Pendleton et al. [91]: (Manual) CVSS-based, Deterministic, and Probabilistic assessment systems. The vulnerability-related subset of the taxonomy from Pendleton et al. [91] is shown in Figure 3, with the three categories we use in our survey highlighted in red.
我们的类别与 Pendleton 等人[91]提出的“衡量软件漏洞严重程度的指标”中的三个类别紧密对应：（手动）基于 CVSS 的、确定性评估系统和概率评估系统。Pendleton 等人[91]的分类学中的漏洞相关子集如图 3 所示，我们调查中使用的三个类别用红色突出显示。

Based on our initial keywording and our comparison with Pendleton et al. [91], we identified three primary categories of studies based on the techniques used in each study: (Manual) CVSS-based, Deterministic, and Probabilistic assessment systems. We describe our categories in Section 5.
基于我们最初的关键词筛选以及与 Pendleton 等人[91]的比较，我们根据每项研究中使用的技巧确定了三个主要研究类别：（手动）基于 CVSS 的、确定性和概率性评估系统。我们在第 5 节中描述了我们的类别。

In S34, Allodi et al. [3] evaluate what information is helpful in determining the CVSS Base score with a user study of CVSS v3. Using the CVSS scores provided in the CVSS v3 example document [24] as ground truth, the researchers provided study participants with a tutorial on determining CVSS scores. The control group was then asked to determine CVSS scores based on the original vulnerability descriptions from NVD alone. The treatment group was given vulnerability descriptions with additional information from the CVSS example document [24]. The authors examined four categories of information: information about the vulnerable asset (i.e., information on the type of system directly affected by the vulnerability), attack procedures that could be used against the vulnerability, vulnerability type information characterizing the technical root cause and result of exploiting the vulnerability, and known threats (i.e. whether there is evidence that malicious actors have exploited the vulnerability on production systems). The researchers assessed how the addition and removal of each category of information impacted participants’ error rates. They found that information on assets reduced the error rate for the A (Availability) metric of the Impact group, but asset information had little other effect. Information on the attack reduced error rates for the AV and AC metrics in the Exploitability group and the C (Confidentiality) metric of the Impact group. Information on vulnerability type was related to a reduced error rate for AC, UI, and PR metrics in the Exploitability group. However, information about known threats was related to an increased error rate for AV and AC in the Exploitability group and C and A in the Impact group. Individual differences in performance between participants were observed—for example, information on vulnerability type reduced error rates more for individuals with more security expertise [3].
在 S34 中，Allodi 等人[3]通过 CVSS v3 的用户研究评估了在确定 CVSS 基础评分中哪些信息是有帮助的。使用 CVSS v3 示例文档[24]中提供的 CVSS 评分作为基准，研究人员向研究参与者提供了确定 CVSS 评分的教程。对照组被要求仅根据 NVD 的原始漏洞描述来确定 CVSS 评分。处理组则获得了包含 CVSS 示例文档[24]中额外信息的漏洞描述。作者考察了四个类别的信息：关于易受攻击资产的信息（即受漏洞直接影响系统的类型信息）、可用于针对漏洞的攻击程序、描述漏洞技术根本原因和结果的漏洞类型信息，以及已知威胁（即是否有证据表明恶意行为者在生产系统中利用了该漏洞）。研究人员评估了添加和删除每个类别信息对参与者错误率的影响。 他们发现，关于资产的信息降低了影响组 A（可用性）指标的误差率，但资产信息对其他方面影响不大。关于攻击的信息降低了可利用性组 AV 和 AC 指标的误差率以及影响组 C（机密性）指标的误差率。关于漏洞类型的信息与可利用性组 AC、UI 和 PR 指标的误差率降低有关。然而，关于已知威胁的信息与可利用性组 AV 和 AC 以及影响组 C 和 A 的误差率增加有关。观察到参与者之间在性能上的个体差异——例如，关于漏洞类型的信息对具有更多安全专业知识的人的误差率降低作用更大[3]。

In S23 (see [117]), the authors perform text mining on descriptions of CVEs in the NVD and analyze the correlation between terms and the values of the different CVSS v2 metrics (AV, AC, AU, C, I, and A) using Spearman’s rho. Their results somewhat support the analysis in S34. In S23, the authors found that the term “attack” had a moderate positive correlation (0.30) with AV and a moderate negative correlation (–0.35) with AU, whereas the correlation with AC was weak (0.02). However, terms associated with a particular type of attack, such as “cross-site-script” and “script” had a higher correlation with AC (0.42 and 0.36, respectively). Other terms that had a moderate or strong correlation with AV included “remote,” which had a positive correlation (0.53), as well as “local” and “user,” which had negative correlations (–0.70 and –0.49, respectively). Other terms that had a moderate correlation with AC included “html” and “web,” which both had positive correlations (0.40 and 0.36, respectively). Additionally, the terms “authent” and “user” had moderate to high positive correlations with AU (0.65 and 0.43, respectively). Correlation is not causation. We cannot be sure, based on S23 alone, how much these terms contribute to an individual’s ability to score the metric. However, when combined with work such as S34, we see a fuller picture.
在 S23（见[117]），作者对 NVD 中 CVE 描述进行文本挖掘，并使用 Spearman 的 rho 分析不同 CVSS v2 指标（AV、AC、AU、C、I 和 A）的术语与值之间的相关性。他们的结果在一定程度上支持 S34 中的分析。在 S23 中，作者发现“攻击”一词与 AV 有中等正相关（0.30），与 AU 有中等负相关（–0.35），而与 AC 的相关性较弱（0.02）。然而，与特定类型攻击相关的术语，如“跨站脚本”和“脚本”，与 AC 的相关性更高（分别为 0.42 和 0.36）。其他与 AV 有中等或强相关性的术语包括“远程”，其具有正相关（0.53），以及“本地”和“用户”，它们分别具有负相关（–0.70 和–0.49）。其他与 AC 有中等相关性的术语包括“html”和“web”，它们都具有正相关（分别为 0.40 和 0.36）。此外，“authent”和“user”这两个术语与 AU 有中等至高正相关（分别为 0.65 和 0.43）。相关性不是因果关系。 仅基于 S23，我们无法确定这些术语对个人得分能力贡献了多少。然而，当与 S34 等研究相结合时，我们看到了更完整的图景。

In S44, Allodi et al. [4] further examine how the actual expertise of the individuals applying CVSS influences the results. The authors use CVSS v3 for their experiment. They divided the participants into three groups based on expertise. The first group was graduate students in computer science with no security expertise. The second group was graduate students in computer science who had taken security courses but did not have industry expertise, and the third group was security professionals. The assessment was based on 30 examples randomly selected from 100 vulnerabilities used by the CVSS SIG when developing the standard [4]. The CVSS scores had therefore been determined previously by the CVSS SIG and could be used as “ground truth.” The authors found that computer science students who had taken security courses performed significantly better than students who had not taken security courses for the AC metric and the Impact metrics (C, I, and A). Computer science students who had taken security courses also had “borderline” better performance with the AV and PR metrics. Allodi et al. found no statistically significant difference between the two groups of students for the UI metrics. The authors found that security professionals were less likely to err on the UI metric compared with both groups of students. On the AV metric, there was a borderline difference between professionals and students who had taken security courses. However, the authors found no statistically significant difference in the performance of the students who had taken security courses and the security professionals across the AC, PR, C, I, and A metrics. Their results suggest that security knowledge is helpful in producing CVSS assessments. However, students with security experience were relatively competent, producing compatible scores with professionals for four of the six metrics.
在 S44 中，Allodi 等人[4]进一步研究了申请 CVSS 的个人实际专业知识如何影响结果。作者们使用 CVSS v3 进行实验。他们根据专业知识将参与者分为三组。第一组是没有任何安全专业知识的计算机科学研究生。第二组是已经修过安全课程但没有行业专业知识的计算机科学研究生，第三组是安全专业人士。评估基于从 CVSS SIG 在制定标准时使用的 100 个漏洞中随机选取的 30 个示例[4]。因此，CVSS 评分先前已由 CVSS SIG 确定，可作为“真实情况”使用。作者发现，修过安全课程的计算机科学学生在 AC 指标和影响指标（C、I 和 A）上表现显著优于没有修过安全课程的学生。修过安全课程的计算机科学学生在 AV 和 PR 指标上也有“边缘”更好的表现。Allodi 等人 两组学生在 UI 指标上没有发现统计学上显著差异。作者发现，与两组学生相比，安全专业人士在 UI 指标上犯错的概率较低。在 AV 指标上，专业人员和接受过安全课程的学生之间存在边缘差异。然而，作者在 AC、PR、C、I 和 A 指标上发现，接受过安全课程的学生与安全专业人士的表现没有统计学上显著差异。他们的结果表明，安全知识有助于产生 CVSS 评估。然而，具有安全经验的学生相对熟练，在六个指标中的四个指标上产生了与专业人士相匹配的分数。

In S11 (see [57]), the authors propose combining multiple expert analyses of CVSS scores via the Delphi method. The Delphi method is a consensus-building technique frequently used in areas such as Systems engineering [105]. The authors of S11 illustrate the possibility of their approach through an illustrative example with four CVEs. The authors also provide the questionnaire that would be used in their method.
在 S11（见[57]）中，作者们提出通过德尔菲法结合多个 CVSS 评分的专家分析。德尔菲法是一种在系统工程[105]等领域经常使用的共识构建技术。S11 的作者通过四个 CVE 的实例说明了他们方法的可能性。作者们还提供了他们方法中将要使用的问卷。

We found three primary groups of evaluations of CVSS scores collected from industry organizations such as the NVD. The first set of evaluations, which we discuss in Section 6.2.1, looks at the Reliability of CVSS—in other words, how consistently a particular vulnerability is scored using CVSS. The second group examines the relationship between CVSS and other exploit-related indicators, such as the presence of an exploit in ExploitDB. Finally, in Section 6.2.3, we discuss four studies that examine the distribution of CVSS scores in the NVD—for example, the percentage of scores that are classified with “Low,” “Medium,” and “High” severity or exploitability.
我们发现从 NVD 等行业组织收集的 CVSS 评分有三个主要评价组。第一组评价，我们在第 6.2.1 节中讨论，考察 CVSS 的可靠性——换句话说，就是使用 CVSS 对特定漏洞进行评分的一致性。第二组研究 CVSS 与其他与利用相关的指标之间的关系，例如 ExploitDB 中是否存在利用。最后，在第 6.2.3 节中，我们讨论了四项研究，这些研究考察了 NVD 中 CVSS 评分的分布——例如，被评为“低”、“中”和“高”严重性或利用性的评分百分比。

Based on analyses and criticisms discussed in Section 6.2, at least four studies suggest potential changes that could be made to the NVD. We divide the proposed changes into two groups. First, in Section 6.3.1, we discuss two studies, S04 (see [80]) and S10 (see [115]), which propose altering the equations used to calculate CVSS, including the equation used to calculate the Base Exploitability score, such as by altering how each metric (e.g. AV, AC) is weighted. They do not propose altering the underlying metrics. In Section 6.3.2, we discuss two studies, 10 (see [81]) and S15 (see [56]), which propose adding or altering the metrics themselves. In S10, S04, and S12, which are based on criticisms of the distribution of CVSS scores in the NVD as discussed in Section 6.2.3, the authors evaluate their changes by illustrating how they produce a distribution of severity scores different from the distribution of scores in the NVD. In S15 (see [56]), the proposed changes are based on survey responses. The survey responses are used to suggest that the existing categories of CVSS are inadequate.
基于第 6.2 节中讨论的分析和批评，至少有四项研究提出了可以对 NVD 进行改进的潜在变化。我们将这些建议的改进分为两组。首先，在第 6.3.1 节中，我们讨论了两项研究，S04（见[80]）和 S10（见[115]），它们提出了修改用于计算 CVSS 的方程，包括用于计算基本可利用性得分的方程，例如通过改变每个指标（例如 AV、AC）的权重。它们不提出修改基础指标。在第 6.3.2 节中，我们讨论了两项研究，10（见[81]）和 S15（见[56]），它们提出了添加或修改指标本身。在 S10、S04 和 S12 中，这些研究基于第 6.2.3 节中讨论的 NVD 中 CVSS 分数分布的批评，作者通过说明他们的变化如何产生与 NVD 中分数分布不同的严重程度分数来评估他们的变化。在 S15（见[56]）中，提出的改进基于调查反馈。调查反馈被用来表明现有的 CVSS 类别是不充分的。

In this section, we discuss automated, Deterministic assessments that examine how a sequence of program states may lead to an “exploitability property” [12] being met. The exploitability property is based on the intended output, such as the type of exploit to be produced or the sub-metrics of the CVSS score. The exploitability property is defined in terms of the types of program analysis performed by the tool, such as static binary analysis or memory monitoring.
在本节中，我们讨论了自动的确定性评估，这些评估检查程序状态序列如何导致满足“可利用性属性” [12]。可利用性属性基于预期输出，例如要生成的漏洞类型或 CVSS 评分的子指标。可利用性属性是在工具执行程序分析的类型（如静态二进制分析或内存监控）的术语中定义的。

For example, the exploitability properties in S06 (see [11, 12, 16, 107]) are defined in terms of elements of the internal program state space such as functions and register calls—for example, “the IP register holds a value that corresponds to some function f of user input i such as f may be a call to tolower on the input i and the resulting IP points to shellcode” [12]. The authors use static analysis and binary instrumentation to extract assembly language functions from the program executable (also referred to as a “binary”) [11, 16], gathering information about the program state space. The information about program state space is used by an SMT (Satisfiability Modulo Theory) solver [12] to determine if there is an execution trace that results in the exploitability property being met. The tool uses the SMT result to produce an exploit.
例如，S06 中的可利用性属性（参见[11, 12, 16, 107]）是在内部程序状态空间元素（如函数和寄存器调用）的术语下定义的——例如，“IP 寄存器持有与用户输入 i 相关的某个函数 f 的值，例如 f 可能是对 tolower 的调用，并且结果 IP 指向 shellcode” [12]。作者使用静态分析和二进制插装从程序可执行文件（也称为“二进制”）[11, 16]中提取汇编语言函数，收集有关程序状态空间的信息。程序状态空间的信息被 SMT（可满足性模理论）求解器[12]用于确定是否存在导致满足可利用性属性的执行轨迹。该工具使用 SMT 结果生成漏洞利用。

As a different example, in S43 (see [144]), the properties are defined based on CVSS and determined based on outputs of instrumented binary analysis. For example, the property based on the AV metric was determined using the existence and observed behavior of function calls such as socket and connect when a dynamic trigger for the vulnerability is run against the target application.
作为一个不同的例子，在 S43（见[144]）中，属性是基于 CVSS 定义的，并基于仪器化二进制分析的结果确定的。例如，基于 AV 指标的属性是通过在针对目标应用程序运行漏洞的动态触发器时，检查 socket 和 connect 等函数调用的存在和观察到的行为来确定的。

Table 5 highlights characteristics of the studies of automated, Deterministic exploitability assessments. The first two columns of Table 5 indicate the study ID and associated bibliography entries. The third column indicates the years in which the studies were published. The fourth and fifth columns pertain to the tool outputs, which we will discuss in Section 7.1.1, including the type of exploit that the tool can generate, if applicable. Columns 6 through 9 show the primary inputs to the tools, which we discuss in Section 7.1.2. Column 10 is the type of vulnerability the tool is designed to analyze. Column 11 is the language(s) of the vulnerable applications to which the tools can be applied, which relates to the type of vulnerability. Columns 10 and 11 are discussed in Section 7.1.3. Columns 12 through 14 highlight aspects of the evaluation performed in each study, which are discussed in Section 7.1.4. Column 12 indicates the types of programs (applications) that the tool is intended to be used against. Column 13 indicates the size of the dataset used in the evaluation in terms of the number of vulnerabilities. Column 14 indicates the Time to Run the tool per vuln (unless otherwise noted). The rows of Table 5 are ordered based on their outputs and then on the year(s) in which the work was published since, as we will discuss in Section 7.1.1, the tool output is a key distinguishing factor between studies.
表 5 突出了自动化、确定性可利用性评估研究的特点。表 5 的前两列表明了研究 ID 和相关参考文献条目。第三列指出了研究发表的年份。第四和第五列涉及工具输出，我们将在 7.1.1 节中讨论，包括工具可以生成的攻击类型（如果适用）。第 6 至 9 列显示了工具的主要输入，我们将在 7.1.2 节中讨论。第 10 列是工具设计用于分析的安全漏洞类型。第 11 列是工具可以应用于的易受攻击应用程序的语言（们），这与漏洞类型相关。第 10 和 11 列在 7.1.3 节中讨论。第 12 至 14 列突出了每项研究中进行的评估方面，这些将在 7.1.4 节中讨论。第 12 列指出了工具打算用于对抗的程序（应用）类型。第 13 列指出了评估中使用的数据集大小，以漏洞数量来衡量。 第 14 列表示每个漏洞运行工具所需的时间（除非另有说明）。表 5 的行按其输出顺序排列，然后按发表年份排序，因为，正如我们在第 7.1.1 节中将要讨论的，工具的输出是区分研究的关键因素。

We identified one study, S01 (see [123]), which predates the program analysis work but leverages a similar, Deterministic vulnerability exploitability metric as part of their network security analysis. They model the network as a graph of system states and measure the exploitability of a vulnerability as the in-degree of the vulnerability within the system state graph. Although this graph is based on a network and not a program, their graph of system states is quite similar to the graphs used to analyze the system state of a program in other Deterministic systems such as S16 (see [44]) and S13 (see [135, 137]). The assessment in S01 focuses on system-level metrics that their vulnerability-level metric contributes to rather than evaluating the vulnerability exploitability metric.
我们确定了一项研究，S01（见[123]），该研究早于程序分析工作，但将其网络安全性分析中利用了类似的、确定性的漏洞可利用性指标。他们将网络建模为系统状态的图，并将漏洞的可利用性度量为其在系统状态图中的入度。尽管这个图是基于网络而不是程序，但他们的系统状态图与其他确定性系统（如 S16[44]和 S13[135, 137]）中分析程序系统状态所使用的图相当相似。S01 的评估侧重于系统级指标，而不是评估漏洞可利用性指标，而是他们的漏洞级指标所贡献的。

We identified one unique study, S27 (see [31]), in which “exploitability” was defined based on attacker characteristics and activity. In S27 (see [31]), the authors propose a metric Threat Agent Count (TAC) that indicates the quantity of threat actors capable of exploiting a particular vulnerability. The authors evaluate their metric by comparing six vulnerability prioritization policies: (1) a FIFO (first-in-first-out policy; (2) prioritizing vulnerabilities with the highest CVSS score first; (3) prioritizing vulnerabilities with the highest score from the adapted CVSS framework proposed in S04 (see [80]) first; (4) prioritizing fixing vulnerabilities with the highest TAC score first; (5) prioritizing fixing vulnerabilities based on highest CVSS score, then highest TAC score in case of a tie; and (6) prioritizing fixing vulnerabilities based on highest TAC value, then the highest CVSS value in the case of a tie. They evaluate and compare the policies based on how much the policy exposes a software system to vulnerabilities with a known exploit. Exposure is measured as a function of time t where $E_t$ is the set of vulnerabilities with known exploits that have yet to be fixed, such that $Exposure\left(t\right)=\sum _{i=0}^t|E_i|$. The authors used 1,000 randomly selected vulnerabilities to represent the vulnerable Information System, which was evaluated via simulations of 1,000 timesteps t. At each timestep, 1 vulnerability was fixed according to the policy under evaluation. The authors report their results at the end of each quarter of the simulation (i.e., every 250 t). The policy of prioritizing vulnerabilities based on TAC, then by CVSS in the case of a tie (policy 6), had the lowest exposure at the end of each quarter, followed by prioritization entirely based on TAC, where in the case of ties, the first-found vulnerability is removed first (policy 4). This leads the authors to claim that TAC “is a significantly better predictor of exploitable vulnerabilities than CVSS score” [31].
我们确定了一项独特的研究，S27（见[31]），其中“可利用性”是根据攻击者特征和活动来定义的。在 S27（见[31]）中，作者提出了一个指标威胁代理计数（TAC），它表示能够利用特定漏洞的威胁行为者的数量。作者通过比较六种漏洞优先级策略来评估他们的指标：（1）先进先出（FIFO）策略；（2）优先处理 CVSS 评分最高的漏洞；（3）优先处理来自 S04（见[80]）中提出的改进 CVSS 框架的最高评分的漏洞；（4）优先修复 TAC 评分最高的漏洞；（5）在 CVSS 评分相同的情况下，优先修复 CVSS 评分最高、TAC 评分次之的漏洞；（6）在 TAC 值相同的情况下，优先修复 CVSS 值最高的漏洞。他们根据策略暴露软件系统于已知漏洞可利用性的程度来评估和比较这些策略。 暴露被测量为时间 t 的函数，其中 $E_t$ 是已知漏洞利用但尚未修复的漏洞集合，即 $Exposure\left(t\right)=\sum _{i=0}^t|E_i|$ 。作者使用了 1,000 个随机选择的漏洞来代表易受攻击的信息系统，该系统通过 1,000 个时间步长 t 的模拟进行评估。在每个时间步长，根据评估的政策修复 1 个漏洞。作者在每个季度结束时报告他们的结果（即，每 250t）。基于 TAC 优先处理漏洞的政策，在出现平局时按 CVSS 排序（政策 6），在每个季度结束时具有最低的暴露度，其次是完全基于 TAC 的优先处理，在出现平局时，首先移除最早发现的漏洞（政策 4）。这导致作者声称 TAC“是可利用漏洞的显著更好的预测因子，比 CVSS 评分要好” [ 31]。

All of the models in this section are supervised machine learning or neural network models, requiring training data that includes a set of features on which the model is based and labels indicating the ground truth for each instance (e.g., the exploitability for each vulnerability). Empirical studies of machine learning models typically perform their evaluation by withholding a subset of the labeled data to use in testing the model once it has been trained.
本节中的所有模型均为监督机器学习或神经网络模型，需要包含模型所基于的特征集和每个实例的地面真实标签的训练数据（例如，每个漏洞的可利用性）。机器学习模型的实证研究通常通过保留一部分标记数据来评估模型，在模型训练完成后，使用这部分数据来测试模型。

The models can be split into two distinct groups based on their GT. The first group is models that use the base CVSS score directly or indirectly from NVD as the target of their model, which we discuss in Section 8.1.1. The second group uses one or more of a combination of indicators, including exploits from a database like ExploitDB, Exploit Signatures, specially dedicated exploit flags in vulnerability databases, and other exploit-related indicators. As shown in Table 6, authors can combine multiple sources as part of the same GT, hampering further orthogonal classification. We discuss this second group of Other Exploit-Related GT in Section 8.1.2
模型可以根据其 GT 分为两个不同的组。第一组是直接或间接使用 NVD 的 CVSS 基础分数作为模型目标的模型，我们将在第 8.1.1 节中讨论。第二组使用一个或多个指标的组合，包括来自 ExploitDB 等数据库的漏洞利用、漏洞签名、专门用于漏洞数据库的漏洞利用标志以及其他与漏洞利用相关的指标。如表 6 所示，作者可以将多个来源组合为同一 GT，从而阻碍进一步的正交分类。我们将在第 8.1.2 节中讨论这一组其他与漏洞利用相关的 GT。

As we discuss in Section 6, and as is documented in work on LMs such as EPSS where the maintainers report the features that contribute most to the LM, exploit and exploitability indicators are only loosely related to each other and are also related to factors such as the organization who maintains the software (e.g. Microsoft). Hence, there is no clear “winner” among types and sources of GT. Furthermore, as can be seen in the Data Source(s) column (Column 15) of Table 6, the existing work primarily uses publicly available datasets, which are known to provide higher coverage of large organizations such as Microsoft. More work is needed to understand how these models generalize to less well known organizations and smaller projects, or with data collected internally, which may be too sensitive to release publicly.
如我们在第 6 节中讨论的，正如在 EPSS 等 LM（语言模型）的研究工作中所记录的，维护者报告了对 LM 贡献最大的特征，利用和可利用性指标彼此之间只有松散的联系，并且也与维护软件的组织（例如微软）等因素相关。因此，在 GT（生成式翻译）的类型和来源中并没有明确的“赢家”。此外，如表 6 中的数据源（第 15 列）所示，现有工作主要使用公开可用的数据集，这些数据集已知可以提供对微软等大型组织的高覆盖率。需要更多的工作来理解这些模型如何推广到不太知名的组织和较小的项目，或者使用内部收集的数据，这些数据可能过于敏感而不宜公开发布。

We categorize the features used in the different LMs into several categories, each of which is described in more detail in this section. Table 6 also shows the features used in each paper.
我们将不同语言模型中使用的特征分为几个类别，每个类别在本节中都有更详细的描述。表 6 还展示了每篇论文中使用的特征。

The final column of Table 5 highlights type types of modeling techniques examined in each study. As can be seen in the table, neural networks and large-language models, such as Bi-directional Long Short Term Memory, Gated Recurrent Unit, and the CodeBERT model, have been increasingly popular avenues of research since 2019. However, gaps remain in determining the efficacy and usability of these models in a real-world scenario.
表 5 的最后一列突出了每项研究中考察的建模技术类型。如表中所示，自 2019 年以来，神经网络和大型语言模型，如双向长短期记忆网络、门控循环单元以及 CodeBERT 模型，已成为越来越受欢迎的研究途径。然而，在确定这些模型在实际场景中的有效性和可用性方面仍存在差距。

In 2006, Frei et al. [41] examined vulnerability and exploit information based on vulnerabilities published in the OSVDB and NVD from 1996 to 2006. Their analysis focused on four points in the vulnerability lifecycle: the time of vulnerability Discovery, the time of vulnerability Disclosure, the (earliest) time of Exploit availability for the vulnerability, and the time of Patch availability. The timeline for each vulnerability was extracted through a number of sources including three public organizations: the CERT organization at Carnegie Mellon and the French Security Incident Response Team, and the milw0rm hacktivist group, as well as four companies that provide security-related software: ISS (Internet Security Systems) X-Force (now IBM X-Force), Secunia, Symantec SecurityFocus, Packetstorm, and Metasploit [41]. As part of their analysis, they identified a function matching the distribution of Exploit Availability in terms of time t since Disclosure:
2006 年，Frei 等人[41]基于 1996 年至 2006 年 OSVDB 和 NVD 发布的漏洞信息，考察了漏洞和利用信息。他们的分析集中在漏洞生命周期的四个方面：漏洞发现时间、漏洞披露时间、漏洞利用（最早）可用时间以及补丁可用时间。每个漏洞的时间线是通过多个来源提取的，包括三个公共组织：卡内基梅隆大学的 CERT 组织、法国安全事件响应团队以及 milw0rm 黑客组织，以及四家提供安全相关软件的公司：ISS（互联网安全系统）X-Force（现为 IBM X-Force）、Secunia、Symantec SecurityFocus、Packetstorm 和 Metasploit[41]。作为他们分析的一部分，他们确定了一个与自披露以来时间 t 的 Exploit 可用性分布相匹配的函数：The Exploit Availability function determined by Frei et al. was used by two studies in our survey, S02 (see [43]) and S24 (see [99, 100]) to estimate exploitability.
Frei 等人确定的漏洞可用性函数被我们调查中的两项研究使用，S02（见[43]）和 S24（见[99, 100]）用于估计漏洞利用性。

As discussed in Section 3, the NVD does not provide estimates of the Temporal or Environmental scores. In S02, Frühwirth and Mannisto [43] analyze the difference in CVSS v2 metrics when they use probability distributions to approximate the Temporal and Environmental metrics. For the “Exploit Code Maturity Metric,” the primary exploitability-specific Temporal or Environmental metric in CVSS v2, the authors use the equation from Fre. et al. [41]. In S02 (see [43]), the authors found that on a set of 720 vulnerabilities recorded between January 5 and March 20, 2009, the CVSS score computed using their probability-based system for the Environmental and Temporal metrics resulted in differences ranging from a 2.5 point decrease to a 2.5 increase from the score computed using default values, with 60% of scores having a 0.5% to 1% decrease from the default values. Overall, this increased the number of “Low” severity vulnerabilities, which the authors argue would result in cost savings, assuming that lower-severity vulnerabilities are less expensive to fix [43].
如第 3 节所述，NVD 不提供时间或环境评分的估计。在 S02 中，Frühwirth 和 Mannisto[43]分析了当他们使用概率分布来近似时间和环境指标时，CVSS v2 指标的差异。对于“漏洞利用代码成熟度指标”，CVSS v2 中的主要可利用性特定时间和环境指标，作者使用了 Fre.等人的方程[41]。在 S02（见[43]），作者发现，在 2009 年 1 月 5 日至 3 月 20 日记录的 720 个漏洞集合中，使用基于概率的系统计算的环境和时间指标得出的 CVSS 评分与使用默认值计算出的评分相比，差异从 2.5 分下降到 2.5 分上升，其中 60%的评分从默认值下降了 0.5%至 1%。总体而言，这增加了“低”严重性漏洞的数量，作者认为这将导致成本节约，假设低严重性漏洞的修复成本较低[43]。

Similarly, in S24 (see [99, 100]), the authors use the same model fit by Frei et al. to estimate an “exploitability” score they describe as the likelihood that a vulnerability will be exploited before being patched or disclosed. The authors combine this exploitability score with the overall CVSS severity score and an estimate of the likelihood that a vulnerability will be patched, also based on Frei’s work, to develop non-linear statistical models to estimate the probability of the exploitation of vulnerability as a function of time. This model is intended to improve on their previous model (see [99]), which used the CVSS Base Exploitability metric as part of their process. The authors of S24 refer to their overall models of the probability of exploitation as “exploitability” models [100]. In S24, the authors evaluate their risk models using $R^2$ and Residual analysis. $R^2$ values can range between 0 and 1, where 1 indicates that the targeted phenomenon (e.g., risk) can be perfectly explained by the predictors [30]. The adjusted $R^2$ value is used to account for larger numbers of predictor variables in a model. The adjusted $R^2$ for the initial model in S24 was 0.85 [99], whereas the most advanced model had an adjusted $R^2$ of 0.96 [99].
同样，在 S24（见[99, 100]）中，作者们使用了 Frei 等人提出的相同模型来估算一个他们描述为“可利用性”的分数，即漏洞在被修补或公开之前被利用的可能性。作者们将这个可利用性分数与整体 CVSS 严重性评分以及一个基于 Frei 工作的漏洞被修补可能性的估计相结合，开发出非线性统计模型来估计漏洞被利用的概率作为时间的函数。这个模型旨在改进他们之前使用的模型（见[99]），该模型将 CVSS 基本可利用性指标作为其过程的一部分。S24 的作者们将他们关于利用概率的整体模型称为“可利用性”模型[100]。在 S24 中，作者们使用 $R^2$ 和残差分析来评估他们的风险模型。 $R^2$ 的值介于 0 和 1 之间，其中 1 表示目标现象（例如，风险）可以被预测因子完美解释[30]。调整后的 $R^2$ 值用于考虑模型中预测变量数量的增加。S24 中初始模型的调整后 $R^2$ 为 0。85 [ 99]，而最先进的模型调整后的 $R^2$ 为 0.96 [ 99]。

In S22 (see [111]), the authors use a Cox Proportional Hazard survival model to examine how vulnerability characteristics, such as Severity, relate to the amount of time between when a vulnerability is discovered and when an exploit is published. The authors found that Disclosure Status (whether the vulnerability had been disclosed at the time of discovery) was the strongest predictor of the length of time between when a vulnerability is discovered an the exploit is published, followed by severity. The authors propose a set of rules based on the results of their analysis, such as “Immediately disclosed, highly severe, remote vulnerability targeted at open-source, infrastructure software” should be a top priority for patching [111]. The authors suggest that managers could use a similar modeling and rule-development process to prioritize patching efforts.
在 S22（见[111]）中，作者使用 Cox 比例风险生存模型来检验脆弱性特征，如严重程度，与发现漏洞到发布漏洞利用之间的时间长度之间的关系。作者发现，披露状态（在发现时漏洞是否已披露）是预测发现漏洞到发布漏洞利用之间时间长度最强的预测因子，其次是严重程度。作者根据他们的分析结果提出了一组规则，例如“立即披露、高度严重、针对开源、基础设施软件的远程漏洞”应优先进行修补[111]。作者建议管理者可以使用类似的建模和规则开发过程来优先考虑修补工作。

In this section, we examine temporal trends and cross-cutting factors in exploitability assessment systems. Since Table 4 is ordered by publication year, it is a useful reference for observing and discussing temporal trends.
在这一节中，我们探讨了可利用性评估系统中的时间趋势和交叉因素。由于表 4 按出版年份排序，它对于观察和讨论时间趋势非常有用。

As described by de Smale et al. [28], practitioners make tradeoffs in determining what vulnerabilities to prioritize or even what information to collect for vulnerability management. What methods and tools are useful for a particular organization will depend on the tradeoffs they make. We use the three sets of value tradeoffs identified by de Smale et al. obtained via interviews with practitioners to frame our discussion of the different exploitability assessment techniques. In this discussion, we are not proposing that one technique should be used instead of another. We use de Smale’s sets of opposing values to highlight how some categories of technique may align with specific values.
如 de Smale 等人[28]所述，实践者在确定优先考虑哪些漏洞或甚至收集哪些信息以进行漏洞管理时进行权衡。对特定组织有用的方法和工具将取决于他们所做的权衡。我们使用 de Smale 等人通过访谈实践者获得的三个价值权衡集来构建我们对不同可利用性评估技术的讨论。在这次讨论中，我们并不是提议用一种技术代替另一种技术。我们使用 de Smale 的相反价值集来强调某些技术类别可能如何与特定价值相一致。

Independent analysis vs. Trust. The first set of values discussed by de Smale et al. [28] is how much organizations trust information provided by others, compared with the time and effort required to perform an in-depth, independent analysis of the vulnerability in the organization’s context. CVSS scores provided by a third party, such as the NVD, require a high amount of trust. As noted by Ponta et al. [97] in their “Lessons Learned” as part of transitioning S21 into industry practice, developers may be particularly skeptical of metadata-based assessments, and as shown in Table 6, many of the proposed LM approaches for exploitability assessment are based on metadata from vulnerability reports. However, if an organization has sufficient data from its sources, it may be able to build and use a more independent LM. Using Deterministic tools may provide more independent information for that organization’s context but do require Time to Run. Similarly, evaluating CVSS manually requires expertise and time but can be done independently, with the Environmental metrics of CVSS specifically designed to be tailored to a particular context.
独立分析 vs. 信任。de Smale 等人[28]讨论的第一组价值观是组织对他人提供的信息信任程度，与在组织环境中对漏洞进行深入独立分析所需的时间和精力相比。第三方提供的 CVSS 评分，如 NVD，需要很高的信任度。正如 Ponta 等人[97]在他们将 S21 过渡到行业实践中的“经验教训”中所述，开发者可能特别怀疑基于元数据的评估，如表 6 所示，许多针对可利用性评估的 LM 方法都是基于漏洞报告的元数据。然而，如果一个组织拥有足够的数据来源，它可能能够构建和使用一个更独立的 LM。使用确定性工具可能为该组织的特定环境提供更多独立信息，但确实需要运行时间。同样，手动评估 CVSS 需要专业知识和时间，但可以独立完成，CVSS 的环境指标专门设计为可以针对特定环境进行调整。

Proactive vs. Reactive. Smaller organizations tend to take a primarily reactive approach [28], often relying on information pushed to them by government organizations—and may therefore be more likely to rely heavily on CVSS scores from the NVD. Gathering sufficient information to leverage an LM model requires a highly proactive approach, whereas Deterministic methods lie somewhere in between. Having LM scores publicly available, such as with EPSS, may facilitate the use of LM results in a reactive context.
主动与被动。较小的组织往往采取主要被动的策略[28]，通常依赖政府机构推送给他们的信息——因此可能更倾向于高度依赖 NVD 的 CVSS 评分。收集足够的信息以利用 LM 模型需要高度主动的方法，而确定性方法则介于两者之间。LM 评分的公开，例如 EPSS，可能有助于在被动环境中使用 LM 结果。

Formalized vs. Ad-Hoc Processes. Probabilistic approaches, which require a collection of vulnerabilities to perform statistical analysis, may be challenging to incorporate into ad-hoc processes. As with Proactive vs. Reactive decisions, the availability of a pre-calculated EPSS (LM) score may mitigate the need for a formalized process but also reduces the ability to tailor the score to the organization’s context. CVSS-based scores can be more readily calculated on an ad-hoc basis. Similarly, the Program State based Deterministic assessments may be more readily incorporated into ad-hoc processes, although some work may be required to set up, configure, and run the program analysis tools. As discussed in Section 7.1.4, given the time required to run many of the Deterministic Program State based assessments, these approaches may be less useful if an organization is attempting to analyze all vulnerabilities at once on a regular basis as part of a formalized processes such as in Continuous Integration.
形式化过程与临时过程。需要收集漏洞集以进行统计分析的概率方法可能难以融入临时过程。正如主动与被动决策一样，预先计算的 EPSS（LM）分数可能减轻了对形式化过程的需求，但也降低了根据组织环境调整分数的能力。基于 CVSS 的分数可以更方便地在临时基础上计算。同样，基于程序状态的确定性评估可能更容易融入临时过程，尽管可能需要一些工作来设置、配置和运行程序分析工具。如第 7.1.4 节所述，考虑到许多基于确定性程序状态的评估所需的时间，如果组织试图定期作为形式化过程（如持续集成）的一部分同时分析所有漏洞，这些方法可能不太有用。

We found studies that used other assessment methods as a control group in their experiments (i.e., to demonstrate that their proposed technique is an improvement). However, we found relatively little research on how assessment methods might work well together. One of the closest studies in this regard may be S22 (see [111]) discussed in Section 9.2, in which the authors propose building more deterministic rules for vulnerability prioritization based on the results of a statistical model. However, even in S22, the study focuses on the relationship between factors within the statistical model rather than evaluating the combination of techniques. The length of time and resources to run many of the Deterministic, Program State based models discussed in Section 7.1 may make it difficult for such tools to be used at a scale that could contribute to developing an LM. Future research may focus on the assessment techniques that could be used to better determine when to deploy Program State based models.
我们发现了一些研究，它们在实验中将其他评估方法作为对照组（即，为了证明他们提出的技术是一种改进）。然而，我们发现关于评估方法如何协同工作的研究相对较少。在这方面最接近的研究可能是 S22（见[111]），在 9.2 节中讨论了这项研究，其中作者提出根据统计模型的结果建立更多确定性规则来进行漏洞优先级排序。然而，即使在 S22 中，该研究也侧重于统计模型内部因素之间的关系，而不是评估技术的组合。7.1 节中讨论的许多基于程序状态的确定性模型的运行时间和资源可能使得这些工具难以在可能对开发 LM 做出贡献的规模上使用。未来的研究可能将重点放在评估技术，这些技术可以更好地确定何时部署基于程序状态的模型。

The usability of exploitability assessment techniques is under-studied. Studies S34 (see [3]) and S44 (see [4]) examine user concerns when applying CVSS. For Deterministic methods, S57 determined that there were misconceptions among experts about the relative merits and drawbacks of different features in EG tools for heap-based vulnerabilities and S21 (see [60, 95, 96, 97]) includes a summary of lessons learned from transitioning their Deterministic tool into practical use at SAP. However, we could not find studies looking into any usability-related aspects of the Probabilistic models and no formal usability studies of many of the Deterministic models.
可利用性评估技术的可用性研究不足。研究 S34（见[3]）和 S44（见[4]）考察了在应用 CVSS 时用户关注的问题。对于确定性方法，S57 确定专家对基于堆漏洞的 EG 工具中不同特征的相对优缺点存在误解，而 S21（见[60, 95, 96, 97]）总结了将他们的确定性工具过渡到实际应用在 SAP 中学到的经验教训。然而，我们未能找到研究概率模型任何可用性相关方面的研究，也没有许多确定性模型的形式可用性研究。

Finally, while work such as the reliability analysis of exploit stabilization in EG tools for heap overflow vulnerabilities in S57 (see [140]) and larger-scale analyses in studies such as S21 (see [60, 95, 96, 97]) begin to look at effectiveness using more systematic, scalable evaluations, most studies examining Program State based programs focus on smaller vulnerability datasets. While smaller datasets may be understandable due to the complications that arise from the state-space explosion, their generalizability is less known. To build datasets to provide accurate information such as general precision and recall [97], it may be beneficial to perform additional research into what practitioners mean by “exploitability” and what they expect from exploitability assessment.
最终，尽管像在 S57 中针对堆溢出漏洞的 EG 工具的利用稳定性可靠性分析（见[140]）以及 S21 等研究中的更大规模分析（见[60, 95, 96, 97]）开始使用更系统、可扩展的评估方法来考察有效性，但大多数基于程序状态的研究仍然集中在较小的漏洞数据集上。虽然由于状态空间爆炸引起的复杂性，较小的数据集可能是可以理解的，但它们的泛化能力却鲜为人知。为了构建提供准确信息，如一般精度和召回率[97]的数据集，可能有益于进一步研究实践者对“可利用性”的理解以及他们对可利用性评估的期望。