CyRating® Explained

Learn How CYR3CON PR1ORITY Delivers Accurate Predictions of Weaponized Exploits  

What is it exactly?


CyRating is a risk measurement that reflects the current real-world threat to vulnerabilities. Every vulnerability is at risk of exploitation, but each does not carry the same level of risk.

The CyRating Score is scaled from 1.00 to 38.46 to reflect the relative likelihood of exploitation.

A vulnerability with a CyRating of 10.00 is 10xspan more likely to be exploited than a vulnerability with a CyRating of 1.00.

Why is it needed?


As software permeates our lives – especially networking & communications software – coding flaws/bugs proliferate along with it. Since January 2017, we have seen well over 1000 CVEs per month (on average) published to the National Vulnerability Database (NVD).

Organizations need a way to rapidly triage threat intelligence for two related reasons:

circle 1The sheer volume of published vulnerabilities has made it impossible for almost all organizations to follow a “patch everything” strategy.

circle 2The vast majority of vulnerabilities have never been exploited in the wild.

 

 

step 1

In 2019, a single worker dedicated to monitoring published CVEs needed to read, evaluate, and direct action on one vulnerability every seven minutes to keep pace.

CVEs 1999 - Jun 2020
NUMBER OF VULNERABILITIES PUBLISHED EACH YEAR 1999 – JUNE 2020
Chart 1
2019 CVEs
NUMBER OF VULNERABILITIES PUBLISHED MONTHLY DURING 2019
Chart 2
step 1

While all vulnerabilities pose some degree of threat, they are not of equal interest to hackers for a variety of reasons.

3%

Of the 138,000+ CVEs published in the NVD as of June 2020, multiple studies have shown that only a small minority have ever actually been exploited in the wild. Our research suggests that it is less than 3%. With no additional threat intelligence, it would be reasonable to assume that each vulnerability had about a 3% chance of exploitation*.

*M. Almukaynizi, A. Grimm, E. Nunes, J. Shakarian, P. Shakarian Predicting Cyber Threats through the Dynamics of User Connectivity in Darkweb and Deepweb Forums
ACM Computational Social Science (CSS-2017) (Oct. 2017)
https://dl.acm.org/doi/abs/10.1145/3145574.3145590

The likelihood of exploitation for a relatively few number of vulnerabilities is much higher than for the vast majority of ordinary vulnerabilities. The scale of the problem is represented by the image below. In this symbolic IT world, white dots represent all non-exploited vulnerabilities and red dots signify exploited vulnerabilities.

ratio
2019 CVEs
NUMBER OF VULNERABILITIES PUBLISHED MONTHLY DURING 2019

 CyRating enables organizations to scale their cyber threat analysis and become much more efficient.

While it has become impossible for almost all organizations to follow a “patch everything” strategy, even those that are able will balk at the realization that a large amount of all resources – people, time, money, etc. – dedicated to “patch everything” are arguably being squandered. For the many organizations drowning in vulnerabilities, CyRating cuts through the paralyzing confusion often caused by too much information and provides laser focus on the most important vulnerabilities to mitigate.

What goes into the score?


All vulnerabilities published in the NVD start off with a CyRating Score of 1.00 (hence, average likelihood of exploitation 1x as likely as average) to reflect the base probability of exploitation. A CyRating Score of 5.00 means that a vulnerability is 5x more likely to be exploited than an ordinary vulnerability with a score of 1.00.

CYR3CON®’s CyRating combines a great variety of factors. These include:

  • Hacker-community data from a wide array of sources (including darkweb, deepweb, social media, Telegram, and others)
  • Underlying social structure of these communities
  • Skill level/reputation of actor posting information
  • Metadata of the hacker source (e.g. language, # of users, etc.)
  • Vulnerability technical information (e.g. operating system, etc.)
Score Sources
Chart 3

CYRATING CHANGES FOR CVE-2019-19781 FOR THE FIRST TWO MONTHS AFTER BEING RELEASED

CyRating Scores currently range from a low of 1.00 to a high of 38.46, though the high-end could vary slightly as the vulnerability landscape is constantly changing. Scores increase (or decrease) when CYR3CON®’s continuous, automatic, machine-learning driven analysis of threat intelligence warrants. This happens for any number of reasons. An example snapshot of one vuln’s CyRating changing over time and some of the reasons for the changes is provided in the figure.

CyRating is an indication of the current, external threat. So, mature cybersecurity programs can combine the external threat intelligence provided by CyRating Scores with internal considerations, e.g. organizational policies, controls, topology, to better assess overall risk.

While vulnerabilities can be mitigated by patching, another remediation strategy may make more sense if a patch is not readily available or if the risk to disrupting operations is high, i.e. applying a patch breaks a complex system. Non-patching approaches include adding firewall rules, changing configuration settings, modifying network access control lists, updating IDS/IPS signatures, ensuring network segmentation, etc.

step 1

The CyRating for each vulnerability is integrated into CYR3CON®’s PR1ORITY  product which is available via a full REST-based API or through a web-based interface (screenshot shown below).

Updated in real time, PR1ORITY provides information relevant to all CVEs including the dates that related hacker chatter was first seen and when it was last seen. As well, clicking on the CVE number opens an additional pane that displays, by date, the raw threat intel analyzed as part of the CyRating generation procedure.

main

SCREENSHOT OF THE PR1ORITY WEB USER INTERFACE WHICH INCORPORATES CYRATING

Who provides the analysis?


CYR3CON®’s artificial intelligence (AI) is the embodiment of a cybersecurity analyst who never gets tired, never needs to go to the bathroom, never eats, and never sleeps.

CyRating Scores are calculated using advanced machine learning (ML) algorithms designed to make attacker-focused predictions about the exploitability of all vulnerabilities.

The resultant AI-generated scores are the manifestation of an expert cybersecurity analyst’s evaluation of the real-world threat of exploitation posed by a vulnerability. The cybersecurity landscape is dynamic and cyber criminals are adaptive threat

Of course, computer scientists and engineers monitor the CYR3CON® system, conduct quality checks, develop new capabilities, etc., but AI conducts all primary tasks from mining data to conducting the analysis needed to generate CyRating Scores.

CyRating Scores are calculated using advanced machine learning (ML) algorithms designed to make attacker-focused predictions about the exploitability of all vulnerabilities.
The resultant AI-generated scores are the manifestation of an expert cybersecurity analyst’s evaluation of the real-world threat of exploitation posed by a vulnerability. The cybersecurity landscape is dynamic and cyber criminals are adaptive threat actors who take initiative and react to changing conditions, so CYR3CON®’s ML models are regularly retrained to ensure they are properly attuned to current threat conditions.
The cybersecurity landscape is dynamic and cyber criminals are adaptive threat actors who take initiative and react to changing conditions, so CYR3CON®’s ML models are regularly retrained to ensure they are properly attuned to current threat conditions.
Of course, computer scientists and engineers monitor the CYR3CON® system, conduct quality checks, develop new capabilities, etc., but AI conducts all primary tasks from mining data to conducting the analysis needed to generate CyRating Scores.
Case Study

The CyRating for each vulnerability is integrated into CYR3CON®’s PR1ORITY  product which is available via a full REST-based API or through a web-based interface (screenshot shown below).

step 1

The colloquial understanding of the word “accuracy” is likely different for most people from the probabilistic meaning.

The colloquial understanding of the word “accuracy” is likely different for most people from the probabilistic meaning. Because <3% of vulnerabilities are actually exploited in the wild, someone could predict that no vulnerabilities will ever be exploited and they would be correct 97% of the time – so, claims of high accuracy alone are potentially misleading. Another significant problem here is that relatively high accuracy with an unbalanced dataset (like exists with exploited vs. non-exploited vulnerabilities) potentially generates many false positives. Beware of vendors only touting accuracy.


To evaluate how well a vulnerability exploitation prediction algorithm is working,
precision is a reasonably good value to also check.

step 1

To evaluate how well a vulnerability exploitation prediction algorithm is working,
precision is a reasonably good value to also check.

It strikes a respectable balance between conveying useful information while remaining relatively easy to understand. The closer precision is to 100%, the fewer false positives there are and avoiding false positives (i.e. avoiding patching low-threat vulnerabilities that are not likely to be exploited) is the point of prediction when managing large numbers of vulnerabilities.

At the time of this writing, for vulnerabilities with a CyRating of 20.00 or higher the associated precision is 86%. This means that we could expect ~6 of 7 vulnerabilities with CyRating Scores >=20.00 to be exploited in the wild at some point, while only ~1 in 7 would be a false positive (not exploited in the wild).

Cyr3con logo

While other companies have been around and doing manual, human-based threat analysis, CYR3CON® has led the way in using AI to automate all aspects of the process pipeline from mining to analysis. CYR3CON’s patented, AI-driven, predictive technology has generated research results that have been published in multiple peer-reviewed venues. CYR3CON PR1ORITY was the first commercial product to rank vulnerabilities according to likelihood of exploitation and since 2017, we have been providing customers with threat-based analysis of vulnerabilities using continuous, ML-driven analysis.

What does the vulnerability data look like - big picture?


Unique vulnerabilities have been tracked by NIST in the NVD and assigned a Common Vulnerabilities and Exposures Identifier (CVE ID) since 1999. While all discovered vulnerabilities are not necessarily publicly disclosed and all publicly disclosed vulnerabilities are not assigned a CVE ID, the CVE list maintained by MITRE is intended to be as comprehensive as possible and currently provides the best publicly available list of known software vulnerabilities.

Though CVE IDs include a year, e.g. CVE-2019-19781, this does not necessarily mean that a CVE was published in the year that’s included in the ID. For example, CVE IDs can be assigned in a prior year when a vulnerability is first discovered, then once more details about the vulnerability become available it may be published in a later year.

There were 21,366 CVE IDs assigned that start with CVE-2019-* and 20,348 have been published. When stats are provided for vulnerabilities published by year (like in our earlier charts), those vulnerabilities include CVE IDs with multiple different years as part of the ID list. For example, in June 2020, 1869 vulnerabilities were published and while most start with CVE-2020-, there were 215 that started with CVE-2019- and even 2 that started with CVE-2011-.

Chart 4

2019 VULNERABILITIES

Distribution by CyRating of vulnerabilities assigned CVE IDs in 2019

*Note this does not include CVE numbers that were deprecated or re-assigned.

Scientific Papers Published


The original ideas of CyRating come from years of scientific research – check out the following studies that led to today’s CyRating used in CYR3CON products.

Cy Bullet

E. Nunes, A. Diab, Andrew Gunn, E. Marin, V. Mishra, V. Paliath, J. Robertson, J. Shakarian,
A. Thart, P. Shakarian
Darknet and Deepnet Mining for Proactive Cybersecurity Threat Intelligence
2016 IEEE Conference on Intelligence and Security Informatics (ISI-16) (Sep. 2016)
https://arxiv.org/pdf/1607.08583.pdf

Cy Bullet

J. Robertson, A. Diab, E. Marin, E. Nunes, J. Shakarian, P. Shakarian
Darkweb Cyber Threat Intelligence Mining
Cambridge University Press, 2017
https://books.google.com/books?hl=en&lr=&id=g-VsDgAAQBAJ&oi=fnd&pg=PR7&dq=info:NJgWyPbZ7z sJ:scholar.google.com&ots=Gsy5ruR8Or&sig=w4tLm0KD2jusOzUmJDmVKaxkEzI#v=onepage&q&f=false

Cy Bullet

M. Almukaynizi, A. Grimm, E. Nunes, J. Shakarian, P. Shakarian
Predicting Cyber Threats through the Dynamics of User Connectivity in Darkweb and Deepweb Forums
ACM Computational Social Science (CSS-2017) (Oct. 2017)
https://dl.acm.org/doi/abs/10.1145/3145574.3145590

Cy Bullet

M. Almukaynizi, E. Nunes, K. Dharaiya, M. Senguttuvan, J. Shakarian, P. Shakarian Proactive Identification of Exploits in the Wild Through Vulnerability Mentions Online

2017 International Conference on Cyber Conflict (CyCon-US) (Nov. 2017)

https://admin.govexec.com/media/cycon_paper_camera_version.pdf

Cy Bullet

N. Tavabi, P. Goyal, M. Almukaynizi, P. Shakarian, K. Lerman
DarkEmbed: Exploit Prediction with Neural Language Models
30th Innovative Applications of Artificial Intelligence (IAAI-18) (Feb. 2018)
https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17304

Cy Bullet

Almukaynizi, E. Nunes, K. Dharaiya, M. Senguttuvan, J. Shakarian, P. Shakarian
Patch Before Exploited: An Approach to Identify Targeted Software Vulnerabilities
Intelligent Systems Reference Library: AI in Cybersecurity, Springer
https://www.researchgate.net/profile/Mohammed_Almukaynizi/publication/327730183_ Patch_Before_Exploited_An_Approach_to_Identify_Targeted_Software_Vulnerabilities/links/5cedc01092851c1ad49a57de/Patch-Before-Exploited-An-Approach-to-Identify-Targeted-Software-Vulnerabilities.pdf

© 2020 Cyber Reconnaissance, Inc.
The technology represented by CyRating consists of multiple pending and issued patents in the US, EU, and other countries. Scientific papers shown here are associated with one or more inventions either owned or exclusively licensed to Cyber Reconnaissnace, Inc. CYR3CON and CyRating are registered trademarks of Cyber Reconnaissance, Inc.