Pen Tests vs. Vulnerability Assessments

In the world of cybersecurity, nothing is static. The cyberthreat environment is dynamic and evolving. There are new vulnerabilities discovered seemingly every day. Attacks are getting more sophisticated – they're getting more complex and flying under the radar of traditional detection technologies.

Your organization's environment isn't static either. You introduce new network equipment, bring in new people, engage with new third-party vendors, etc., and security needs to be a consideration with every change made. That's why cybersecurity must be a managed process where you are constantly evaluating, remediating, and tracking what's working and what's not.  

Regularly testing your environment from both an internal and external perspective should be an integral part of your evaluation process – including performing vulnerability assessments and penetration tests (pen test for short). Unfortunately, there is a misconception that these engagements are synonymous. It's common to hear the terms used interchangeably, when in reality, they are very different engagements.

It's imperative you understand the differences between them, so you can select which is most appropriate for your organization at any given time. Here are four distinctions to be aware of.

Objective

The objective of a vulnerability assessment is to identify known weaknesses in your environment. It can provide you with important information, including unapplied patches, vulnerable software versions, and gaps in network controls, like firewalls.

A pen test simulates a real-world attack and tests your existing defensive controls. It goes beyond identifying vulnerabilities, by attempting to exploit found vulnerabilities and performing manual testing to gain access to systems / sensitive data. Manual testing routinely finds vulnerabilities that automated tools are incapable of finding.

Tools

Vulnerability assessments are primarily performed using automated scanning tools such as Nessus, Qualys, or OpenVas, which are off-the-shelf software packages.

A comprehensive pen test is mostly a manual process (although an automated vulnerability scan is often performed during the reconnaissance phase of a pen test). There are commercial tools for pen testing, including Metasploit and CoreImpact; however, skilled pen testers will often write their own exploits as needed.

Deliverables

Following a vulnerability assessment, you are typically provided with a list of known vulnerabilities found during the scan, prioritized by severity and/or business criticality. A stock scanner report could be hundreds of pages and will likely include false positives. Some third-party vendors, like Tyler Technologies, provide a more consolidated report that's easier to navigate and is focused more on its practical use and not on the sheer number of vulnerabilities reported.

Download Our Guide to Penetration Testing

Read our Guide to Penetration Testing to learn how to choose the right type of pen test for your organization, what makes an effective pen test, and see how you can reduce common vulnerabilities that may be found on your network.

Results from a pen test will also provide information on vulnerabilities, ranked by severity, with remediation recommendations; however, it will also include the steps taken to exploit a vulnerability. At Tyler, our reports provide the steps we took or examples used to exploit the vulnerability, so you have all the details on how an attacker could breach your defenses. We also provide an action plan document for you to use to assign and track the individual findings until the risk has been remediated.

Skill Level

Because testing is mostly automated, very little skill is needed to perform a vulnerability assessment.

When it comes to pen tests, the experience, training, and expertise of who is performing it is directly linked to the value the results will provide you. Continuous education is a fundamental element of ensuring quality testing and there are several professional credentials for pen testers including Offensive Security Certified Professional (OSCP), GIAC Web Application Penetration Tester (GWAPT), and GIAC Exploit Researcher, and Advanced Penetration Tester (GXPN).

Pen Test vs. Vulnerability Assessment: Which is Right for my Organization?

In short, both are critical components of a threat and vulnerability management process, but in certain cases, one may be more appropriate than the other.

A vulnerability assessment delivers breadth over depth. It tells you where some of your weaknesses are and how to fix them. Vulnerability assessments are ideal for periodic testing between penetration testing engagements and as a quick verification/sanity check when changes are made to the environment. A targeted vulnerability assessment can be run when a new critical vulnerability is announced to identify the organization's exposure.

Organizations just getting started (thinking about cybersecurity or with a developing cybersecurity program that would like to get a basic understanding of their current vulnerabilities) could start their program off with vulnerability assessments.

In contrast, a pen test delivers depth over breadth. It tells you if someone can exploit your weaknesses to break in, and if so, what information they can access. It is suited for organizations that are compliance-driven, are high-value targets, or have a mature, integrated cybersecurity program. Pen tests should be performed at least annually, and any time significant changes are made to your environment.