# Penetration testing vs vulnerability scanning: what each proves and when you need both > A vulnerability scan is automated and fast. A penetration test has a person try to exploit weaknesses. What each shows, how often, and what to ask a provider. - URL: https://computese.com/penetration-testing-vs-vulnerability-scanning/ - Author: Duong Quan Nguyen, CEO, Computese - Published: 2026-11-12 - Updated: 2026-11-12 - Topics: Security ## In short - A vulnerability scan is automated and reports the possible existence of weaknesses. A penetration test exploits them to confirm they are real, and most tests look for combinations that give an attacker more access than any single flaw. - Scanners can report problems that do not exist, miss ones that do, and rate risk on their own scales. A person should interpret the output and set the risk level instead of accepting the scanner's labels. - PCI DSS v4.0 asks for internal and external vulnerability scans at least every three months (external ones by an Approved Scanning Vendor) and penetration tests at least every 12 months and after significant change. - A report that only lists vulnerabilities does not meet the intent of a penetration test, according to PCI SSC. Look for scope, method, limitations, a testing narrative, ranked findings and a retest after fixes. - A test can only validate that your systems are not vulnerable to known issues on the day of the test. Scan between tests, fix by risk and retest each fix, and agree scope, test windows, contacts and data handling in writing before work starts. A vulnerability scan is an automated check that reports possible weaknesses, quickly and on a schedule. A penetration test is a person, using tools, who tries to exploit and chain those weaknesses inside an agreed scope to show what an attacker could reach. A scan lists candidates. A test confirms what an attacker could exploit. If the basics are missing, start with our [small business security checklist](https://computese.com/cybersecurity-best-practices/), because testing does not replace them. PCI DSS wording below is from version 4.0, and cloud provider rules are as read in October 2026. ## What is the difference between a vulnerability scan and a penetration test? NIST SP 800-115 draws the line in one sentence: scanners "check only for the possible existence of a vulnerability", while the attack phase of a penetration test "exploits the vulnerability to confirm its existence". Most tests also look for combinations of vulnerabilities that give more access than any single one. | Aspect | Vulnerability scan | Penetration test | | ------------------- | ---------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | | Method | Typically automated tools combined with manual verification of identified issues | A manual process that may use vulnerability scanning or other automated tools | | Depth | Surface vulnerabilities, weaknesses that exist in isolation | Goes beyond automated scanning, with more in-depth analysis | | What it reports | Potential risks posed by known vulnerabilities, ranked by NVD and CVSS base scores | Each vulnerability verified or potential issue found, including how and to what extent it may be exploited | | False positives | Can be high, so an expert should interpret the results | The attack phase exploits a vulnerability to confirm it exists | | Frequency (PCI DSS) | At least quarterly and after significant changes | At least annually and upon significant changes | | Duration | Typically seconds to several minutes per scanned host | Days or weeks, depending on scope | The rows draw on [PCI SSC's penetration testing guidance](https://www.pcisecuritystandards.org/documents/Penetration-Testing-Guidance-v1_1.pdf) (version 1.1, 2017), NIST SP 800-115 and NIST SP 800-53. The guidance adds that some entities may have to test more often. ### Is a vulnerability assessment the same as a scan? No. The NIST glossary defines a [vulnerability assessment](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-53r5.pdf) as a systematic examination of an information system or product to determine the adequacy of security measures and identify security deficiencies, among other goals. NIST SP 800-115 lists port and service identification, vulnerability scanning, security configuration review and password cracking as activities a vulnerability assessment plan may authorize. The NCSC adds that a penetration test is [a method for gaining assurance in your vulnerability assessment and management processes](https://www.ncsc.gov.uk/guidance/penetration-testing), not a primary way to find vulnerabilities. ## How does vulnerability scanning work? A scanner identifies the operating systems and major software on each host, submits attack patterns, compares the responses with the signatures of known vulnerabilities and [reports any matches](https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-115.pdf). That can find outdated software versions, missing patches and misconfigurations. It relies on a repository of signatures, so its database should be updated before each run. It sees surface vulnerabilities, weaknesses that exist in isolation, so it cannot detect ones revealed only by combinations of attack patterns. NIST says scanners can have high false positive and false negative rates. ![A magnifying glass hangs from a rail above a row of identical closed doors and has left small flags beside three of them. The magnifying glass is orange; everything else is grey.](https://computese.com/images/blog/penetration-testing-vs-vulnerability-scanning/door-sweep.ab0a11211b-1536.webp) *A scan passes along the row and flags the doors that look weak, but it does not walk through any of them.* Scans come in several kinds. | Kind | What it checks | Limits | | --------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | | Network scan | Hosts and open ports, and the known vulnerabilities on them, from outside or inside the network | Finds vulnerabilities only on active systems, mostly surface ones | | Authenticated scan | The same hosts, signed in with enough privilege for a thorough scan | Needs credentials; some network and security appliances, mainframes and containers may not accept them | | [Web application scan (DAST)](https://community.owasp.org/Vulnerability_Scanning_Tools) | Web applications, normally from the outside | Application scanners typically have high false positive rates | The [OWASP testing guide](https://wstg.owasp.org/v4.2/2-Introduction/) suggests building DAST, static application security testing (SAST) and software composition analysis (SCA, or dependency tracking) tools into CI/CD workflows, or running them on a regular schedule, to catch "low hanging fruit" weaknesses. Our [secure coding checklist](https://computese.com/best-practices-for-secure-coding/) shows when each check runs, the [website security checklist](https://computese.com/essential-tips-to-secure-your-website/) lists free scans for your own site, and the [SBOM guide](https://computese.com/sbom/) covers the component inventory behind dependency checks. ## What does a penetration test do that a scan cannot? NIST SP 800-115 splits a test into four phases: planning, discovery, attack and reporting. Discovery includes the scanning and vulnerability analysis that many scanners automate, and manual analysis can identify new or obscure vulnerabilities that scanners may miss, though it is much slower. The attack phase then tries to exploit what was found, and a feedback loop returns the tester to discovery, because penetration testing is an iterative process that uses minimal access to gain greater access. **An example, invented for illustration.** A scan of a small web business reports three findings: a login form that reveals whether a username exists, an old admin panel on a high port, and a database that accepts connections from the whole internal network. The scanner rates each low or medium. A tester chains them: confirm a staff username, guess a weak password on the admin panel, run commands on the web server through the panel, then read the customer database. NIST warns that a scanner may assign "a low risk to each vulnerability", leaving the assessor "falsely confident", and names penetration testing as the more reliable way to find the risk of vulnerabilities in aggregate. ![An isometric scene: a dashed orange path leaves a laptop, passes through the open doorway of each of three walls in turn, and ends at a database cylinder behind the last wall.](https://computese.com/images/blog/penetration-testing-vs-vulnerability-scanning/door-chain.ee8ae518dd-1536.webp) *A test can follow one weakness into the next until it reaches the data, a route that separate findings do not show.* OWASP's testing guide says [automated tools do a poor job of testing for vulnerabilities](https://wstg.owasp.org/v4.2/2-Introduction/) in part because they do not think creatively, and that creative thinking is needed case by case because most web applications are built in a unique way. [PCI SSC's guidance](https://www.pcisecuritystandards.org/documents/Penetration-Testing-Guidance-v1_1.pdf) gives an example of a finding a report should include even though it was not exploited: credentials obtained by manipulating a web application error message, which an ASV scan had not flagged because of a low CVSS base score. A test has limits too. NIST says penetration testing is labour-intensive, requires great expertise and may damage systems or render them inoperable, a risk experienced testers can reduce but never fully eliminate. The [NCSC](https://www.ncsc.gov.uk/guidance/penetration-testing) says a test can only validate that your systems are not vulnerable to known issues on the day of the test, and that checking whether defined security controls work is not a valuable use of penetration testing resources. ## Which type of penetration test do you need? Pick by what you expose and what you cannot afford to lose. | Type | What it covers | | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | External network | The perimeter as seen from the internet. NIST says external testing usually comes first when both are done | | Internal network | Testers behind the firewall with some access granted, trying to gain more through privilege escalation | | Web application and API | Testers work from the application's defined roles, and a back-end API may be in scope. [OWASP's Web Security Testing Guide](https://wstg.owasp.org/v4.2/2-Introduction/) has sections on authentication, authorization, sessions, input validation, business logic and APIs | | Mobile app | Verifying an app's controls, as in the [OWASP MASTG](https://mas.owasp.org/MASTG/) | | Cloud configuration | Your own cloud resources, within the provider's published testing rules | | Wireless | Your wireless environment. NIST notes that only active wireless devices are discoverable during a scan | | Social engineering | People and procedures. PCI DSS does not require it, but an entity can include it | Each type can run with different amounts of knowledge. [PCI SSC's guidance](https://www.pcisecuritystandards.org/documents/Penetration-Testing-Guidance-v1_1.pdf) defines black-box testing as testing without prior knowledge of the internal structure, grey-box as partial knowledge and white-box as knowledge of it, and says PCI DSS penetration tests are typically white-box or grey-box, which give more accurate results than a pure black-box test. The [NCSC](https://www.ncsc.gov.uk/guidance/penetration-testing) calls the two ends closed box and open box, and says closed-box testing models unknown attackers more accurately but can leave vulnerabilities undiscovered in the time available. A red team exercise goes further. [NIST SP 800-53](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-53r5.pdf) says red team exercises extend the objectives of penetration testing by examining an organization's security and privacy posture and its capability to implement effective cyber defences, and that they may include social engineering. A penetration test is conducted on systems or individual system components. ## How do you rank what scans and tests find? Scanners often use [their own proprietary methods for defining risk levels](https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-115.pdf), so findings are hard to compare between tools. NIST's example: a scanner may call an FTP server a moderate risk because it sends passwords in cleartext, when the real risk is lower if the server is an anonymous public one that uses no passwords. Assessors should set the risk level for each vulnerability and not simply accept the scanner's. ![Many small grey cards drop from a chute onto a slanted mesh screen. Most fall through into a deep bin below, while a few larger cards slide down to a tray at the right and form a short orange stack.](https://computese.com/images/blog/penetration-testing-vs-vulnerability-scanning/triage-sieve.a162a759af-1536.webp) *Triage filters a scanner's list: false positives drop out and an expert ranks the rest in context.* Three public inputs help you set that level: - **CVSS v4.0** rates severity, and its [qualitative scale](https://www.first.org/cvss/v4.0/specification-document) runs from Low (0.1 to 3.9) to Critical (9.0 to 10.0). FIRST's FAQ says [Base scores "are not risk"](https://www.first.org/cvss/v4.0/faq) and should not be used alone for patch prioritization. - **EPSS** is [a calibrated probability](https://www.first.org/epss/using-epss) that exploitation activity will be observed across EPSS data partners in the next 30 days. FIRST says it is not a severity score and not a complete risk score. - **CISA's KEV catalog** is, in CISA's words, [the authoritative source of vulnerabilities that have been exploited in the wild](https://www.cisa.gov/known-exploited-vulnerabilities-catalog). CISA says organizations should use it as an input to their prioritization framework. > [!TIP] > Ask which metrics produced each CVSS score. FIRST says scores should be labelled CVSS-B, CVSS-BT, CVSS-BE or CVSS-BTE to show which metric groups produced them. NVD and similar sources typically publish only CVSS-B, so applying Threat and Environmental metrics is up to the CVSS consumer, meaning you. Our guide to [which vulnerabilities to patch first](https://computese.com/the-future-of-cybersecurity/) compares these inputs and ends with a patch policy a small IT team can adopt. Testers rank findings too. The [NCSC](https://www.ncsc.gov.uk/guidance/penetration-testing) says they may lack your business context and rate an issue lower or higher than you would, and that "risk assessment and decisions on the application of fixes are your responsibility". ## What do PCI DSS and NIST ask for, and how often should you test? The rows below are from [PCI DSS v4.0, as printed in its SAQ D for Merchants](https://listings.pcisecuritystandards.org/documents/PCI-DSS-v4-0-SAQ-D-Merchant.pdf) (April 2022). The Council's note on [v4.0.1](https://blog.pcisecuritystandards.org/just-published-pci-dss-v4-0-1) calls it a limited revision with no new or deleted requirements and says that once v4.0 was retired on 31 December 2024, v4.0.1 became the only active version, so confirm the current text before you quote it to an assessor. | Requirement | What it asks | How often | | --------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | | 11.3.1 Internal vulnerability scans | High-risk and critical vulnerabilities resolved and rescanned, by qualified personnel with organizational independence. No QSA or ASV is required | At least once every three months | | 11.3.1.2 Authenticated scanning | Internal scans are authenticated; systems that cannot accept credentials are documented | Required since 31 March 2025 (a best practice before then) | | 11.3.2 External vulnerability scans | Done by a PCI SSC Approved Scanning Vendor (ASV), to the ASV Program Guide's passing-scan rules | At least once every three months | | 11.3.1.3 and 11.3.2.1 Scans after a change | Internal: high-risk and critical vulnerabilities resolved. External: vulnerabilities scored 4.0 or higher by CVSS resolved. Neither requires an ASV | After any significant change | | 11.4.2 and 11.4.3 Internal and external penetration tests | Per the entity's methodology, by a qualified internal resource or external third party with organizational independence | At least once every 12 months, and after any significant infrastructure or application upgrade or change | | 11.4.4 Fix and retest | Exploitable vulnerabilities corrected according to the entity's assessment of risk, then penetration testing repeated to verify the corrections | After the fixes | PCI DSS leaves "significant change" to the entity. [PCI SSC's guidance](https://www.pcisecuritystandards.org/documents/Penetration-Testing-Guidance-v1_1.pdf) says it depends on your risk-assessment process, and that a change which could affect network security or allow access to cardholder data may count. The same guidance says simply running an automated tool does not satisfy the penetration testing requirement, because tools cannot interpret vulnerabilities or exposed services to assess the true risk. Outside PCI DSS, [NIST SP 800-115](https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-115.pdf) says that, given its high cost and potential impact, annual penetration testing may be sufficient, and that regularly scheduled scanning interspersed with periodic penetration testing can help prevent many types of attacks and reduce the impact of successful ones. Scan between tests, because the [NCSC](https://www.ncsc.gov.uk/guidance/penetration-testing) notes it is not uncommon for a year or more to pass between tests, so vulnerabilities could exist for long periods without you knowing. The program below is an example for a small or mid-sized company, not a standard. Where PCI DSS applies, it sets the floors. ![A wall planner: a thick timeline bar split into four equal parts carries many small grey ticks. One wide orange block stands on short legs above the third part, followed by a small loop arrow.](https://computese.com/images/blog/penetration-testing-vs-vulnerability-scanning/test-calendar.bbf1e83296-1536.webp) *Scans fill the months between tests, and a retest after the fixes closes the loop.* | When | What | Basis | | -------------------------- | -------------------------------------------------------------------------- | --------------------------------------------- | | Every build | DAST, SAST and SCA in the delivery pipeline | OWASP testing guide | | Monthly | Internal (authenticated) and external scans, fixed in risk order | PCI DSS minimum is quarterly | | After a significant change | Scan the change, and test the changed part | PCI DSS 11.3.1.3, 11.3.2.1, 11.4.2 and 11.4.3 | | Once a year | External and internal penetration test, plus web application and API tests | PCI DSS 11.4.1 to 11.4.3; NIST SP 800-115 | | After each round of fixes | Retest | PCI DSS 11.4.4 | ## What should you agree in writing before a test starts? NIST notes that issuing attacks to compromise systems is usually prohibited by an organization's own policies, and that [the rules of engagement (ROE) provide the authorization](https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-115.pdf) for assessors to do it. It recommends involving legal advisors for intrusive tests such as penetration testing. ![A dashed orange boundary encloses three linked servers and a laptop, while a fourth server outside it carries a closed padlock. A signed page with a pen, a clock and a telephone sit nearby.](https://computese.com/images/blog/penetration-testing-vs-vulnerability-scanning/scope-boundary.adffd3cb2b-1536.webp) *Whatever sits inside the written boundary can be tested; whatever sits outside stays untouched.* Settle these points in the scope document and the ROE, drawing on the [NCSC's scoping guidance](https://www.ncsc.gov.uk/guidance/penetration-testing) and PCI SSC's list of ROE considerations: 1. **Scope and exclusions.** List the hosts, applications, APIs and accounts in scope, and any system not authorized for testing, which NIST calls the "exclude list". NIST says assessors should verify that public IP addresses are under your organization's purview before testing begins. 2. **Test window.** State the hours, and name legacy systems with known issues with automated scanning. NIST's sample ROE notes it may be prudent to test an operational site in the evening rather than at peak business periods. 3. **Contacts and stop rules.** Name a technical contact who is available at short notice. PCI SSC's example for a previous or active compromise is to activate incident response and stop the test until it is resolved. 4. **Data handling.** Decide whether the tester must disclose passwords or sensitive data they reach, and whether they keep any. PCI SSC says only a proof-of-concept test should be performed, and any cardholder data obtained must be secured in accordance with PCI DSS. 5. **Security controls.** If an intrusion prevention system or web application firewall protects the application, decide whether it may block test traffic. PCI SSC advises avoiding interference, because the test should evaluate the services' susceptibility to exploitation, not the protection system. 6. **Success criteria and retest.** Define when the test is complete, for example compromise of the domain used by privileged users. Without that, PCI SSC says, the tester may exceed your boundaries and expectations. Include a retest of the fixes. Cloud providers add rules of their own. [AWS](https://aws.amazon.com/security/penetration-testing/) lets customers test their AWS infrastructure on a list of permitted services, such as EC2 instances, RDS and Lambda, without prior approval, but not AWS's own infrastructure or the AWS services themselves. It prohibits, among other things, denial of service and port, protocol and request flooding; DDoS simulation falls under a separate policy. [Microsoft's rules](https://www.microsoft.com/en-us/msrc/pentest-rules-of-engagement) allow fuzzing, port scanning and vulnerability assessments on your own Azure virtual machines, prohibit denial-of-service testing and network-intensive fuzzing, and apply equally to the resource owner and their authorized agents. > [!WARNING] > Your scope covers systems you manage, not your provider's. PCI SSC's guidance says scope may not include infrastructure that a third party provides, and that where a service agreement requires the third party's approval, you must receive it before the assessment takes place. ## What does a good penetration test report contain? [PCI SSC's guidance](https://www.pcisecuritystandards.org/documents/Penetration-Testing-Guidance-v1_1.pdf) is blunt: "Merely reporting lists of vulnerabilities is not helpful" and does not meet the intent of the penetration test. ![Two documents side by side. The left is a long page of identical grey rows. The right is shorter and shows three boxes joined by an orange arrow path that ends at a database cylinder.](https://computese.com/images/blog/penetration-testing-vs-vulnerability-scanning/report-route.1b6a17c140-1536.webp) *A scanner export lists rows; a real report shows the route an attacker took and what it reached.* Use PCI SSC's suggested outline and the NCSC's list as a checklist. A report worth paying for has: - A statement of scope, including the systems tested and why. - Methodology and limitations, such as the approach followed, test hours and restrictions. - A testing narrative: how testing progressed, and any issues met along the way. - Findings with a risk ranking, affected targets, references such as CVE or CWE, and how each was exploited. - Findings that were not exploited but still pose a risk. - How each severity was derived, with reasons for any change to a standard score. - An opinion on your own vulnerability assessment, with advice to improve it. - Cleanup steps, and a retest report listing the original findings and the retest results. A report that mainly lists scanner findings with the scanner's own severity labels is a scan export, not a test report. Ask for a debrief, where the testers walk through their findings and you can request clarification, and for a retest. ## What drives the cost, and what should you ask a provider? This guide quotes no prices, because none is honest without your scope. The NCSC calls penetration testing a [powerful but expensive operation](https://www.ncsc.gov.uk/guidance/penetration-testing) and says the scoping document should state the time frame and effort, usually in resource days. Three things the sources link to that effort: - **Size of scope.** The technical boundaries and the types of test expected. The NCSC says changing scope is likely to alter time frames and cost. - **How much the tester is told.** PCI SSC says black-box testing may need more time, money and resources. - **Constraints and reporting.** The NCSC says the scope should record out-of-hours testing, systems that need special handling and reporting requirements such as CVSS scores, plus any time constraints a provider must consider when allocating resources. Put these questions to every provider, against the same written scope: 1. **Who will test, and what have they tested before?** The NCSC says a test's quality is closely linked to the abilities of the testers. PCI SSC suggests asking about years of experience (the tester's and the firm's), work for organizations of similar size and scope, experience with your technologies, and references. 2. **What do their certifications show?** PCI SSC lists OSCP, CEH, GIAC, CREST and CHECK as examples, says certifications may indicate skill, does not endorse them, and says qualifications cannot be met by certifications alone, so ask about actual engagements. 3. **Are they independent of your systems?** PCI SSC says the tester must be organizationally separate from the management of the target systems. 4. **Which method will they follow?** PCI DSS asks for industry-accepted approaches. PCI SSC's guidance names examples, among them OSSTMM, NIST SP 800-115, the OWASP Testing Guide and the Penetration Testing Execution Standard. 5. **Is a retest included, and what does it cover?** ## A plan you can start this month 1. **List what you expose.** Domains, public IP ranges, web applications, APIs, cloud accounts, anything a third party hosts for you, and any [reverse proxy](https://computese.com/reverse-proxy/) in front of them. 2. **Get the requirement in writing.** Ask the customer, insurer or auditor which document asks for the test, how often, who may perform it and what the report must show. 3. **Scan and fix by risk.** Run internal (authenticated) and external scans, rank findings as above and rescan. Routine updates, covered in our website maintenance guide, address a common root cause NIST names: insufficient patch management. 4. **Write one scope sheet** from the checklist above, and collect two or three quotes against it. 5. **Book the test after your next big change, and book the retest with it.** 6. **File the evidence.** Keep reports, fix records and retest results for at least 12 months, the period PCI DSS 11.4.1 sets for penetration testing results and remediation activities results. Computese offers [security testing](https://computese.com/services/security-testing/), which covers vulnerability scanning and penetration testing. ## Key terms - **Vulnerability scan**: An automated check that matches hosts and software against a database of known vulnerabilities and reports where one may exist. It checks only for the possible existence of a weakness and does not exploit it to confirm it. - **Penetration test**: Security testing in which assessors mimic real-world attacks to identify ways around the security features of an application, system or network, often by combining several vulnerabilities to gain more access than one allows. - **Vulnerability assessment**: A systematic examination of an information system or product to determine the adequacy of security measures and identify security deficiencies, among other goals. This is NIST's glossary definition. - **Authenticated scan**: A vulnerability scan run with an account on the target, using privileges sufficient for a thorough scan. PCI DSS v4.0 has required authenticated internal scans since 31 March 2025, except on systems that cannot accept credentials. - **DAST**: Dynamic application security testing. Automated tools that scan a web application, normally from the outside, for flaws such as cross-site scripting, SQL injection and insecure server configuration. - **Rules of engagement (ROE)**: The document that authorizes assessors to carry out activities an organization's policies usually prohibit, such as attacks that compromise systems. It also sets the conditions and the degree of exploitation allowed. - **Black-, grey- and white-box testing**: Testing with no, partial or full knowledge of the target's internals. PCI SSC says PCI DSS penetration tests are typically white-box or grey-box. The NCSC calls the two ends closed-box and open-box testing. - **Approved Scanning Vendor (ASV)**: A scanning vendor approved by the PCI Security Standards Council. PCI DSS v4.0 requires external vulnerability scans at least every three months, performed by an ASV and meeting the ASV Program Guide's rules for a passing scan. - **CVSS and EPSS**: CVSS is an open framework that communicates the characteristics and severity of vulnerabilities, with base scores from 0.0 to 10.0. EPSS is the estimated likelihood that exploitation activity for a given vulnerability will be observed across EPSS data partners in the next 30 days. ## Common questions ### What is the difference between a vulnerability scan and a penetration test? A vulnerability scan is automated and checks hosts and applications only for the possible existence of known weaknesses. A penetration test is a manual process in which a tester tries to exploit weaknesses, often chaining several, to show how far an attacker could get. A scan takes seconds to minutes per host and PCI DSS asks for it at least quarterly; a test takes days or weeks and is asked for at least annually. ### Is a vulnerability assessment the same as a penetration test? No. NIST defines a vulnerability assessment as a systematic examination of an information system or product to determine the adequacy of security measures and identify security deficiencies, and says penetration testing goes beyond automated vulnerability scanning. The NCSC adds that a penetration test gives assurance in your vulnerability assessment and management processes; it is not a primary way to find vulnerabilities. ### How often should you run a penetration test? PCI DSS v4.0 requires internal and external penetration tests at least once every 12 months and after any significant infrastructure or application upgrade or change. NIST SP 800-115 says annual testing may be sufficient, given its high cost and potential impact. The NCSC notes it is not uncommon for a year or more to pass between tests, and NIST suggests regular scanning in between. ### Does PCI DSS require both vulnerability scanning and penetration testing? PCI DSS v4.0 sets them as separate requirements. Requirement 11.3 asks for internal and external vulnerability scans at least once every three months, with external scans done by a PCI SSC Approved Scanning Vendor. Requirement 11.4 asks for internal and external penetration testing at least once every 12 months, plus segmentation tests where segmentation is used. Check the current version of the standard before relying on this wording. ### What is the difference between a red team exercise and a penetration test? NIST SP 800-53 says red team exercises extend the objectives of penetration testing by examining the security and privacy posture of organizations and their capability to implement effective cyber defences. They simulate adversary attempts to compromise mission and business functions and may include social engineering. A penetration test is conducted on systems or individual system components. ### How much does a penetration test cost? It depends on the scope, and this guide quotes no prices. The NCSC describes the effort in resource days and says changing scope is likely to alter both time frames and cost. PCI SSC's guidance says a black-box test, where the tester gets no details, may require more time, money and resources. Ask every provider to quote against the same written scope. ## Sources 1. [SP 800-115: Technical Guide to Information Security Testing and Assessment (September 2008)](https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-115.pdf), NIST 2. [SP 800-53 Rev. 5: Security and Privacy Controls for Information Systems and Organizations](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-53r5.pdf), NIST 3. [Penetration testing: how to get the most from penetration testing](https://www.ncsc.gov.uk/guidance/penetration-testing), UK National Cyber Security Centre (NCSC) 4. [Information Supplement: Penetration Testing Guidance (version 1.1, September 2017)](https://www.pcisecuritystandards.org/documents/Penetration-Testing-Guidance-v1_1.pdf), PCI Security Standards Council 5. [PCI DSS v4.0 SAQ D for Merchants (April 2022)](https://listings.pcisecuritystandards.org/documents/PCI-DSS-v4-0-SAQ-D-Merchant.pdf), PCI Security Standards Council 6. [Just Published: PCI DSS v4.0.1](https://blog.pcisecuritystandards.org/just-published-pci-dss-v4-0-1), PCI Security Standards Council Blog 7. [Web Security Testing Guide v4.2: Introduction](https://wstg.owasp.org/v4.2/2-Introduction/), OWASP 8. [Vulnerability Scanning Tools](https://community.owasp.org/Vulnerability_Scanning_Tools), OWASP Foundation 9. [OWASP Mobile Application Security Testing Guide (MASTG)](https://mas.owasp.org/MASTG/), OWASP 10. [CVSS v4.0 Specification Document](https://www.first.org/cvss/v4.0/specification-document), FIRST 11. [CVSS v4.0 Frequently Asked Questions](https://www.first.org/cvss/v4.0/faq), FIRST 12. [Using EPSS](https://www.first.org/epss/using-epss), FIRST 13. [Known Exploited Vulnerabilities Catalog](https://www.cisa.gov/known-exploited-vulnerabilities-catalog), CISA 14. [Penetration Testing: AWS Customer Support Policy for Penetration Testing](https://aws.amazon.com/security/penetration-testing/), Amazon Web Services 15. [Microsoft Security Testing Rules of Engagement](https://www.microsoft.com/en-us/msrc/pentest-rules-of-engagement), Microsoft Security Response Center