In this editorial, we will delve into the characteristics and differences of the most authoritative and currently relevant standards for designing and executing Penetration Tests:
Penetration Testing is a serious matter
The growth of a company, whether small or large, today passes through IT. Networks, the web, and computers are powerful tools capable of supporting business processes.
These technologies must receive preferential treatment because the benefits they bring far outweigh their cost. However, the mistake often made is believing that once implemented, computer networks, applications, and management systems do not require maintenance and verification of IT security (which in recent years we call Cyber Security).
It is essential that the entrepreneur and their management team request the execution of penetration tests, a type of quality check capable of identifying design errors and vulnerabilities.
Only in this way can one understand the risk to which they are exposed and have the necessary information to act accordingly.
A breach in a company's IT security causes economic losses due to the inability to operate normally, but also costs due to the work hours needed to restore the situation to normal.
Furthermore, the compromise of systems can lead to violations of customer privacy protection laws (see GDPR, for example) and, with the obligation to report Data Breaches that these regulations introduce, ruin the company's reputation.
It is possible to mitigate security-related risks by discovering vulnerabilities in one's infrastructure through Penetration Tests and Vulnerability Assessments (VA/PT).
Turning to an expert CyberSecurity team capable of conducting complete and effective Penetration Tests will allow you to know the risks you face and learn how to mitigate them.
The importance of Penetration Test methodologies
The Penetration Testing process must produce founded, complete, and structured results. With the spread of this practice, it has become necessary to develop methodologies to standardize the process and make it reliable and complete.
Thus, several standards have emerged, some with the aim of providing guidelines for testing a company's security in all aspects, others focusing on more restricted application domains.
These methodologies are adopted worldwide by professionals and companies to manage their needs in terms of CyberSecurity and Pentest and represent the de facto standard for security testing and secure development procedures.
OSSTMM (Open Source Security Testing Methodology Manual)
The OSSTMM methodology is a "peer-reviewed" methodology (i.e., subjected to a peer review process) for testing corporate security. This methodology, distributed in the form of a manual, as the name suggests, is maintained by the Institute for Security and Open Methodologies (ISECOM) and updated semi-annually to keep up with new technologies available on the market.
OSSTMM is internationally recognized and is mainly used for Network Penetration Test activities, although its scope is not limited to this, as we will see later.
The purpose of this methodology is to apply the scientific method to any type of security test applicable in a company. This is to provide companies that employ the OSSTMM methodology with the assurance of having undertaken a concrete process.
As mentioned earlier, OSSTMM not only covers ethical hacking and penetration testing, but it also provides guidelines for three classes of security verification:
-
Physical Security (PHYSSEC)
-
Spectrum Security (SPECSEC)
-
Communications Security (COMSEC)
These classes are further divided into channels, which dictate more detailed rules for testing.
Part of the PHYSSEC class includes:
-
Human Security
That is, testing human security. The purpose of this type of testing is to evaluate the security of interactions between people and between people and technology. -
Physical Security
It is the testing of the tangible part of the company. This type of security test evaluates the security of physical security elements such as doors and other entryways.
SPECSEC includes only one channel:
-
Wireless Security
Security testing related to wireless communication. More generally, this type of test evaluates the security of all types of signals on the electromagnetic spectrum considered communication, such as radio signals.
The channels part of COMSEC are:
-
Telecommunications Security
Testing aimed at evaluating the security of analog and digital communications that pass through telephone or wired networks. -
Data Network Security
It is the testing of network security. It involves evaluating the security of electronic systems that distribute or route data. This testing includes both wired and wireless components.

OWASP (Open Web Application Security Project)
The OWASP project offers free and open-source standards and tools for security testing and secure development of web applications.
The goal of this project is to raise awareness of web security and enable companies and anyone offering a web service to make informed security choices.
As mentioned earlier, OWASP offers several free software and tools. Particularly important are the framework for secure web application development and a methodology for penetration testing of web applications based on HTTP (i.e., websites and web applications).
The OWASP standard includes the following steps for testing web applications:
-
Information gathering
That is, the process of collecting information about the application in question. -
Configuration management testing
Collection of technical data on the operation of the application, such as the version of the protocols used, authentication methods, and related aspects. -
Authentication testing
Testing of user authentication processes. This step includes auditing password security and user enumeration. -
Session management testing
Evaluation of how the site interacts with the user. -
Authorization testing
Phase aimed at evaluating how a user can interact with the service and what actions they can perform. -
Business logic testing
Tests to evaluate whether the logic implemented in the application works as expected or contains vulnerabilities. -
Data validation testing
Evaluation of the methods for validating data entered by the user. This test is particularly relevant because it focuses on discovering some of the most common vulnerabilities such as SQL injection and Cross Site Scripting (XSS). -
Denial of service attacks testing
Tests to evaluate the site's behavior under DoS attacks not only based on the excessive amount of traffic sent to the server. -
Web services testing
Vulnerability testing on the standards adopted by the web platform. -
AJAX testing
Vulnerability testing in the methods of using AJAX for asynchronous requests.
As you can see, the OWASP standard is extremely detailed and rich in guidelines for Penetration Testing of web applications.
This type of testing can be useful for companies with a strong web presence to verify the security of their e-commerce portals, applications, and web-accessible management systems, and reserved areas.
NIST
The NIST methodology offers specific guidelines for penetration testing and focuses on keeping critical infrastructures secure.
The typical targets for Pentest according to the NIST methodology are usually in the banking, energy, or communications sectors.
To understand what the NIST standard is, you can see in the figure the fundamental steps of the NIST CyberSecurity framework:

PTES
The PTES is a penetration testing methodology slightly different from those described so far.
In fact, according to this standard, the test should begin with a communication phase between the penetration tester and the company in which the details of the infrastructure are communicated.
This will allow the penetration tester to focus on the exploitation and post-exploitation phases, useful for the company to understand the concrete risks in the event of a cyber attack.
Although testers are influenced by knowledge of the infrastructure, the results obtained can be similar to those of an attack by an employee and therefore not unrealistic.
An open dialogue about the technologies to be used also allows for precisely defining the perimeter on which penetration testers should focus, possibly limiting the exposure of sensitive data.
ISSAF (Information Systems Security Assessment Framework)
Testing according to the ISSAF framework focuses on systems, networks, and applications.
Similar to OSSTMM, it proposes itself as a standard for testing various aspects of the company, with the difference that ISSAF focuses only on the IT aspects of security.
The steps of ISSAF testing are similar to those already encountered earlier. You can see a testing scheme in the image below.

RSA
The RSA framework is different from the methodologies analyzed earlier.
Although the purpose is similar, namely to assess the security of an infrastructure, the RSA framework does not provide guidelines on how to conduct the penetration test. Rather, given an assessment of the company's security, it estimates a maturity model of the security in place.
The result of a test with the RSA framework will result in a security assessment, which can highlight risks with one of these five levels of severity:
-
Critical
-
High
-
Medium
-
Low
-
Informational
The RSA methodology is therefore useful where one wants to assess security as a whole and understand how to improve it at a macroscopic level.
Conclusions
Penetration Testing is a powerful tool capable of providing a company with indications on how to improve its security and that of its customers. Identifying risk based on objective evidence, with detailed and structured results in an official report from a third party, is essential to strengthen security and avoid potentially exorbitant economic damage.
It is important to carefully choose the penetration testing methodologies most suited to your needs, perhaps seeking advice, and turning to a serious partner.
References and citations:
(1) Comparative Study of Penetration Test Methods
Yong-Suk Kang, Hee-Hoon Cho, Yongtae Shin, and Jong-Bae Kim
https://pdfs.semanticscholar.org/be2d/4acf60ff2a37f196aaef48649eaa89ca36ac.pdf
(2) Study: A Penetration Testing Model
(3) Selection of penetration testing methodologies: A comparison and evaluation
Aleatha Shanley, Michael N. Johnstone
OSSTMM Testing Classes | |||
---|---|---|---|
Category | Channel | Test Object | Verification Elements |
PHYSSEC | Human Security | People | Policy compliance, Social Engineering, Phishing |
PHYSSEC | Physical Security | Places and things | Unauthorized access to premises, keyloggers, spies, and environmental interceptions |
SPECSEC | Wireless Security | Radio communications | Open networks, unauthorized networks, weak passwords, architecture and configuration errors |
COMSEC | Telecommunications Security | Digital communications | Network architectures, Switch security, Firewall rules, Encryption, Authentication |
COMSEC | Data Network Security | Systems and applications | Configuration, Update, Code vulnerabilities |