Writing a technical good report is just one key aim when it comes to formalizing a security report after a Penetration Test or Attack Simulation (e.g. Red Team, TIBER) exercise. Beside creating a good structure, the writer of a report also needs to know and understand their audience. This includes that the reader might not has in-depth knowledge of described vulnerabilities or that the reader will read the full report.

In this post we give some insights into our approach and key aims of our reporting format and how we tackle common problems with security reports.

Understanding the Audience & Requirements

We have written hundreds of reports over the last decades and also seen reports written by competitor (you can find some of them in this GitHub repository). We have reviewed reports from colleagues and there have been good examples, which were well structured and also had good technical descriptions to proof what has been done. On the other hand we have seen reports, which were written on such a technical level that likely no one who doesn’t deal with security on a daily base could understand it completely.
Over the years we have learned that three types of reader are usually receiving the report, who have different experience and aims when processing the report.

Understanding the requirements of each reader and assuming that every single piece of information may be useful has helped us to create new holistic reporting styles for Penetration Testing reports and Attack Simulation reports.

Aims of our reports

There was one functionality, which we missed in most (if not all) reports we have read: Easy navigation throughout the report. Reports were written in a form that required you to read them from top to the end. In some cases you had to jump a few pages (or a few more) down to the Appendix of the document to get further details about how a vulnerability was exploited. Unfortunately, there was no way back at that point to the position you have been beforehand. The only solution available for the reader was scrolling back or searching for a text string in the document to find the position they have been. Therefore, Interactive Navigation has been set as one of the key goals of our reports.

The second key goal we have set was Technical Accuracy. Mostly, developer have to work with the technical findings to reproduce them and deploy fixes to remediate the vulnerability. The goal we have set for this was to provide as much information as possible within the report, which can be read, but does not have to be. Therefore, we have included a generic description of the vulnerably, an in-depth remediation recommendation (where possible) and redirects to external sources holding further information. This is all combined with step-by-step instructions to replay an attack and detailed evidence including screenshots and exploit code. Especially in our Attack Simulation reports this is also supplemented by a per-minute time log of actions performed.

Readability and optical look was the third key goal we defined. Some vulnerabilities exist, which you will find in nearly every report and there is not much benefit for a reader to re-read the whole technical description again and again – just to find the recommendation that should be implemented. In our reports we give information on different levels starting from a two sentence vulnerability description and recommendation to a full in-depth description with all information needed to reproduce the issue.

Pentest Report vs Attack Simulation Report

Handling a Penetration Test report is something completely different to handling an Attack Simulation (e.g. Red Team, TIBER) report. Even though they both provide insights into the project, the Attack Simulation report is more likely to affect an organization as a whole. Whenever we describe the main difference between Penetration Testing and Attack Simulation, we give this example.

If we have the goal set to gain access to an administrative panel of a web application, then we would only use the web application itself as a vector in a Penetration Test. In an Attack Simulation we would also use alternative vectors like for example Spear-Phishing against an administrator of the application or other applications possibly residing in the same network segment.

Vulnerabilities discovered in an Attack Simulation are also more likely to affect existing procedures and global defense mechanisms. This also means that a broader audience with different background knowledge will (partially) read the report as they will be involved in remediation planning and risk assessments. Further information about our approaches to Attack Simulation and TIBER are also mentioned in more depth in our recent posts.

The best experience you can get from our reports is by navigating through them yourself, but we have also prepared teaser showing some parts of each report. Please note that the reports also contain further sections and information not shown in the teaser video.

Penetration Testing Report

The Penetration Test report provides insights for the Management and Project Owner as part of two test summaries with a different level of technical knowledge required. The Project Owner can easily identify, which vulnerabilities affects a specific user group and jump throughout the report from e.g. the Overview of Findings page to get a rough understanding of identified vulnerabilities. The Overview of Findings holds a short description and associated risk key together with references to further technical information, which are mostly read by Technical Member of staff.

The vulnerability itself contains references to external sources such as CWE, OWASP Web Security Testing Guide and further sources where required. With the introduced Interactive Navigation the reader can easily jump forwards and backwards within the report and always jump back to the position they have been before.

Attack Simulation Report

The Attack Simulation Report provides insights for the Project Owner and Manager with details about the Attack Simulation (e.g. scope, scenarios), results, and recommendation to enhance existing security controls and increase the cyber resilience. The sections intended for those groups once again have references to each others which allow the reader to navigate from e.g. a Scenario Overview, into the High Level details of an Attack, through the kill-chain and back. A video is worth a thousand words 🙂

As a Technical Member of staff (especially when working in the Blue Team) you are likely interested in the performed attack scenario and what TTP have been combined to reach the goal of the assessment. One starting point can be the Overview of Findings page, which lets the user easily navigate to identified core issues, associated vulnerabilities and detailed evidence about the exploitation. With each attack action we also list key events directly within the report as those might be the first step to handle in the remediation plan (more detailed information is given in a second document). Key Events can be followed to navigate through the technical information and then navigate back to the starting point. Once again, a video is worth a thousand words:

Get In Touch With Us For Your Copy

We have created new reporting templates which combine our long term experience in holistic reports. They hold references which make it easy to navigate throughout the reports and the reports provide reproducible attack chain insights to make a risk aware decisions. If you are interested in our sample reports, then give us a Ping via request@y-security.de.

Author