Improving vulnerability management at Masaryk University

We had no overview of our network's vulnerabilities, and we needed a solution that would help us identify, prioritize, and report them to different administrators.

To achieve this, we built a modular, automated, and open-source solution using trusted tools like BBOT for asset discovery and Nuclei for vulnerability scanning orchestrated with custom Python scripts.

Now, we have a much better overview of present vulnerabilities in our network and can prioritize them based on their severity.

Table of contents

1. Introduction
2. Core principles
   • Extensibility and flexibility
   • Automation
   • Open-Source
3. Technical details
   • Features
   • Deployment
   • Example use-case
4. Conclusion

Introduction

Vulnerabilities are inevitable in software systems, what makes a difference is organizations' ability to manage them.

Lately, as the cybersecurity team at Masaryk University, we have been working on improving our approach to handling vulnerabilities. This post is about our processes regarding vulnerability discovery, not much about vulnerability mitigation.

After testing countless tools and methods, we have produced something resembling a solution, and we strongly   believe that our approach can be adapted by universities and organizations worldwide, helping them achieve more graceful and effective vulnerability management processes.

In our approach, we combine multiple trusted open-source tools that have proved to be helpful for our everyday operations. We are delighted to see that we are not alone in our vision, and other cybersecurity companies are taking a similar route -- reinforcing the idea of cleverly combining already established tools.

Our approach is based on the recursive internet scanner BBOT from Black Lantern Security and the vulnerability scanner Nuclei from Project Discovery glued together with some Python code for orchestration.
 

Core principles

In our approach we focus on three core principles:

• Extensibility and flexibility via a modular architecture.
• Full-fledged Automation.
• Staying open source.

Extensibility and flexibility

Our first guiding principle was to create a solution that can be easily changed and extended. We wanted to avoid "locking" into specific tools and technologies; instead, we focused on defining the tasks needed to achieve our goal. The following tasks, can be then backed by different technologies:

1. New Targets: A mechanism to gather and store a list of target hosts in a database.
2. Recurring Scans: Fetching the actual set of targets from the database and scanning them regularly.
3. New Vulnerabilities: Continuously receiving updates on newly discovered vulnerabilities.
4. Reporting: Presenting findings in a user-friendly and actionable format.

We currently use BBOT & Massscan, Nuclei and GitLab to perform these tasks. However, we remain flexible and can add or replace sources of targets, scanners, or report destinations.

Automation

Automation is at the heart of our approach. The goal is to set up the tooling once and let it run with minimal user intervention. We still recognize that some tasks require human expertise, such as triaging vulnerabilities, filtering out false positives, and communicating with system owners.

These tasks require human expertise, but if feasible, we aim to automate as much as possible. Deployment of the entire solution is simplified through Ansible, ensuring that anyone can easily set up and deploy the tooling.

Open-Source

We prioritize the use of open-source tools. Even though we did experiments with commercial scanners, our methodology for discovering new vulnerabilities benefits from the agility and responsiveness of community-driven tools. The open-source community's rapid pace of innovation is an advantage for us in the short term, surpassing the possible challenges in the long term.

The applicability of our architecture

The most valuable aspect of our work is not the code itself but the architecture. Each university, company, or organization has unique needs and an existing infrastructure. However, the presented concepts can be adapted to fit a wide range of vulnerability management scenarios.

We have created a robust, adaptable, and efficient solution for vulnerability discovery and reporting by adhering to the mentioned core principles.

Technical details

The solution consists of the following three main components.

• Asset discovery
• Database
• Vulnerability scanner

Target scanners are responsible for finding and storing new targets in the database. We use BBOT and Masscan for this purpose. Nuclei and other tools fetch targets from the database and scan them for vulnerabilities. New vulnerabilities are saved in the database and reported to GitLab.

The following diagram displays the architecture of our approach [see Img-1].

Features

Our framework enhances tools like BBOT and Nuclei (in a standalone deployment) via the unique features provided by our custom orchestration code. For instance:

• To optimize efficiency and avoid redundant scans, we enforce that only domains that have not been scanned within a specified period are checked.
• To guarantee that our scans remain current and comprehensive, templates for newly discovered vulnerabilities are periodically retrieved
• To simplify communication with system owners, reports are generated based on predefined templates. These templates include prepared remediation steps, relevant resources, and additional information tailored to specific vulnerabilities, such as XSS, SQLi, etc.
• To ease data retrieval and management, the inventory is simplified to store all targets in a database.
• To provide visibility into the scanning process and quick issue identification and resolution, logging and monitoring are built in.

Deployment

Our solution is organized across three main repositories, each serving a distinct purpose, the first being the Ansible repository which is responsible for deploying both the Domain Discovery and Nuclei repositories.

1. Ansible Repository
   
   This repository contains the Ansible playbooks and roles required to deploy the framework. It manages the configuration and setup of all necessary components, ensuring a seamless deployment process. Ansible sets up two repositories as systemd services and installs all the required dependencies.
    
2. Domain Discovery Repository This repository focuses on identifying new targets. It includes:
   • Python Wrappers for tools like BBOT (our primary source for discovering new targets) and Masscan.
   • A Parsing Script that processes scanners' output and stores the parsed results in a PostgreSQL database. [see Img-2]
3. Nuclei Repository This repository handles vulnerability scanning and reporting. It includes:
   • Python Wrappers for Nuclei, which scans identified targets for vulnerabilities.
   • An after-scan script is used to store findings in a database. Alternatively, GitLab issues are created and assigned to the appropriate person. [see Img-3]

The after-scan script uses regex to recognize the type of vulnerability and assign it to the appropriate template to make reporting to the system owner easier.

All scripts are designed to work with a database, whether it is hosted locally or remotely. While our current implementation uses PostgreSQL, the flexibility provided by SQLAlchemy enables adaptation to other database backends if required. Additionally, Ansible also includes a playbook to set up the PostgreSQL database.

Our philosophy of choosing technologies, languages, and tools emphasizes simplicity and ease of use. This can be beneficial for other Universities looking to improve their vulnerability management processes.

[see Img-4]

Source: https://www.ibm.com/think/topics/vulnerability-management-lifecycle

Example use-case

To illustrate how our solution works in practice, let's consider a scenario:

A new vulnerability emerges; within a few hours, a Nuclei template is created. Within weeks, it gets merged into the main repository. Our Nuclei scanner eventually picks up the new template and scans our network to see if a new vulnerability is present in our infrastructure. Moreover, if a vulnerability has critical severity, we perform the immediate ad-hoc scan.

If a vulnerability is present, the after-scan script will create a GitLab issue and assign it to the responsible personnel, responsible for triaging the vulnerability and reporting it to the system owners.

Conclusion

This blog post presents the approach of improving vulnerability management at Masaryk University. Guided by flexibility, automation, and open-source collaboration principles, the proposed approach demonstrates how established tools like BBOT and Nuclei, governed by custom integrations, can streamline the complex process of vulnerability discovery and reporting.

We encourage other universities and organizations to explore and adapt our methodology to their own environments. We are happy to assist you and collaborate if you want to improve your vulnerability management processes or build a similar solution. Feel free to reach out to us at csirt-info@muni.cz.

For more details on setting up our tooling, refer to the links below, and start with ansible.

1. Ansible Repository: https://github.com/SOCCER-Project-DEP/vulnman-ansible
2. Domain Discovery Repository: https://github.com/SOCCER-Project-DEP/vulnman-domain-discovery
3. Nuclei Repository: https://github.com/SOCCER-Project-DEP/vulnman-nuclei-orchestrator

Thank you for reading!