Skip to content

👨‍🔧 Vulnerability Management and Remediation

Reports from different tools may be very hard to be orchestrated, and here is where Management tools like DefectDojo come in handy.

SAST Export Results

Most SAST tools allow to export the results as reports. For this example, we added output attributes aswell as artifact funtionality to upload these results to the pipeline build.

# Scan for Git Leaked credentials
gitleaks:
  stage: test
  image:
    name: zricethezav/gitleaks:latest
    entrypoint: [""]
  script:
    - gitleaks detect --verbose --source . -f json -r gitleaks.json #(1)
  allow_failure: true
  artifacts: 
    when: always
    paths:
      - gitleaks.json #(2)

# Scan for NodeJS SAST vulnerabilies
njsscan:
  stage: test
  image: python
  before_script:
    - pip3 install --upgrade njsscan
  script:
    - njsscan --exit-warning . --sarif -o njsscan.sarif #(3)
  allow_failure: true
  artifacts: 
    when: always
    paths:
      - njsscan.sarif #(4)

# Scan with SemGrep
semgrep:
  stage: test
  image: semgrep/semgrep
  variables:
    SEMGREP_RULES: p/javascript
  script:
    - semgrep ci --json --output semgrep.json #(5)
  allow_failure: true
  artifacts:
    when: always
    paths:
      - semgrep.json #(6)
  1. -f json -r gitleaks.json exports the file as json and as gitleaks.json.
  2. This artifact block will upload the gitleaks.json file on the current build always (even if task fails).
  3. --sarif -o njsscan.sarif exports the file as sarif and as njsscan.sarif.
  4. This artifact block will upload the njsscan.sarif file on the current build always (even if task fails).
  5. --json --output semgrep.json exports the file as json and as semgrep.json.
  6. This artifact block will upload the semgrep.json file on the current build always (even if task fails).

DefectDojo

We will use this tool to deploy it locally in order to understand how it works. Nana on the course used the Demo site, however, I wanted to try the Docker-compose version.

  • Deploy DefectDojo locally.
# 👨‍🔧 Vulnerability Management and Remediation

Reports from different tools may be very hard to be orchestrated, and here is where Management tools like [DefectDojo](https://defectdojo.com/) come in handy.

## SAST Export Results

Most SAST tools allow to export the results as reports. For this example, we added output attributes aswell as artifact funtionality to upload these results to the pipeline build.

```yaml
# Scan for Git Leaked credentials
gitleaks:
  stage: test
  image:
    name: zricethezav/gitleaks:latest
    entrypoint: [""]
  script:
    - gitleaks detect --verbose --source . -f json -r gitleaks.json #(1)
  allow_failure: true
  artifacts: 
    when: always
    paths:
      - gitleaks.json #(2)

# Scan for NodeJS SAST vulnerabilies
njsscan:
  stage: test
  image: python
  before_script:
    - pip3 install --upgrade njsscan
  script:
    - njsscan --exit-warning . --sarif -o njsscan.sarif #(3)
  allow_failure: true
  artifacts: 
    when: always
    paths:
      - njsscan.sarif #(4)

# Scan with SemGrep
semgrep:
  stage: test
  image: semgrep/semgrep
  variables:
    SEMGREP_RULES: p/javascript
  script:
    - semgrep ci --json --output semgrep.json #(5)
  allow_failure: true
  artifacts:
    when: always
    paths:
      - semgrep.json #(6)
  1. -f json -r gitleaks.json exports the file as json and as gitleaks.json.
  2. This artifact block will upload the gitleaks.json file on the current build always (even if task fails).
  3. --sarif -o njsscan.sarif exports the file as sarif and as njsscan.sarif.
  4. This artifact block will upload the njsscan.sarif file on the current build always (even if task fails).
  5. --json --output semgrep.json exports the file as json and as semgrep.json.
  6. This artifact block will upload the semgrep.json file on the current build always (even if task fails).

DefectDojo

We will use this tool to deploy it locally in order to understand how it works. Nana on the course used the Demo site, however, I wanted to try the Docker-compose version instead of the ephimeral DefectDojo Demo site.

  • Deploy DefectDojo locally.
# Clone the project
git clone https://github.com/DefectDojo/django-DefectDojo
cd django-DefectDojo

# Check if your installed toolkit is compatible
./docker/docker-compose-check.sh

# Building Docker images
docker compose build

# Run the application (for other profiles besides postgres-redis see  
# https://github.com/DefectDojo/django-DefectDojo/blob/dev/readme-docs/DOCKER.md)
docker compose up -d

# Obtain admin credentials. The initializer can take up to 3 minutes to run.
# Use docker compose logs -f initializer to track its progress.
docker compose logs initializer | grep "Admin password:"

Upload Reports to DefectDojo

We will upload reports automatically to DefectDojo form the pipeline.

  • Head to DefectDojo page and get the API Key: DefectDojo page > User Profile > API v2 Key and copy the key. For example 7e494885053ffdeb621d02909201581581872bc4.
import requests
import sys
import os

api_key = os.environ["DEFECTDOJO_API_KEY"]
file_name = sys.argv[1]
scan_type = '' 

if file_name == 'gitleaks.json':
    scan_type = 'Gitleaks Scan'
elif file_name == 'njsscan.sarif':
    scan_type = 'SARIF'
elif file_name == 'semgrep.json':
    scan_type = 'Semgrep JSON Report'
elif file_name == 'retire.json':
    scan_type = 'Retire.js Scan'

headers = {
  'Authorization': f'Token {api_key}'

}

url = 'http://95.17.91.21:8080/api/v2/import-scan/'

data = {
    'auto_create_context': True,
    'active': True, # it will mark the findings as active
    'verified': True, # mark as findings as verified automatically
    'scan_type': scan_type, # this is the type of scan we specified earlyer manually
    'minimum_severity': 'Low', # results with "info" severity won't be imported
    'product_name': 'Juice Shop',
    'engagement': 5
}

files = {
    'file': open(file_name, 'rb')
}

response = requests.post(url, headers=headers, data=data, files=files)


if response.status_code == 201:
    print('Scan results imported successfully')
else:
    print(f'Failed to import scan results: Status_Code: { response.status_code} - Response: {response.content}')

However, this is how the complete CICD pipeline looks like. I also added the "DEFECTDOJO_API_KEY" as a variable to the code, to avoid hardcoding any secret value.

variables:
  IMAGE_NAME: onisimiacob/demo-app
  IMAGE_VERSION: juice-shop-1.0

stages:
  - cache
  - test
  - build

# Improove build times with a cache task that saves the dependencies as an artifact
create_cache:
  image: node:18-bullseye
  stage: cache
  script:
    - yarn install --ignore-engines 
  cache:
    key:
      files:
        - yarn.lock
    paths:
      - node_modules
      - yarn.lock
      - .yarn
    policy: pull-push

# Test using the cache to speedup testing
yarn_test:
  image: node:18-bullseye
  stage: test
  script:
    - yarn install --ignore-engines # fix The engine "node" is incompatible with this module. Expected version "20 || >=22". Got "18.20.8"
    - yarn test
  cache:
    key:
      files:
        - yarn.lock
    paths:
      - node_modules
      - yarn.lock
      - .yarn
    policy: pull

# Scan for Git Leaked credentials
gitleaks:
  stage: test
  image:
    name: zricethezav/gitleaks:latest
    entrypoint: [""]
  script:
    - gitleaks detect --verbose --source . -f json -r gitleaks.json
  allow_failure: true
  artifacts:
    when: always
    paths:
      - gitleaks.json

# Scan for NodeJS SAST vulnerabilies
njsscan:
  stage: test
  image: python
  before_script:
    - pip3 install --upgrade njsscan
  script:
    - njsscan --exit-warning . --sarif -o njsscan.sarif
  allow_failure: true
  artifacts:
    when: always
    paths:
      - njsscan.sarif

# Scan with SemGrep
semgrep:
  stage: test
  image: semgrep/semgrep
  variables:
    SEMGREP_RULES: p/javascript
  script:
    - semgrep ci --json --output semgrep.json
  allow_failure: true
  artifacts:
    when: always
    paths:
      - semgrep.json

# Upload results to DefectDojo
upload_reports:
  stage: test
  image: python
  needs: ["gitleaks", "njsscan", "semgrep"]
  when: always
  variables:
    DEFECTDOJO_API_KEY: $DEFECTDOJO_API_KEY
  before_script:
    - pip3 install requests
  script:
    - python3 upload-reports.py gitleaks.json
    - python3 upload-reports.py njsscan.sarif
    - python3 upload-reports.py semgrep.json

# Build the docker image
build_image:
  stage: build
  image: docker:24
  services:
    - docker:24-dind
  variables:
    DOCKER_USER: $DOCKER_USER
    DOCKER_PASS: $DOCKER_PASS
  before_script:
    - echo $DOCKER_PASS | docker login -u $DOCKER_USER --password-stdin
  script:
    - docker build -t $IMAGE_NAME:$IMAGE_VERSION .
    - docker push $IMAGE_NAME:$IMAGE_VERSION

Gitlab Pipelines

This is how DefectDojo looks after pipeline gets executed and scans got imported.

Defectdojo