Skip to main content

Scorecard Data Sources

Harness IDP allows you to integrate various data sources and implement custom checks to ensure your software components adhere to best practices and compliance. In this docs, we'll walk through how to add custom checks and data sources for scorecards in Harness IDP.

Overview

Harness IDP allows you to integrate various data sources, such as GitHub, GitLab, Bitbucket, Azure DevOps, and many more, to collect specific data points for each software component. Once a data source is enabled, you can use them to create checks to be used in scorecards.

Add Custom Checks

  1. Under the Admin go to Scorecards.
  2. In the Scorecards go to the Checks tab and select Create Custom Check.
  3. Now on the Create Check page add a name and description for your check.
  4. Under Rules you can find the following Data Sources to select from.
info

There's a tab called Data Sources available in Scorecards page to check for supported data sources and the corresponding data points.

caution

The git (GitHub, GitLab, Bitbucket) datasources doesn't support monorepos.

The following Data Points are available for GitHub Data Source.

  1. Branch Protection
  • Objective: Ensure that branch protection rules disallow force push and delete.
  • Calculation Method: Fetch backstage.io/source-location annotation from the catalog YAML file to find repository details and verify the branch protection rules.
  • Prerequisites: Github Connector with Admin access. Provide suitable backstage.io/source-location annotation if the catalog YAML file is present outside the source GitHub repository.

Example YAML

kind: "Component"
apiVersion: "backstage.io/v1alpha1"
metadata:
name: order-service
annotations:
backstage.io/source-location: 'url:https://github.com/kubernetes/kubernetes/tree/master'
...
spec:
...
  1. File Existence
  • Objective: Verify the existence of a specified file in the repository.
  • Calculation Method: Use the backstage.io/source-location annotation to locate the repository and check for the file’s presence. Make sure to mention the filename with extension or relative path from the root folder (Eg: README.md or docs/README.md) in the conditional input field.
  • Prerequisites: Provide suitable backstage.io/source-location annotation if the catalog YAML file is present outside the source GitHub repository.

Example YAML

kind: "Component"
apiVersion: "backstage.io/v1alpha1"
metadata:
name: order-service
annotations:
backstage.io/source-location: 'url:https://github.com/kubernetes/kubernetes/tree/master'
...
spec:
...
  1. Mean Time to Merge Pull Request
  • Objective: Calculate the average time taken to merge the last 100 pull requests.
  • Calculation Method: Retrieve repository details using backstage.io/source-location and calculate the average merge time.
  • Prerequisites: Provide suitable backstage.io/source-location annotation if the catalog YAML file is present outside the source GitHub repository. Make sure to mention the branch name in the conditional input field.

Example YAML

kind: "Component"
apiVersion: "backstage.io/v1alpha1"
metadata:
name: order-service
annotations:
backstage.io/source-location: 'url:https://github.com/kubernetes/kubernetes/tree/master'
...
spec:
...
  1. Average time to complete successful workflow runs (in minutes)
  • Objective: Calculate the average time taken to complete successful workflow runs (in minutes).
  • Calculation Method: Fetches backstage.io/source-location annotation from catalog YAML file to find repository details and calculates the average time for the last 100 successful workflow runs to complete.
  • Prerequisites: Provide suitable backstage.io/source-location annotation if the catalog YAML file is present outside the source GitHub repository. Make sure to mention the workflow id or filename in the conditional input field.
  1. Average time to complete workflow runs (in minutes)
  • Objective: Calculate the average time taken to complete workflow runs (in minutes).
  • Calculation Method: Fetches backstage.io/source-location annotation from catalog YAML file to find repository details and calculates the average time for the last 100 workflow runs to complete.
  • Prerequisites: Provide suitable backstage.io/source-location annotation if the catalog YAML file is present outside the source GitHub repository. Make sure to mention the workflow id or filename in the conditional input field.
  1. Workflow success rate
  • Objective: Calculates success rate for the given workflow.
  • Calculation Method: Fetches backstage.io/source-location annotation from catalog YAML file to find repository details and calculates the success rate for the workflow.
  • Prerequisites: Provide suitable backstage.io/source-location annotation if the catalog YAML file is present outside the source GitHub repository. Make sure to mention the workflow id or filename in the conditional input field.
  1. Workflows count
  • Objective: Calculates total number of workflows.
  • Calculation Method: Fetches backstage.io/source-location annotation from catalog YAML file to find repository details and calculates the total number of workflows
  • Prerequisites: Provide suitable backstage.io/source-location annotation if the catalog YAML file is present outside the source GitHub repository.
  1. Open code scanning alerts
  • Objective: Calculates the total number of open alerts reported in code scanning for the given severity.
  • Calculation Method: Fetches backstage.io/source-location annotation from catalog YAML file to find repository details and calculates the total number of open alerts reported in code scanning.
  • Prerequisites: GitHub Connector with read access for code scanning alerts. Provide suitable backstage.io/source-location annotation if the catalog YAML file is present outside the source GitHub repository. Make sure to mention the severity type in the conditional input field.
  1. Open Dependabot alerts
  • Objective: Calculates the total number of open alerts reported by Dependabot for the given severity.
  • Calculation Method: Fetches backstage.io/source-location annotation from catalog YAML file to find repository details and calculates the total number of open alerts reported by Dependabot.
  • Prerequisites: Provide suitable backstage.io/source-location annotation if the catalog YAML file is present outside the source GitHub repository. Make sure to mention the severity type in the conditional input field.
  1. Open secret scanning alerts
  • Objective: Calculates the total number of open alerts reported in secret scanning.
  • Calculation Method: Fetches backstage.io/source-location annotation from catalog YAML file to find repository details and calculates the total number of open alerts reported in secret scanning.
  • Prerequisites: GitHub Connector with read access for secret scanning alerts. Provide suitable backstage.io/source-location annotation if the catalog YAML file is present outside the source GitHub repository.
  1. Open pull requests by account
  • Objective: Calculates the total number of open pull requests raised by the given account.
  • Calculation Method: Fetches backstage.io/source-location annotation from catalog YAML file to find repository details and calculates the total number of open pull requests raised by account.
  • Prerequisites: Provide suitable backstage.io/source-location annotation if the catalog YAML file is present outside the source GitHub repository. Make sure to mention the username in the conditional input field.
  1. Now add a tag under which category your check belongs to , Ex. "Developer Productivity", "Software Maturity" and click enter to add each tags.

  2. Now add the default result in case of missing data and Save Changes. Your checks will be added.