diff --git a/README.md b/README.md index a4cacda2..42b54828 100644 --- a/README.md +++ b/README.md @@ -119,19 +119,20 @@ Here are some advantages of PR-Agent: PR-Agent and Qodo Merge offer comprehensive pull request functionalities integrated with various git providers: -| | | GitHub | GitLab | Bitbucket | Azure DevOps | Gitea | -|---------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------|:------:|:------:|:---------:|:------------:|:-----:| -| [TOOLS](https://qodo-merge-docs.qodo.ai/tools/) | [Describe](https://qodo-merge-docs.qodo.ai/tools/describe/) | ✅ | ✅ | ✅ | ✅ | ✅ | -| | [Review](https://qodo-merge-docs.qodo.ai/tools/review/) | ✅ | ✅ | ✅ | ✅ | ✅ | -| | [Improve](https://qodo-merge-docs.qodo.ai/tools/improve/) | ✅ | ✅ | ✅ | ✅ | ✅ | -| | [Ask](https://qodo-merge-docs.qodo.ai/tools/ask/) | ✅ | ✅ | ✅ | ✅ | | -| | ⮑ [Ask on code lines](https://qodo-merge-docs.qodo.ai/tools/ask/#ask-lines) | ✅ | ✅ | | | | -| | [Help Docs](https://qodo-merge-docs.qodo.ai/tools/help_docs/?h=auto#auto-approval) | ✅ | ✅ | ✅ | | | -| | [Update CHANGELOG](https://qodo-merge-docs.qodo.ai/tools/update_changelog/) | ✅ | ✅ | ✅ | ✅ | | -| | [Add Documentation](https://qodo-merge-docs.qodo.ai/tools/documentation/) 💎 | ✅ | ✅ | | | | -| | [Analyze](https://qodo-merge-docs.qodo.ai/tools/analyze/) 💎 | ✅ | ✅ | | | | -| | [Auto-Approve](https://qodo-merge-docs.qodo.ai/tools/improve/?h=auto#auto-approval) 💎 | ✅ | ✅ | ✅ | | | -| | [CI Feedback](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) 💎 | ✅ | | | | | +| | | GitHub | GitLab | Bitbucket | Azure DevOps | Gitea | +|---------------------------------------------------------|----------------------------------------------------------------------------------------|:------:|:------:|:---------:|:------------:|:-----:| +| [TOOLS](https://qodo-merge-docs.qodo.ai/tools/) | [Describe](https://qodo-merge-docs.qodo.ai/tools/describe/) | ✅ | ✅ | ✅ | ✅ | ✅ | +| | [Review](https://qodo-merge-docs.qodo.ai/tools/review/) | ✅ | ✅ | ✅ | ✅ | ✅ | +| | [Improve](https://qodo-merge-docs.qodo.ai/tools/improve/) | ✅ | ✅ | ✅ | ✅ | ✅ | +| | [Ask](https://qodo-merge-docs.qodo.ai/tools/ask/) | ✅ | ✅ | ✅ | ✅ | | +| | ⮑ [Ask on code lines](https://qodo-merge-docs.qodo.ai/tools/ask/#ask-lines) | ✅ | ✅ | | | | +| | [Help Docs](https://qodo-merge-docs.qodo.ai/tools/help_docs/?h=auto#auto-approval) | ✅ | ✅ | ✅ | | | +| | [Update CHANGELOG](https://qodo-merge-docs.qodo.ai/tools/update_changelog/) | ✅ | ✅ | ✅ | ✅ | | +| | [Add Documentation](https://qodo-merge-docs.qodo.ai/tools/documentation/) 💎 | ✅ | ✅ | | | | +| | [Analyze](https://qodo-merge-docs.qodo.ai/tools/analyze/) 💎 | ✅ | ✅ | | | | +| | [Auto-Approve](https://qodo-merge-docs.qodo.ai/tools/improve/?h=auto#auto-approval) 💎 | ✅ | ✅ | ✅ | | | +| | [CI Feedback](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) 💎 | ✅ | | | | | +| | [Compliance](https://qodo-merge-docs.qodo.ai/tools/compliance/) 💎 | ✅ | ✅ | ✅ | | | | | [Custom Prompt](https://qodo-merge-docs.qodo.ai/tools/custom_prompt/) 💎 | ✅ | ✅ | ✅ | | | | | [Generate Custom Labels](https://qodo-merge-docs.qodo.ai/tools/custom_labels/) 💎 | ✅ | ✅ | | | | | | [Generate Tests](https://qodo-merge-docs.qodo.ai/tools/test/) 💎 | ✅ | ✅ | | | | diff --git a/docs/docs/index.md b/docs/docs/index.md index f67a8dbb..b4738d80 100644 --- a/docs/docs/index.md +++ b/docs/docs/index.md @@ -39,6 +39,7 @@ PR-Agent and Qodo Merge offer comprehensive pull request functionalities integra | | [Analyze](https://qodo-merge-docs.qodo.ai/tools/analyze/) 💎 | ✅ | ✅ | | | | | | [Auto-Approve](https://qodo-merge-docs.qodo.ai/tools/improve/?h=auto#auto-approval) 💎 | ✅ | ✅ | ✅ | | | | | [CI Feedback](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) 💎 | ✅ | | | | | +| | [Compliance](https://qodo-merge-docs.qodo.ai/tools/compliance/) 💎 | ✅ | ✅ | ✅ | | | | | [Custom Prompt](https://qodo-merge-docs.qodo.ai/tools/custom_prompt/) 💎 | ✅ | ✅ | ✅ | | | | | [Generate Custom Labels](https://qodo-merge-docs.qodo.ai/tools/custom_labels/) 💎 | ✅ | ✅ | | | | | | [Generate Tests](https://qodo-merge-docs.qodo.ai/tools/test/) 💎 | ✅ | ✅ | | | | diff --git a/docs/docs/tools/compliance.md b/docs/docs/tools/compliance.md new file mode 100644 index 00000000..4e5ae111 --- /dev/null +++ b/docs/docs/tools/compliance.md @@ -0,0 +1,300 @@ +`Platforms supported: GitHub, GitLab, Bitbucket` + +## Overview + +The `compliance` tool performs comprehensive compliance checks on PR code changes, validating them against security standards, ticket requirements, and custom organizational compliance checklists. + +=== "Fully Compliant" + ![compliance_overview](https://codium.ai/images/pr_agent/compliance_full.png){width=256} + +=== "Partially Compliant" + ![compliance_overview](https://codium.ai/images/pr_agent/compliance_partial.png){width=256} + +___ + +[//]: # (???+ note "The following features are available only for Qodo Merge 💎 users:") + +[//]: # ( - Custom compliance checklists and hierarchical compliance checklists) + +[//]: # ( - Ticket compliance validation with Jira/Linear integration) + +[//]: # ( - Auto-approval based on compliance status) + +[//]: # ( - Compliance labels and automated enforcement) + +## Example Usage + +### Manual Triggering + +Invoke the tool manually by commenting `/compliance` on any PR. The compliance results are presented in a comprehensive table: + +To edit [configurations](#configuration-options) related to the `compliance` tool, use the following template: + +```toml +/compliance --pr_compliance.some_config1=... --pr_compliance.some_config2=... +``` + +For example, you can enable ticket compliance labels by running: + +```toml +/compliance --pr_compliance.enable_ticket_labels=true +``` + +### Automatic Triggering + + +The tool can be triggered automatically every time a new PR is [opened](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#github-app-automatic-tools-when-a-new-pr-is-opened), or in a [push](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/?h=push#github-app-automatic-tools-for-push-actions-commits-to-an-open-pr) event to an existing PR. + +To run the `compliance` tool automatically when a PR is opened, define the following in the [configuration file](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/): + +```toml +[github_app] # for example +pr_commands = [ + "/compliance", + ... +] + +``` + +## Compliance Categories + +The compliance tool evaluates three main categories: + + +### 1. Security Compliance + +Scans for security vulnerabilities and potential exploits in the PR code changes: + +- **Verified Security Concerns** 🔴: Clear security vulnerabilities that require immediate attention +- **Possible Security Risks** ⚪: Potential security issues that need human verification +- **No Security Concerns** 🟢: No security vulnerabilities were detected + +Examples of security issues: + +- Exposure of sensitive information (API keys, passwords, secrets) +- SQL injection vulnerabilities +- Cross-site scripting (XSS) risks +- Cross-site request forgery (CSRF) vulnerabilities +- Insecure data handling patterns + + +### 2. Ticket Compliance + +???+ tip "How to set up ticket compliance" + Follow the guide on how to set up [ticket compliance](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) with Qodo Merge. + +Validates that PR changes fulfill the requirements specified in linked tickets: + +- **Fully Compliant** 🟢: All ticket requirements are satisfied +- **Partially Compliant** 🟡: Some requirements are met, others need attention +- **Not Compliant** 🔴: Clear violations of ticket requirements +- **Requires Verification** ⚪: Requirements that need human review + + +### 3. Custom Compliance + +Validates against an organization-specific compliance checklist: + +- **Fully Compliant** 🟢: All custom compliance are satisfied +- **Not Compliant** 🔴: Violations of custom compliance +- **Requires Verification** ⚪: Compliance that need human assessment + +## Custom Compliance + +### Setting Up Custom Compliance + +Each compliance is defined in a YAML file, as follows: +- `title`: Used to provide a clear name for the compliance +- `compliance_label`: Used to automatically generate labels for non-compliance issues +- `objective`, `success_criteria`, and `failure_criteria`: These fields are used to clearly define what constitutes compliance + +???+ tip "Example of a compliance checklist" + + ```yaml + # pr_compliance_checklist.yaml + pr_compliances: + - title: "Error Handling" + compliance_label: true + objective: "All external API calls must have proper error handling" + success_criteria: "Try-catch blocks around external calls with appropriate logging" + failure_criteria: "External API calls without error handling or logging" + + ... + ``` + +???+ tip "Writing effective compliance checklist" + - Avoid overly complex or subjective compliances that are hard to verify + - Keep compliances focused on security, business requirements, and critical standards + - Use clear, actionable language that developers can understand + - Focus on meaningful compliance requirements, not style preferences + + +### Global Hierarchical Compliance + +Qodo Merge supports hierarchical compliance checklists using a dedicated global configuration repository. + +#### Setting up global hierarchical compliance + +1\. Create a new repository named `pr-agent-settings` in your organization/workspace. + +2\. Build the folder hierarchy in your `pr-agent-settings` repository: + +```bash +pr-agent-settings/ +├── metadata.yaml # Maps repos/folders to compliance paths +└── compliance_standards/ # Root for all compliance definitions + ├── global/ # Global compliance, inherited widely + │ └── pr_compliance_checklist.yaml + ├── groups/ # For groups of repositories + │ ├── frontend_repos/ + │ │ └── pr_compliance_checklist.yaml + │ └── backend_repos/ + │ └── pr_compliance_checklist.yaml + ├── qodo-merge/ # For standalone repositories + │ └── pr_compliance_checklist.yaml + └── qodo-monorepo/ # For monorepo-specific compliance + ├── pr_compliance_checklist.yaml # Root level monorepo compliance + ├── qodo-github/ # Subproject compliance + │ └── pr_compliance_checklist.yaml + └── qodo-gitlab/ # Another subproject + └── pr_compliance_checklist.yaml +``` + +3\. Define the metadata file `metadata.yaml` in the `pr-agent-settings` root: + +```yaml +# Standalone repos +qodo-merge: + pr_compliance_checklist_paths: + - "qodo-merge" + +# Group-associated repos +repo_b: + pr_compliance_checklist_paths: + - "groups/backend_repos" + +# Multi-group repos +repo_c: + pr_compliance_checklist_paths: + - "groups/frontend_repos" + - "groups/backend_repos" + +# Monorepo with subprojects +qodo-monorepo: + pr_compliance_checklist_paths: + - "qodo-monorepo" + monorepo_subprojects: + frontend: + pr_compliance_checklist_paths: + - "qodo-monorepo/qodo-github" + backend: + pr_compliance_checklist_paths: + - "qodo-monorepo/qodo-gitlab" +``` + +4\. Set the following configuration: + +```toml +[pr_compliance] +enable_global_pr_compliance = true +``` + +???- info "Compliance priority and fallback behavior" + + 1\. **Primary**: Global hierarchical compliance checklists from the `pr-agent-settings` repository: + + 1.1 If the repository is mapped in `metadata.yaml`, it uses the specified paths + + 1.2 For monorepos, it automatically collects compliance checklists matching PR file paths + + 1.3 If no mapping exists, it falls back to the global compliance checklists + + 2. **Fallback**: Local repository wiki `pr_compliance_checklist.yaml` file: + + 2.1 Used when global compliance checklists are not found or configured + + 2.2 Acts as a safety net for repositories not yet configured in the global system + + 2.3 Local wiki compliance checklists are completely ignored when compliance checklists are successfully loaded + + +## Configuration Options + +???+ example "General options" + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
extra_instructionsOptional extra instructions for the tool. For example: "focus on the changes in the file X. Ignore changes in ...". Default is empty string.
persistent_commentIf set to true, the compliance comment will be persistent, meaning that every new compliance request will edit the previous one. Default is true.
enable_user_defined_compliance_labelsIf set to true, the tool will add labels for custom compliance violations. Default is true.
enable_estimate_effort_to_reviewIf set to true, the tool will estimate the effort required to review the PR (1-5 scale). Default is true.
enable_todo_scanIf set to true, the tool will scan for TODO comments in the PR code. Default is false.
enable_update_pr_compliance_checkboxIf set to true, the tool will add an update checkbox to refresh compliance status. Default is true.
enable_help_textIf set to true, the tool will display a help text in the comment. Default is false.
+ +???+ example "Security compliance options" + + + + + + + + + + +
enable_security_complianceIf set to true, the tool will check for security vulnerabilities. Default is true.
enable_compliance_labels_securityIf set to true, the tool will add security-related labels to the PR. Default is true.
+ +???+ example "Ticket compliance options" + + + + + + + + + + + + + + +
enable_ticket_labelsIf set to true, the tool will add ticket compliance labels to the PR. Default is false.
enable_no_ticket_labelsIf set to true, the tool will add a label when no ticket is found. Default is false.
check_pr_additional_contentIf set to true, the tool will check if the PR contains content not related to the ticket. Default is false.
+ + +## Usage Tips + +### Blocking PRs Based on Compliance + +!!! tip "" + You can configure CI/CD Actions to prevent merging PRs with specific compliance labels: + + - `Possible security concern` - Block PRs with potential security issues + - `Failed compliance check` - Block PRs that violate custom compliance checklists + + Implement a dedicated [GitHub Action](https://medium.com/sequra-tech/quick-tip-block-pull-request-merge-using-labels-6cc326936221) to enforce these checklists. + + +The compliance tool provides a comprehensive view of PR quality and adherence to organizational standards, helping teams maintain consistent code quality and security practices while ensuring that development work aligns with business requirements. diff --git a/docs/docs/tools/index.md b/docs/docs/tools/index.md index e50e0785..cce65834 100644 --- a/docs/docs/tools/index.md +++ b/docs/docs/tools/index.md @@ -14,6 +14,7 @@ Here is a list of Qodo Merge tools, each with a dedicated page that explains how | **💎 [Add Documentation (`/add_docs`](./documentation.md))** | Generates documentation to methods/functions/classes that changed in the PR | | **💎 [Analyze (`/analyze`](./analyze.md))** | Identify code components that changed in the PR, and enables to interactively generate tests, docs, and code suggestions for each component | | **💎 [CI Feedback (`/checks ci_job`](./ci_feedback.md))** | Automatically generates feedback and analysis for a failed CI job | +| **💎 [Compliance (`/compliance`](./compliance.md))** | Comprehensive compliance checks for security, ticket requirements, and custom organizational rules | | **💎 [Custom Prompt (`/custom_prompt`](./custom_prompt.md))** | Automatically generates custom suggestions for improving the PR code, based on specific guidelines defined by the user | | **💎 [Generate Custom Labels (`/generate_labels`](./custom_labels.md))** | Generates custom labels for the PR, based on specific guidelines defined by the user | | **💎 [Generate Tests (`/test`](./test.md))** | Automatically generates unit tests for a selected component, based on the PR code changes | diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 894de407..746341a6 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -34,6 +34,7 @@ nav: - 💎 Add Documentation: 'tools/documentation.md' - 💎 Analyze: 'tools/analyze.md' - 💎 CI Feedback: 'tools/ci_feedback.md' + - 💎 Compliance: 'tools/compliance.md' - 💎 Custom Prompt: 'tools/custom_prompt.md' - 💎 Generate Labels: 'tools/custom_labels.md' - 💎 Generate Tests: 'tools/test.md'