mirror of
https://github.com/qodo-ai/pr-agent.git
synced 2025-07-04 12:50:38 +08:00
Compare commits
149 Commits
hl/fix_azu
...
v0.30
| Author | SHA1 | Date | |
|---|---|---|---|
| eb4cdbb115 | |||
| 7f54b14b4d | |||
| 938ab9a139 | |||
| 75bde39b03 | |||
| ee36c0208c | |||
| 7c02678ba5 | |||
| 235df737d0 | |||
| 37ef4bad8f | |||
| ab7e0d9141 | |||
| 7db4d97fc2 | |||
| 3e1cf2deed | |||
| 4a00854b15 | |||
| d4a52ffc93 | |||
| c4ccfd865d | |||
| 38f10e10fa | |||
| 2cfe52e294 | |||
| 1c34450645 | |||
| 8325a8aeb1 | |||
| 081310b943 | |||
| c75fb2137b | |||
| 6de821719f | |||
| 4662f65146 | |||
| 9789e5d701 | |||
| 5b7f8a4bb6 | |||
| 608065f2ad | |||
| 7ee4c4aad1 | |||
| 1c5f203b52 | |||
| d4e8f618ce | |||
| b7b4d00f57 | |||
| 76aa66b534 | |||
| 9a2bfb4d64 | |||
| f39e0a13f4 | |||
| 5d721376fe | |||
| 1963b80b46 | |||
| 678bc9d911 | |||
| a5c61e33d3 | |||
| 99904601ce | |||
| cbea0380ec | |||
| 4e466c07f9 | |||
| 00daae1405 | |||
| 10350ae58e | |||
| cf31e7c974 | |||
| 96150495f8 | |||
| 25b07441a9 | |||
| ce062adf39 | |||
| 4bac78b1b9 | |||
| de8913e905 | |||
| 9b19935f47 | |||
| 3b42b0e152 | |||
| eff45e832c | |||
| 94253853cd | |||
| e8770beba1 | |||
| 8d2e92fe83 | |||
| 9c17a3902b | |||
| 3faab424a4 | |||
| 7a0c350760 | |||
| 3c572306cf | |||
| 3c1f47b4e9 | |||
| 39067a07ef | |||
| d812b7ed7e | |||
| 4fa65fec93 | |||
| 7f02ba5c8f | |||
| d4c2e2f483 | |||
| ca05b798ca | |||
| b0711929c3 | |||
| f09e1edb13 | |||
| 9230be86e9 | |||
| 6c05c6685e | |||
| 3904743e85 | |||
| fe9afb826d | |||
| a550c17466 | |||
| 04cb8af65d | |||
| 122248ef9c | |||
| 277c6abf0f | |||
| 829417ce6e | |||
| c635887949 | |||
| 4fdbcdf86a | |||
| 4c006eba47 | |||
| 3b84203d4c | |||
| ee4b847da0 | |||
| fa435cfc1e | |||
| dc9e9af9f8 | |||
| dc6460361b | |||
| 37781c59e7 | |||
| 865798ef3f | |||
| c190b93c4f | |||
| 1bbc673588 | |||
| ac72d6f6c2 | |||
| 4977db5578 | |||
| a2500633e2 | |||
| d8fb24c971 | |||
| 3f9cade14d | |||
| 4a7757be3f | |||
| 2c976fef72 | |||
| 2a84c841da | |||
| a9d6a615fd | |||
| e561f0358c | |||
| 9e238dd991 | |||
| 5917903a5a | |||
| 7d21c84c93 | |||
| 50e331476d | |||
| 48ab5a4018 | |||
| be20c9ab3b | |||
| ecb39856ee | |||
| b6787fa6de | |||
| c520a8658f | |||
| e2867f3a19 | |||
| 5856a9e548 | |||
| 8b563b0143 | |||
| 9b06f46563 | |||
| 75d24791a4 | |||
| 0dba39566f | |||
| d2caf0352c | |||
| 8be2e43a0f | |||
| 27479d87b7 | |||
| 76172bd3ec | |||
| 4893552f20 | |||
| 520faa7f2c | |||
| d5080a35f6 | |||
| cc06da3b7f | |||
| ac8aa9c2ef | |||
| 9c1f5ad497 | |||
| 8baf6dba93 | |||
| 403efcae22 | |||
| d857132d1d | |||
| e7f85cf858 | |||
| c57f8aff9b | |||
| cc1b1871d0 | |||
| 984d627300 | |||
| d1e8d267f6 | |||
| 32b1fb91c3 | |||
| b932b96e0c | |||
| 53e232d7f4 | |||
| 5e535a8868 | |||
| cd96f6b911 | |||
| 1955157e9a | |||
| 8c7c087931 | |||
| 53b913a4cb | |||
| c6d6e08618 | |||
| 5abc4028c9 | |||
| a540cd24e6 | |||
| 498e5ff0a7 | |||
| 7d38814cae | |||
| 08440d8ebd | |||
| bab8ee9633 | |||
| ca3df352ab | |||
| b83085ea00 | |||
| 66131854c1 | |||
| 788c0c12e6 |
172
README.md
172
README.md
@ -29,19 +29,47 @@ PR-Agent aims to help efficiently review and handle pull requests, by providing
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Getting Started](#getting-started)
|
||||
- [News and Updates](#news-and-updates)
|
||||
- [Overview](#overview)
|
||||
- [Example results](#example-results)
|
||||
- [Try it now](#try-it-now)
|
||||
- [Qodo Merge](https://qodo-merge-docs.qodo.ai/overview/pr_agent_pro/)
|
||||
- [How it works](#how-it-works)
|
||||
- [Why use PR-Agent?](#why-use-pr-agent)
|
||||
- [Data privacy](#data-privacy)
|
||||
- [See It in Action](#see-it-in-action)
|
||||
- [Try It Now](#try-it-now)
|
||||
- [Qodo Merge 💎](#qodo-merge-)
|
||||
- [How It Works](#how-it-works)
|
||||
- [Why Use PR-Agent?](#why-use-pr-agent)
|
||||
- [Data Privacy](#data-privacy)
|
||||
- [Contributing](#contributing)
|
||||
- [Links](#links)
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Try it Instantly
|
||||
Test PR-Agent on any public GitHub repository by commenting `@CodiumAI-Agent /improve`
|
||||
|
||||
### GitHub Action
|
||||
Add automated PR reviews to your repository with a simple workflow file using [GitHub Action setup guide](https://qodo-merge-docs.qodo.ai/installation/github/#run-as-a-github-action)
|
||||
|
||||
#### Other Platforms
|
||||
- [GitLab webhook setup](https://qodo-merge-docs.qodo.ai/installation/gitlab/)
|
||||
- [BitBucket app installation](https://qodo-merge-docs.qodo.ai/installation/bitbucket/)
|
||||
- [Azure DevOps setup](https://qodo-merge-docs.qodo.ai/installation/azure/)
|
||||
|
||||
### CLI Usage
|
||||
Run PR-Agent locally on your repository via command line: [Local CLI setup guide](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#local-repo-cli)
|
||||
|
||||
### Discover Qodo Merge 💎
|
||||
Zero-setup hosted solution with advanced features and priority support
|
||||
- [Intro and Installation guide](https://qodo-merge-docs.qodo.ai/installation/qodo_merge/)
|
||||
- [Plans & Pricing](https://www.qodo.ai/pricing/)
|
||||
|
||||
|
||||
## News and Updates
|
||||
|
||||
## Jun 3, 2025
|
||||
|
||||
Qodo Merge now offers a simplified free tier 💎.
|
||||
Organizations can use Qodo Merge at no cost, with a [monthly limit](https://qodo-merge-docs.qodo.ai/installation/qodo_merge/#cloud-users) of 75 PR reviews per organization.
|
||||
|
||||
## May 17, 2025
|
||||
|
||||
- v0.29 was [released](https://github.com/qodo-ai/pr-agent/releases)
|
||||
@ -70,84 +98,58 @@ Read more about it [here](https://qodo-merge-docs.qodo.ai/tools/scan_repo_discus
|
||||
|
||||
Supported commands per platform:
|
||||
|
||||
| | | GitHub | GitLab | Bitbucket | Azure DevOps | Gitea |
|
||||
| ----- |---------------------------------------------------------------------------------------------------------|:------:|:------:|:---------:|:------------:|:-----:|
|
||||
| TOOLS | [Review](https://qodo-merge-docs.qodo.ai/tools/review/) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Describe](https://qodo-merge-docs.qodo.ai/tools/describe/) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Improve](https://qodo-merge-docs.qodo.ai/tools/improve/) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Ask](https://qodo-merge-docs.qodo.ai/tools/ask/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | ⮑ [Ask on code lines](https://qodo-merge-docs.qodo.ai/tools/ask/#ask-lines) | ✅ | ✅ | | | |
|
||||
| | [Update CHANGELOG](https://qodo-merge-docs.qodo.ai/tools/update_changelog/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Help Docs](https://qodo-merge-docs.qodo.ai/tools/help_docs/?h=auto#auto-approval) | ✅ | ✅ | ✅ | | |
|
||||
| | [Ticket Context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [Utilizing Best Practices](https://qodo-merge-docs.qodo.ai/tools/improve/#best-practices) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [PR Chat](https://qodo-merge-docs.qodo.ai/chrome-extension/features/#pr-chat) 💎 | ✅ | | | | |
|
||||
| | [Suggestion Tracking](https://qodo-merge-docs.qodo.ai/tools/improve/#suggestion-tracking) 💎 | ✅ | ✅ | | | |
|
||||
| | [CI Feedback](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) 💎 | ✅ | | | | |
|
||||
| | [PR Documentation](https://qodo-merge-docs.qodo.ai/tools/documentation/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Custom Labels](https://qodo-merge-docs.qodo.ai/tools/custom_labels/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Analyze](https://qodo-merge-docs.qodo.ai/tools/analyze/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Similar Code](https://qodo-merge-docs.qodo.ai/tools/similar_code/) 💎 | ✅ | | | | |
|
||||
| | [Custom Prompt](https://qodo-merge-docs.qodo.ai/tools/custom_prompt/) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [Test](https://qodo-merge-docs.qodo.ai/tools/test/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Implement](https://qodo-merge-docs.qodo.ai/tools/implement/) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [Scan Repo Discussions](https://qodo-merge-docs.qodo.ai/tools/scan_repo_discussions/) 💎 | ✅ | | | | |
|
||||
| | [Auto-Approve](https://qodo-merge-docs.qodo.ai/tools/improve/?h=auto#auto-approval) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | | | | | | |
|
||||
| USAGE | [CLI](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#local-repo-cli) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [App / webhook](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#github-app) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Tagging bot](https://github.com/Codium-ai/pr-agent#try-it-now) | ✅ | | | | |
|
||||
| | [Actions](https://qodo-merge-docs.qodo.ai/installation/github/#run-as-a-github-action) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | | | | | | |
|
||||
| CORE | [PR compression](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | Adaptive and token-aware file patch fitting | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Multiple models support](https://qodo-merge-docs.qodo.ai/usage-guide/changing_a_model/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Local and global metadata](https://qodo-merge-docs.qodo.ai/core-abilities/metadata/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Dynamic context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic_context/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Self reflection](https://qodo-merge-docs.qodo.ai/core-abilities/self_reflection/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Static code analysis](https://qodo-merge-docs.qodo.ai/core-abilities/static_code_analysis/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Global and wiki configurations](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [PR interactive actions](https://www.qodo.ai/images/pr_agent/pr-actions.mp4) 💎 | ✅ | ✅ | | | |
|
||||
| | [Impact Evaluation](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Code Validation 💎](https://qodo-merge-docs.qodo.ai/core-abilities/code_validation/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Auto Best Practices 💎](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/) | ✅ | | | | |
|
||||
| | | GitHub | GitLab | Bitbucket | Azure DevOps | Gitea |
|
||||
|---------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------|:------:|:------:|:---------:|:------------:|:-----:|
|
||||
| [TOOLS](https://qodo-merge-docs.qodo.ai/tools/) | [Describe](https://qodo-merge-docs.qodo.ai/tools/describe/) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Review](https://qodo-merge-docs.qodo.ai/tools/review/) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Improve](https://qodo-merge-docs.qodo.ai/tools/improve/) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Ask](https://qodo-merge-docs.qodo.ai/tools/ask/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | ⮑ [Ask on code lines](https://qodo-merge-docs.qodo.ai/tools/ask/#ask-lines) | ✅ | ✅ | | | |
|
||||
| | [Help Docs](https://qodo-merge-docs.qodo.ai/tools/help_docs/?h=auto#auto-approval) | ✅ | ✅ | ✅ | | |
|
||||
| | [Update CHANGELOG](https://qodo-merge-docs.qodo.ai/tools/update_changelog/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Add Documentation](https://qodo-merge-docs.qodo.ai/tools/documentation/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Analyze](https://qodo-merge-docs.qodo.ai/tools/analyze/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Auto-Approve](https://qodo-merge-docs.qodo.ai/tools/improve/?h=auto#auto-approval) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [CI Feedback](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) 💎 | ✅ | | | | |
|
||||
| | [Custom Prompt](https://qodo-merge-docs.qodo.ai/tools/custom_prompt/) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [Generate Custom Labels](https://qodo-merge-docs.qodo.ai/tools/custom_labels/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Generate Tests](https://qodo-merge-docs.qodo.ai/tools/test/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Implement](https://qodo-merge-docs.qodo.ai/tools/implement/) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [Scan Repo Discussions](https://qodo-merge-docs.qodo.ai/tools/scan_repo_discussions/) 💎 | ✅ | | | | |
|
||||
| | [Similar Code](https://qodo-merge-docs.qodo.ai/tools/similar_code/) 💎 | ✅ | | | | |
|
||||
| | [Ticket Context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [Utilizing Best Practices](https://qodo-merge-docs.qodo.ai/tools/improve/#best-practices) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [PR Chat](https://qodo-merge-docs.qodo.ai/chrome-extension/features/#pr-chat) 💎 | ✅ | | | | |
|
||||
| | [Suggestion Tracking](https://qodo-merge-docs.qodo.ai/tools/improve/#suggestion-tracking) 💎 | ✅ | ✅ | | | |
|
||||
| | | | | | | |
|
||||
| [USAGE](https://qodo-merge-docs.qodo.ai/usage-guide/) | [CLI](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#local-repo-cli) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [App / webhook](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#github-app) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Tagging bot](https://github.com/Codium-ai/pr-agent#try-it-now) | ✅ | | | | |
|
||||
| | [Actions](https://qodo-merge-docs.qodo.ai/installation/github/#run-as-a-github-action) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | | | | | | |
|
||||
| [CORE](https://qodo-merge-docs.qodo.ai/core-abilities/) | [Adaptive and token-aware file patch fitting](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Auto Best Practices 💎](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/) | ✅ | | | | |
|
||||
| | [Chat on code suggestions](https://qodo-merge-docs.qodo.ai/core-abilities/chat_on_code_suggestions/) | ✅ | ✅ | | | |
|
||||
| | [Code Validation 💎](https://qodo-merge-docs.qodo.ai/core-abilities/code_validation/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Dynamic context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic_context/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Fetching ticket context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) | ✅ | ✅ | ✅ | | |
|
||||
| | [Global and wiki configurations](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [Impact Evaluation](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Incremental Update](https://qodo-merge-docs.qodo.ai/core-abilities/incremental_update/) | ✅ | | | | |
|
||||
| | [Interactivity](https://qodo-merge-docs.qodo.ai/core-abilities/interactivity/) | ✅ | ✅ | | | |
|
||||
| | [Local and global metadata](https://qodo-merge-docs.qodo.ai/core-abilities/metadata/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Multiple models support](https://qodo-merge-docs.qodo.ai/usage-guide/changing_a_model/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [PR compression](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [PR interactive actions](https://www.qodo.ai/images/pr_agent/pr-actions.mp4) 💎 | ✅ | ✅ | | | |
|
||||
| | [RAG context enrichment](https://qodo-merge-docs.qodo.ai/core-abilities/rag_context_enrichment/) | ✅ | | ✅ | | |
|
||||
| | [Self reflection](https://qodo-merge-docs.qodo.ai/core-abilities/self_reflection/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Static code analysis](https://qodo-merge-docs.qodo.ai/core-abilities/static_code_analysis/) 💎 | ✅ | ✅ | | | |
|
||||
- 💎 means this feature is available only in [Qodo Merge](https://www.qodo.ai/pricing/)
|
||||
|
||||
[//]: # (- Support for additional git providers is described in [here](./docs/Full_environments.md))
|
||||
___
|
||||
|
||||
‣ **Auto Description ([`/describe`](https://qodo-merge-docs.qodo.ai/tools/describe/))**: Automatically generating PR description - title, type, summary, code walkthrough and labels.
|
||||
\
|
||||
‣ **Auto Review ([`/review`](https://qodo-merge-docs.qodo.ai/tools/review/))**: Adjustable feedback about the PR, possible issues, security concerns, review effort and more.
|
||||
\
|
||||
‣ **Code Suggestions ([`/improve`](https://qodo-merge-docs.qodo.ai/tools/improve/))**: Code suggestions for improving the PR.
|
||||
\
|
||||
‣ **Question Answering ([`/ask ...`](https://qodo-merge-docs.qodo.ai/tools/ask/))**: Answering free-text questions about the PR.
|
||||
\
|
||||
‣ **Update Changelog ([`/update_changelog`](https://qodo-merge-docs.qodo.ai/tools/update_changelog/))**: Automatically updating the CHANGELOG.md file with the PR changes.
|
||||
\
|
||||
‣ **Help Docs ([`/help_docs`](https://qodo-merge-docs.qodo.ai/tools/help_docs/))**: Answers a question on any repository by utilizing given documentation.
|
||||
\
|
||||
‣ **Add Documentation 💎 ([`/add_docs`](https://qodo-merge-docs.qodo.ai/tools/documentation/))**: Generates documentation to methods/functions/classes that changed in the PR.
|
||||
\
|
||||
‣ **Generate Custom Labels 💎 ([`/generate_labels`](https://qodo-merge-docs.qodo.ai/tools/custom_labels/))**: Generates custom labels for the PR, based on specific guidelines defined by the user.
|
||||
\
|
||||
‣ **Analyze 💎 ([`/analyze`](https://qodo-merge-docs.qodo.ai/tools/analyze/))**: Identify code components that changed in the PR, and enables to interactively generate tests, docs, and code suggestions for each component.
|
||||
\
|
||||
‣ **Test 💎 ([`/test`](https://qodo-merge-docs.qodo.ai/tools/test/))**: Generate tests for a selected component, based on the PR code changes.
|
||||
\
|
||||
‣ **Custom Prompt 💎 ([`/custom_prompt`](https://qodo-merge-docs.qodo.ai/tools/custom_prompt/))**: Automatically generates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
|
||||
\
|
||||
‣ **Generate Tests 💎 ([`/test component_name`](https://qodo-merge-docs.qodo.ai/tools/test/))**: Generates unit tests for a selected component, based on the PR code changes.
|
||||
\
|
||||
‣ **CI Feedback 💎 ([`/checks ci_job`](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/))**: Automatically generates feedback and analysis for a failed CI job.
|
||||
\
|
||||
‣ **Similar Code 💎 ([`/find_similar_component`](https://qodo-merge-docs.qodo.ai/tools/similar_code/))**: Retrieves the most similar code components from inside the organization's codebase, or from open-source code.
|
||||
\
|
||||
‣ **Implement 💎 ([`/implement`](https://qodo-merge-docs.qodo.ai/tools/implement/))**: Generates implementation code from review suggestions.
|
||||
___
|
||||
|
||||
## Example results
|
||||
## See It in Action
|
||||
|
||||
</div>
|
||||
<h4><a href="https://github.com/Codium-ai/pr-agent/pull/530">/describe</a></h4>
|
||||
@ -182,7 +184,7 @@ ___
|
||||
</div>
|
||||
<hr>
|
||||
|
||||
## Try it now
|
||||
## Try It Now
|
||||
|
||||
Try the Claude Sonnet powered PR-Agent instantly on _your public GitHub repository_. Just mention `@CodiumAI-Agent` and add the desired command in any PR comment. The agent will generate a response based on your command.
|
||||
For example, add a comment to any pull request with the following text:
|
||||
@ -208,7 +210,7 @@ It does not have 'edit' access to your repo, for example, so it cannot update th
|
||||
4. **Extra features** - In addition to the benefits listed above, Qodo Merge will emphasize more customization, and the usage of static code analysis, in addition to LLM logic, to improve results.
|
||||
See [here](https://qodo-merge-docs.qodo.ai/overview/pr_agent_pro/) for a list of features available in Qodo Merge.
|
||||
|
||||
## How it works
|
||||
## How It Works
|
||||
|
||||
The following diagram illustrates PR-Agent tools and their flow:
|
||||
|
||||
@ -216,7 +218,7 @@ The following diagram illustrates PR-Agent tools and their flow:
|
||||
|
||||
Check out the [PR Compression strategy](https://qodo-merge-docs.qodo.ai/core-abilities/#pr-compression-strategy) page for more details on how we convert a code diff to a manageable LLM prompt
|
||||
|
||||
## Why use PR-Agent?
|
||||
## Why Use PR-Agent?
|
||||
|
||||
A reasonable question that can be asked is: `"Why use PR-Agent? What makes it stand out from existing tools?"`
|
||||
|
||||
@ -224,10 +226,10 @@ Here are some advantages of PR-Agent:
|
||||
|
||||
- We emphasize **real-life practical usage**. Each tool (review, improve, ask, ...) has a single LLM call, no more. We feel that this is critical for realistic team usage - obtaining an answer quickly (~30 seconds) and affordably.
|
||||
- Our [PR Compression strategy](https://qodo-merge-docs.qodo.ai/core-abilities/#pr-compression-strategy) is a core ability that enables to effectively tackle both short and long PRs.
|
||||
- Our JSON prompting strategy enables to have **modular, customizable tools**. For example, the '/review' tool categories can be controlled via the [configuration](pr_agent/settings/configuration.toml) file. Adding additional categories is easy and accessible.
|
||||
- Our JSON prompting strategy enables us to have **modular, customizable tools**. For example, the '/review' tool categories can be controlled via the [configuration](pr_agent/settings/configuration.toml) file. Adding additional categories is easy and accessible.
|
||||
- We support **multiple git providers** (GitHub, GitLab, BitBucket), **multiple ways** to use the tool (CLI, GitHub Action, GitHub App, Docker, ...), and **multiple models** (GPT, Claude, Deepseek, ...)
|
||||
|
||||
## Data privacy
|
||||
## Data Privacy
|
||||
|
||||
### Self-hosted PR-Agent
|
||||
|
||||
@ -252,7 +254,7 @@ To contribute to the project, get started by reading our [Contributing Guide](ht
|
||||
|
||||
## Links
|
||||
|
||||
- Discord community: https://discord.gg/kG35uSHDBc
|
||||
- Discord community: https://discord.com/invite/SgSxuQ65GF
|
||||
- Qodo site: https://www.qodo.ai/
|
||||
- Blog: https://www.qodo.ai/blog/
|
||||
- Troubleshooting: https://www.qodo.ai/blog/technical-faq-and-troubleshooting/
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
FROM public.ecr.aws/lambda/python:3.12
|
||||
FROM public.ecr.aws/lambda/python:3.12 AS base
|
||||
|
||||
RUN dnf update -y && \
|
||||
dnf install -y gcc python3-devel git && \
|
||||
@ -9,4 +9,10 @@ RUN pip install --no-cache-dir . && rm pyproject.toml
|
||||
RUN pip install --no-cache-dir mangum==0.17.0
|
||||
COPY pr_agent/ ${LAMBDA_TASK_ROOT}/pr_agent/
|
||||
|
||||
CMD ["pr_agent.servers.serverless.serverless"]
|
||||
FROM base AS github_lambda
|
||||
CMD ["pr_agent.servers.github_lambda_webhook.lambda_handler"]
|
||||
|
||||
FROM base AS gitlab_lambda
|
||||
CMD ["pr_agent.servers.gitlab_lambda_webhook.lambda_handler"]
|
||||
|
||||
FROM github_lambda
|
||||
|
||||
@ -4,7 +4,7 @@ With a single-click installation you will gain access to a context-aware chat on
|
||||
|
||||
The extension is powered by top code models like Claude 3.7 Sonnet and o4-mini. All the extension's features are free to use on public repositories.
|
||||
|
||||
For private repositories, you will need to install [Qodo Merge](https://github.com/apps/qodo-merge-pro){:target="_blank"} in addition to the extension (Quick GitHub app setup with a 14-day free trial. No credit card needed).
|
||||
For private repositories, you will need to install [Qodo Merge](https://github.com/apps/qodo-merge-pro){:target="_blank"} in addition to the extension.
|
||||
For a demonstration of how to install Qodo Merge and use it with the Chrome extension, please refer to the tutorial video at the provided [link](https://codium.ai/images/pr_agent/private_repos.mp4){:target="_blank"}.
|
||||
|
||||
<img src="https://codium.ai/images/pr_agent/PR-AgentChat.gif" width="768">
|
||||
|
||||
83
docs/docs/core-abilities/auto_approval.md
Normal file
83
docs/docs/core-abilities/auto_approval.md
Normal file
@ -0,0 +1,83 @@
|
||||
# Auto-approval 💎
|
||||
|
||||
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
|
||||
|
||||
Under specific conditions, Qodo Merge can auto-approve a PR when a manual comment is invoked, or when the PR meets certain criteria.
|
||||
|
||||
**To ensure safety, the auto-approval feature is disabled by default.**
|
||||
To enable auto-approval features, you need to actively set one or both of the following options in a pre-defined _configuration file_:
|
||||
|
||||
```toml
|
||||
[config]
|
||||
enable_comment_approval = true # For approval via comments
|
||||
enable_auto_approval = true # For criteria-based auto-approval
|
||||
```
|
||||
|
||||
!!! note "Notes"
|
||||
- These flags above cannot be set with a command line argument, only in the configuration file, committed to the repository.
|
||||
- Enabling auto-approval must be a deliberate decision by the repository owner.
|
||||
|
||||
## **Approval by commenting**
|
||||
|
||||
To enable approval by commenting, set in the configuration file:
|
||||
|
||||
```toml
|
||||
[config]
|
||||
enable_comment_approval = true
|
||||
```
|
||||
|
||||
After enabling, by commenting on a PR:
|
||||
|
||||
```
|
||||
/review auto_approve
|
||||
```
|
||||
|
||||
Qodo Merge will approve the PR and add a comment with the reason for the approval.
|
||||
|
||||
## **Auto-approval when the PR meets certain criteria**
|
||||
|
||||
To enable auto-approval based on specific criteria, first, you need to enable the top-level flag:
|
||||
|
||||
```toml
|
||||
[config]
|
||||
enable_auto_approval = true
|
||||
```
|
||||
|
||||
There are two possible paths leading to this auto-approval - one via the `review` tool, and one via the `improve` tool. Each tool can independently trigger auto-approval.
|
||||
|
||||
### Auto-approval via the `review` tool
|
||||
|
||||
- **Review effort score criteria**
|
||||
|
||||
```toml
|
||||
[config]
|
||||
enable_auto_approval = true
|
||||
auto_approve_for_low_review_effort = X # X is a number between 1 and 5
|
||||
```
|
||||
|
||||
When the [review effort score](https://www.qodo.ai/images/pr_agent/review3.png) is lower than or equal to X, the PR will be auto-approved (unless ticket compliance is enabled and fails, see below).
|
||||
|
||||
- **Ticket compliance criteria**
|
||||
|
||||
```toml
|
||||
[config]
|
||||
enable_auto_approval = true
|
||||
ensure_ticket_compliance = true # Default is false
|
||||
```
|
||||
|
||||
If `ensure_ticket_compliance` is set to `true`, auto-approval for the `review` toll path will be disabled if no ticket is linked to the PR, or if the PR is not fully compliant with a linked ticket. This ensures that PRs are only auto-approved if their associated tickets are properly resolved.
|
||||
|
||||
You can also prevent auto-approval if the PR exceeds the ticket's scope (see [here](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/#configuration-options)).
|
||||
|
||||
|
||||
### Auto-approval via the `improve` tool
|
||||
|
||||
PRs can be auto-approved when the `improve` tool doesn't find code suggestions.
|
||||
To enable this feature, set the following in the configuration file:
|
||||
|
||||
```toml
|
||||
[config]
|
||||
enable_auto_approval = true
|
||||
auto_approve_for_no_suggestions = true
|
||||
```
|
||||
|
||||
@ -1,3 +1,8 @@
|
||||
# Code Validation 💎
|
||||
|
||||
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
|
||||
|
||||
|
||||
## Introduction
|
||||
|
||||
The Git environment usually represents the final stage before code enters production. Hence, Detecting bugs and issues during the review process is critical.
|
||||
|
||||
@ -1,5 +1,8 @@
|
||||
|
||||
## Overview - PR Compression Strategy
|
||||
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
|
||||
|
||||
|
||||
## Overview
|
||||
|
||||
There are two scenarios:
|
||||
|
||||
|
||||
@ -1,4 +1,5 @@
|
||||
## TL;DR
|
||||
|
||||
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
|
||||
|
||||
Qodo Merge uses an **asymmetric and dynamic context strategy** to improve AI analysis of code changes in pull requests.
|
||||
It provides more context before changes than after, and dynamically adjusts the context based on code structure (e.g., enclosing functions or classes).
|
||||
|
||||
@ -39,17 +39,40 @@ By understanding the reasoning and intent behind modifications, the LLM can offe
|
||||
Similarly to the `describe` tool, the `review` tool will use the ticket content to provide additional context for the code changes.
|
||||
|
||||
In addition, this feature will evaluate how well a Pull Request (PR) adheres to its original purpose/intent as defined by the associated ticket or issue mentioned in the PR description.
|
||||
Each ticket will be assigned a label (Compliance/Alignment level), Indicates the degree to which the PR fulfills its original purpose, Options: Fully compliant, Partially compliant or Not compliant.
|
||||
Each ticket will be assigned a label (Compliance/Alignment level), Indicates the degree to which the PR fulfills its original purpose:
|
||||
|
||||
- Fully Compliant
|
||||
- Partially Compliant
|
||||
- Not Compliant
|
||||
- PR Code Verified
|
||||
|
||||
{width=768}
|
||||
|
||||
By default, the tool will automatically validate if the PR complies with the referenced ticket.
|
||||
If you want to disable this feedback, add the following line to your configuration file:
|
||||
A `PR Code Verified` label indicates the PR code meets ticket requirements, but requires additional manual testing beyond the code scope. For example - validating UI display across different environments (Mac, Windows, mobile, etc.).
|
||||
|
||||
```toml
|
||||
[pr_reviewer]
|
||||
require_ticket_analysis_review=false
|
||||
```
|
||||
|
||||
#### Configuration options
|
||||
|
||||
-
|
||||
|
||||
By default, the tool will automatically validate if the PR complies with the referenced ticket.
|
||||
If you want to disable this feedback, add the following line to your configuration file:
|
||||
|
||||
```toml
|
||||
[pr_reviewer]
|
||||
require_ticket_analysis_review=false
|
||||
```
|
||||
|
||||
-
|
||||
|
||||
If you set:
|
||||
```toml
|
||||
[pr_reviewer]
|
||||
check_pr_additional_content=true
|
||||
```
|
||||
(default: `false`)
|
||||
|
||||
the `review` tool will also validate that the PR code doesn't contain any additional content that is not related to the ticket. If it does, the PR will be labeled at best as `PR Code Verified`, and the `review` tool will provide a comment with the additional unrelated content found in the PR code.
|
||||
|
||||
## GitHub Issues Integration
|
||||
|
||||
@ -195,7 +218,7 @@ This following steps will help you check if the basic auth is working correctly,
|
||||
|
||||
2. run the following Python script (after replacing the placeholders with your actual values):
|
||||
|
||||
??? example "Script to validate basic auth"
|
||||
???- example "Script to validate basic auth"
|
||||
|
||||
```python
|
||||
from jira import JIRA
|
||||
@ -251,7 +274,7 @@ This following steps will help you check if the token is working correctly, and
|
||||
|
||||
2. run the following Python script (after replacing the placeholders with your actual values):
|
||||
|
||||
??? example "Script to validate PAT token"
|
||||
??? example- "Script to validate PAT token"
|
||||
|
||||
```python
|
||||
from jira import JIRA
|
||||
@ -286,13 +309,90 @@ This following steps will help you check if the token is working correctly, and
|
||||
print(f"Error fetching JIRA ticket details: {e}")
|
||||
```
|
||||
|
||||
|
||||
### Multi-JIRA Server Configuration 💎
|
||||
|
||||
Qodo Merge supports connecting to multiple JIRA servers using different authentication methods.
|
||||
|
||||
=== "Email/Token (Basic Auth)"
|
||||
|
||||
Configure multiple servers using Email/Token authentication:
|
||||
|
||||
- `jira_servers`: List of JIRA server URLs
|
||||
- `jira_api_token`: List of API tokens (for Cloud) or passwords (for Data Center)
|
||||
- `jira_api_email`: List of emails (for Cloud) or usernames (for Data Center)
|
||||
- `jira_base_url`: Default server for ticket IDs like `PROJ-123`, Each repository can configure (local config file) its own `jira_base_url` to choose which server to use by default.
|
||||
|
||||
**Example Configuration:**
|
||||
```toml
|
||||
[jira]
|
||||
# Server URLs
|
||||
jira_servers = ["https://company.atlassian.net", "https://datacenter.jira.com"]
|
||||
|
||||
# API tokens/passwords
|
||||
jira_api_token = ["cloud_api_token_here", "datacenter_password"]
|
||||
|
||||
# Emails/usernames (both required)
|
||||
jira_api_email = ["user@company.com", "datacenter_username"]
|
||||
|
||||
# Default server for ticket IDs
|
||||
jira_base_url = "https://company.atlassian.net"
|
||||
```
|
||||
|
||||
=== "PAT Auth"
|
||||
|
||||
Configure multiple servers using Personal Access Token authentication:
|
||||
|
||||
- `jira_servers`: List of JIRA server URLs
|
||||
- `jira_api_token`: List of PAT tokens
|
||||
- `jira_api_email`: Not needed (can be omitted or left empty)
|
||||
- `jira_base_url`: Default server for ticket IDs like `PROJ-123`, Each repository can configure (local config file) its own `jira_base_url` to choose which server to use by default.
|
||||
|
||||
**Example Configuration:**
|
||||
```toml
|
||||
[jira]
|
||||
# Server URLs
|
||||
jira_servers = ["https://server1.jira.com", "https://server2.jira.com"]
|
||||
|
||||
# PAT tokens only
|
||||
jira_api_token = ["pat_token_1", "pat_token_2"]
|
||||
|
||||
# Default server for ticket IDs
|
||||
jira_base_url = "https://server1.jira.com"
|
||||
```
|
||||
|
||||
**Mixed Authentication (Email/Token + PAT):**
|
||||
```toml
|
||||
[jira]
|
||||
jira_servers = ["https://company.atlassian.net", "https://server.jira.com"]
|
||||
jira_api_token = ["cloud_api_token", "server_pat_token"]
|
||||
jira_api_email = ["user@company.com", ""] # Empty for PAT
|
||||
```
|
||||
|
||||
=== "Jira Cloud App"
|
||||
|
||||
For Jira Cloud instances using App Authentication:
|
||||
|
||||
1. Install the Qodo Merge app on each JIRA Cloud instance you want to connect to
|
||||
2. Set the default server for ticket ID resolution:
|
||||
|
||||
```toml
|
||||
[jira]
|
||||
jira_base_url = "https://primary-team.atlassian.net"
|
||||
```
|
||||
|
||||
Full URLs (e.g., `https://other-team.atlassian.net/browse/TASK-456`) will automatically use the correct connected instance.
|
||||
|
||||
|
||||
|
||||
|
||||
### How to link a PR to a Jira ticket
|
||||
|
||||
To integrate with Jira, you can link your PR to a ticket using either of these methods:
|
||||
|
||||
**Method 1: Description Reference:**
|
||||
|
||||
Include a ticket reference in your PR description using either the complete URL format https://<JIRA_ORG>.atlassian.net/browse/ISSUE-123 or the shortened ticket ID ISSUE-123.
|
||||
Include a ticket reference in your PR description, using either the complete URL format `https://<JIRA_ORG>.atlassian.net/browse/ISSUE-123` or the shortened ticket ID `ISSUE-123` (without prefix or suffix for the shortened ID).
|
||||
|
||||
**Method 2: Branch Name Detection:**
|
||||
|
||||
@ -305,6 +405,7 @@ Name your branch with the ticket ID as a prefix (e.g., `ISSUE-123-feature-descri
|
||||
[jira]
|
||||
jira_base_url = "https://<JIRA_ORG>.atlassian.net"
|
||||
```
|
||||
Where `<JIRA_ORG>` is your Jira organization identifier (e.g., `mycompany` for `https://mycompany.atlassian.net`).
|
||||
|
||||
## Linear Integration 💎
|
||||
|
||||
@ -339,11 +440,11 @@ Include a ticket reference in your PR description using either:
|
||||
Name your branch with the ticket ID as a prefix (e.g., `ABC-123-feature-description` or `feature/ABC-123/feature-description`).
|
||||
|
||||
!!! note "Linear Base URL"
|
||||
For shortened ticket IDs or branch detection (method 2), you must configure the Linear base URL in your configuration file under the [linear] section:
|
||||
|
||||
```toml
|
||||
[linear]
|
||||
linear_base_url = "https://linear.app/[ORG_ID]"
|
||||
```
|
||||
|
||||
Replace `[ORG_ID]` with your Linear organization identifier.
|
||||
For shortened ticket IDs or branch detection (method 2), you must configure the Linear base URL in your configuration file under the [linear] section:
|
||||
|
||||
```toml
|
||||
[linear]
|
||||
linear_base_url = "https://linear.app/[ORG_ID]"
|
||||
```
|
||||
|
||||
Replace `[ORG_ID]` with your Linear organization identifier.
|
||||
|
||||
@ -1,4 +1,6 @@
|
||||
# Overview - Impact Evaluation 💎
|
||||
# Impact Evaluation 💎
|
||||
|
||||
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
|
||||
|
||||
Demonstrating the return on investment (ROI) of AI-powered initiatives is crucial for modern organizations.
|
||||
To address this need, Qodo Merge has developed an AI impact measurement tools and metrics, providing advanced analytics to help businesses quantify the tangible benefits of AI adoption in their PR review process.
|
||||
|
||||
@ -2,6 +2,7 @@
|
||||
|
||||
Qodo Merge utilizes a variety of core abilities to provide a comprehensive and efficient code review experience. These abilities include:
|
||||
|
||||
- [Auto approval](https://qodo-merge-docs.qodo.ai/core-abilities/auto_approval/)
|
||||
- [Auto best practices](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/)
|
||||
- [Chat on code suggestions](https://qodo-merge-docs.qodo.ai/core-abilities/chat_on_code_suggestions/)
|
||||
- [Code validation](https://qodo-merge-docs.qodo.ai/core-abilities/code_validation/)
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Interactivity
|
||||
# Interactivity 💎
|
||||
|
||||
`Supported Git Platforms: GitHub, GitLab`
|
||||
|
||||
|
||||
@ -1,4 +1,6 @@
|
||||
## Local and global metadata injection with multi-stage analysis
|
||||
# Local and global metadata injection with multi-stage analysis
|
||||
|
||||
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
|
||||
|
||||
1\.
|
||||
Qodo Merge initially retrieves for each PR the following data:
|
||||
|
||||
@ -27,7 +27,7 @@ In order to enable the RAG feature, add the following lines to your configuratio
|
||||
enable_rag=true
|
||||
```
|
||||
|
||||
!!! example "RAG Arguments Options"
|
||||
???+ example "RAG Arguments Options"
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
## TL;DR
|
||||
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
|
||||
|
||||
Qodo Merge implements a **self-reflection** process where the AI model reflects, scores, and re-ranks its own suggestions, eliminating irrelevant or incorrect ones.
|
||||
This approach improves the quality and relevance of suggestions, saving users time and enhancing their experience.
|
||||
|
||||
@ -1,11 +1,14 @@
|
||||
## Overview - Static Code Analysis 💎
|
||||
# Static Code Analysis 💎
|
||||
|
||||
` Supported Git Platforms: GitHub, GitLab, Bitbucket`
|
||||
|
||||
|
||||
By combining static code analysis with LLM capabilities, Qodo Merge can provide a comprehensive analysis of the PR code changes on a component level.
|
||||
|
||||
It scans the PR code changes, finds all the code components (methods, functions, classes) that changed, and enables to interactively generate tests, docs, code suggestions and similar code search for each component.
|
||||
|
||||
!!! note "Language that are currently supported:"
|
||||
Python, Java, C++, JavaScript, TypeScript, C#.
|
||||
Python, Java, C++, JavaScript, TypeScript, C#, Go.
|
||||
|
||||
## Capabilities
|
||||
|
||||
|
||||
@ -26,48 +26,52 @@ To search the documentation site using natural language:
|
||||
|
||||
PR-Agent and Qodo Merge offers extensive pull request functionalities across various git providers:
|
||||
|
||||
| | | GitHub | GitLab | Bitbucket | Azure DevOps |
|
||||
| ----- | ------------------------------------------------------------------------------------------------------- |:------:|:------:|:---------:|:------------:|
|
||||
| TOOLS | [Review](https://qodo-merge-docs.qodo.ai/tools/review/) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Describe](https://qodo-merge-docs.qodo.ai/tools/describe/) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Improve](https://qodo-merge-docs.qodo.ai/tools/improve/) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Ask](https://qodo-merge-docs.qodo.ai/tools/ask/) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | ⮑ [Ask on code lines](https://qodo-merge-docs.qodo.ai/tools/ask/#ask-lines) | ✅ | ✅ | | |
|
||||
| | [Update CHANGELOG](https://qodo-merge-docs.qodo.ai/tools/update_changelog/) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Help Docs](https://qodo-merge-docs.qodo.ai/tools/help_docs/?h=auto#auto-approval) | ✅ | ✅ | ✅ | |
|
||||
| | [Ticket Context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) 💎 | ✅ | ✅ | ✅ | |
|
||||
| | [Utilizing Best Practices](https://qodo-merge-docs.qodo.ai/tools/improve/#best-practices) 💎 | ✅ | ✅ | ✅ | |
|
||||
| | [PR Chat](https://qodo-merge-docs.qodo.ai/chrome-extension/features/#pr-chat) 💎 | ✅ | | | |
|
||||
| | [Suggestion Tracking](https://qodo-merge-docs.qodo.ai/tools/improve/#suggestion-tracking) 💎 | ✅ | ✅ | | |
|
||||
| | [CI Feedback](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) 💎 | ✅ | | | |
|
||||
| | [PR Documentation](https://qodo-merge-docs.qodo.ai/tools/documentation/) 💎 | ✅ | ✅ | | |
|
||||
| | [Custom Labels](https://qodo-merge-docs.qodo.ai/tools/custom_labels/) 💎 | ✅ | ✅ | | |
|
||||
| | [Analyze](https://qodo-merge-docs.qodo.ai/tools/analyze/) 💎 | ✅ | ✅ | | |
|
||||
| | [Similar Code](https://qodo-merge-docs.qodo.ai/tools/similar_code/) 💎 | ✅ | | | |
|
||||
| | [Custom Prompt](https://qodo-merge-docs.qodo.ai/tools/custom_prompt/) 💎 | ✅ | ✅ | ✅ | |
|
||||
| | [Test](https://qodo-merge-docs.qodo.ai/tools/test/) 💎 | ✅ | ✅ | | |
|
||||
| | [Implement](https://qodo-merge-docs.qodo.ai/tools/implement/) 💎 | ✅ | ✅ | ✅ | |
|
||||
| | [Scan Repo Discussions](https://qodo-merge-docs.qodo.ai/tools/scan_repo_discussions/) 💎 | ✅ | | | |
|
||||
| | [Auto-Approve](https://qodo-merge-docs.qodo.ai/tools/improve/?h=auto#auto-approval) 💎 | ✅ | ✅ | ✅ | |
|
||||
| | | | | | |
|
||||
| USAGE | [CLI](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#local-repo-cli) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [App / webhook](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#github-app) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Tagging bot](https://github.com/Codium-ai/pr-agent#try-it-now) | ✅ | | | |
|
||||
| | [Actions](https://qodo-merge-docs.qodo.ai/installation/github/#run-as-a-github-action) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | | | | | |
|
||||
| CORE | [PR compression](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | Adaptive and token-aware file patch fitting | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Multiple models support](https://qodo-merge-docs.qodo.ai/usage-guide/changing_a_model/) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Local and global metadata](https://qodo-merge-docs.qodo.ai/core-abilities/metadata/) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Dynamic context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic_context/) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Self reflection](https://qodo-merge-docs.qodo.ai/core-abilities/self_reflection/) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Static code analysis](https://qodo-merge-docs.qodo.ai/core-abilities/static_code_analysis/) 💎 | ✅ | ✅ | | |
|
||||
| | [Global and wiki configurations](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/) 💎 | ✅ | ✅ | ✅ | |
|
||||
| | [PR interactive actions](https://www.qodo.ai/images/pr_agent/pr-actions.mp4) 💎 | ✅ | ✅ | | |
|
||||
| | [Impact Evaluation](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) 💎 | ✅ | ✅ | | |
|
||||
| | [Code Validation 💎](https://qodo-merge-docs.qodo.ai/core-abilities/code_validation/) | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Auto Best Practices 💎](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/) | ✅ | | | |
|
||||
| | [Incremental Update 💎](https://qodo-merge-docs.qodo.ai/core-abilities/incremental_update/) | ✅ | | | |
|
||||
| | | GitHub | GitLab | Bitbucket | Azure DevOps | Gitea |
|
||||
| ----- |---------------------------------------------------------------------------------------------------------------------|:------:|:------:|:---------:|:------------:|:-----:|
|
||||
| [TOOLS](https://qodo-merge-docs.qodo.ai/tools/) | [Describe](https://qodo-merge-docs.qodo.ai/tools/describe/) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Review](https://qodo-merge-docs.qodo.ai/tools/review/) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Improve](https://qodo-merge-docs.qodo.ai/tools/improve/) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Ask](https://qodo-merge-docs.qodo.ai/tools/ask/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | ⮑ [Ask on code lines](https://qodo-merge-docs.qodo.ai/tools/ask/#ask-lines) | ✅ | ✅ | | | |
|
||||
| | [Help Docs](https://qodo-merge-docs.qodo.ai/tools/help_docs/?h=auto#auto-approval) | ✅ | ✅ | ✅ | | |
|
||||
| | [Update CHANGELOG](https://qodo-merge-docs.qodo.ai/tools/update_changelog/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Add Documentation](https://qodo-merge-docs.qodo.ai/tools/documentation/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Analyze](https://qodo-merge-docs.qodo.ai/tools/analyze/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Auto-Approve](https://qodo-merge-docs.qodo.ai/tools/improve/?h=auto#auto-approval) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [CI Feedback](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) 💎 | ✅ | | | | |
|
||||
| | [Custom Prompt](https://qodo-merge-docs.qodo.ai/tools/custom_prompt/) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [Generate Custom Labels](https://qodo-merge-docs.qodo.ai/tools/custom_labels/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Generate Tests](https://qodo-merge-docs.qodo.ai/tools/test/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Implement](https://qodo-merge-docs.qodo.ai/tools/implement/) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [Scan Repo Discussions](https://qodo-merge-docs.qodo.ai/tools/scan_repo_discussions/) 💎 | ✅ | | | | |
|
||||
| | [Similar Code](https://qodo-merge-docs.qodo.ai/tools/similar_code/) 💎 | ✅ | | | | |
|
||||
| | [Ticket Context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [Utilizing Best Practices](https://qodo-merge-docs.qodo.ai/tools/improve/#best-practices) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [PR Chat](https://qodo-merge-docs.qodo.ai/chrome-extension/features/#pr-chat) 💎 | ✅ | | | | |
|
||||
| | [Suggestion Tracking](https://qodo-merge-docs.qodo.ai/tools/improve/#suggestion-tracking) 💎 | ✅ | ✅ | | | |
|
||||
| | | | | | | |
|
||||
| [USAGE](https://qodo-merge-docs.qodo.ai/usage-guide/) | [CLI](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#local-repo-cli) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [App / webhook](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#github-app) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| | [Tagging bot](https://github.com/Codium-ai/pr-agent#try-it-now) | ✅ | | | | |
|
||||
| | [Actions](https://qodo-merge-docs.qodo.ai/installation/github/#run-as-a-github-action) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | | | | | | |
|
||||
| [CORE](https://qodo-merge-docs.qodo.ai/core-abilities/) | [Adaptive and token-aware file patch fitting](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Auto Best Practices 💎](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/) | ✅ | | | | |
|
||||
| | [Chat on code suggestions](https://qodo-merge-docs.qodo.ai/core-abilities/chat_on_code_suggestions/) | ✅ | ✅ | | | |
|
||||
| | [Code Validation 💎](https://qodo-merge-docs.qodo.ai/core-abilities/code_validation/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Dynamic context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic_context/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Fetching ticket context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) | ✅ | ✅ | ✅ | | |
|
||||
| | [Global and wiki configurations](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/) 💎 | ✅ | ✅ | ✅ | | |
|
||||
| | [Impact Evaluation](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) 💎 | ✅ | ✅ | | | |
|
||||
| | [Incremental Update 💎](https://qodo-merge-docs.qodo.ai/core-abilities/incremental_update/) | ✅ | | | | |
|
||||
| | [Interactivity](https://qodo-merge-docs.qodo.ai/core-abilities/interactivity/) | ✅ | ✅ | | | |
|
||||
| | [Local and global metadata](https://qodo-merge-docs.qodo.ai/core-abilities/metadata/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Multiple models support](https://qodo-merge-docs.qodo.ai/usage-guide/changing_a_model/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [PR compression](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [PR interactive actions](https://www.qodo.ai/images/pr_agent/pr-actions.mp4) 💎 | ✅ | ✅ | | | |
|
||||
| | [RAG context enrichment](https://qodo-merge-docs.qodo.ai/core-abilities/rag_context_enrichment/) | ✅ | | ✅ | | |
|
||||
| | [Self reflection](https://qodo-merge-docs.qodo.ai/core-abilities/self_reflection/) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| | [Static code analysis](https://qodo-merge-docs.qodo.ai/core-abilities/static_code_analysis/) 💎 | ✅ | ✅ | | | |
|
||||
!!! note "💎 means Qodo Merge only"
|
||||
All along the documentation, 💎 marks a feature available only in [Qodo Merge](https://www.codium.ai/pricing/){:target="_blank"}, and not in the open-source version.
|
||||
|
||||
|
||||
@ -39,6 +39,8 @@ GITEA__PERSONAL_ACCESS_TOKEN=<personal_access_token>
|
||||
GITEA__WEBHOOK_SECRET=<webhook_secret>
|
||||
GITEA__URL=https://gitea.com # Or self host
|
||||
OPENAI__KEY=<your_openai_api_key>
|
||||
GITEA__SKIP_SSL_VERIFICATION=false # or true
|
||||
GITEA__SSL_CA_CERT=/path/to/cacert.pem
|
||||
```
|
||||
|
||||
8. Create a webhook in your Gitea project. Set the URL to `http[s]://<PR_AGENT_HOSTNAME>/api/v1/gitea_webhooks`, the secret token to the generated secret from step 3, and enable the triggers `push`, `comments` and `merge request events`.
|
||||
|
||||
@ -187,14 +187,15 @@ For example: `GITHUB.WEBHOOK_SECRET` --> `GITHUB__WEBHOOK_SECRET`
|
||||
2. Build a docker image that can be used as a lambda function
|
||||
|
||||
```shell
|
||||
docker buildx build --platform=linux/amd64 . -t codiumai/pr-agent:serverless -f docker/Dockerfile.lambda
|
||||
# Note: --target github_lambda is optional as it's the default target
|
||||
docker buildx build --platform=linux/amd64 . -t codiumai/pr-agent:github_lambda --target github_lambda -f docker/Dockerfile.lambda
|
||||
```
|
||||
|
||||
3. Push image to ECR
|
||||
|
||||
```shell
|
||||
docker tag codiumai/pr-agent:serverless <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:serverless
|
||||
docker push <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:serverless
|
||||
docker tag codiumai/pr-agent:github_lambda <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:github_lambda
|
||||
docker push <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:github_lambda
|
||||
```
|
||||
|
||||
4. Create a lambda function that uses the uploaded image. Set the lambda timeout to be at least 3m.
|
||||
@ -203,6 +204,28 @@ For example: `GITHUB.WEBHOOK_SECRET` --> `GITHUB__WEBHOOK_SECRET`
|
||||
7. Go back to steps 8-9 of [Method 5](#run-as-a-github-app) with the function url as your Webhook URL.
|
||||
The Webhook URL would look like `https://<LAMBDA_FUNCTION_URL>/api/v1/github_webhooks`
|
||||
|
||||
### Using AWS Secrets Manager
|
||||
|
||||
For production Lambda deployments, use AWS Secrets Manager instead of environment variables:
|
||||
|
||||
1. Create a secret in AWS Secrets Manager with JSON format like this:
|
||||
|
||||
```json
|
||||
{
|
||||
"openai.key": "sk-proj-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
|
||||
"github.webhook_secret": "your-webhook-secret-from-step-2",
|
||||
"github.private_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEA...\n-----END RSA PRIVATE KEY-----"
|
||||
}
|
||||
```
|
||||
|
||||
2. Add IAM permission `secretsmanager:GetSecretValue` to your Lambda execution role
|
||||
3. Set these environment variables in your Lambda:
|
||||
|
||||
```bash
|
||||
AWS_SECRETS_MANAGER__SECRET_ARN=arn:aws:secretsmanager:us-east-1:123456789012:secret:pr-agent-secrets-AbCdEf
|
||||
CONFIG__SECRET_PROVIDER=aws_secrets_manager
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## AWS CodeCommit Setup
|
||||
|
||||
@ -61,12 +61,12 @@ git clone https://github.com/qodo-ai/pr-agent.git
|
||||
```
|
||||
|
||||
5. Prepare variables and secrets. Skip this step if you plan on setting these as environment variables when running the agent:
|
||||
1. In the configuration file/variables:
|
||||
- Set `config.git_provider` to "gitlab"
|
||||
1. In the configuration file/variables:
|
||||
- Set `config.git_provider` to "gitlab"
|
||||
|
||||
2. In the secrets file/variables:
|
||||
- Set your AI model key in the respective section
|
||||
- In the [gitlab] section, set `personal_access_token` (with token from step 2) and `shared_secret` (with secret from step 3)
|
||||
2. In the secrets file/variables:
|
||||
- Set your AI model key in the respective section
|
||||
- In the [gitlab] section, set `personal_access_token` (with token from step 2) and `shared_secret` (with secret from step 3)
|
||||
|
||||
6. Build a Docker image for the app and optionally push it to a Docker repository. We'll use Dockerhub as an example:
|
||||
|
||||
@ -88,3 +88,63 @@ OPENAI__KEY=<your_openai_api_key>
|
||||
8. Create a webhook in your GitLab project. Set the URL to `http[s]://<PR_AGENT_HOSTNAME>/webhook`, the secret token to the generated secret from step 3, and enable the triggers `push`, `comments` and `merge request events`.
|
||||
|
||||
9. Test your installation by opening a merge request or commenting on a merge request using one of PR Agent's commands.
|
||||
|
||||
## Deploy as a Lambda Function
|
||||
|
||||
Note that since AWS Lambda env vars cannot have "." in the name, you can replace each "." in an env variable with "__".<br>
|
||||
For example: `GITLAB.PERSONAL_ACCESS_TOKEN` --> `GITLAB__PERSONAL_ACCESS_TOKEN`
|
||||
|
||||
1. Follow steps 1-5 from [Run a GitLab webhook server](#run-a-gitlab-webhook-server).
|
||||
2. Build a docker image that can be used as a lambda function
|
||||
|
||||
```shell
|
||||
docker buildx build --platform=linux/amd64 . -t codiumai/pr-agent:gitlab_lambda --target gitlab_lambda -f docker/Dockerfile.lambda
|
||||
```
|
||||
|
||||
3. Push image to ECR
|
||||
|
||||
```shell
|
||||
docker tag codiumai/pr-agent:gitlab_lambda <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:gitlab_lambda
|
||||
docker push <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:gitlab_lambda
|
||||
```
|
||||
|
||||
4. Create a lambda function that uses the uploaded image. Set the lambda timeout to be at least 3m.
|
||||
5. Configure the lambda function to have a Function URL.
|
||||
6. In the environment variables of the Lambda function, specify `AZURE_DEVOPS_CACHE_DIR` to a writable location such as /tmp. (see [link](https://github.com/Codium-ai/pr-agent/pull/450#issuecomment-1840242269))
|
||||
7. Go back to steps 8-9 of [Run a GitLab webhook server](#run-a-gitlab-webhook-server) with the function url as your Webhook URL.
|
||||
The Webhook URL would look like `https://<LAMBDA_FUNCTION_URL>/webhook`
|
||||
|
||||
### Using AWS Secrets Manager
|
||||
|
||||
For production Lambda deployments, use AWS Secrets Manager instead of environment variables:
|
||||
|
||||
1. Create individual secrets for each GitLab webhook with this JSON format (e.g., secret name: `project-webhook-secret-001`)
|
||||
|
||||
```json
|
||||
{
|
||||
"gitlab_token": "glpat-xxxxxxxxxxxxxxxxxxxxxxxx",
|
||||
"token_name": "project-webhook-001"
|
||||
}
|
||||
```
|
||||
|
||||
2. Create a main configuration secret for common settings (e.g., secret name: `pr-agent-main-config`)
|
||||
|
||||
```json
|
||||
{
|
||||
"openai.key": "sk-proj-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
|
||||
}
|
||||
```
|
||||
|
||||
3. Set these environment variables in your Lambda:
|
||||
|
||||
```bash
|
||||
CONFIG__SECRET_PROVIDER=aws_secrets_manager
|
||||
AWS_SECRETS_MANAGER__SECRET_ARN=arn:aws:secretsmanager:us-east-1:123456789012:secret:pr-agent-main-config-AbCdEf
|
||||
```
|
||||
|
||||
4. In your GitLab webhook configuration, set the **Secret Token** to the **Secret name** created in step 1:
|
||||
- Example: `project-webhook-secret-001`
|
||||
|
||||
**Important**: When using Secrets Manager, GitLab's webhook secret must be the Secrets Manager secret name.
|
||||
|
||||
5. Add IAM permission `secretsmanager:GetSecretValue` to your Lambda execution role
|
||||
@ -1,20 +1,20 @@
|
||||
Qodo Merge is a versatile application compatible with GitHub, GitLab, and BitBucket, hosted by QodoAI.
|
||||
See [here](https://qodo-merge-docs.qodo.ai/overview/pr_agent_pro/) for more details about the benefits of using Qodo Merge.
|
||||
|
||||
## Trial Period and Licensing
|
||||
## Usage and Licensing
|
||||
|
||||
### Cloud Users with Teams Account
|
||||
### Cloud Users
|
||||
|
||||
A complimentary two-week trial is provided to all new users (with three additional grace usages). When the trial period ends, users will stop receiving feedback from Qodo Merge.
|
||||
Non-paying users will enjoy feedback on up to 75 PRs per git organization per month. Above this limit, PRs will not receive feedback until a new month begins.
|
||||
|
||||
Following the trial period, user licenses (seats) are required for continued access. Each user requires an individual seat license.
|
||||
For unlimited access, user licenses (seats) are required. Each user requires an individual seat license.
|
||||
After purchasing seats, the team owner can assign them to specific users through the management portal.
|
||||
|
||||
With an assigned seat, users can seamlessly deploy the application across any of their code repositories.
|
||||
With an assigned seat, users can seamlessly deploy the application across any of their code repositories in a git organization, and receive feedback on all their PRs.
|
||||
|
||||
### Enterprise Account
|
||||
|
||||
For organizations who require an Enterprise account, please [contact](https://www.qodo.ai/contact/#pricing) us to initiate a trial period, and to discuss pricing and licensing options.
|
||||
For companies who require an Enterprise account, please [contact](https://www.qodo.ai/contact/#pricing) us to initiate a trial period, and to discuss pricing and licensing options.
|
||||
|
||||
|
||||
## Install Qodo Merge for GitHub
|
||||
@ -27,7 +27,9 @@ Qodo Merge for GitHub cloud is available for installation through the [GitHub Ma
|
||||
|
||||
### GitHub Enterprise Server
|
||||
|
||||
To use Qodo Merge application on your private GitHub Enterprise Server, you will need to [contact](https://www.qodo.ai/contact/#pricing) Qodo for starting an Enterprise trial.
|
||||
To use Qodo Merge on your private GitHub Enterprise Server, you will need to [contact](https://www.qodo.ai/contact/#pricing) Qodo for starting an Enterprise trial.
|
||||
|
||||
(Note: The marketplace app is not compatible with GitHub Enterprise Server. Installation requires creating a private GitHub App instead.)
|
||||
|
||||
### GitHub Open Source Projects
|
||||
|
||||
@ -95,4 +97,4 @@ Open a new merge request or add a MR comment with one of Qodo Merge’s commands
|
||||
|
||||
### GitLab Server
|
||||
|
||||
For a trial period of two weeks on your private GitLab Server, the same [installation steps](#gitlab-cloud) as for GitLab Cloud apply. After the trial period, you will need to [contact](https://www.qodo.ai/contact/#pricing) Qodo for moving to an Enterprise account.
|
||||
For [limited free usage](https://qodo-merge-docs.qodo.ai/installation/qodo_merge/#cloud-users) on private GitLab Server, the same [installation steps](#gitlab-cloud) as for GitLab Cloud apply. For unlimited usage, you will need to [contact](https://www.qodo.ai/contact/#pricing) Qodo for moving to an Enterprise account.
|
||||
|
||||
@ -1,7 +1,11 @@
|
||||
### Overview
|
||||
|
||||
[Qodo Merge](https://www.codium.ai/pricing/){:target="_blank"} is a paid, hosted version of open-source [PR-Agent](https://github.com/Codium-ai/pr-agent){:target="_blank"}. A complimentary two-week trial is offered, followed by a monthly subscription fee.
|
||||
Qodo Merge is designed for companies and teams that require additional features and capabilities. It provides the following benefits:
|
||||
[Qodo Merge](https://www.codium.ai/pricing/){:target="_blank"} is a hosted version of the open-source [PR-Agent](https://github.com/Codium-ai/pr-agent){:target="_blank"}.
|
||||
It is designed for companies and teams that require additional features and capabilities.
|
||||
|
||||
Free users receive a monthly quota of 75 PR reviews per git organization, while unlimited usage requires a paid subscription. See [details](https://qodo-merge-docs.qodo.ai/installation/qodo_merge/#cloud-users).
|
||||
|
||||
Qodo Merge provides the following benefits:
|
||||
|
||||
1. **Fully managed** - We take care of everything for you - hosting, models, regular updates, and more. Installation is as simple as signing up and adding the Qodo Merge app to your GitHub\GitLab\BitBucket repo.
|
||||
|
||||
@ -45,7 +49,7 @@ Here are additional tools that are available only for Qodo Merge users:
|
||||
|
||||
### Supported languages
|
||||
|
||||
Qodo Merge leverages the world's leading code models, such as Claude 3.7 Sonnet and o3-mini.
|
||||
Qodo Merge leverages the world's leading code models, such as Claude 4 Sonnet, o4-mini and Gemini-2.5-Pro.
|
||||
As a result, its primary tools such as `describe`, `review`, and `improve`, as well as the PR-chat feature, support virtually all programming languages.
|
||||
|
||||
For specialized commands that require static code analysis, Qodo Merge offers support for specific languages. For more details about features that require static code analysis, please refer to the [documentation](https://qodo-merge-docs.qodo.ai/tools/analyze/#overview).
|
||||
|
||||
@ -2,200 +2,231 @@
|
||||
|
||||
## Methodology
|
||||
|
||||
Qodo Merge PR Benchmark evaluates and compares the performance of two Large Language Models (LLMs) in analyzing pull request code and providing meaningful code suggestions.
|
||||
Qodo Merge PR Benchmark evaluates and compares the performance of Large Language Models (LLMs) in analyzing pull request code and providing meaningful code suggestions.
|
||||
Our diverse dataset comprises of 400 pull requests from over 100 repositories, spanning various programming languages and frameworks to reflect real-world scenarios.
|
||||
|
||||
- For each pull request, two distinct LLMs process the same prompt using the Qodo Merge `improve` tool, each generating two sets of responses. The prompt for response generation can be found [here](https://github.com/qodo-ai/pr-agent/blob/main/pr_agent/settings/code_suggestions/pr_code_suggestions_prompts_not_decoupled.toml).
|
||||
- For each pull request, we have pre-generated suggestions from [11](https://qodo-merge-docs.qodo.ai/pr_benchmark/#models-used-for-generating-the-benchmark-baseline) different top-performing models using the Qodo Merge `improve` tool. The prompt for response generation can be found [here](https://github.com/qodo-ai/pr-agent/blob/main/pr_agent/settings/code_suggestions/pr_code_suggestions_prompts_not_decoupled.toml).
|
||||
|
||||
- Subsequently, a high-performing third model (an AI judge) evaluates the responses from the initial two models to determine the superior one. We utilize OpenAI's `o3` model as the judge, though other models have yielded consistent results. The prompt for this comparative judgment is available [here](https://github.com/Codium-ai/pr-agent-settings/tree/main/benchmark).
|
||||
- To benchmark a model, we generate its suggestions for the same pull requests and ask a high-performing judge model to **rank** the new model's output against the 11 pre-generated baseline suggestions. We utilize OpenAI's `o3` model as the judge, though other models have yielded consistent results. The prompt for this ranking judgment is available [here](https://github.com/Codium-ai/pr-agent-settings/tree/main/benchmark).
|
||||
|
||||
- We aggregate comparison outcomes across all the pull requests, calculating the win rate for each model. We also analyze the qualitative feedback (the "why" explanations from the judge) to identify each model's comparative strengths and weaknesses.
|
||||
- We aggregate ranking outcomes across all pull requests, calculating performance metrics for the evaluated model. We also analyze the qualitative feedback from the judge to identify the model's comparative strengths and weaknesses against the established baselines.
|
||||
This approach provides not just a quantitative score but also a detailed analysis of each model's strengths and weaknesses.
|
||||
|
||||
- For each model we build a "Model Card", comparing it against others. To ensure full transparency and enable community scrutiny, we also share the raw code suggestions generated by each model, and the judge's specific feedback. See example for the full output [here](https://github.com/Codium-ai/pr-agent-settings/blob/main/benchmark/sonnet_37_vs_gemini-2.5-pro-preview-05-06.md)
|
||||
|
||||
Note that this benchmark focuses on quality: the ability of an LLM to process complex pull request with multiple files and nuanced task to produce high-quality code suggestions.
|
||||
Other factors like speed, cost, and availability, while also relevant for model selection, are outside this benchmark's scope.
|
||||
[//]: # (Note that this benchmark focuses on quality: the ability of an LLM to process complex pull request with multiple files and nuanced task to produce high-quality code suggestions.)
|
||||
|
||||
## TL;DR
|
||||
[//]: # (Other factors like speed, cost, and availability, while also relevant for model selection, are outside this benchmark's scope. We do specify the thinking budget used by each model, which can be a factor in the model's performance.)
|
||||
|
||||
Here's a summary of the win rates based on the benchmark:
|
||||
[//]: # ()
|
||||
|
||||
[//]: # (| Model A | Model B | Model A Win Rate | Model B Win Rate |)
|
||||
|
||||
[//]: # (|:-------------------------------|:-------------------------------|:----------------:|:----------------:|)
|
||||
|
||||
[//]: # (| Gemini-2.5-pro-preview-05-06 | GPT-4.1 | 70.4% | 29.6% |)
|
||||
|
||||
[//]: # (| Gemini-2.5-pro-preview-05-06 | Sonnet 3.7 | 78.1% | 21.9% |)
|
||||
|
||||
[//]: # (| GPT-4.1 | Sonnet 3.7 | 61.0% | 39.0% |)
|
||||
## Results
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th style="text-align:left;">Model A</th>
|
||||
<th style="text-align:left;">Model B</th>
|
||||
<th style="text-align:center;">Model A Win Rate</th> <th style="text-align:center;">Model B Win Rate</th> </tr>
|
||||
<th style="text-align:left;">Model Name</th>
|
||||
<th style="text-align:left;">Version (Date)</th>
|
||||
<th style="text-align:left;">Thinking budget tokens</th>
|
||||
<th style="text-align:center;">Score</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td style="text-align:left;">Gemini-2.5-pro-preview-05-06</td>
|
||||
<td style="text-align:left;">GPT-4.1</td>
|
||||
<td style="text-align:center; color: #1E8449;"><b>70.4%</b></td> <td style="text-align:center; color: #D8000C;"><b>29.6%</b></td> </tr>
|
||||
<td style="text-align:left;">o3</td>
|
||||
<td style="text-align:left;">2025-04-16</td>
|
||||
<td style="text-align:left;">'medium' (<a href="https://ai.google.dev/gemini-api/docs/openai">8000</a>)</td>
|
||||
<td style="text-align:center;"><b>62.5</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">Gemini-2.5-pro-preview-05-06</td>
|
||||
<td style="text-align:left;">Sonnet 3.7</td>
|
||||
<td style="text-align:center; color: #1E8449;"><b>78.1%</b></td> <td style="text-align:center; color: #D8000C;"><b>21.9%</b></td> </tr>
|
||||
<td style="text-align:left;">o4-mini</td>
|
||||
<td style="text-align:left;">2025-04-16</td>
|
||||
<td style="text-align:left;">'medium' (<a href="https://ai.google.dev/gemini-api/docs/openai">8000</a>)</td>
|
||||
<td style="text-align:center;"><b>57.7</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">Gemini-2.5-pro-preview-05-06</td>
|
||||
<td style="text-align:left;">Gemini-2.5-flash-preview-04-17</td>
|
||||
<td style="text-align:center; color: #1E8449;"><b>73.0%</b></td> <td style="text-align:center; color: #D8000C;"><b>27.0%</b></td> </tr>
|
||||
<td style="text-align:left;">Gemini-2.5-pro</td>
|
||||
<td style="text-align:left;">2025-06-05</td>
|
||||
<td style="text-align:left;">4096</td>
|
||||
<td style="text-align:center;"><b>56.3</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">Gemini-2.5-flash-preview-04-17</td>
|
||||
<td style="text-align:left;">GPT-4.1</td>
|
||||
<td style="text-align:center; color: #1E8449;"><b>54.6%</b></td> <td style="text-align:center; color: #D8000C;"><b>45.4%</b></td> </tr>
|
||||
<td style="text-align:left;">Gemini-2.5-pro</td>
|
||||
<td style="text-align:left;">2025-06-05</td>
|
||||
<td style="text-align:left;">1024</td>
|
||||
<td style="text-align:center;"><b>44.3</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">Gemini-2.5-flash-preview-04-17</td>
|
||||
<td style="text-align:left;">Sonnet 3.7</td>
|
||||
<td style="text-align:center; color: #1E8449;"><b>60.6%</b></td> <td style="text-align:center; color: #D8000C;"><b>39.4%</b></td> </tr>
|
||||
<td style="text-align:left;">Claude-4-sonnet</td>
|
||||
<td style="text-align:left;">2025-05-14</td>
|
||||
<td style="text-align:left;">4096</td>
|
||||
<td style="text-align:center;"><b>39.7</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">Claude-4-sonnet</td>
|
||||
<td style="text-align:left;">2025-05-14</td>
|
||||
<td style="text-align:left;"></td>
|
||||
<td style="text-align:center;"><b>39.0</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">Gemini-2.5-flash</td>
|
||||
<td style="text-align:left;">2025-04-17</td>
|
||||
<td style="text-align:left;"></td>
|
||||
<td style="text-align:center;"><b>33.5</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">Claude-3.7-sonnet</td>
|
||||
<td style="text-align:left;">2025-02-19</td>
|
||||
<td style="text-align:left;"></td>
|
||||
<td style="text-align:center;"><b>32.4</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="text-align:left;">GPT-4.1</td>
|
||||
<td style="text-align:left;">Sonnet 3.7</td>
|
||||
<td style="text-align:center; color: #1E8449;"><b>61.0%</b></td> <td style="text-align:center; color: #D8000C;"><b>39.0%</b></td> </tr>
|
||||
<td style="text-align:left;">2025-04-14</td>
|
||||
<td style="text-align:left;"></td>
|
||||
<td style="text-align:center;"><b>26.5</b></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## Results Analysis
|
||||
|
||||
## Gemini-2.5-pro-preview-05-06 - Model Card
|
||||
### O3
|
||||
|
||||
### Comparison against GPT-4.1
|
||||
Final score: **62.5**
|
||||
|
||||
{width=768}
|
||||
strengths:
|
||||
|
||||
#### Analysis Summary
|
||||
- **High precision & compliance:** Generally respects task rules (limits, “added lines” scope, YAML schema) and avoids false-positive advice, often returning an empty list when appropriate.
|
||||
- **Clear, actionable output:** Suggestions are concise, well-explained and include correct before/after patches, so reviewers can apply them directly.
|
||||
- **Good critical-bug detection rate:** Frequently spots compile-breakers or obvious runtime faults (nil / NPE, overflow, race, wrong selector, etc.), putting it at least on par with many peers.
|
||||
- **Consistent formatting:** Produces syntactically valid YAML with correct labels, making automated consumption easy.
|
||||
|
||||
Model 'Gemini-2.5-pro-preview-05-06' is generally more useful thanks to wider and more accurate bug detection and concrete patches, but it sacrifices compliance discipline and sometimes oversteps the task rules. Model 'GPT-4.1' is safer and highly rule-abiding, yet often too timid—missing many genuine issues and providing limited insight. An ideal reviewer would combine 'GPT-4.1’ restraint with 'Gemini-2.5-pro-preview-05-06' thoroughness.
|
||||
weaknesses:
|
||||
|
||||
#### Detailed Analysis
|
||||
- **Narrow coverage:** Tends to stop after 1-2 issues; regularly misses additional critical defects that better answers catch, so it is seldom the top-ranked review.
|
||||
- **Occasional inaccuracies:** A few replies introduce new bugs, give partial/duplicate fixes, or (rarely) violate rules (e.g., import suggestions), hurting trust.
|
||||
- **Conservative bias:** Prefers silence over risk; while this keeps precision high, it lowers recall and overall usefulness on larger diffs.
|
||||
- **Little added insight:** Rarely offers broader context, optimisations or holistic improvements, causing it to rank only mid-tier in many comparisons.
|
||||
|
||||
Gemini-2.5-pro-preview-05-06 strengths:
|
||||
### O4 Mini ('medium' thinking tokens)
|
||||
|
||||
- better_bug_coverage: Detects and explains more critical issues, winning in ~70 % of comparisons and achieving a higher average score.
|
||||
- actionable_fixes: Supplies clear code snippets, correct language labels, and often multiple coherent suggestions per diff.
|
||||
- deeper_reasoning: Shows stronger grasp of logic, edge cases, and cross-file implications, leading to broader, high-impact reviews.
|
||||
Final score: **57.7**
|
||||
|
||||
Gemini-2.5-pro-preview-05-06 weaknesses:
|
||||
strengths:
|
||||
|
||||
- guideline_violations: More prone to over-eager advice—non-critical tweaks, touching unchanged code, suggesting new imports, or minor format errors.
|
||||
- occasional_overreach: Some fixes are speculative or risky, potentially introducing new bugs.
|
||||
- redundant_or_duplicate: At times repeats the same point or exceeds the required brevity.
|
||||
- **Good rule adherence:** Most answers respect the “new-lines only”, 3-suggestion, and YAML-schema limits, and frequently choose the safe empty list when the diff truly adds no critical bug.
|
||||
- **Clear, minimal patches:** When the model does spot a defect it usually supplies terse, valid before/after snippets and short, targeted explanations, making fixes easy to read and apply.
|
||||
- **Language & domain breadth:** Demonstrates competence across many ecosystems (C/C++, Java, TS/JS, Go, Rust, Python, Bash, Markdown, YAML, SQL, CSS, translation files, etc.) and can detect both compile-time and runtime mistakes.
|
||||
- **Often competitive:** In a sizeable minority of cases the model ties for best or near-best answer, occasionally being the only response to catch a subtle crash or build blocker.
|
||||
|
||||
weaknesses:
|
||||
|
||||
- **High miss rate:** A large share of examples show the model returning an empty list or only minor advice while other reviewers catch clear, high-impact bugs—indicative of weak defect-detection recall.
|
||||
- **False or harmful fixes:** Several answers introduce new compilation errors, propose out-of-scope changes, or violate explicit rules (e.g., adding imports, version bumps, touching untouched lines), reducing trustworthiness.
|
||||
- **Shallow coverage:** Even when it identifies one real issue it often stops there, missing additional critical problems found by stronger peers; breadth and depth are inconsistent.
|
||||
|
||||
### Gemini-2.5 Pro (4096 thinking tokens)
|
||||
|
||||
Final score: **56.3**
|
||||
|
||||
strengths:
|
||||
|
||||
- **High formatting compliance:** The model almost always produces valid YAML, respects the three-suggestion limit, and supplies clear before/after code snippets and short rationales.
|
||||
- **Good “first-bug” detection:** It frequently notices the single most obvious regression (crash, compile error, nil/NPE risk, wrong path, etc.) and gives a minimal, correct patch—often judged “on-par” with other solid answers.
|
||||
- **Clear, concise writing:** Explanations are brief yet understandable for reviewers; fixes are scoped to the changed lines and rarely include extraneous context.
|
||||
- **Low rate of harmful fixes:** Truly dangerous or build-breaking advice is rare; most mistakes are omissions rather than wrong code.
|
||||
|
||||
weaknesses:
|
||||
|
||||
- **Limited breadth of review:** The model regularly stops after the first or second issue, missing additional critical problems that stronger answers surface, so it is often out-ranked by more comprehensive peers.
|
||||
- **Occasional guideline violations:** A noticeable minority of answers touch unchanged lines, exceed the 3-item cap, suggest adding imports, or drop the required YAML wrapper, leading to automatic downgrades.
|
||||
- **False positives / speculative fixes:** In several cases it flags non-issues (style, performance, redundant code) or supplies debatable “improvements”, lowering precision and sometimes breaching the “critical bugs only” rule.
|
||||
- **Inconsistent error coverage:** For certain domains (build scripts, schema files, test code) it either returns an empty list when real regressions exist or proposes cosmetic edits, indicating gaps in specialised knowledge.
|
||||
|
||||
### Claude-4 Sonnet (4096 thinking tokens)
|
||||
|
||||
Final score: **39.7**
|
||||
|
||||
strengths:
|
||||
|
||||
- **High guideline & format compliance:** Almost always returns valid YAML, keeps ≤ 3 suggestions, avoids forbidden import/boiler-plate changes and provides clear before/after snippets.
|
||||
- **Good pinpoint accuracy on single issues:** Frequently spots at least one real critical bug and proposes a concise, technically correct fix that compiles/runs.
|
||||
- **Clarity & brevity of patches:** Explanations are short, actionable, and focused on changed lines, making the advice easy for reviewers to apply.
|
||||
|
||||
weaknesses:
|
||||
|
||||
- **Low coverage / recall:** Regularly surfaces only one minor issue (or none) while missing other, often more severe, problems caught by peer models.
|
||||
- **High “empty-list” rate:** In many diffs the model returns no suggestions even when clear critical bugs exist, offering zero reviewer value.
|
||||
- **Occasional incorrect or harmful fixes:** A non-trivial number of suggestions are speculative, contradict code intent, or would break compilation/runtime; sometimes duplicates or contradicts itself.
|
||||
- **Inconsistent severity labelling & duplication:** Repeats the same point in multiple slots, marks cosmetic edits as “critical”, or leaves `improved_code` identical to original.
|
||||
|
||||
|
||||
### Comparison against Sonnet 3.7
|
||||
### Claude-4 Sonnet
|
||||
|
||||
{width=768}
|
||||
Final score: **39.0**
|
||||
|
||||
#### Analysis Summary
|
||||
strengths:
|
||||
|
||||
Model 'Gemini-2.5-pro-preview-05-06' is the stronger reviewer—more frequently identifies genuine, high-impact bugs and provides well-formed, actionable fixes. Model 'Sonnet 3.7' is safer against false positives and tends to be concise but often misses important defects or offers low-value or incorrect suggestions.
|
||||
- **Consistently well-formatted & rule-compliant output:** Almost every answer follows the required YAML schema, keeps within the 3-suggestion limit, and returns an empty list when no issues are found, showing good instruction following.
|
||||
|
||||
See raw results [here](https://github.com/Codium-ai/pr-agent-settings/blob/main/benchmark/sonnet_37_vs_gemini-2.5-pro-preview-05-06.md)
|
||||
- **Actionable, code-level patches:** When it does spot a defect the model usually supplies clear, minimal diffs or replacement snippets that compile / run, making the fix easy to apply.
|
||||
|
||||
- **Decent hit-rate on “obvious” bugs:** The model reliably catches the most blatant syntax errors, null-checks, enum / cast problems, and other first-order issues, so it often ties or slightly beats weaker baseline replies.
|
||||
|
||||
weaknesses:
|
||||
|
||||
- **Shallow coverage:** It frequently stops after one easy bug and overlooks additional, equally-critical problems that stronger reviewers find, leaving significant risks unaddressed.
|
||||
|
||||
- **False positives & harmful fixes:** In a noticeable minority of cases it misdiagnoses code, suggests changes that break compilation or behaviour, or flags non-issues, sometimes making its output worse than doing nothing.
|
||||
|
||||
- **Drifts into non-critical or out-of-scope advice:** The model regularly proposes style tweaks, documentation edits, or changes to unchanged lines, violating the “critical new-code only” requirement.
|
||||
|
||||
|
||||
#### Detailed Analysis
|
||||
### Gemini-2.5 Flash
|
||||
|
||||
Gemini-2.5-pro-preview-05-06 strengths:
|
||||
strengths:
|
||||
|
||||
- higher_accuracy_and_coverage: finds real critical bugs and supplies actionable patches in most examples (better in 78 % of cases).
|
||||
- guideline_awareness: usually respects new-lines-only scope, ≤3 suggestions, proper YAML, and stays silent when no issues exist.
|
||||
- detailed_reasoning_and_patches: explanations tie directly to the diff and fixes are concrete, often catching multiple related defects that 'Sonnet 3.7' overlooks.
|
||||
- **High precision / low false-positive rate:** The model often stays silent or gives a single, well-justified fix, so when it does speak the suggestion is usually correct and seldom touches unchanged lines, keeping guideline compliance high.
|
||||
- **Good guideline awareness:** YAML structure is consistently valid; suggestions rarely exceed the 3-item limit and generally restrict themselves to newly-added lines.
|
||||
- **Clear, concise patches:** When a defect is found, the model produces short rationales and tidy “improved_code” blocks that reviewers can apply directly.
|
||||
- **Risk-averse behaviour pays off in “no-bug” PRs:** In examples where the diff truly contained no critical issue, the model’s empty output ranked above peers that offered speculative or stylistic advice.
|
||||
|
||||
Gemini-2.5-pro-preview-05-06 weaknesses:
|
||||
weaknesses:
|
||||
|
||||
- occasional_rule_violations: sometimes proposes new imports, package-version changes, or edits outside the added lines.
|
||||
- overzealous_suggestions: may add speculative or stylistic fixes that exceed the “critical” scope, or mis-label severity.
|
||||
- sporadic_technical_slips: a few patches contain minor coding errors, oversized snippets, or duplicate/contradicting advice.
|
||||
- **Very low recall / shallow coverage:** In a large majority of cases it gives 0-1 suggestions and misses other evident, critical bugs highlighted by peer models, leading to inferior rankings.
|
||||
- **Occasional incorrect or harmful fixes:** A noticeable subset of answers propose changes that break functionality or misunderstand the code (e.g. bad constant, wrong header logic, speculative rollbacks).
|
||||
- **Non-actionable placeholders:** Some “improved_code” sections contain comments or “…” rather than real patches, reducing practical value.
|
||||
-
|
||||
### GPT-4.1
|
||||
|
||||
## GPT-4.1 - Model Card
|
||||
Final score: **26.5**
|
||||
|
||||
### Comparison against Sonnet 3.7
|
||||
strengths:
|
||||
|
||||
{width=768}
|
||||
- **Consistent format & guideline obedience:** Output is almost always valid YAML, within the 3-suggestion limit, and rarely touches lines not prefixed with “+”.
|
||||
- **Low false-positive rate:** When no real defect exists, the model correctly returns an empty list instead of inventing speculative fixes, avoiding the “noise” many baseline answers add.
|
||||
- **Clear, concise patches when it does act:** In the minority of cases where it detects a bug (e.g., ex-13, 46, 212), the fix is usually correct, minimal, and easy to apply.
|
||||
|
||||
#### Analysis Summary
|
||||
weaknesses:
|
||||
|
||||
Model 'GPT-4.1' is safer and more compliant, preferring silence over speculation, which yields fewer rule breaches and false positives but misses some real bugs.
|
||||
Model 'Sonnet 3.7' is more adventurous and often uncovers important issues that 'GPT-4.1' ignores, yet its aggressive style leads to frequent guideline violations and a higher proportion of incorrect or non-critical advice.
|
||||
|
||||
See raw results [here](https://github.com/Codium-ai/pr-agent-settings/blob/main/benchmark/gpt-4.1_vs_sonnet_3.7_judge_o3.md)
|
||||
- **Very low recall / coverage:** In a large majority of examples it outputs an empty list or only 1 trivial suggestion while obvious critical issues remain unfixed; it systematically misses circular bugs, null-checks, schema errors, etc.
|
||||
- **Shallow analysis:** Even when it finds one problem it seldom looks deeper, so more severe or additional bugs in the same diff are left unaddressed.
|
||||
- **Occasional technical inaccuracies:** A noticeable subset of suggestions are wrong (mis-ordered assertions, harmful Bash `set` change, false dangling-reference claims) or carry metadata errors (mis-labeling files as “python”).
|
||||
- **Repetitive / derivative fixes:** Many outputs duplicate earlier simplistic ideas (e.g., single null-check) without new insight, showing limited reasoning breadth.
|
||||
|
||||
|
||||
#### Detailed Analysis
|
||||
## Appendix - models used for generating the benchmark baseline
|
||||
|
||||
GPT-4.1 strengths:
|
||||
- Strong guideline adherence: usually stays strictly on `+` lines, avoids non-critical or stylistic advice, and rarely suggests forbidden imports; often outputs an empty list when no real bug exists.
|
||||
- Lower false-positive rate: suggestions are more accurate and seldom introduce new bugs; fixes compile more reliably.
|
||||
- Good schema discipline: YAML is almost always well-formed and fields are populated correctly.
|
||||
|
||||
GPT-4.1 weaknesses:
|
||||
- Misses bugs: often returns an empty list even when a clear critical issue is present, so coverage is narrower.
|
||||
- Sparse feedback: when it does comment, it tends to give fewer suggestions and sometimes lacks depth or completeness.
|
||||
- Occasional metadata/slip-ups (wrong language tags, overly broad code spans), though less harmful than Sonnet 3.7 errors.
|
||||
|
||||
### Comparison against Gemini-2.5-pro-preview-05-06
|
||||
|
||||
{width=768}
|
||||
|
||||
#### Analysis Summary
|
||||
|
||||
Model 'Gemini-2.5-pro-preview-05-06' is generally more useful thanks to wider and more accurate bug detection and concrete patches, but it sacrifices compliance discipline and sometimes oversteps the task rules. Model 'GPT-4.1' is safer and highly rule-abiding, yet often too timid—missing many genuine issues and providing limited insight. An ideal reviewer would combine 'GPT-4.1’ restraint with 'Gemini-2.5-pro-preview-05-06' thoroughness.
|
||||
|
||||
#### Detailed Analysis
|
||||
|
||||
GPT-4.1 strengths:
|
||||
- strict_compliance: Usually sticks to the “critical bugs only / new ‘+’ lines only” rule, so outputs rarely violate task constraints.
|
||||
- low_risk: Conservative behaviour avoids harmful or speculative fixes; safer when no obvious issue exists.
|
||||
- concise_formatting: Tends to produce minimal, correctly-structured YAML without extra noise.
|
||||
|
||||
GPT-4.1 weaknesses:
|
||||
- under_detection: Frequently returns an empty list even when real bugs are present, missing ~70 % of the time.
|
||||
- shallow_analysis: When it does suggest fixes, coverage is narrow and technical depth is limited, sometimes with wrong language tags or minor format slips.
|
||||
- occasional_inaccuracy: A few suggestions are unfounded or duplicate, and rare guideline breaches (e.g., import advice) still occur.
|
||||
- anthropic_sonnet_3.7_v1:0
|
||||
- claude-4-opus-20250514
|
||||
- claude-4-sonnet-20250514
|
||||
- claude-4-sonnet-20250514_thinking_2048
|
||||
- gemini-2.5-flash-preview-04-17
|
||||
- gemini-2.5-pro-preview-05-06
|
||||
- gemini-2.5-pro-preview-06-05_1024
|
||||
- gemini-2.5-pro-preview-06-05_4096
|
||||
- gpt-4.1
|
||||
- o3
|
||||
- o4-mini_medium
|
||||
|
||||
|
||||
## Sonnet 3.7 - Model Card
|
||||
|
||||
### Comparison against GPT-4.1
|
||||
|
||||
{width=768}
|
||||
|
||||
#### Analysis Summary
|
||||
|
||||
Model 'GPT-4.1' is safer and more compliant, preferring silence over speculation, which yields fewer rule breaches and false positives but misses some real bugs.
|
||||
Model 'Sonnet 3.7' is more adventurous and often uncovers important issues that 'GPT-4.1' ignores, yet its aggressive style leads to frequent guideline violations and a higher proportion of incorrect or non-critical advice.
|
||||
|
||||
See raw results [here](https://github.com/Codium-ai/pr-agent-settings/blob/main/benchmark/gpt-4.1_vs_sonnet_3.7_judge_o3.md)
|
||||
|
||||
#### Detailed Analysis
|
||||
|
||||
'Sonnet 3.7' strengths:
|
||||
- Better bug discovery breadth: more willing to dive into logic and spot critical problems that 'GPT-4.1' overlooks; often supplies multiple, detailed fixes.
|
||||
- Richer explanations & patches: gives fuller context and, when correct, proposes more functional or user-friendly solutions.
|
||||
- Generally correct language/context tagging and targeted code snippets.
|
||||
|
||||
'Sonnet 3.7' weaknesses:
|
||||
- Guideline violations: frequently flags non-critical issues, edits untouched code, or recommends adding imports, breaching task rules.
|
||||
- Higher error rate: suggestions are more speculative and sometimes introduce new defects or duplicate work already done.
|
||||
- Occasional schema or formatting mistakes (missing list value, duplicated suggestions), reducing reliability.
|
||||
|
||||
|
||||
### Comparison against Gemini-2.5-pro-preview-05-06
|
||||
|
||||
{width=768}
|
||||
|
||||
#### Analysis Summary
|
||||
|
||||
Model 'Gemini-2.5-pro-preview-05-06' is the stronger reviewer—more frequently identifies genuine, high-impact bugs and provides well-formed, actionable fixes. Model 'Sonnet 3.7' is safer against false positives and tends to be concise but often misses important defects or offers low-value or incorrect suggestions.
|
||||
|
||||
See raw results [here](https://github.com/Codium-ai/pr-agent-settings/blob/main/benchmark/sonnet_37_vs_gemini-2.5-pro-preview-05-06.md)
|
||||
|
||||
@ -1,22 +1,23 @@
|
||||
# Recent Updates and Future Roadmap
|
||||
|
||||
`Page last updated: 2025-05-11`
|
||||
`Page last updated: 2025-06-01`
|
||||
|
||||
This page summarizes recent enhancements to Qodo Merge (last three months).
|
||||
|
||||
It also outlines our development roadmap for the upcoming three months. Please note that the roadmap is subject to change, and features may be adjusted, added, or reprioritized.
|
||||
|
||||
=== "Recent Updates"
|
||||
- **Simplified Free Tier**: Qodo Merge now offers a simplified free tier with a monthly limit of 75 PR reviews per organization, replacing the previous two-week trial. ([Learn more](https://qodo-merge-docs.qodo.ai/installation/qodo_merge/#cloud-users))
|
||||
- **CLI Endpoint**: A new Qodo Merge endpoint that accepts a lists of before/after code changes, executes Qodo Merge commands, and return the results. Currently available for enterprise customers. Contact [Qodo](https://www.qodo.ai/contact/) for more information.
|
||||
- **Linear tickets support**: Qodo Merge now supports Linear tickets. ([Learn more](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/#linear-integration))
|
||||
- **Smart Update**: Upon PR updates, Qodo Merge will offer tailored code suggestions, addressing both the entire PR and the specific incremental changes since the last feedback ([Learn more](https://qodo-merge-docs.qodo.ai/core-abilities/incremental_update//))
|
||||
- **Qodo Merge Pull Request Benchmark** - evaluating the performance of LLMs in analyzing pull request code ([Learn more](https://qodo-merge-docs.qodo.ai/pr_benchmark/))
|
||||
- **Chat on Suggestions**: Users can now chat with Qodo Merge code suggestions ([Learn more](https://qodo-merge-docs.qodo.ai/tools/improve/#chat-on-code-suggestions))
|
||||
- **Chat on Suggestions**: Users can now chat with code suggestions ([Learn more](https://qodo-merge-docs.qodo.ai/tools/improve/#chat-on-code-suggestions))
|
||||
- **Scan Repo Discussions Tool**: A new tool that analyzes past code discussions to generate a `best_practices.md` file, distilling key insights and recommendations. ([Learn more](https://qodo-merge-docs.qodo.ai/tools/scan_repo_discussions/))
|
||||
- **Enhanced Models**: Qodo Merge now defaults to a combination of top models (Claude Sonnet 3.7 and Gemini 2.5 Pro) and incorporates dedicated code validation logic for improved results. ([Details 1](https://qodo-merge-docs.qodo.ai/usage-guide/qodo_merge_models/), [Details 2](https://qodo-merge-docs.qodo.ai/core-abilities/code_validation/))
|
||||
- **Chrome Extension Update**: Qodo Merge Chrome extension now supports single-tenant users. ([Learn more](https://qodo-merge-docs.qodo.ai/chrome-extension/options/#configuration-options/))
|
||||
- **Installation Metrics**: Upon installation, Qodo Merge analyzes past PRs for key metrics (e.g., time to merge, time to first reviewer feedback), enabling pre/post-installation comparison to calculate ROI.
|
||||
|
||||
|
||||
=== "Future Roadmap"
|
||||
- **Smart Update**: Upon PR updates, Qodo Merge will offer tailored code suggestions, addressing both the entire PR and the specific incremental changes since the last feedback.
|
||||
- **CLI Endpoint**: A new Qodo Merge endpoint will accept lists of before/after code changes, execute Qodo Merge commands, and return the results.
|
||||
- **Simplified Free Tier**: We plan to transition from a two-week free trial to a free tier offering a limited number of suggestions per month per organization.
|
||||
- **Best Practices Hierarchy**: Introducing support for structured best practices, such as for folders in monorepos or a unified best practice file for a group of repositories.
|
||||
- **Enhanced `review` tool**: Enhancing the `review` tool validate compliance across multiple categories including security, tickets, and custom best practices.
|
||||
- **Smarter context retrieval**: Leverage AST and LSP analysis to gather relevant context from across the entire repository.
|
||||
- **Enhanced portal experience**: Improved user experience in the Qodo Merge portal with new options and capabilities.
|
||||
|
||||
@ -17,4 +17,4 @@ An example result:
|
||||
{width=750}
|
||||
|
||||
!!! note "Language that are currently supported:"
|
||||
Python, Java, C++, JavaScript, TypeScript, C#.
|
||||
Python, Java, C++, JavaScript, TypeScript, C#, Go.
|
||||
|
||||
@ -73,7 +73,7 @@ enable_pr_diagram = true
|
||||
|
||||
## Configuration options
|
||||
|
||||
!!! example "Possible configurations"
|
||||
???+ example "Possible configurations"
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
|
||||
@ -46,9 +46,6 @@ pr_commands = [
|
||||
The `pr_commands` list defines commands that run automatically when a PR is opened.
|
||||
Since this is under the [github_app] section, it only applies when using the Qodo Merge GitHub App in GitHub environments.
|
||||
|
||||
!!! note
|
||||
By default, /add_docs is not triggered automatically. You must explicitly include it in pr_commands to enable this behavior.
|
||||
|
||||
## Configuration options
|
||||
|
||||
- `docs_style`: The exact style of the documentation (for python docstring). you can choose between: `google`, `numpy`, `sphinx`, `restructuredtext`, `plain`. Default is `sphinx`.
|
||||
|
||||
@ -144,84 +144,216 @@ Use triple quotes to write multi-line instructions. Use bullet points or numbers
|
||||
|
||||
> `💎 feature. Platforms supported: GitHub, GitLab, Bitbucket`
|
||||
|
||||
Another option to give additional guidance to the AI model is by creating a `best_practices.md` file in your repository's root directory.
|
||||
This page can contain a list of best practices, coding standards, and guidelines that are specific to your repo/organization.
|
||||
Qodo Merge supports both simple and hierarchical best practices configurations to provide guidance to the AI model for generating relevant code suggestions.
|
||||
|
||||
The AI model will use this `best_practices.md` file as a reference, and in case the PR code violates any of the guidelines, it will create additional suggestions, with a dedicated label: `Organization
|
||||
best practice`.
|
||||
???- tip "Writing effective best practices files"
|
||||
|
||||
The following guidelines apply to all best practices files:
|
||||
|
||||
- Write clearly and concisely
|
||||
- Include brief code examples when helpful with before/after patterns
|
||||
- Focus on project-specific guidelines that will result in relevant suggestions you actually want to get
|
||||
- Keep each file relatively short, under 800 lines, since:
|
||||
- AI models may not process effectively very long documents
|
||||
- Long files tend to contain generic guidelines already known to AI
|
||||
- Use pattern-based structure rather than simple bullet points for better clarity
|
||||
|
||||
Example for a Python `best_practices.md` content:
|
||||
???- tip "Example of a best practices file"
|
||||
|
||||
Pattern 1: Add proper error handling with try-except blocks around external function calls.
|
||||
|
||||
Example code before:
|
||||
|
||||
```markdown
|
||||
## Project best practices
|
||||
- Make sure that I/O operations are encapsulated in a try-except block
|
||||
- Use the `logging` module for logging instead of `print` statements
|
||||
- Use `is` and `is not` to compare with `None`
|
||||
- Use `if __name__ == '__main__':` to run the code only when the script is executed
|
||||
- Use `with` statement to open files
|
||||
...
|
||||
```
|
||||
```python
|
||||
# Some code that might raise an exception
|
||||
return process_pr_data(data)
|
||||
```
|
||||
|
||||
Tips for writing an effective `best_practices.md` file:
|
||||
Example code after:
|
||||
|
||||
- Write clearly and concisely
|
||||
- Include brief code examples when helpful
|
||||
- Focus on project-specific guidelines, that will result in relevant suggestions you actually want to get
|
||||
- Keep the file relatively short, under 800 lines, since:
|
||||
- AI models may not process effectively very long documents
|
||||
- Long files tend to contain generic guidelines already known to AI
|
||||
```python
|
||||
try:
|
||||
# Some code that might raise an exception
|
||||
return process_pr_data(data)
|
||||
except Exception as e:
|
||||
logger.exception("Failed to process request", extra={"error": e})
|
||||
```
|
||||
|
||||
To control the number of best practices suggestions generated by the `improve` tool, give the following configuration:
|
||||
Pattern 2: Add defensive null/empty checks before accessing object properties or performing operations on potentially null variables to prevent runtime errors.
|
||||
|
||||
Example code before:
|
||||
|
||||
```toml
|
||||
[best_practices]
|
||||
num_best_practice_suggestions = 2
|
||||
```
|
||||
```python
|
||||
def get_pr_code(pr_data):
|
||||
if "changed_code" in pr_data:
|
||||
return pr_data.get("changed_code", "")
|
||||
return ""
|
||||
```
|
||||
|
||||
#### Local and global best practices
|
||||
Example code after:
|
||||
|
||||
By default, Qodo Merge will look for a local `best_practices.md` in the root of the relevant local repo.
|
||||
```python
|
||||
def get_pr_code(pr_data):
|
||||
if pr_data is None:
|
||||
return ""
|
||||
if "changed_code" in pr_data:
|
||||
return pr_data.get("changed_code", "")
|
||||
return ""
|
||||
```
|
||||
|
||||
If you want to enable also a global `best_practices.md` file, set first in the global configuration file:
|
||||
#### Local best practices
|
||||
|
||||
```toml
|
||||
[best_practices]
|
||||
enable_global_best_practices = true
|
||||
```
|
||||
For basic usage, create a `best_practices.md` file in your repository's root directory containing a list of best practices, coding standards, and guidelines specific to your repository.
|
||||
|
||||
Then, create a `best_practices.md` file in the root of [global](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/#global-configuration-file) configuration repository, `pr-agent-settings`.
|
||||
The AI model will use this `best_practices.md` file as a reference, and in case the PR code violates any of the guidelines, it will create additional suggestions, with a dedicated label: `Organization best practice`.
|
||||
|
||||
#### Best practices for multiple languages
|
||||
#### Global hierarchical best practices
|
||||
|
||||
For a git organization working with multiple programming languages, you can maintain a centralized global `best_practices.md` file containing language-specific guidelines.
|
||||
When reviewing pull requests, Qodo Merge automatically identifies the programming language and applies the relevant best practices from this file.
|
||||
|
||||
To do this, structure your `best_practices.md` file using the following format:
|
||||
For organizations managing multiple repositories with different requirements, Qodo Merge supports a hierarchical best practices system using a dedicated global configuration repository.
|
||||
|
||||
```
|
||||
# [Python]
|
||||
...
|
||||
# [Java]
|
||||
...
|
||||
# [JavaScript]
|
||||
...
|
||||
```
|
||||
**Supported scenarios:**
|
||||
|
||||
#### Dedicated label for best practices suggestions
|
||||
1. **Standalone repositories**: Individual repositories can have their own specific best practices tailored to their unique requirements
|
||||
2. **Groups of repositories**: Repositories can be mapped to shared group-level best practices for consistent standards across similar projects
|
||||
3. **Monorepos with subprojects**: Large monorepos can have both repository-level and subproject-level best practices, with automatic path-based matching
|
||||
|
||||
Best practice suggestions are labeled as `Organization best practice` by default.
|
||||
To customize this label, modify it in your configuration file:
|
||||
#### Setting up global hierarchical best practices
|
||||
|
||||
1\. Create a new repository named `pr-agent-settings` in your organization/workspace.
|
||||
|
||||
```toml
|
||||
[best_practices]
|
||||
organization_name = "..."
|
||||
```
|
||||
2\. Build the folder hierarchy in your `pr-agent-settings` repository, for example:
|
||||
|
||||
And the label will be: `{organization_name} best practice`.
|
||||
```bash
|
||||
pr-agent-settings/
|
||||
├── metadata.yaml # Maps repos/folders to best practice paths
|
||||
└── codebase_standards/ # Root for all best practice definitions
|
||||
├── global/ # Global rules, inherited widely
|
||||
│ └── best_practices.md
|
||||
├── groups/ # For groups of repositories
|
||||
│ ├── frontend_repos/
|
||||
│ │ └── best_practices.md
|
||||
│ ├── backend_repos/
|
||||
│ │ └── best_practices.md
|
||||
│ └── ...
|
||||
├── qodo-merge/ # For standalone repositories
|
||||
│ └── best_practices.md
|
||||
├── qodo-monorepo/ # For monorepo-specific rules
|
||||
│ ├── best_practices.md # Root level monorepo rules
|
||||
│ ├── qodo-github/ # Subproject best practices
|
||||
│ │ └── best_practices.md
|
||||
│ └── qodo-gitlab/ # Another subproject
|
||||
│ └── best_practices.md
|
||||
└── ... # More repositories
|
||||
```
|
||||
|
||||
#### Example results
|
||||
3\. Define the metadata file `metadata.yaml` that maps your repositories to their relevant best practices paths, for example:
|
||||
|
||||
{width=512}
|
||||
```yaml
|
||||
# Standalone repos
|
||||
qodo-merge:
|
||||
best_practices_paths:
|
||||
- "qodo-merge"
|
||||
|
||||
# Group-associated repos
|
||||
repo_b:
|
||||
best_practices_paths:
|
||||
- "groups/backend_repos"
|
||||
|
||||
# Multi-group repos
|
||||
repo_c:
|
||||
best_practices_paths:
|
||||
- "groups/frontend_repos"
|
||||
- "groups/backend_repos"
|
||||
|
||||
# Monorepo with subprojects
|
||||
qodo-monorepo:
|
||||
best_practices_paths:
|
||||
- "qodo-monorepo"
|
||||
monorepo_subprojects:
|
||||
qodo-github:
|
||||
best_practices_paths:
|
||||
- "qodo-monorepo/qodo-github"
|
||||
qodo-gitlab:
|
||||
best_practices_paths:
|
||||
- "qodo-monorepo/qodo-gitlab"
|
||||
```
|
||||
|
||||
4\. Set the following configuration in your global configuration file:
|
||||
|
||||
```toml
|
||||
[best_practices]
|
||||
enable_global_best_practices = true
|
||||
```
|
||||
|
||||
???- info "Best practices priority and fallback behavior"
|
||||
|
||||
When global best practices are enabled, Qodo Merge follows this priority order:
|
||||
|
||||
1\. **Primary**: Global hierarchical best practices from `pr-agent-settings` repository:
|
||||
|
||||
1.1 If the repository is mapped in `metadata.yaml`, it uses the specified paths
|
||||
|
||||
1.2 For monorepos, it automatically collects best practices matching PR file paths
|
||||
|
||||
1.3 If no mapping exists, it falls back to the global best practices
|
||||
|
||||
2\. **Fallback**: Local repository `best_practices.md` file:
|
||||
|
||||
2.1 Used when global best practices are not found or configured
|
||||
|
||||
2.2 Acts as a safety net for repositories not yet configured in the global system
|
||||
|
||||
2.3 Local best practices are completely ignored when global best practices are successfully loaded
|
||||
|
||||
???- info "Edge cases and behavior"
|
||||
|
||||
- **Missing paths**: If specified paths in `metadata.yaml` don't exist in the file system, those paths are skipped
|
||||
- **Monorepo subproject matching**: For monorepos, Qodo Merge automatically matches PR file paths against subproject paths to apply relevant best practices
|
||||
- **Multiple group inheritance**: Repositories can inherit from multiple groups, and all applicable best practices are combined
|
||||
|
||||
[//]: # (#### Best practices for multiple languages)
|
||||
|
||||
[//]: # ()
|
||||
[//]: # (For a git organization working with multiple programming languages, you can maintain a centralized global `best_practices.md` file containing language-specific guidelines.)
|
||||
|
||||
[//]: # (When reviewing pull requests, Qodo Merge automatically identifies the programming language and applies the relevant best practices from this file.)
|
||||
|
||||
[//]: # ()
|
||||
[//]: # (To do this, structure your `best_practices.md` file using the following format:)
|
||||
|
||||
[//]: # ()
|
||||
[//]: # (```)
|
||||
|
||||
[//]: # (# [Python])
|
||||
|
||||
[//]: # (...)
|
||||
|
||||
[//]: # (# [Java])
|
||||
|
||||
[//]: # (...)
|
||||
|
||||
[//]: # (# [JavaScript])
|
||||
|
||||
[//]: # (...)
|
||||
|
||||
[//]: # (```)
|
||||
|
||||
???- info "Dedicated label for best practices suggestions"
|
||||
|
||||
Best practice suggestions are labeled as `Organization best practice` by default.
|
||||
To customize this label, modify it in your configuration file:
|
||||
|
||||
```toml
|
||||
[best_practices]
|
||||
organization_name = "..."
|
||||
```
|
||||
|
||||
And the label will be: `{organization_name} best practice`.
|
||||
|
||||
#### Example results
|
||||
|
||||
{width=512}
|
||||
|
||||
### Auto best practices
|
||||
|
||||
@ -351,86 +483,6 @@ code_suggestions_self_review_text = "... (your text here) ..."
|
||||
|
||||
To prevent unauthorized approvals, this configuration defaults to false, and cannot be altered through online comments; enabling requires a direct update to the configuration file and a commit to the repository. This ensures that utilizing the feature demands a deliberate documented decision by the repository owner.
|
||||
|
||||
### Auto-approval
|
||||
|
||||
> `💎 feature. Platforms supported: GitHub, GitLab, Bitbucket`
|
||||
|
||||
Under specific conditions, Qodo Merge can auto-approve a PR when a specific comment is invoked, or when the PR meets certain criteria.
|
||||
|
||||
**To ensure safety, the auto-approval feature is disabled by default.**
|
||||
To enable auto-approval features, you need to actively set one or both of the following options in a pre-defined _configuration file_:
|
||||
|
||||
```toml
|
||||
[config]
|
||||
enable_comment_approval = true # For approval via comments
|
||||
enable_auto_approval = true # For criteria-based auto-approval
|
||||
```
|
||||
|
||||
!!! note "Notes"
|
||||
- Note that this specific flag cannot be set with a command line argument, only in the configuration file, committed to the repository.
|
||||
- Enabling auto-approval must be a deliberate decision by the repository owner.
|
||||
|
||||
1\. **Auto-approval by commenting**
|
||||
|
||||
To enable auto-approval by commenting, set in the configuration file:
|
||||
|
||||
```toml
|
||||
[config]
|
||||
enable_comment_approval = true
|
||||
```
|
||||
|
||||
After enabling, by commenting on a PR:
|
||||
|
||||
```
|
||||
/review auto_approve
|
||||
```
|
||||
|
||||
Qodo Merge will automatically approve the PR, and add a comment with the approval.
|
||||
|
||||
2\. **Auto-approval when the PR meets certain criteria**
|
||||
|
||||
To enable auto-approval based on specific criteria, first, you need to enable the top-level flag:
|
||||
|
||||
```toml
|
||||
[config]
|
||||
enable_auto_approval = true
|
||||
```
|
||||
|
||||
There are several criteria that can be set for auto-approval:
|
||||
|
||||
- **Review effort score**
|
||||
|
||||
```toml
|
||||
[config]
|
||||
enable_auto_approval = true
|
||||
auto_approve_for_low_review_effort = X # X is a number between 1 to 5
|
||||
```
|
||||
|
||||
When the [review effort score](https://www.qodo.ai/images/pr_agent/review3.png) is lower or equal to X, the PR will be auto-approved.
|
||||
|
||||
___
|
||||
|
||||
- **No code suggestions**
|
||||
|
||||
```toml
|
||||
[config]
|
||||
enable_auto_approval = true
|
||||
auto_approve_for_no_suggestions = true
|
||||
```
|
||||
|
||||
When no [code suggestions](https://www.qodo.ai/images/pr_agent/code_suggestions_as_comment_closed.png) were found for the PR, the PR will be auto-approved.
|
||||
|
||||
___
|
||||
|
||||
- **Ticket Compliance**
|
||||
|
||||
```toml
|
||||
[config]
|
||||
enable_auto_approval = true
|
||||
ensure_ticket_compliance = true # Default is false
|
||||
```
|
||||
|
||||
If `ensure_ticket_compliance` is set to `true`, auto-approval will be disabled if a ticket is linked to the PR and the ticket is not compliant (e.g., the `review` tool did not mark the PR as fully compliant with the ticket). This ensures that PRs are only auto-approved if their associated tickets are properly resolved.
|
||||
|
||||
### How many code suggestions are generated?
|
||||
|
||||
@ -454,7 +506,7 @@ Note: Chunking is primarily relevant for large PRs. For most PRs (up to 600 line
|
||||
|
||||
## Configuration options
|
||||
|
||||
??? example "General options"
|
||||
???+ example "General options"
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
@ -514,7 +566,7 @@ Note: Chunking is primarily relevant for large PRs. For most PRs (up to 600 line
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
??? example "Params for number of suggestions and AI calls"
|
||||
???+ example "Params for number of suggestions and AI calls"
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
|
||||
@ -3,22 +3,23 @@
|
||||
Here is a list of Qodo Merge tools, each with a dedicated page that explains how to use it:
|
||||
|
||||
| Tool | Description |
|
||||
| ---------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| **[PR Description (`/describe`](./describe.md))** | Automatically generating PR description - title, type, summary, code walkthrough and labels |
|
||||
| **[PR Review (`/review`](./review.md))** | Adjustable feedback about the PR, possible issues, security concerns, review effort and more |
|
||||
| **[Code Suggestions (`/improve`](./improve.md))** | Code suggestions for improving the PR |
|
||||
| **[Question Answering (`/ask ...`](./ask.md))** | Answering free-text questions about the PR, or on specific code lines |
|
||||
| **[Update Changelog (`/update_changelog`](./update_changelog.md))** | Automatically updating the CHANGELOG.md file with the PR changes |
|
||||
| **[Help (`/help`](./help.md))** | Provides a list of all the available tools. Also enables to trigger them interactively (💎) |
|
||||
| **[Help Docs (`/help_docs`](./help_docs.md))** | Answer a free-text question based on a git documentation folder. |
|
||||
| **[Update Changelog (`/update_changelog`](./update_changelog.md))** | Automatically updating the CHANGELOG.md file with the PR changes |
|
||||
| **💎 [Add Documentation (`/add_docs`](./documentation.md))** | Generates documentation to methods/functions/classes that changed in the PR |
|
||||
| **💎 [Generate Custom Labels (`/generate_labels`](./custom_labels.md))** | Generates custom labels for the PR, based on specific guidelines defined by the user |
|
||||
| **💎 [Analyze (`/analyze`](./analyze.md))** | Identify code components that changed in the PR, and enables to interactively generate tests, docs, and code suggestions for each component |
|
||||
| **💎 [Test (`/test`](./test.md))** | generate tests for a selected component, based on the PR code changes |
|
||||
| **💎 [Custom Prompt (`/custom_prompt`](./custom_prompt.md))** | Automatically generates custom suggestions for improving the PR code, based on specific guidelines defined by the user |
|
||||
| **💎 [Generate Tests (`/test component_name`](./test.md))** | Automatically generates unit tests for a selected component, based on the PR code changes |
|
||||
| **💎 [Improve Component (`/improve_component component_name`](./improve_component.md))** | Generates code suggestions for a specific code component that changed in the PR |
|
||||
| **💎 [CI Feedback (`/checks ci_job`](./ci_feedback.md))** | Automatically generates feedback and analysis for a failed CI job |
|
||||
| **💎 [Custom Prompt (`/custom_prompt`](./custom_prompt.md))** | Automatically generates custom suggestions for improving the PR code, based on specific guidelines defined by the user |
|
||||
| **💎 [Generate Custom Labels (`/generate_labels`](./custom_labels.md))** | Generates custom labels for the PR, based on specific guidelines defined by the user |
|
||||
| **💎 [Generate Tests (`/test`](./test.md))** | Automatically generates unit tests for a selected component, based on the PR code changes |
|
||||
| **💎 [Implement (`/implement`](./implement.md))** | Generates implementation code from review suggestions |
|
||||
| **💎 [Improve Component (`/improve_component component_name`](./improve_component.md))** | Generates code suggestions for a specific code component that changed in the PR |
|
||||
| **💎 [Scan Repo Discussions (`/scan_repo_discussions`](./scan_repo_discussions.md))** | Generates `best_practices.md` file based on previous discussions in the repository |
|
||||
| **💎 [Similar Code (`/similar_code`](./similar_code.md))** | Retrieves the most similar code components from inside the organization's codebase, or from open-source code. |
|
||||
|
||||
Note that the tools marked with 💎 are available only for Qodo Merge users.
|
||||
Note that the tools marked with 💎 are available only for Qodo Merge users.
|
||||
@ -51,7 +51,7 @@ extra_instructions = "..."
|
||||
|
||||
## Configuration options
|
||||
|
||||
!!! example "General options"
|
||||
???+ example "General options"
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
@ -76,7 +76,7 @@ extra_instructions = "..."
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
!!! example "Enable\\disable specific sub-sections"
|
||||
???+ example "Enable\\disable specific sub-sections"
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
@ -98,6 +98,11 @@ extra_instructions = "..."
|
||||
<tr>
|
||||
<td><b>require_security_review</b></td>
|
||||
<td>If set to true, the tool will add a section that checks if the PR contains a possible security or vulnerability issue. Default is true.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>require_todo_scan</b></td>
|
||||
<td>If set to true, the tool will add a section that lists TODO comments found in the PR code changes. Default is false.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>require_ticket_analysis_review</b></td>
|
||||
@ -105,7 +110,7 @@ extra_instructions = "..."
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
!!! example "Adding PR labels"
|
||||
???+ example "Adding PR labels"
|
||||
|
||||
You can enable\disable the `review` tool to add specific labels to the PR:
|
||||
|
||||
@ -155,7 +160,7 @@ extra_instructions = "..."
|
||||
- **`ticket compliance`**: Adds a label indicating code compliance level ("Fully compliant" | "PR Code Verified" | "Partially compliant" | "Not compliant") to any GitHub/Jira/Linea ticket linked in the PR. Controlled by the 'require_ticket_labels' flag (default: false). If 'require_no_ticket_labels' is also enabled, PRs without ticket links will receive a "No ticket found" label.
|
||||
|
||||
|
||||
### Blocking PRs from merging based on the generated labels
|
||||
### Auto-blocking PRs from being merged based on the generated labels
|
||||
|
||||
!!! tip ""
|
||||
|
||||
|
||||
@ -249,4 +249,4 @@ ignore_pr_authors = ["my-special-bot-user", ...]
|
||||
Where the `ignore_pr_authors` is a list of usernames that you want to ignore.
|
||||
|
||||
!!! note
|
||||
There is one specific case where bots will receive an automatic response - when they generated a PR with a _failed test_. In that case, the [`ci_feedback`](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) tool will be invoked.
|
||||
There is one specific case where bots will receive an automatic response - when they generated a PR with a _failed test_. In that case, the [`ci_feedback`](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) tool will be invoked.
|
||||
|
||||
@ -90,7 +90,7 @@ duplicate_examples=true # will duplicate the examples in the prompt, to help the
|
||||
api_base = "http://localhost:11434" # or whatever port you're running Ollama on
|
||||
```
|
||||
|
||||
By default, Ollama uses a context window size of 2048 tokens. In most cases this is not enough to cover pr-agent promt and pull-request diff. Context window size can be overridden with the `OLLAMA_CONTEXT_LENGTH` environment variable. For example, to set the default context length to 8K, use: `OLLAMA_CONTEXT_LENGTH=8192 ollama serve`. More information you can find on the [official ollama faq](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-specify-the-context-window-size).
|
||||
By default, Ollama uses a context window size of 2048 tokens. In most cases this is not enough to cover pr-agent prompt and pull-request diff. Context window size can be overridden with the `OLLAMA_CONTEXT_LENGTH` environment variable. For example, to set the default context length to 8K, use: `OLLAMA_CONTEXT_LENGTH=8192 ollama serve`. More information you can find on the [official ollama faq](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-specify-the-context-window-size).
|
||||
|
||||
Please note that the `custom_model_max_tokens` setting should be configured in accordance with the `OLLAMA_CONTEXT_LENGTH`. Failure to do so may result in unexpected model output.
|
||||
|
||||
|
||||
@ -88,7 +88,7 @@ Create a dedicated project to hold a global configuration file that affects all
|
||||
1. Create a new project with both the name and key: PR_AGENT_SETTINGS.
|
||||
2. Inside the PR_AGENT_SETTINGS project, create a repository named pr-agent-settings.
|
||||
3. In this repository, add a `.pr_agent.toml` configuration file—structured similarly to the global configuration file described above.
|
||||
4. Optionally, you can add organizational-level [global best practices file](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/#global-configuration-file).
|
||||
4. Optionally, you can add organizational-level [global best practices](https://qodo-merge-docs.qodo.ai/tools/improve/#global-hierarchical-best-practices).
|
||||
|
||||
Repositories across your entire Bitbucket organization will inherit the configuration from this file.
|
||||
|
||||
|
||||
@ -1,12 +1,12 @@
|
||||
|
||||
The default models used by Qodo Merge (April 2025) are a combination of Claude Sonnet 3.7 and Gemini 2.5 Pro.
|
||||
The default models used by Qodo Merge (June 2025) are a combination of Claude Sonnet 4 and Gemini 2.5 Pro.
|
||||
|
||||
### Selecting a Specific Model
|
||||
|
||||
Users can configure Qodo Merge to use only a specific model by editing the [configuration](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/) file.
|
||||
The models supported by Qodo Merge are:
|
||||
|
||||
- `claude-3-7-sonnet`
|
||||
- `claude-4-sonnet`
|
||||
- `o4-mini`
|
||||
- `gpt-4.1`
|
||||
- `gemini-2.5-pro`
|
||||
|
||||
@ -23,30 +23,31 @@ nav:
|
||||
- Frequently Asked Questions: 'faq/index.md'
|
||||
- 💎 Qodo Merge Models: 'usage-guide/qodo_merge_models.md'
|
||||
- Tools:
|
||||
- 'tools/index.md'
|
||||
- Describe: 'tools/describe.md'
|
||||
- Review: 'tools/review.md'
|
||||
- Improve: 'tools/improve.md'
|
||||
- Ask: 'tools/ask.md'
|
||||
- Update Changelog: 'tools/update_changelog.md'
|
||||
- Help Docs: 'tools/help_docs.md'
|
||||
- Help: 'tools/help.md'
|
||||
- 💎 Analyze: 'tools/analyze.md'
|
||||
- 💎 Test: 'tools/test.md'
|
||||
- 💎 Improve Component: 'tools/improve_component.md'
|
||||
- 💎 Documentation: 'tools/documentation.md'
|
||||
- 💎 Custom Labels: 'tools/custom_labels.md'
|
||||
- 💎 Custom Prompt: 'tools/custom_prompt.md'
|
||||
- 💎 CI Feedback: 'tools/ci_feedback.md'
|
||||
- 💎 Similar Code: 'tools/similar_code.md'
|
||||
- 💎 Implement: 'tools/implement.md'
|
||||
- 💎 Scan Repo Discussions: 'tools/scan_repo_discussions.md'
|
||||
- 'tools/index.md'
|
||||
- Describe: 'tools/describe.md'
|
||||
- Review: 'tools/review.md'
|
||||
- Improve: 'tools/improve.md'
|
||||
- Ask: 'tools/ask.md'
|
||||
- Help: 'tools/help.md'
|
||||
- Help Docs: 'tools/help_docs.md'
|
||||
- Update Changelog: 'tools/update_changelog.md'
|
||||
- 💎 Add Documentation: 'tools/documentation.md'
|
||||
- 💎 Analyze: 'tools/analyze.md'
|
||||
- 💎 CI Feedback: 'tools/ci_feedback.md'
|
||||
- 💎 Custom Prompt: 'tools/custom_prompt.md'
|
||||
- 💎 Generate Labels: 'tools/custom_labels.md'
|
||||
- 💎 Generate Tests: 'tools/test.md'
|
||||
- 💎 Implement: 'tools/implement.md'
|
||||
- 💎 Improve Components: 'tools/improve_component.md'
|
||||
- 💎 Scan Repo Discussions: 'tools/scan_repo_discussions.md'
|
||||
- 💎 Similar Code: 'tools/similar_code.md'
|
||||
- Core Abilities:
|
||||
- 'core-abilities/index.md'
|
||||
- Auto approval: 'core-abilities/auto_approval.md'
|
||||
- Auto best practices: 'core-abilities/auto_best_practices.md'
|
||||
- Chat on code suggestions: 'core-abilities/chat_on_code_suggestions.md'
|
||||
- Code validation: 'core-abilities/code_validation.md'
|
||||
- Compression strategy: 'core-abilities/compression_strategy.md'
|
||||
# - Compression strategy: 'core-abilities/compression_strategy.md'
|
||||
- Dynamic context: 'core-abilities/dynamic_context.md'
|
||||
- Fetching ticket context: 'core-abilities/fetching_ticket_context.md'
|
||||
- Impact evaluation: 'core-abilities/impact_evaluation.md'
|
||||
|
||||
@ -61,6 +61,7 @@ MAX_TOKENS = {
|
||||
'vertex_ai/gemini-1.5-pro': 1048576,
|
||||
'vertex_ai/gemini-2.5-pro-preview-03-25': 1048576,
|
||||
'vertex_ai/gemini-2.5-pro-preview-05-06': 1048576,
|
||||
'vertex_ai/gemini-2.5-pro-preview-06-05': 1048576,
|
||||
'vertex_ai/gemini-1.5-flash': 1048576,
|
||||
'vertex_ai/gemini-2.0-flash': 1048576,
|
||||
'vertex_ai/gemini-2.5-flash-preview-04-17': 1048576,
|
||||
@ -73,6 +74,7 @@ MAX_TOKENS = {
|
||||
'gemini/gemini-2.5-flash-preview-05-20': 1048576,
|
||||
'gemini/gemini-2.5-pro-preview-03-25': 1048576,
|
||||
'gemini/gemini-2.5-pro-preview-05-06': 1048576,
|
||||
'gemini/gemini-2.5-pro-preview-06-05': 1048576,
|
||||
'codechat-bison': 6144,
|
||||
'codechat-bison-32k': 32000,
|
||||
'anthropic.claude-instant-v1': 100000,
|
||||
@ -101,6 +103,9 @@ MAX_TOKENS = {
|
||||
"bedrock/us.anthropic.claude-3-5-sonnet-20241022-v2:0": 100000,
|
||||
"bedrock/us.anthropic.claude-3-7-sonnet-20250219-v1:0": 200000,
|
||||
"bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0": 200000,
|
||||
"bedrock/apac.anthropic.claude-3-5-sonnet-20241022-v2:0": 100000,
|
||||
"bedrock/apac.anthropic.claude-3-7-sonnet-20250219-v1:0": 200000,
|
||||
"bedrock/apac.anthropic.claude-sonnet-4-20250514-v1:0": 200000,
|
||||
'claude-3-5-sonnet': 100000,
|
||||
'groq/meta-llama/llama-4-scout-17b-16e-instruct': 131072,
|
||||
'groq/meta-llama/llama-4-maverick-17b-128e-instruct': 131072,
|
||||
|
||||
@ -131,7 +131,7 @@ class LiteLLMAIHandler(BaseAiHandler):
|
||||
self.api_base = openrouter_api_base
|
||||
litellm.api_base = openrouter_api_base
|
||||
|
||||
# Models that only use user meessage
|
||||
# Models that only use user message
|
||||
self.user_message_only_models = USER_MESSAGE_ONLY_MODELS
|
||||
|
||||
# Model that doesn't support temperature argument
|
||||
@ -212,7 +212,7 @@ class LiteLLMAIHandler(BaseAiHandler):
|
||||
|
||||
return kwargs
|
||||
|
||||
def add_litellm_callbacks(selfs, kwargs) -> dict:
|
||||
def add_litellm_callbacks(self, kwargs) -> dict:
|
||||
captured_extra = []
|
||||
|
||||
def capture_logs(message):
|
||||
|
||||
@ -1,5 +1,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import copy
|
||||
import difflib
|
||||
import hashlib
|
||||
@ -14,7 +15,7 @@ import traceback
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
from importlib.metadata import PackageNotFoundError, version
|
||||
from typing import Any, List, Tuple
|
||||
from typing import Any, List, Tuple, TypedDict
|
||||
|
||||
import html2text
|
||||
import requests
|
||||
@ -37,21 +38,31 @@ def get_model(model_type: str = "model_weak") -> str:
|
||||
return get_settings().config.model_reasoning
|
||||
return get_settings().config.model
|
||||
|
||||
|
||||
class Range(BaseModel):
|
||||
line_start: int # should be 0-indexed
|
||||
line_end: int
|
||||
column_start: int = -1
|
||||
column_end: int = -1
|
||||
|
||||
|
||||
class ModelType(str, Enum):
|
||||
REGULAR = "regular"
|
||||
WEAK = "weak"
|
||||
REASONING = "reasoning"
|
||||
|
||||
|
||||
class TodoItem(TypedDict):
|
||||
relevant_file: str
|
||||
line_range: Tuple[int, int]
|
||||
content: str
|
||||
|
||||
|
||||
class PRReviewHeader(str, Enum):
|
||||
REGULAR = "## PR Reviewer Guide"
|
||||
INCREMENTAL = "## Incremental PR Reviewer Guide"
|
||||
|
||||
|
||||
class ReasoningEffort(str, Enum):
|
||||
HIGH = "high"
|
||||
MEDIUM = "medium"
|
||||
@ -109,6 +120,7 @@ def unique_strings(input_list: List[str]) -> List[str]:
|
||||
seen.add(item)
|
||||
return unique_list
|
||||
|
||||
|
||||
def convert_to_markdown_v2(output_data: dict,
|
||||
gfm_supported: bool = True,
|
||||
incremental_review=None,
|
||||
@ -131,6 +143,7 @@ def convert_to_markdown_v2(output_data: dict,
|
||||
"Focused PR": "✨",
|
||||
"Relevant ticket": "🎫",
|
||||
"Security concerns": "🔒",
|
||||
"Todo sections": "📝",
|
||||
"Insights from user's answers": "📝",
|
||||
"Code feedback": "🤖",
|
||||
"Estimated effort to review [1-5]": "⏱️",
|
||||
@ -151,6 +164,7 @@ def convert_to_markdown_v2(output_data: dict,
|
||||
if gfm_supported:
|
||||
markdown_text += "<table>\n"
|
||||
|
||||
todo_summary = output_data['review'].pop('todo_summary', '')
|
||||
for key, value in output_data['review'].items():
|
||||
if value is None or value == '' or value == {} or value == []:
|
||||
if key.lower() not in ['can_be_split', 'key_issues_to_review']:
|
||||
@ -209,6 +223,23 @@ def convert_to_markdown_v2(output_data: dict,
|
||||
markdown_text += f"### {emoji} Security concerns\n\n"
|
||||
value = emphasize_header(value.strip(), only_markdown=True)
|
||||
markdown_text += f"{value}\n\n"
|
||||
elif 'todo sections' in key_nice.lower():
|
||||
if gfm_supported:
|
||||
markdown_text += "<tr><td>"
|
||||
if is_value_no(value):
|
||||
markdown_text += f"✅ <strong>No TODO sections</strong>"
|
||||
else:
|
||||
markdown_todo_items = format_todo_items(value, git_provider, gfm_supported)
|
||||
markdown_text += f"{emoji} <strong>TODO sections</strong>\n<br><br>\n"
|
||||
markdown_text += markdown_todo_items
|
||||
markdown_text += "</td></tr>\n"
|
||||
else:
|
||||
if is_value_no(value):
|
||||
markdown_text += f"### ✅ No TODO sections\n\n"
|
||||
else:
|
||||
markdown_todo_items = format_todo_items(value, git_provider, gfm_supported)
|
||||
markdown_text += f"### {emoji} TODO sections\n\n"
|
||||
markdown_text += markdown_todo_items
|
||||
elif 'can be split' in key_nice.lower():
|
||||
if gfm_supported:
|
||||
markdown_text += f"<tr><td>"
|
||||
@ -1289,7 +1320,7 @@ def process_description(description_full: str) -> Tuple[str, List]:
|
||||
pattern_back = r'<details>\s*<summary><strong>(.*?)</strong><dd><code>(.*?)</code>.*?</summary>\s*<hr>\s*(.*?)\n\n\s*(.*?)</details>'
|
||||
res = re.search(pattern_back, file_data, re.DOTALL)
|
||||
if not res or res.lastindex != 4:
|
||||
pattern_back = r'<details>\s*<summary><strong>(.*?)</strong>\s*<dd><code>(.*?)</code>.*?</summary>\s*<hr>\s*(.*?)\s*-\s*(.*?)\s*</details>' # looking for hypen ('- ')
|
||||
pattern_back = r'<details>\s*<summary><strong>(.*?)</strong>\s*<dd><code>(.*?)</code>.*?</summary>\s*<hr>\s*(.*?)\s*-\s*(.*?)\s*</details>' # looking for hyphen ('- ')
|
||||
res = re.search(pattern_back, file_data, re.DOTALL)
|
||||
if res and res.lastindex == 4:
|
||||
short_filename = res.group(1).strip()
|
||||
@ -1367,3 +1398,47 @@ def set_file_languages(diff_files) -> List[FilePatchInfo]:
|
||||
get_logger().exception(f"Failed to set file languages: {e}")
|
||||
|
||||
return diff_files
|
||||
|
||||
def format_todo_item(todo_item: TodoItem, git_provider, gfm_supported) -> str:
|
||||
relevant_file = todo_item.get('relevant_file', '').strip()
|
||||
line_number = todo_item.get('line_number', '')
|
||||
content = todo_item.get('content', '')
|
||||
reference_link = git_provider.get_line_link(relevant_file, line_number, line_number)
|
||||
file_ref = f"{relevant_file} [{line_number}]"
|
||||
if reference_link:
|
||||
if gfm_supported:
|
||||
file_ref = f"<a href='{reference_link}'>{file_ref}</a>"
|
||||
else:
|
||||
file_ref = f"[{file_ref}]({reference_link})"
|
||||
|
||||
if content:
|
||||
return f"{file_ref}: {content.strip()}"
|
||||
else:
|
||||
# if content is empty, return only the file reference
|
||||
return file_ref
|
||||
|
||||
|
||||
def format_todo_items(value: list[TodoItem] | TodoItem, git_provider, gfm_supported) -> str:
|
||||
markdown_text = ""
|
||||
MAX_ITEMS = 5 # limit the number of items to display
|
||||
if gfm_supported:
|
||||
if isinstance(value, list):
|
||||
markdown_text += "<ul>\n"
|
||||
if len(value) > MAX_ITEMS:
|
||||
get_logger().debug(f"Truncating todo items to {MAX_ITEMS} items")
|
||||
value = value[:MAX_ITEMS]
|
||||
for todo_item in value:
|
||||
markdown_text += f"<li>{format_todo_item(todo_item, git_provider, gfm_supported)}</li>\n"
|
||||
markdown_text += "</ul>\n"
|
||||
else:
|
||||
markdown_text += f"<p>{format_todo_item(value, git_provider, gfm_supported)}</p>\n"
|
||||
else:
|
||||
if isinstance(value, list):
|
||||
if len(value) > MAX_ITEMS:
|
||||
get_logger().debug(f"Truncating todo items to {MAX_ITEMS} items")
|
||||
value = value[:MAX_ITEMS]
|
||||
for todo_item in value:
|
||||
markdown_text += f"- {format_todo_item(todo_item, git_provider, gfm_supported)}\n"
|
||||
else:
|
||||
markdown_text += f"- {format_todo_item(value, git_provider, gfm_supported)}\n"
|
||||
return markdown_text
|
||||
@ -81,3 +81,62 @@ def _find_pyproject() -> Optional[Path]:
|
||||
pyproject_path = _find_pyproject()
|
||||
if pyproject_path is not None:
|
||||
get_settings().load_file(pyproject_path, env=f'tool.{PR_AGENT_TOML_KEY}')
|
||||
|
||||
|
||||
def apply_secrets_manager_config():
|
||||
"""
|
||||
Retrieve configuration from AWS Secrets Manager and override existing settings
|
||||
"""
|
||||
try:
|
||||
# Dynamic imports to avoid circular dependency (secret_providers imports config_loader)
|
||||
from pr_agent.secret_providers import get_secret_provider
|
||||
from pr_agent.log import get_logger
|
||||
|
||||
secret_provider = get_secret_provider()
|
||||
if not secret_provider:
|
||||
return
|
||||
|
||||
if (hasattr(secret_provider, 'get_all_secrets') and
|
||||
get_settings().get("CONFIG.SECRET_PROVIDER") == 'aws_secrets_manager'):
|
||||
try:
|
||||
secrets = secret_provider.get_all_secrets()
|
||||
if secrets:
|
||||
apply_secrets_to_config(secrets)
|
||||
get_logger().info("Applied AWS Secrets Manager configuration")
|
||||
except Exception as e:
|
||||
get_logger().error(f"Failed to apply AWS Secrets Manager config: {e}")
|
||||
except Exception as e:
|
||||
try:
|
||||
from pr_agent.log import get_logger
|
||||
get_logger().debug(f"Secret provider not configured: {e}")
|
||||
except:
|
||||
# Fail completely silently if log module is not available
|
||||
pass
|
||||
|
||||
|
||||
def apply_secrets_to_config(secrets: dict):
|
||||
"""
|
||||
Apply secret dictionary to configuration
|
||||
"""
|
||||
try:
|
||||
# Dynamic import to avoid potential circular dependency
|
||||
from pr_agent.log import get_logger
|
||||
except:
|
||||
def get_logger():
|
||||
class DummyLogger:
|
||||
def debug(self, msg): pass
|
||||
return DummyLogger()
|
||||
|
||||
for key, value in secrets.items():
|
||||
if '.' in key: # nested key like "openai.key"
|
||||
parts = key.split('.')
|
||||
if len(parts) == 2:
|
||||
section, setting = parts
|
||||
section_upper = section.upper()
|
||||
setting_upper = setting.upper()
|
||||
|
||||
# Set only when no existing value (prioritize environment variables)
|
||||
current_value = get_settings().get(f"{section_upper}.{setting_upper}")
|
||||
if current_value is None or current_value == "":
|
||||
get_settings().set(f"{section_upper}.{setting_upper}", value)
|
||||
get_logger().debug(f"Set {section}.{setting} from AWS Secrets Manager")
|
||||
|
||||
@ -86,7 +86,7 @@ class BitbucketServerProvider(GitProvider):
|
||||
|
||||
def get_repo_settings(self):
|
||||
try:
|
||||
content = self.bitbucket_client.get_content_of_file(self.workspace_slug, self.repo_slug, ".pr_agent.toml", self.get_pr_branch())
|
||||
content = self.bitbucket_client.get_content_of_file(self.workspace_slug, self.repo_slug, ".pr_agent.toml")
|
||||
|
||||
return content
|
||||
except Exception as e:
|
||||
|
||||
@ -41,6 +41,12 @@ class GiteaProvider(GitProvider):
|
||||
configuration.host = "{}/api/v1".format(self.base_url)
|
||||
configuration.api_key['Authorization'] = f'token {gitea_access_token}'
|
||||
|
||||
if get_settings().get("GITEA.SKIP_SSL_VERIFICATION", False):
|
||||
configuration.verify_ssl = False
|
||||
|
||||
# Use custom cert (self-signed)
|
||||
configuration.ssl_ca_cert = get_settings().get("GITEA.SSL_CA_CERT", None)
|
||||
|
||||
client = giteapy.ApiClient(configuration)
|
||||
self.repo_api = RepoApi(client)
|
||||
self.owner = None
|
||||
@ -409,7 +415,7 @@ class GiteaProvider(GitProvider):
|
||||
def _get_file_content_from_base(self, filename: str) -> str:
|
||||
return self.repo_api.get_file_content(
|
||||
owner=self.owner,
|
||||
repo=self.base_ref,
|
||||
repo=self.repo,
|
||||
commit_sha=self.base_sha,
|
||||
filepath=filename
|
||||
)
|
||||
@ -417,7 +423,7 @@ class GiteaProvider(GitProvider):
|
||||
def _get_file_content_from_latest_commit(self, filename: str) -> str:
|
||||
return self.repo_api.get_file_content(
|
||||
owner=self.owner,
|
||||
repo=self.base_ref,
|
||||
repo=self.repo,
|
||||
commit_sha=self.last_commit.sha,
|
||||
filepath=filename
|
||||
)
|
||||
@ -471,11 +477,11 @@ class GiteaProvider(GitProvider):
|
||||
|
||||
if status == 'added':
|
||||
edit_type = EDIT_TYPE.ADDED
|
||||
elif status == 'removed':
|
||||
elif status == 'removed' or status == 'deleted':
|
||||
edit_type = EDIT_TYPE.DELETED
|
||||
elif status == 'renamed':
|
||||
edit_type = EDIT_TYPE.RENAMED
|
||||
elif status == 'modified':
|
||||
elif status == 'modified' or status == 'changed':
|
||||
edit_type = EDIT_TYPE.MODIFIED
|
||||
else:
|
||||
self.logger.error(f"Unknown edit type: {status}")
|
||||
|
||||
@ -1,16 +1,17 @@
|
||||
import difflib
|
||||
import hashlib
|
||||
import re
|
||||
from typing import Optional, Tuple
|
||||
from urllib.parse import urlparse
|
||||
from typing import Optional, Tuple, Any, Union
|
||||
from urllib.parse import urlparse, parse_qs
|
||||
|
||||
import gitlab
|
||||
import requests
|
||||
from gitlab import GitlabGetError
|
||||
from gitlab import GitlabGetError, GitlabAuthenticationError, GitlabCreateError, GitlabUpdateError
|
||||
|
||||
from pr_agent.algo.types import EDIT_TYPE, FilePatchInfo
|
||||
|
||||
from ..algo.file_filter import filter_ignored
|
||||
from ..algo.git_patch_processing import decode_if_bytes
|
||||
from ..algo.language_handler import is_valid_file
|
||||
from ..algo.utils import (clip_tokens,
|
||||
find_line_number_of_relevant_line_in_file,
|
||||
@ -112,14 +113,50 @@ class GitLabProvider(GitProvider):
|
||||
get_logger().error(f"Could not get diff for merge request {self.id_mr}")
|
||||
raise DiffNotFoundError(f"Could not get diff for merge request {self.id_mr}") from e
|
||||
|
||||
|
||||
def get_pr_file_content(self, file_path: str, branch: str) -> str:
|
||||
try:
|
||||
return self.gl.projects.get(self.id_project).files.get(file_path, branch).decode()
|
||||
file_obj = self.gl.projects.get(self.id_project).files.get(file_path, branch)
|
||||
content = file_obj.decode()
|
||||
return decode_if_bytes(content)
|
||||
except GitlabGetError:
|
||||
# In case of file creation the method returns GitlabGetError (404 file not found).
|
||||
# In this case we return an empty string for the diff.
|
||||
return ''
|
||||
except Exception as e:
|
||||
get_logger().warning(f"Error retrieving file {file_path} from branch {branch}: {e}")
|
||||
return ''
|
||||
|
||||
def create_or_update_pr_file(self, file_path: str, branch: str, contents="", message="") -> None:
|
||||
"""Create or update a file in the GitLab repository."""
|
||||
try:
|
||||
project = self.gl.projects.get(self.id_project)
|
||||
|
||||
if not message:
|
||||
action = "Update" if contents else "Create"
|
||||
message = f"{action} {file_path}"
|
||||
|
||||
try:
|
||||
existing_file = project.files.get(file_path, branch)
|
||||
existing_file.content = contents
|
||||
existing_file.save(branch=branch, commit_message=message)
|
||||
get_logger().debug(f"Updated file {file_path} in branch {branch}")
|
||||
except GitlabGetError:
|
||||
project.files.create({
|
||||
'file_path': file_path,
|
||||
'branch': branch,
|
||||
'content': contents,
|
||||
'commit_message': message
|
||||
})
|
||||
get_logger().debug(f"Created file {file_path} in branch {branch}")
|
||||
except GitlabAuthenticationError as e:
|
||||
get_logger().error(f"Authentication failed while creating/updating file {file_path} in branch {branch}: {e}")
|
||||
raise
|
||||
except (GitlabCreateError, GitlabUpdateError) as e:
|
||||
get_logger().error(f"Permission denied or validation error for file {file_path} in branch {branch}: {e}")
|
||||
raise
|
||||
except Exception as e:
|
||||
get_logger().exception(f"Unexpected error creating/updating file {file_path} in branch {branch}: {e}")
|
||||
raise
|
||||
|
||||
def get_diff_files(self) -> list[FilePatchInfo]:
|
||||
"""
|
||||
@ -167,14 +204,9 @@ class GitLabProvider(GitProvider):
|
||||
original_file_content_str = ''
|
||||
new_file_content_str = ''
|
||||
|
||||
try:
|
||||
if isinstance(original_file_content_str, bytes):
|
||||
original_file_content_str = bytes.decode(original_file_content_str, 'utf-8')
|
||||
if isinstance(new_file_content_str, bytes):
|
||||
new_file_content_str = bytes.decode(new_file_content_str, 'utf-8')
|
||||
except UnicodeDecodeError:
|
||||
get_logger().warning(
|
||||
f"Cannot decode file {diff['old_path']} or {diff['new_path']} in merge request {self.id_mr}")
|
||||
# Ensure content is properly decoded
|
||||
original_file_content_str = decode_if_bytes(original_file_content_str)
|
||||
new_file_content_str = decode_if_bytes(new_file_content_str)
|
||||
|
||||
edit_type = EDIT_TYPE.MODIFIED
|
||||
if diff['new_file']:
|
||||
|
||||
@ -13,5 +13,12 @@ def get_secret_provider():
|
||||
return GoogleCloudStorageSecretProvider()
|
||||
except Exception as e:
|
||||
raise ValueError(f"Failed to initialize google_cloud_storage secret provider {provider_id}") from e
|
||||
elif provider_id == 'aws_secrets_manager':
|
||||
try:
|
||||
from pr_agent.secret_providers.aws_secrets_manager_provider import \
|
||||
AWSSecretsManagerProvider
|
||||
return AWSSecretsManagerProvider()
|
||||
except Exception as e:
|
||||
raise ValueError(f"Failed to initialize aws_secrets_manager secret provider {provider_id}") from e
|
||||
else:
|
||||
raise ValueError("Unknown SECRET_PROVIDER")
|
||||
|
||||
57
pr_agent/secret_providers/aws_secrets_manager_provider.py
Normal file
57
pr_agent/secret_providers/aws_secrets_manager_provider.py
Normal file
@ -0,0 +1,57 @@
|
||||
import json
|
||||
import boto3
|
||||
from botocore.exceptions import ClientError
|
||||
|
||||
from pr_agent.config_loader import get_settings
|
||||
from pr_agent.log import get_logger
|
||||
from pr_agent.secret_providers.secret_provider import SecretProvider
|
||||
|
||||
|
||||
class AWSSecretsManagerProvider(SecretProvider):
|
||||
def __init__(self):
|
||||
try:
|
||||
region_name = get_settings().get("aws_secrets_manager.region_name") or \
|
||||
get_settings().get("aws.AWS_REGION_NAME")
|
||||
if region_name:
|
||||
self.client = boto3.client('secretsmanager', region_name=region_name)
|
||||
else:
|
||||
self.client = boto3.client('secretsmanager')
|
||||
|
||||
self.secret_arn = get_settings().get("aws_secrets_manager.secret_arn")
|
||||
if not self.secret_arn:
|
||||
raise ValueError("AWS Secrets Manager ARN is not configured")
|
||||
except Exception as e:
|
||||
get_logger().error(f"Failed to initialize AWS Secrets Manager Provider: {e}")
|
||||
raise e
|
||||
|
||||
def get_secret(self, secret_name: str) -> str:
|
||||
"""
|
||||
Retrieve individual secret by name (for webhook tokens)
|
||||
"""
|
||||
try:
|
||||
response = self.client.get_secret_value(SecretId=secret_name)
|
||||
return response['SecretString']
|
||||
except Exception as e:
|
||||
get_logger().warning(f"Failed to get secret {secret_name} from AWS Secrets Manager: {e}")
|
||||
return ""
|
||||
|
||||
def get_all_secrets(self) -> dict:
|
||||
"""
|
||||
Retrieve all secrets for configuration override
|
||||
"""
|
||||
try:
|
||||
response = self.client.get_secret_value(SecretId=self.secret_arn)
|
||||
return json.loads(response['SecretString'])
|
||||
except Exception as e:
|
||||
get_logger().error(f"Failed to get secrets from AWS Secrets Manager {self.secret_arn}: {e}")
|
||||
return {}
|
||||
|
||||
def store_secret(self, secret_name: str, secret_value: str):
|
||||
try:
|
||||
self.client.put_secret_value(
|
||||
SecretId=secret_name,
|
||||
SecretString=secret_value
|
||||
)
|
||||
except Exception as e:
|
||||
get_logger().error(f"Failed to store secret {secret_name} in AWS Secrets Manager: {e}")
|
||||
raise e
|
||||
@ -30,5 +30,9 @@
|
||||
"url": "/webhook"
|
||||
}
|
||||
]
|
||||
},
|
||||
"links": {
|
||||
"privacy": "https://qodo.ai/privacy-policy",
|
||||
"terms": "https://qodo.ai/terms"
|
||||
}
|
||||
}
|
||||
|
||||
@ -80,6 +80,30 @@ async def run_action():
|
||||
except Exception as e:
|
||||
get_logger().info(f"github action: failed to apply repo settings: {e}")
|
||||
|
||||
# Append the response language in the extra instructions
|
||||
try:
|
||||
response_language = get_settings().config.get('response_language', 'en-us')
|
||||
if response_language.lower() != 'en-us':
|
||||
get_logger().info(f'User has set the response language to: {response_language}')
|
||||
|
||||
lang_instruction_text = f"Your response MUST be written in the language corresponding to locale code: '{response_language}'. This is crucial."
|
||||
separator_text = "\n======\n\nIn addition, "
|
||||
|
||||
for key in get_settings():
|
||||
setting = get_settings().get(key)
|
||||
if str(type(setting)) == "<class 'dynaconf.utils.boxing.DynaBox'>":
|
||||
if key.lower() in ['pr_description', 'pr_code_suggestions', 'pr_reviewer']:
|
||||
if hasattr(setting, 'extra_instructions'):
|
||||
extra_instructions = setting.extra_instructions
|
||||
|
||||
if lang_instruction_text not in str(extra_instructions):
|
||||
updated_instructions = (
|
||||
str(extra_instructions) + separator_text + lang_instruction_text
|
||||
if extra_instructions else lang_instruction_text
|
||||
)
|
||||
setting.extra_instructions = updated_instructions
|
||||
except Exception as e:
|
||||
get_logger().info(f"github action: failed to apply language-specific instructions: {e}")
|
||||
# Handle pull request opened event
|
||||
if GITHUB_EVENT_NAME == "pull_request" or GITHUB_EVENT_NAME == "pull_request_target":
|
||||
action = event_payload.get("action")
|
||||
@ -102,7 +126,7 @@ async def run_action():
|
||||
auto_improve = get_setting_or_env("GITHUB_ACTION_CONFIG.AUTO_IMPROVE", None)
|
||||
|
||||
# Set the configuration for auto actions
|
||||
get_settings().config.is_auto_command = True # Set the flag to indicate that the command is auto
|
||||
get_settings().config.is_auto_command = True # Set the flag to indicate that the command is auto
|
||||
get_settings().pr_description.final_update_message = False # No final update message when auto_describe is enabled
|
||||
get_logger().info(f"Running auto actions: auto_describe={auto_describe}, auto_review={auto_review}, auto_improve={auto_improve}")
|
||||
|
||||
|
||||
27
pr_agent/servers/github_lambda_webhook.py
Normal file
27
pr_agent/servers/github_lambda_webhook.py
Normal file
@ -0,0 +1,27 @@
|
||||
from fastapi import FastAPI
|
||||
from mangum import Mangum
|
||||
from starlette.middleware import Middleware
|
||||
from starlette_context.middleware import RawContextMiddleware
|
||||
|
||||
from pr_agent.servers.github_app import router
|
||||
|
||||
try:
|
||||
from pr_agent.config_loader import apply_secrets_manager_config
|
||||
apply_secrets_manager_config()
|
||||
except Exception as e:
|
||||
try:
|
||||
from pr_agent.log import get_logger
|
||||
get_logger().debug(f"AWS Secrets Manager initialization failed, falling back to environment variables: {e}")
|
||||
except:
|
||||
# Fail completely silently if log module is not available
|
||||
pass
|
||||
|
||||
middleware = [Middleware(RawContextMiddleware)]
|
||||
app = FastAPI(middleware=middleware)
|
||||
app.include_router(router)
|
||||
|
||||
handler = Mangum(app, lifespan="off")
|
||||
|
||||
|
||||
def lambda_handler(event, context):
|
||||
return handler(event, context)
|
||||
27
pr_agent/servers/gitlab_lambda_webhook.py
Normal file
27
pr_agent/servers/gitlab_lambda_webhook.py
Normal file
@ -0,0 +1,27 @@
|
||||
from fastapi import FastAPI
|
||||
from mangum import Mangum
|
||||
from starlette.middleware import Middleware
|
||||
from starlette_context.middleware import RawContextMiddleware
|
||||
|
||||
from pr_agent.servers.gitlab_webhook import router
|
||||
|
||||
try:
|
||||
from pr_agent.config_loader import apply_secrets_manager_config
|
||||
apply_secrets_manager_config()
|
||||
except Exception as e:
|
||||
try:
|
||||
from pr_agent.log import get_logger
|
||||
get_logger().debug(f"AWS Secrets Manager initialization failed, falling back to environment variables: {e}")
|
||||
except:
|
||||
# Fail completely silently if log module is not available
|
||||
pass
|
||||
|
||||
middleware = [Middleware(RawContextMiddleware)]
|
||||
app = FastAPI(middleware=middleware)
|
||||
app.include_router(router)
|
||||
|
||||
handler = Mangum(app, lifespan="off")
|
||||
|
||||
|
||||
def lambda_handler(event, context):
|
||||
return handler(event, context)
|
||||
@ -1,16 +0,0 @@
|
||||
from fastapi import FastAPI
|
||||
from mangum import Mangum
|
||||
from starlette.middleware import Middleware
|
||||
from starlette_context.middleware import RawContextMiddleware
|
||||
|
||||
from pr_agent.servers.github_app import router
|
||||
|
||||
middleware = [Middleware(RawContextMiddleware)]
|
||||
app = FastAPI(middleware=middleware)
|
||||
app.include_router(router)
|
||||
|
||||
handler = Mangum(app, lifespan="off")
|
||||
|
||||
|
||||
def serverless(event, context):
|
||||
return handler(event, context)
|
||||
@ -121,4 +121,8 @@ api_base = ""
|
||||
[aws]
|
||||
AWS_ACCESS_KEY_ID = ""
|
||||
AWS_SECRET_ACCESS_KEY = ""
|
||||
AWS_REGION_NAME = ""
|
||||
AWS_REGION_NAME = ""
|
||||
|
||||
[aws_secrets_manager]
|
||||
secret_arn = "" # The ARN of the AWS Secrets Manager secret containing PR-Agent configuration
|
||||
region_name = "" # Optional: specific AWS region (defaults to AWS_REGION_NAME or Lambda region)
|
||||
|
||||
@ -39,7 +39,7 @@ allow_dynamic_context=true
|
||||
max_extra_lines_before_dynamic_context = 10 # will try to include up to 10 extra lines before the hunk in the patch, until we reach an enclosing function or class
|
||||
patch_extra_lines_before = 5 # Number of extra lines (+3 default ones) to include before each hunk in the patch
|
||||
patch_extra_lines_after = 1 # Number of extra lines (+3 default ones) to include after each hunk in the patch
|
||||
secret_provider=""
|
||||
secret_provider="" # "" (disabled), "google_cloud_storage", or "aws_secrets_manager" for secure secret management
|
||||
cli_mode=false
|
||||
ai_disclaimer_title="" # Pro feature, title for a collapsible disclaimer to AI outputs
|
||||
ai_disclaimer="" # Pro feature, full text for the AI disclaimer
|
||||
@ -78,8 +78,10 @@ require_tests_review=true
|
||||
require_estimate_effort_to_review=true
|
||||
require_can_be_split_review=false
|
||||
require_security_review=true
|
||||
require_todo_scan=false
|
||||
require_ticket_analysis_review=true
|
||||
# general options
|
||||
publish_output_no_suggestions=true # Set to "false" if you only need the reviewer's remarks (not labels, not "security audit", etc.) and want to avoid noisy "No major issues detected" comments.
|
||||
persistent_comment=true
|
||||
extra_instructions = ""
|
||||
num_max_findings = 3
|
||||
|
||||
@ -1,11 +1,12 @@
|
||||
[pr_description_prompt]
|
||||
system="""You are PR-Reviewer, a language model designed to review a Git Pull Request (PR).
|
||||
Your task is to provide a full description for the PR content - type, description, title and files walkthrough.
|
||||
Your task is to provide a full description for the PR content: type, description, title, and files walkthrough.
|
||||
- Focus on the new PR code (lines starting with '+' in the 'PR Git Diff' section).
|
||||
- Keep in mind that the 'Previous title', 'Previous description' and 'Commit messages' sections may be partial, simplistic, non-informative or out of date. Hence, compare them to the PR diff code, and use them only as a reference.
|
||||
- The generated title and description should prioritize the most significant changes.
|
||||
- If needed, each YAML output should be in block scalar indicator ('|')
|
||||
- When quoting variables, names or file paths from the code, use backticks (`) instead of single quote (').
|
||||
- When needed, use '- ' as bullets
|
||||
|
||||
{%- if extra_instructions %}
|
||||
|
||||
@ -181,4 +182,4 @@ pr_files:
|
||||
|
||||
Response (should be a valid YAML, and nothing else):
|
||||
```yaml
|
||||
"""
|
||||
"""
|
||||
|
||||
@ -1,12 +1,12 @@
|
||||
[pr_help_prompts]
|
||||
system="""You are Doc-helper, a language models designed to answer questions about a documentation website for an open-soure project called "PR-Agent" (recently renamed to "Qodo Merge").
|
||||
You will recieve a question, and the full documentation website content.
|
||||
You will receive a question, and the full documentation website content.
|
||||
Your goal is to provide the best answer to the question using the documentation provided.
|
||||
|
||||
Additional instructions:
|
||||
- Try to be short and concise in your answers. Try to give examples if needed.
|
||||
- The main tools of PR-Agent are 'describe', 'review', 'improve'. If there is ambiguity to which tool the user is referring to, prioritize snippets of these tools over others.
|
||||
- If the question has ambiguity and can relate to different tools or platfroms, provide the best answer possible based on what is available, but also state in your answer what additional information would be needed to give a more accurate answer.
|
||||
- If the question has ambiguity and can relate to different tools or platforms, provide the best answer possible based on what is available, but also state in your answer what additional information would be needed to give a more accurate answer.
|
||||
|
||||
|
||||
The output must be a YAML object equivalent to type $DocHelper, according to the following Pydantic definitions:
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
system="""You are PR-Reviewer, a language model designed to review a Git Pull Request (PR).
|
||||
Given the PR Info and the PR Git Diff, generate 3 short questions about the PR code for the PR author.
|
||||
The goal of the questions is to help the language model understand the PR better, so the questions should be insightful, informative, non-trivial, and relevant to the PR.
|
||||
You should prefer asking yes\\no questions, or multiple choice questions. Also add at least one open-ended question, but make sure they are not too difficult, and can be answered in a sentence or two.
|
||||
You should prefer asking yes/no questions, or multiple choice questions. Also add at least one open-ended question, but make sure they are not too difficult, and can be answered in a sentence or two.
|
||||
|
||||
|
||||
Example output:
|
||||
|
||||
@ -37,9 +37,9 @@ __new hunk__
|
||||
======
|
||||
|
||||
- In the format above, the diff is organized into separate '__new hunk__' and '__old hunk__' sections for each code chunk. '__new hunk__' contains the updated code, while '__old hunk__' shows the removed code. If no code was removed in a specific chunk, the __old hunk__ section will be omitted.
|
||||
- We also added line numbers for the '__new hunk__' code, to help you refer to the code lines in your suggestions. These line numbers are not part of the actual code, and should only used for reference.
|
||||
- We also added line numbers for the '__new hunk__' code, to help you refer to the code lines in your suggestions. These line numbers are not part of the actual code, and should only be used for reference.
|
||||
- Code lines are prefixed with symbols ('+', '-', ' '). The '+' symbol indicates new code added in the PR, the '-' symbol indicates code removed in the PR, and the ' ' symbol indicates unchanged code. \
|
||||
The review should address new code added in the PR code diff (lines starting with '+')
|
||||
The review should address new code added in the PR code diff (lines starting with '+').
|
||||
{%- if is_ai_metadata %}
|
||||
- If available, an AI-generated summary will appear and provide a high-level overview of the file changes. Note that this summary may not be fully accurate or complete.
|
||||
{%- endif %}
|
||||
@ -72,6 +72,13 @@ class KeyIssuesComponentLink(BaseModel):
|
||||
start_line: int = Field(description="The start line that corresponds to this issue in the relevant file")
|
||||
end_line: int = Field(description="The end line that corresponds to this issue in the relevant file")
|
||||
|
||||
{%- if require_todo_scan %}
|
||||
class TodoSection(BaseModel):
|
||||
relevant_file: str = Field(description="The full path of the file containing the TODO comment")
|
||||
line_number: int = Field(description="The line number where the TODO comment starts")
|
||||
content: str = Field(description="The content of the TODO comment. Only include actual TODO comments within code comments (e.g., comments starting with '#', '//', '/*', '<!--', ...). Remove leading 'TODO' prefixes. If more than 10 words, summarize the TODO comment to a single short sentence up to 10 words.")
|
||||
{%- endif %}
|
||||
|
||||
{%- if related_tickets %}
|
||||
|
||||
class TicketCompliance(BaseModel):
|
||||
@ -93,14 +100,17 @@ class Review(BaseModel):
|
||||
score: str = Field(description="Rate this PR on a scale of 0-100 (inclusive), where 0 means the worst possible PR code, and 100 means PR code of the highest quality, without any bugs or performance issues, that is ready to be merged immediately and run in production at scale.")
|
||||
{%- endif %}
|
||||
{%- if require_tests %}
|
||||
relevant_tests: str = Field(description="yes\\no question: does this PR have relevant tests added or updated ?")
|
||||
relevant_tests: str = Field(description="yes/no question: does this PR have relevant tests added or updated ?")
|
||||
{%- endif %}
|
||||
{%- if question_str %}
|
||||
insights_from_user_answers: str = Field(description="shortly summarize the insights you gained from the user's answers to the questions")
|
||||
{%- endif %}
|
||||
key_issues_to_review: List[KeyIssuesComponentLink] = Field("A short and diverse list (0-{{ num_max_findings }} issues) of high-priority bugs, problems or performance concerns introduced in the PR code, which the PR reviewer should further focus on and validate during the review process.")
|
||||
{%- if require_security_review %}
|
||||
security_concerns: str = Field(description="Does this PR code introduce possible vulnerabilities such as exposure of sensitive information (e.g., API keys, secrets, passwords), or security concerns like SQL injection, XSS, CSRF, and others ? Answer 'No' (without explaining why) if there are no possible issues. If there are security concerns or issues, start your answer with a short header, such as: 'Sensitive information exposure: ...', 'SQL injection: ...' etc. Explain your answer. Be specific and give examples if possible")
|
||||
security_concerns: str = Field(description="Does this PR code introduce vulnerabilities such as exposure of sensitive information (e.g., API keys, secrets, passwords), or security concerns like SQL injection, XSS, CSRF, and others ? Answer 'No' (without explaining why) if there are no possible issues. If there are security concerns or issues, start your answer with a short header, such as: 'Sensitive information exposure: ...', 'SQL injection: ...', etc. Explain your answer. Be specific and give examples if possible")
|
||||
{%- endif %}
|
||||
{%- if require_todo_scan %}
|
||||
todo_sections: Union[List[TodoSection], str] = Field(description="A list of TODO comments found in the PR code. Return 'No' (as a string) if there are no TODO comments in the PR")
|
||||
{%- endif %}
|
||||
{%- if require_can_be_split_review %}
|
||||
can_be_split: List[SubPR] = Field(min_items=0, max_items=3, description="Can this PR, which contains {{ num_pr_files }} changed files in total, be divided into smaller sub-PRs with distinct tasks that can be reviewed and merged independently, regardless of the order ? Make sure that the sub-PRs are indeed independent, with no code dependencies between them, and that each sub-PR represent a meaningful independent task. Output an empty list if the PR code does not need to be split.")
|
||||
@ -148,6 +158,10 @@ review:
|
||||
- ...
|
||||
security_concerns: |
|
||||
No
|
||||
{%- if require_todo_scan %}
|
||||
todo_sections: |
|
||||
No
|
||||
{%- endif %}
|
||||
{%- if require_can_be_split_review %}
|
||||
can_be_split:
|
||||
- relevant_files:
|
||||
@ -266,6 +280,10 @@ review:
|
||||
- ...
|
||||
security_concerns: |
|
||||
No
|
||||
{%- if require_todo_scan %}
|
||||
todo_sections: |
|
||||
No
|
||||
{%- endif %}
|
||||
{%- if require_can_be_split_review %}
|
||||
can_be_split:
|
||||
- relevant_files:
|
||||
|
||||
@ -21,7 +21,7 @@ from pr_agent.servers.help import HelpMessage
|
||||
|
||||
#Common code that can be called from similar tools:
|
||||
def modify_answer_section(ai_response: str) -> str | None:
|
||||
# Gets the model's answer and relevant sources section, repacing the heading of the answer section with:
|
||||
# Gets the model's answer and relevant sources section, replacing the heading of the answer section with:
|
||||
# :bulb: Auto-generated documentation-based answer:
|
||||
"""
|
||||
For example: The following input:
|
||||
|
||||
@ -87,6 +87,7 @@ class PRReviewer:
|
||||
"require_estimate_effort_to_review": get_settings().pr_reviewer.require_estimate_effort_to_review,
|
||||
'require_can_be_split_review': get_settings().pr_reviewer.require_can_be_split_review,
|
||||
'require_security_review': get_settings().pr_reviewer.require_security_review,
|
||||
'require_todo_scan': get_settings().pr_reviewer.get("require_todo_scan", False),
|
||||
'question_str': question_str,
|
||||
'answer_str': answer_str,
|
||||
"extra_instructions": get_settings().pr_reviewer.extra_instructions,
|
||||
@ -158,25 +159,32 @@ class PRReviewer:
|
||||
pr_review = self._prepare_pr_review()
|
||||
get_logger().debug(f"PR output", artifact=pr_review)
|
||||
|
||||
if get_settings().config.publish_output:
|
||||
# publish the review
|
||||
if get_settings().pr_reviewer.persistent_comment and not self.incremental.is_incremental:
|
||||
final_update_message = get_settings().pr_reviewer.final_update_message
|
||||
self.git_provider.publish_persistent_comment(pr_review,
|
||||
initial_header=f"{PRReviewHeader.REGULAR.value} 🔍",
|
||||
update_header=True,
|
||||
final_update_message=final_update_message, )
|
||||
else:
|
||||
self.git_provider.publish_comment(pr_review)
|
||||
|
||||
self.git_provider.remove_initial_comment()
|
||||
else:
|
||||
get_logger().info("Review output is not published")
|
||||
should_publish = get_settings().config.publish_output and self._should_publish_review_no_suggestions(pr_review)
|
||||
if not should_publish:
|
||||
reason = "Review output is not published"
|
||||
if get_settings().config.publish_output:
|
||||
reason += ": no major issues detected."
|
||||
get_logger().info(reason)
|
||||
get_settings().data = {"artifact": pr_review}
|
||||
return
|
||||
|
||||
# publish the review
|
||||
if get_settings().pr_reviewer.persistent_comment and not self.incremental.is_incremental:
|
||||
final_update_message = get_settings().pr_reviewer.final_update_message
|
||||
self.git_provider.publish_persistent_comment(pr_review,
|
||||
initial_header=f"{PRReviewHeader.REGULAR.value} 🔍",
|
||||
update_header=True,
|
||||
final_update_message=final_update_message, )
|
||||
else:
|
||||
self.git_provider.publish_comment(pr_review)
|
||||
|
||||
self.git_provider.remove_initial_comment()
|
||||
except Exception as e:
|
||||
get_logger().error(f"Failed to review PR: {e}")
|
||||
|
||||
def _should_publish_review_no_suggestions(self, pr_review: str) -> bool:
|
||||
return get_settings().pr_reviewer.get('publish_output_no_suggestions', True) or "No major issues detected" not in pr_review
|
||||
|
||||
async def _prepare_prediction(self, model: str) -> None:
|
||||
self.patches_diff = get_pr_diff(self.git_provider,
|
||||
self.token_handler,
|
||||
|
||||
@ -58,7 +58,7 @@ class PRUpdateChangelog:
|
||||
'config': dict(get_settings().config)}
|
||||
get_logger().debug("Relevant configs", artifacts=relevant_configs)
|
||||
|
||||
# currently only GitHub is supported for pushing changelog changes
|
||||
# check if the git provider supports pushing changelog changes
|
||||
if get_settings().pr_update_changelog.push_changelog_changes and not hasattr(
|
||||
self.git_provider, "create_or_update_pr_file"
|
||||
):
|
||||
@ -128,6 +128,7 @@ class PRUpdateChangelog:
|
||||
existing_content = self.changelog_file
|
||||
else:
|
||||
existing_content = ""
|
||||
|
||||
if existing_content:
|
||||
new_file_content = answer + "\n\n" + self.changelog_file
|
||||
else:
|
||||
@ -186,12 +187,18 @@ Example:
|
||||
self.changelog_file = self.git_provider.get_pr_file_content(
|
||||
"CHANGELOG.md", self.git_provider.get_pr_branch()
|
||||
)
|
||||
|
||||
if isinstance(self.changelog_file, bytes):
|
||||
self.changelog_file = self.changelog_file.decode('utf-8')
|
||||
|
||||
changelog_file_lines = self.changelog_file.splitlines()
|
||||
changelog_file_lines = changelog_file_lines[:CHANGELOG_LINES]
|
||||
self.changelog_file_str = "\n".join(changelog_file_lines)
|
||||
except Exception:
|
||||
except Exception as e:
|
||||
get_logger().warning(f"Error getting changelog file: {e}")
|
||||
self.changelog_file_str = ""
|
||||
self.changelog_file = ""
|
||||
return
|
||||
|
||||
if not self.changelog_file_str:
|
||||
self.changelog_file_str = self._get_default_changelog()
|
||||
|
||||
89
tests/unittest/test_aws_secrets_manager_provider.py
Normal file
89
tests/unittest/test_aws_secrets_manager_provider.py
Normal file
@ -0,0 +1,89 @@
|
||||
import json
|
||||
import pytest
|
||||
from unittest.mock import MagicMock, patch
|
||||
from botocore.exceptions import ClientError
|
||||
|
||||
from pr_agent.secret_providers.aws_secrets_manager_provider import AWSSecretsManagerProvider
|
||||
|
||||
|
||||
class TestAWSSecretsManagerProvider:
|
||||
|
||||
def _provider(self):
|
||||
"""Create provider following existing pattern"""
|
||||
with patch('pr_agent.secret_providers.aws_secrets_manager_provider.get_settings') as mock_get_settings, \
|
||||
patch('pr_agent.secret_providers.aws_secrets_manager_provider.boto3.client') as mock_boto3_client:
|
||||
|
||||
settings = MagicMock()
|
||||
settings.get.side_effect = lambda k, d=None: {
|
||||
'aws_secrets_manager.secret_arn': 'arn:aws:secretsmanager:us-east-1:123456789012:secret:test-secret',
|
||||
'aws_secrets_manager.region_name': 'us-east-1',
|
||||
'aws.AWS_REGION_NAME': 'us-east-1'
|
||||
}.get(k, d)
|
||||
settings.aws_secrets_manager.secret_arn = 'arn:aws:secretsmanager:us-east-1:123456789012:secret:test-secret'
|
||||
mock_get_settings.return_value = settings
|
||||
|
||||
# Mock boto3 client
|
||||
mock_client = MagicMock()
|
||||
mock_boto3_client.return_value = mock_client
|
||||
|
||||
provider = AWSSecretsManagerProvider()
|
||||
provider.client = mock_client # Set client directly for testing
|
||||
return provider, mock_client
|
||||
|
||||
# Positive test cases
|
||||
def test_get_secret_success(self):
|
||||
provider, mock_client = self._provider()
|
||||
mock_client.get_secret_value.return_value = {'SecretString': 'test-secret-value'}
|
||||
|
||||
result = provider.get_secret('test-secret-name')
|
||||
assert result == 'test-secret-value'
|
||||
mock_client.get_secret_value.assert_called_once_with(SecretId='test-secret-name')
|
||||
|
||||
def test_get_all_secrets_success(self):
|
||||
provider, mock_client = self._provider()
|
||||
secret_data = {'openai.key': 'sk-test', 'github.webhook_secret': 'webhook-secret'}
|
||||
mock_client.get_secret_value.return_value = {'SecretString': json.dumps(secret_data)}
|
||||
|
||||
result = provider.get_all_secrets()
|
||||
assert result == secret_data
|
||||
|
||||
# Negative test cases (following Google Cloud Storage pattern)
|
||||
def test_get_secret_failure(self):
|
||||
provider, mock_client = self._provider()
|
||||
mock_client.get_secret_value.side_effect = Exception("AWS error")
|
||||
|
||||
result = provider.get_secret('nonexistent-secret')
|
||||
assert result == "" # Confirm empty string is returned
|
||||
|
||||
def test_get_all_secrets_failure(self):
|
||||
provider, mock_client = self._provider()
|
||||
mock_client.get_secret_value.side_effect = Exception("AWS error")
|
||||
|
||||
result = provider.get_all_secrets()
|
||||
assert result == {} # Confirm empty dictionary is returned
|
||||
|
||||
def test_store_secret_update_existing(self):
|
||||
provider, mock_client = self._provider()
|
||||
mock_client.update_secret.return_value = {}
|
||||
|
||||
provider.store_secret('test-secret', 'test-value')
|
||||
mock_client.put_secret_value.assert_called_once_with(
|
||||
SecretId='test-secret',
|
||||
SecretString='test-value'
|
||||
)
|
||||
|
||||
def test_init_failure_invalid_config(self):
|
||||
with patch('pr_agent.secret_providers.aws_secrets_manager_provider.get_settings') as mock_get_settings:
|
||||
settings = MagicMock()
|
||||
settings.aws_secrets_manager.secret_arn = None # Configuration error
|
||||
mock_get_settings.return_value = settings
|
||||
|
||||
with pytest.raises(Exception):
|
||||
AWSSecretsManagerProvider()
|
||||
|
||||
def test_store_secret_failure(self):
|
||||
provider, mock_client = self._provider()
|
||||
mock_client.put_secret_value.side_effect = Exception("AWS error")
|
||||
|
||||
with pytest.raises(Exception):
|
||||
provider.store_secret('test-secret', 'test-value')
|
||||
120
tests/unittest/test_config_loader_secrets.py
Normal file
120
tests/unittest/test_config_loader_secrets.py
Normal file
@ -0,0 +1,120 @@
|
||||
import pytest
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from pr_agent.config_loader import apply_secrets_manager_config, apply_secrets_to_config
|
||||
|
||||
|
||||
class TestConfigLoaderSecrets:
|
||||
|
||||
def test_apply_secrets_manager_config_success(self):
|
||||
with patch('pr_agent.secret_providers.get_secret_provider') as mock_get_provider, \
|
||||
patch('pr_agent.config_loader.apply_secrets_to_config') as mock_apply_secrets, \
|
||||
patch('pr_agent.config_loader.get_settings') as mock_get_settings:
|
||||
|
||||
# Mock secret provider
|
||||
mock_provider = MagicMock()
|
||||
mock_provider.get_all_secrets.return_value = {'openai.key': 'sk-test'}
|
||||
mock_get_provider.return_value = mock_provider
|
||||
|
||||
# Mock settings
|
||||
settings = MagicMock()
|
||||
settings.get.return_value = "aws_secrets_manager"
|
||||
mock_get_settings.return_value = settings
|
||||
|
||||
apply_secrets_manager_config()
|
||||
|
||||
mock_apply_secrets.assert_called_once_with({'openai.key': 'sk-test'})
|
||||
|
||||
def test_apply_secrets_manager_config_no_provider(self):
|
||||
with patch('pr_agent.secret_providers.get_secret_provider') as mock_get_provider:
|
||||
mock_get_provider.return_value = None
|
||||
|
||||
# Confirm no exception is raised
|
||||
apply_secrets_manager_config()
|
||||
|
||||
def test_apply_secrets_manager_config_not_aws(self):
|
||||
with patch('pr_agent.secret_providers.get_secret_provider') as mock_get_provider, \
|
||||
patch('pr_agent.config_loader.get_settings') as mock_get_settings:
|
||||
|
||||
# Mock Google Cloud Storage provider
|
||||
mock_provider = MagicMock()
|
||||
mock_get_provider.return_value = mock_provider
|
||||
|
||||
# Mock settings (Google Cloud Storage)
|
||||
settings = MagicMock()
|
||||
settings.get.return_value = "google_cloud_storage"
|
||||
mock_get_settings.return_value = settings
|
||||
|
||||
# Confirm execution is skipped for non-AWS Secrets Manager
|
||||
apply_secrets_manager_config()
|
||||
|
||||
# Confirm get_all_secrets is not called
|
||||
assert not hasattr(mock_provider, 'get_all_secrets') or \
|
||||
not mock_provider.get_all_secrets.called
|
||||
|
||||
def test_apply_secrets_to_config_nested_keys(self):
|
||||
with patch('pr_agent.config_loader.get_settings') as mock_get_settings:
|
||||
settings = MagicMock()
|
||||
settings.get.return_value = None # No existing value
|
||||
settings.set = MagicMock()
|
||||
mock_get_settings.return_value = settings
|
||||
|
||||
secrets = {
|
||||
'openai.key': 'sk-test',
|
||||
'github.webhook_secret': 'webhook-secret'
|
||||
}
|
||||
|
||||
apply_secrets_to_config(secrets)
|
||||
|
||||
# Confirm settings are applied correctly
|
||||
settings.set.assert_any_call('OPENAI.KEY', 'sk-test')
|
||||
settings.set.assert_any_call('GITHUB.WEBHOOK_SECRET', 'webhook-secret')
|
||||
|
||||
def test_apply_secrets_to_config_existing_value_preserved(self):
|
||||
with patch('pr_agent.config_loader.get_settings') as mock_get_settings:
|
||||
settings = MagicMock()
|
||||
settings.get.return_value = "existing-value" # Existing value present
|
||||
settings.set = MagicMock()
|
||||
mock_get_settings.return_value = settings
|
||||
|
||||
secrets = {'openai.key': 'sk-test'}
|
||||
|
||||
apply_secrets_to_config(secrets)
|
||||
|
||||
# Confirm settings are not overridden when existing value present
|
||||
settings.set.assert_not_called()
|
||||
|
||||
def test_apply_secrets_to_config_single_key(self):
|
||||
with patch('pr_agent.config_loader.get_settings') as mock_get_settings:
|
||||
settings = MagicMock()
|
||||
settings.get.return_value = None
|
||||
settings.set = MagicMock()
|
||||
mock_get_settings.return_value = settings
|
||||
|
||||
secrets = {'simple_key': 'simple_value'}
|
||||
|
||||
apply_secrets_to_config(secrets)
|
||||
|
||||
# Confirm non-dot notation keys are ignored
|
||||
settings.set.assert_not_called()
|
||||
|
||||
def test_apply_secrets_to_config_multiple_dots(self):
|
||||
with patch('pr_agent.config_loader.get_settings') as mock_get_settings:
|
||||
settings = MagicMock()
|
||||
settings.get.return_value = None
|
||||
settings.set = MagicMock()
|
||||
mock_get_settings.return_value = settings
|
||||
|
||||
secrets = {'section.subsection.key': 'value'}
|
||||
|
||||
apply_secrets_to_config(secrets)
|
||||
|
||||
# Confirm keys with multiple dots are ignored
|
||||
settings.set.assert_not_called()
|
||||
|
||||
def test_apply_secrets_manager_config_exception_handling(self):
|
||||
with patch('pr_agent.secret_providers.get_secret_provider') as mock_get_provider:
|
||||
mock_get_provider.side_effect = Exception("Provider error")
|
||||
|
||||
# Confirm processing continues even when exception occurs
|
||||
apply_secrets_manager_config() # Confirm no exception is raised
|
||||
147
tests/unittest/test_gitlab_provider.py
Normal file
147
tests/unittest/test_gitlab_provider.py
Normal file
@ -0,0 +1,147 @@
|
||||
import pytest
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from pr_agent.git_providers.gitlab_provider import GitLabProvider
|
||||
from gitlab import Gitlab
|
||||
from gitlab.v4.objects import Project, ProjectFile
|
||||
from gitlab.exceptions import GitlabGetError
|
||||
|
||||
|
||||
class TestGitLabProvider:
|
||||
"""Test suite for GitLab provider functionality."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_gitlab_client(self):
|
||||
client = MagicMock()
|
||||
return client
|
||||
|
||||
@pytest.fixture
|
||||
def mock_project(self):
|
||||
project = MagicMock()
|
||||
return project
|
||||
|
||||
@pytest.fixture
|
||||
def gitlab_provider(self, mock_gitlab_client, mock_project):
|
||||
with patch('pr_agent.git_providers.gitlab_provider.gitlab.Gitlab', return_value=mock_gitlab_client), \
|
||||
patch('pr_agent.git_providers.gitlab_provider.get_settings') as mock_settings:
|
||||
|
||||
mock_settings.return_value.get.side_effect = lambda key, default=None: {
|
||||
"GITLAB.URL": "https://gitlab.com",
|
||||
"GITLAB.PERSONAL_ACCESS_TOKEN": "fake_token"
|
||||
}.get(key, default)
|
||||
|
||||
mock_gitlab_client.projects.get.return_value = mock_project
|
||||
provider = GitLabProvider("https://gitlab.com/test/repo/-/merge_requests/1")
|
||||
provider.gl = mock_gitlab_client
|
||||
provider.id_project = "test/repo"
|
||||
return provider
|
||||
|
||||
def test_get_pr_file_content_success(self, gitlab_provider, mock_project):
|
||||
mock_file = MagicMock(ProjectFile)
|
||||
mock_file.decode.return_value = "# Changelog\n\n## v1.0.0\n- Initial release"
|
||||
mock_project.files.get.return_value = mock_file
|
||||
|
||||
content = gitlab_provider.get_pr_file_content("CHANGELOG.md", "main")
|
||||
|
||||
assert content == "# Changelog\n\n## v1.0.0\n- Initial release"
|
||||
mock_project.files.get.assert_called_once_with("CHANGELOG.md", "main")
|
||||
mock_file.decode.assert_called_once()
|
||||
|
||||
def test_get_pr_file_content_with_bytes(self, gitlab_provider, mock_project):
|
||||
mock_file = MagicMock(ProjectFile)
|
||||
mock_file.decode.return_value = b"# Changelog\n\n## v1.0.0\n- Initial release"
|
||||
mock_project.files.get.return_value = mock_file
|
||||
|
||||
content = gitlab_provider.get_pr_file_content("CHANGELOG.md", "main")
|
||||
|
||||
assert content == "# Changelog\n\n## v1.0.0\n- Initial release"
|
||||
mock_project.files.get.assert_called_once_with("CHANGELOG.md", "main")
|
||||
|
||||
def test_get_pr_file_content_file_not_found(self, gitlab_provider, mock_project):
|
||||
mock_project.files.get.side_effect = GitlabGetError("404 Not Found")
|
||||
|
||||
content = gitlab_provider.get_pr_file_content("CHANGELOG.md", "main")
|
||||
|
||||
assert content == ""
|
||||
mock_project.files.get.assert_called_once_with("CHANGELOG.md", "main")
|
||||
|
||||
def test_get_pr_file_content_other_exception(self, gitlab_provider, mock_project):
|
||||
mock_project.files.get.side_effect = Exception("Network error")
|
||||
|
||||
content = gitlab_provider.get_pr_file_content("CHANGELOG.md", "main")
|
||||
|
||||
assert content == ""
|
||||
|
||||
def test_create_or_update_pr_file_create_new(self, gitlab_provider, mock_project):
|
||||
mock_project.files.get.side_effect = GitlabGetError("404 Not Found")
|
||||
mock_file = MagicMock()
|
||||
mock_project.files.create.return_value = mock_file
|
||||
|
||||
new_content = "# Changelog\n\n## v1.1.0\n- New feature"
|
||||
commit_message = "Add CHANGELOG.md"
|
||||
|
||||
gitlab_provider.create_or_update_pr_file(
|
||||
"CHANGELOG.md", "feature-branch", new_content, commit_message
|
||||
)
|
||||
|
||||
mock_project.files.get.assert_called_once_with("CHANGELOG.md", "feature-branch")
|
||||
mock_project.files.create.assert_called_once_with({
|
||||
'file_path': 'CHANGELOG.md',
|
||||
'branch': 'feature-branch',
|
||||
'content': new_content,
|
||||
'commit_message': commit_message,
|
||||
})
|
||||
|
||||
def test_create_or_update_pr_file_update_existing(self, gitlab_provider, mock_project):
|
||||
mock_file = MagicMock(ProjectFile)
|
||||
mock_file.decode.return_value = "# Old changelog content"
|
||||
mock_project.files.get.return_value = mock_file
|
||||
|
||||
new_content = "# New changelog content"
|
||||
commit_message = "Update CHANGELOG.md"
|
||||
|
||||
gitlab_provider.create_or_update_pr_file(
|
||||
"CHANGELOG.md", "feature-branch", new_content, commit_message
|
||||
)
|
||||
|
||||
mock_project.files.get.assert_called_once_with("CHANGELOG.md", "feature-branch")
|
||||
mock_file.content = new_content
|
||||
mock_file.save.assert_called_once_with(branch="feature-branch", commit_message=commit_message)
|
||||
|
||||
def test_create_or_update_pr_file_update_exception(self, gitlab_provider, mock_project):
|
||||
mock_project.files.get.side_effect = Exception("Network error")
|
||||
|
||||
with pytest.raises(Exception):
|
||||
gitlab_provider.create_or_update_pr_file(
|
||||
"CHANGELOG.md", "feature-branch", "content", "message"
|
||||
)
|
||||
|
||||
def test_has_create_or_update_pr_file_method(self, gitlab_provider):
|
||||
assert hasattr(gitlab_provider, "create_or_update_pr_file")
|
||||
assert callable(getattr(gitlab_provider, "create_or_update_pr_file"))
|
||||
|
||||
def test_method_signature_compatibility(self, gitlab_provider):
|
||||
import inspect
|
||||
|
||||
sig = inspect.signature(gitlab_provider.create_or_update_pr_file)
|
||||
params = list(sig.parameters.keys())
|
||||
|
||||
expected_params = ['file_path', 'branch', 'contents', 'message']
|
||||
assert params == expected_params
|
||||
|
||||
@pytest.mark.parametrize("content,expected", [
|
||||
("simple text", "simple text"),
|
||||
(b"bytes content", "bytes content"),
|
||||
("", ""),
|
||||
(b"", ""),
|
||||
("unicode: café", "unicode: café"),
|
||||
(b"unicode: caf\xc3\xa9", "unicode: café"),
|
||||
])
|
||||
def test_content_encoding_handling(self, gitlab_provider, mock_project, content, expected):
|
||||
mock_file = MagicMock(ProjectFile)
|
||||
mock_file.decode.return_value = content
|
||||
mock_project.files.get.return_value = mock_file
|
||||
|
||||
result = gitlab_provider.get_pr_file_content("test.md", "main")
|
||||
|
||||
assert result == expected
|
||||
247
tests/unittest/test_pr_update_changelog.py
Normal file
247
tests/unittest/test_pr_update_changelog.py
Normal file
@ -0,0 +1,247 @@
|
||||
import pytest
|
||||
from unittest.mock import MagicMock, patch, AsyncMock
|
||||
from pr_agent.tools.pr_update_changelog import PRUpdateChangelog
|
||||
|
||||
|
||||
class TestPRUpdateChangelog:
|
||||
"""Test suite for the PR Update Changelog functionality."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_git_provider(self):
|
||||
"""Create a mock git provider."""
|
||||
provider = MagicMock()
|
||||
provider.get_pr_branch.return_value = "feature-branch"
|
||||
provider.get_pr_file_content.return_value = ""
|
||||
provider.pr.title = "Test PR"
|
||||
provider.get_pr_description.return_value = "Test description"
|
||||
provider.get_commit_messages.return_value = "fix: test commit"
|
||||
provider.get_languages.return_value = {"Python": 80, "JavaScript": 20}
|
||||
provider.get_files.return_value = ["test.py", "test.js"]
|
||||
return provider
|
||||
|
||||
@pytest.fixture
|
||||
def mock_ai_handler(self):
|
||||
"""Create a mock AI handler."""
|
||||
handler = MagicMock()
|
||||
handler.chat_completion = AsyncMock(return_value=("Test changelog entry", "stop"))
|
||||
return handler
|
||||
|
||||
@pytest.fixture
|
||||
def changelog_tool(self, mock_git_provider, mock_ai_handler):
|
||||
"""Create a PRUpdateChangelog instance with mocked dependencies."""
|
||||
with patch('pr_agent.tools.pr_update_changelog.get_git_provider', return_value=lambda url: mock_git_provider), \
|
||||
patch('pr_agent.tools.pr_update_changelog.get_main_pr_language', return_value="Python"), \
|
||||
patch('pr_agent.tools.pr_update_changelog.get_settings') as mock_settings:
|
||||
|
||||
# Configure mock settings
|
||||
mock_settings.return_value.pr_update_changelog.push_changelog_changes = False
|
||||
mock_settings.return_value.pr_update_changelog.extra_instructions = ""
|
||||
mock_settings.return_value.pr_update_changelog_prompt.system = "System prompt"
|
||||
mock_settings.return_value.pr_update_changelog_prompt.user = "User prompt"
|
||||
mock_settings.return_value.config.temperature = 0.2
|
||||
|
||||
tool = PRUpdateChangelog("https://gitlab.com/test/repo/-/merge_requests/1", ai_handler=lambda: mock_ai_handler)
|
||||
return tool
|
||||
|
||||
def test_get_changelog_file_with_existing_content(self, changelog_tool, mock_git_provider):
|
||||
"""Test retrieving existing changelog content."""
|
||||
# Arrange
|
||||
existing_content = "# Changelog\n\n## v1.0.0\n- Initial release\n- Bug fixes"
|
||||
mock_git_provider.get_pr_file_content.return_value = existing_content
|
||||
|
||||
# Act
|
||||
changelog_tool._get_changelog_file()
|
||||
|
||||
# Assert
|
||||
assert changelog_tool.changelog_file == existing_content
|
||||
assert "# Changelog" in changelog_tool.changelog_file_str
|
||||
|
||||
def test_get_changelog_file_with_no_existing_content(self, changelog_tool, mock_git_provider):
|
||||
"""Test handling when no changelog file exists."""
|
||||
# Arrange
|
||||
mock_git_provider.get_pr_file_content.return_value = ""
|
||||
|
||||
# Act
|
||||
changelog_tool._get_changelog_file()
|
||||
|
||||
# Assert
|
||||
assert changelog_tool.changelog_file == ""
|
||||
assert "Example:" in changelog_tool.changelog_file_str # Default template
|
||||
|
||||
def test_get_changelog_file_with_bytes_content(self, changelog_tool, mock_git_provider):
|
||||
"""Test handling when git provider returns bytes instead of string."""
|
||||
# Arrange
|
||||
content_bytes = b"# Changelog\n\n## v1.0.0\n- Initial release"
|
||||
mock_git_provider.get_pr_file_content.return_value = content_bytes
|
||||
|
||||
# Act
|
||||
changelog_tool._get_changelog_file()
|
||||
|
||||
# Assert
|
||||
assert isinstance(changelog_tool.changelog_file, str)
|
||||
assert changelog_tool.changelog_file == "# Changelog\n\n## v1.0.0\n- Initial release"
|
||||
|
||||
def test_get_changelog_file_with_exception(self, changelog_tool, mock_git_provider):
|
||||
"""Test handling exceptions during file retrieval."""
|
||||
# Arrange
|
||||
mock_git_provider.get_pr_file_content.side_effect = Exception("Network error")
|
||||
|
||||
# Act
|
||||
changelog_tool._get_changelog_file()
|
||||
|
||||
# Assert
|
||||
assert changelog_tool.changelog_file == ""
|
||||
assert changelog_tool.changelog_file_str == "" # Exception should result in empty string, no default template
|
||||
|
||||
def test_prepare_changelog_update_with_existing_content(self, changelog_tool):
|
||||
"""Test preparing changelog update when existing content exists."""
|
||||
# Arrange
|
||||
changelog_tool.prediction = "## v1.1.0\n- New feature\n- Bug fix"
|
||||
changelog_tool.changelog_file = "# Changelog\n\n## v1.0.0\n- Initial release"
|
||||
changelog_tool.commit_changelog = True
|
||||
|
||||
# Act
|
||||
new_content, answer = changelog_tool._prepare_changelog_update()
|
||||
|
||||
# Assert
|
||||
assert new_content.startswith("## v1.1.0\n- New feature\n- Bug fix\n\n")
|
||||
assert "# Changelog\n\n## v1.0.0\n- Initial release" in new_content
|
||||
assert answer == "## v1.1.0\n- New feature\n- Bug fix"
|
||||
|
||||
def test_prepare_changelog_update_without_existing_content(self, changelog_tool):
|
||||
"""Test preparing changelog update when no existing content."""
|
||||
# Arrange
|
||||
changelog_tool.prediction = "## v1.0.0\n- Initial release"
|
||||
changelog_tool.changelog_file = ""
|
||||
changelog_tool.commit_changelog = True
|
||||
|
||||
# Act
|
||||
new_content, answer = changelog_tool._prepare_changelog_update()
|
||||
|
||||
# Assert
|
||||
assert new_content == "## v1.0.0\n- Initial release"
|
||||
assert answer == "## v1.0.0\n- Initial release"
|
||||
|
||||
def test_prepare_changelog_update_no_commit(self, changelog_tool):
|
||||
"""Test preparing changelog update when not committing."""
|
||||
# Arrange
|
||||
changelog_tool.prediction = "## v1.1.0\n- New feature"
|
||||
changelog_tool.changelog_file = ""
|
||||
changelog_tool.commit_changelog = False
|
||||
|
||||
# Act
|
||||
new_content, answer = changelog_tool._prepare_changelog_update()
|
||||
|
||||
# Assert
|
||||
assert new_content == "## v1.1.0\n- New feature"
|
||||
assert "to commit the new content" in answer
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_run_without_push_support(self, changelog_tool, mock_git_provider):
|
||||
"""Test running changelog update when git provider doesn't support pushing."""
|
||||
# Arrange
|
||||
delattr(mock_git_provider, 'create_or_update_pr_file') # Remove the method
|
||||
changelog_tool.commit_changelog = True
|
||||
|
||||
with patch('pr_agent.tools.pr_update_changelog.get_settings') as mock_settings:
|
||||
mock_settings.return_value.pr_update_changelog.push_changelog_changes = True
|
||||
mock_settings.return_value.config.publish_output = True
|
||||
|
||||
# Act
|
||||
await changelog_tool.run()
|
||||
|
||||
# Assert
|
||||
mock_git_provider.publish_comment.assert_called_once()
|
||||
assert "not currently supported" in str(mock_git_provider.publish_comment.call_args)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_run_with_push_support(self, changelog_tool, mock_git_provider):
|
||||
"""Test running changelog update when git provider supports pushing."""
|
||||
# Arrange
|
||||
mock_git_provider.create_or_update_pr_file = MagicMock()
|
||||
changelog_tool.commit_changelog = True
|
||||
changelog_tool.prediction = "## v1.1.0\n- New feature"
|
||||
|
||||
with patch('pr_agent.tools.pr_update_changelog.get_settings') as mock_settings, \
|
||||
patch('pr_agent.tools.pr_update_changelog.retry_with_fallback_models') as mock_retry, \
|
||||
patch('pr_agent.tools.pr_update_changelog.sleep'):
|
||||
|
||||
mock_settings.return_value.pr_update_changelog.push_changelog_changes = True
|
||||
mock_settings.return_value.pr_update_changelog.get.return_value = True
|
||||
mock_settings.return_value.config.publish_output = True
|
||||
mock_settings.return_value.config.git_provider = "gitlab"
|
||||
mock_retry.return_value = None
|
||||
|
||||
# Act
|
||||
await changelog_tool.run()
|
||||
|
||||
# Assert
|
||||
mock_git_provider.create_or_update_pr_file.assert_called_once()
|
||||
call_args = mock_git_provider.create_or_update_pr_file.call_args
|
||||
assert call_args[1]['file_path'] == 'CHANGELOG.md'
|
||||
assert call_args[1]['branch'] == 'feature-branch'
|
||||
|
||||
def test_push_changelog_update(self, changelog_tool, mock_git_provider):
|
||||
"""Test the push changelog update functionality."""
|
||||
# Arrange
|
||||
mock_git_provider.create_or_update_pr_file = MagicMock()
|
||||
mock_git_provider.get_pr_branch.return_value = "feature-branch"
|
||||
new_content = "# Updated changelog content"
|
||||
answer = "Changes made"
|
||||
|
||||
with patch('pr_agent.tools.pr_update_changelog.get_settings') as mock_settings, \
|
||||
patch('pr_agent.tools.pr_update_changelog.sleep'):
|
||||
|
||||
mock_settings.return_value.pr_update_changelog.get.return_value = True
|
||||
|
||||
# Act
|
||||
changelog_tool._push_changelog_update(new_content, answer)
|
||||
|
||||
# Assert
|
||||
mock_git_provider.create_or_update_pr_file.assert_called_once_with(
|
||||
file_path="CHANGELOG.md",
|
||||
branch="feature-branch",
|
||||
contents=new_content,
|
||||
message="[skip ci] Update CHANGELOG.md"
|
||||
)
|
||||
|
||||
def test_gitlab_provider_method_detection(self, changelog_tool, mock_git_provider):
|
||||
"""Test that the tool correctly detects GitLab provider method availability."""
|
||||
# Arrange
|
||||
mock_git_provider.create_or_update_pr_file = MagicMock()
|
||||
|
||||
# Act & Assert
|
||||
assert hasattr(mock_git_provider, "create_or_update_pr_file")
|
||||
|
||||
@pytest.mark.parametrize("existing_content,new_entry,expected_order", [
|
||||
(
|
||||
"# Changelog\n\n## v1.0.0\n- Old feature",
|
||||
"## v1.1.0\n- New feature",
|
||||
["v1.1.0", "v1.0.0"]
|
||||
),
|
||||
(
|
||||
"",
|
||||
"## v1.0.0\n- Initial release",
|
||||
["v1.0.0"]
|
||||
),
|
||||
(
|
||||
"Some existing content",
|
||||
"## v1.0.0\n- New entry",
|
||||
["v1.0.0", "Some existing content"]
|
||||
),
|
||||
])
|
||||
def test_changelog_order_preservation(self, changelog_tool, existing_content, new_entry, expected_order):
|
||||
"""Test that changelog entries are properly ordered (newest first)."""
|
||||
# Arrange
|
||||
changelog_tool.prediction = new_entry
|
||||
changelog_tool.changelog_file = existing_content
|
||||
changelog_tool.commit_changelog = True
|
||||
|
||||
# Act
|
||||
new_content, _ = changelog_tool._prepare_changelog_update()
|
||||
|
||||
# Assert
|
||||
for i, expected in enumerate(expected_order[:-1]):
|
||||
current_pos = new_content.find(expected)
|
||||
next_pos = new_content.find(expected_order[i + 1])
|
||||
assert current_pos < next_pos, f"Expected {expected} to come before {expected_order[i + 1]}"
|
||||
69
tests/unittest/test_secret_provider_factory.py
Normal file
69
tests/unittest/test_secret_provider_factory.py
Normal file
@ -0,0 +1,69 @@
|
||||
import pytest
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from pr_agent.secret_providers import get_secret_provider
|
||||
|
||||
|
||||
class TestSecretProviderFactory:
|
||||
|
||||
def test_get_secret_provider_none_when_not_configured(self):
|
||||
with patch('pr_agent.secret_providers.get_settings') as mock_get_settings:
|
||||
settings = MagicMock()
|
||||
settings.get.return_value = None
|
||||
mock_get_settings.return_value = settings
|
||||
|
||||
result = get_secret_provider()
|
||||
assert result is None
|
||||
|
||||
def test_get_secret_provider_google_cloud_storage(self):
|
||||
with patch('pr_agent.secret_providers.get_settings') as mock_get_settings:
|
||||
settings = MagicMock()
|
||||
settings.get.return_value = "google_cloud_storage"
|
||||
settings.config.secret_provider = "google_cloud_storage"
|
||||
mock_get_settings.return_value = settings
|
||||
|
||||
with patch('pr_agent.secret_providers.google_cloud_storage_secret_provider.GoogleCloudStorageSecretProvider') as MockProvider:
|
||||
mock_instance = MagicMock()
|
||||
MockProvider.return_value = mock_instance
|
||||
|
||||
result = get_secret_provider()
|
||||
assert result is mock_instance
|
||||
MockProvider.assert_called_once()
|
||||
|
||||
def test_get_secret_provider_aws_secrets_manager(self):
|
||||
with patch('pr_agent.secret_providers.get_settings') as mock_get_settings:
|
||||
settings = MagicMock()
|
||||
settings.get.return_value = "aws_secrets_manager"
|
||||
settings.config.secret_provider = "aws_secrets_manager"
|
||||
mock_get_settings.return_value = settings
|
||||
|
||||
with patch('pr_agent.secret_providers.aws_secrets_manager_provider.AWSSecretsManagerProvider') as MockProvider:
|
||||
mock_instance = MagicMock()
|
||||
MockProvider.return_value = mock_instance
|
||||
|
||||
result = get_secret_provider()
|
||||
assert result is mock_instance
|
||||
MockProvider.assert_called_once()
|
||||
|
||||
def test_get_secret_provider_unknown_provider(self):
|
||||
with patch('pr_agent.secret_providers.get_settings') as mock_get_settings:
|
||||
settings = MagicMock()
|
||||
settings.get.return_value = "unknown_provider"
|
||||
settings.config.secret_provider = "unknown_provider"
|
||||
mock_get_settings.return_value = settings
|
||||
|
||||
with pytest.raises(ValueError, match="Unknown SECRET_PROVIDER"):
|
||||
get_secret_provider()
|
||||
|
||||
def test_get_secret_provider_initialization_error(self):
|
||||
with patch('pr_agent.secret_providers.get_settings') as mock_get_settings:
|
||||
settings = MagicMock()
|
||||
settings.get.return_value = "aws_secrets_manager"
|
||||
settings.config.secret_provider = "aws_secrets_manager"
|
||||
mock_get_settings.return_value = settings
|
||||
|
||||
with patch('pr_agent.secret_providers.aws_secrets_manager_provider.AWSSecretsManagerProvider') as MockProvider:
|
||||
MockProvider.side_effect = Exception("Initialization failed")
|
||||
|
||||
with pytest.raises(ValueError, match="Failed to initialize aws_secrets_manager secret provider"):
|
||||
get_secret_provider()
|
||||
Reference in New Issue
Block a user