Compare commits

..

1 Commits

83 changed files with 649 additions and 2554 deletions

214
README.md
View File

@ -29,62 +29,24 @@ PR-Agent aims to help efficiently review and handle pull requests, by providing
## Table of Contents
- [Getting Started](#getting-started)
- [News and Updates](#news-and-updates)
- [Why Use PR-Agent?](#why-use-pr-agent)
- [Features](#features)
- [See It in Action](#see-it-in-action)
- [Try It Now](#try-it-now)
- [Qodo Merge 💎](#qodo-merge-)
- [How It Works](#how-it-works)
- [Data Privacy](#data-privacy)
- [Overview](#overview)
- [Example results](#example-results)
- [Try it now](#try-it-now)
- [Qodo Merge](https://qodo-merge-docs.qodo.ai/overview/pr_agent_pro/)
- [How it works](#how-it-works)
- [Why use PR-Agent?](#why-use-pr-agent)
- [Data privacy](#data-privacy)
- [Contributing](#contributing)
- [Links](#links)
## Getting Started
### Try it Instantly
Test PR-Agent on any public GitHub repository by commenting `@CodiumAI-Agent /improve`
### GitHub Action
Add automated PR reviews to your repository with a simple workflow file using [GitHub Action setup guide](https://qodo-merge-docs.qodo.ai/installation/github/#run-as-a-github-action)
#### Other Platforms
- [GitLab webhook setup](https://qodo-merge-docs.qodo.ai/installation/gitlab/)
- [BitBucket app installation](https://qodo-merge-docs.qodo.ai/installation/bitbucket/)
- [Azure DevOps setup](https://qodo-merge-docs.qodo.ai/installation/azure/)
### CLI Usage
Run PR-Agent locally on your repository via command line: [Local CLI setup guide](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#local-repo-cli)
### Qodo Merge as post-commit in your local IDE
See [here](https://github.com/qodo-ai/agents/tree/main/agents/qodo-merge-post-commit)
### Discover Qodo Merge 💎
Zero-setup hosted solution with advanced features and priority support
- **[FREE for Open Source](https://github.com/marketplace/qodo-merge-pro-for-open-source)**: Full features, zero cost for public repos
- [Intro and Installation guide](https://qodo-merge-docs.qodo.ai/installation/qodo_merge/)
- [Plans & Pricing](https://www.qodo.ai/pricing/)
### Qodo Merge as a Post-commit in Your Local IDE
You can receive automatic feedback from Qodo Merge on your local IDE after each [commit](https://github.com/qodo-ai/agents/tree/main/agents/qodo-merge-post-commit)
## News and Updates
## Jul 1, 2025
You can now receive automatic feedback from Qodo Merge in your local IDE after each commit. Read more about it [here](https://github.com/qodo-ai/agents/tree/main/agents/qodo-merge-post-commit).
## Jun 21, 2025
v0.30 was [released](https://github.com/qodo-ai/pr-agent/releases)
## Jun 3, 2025
Qodo Merge now offers a simplified free tier 💎.
Organizations can use Qodo Merge at no cost, with a [monthly limit](https://qodo-merge-docs.qodo.ai/installation/qodo_merge/#cloud-users) of 75 PR reviews per organization.
## May 17, 2025
- v0.29 was [released](https://github.com/qodo-ai/pr-agent/releases)
- `Qodo Merge Pull Request Benchmark` was [released](https://qodo-merge-docs.qodo.ai/pr_benchmark/). This benchmark evaluates and compares the performance of LLMs in analyzing pull request code.
- `Recent Updates and Future Roadmap` page was added to the [Qodo Merge Docs](https://qodo-merge-docs.qodo.ai/recent_updates/)
## Apr 30, 2025
@ -102,75 +64,90 @@ New tool for Qodo Merge 💎 - `/scan_repo_discussions`.
Read more about it [here](https://qodo-merge-docs.qodo.ai/tools/scan_repo_discussions/).
## Why Use PR-Agent?
A reasonable question that can be asked is: `"Why use PR-Agent? What makes it stand out from existing tools?"`
Here are some advantages of PR-Agent:
- We emphasize **real-life practical usage**. Each tool (review, improve, ask, ...) has a single LLM call, no more. We feel that this is critical for realistic team usage - obtaining an answer quickly (~30 seconds) and affordably.
- Our [PR Compression strategy](https://qodo-merge-docs.qodo.ai/core-abilities/#pr-compression-strategy) is a core ability that enables to effectively tackle both short and long PRs.
- Our JSON prompting strategy enables us to have **modular, customizable tools**. For example, the '/review' tool categories can be controlled via the [configuration](pr_agent/settings/configuration.toml) file. Adding additional categories is easy and accessible.
- We support **multiple git providers** (GitHub, GitLab, BitBucket), **multiple ways** to use the tool (CLI, GitHub Action, GitHub App, Docker, ...), and **multiple models** (GPT, Claude, Deepseek, ...)
## Features
## Overview
<div style="text-align:left;">
PR-Agent and Qodo Merge offer comprehensive pull request functionalities integrated with various git providers:
Supported commands per platform:
| | | GitHub | GitLab | Bitbucket | Azure DevOps | Gitea |
|---------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------|:------:|:------:|:---------:|:------------:|:-----:|
| [TOOLS](https://qodo-merge-docs.qodo.ai/tools/) | [Describe](https://qodo-merge-docs.qodo.ai/tools/describe/) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [Review](https://qodo-merge-docs.qodo.ai/tools/review/) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [Improve](https://qodo-merge-docs.qodo.ai/tools/improve/) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [Ask](https://qodo-merge-docs.qodo.ai/tools/ask/) | ✅ | ✅ | ✅ | ✅ | |
| | ⮑ [Ask on code lines](https://qodo-merge-docs.qodo.ai/tools/ask/#ask-lines) | ✅ | ✅ | | | |
| | [Help Docs](https://qodo-merge-docs.qodo.ai/tools/help_docs/?h=auto#auto-approval) | ✅ | ✅ | ✅ | | |
| | [Update CHANGELOG](https://qodo-merge-docs.qodo.ai/tools/update_changelog/) | ✅ | ✅ | ✅ | | |
| | [Add Documentation](https://qodo-merge-docs.qodo.ai/tools/documentation/) 💎 | ✅ | ✅ | | | |
| | [Analyze](https://qodo-merge-docs.qodo.ai/tools/analyze/) 💎 | ✅ | ✅ | | | |
| | [Auto-Approve](https://qodo-merge-docs.qodo.ai/tools/improve/?h=auto#auto-approval) 💎 | ✅ | | | | |
| | [CI Feedback](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) 💎 | ✅ | | | | |
| | [Custom Prompt](https://qodo-merge-docs.qodo.ai/tools/custom_prompt/) 💎 | ✅ | | | | |
| | [Generate Custom Labels](https://qodo-merge-docs.qodo.ai/tools/custom_labels/) 💎 | ✅ | ✅ | | | |
| | [Generate Tests](https://qodo-merge-docs.qodo.ai/tools/test/) 💎 | ✅ | ✅ | | | |
| | [Implement](https://qodo-merge-docs.qodo.ai/tools/implement/) 💎 | ✅ | ✅ | | | |
| | [Scan Repo Discussions](https://qodo-merge-docs.qodo.ai/tools/scan_repo_discussions/) 💎 | ✅ | | | | |
| | [Similar Code](https://qodo-merge-docs.qodo.ai/tools/similar_code/) 💎 | ✅ | | | | |
| | [Ticket Context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) 💎 | ✅ | ✅ | | | |
| | [Utilizing Best Practices](https://qodo-merge-docs.qodo.ai/tools/improve/#best-practices) 💎 | ✅ | ✅ | ✅ | | |
| | [PR Chat](https://qodo-merge-docs.qodo.ai/chrome-extension/features/#pr-chat) 💎 | ✅ | | | | |
| | [Suggestion Tracking](https://qodo-merge-docs.qodo.ai/tools/improve/#suggestion-tracking) 💎 | ✅ | ✅ | | | |
| | | | | | | |
| [USAGE](https://qodo-merge-docs.qodo.ai/usage-guide/) | [CLI](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#local-repo-cli) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [App / webhook](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#github-app) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [Tagging bot](https://github.com/Codium-ai/pr-agent#try-it-now) | ✅ | | | | |
| | [Actions](https://qodo-merge-docs.qodo.ai/installation/github/#run-as-a-github-action) | ✅ | ✅ | ✅ | ✅ | |
| | | | | | | |
| [CORE](https://qodo-merge-docs.qodo.ai/core-abilities/) | [Adaptive and token-aware file patch fitting](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/) | ✅ | ✅ | ✅ | ✅ | |
| | [Auto Best Practices 💎](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/) | ✅ | | | | |
| | [Chat on code suggestions](https://qodo-merge-docs.qodo.ai/core-abilities/chat_on_code_suggestions/) | ✅ | ✅ | | | |
| | [Code Validation 💎](https://qodo-merge-docs.qodo.ai/core-abilities/code_validation/) | ✅ | ✅ | ✅ | ✅ | |
| | [Dynamic context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic_context/) | ✅ | ✅ | ✅ | ✅ | |
| | [Fetching ticket context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) | ✅ | ✅ | ✅ | | |
| | [Global and wiki configurations](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/) 💎 | ✅ | ✅ | | | |
| | [Impact Evaluation](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) 💎 | ✅ | ✅ | | | |
| | [Incremental Update](https://qodo-merge-docs.qodo.ai/core-abilities/incremental_update/) | ✅ | | | | |
| | [Interactivity](https://qodo-merge-docs.qodo.ai/core-abilities/interactivity/) | ✅ | ✅ | | | |
| | [Local and global metadata](https://qodo-merge-docs.qodo.ai/core-abilities/metadata/) | ✅ | ✅ | ✅ | ✅ | |
| | [Multiple models support](https://qodo-merge-docs.qodo.ai/usage-guide/changing_a_model/) | ✅ | | | | |
| | [PR compression](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/) | ✅ | ✅ | ✅ | ✅ | |
| | [PR interactive actions](https://www.qodo.ai/images/pr_agent/pr-actions.mp4) 💎 | ✅ | ✅ | | | |
| | [RAG context enrichment](https://qodo-merge-docs.qodo.ai/core-abilities/rag_context_enrichment/) | ✅ | | ✅ | | |
| | [Self reflection](https://qodo-merge-docs.qodo.ai/core-abilities/self_reflection/) | ✅ | ✅ | ✅ | ✅ | |
| | [Static code analysis](https://qodo-merge-docs.qodo.ai/core-abilities/static_code_analysis/) 💎 | ✅ | ✅ | | | |
| | | GitHub | GitLab | Bitbucket | Azure DevOps | Gitea |
| ----- |---------------------------------------------------------------------------------------------------------|:------:|:------:|:---------:|:------------:|:-----:|
| TOOLS | [Review](https://qodo-merge-docs.qodo.ai/tools/review/) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [Describe](https://qodo-merge-docs.qodo.ai/tools/describe/) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [Improve](https://qodo-merge-docs.qodo.ai/tools/improve/) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [Ask](https://qodo-merge-docs.qodo.ai/tools/ask/) | ✅ | ✅ | ✅ | ✅ | |
| | ⮑ [Ask on code lines](https://qodo-merge-docs.qodo.ai/tools/ask/#ask-lines) | ✅ | ✅ | | | |
| | [Update CHANGELOG](https://qodo-merge-docs.qodo.ai/tools/update_changelog/) | ✅ | ✅ | ✅ | | |
| | [Help Docs](https://qodo-merge-docs.qodo.ai/tools/help_docs/?h=auto#auto-approval) | ✅ | ✅ | ✅ | | |
| | [Ticket Context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) 💎 | ✅ | ✅ | | | |
| | [Utilizing Best Practices](https://qodo-merge-docs.qodo.ai/tools/improve/#best-practices) 💎 | ✅ | ✅ | | | |
| | [PR Chat](https://qodo-merge-docs.qodo.ai/chrome-extension/features/#pr-chat) 💎 | ✅ | | | | |
| | [Suggestion Tracking](https://qodo-merge-docs.qodo.ai/tools/improve/#suggestion-tracking) 💎 | ✅ | | | | |
| | [CI Feedback](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) 💎 | ✅ | | | | |
| | [PR Documentation](https://qodo-merge-docs.qodo.ai/tools/documentation/) 💎 | ✅ | ✅ | | | |
| | [Custom Labels](https://qodo-merge-docs.qodo.ai/tools/custom_labels/) 💎 | ✅ | ✅ | | | |
| | [Analyze](https://qodo-merge-docs.qodo.ai/tools/analyze/) 💎 | ✅ | ✅ | | | |
| | [Similar Code](https://qodo-merge-docs.qodo.ai/tools/similar_code/) 💎 | ✅ | | | | |
| | [Custom Prompt](https://qodo-merge-docs.qodo.ai/tools/custom_prompt/) 💎 | ✅ | | | | |
| | [Test](https://qodo-merge-docs.qodo.ai/tools/test/) 💎 | ✅ | ✅ | | | |
| | [Implement](https://qodo-merge-docs.qodo.ai/tools/implement/) 💎 | ✅ | ✅ | ✅ | | |
| | [Scan Repo Discussions](https://qodo-merge-docs.qodo.ai/tools/scan_repo_discussions/) 💎 | ✅ | | | | |
| | [Auto-Approve](https://qodo-merge-docs.qodo.ai/tools/improve/?h=auto#auto-approval) 💎 | ✅ | ✅ | | | |
| | | | | | | |
| USAGE | [CLI](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#local-repo-cli) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [App / webhook](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#github-app) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [Tagging bot](https://github.com/Codium-ai/pr-agent#try-it-now) | ✅ | | | | |
| | [Actions](https://qodo-merge-docs.qodo.ai/installation/github/#run-as-a-github-action) | ✅ | ✅ | ✅ | ✅ | |
| | | | | | | |
| CORE | [PR compression](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/) | ✅ | ✅ | ✅ | ✅ | |
| | Adaptive and token-aware file patch fitting | ✅ | ✅ | | | |
| | [Multiple models support](https://qodo-merge-docs.qodo.ai/usage-guide/changing_a_model/) | ✅ | ✅ | | | |
| | [Local and global metadata](https://qodo-merge-docs.qodo.ai/core-abilities/metadata/) | ✅ | ✅ | ✅ | ✅ | |
| | [Dynamic context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic_context/) | ✅ | ✅ | ✅ | ✅ | |
| | [Self reflection](https://qodo-merge-docs.qodo.ai/core-abilities/self_reflection/) | ✅ | | ✅ | | |
| | [Static code analysis](https://qodo-merge-docs.qodo.ai/core-abilities/static_code_analysis/) 💎 | ✅ | ✅ | | | |
| | [Global and wiki configurations](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/) 💎 | ✅ | ✅ | | | |
| | [PR interactive actions](https://www.qodo.ai/images/pr_agent/pr-actions.mp4) 💎 | ✅ | | | | |
| | [Impact Evaluation](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) 💎 | ✅ | ✅ | | | |
| | [Code Validation 💎](https://qodo-merge-docs.qodo.ai/core-abilities/code_validation/) | ✅ | ✅ | ✅ | ✅ | |
| | [Auto Best Practices 💎](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/) | ✅ | | | | |
- 💎 means this feature is available only in [Qodo Merge](https://www.qodo.ai/pricing/)
[//]: # (- Support for additional git providers is described in [here]&#40;./docs/Full_environments.md&#41;)
___
## See It in Action
**Auto Description ([`/describe`](https://qodo-merge-docs.qodo.ai/tools/describe/))**: Automatically generating PR description - title, type, summary, code walkthrough and labels.
\
**Auto Review ([`/review`](https://qodo-merge-docs.qodo.ai/tools/review/))**: Adjustable feedback about the PR, possible issues, security concerns, review effort and more.
\
**Code Suggestions ([`/improve`](https://qodo-merge-docs.qodo.ai/tools/improve/))**: Code suggestions for improving the PR.
\
**Question Answering ([`/ask ...`](https://qodo-merge-docs.qodo.ai/tools/ask/))**: Answering free-text questions about the PR.
\
**Update Changelog ([`/update_changelog`](https://qodo-merge-docs.qodo.ai/tools/update_changelog/))**: Automatically updating the CHANGELOG.md file with the PR changes.
\
**Help Docs ([`/help_docs`](https://qodo-merge-docs.qodo.ai/tools/help_docs/))**: Answers a question on any repository by utilizing given documentation.
\
**Add Documentation 💎 ([`/add_docs`](https://qodo-merge-docs.qodo.ai/tools/documentation/))**: Generates documentation to methods/functions/classes that changed in the PR.
\
**Generate Custom Labels 💎 ([`/generate_labels`](https://qodo-merge-docs.qodo.ai/tools/custom_labels/))**: Generates custom labels for the PR, based on specific guidelines defined by the user.
\
**Analyze 💎 ([`/analyze`](https://qodo-merge-docs.qodo.ai/tools/analyze/))**: Identify code components that changed in the PR, and enables to interactively generate tests, docs, and code suggestions for each component.
\
**Test 💎 ([`/test`](https://qodo-merge-docs.qodo.ai/tools/test/))**: Generate tests for a selected component, based on the PR code changes.
\
**Custom Prompt 💎 ([`/custom_prompt`](https://qodo-merge-docs.qodo.ai/tools/custom_prompt/))**: Automatically generates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
\
**Generate Tests 💎 ([`/test component_name`](https://qodo-merge-docs.qodo.ai/tools/test/))**: Generates unit tests for a selected component, based on the PR code changes.
\
**CI Feedback 💎 ([`/checks ci_job`](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/))**: Automatically generates feedback and analysis for a failed CI job.
\
**Similar Code 💎 ([`/find_similar_component`](https://qodo-merge-docs.qodo.ai/tools/similar_code/))**: Retrieves the most similar code components from inside the organization's codebase, or from open-source code.
\
**Implement 💎 ([`/implement`](https://qodo-merge-docs.qodo.ai/tools/implement/))**: Generates implementation code from review suggestions.
___
## Example results
</div>
<h4><a href="https://github.com/Codium-ai/pr-agent/pull/530">/describe</a></h4>
@ -205,7 +182,7 @@ ___
</div>
<hr>
## Try It Now
## Try it now
Try the Claude Sonnet powered PR-Agent instantly on _your public GitHub repository_. Just mention `@CodiumAI-Agent` and add the desired command in any PR comment. The agent will generate a response based on your command.
For example, add a comment to any pull request with the following text:
@ -231,7 +208,7 @@ It does not have 'edit' access to your repo, for example, so it cannot update th
4. **Extra features** - In addition to the benefits listed above, Qodo Merge will emphasize more customization, and the usage of static code analysis, in addition to LLM logic, to improve results.
See [here](https://qodo-merge-docs.qodo.ai/overview/pr_agent_pro/) for a list of features available in Qodo Merge.
## How It Works
## How it works
The following diagram illustrates PR-Agent tools and their flow:
@ -239,7 +216,18 @@ The following diagram illustrates PR-Agent tools and their flow:
Check out the [PR Compression strategy](https://qodo-merge-docs.qodo.ai/core-abilities/#pr-compression-strategy) page for more details on how we convert a code diff to a manageable LLM prompt
## Data Privacy
## Why use PR-Agent?
A reasonable question that can be asked is: `"Why use PR-Agent? What makes it stand out from existing tools?"`
Here are some advantages of PR-Agent:
- We emphasize **real-life practical usage**. Each tool (review, improve, ask, ...) has a single LLM call, no more. We feel that this is critical for realistic team usage - obtaining an answer quickly (~30 seconds) and affordably.
- Our [PR Compression strategy](https://qodo-merge-docs.qodo.ai/core-abilities/#pr-compression-strategy) is a core ability that enables to effectively tackle both short and long PRs.
- Our JSON prompting strategy enables to have **modular, customizable tools**. For example, the '/review' tool categories can be controlled via the [configuration](pr_agent/settings/configuration.toml) file. Adding additional categories is easy and accessible.
- We support **multiple git providers** (GitHub, GitLab, BitBucket), **multiple ways** to use the tool (CLI, GitHub Action, GitHub App, Docker, ...), and **multiple models** (GPT, Claude, Deepseek, ...)
## Data privacy
### Self-hosted PR-Agent
@ -264,7 +252,7 @@ To contribute to the project, get started by reading our [Contributing Guide](ht
## Links
- Discord community: https://discord.com/invite/SgSxuQ65GF
- Discord community: https://discord.gg/kG35uSHDBc
- Qodo site: https://www.qodo.ai/
- Blog: https://www.qodo.ai/blog/
- Troubleshooting: https://www.qodo.ai/blog/technical-faq-and-troubleshooting/

View File

@ -1,4 +1,4 @@
FROM public.ecr.aws/lambda/python:3.12 AS base
FROM public.ecr.aws/lambda/python:3.12
RUN dnf update -y && \
dnf install -y gcc python3-devel git && \
@ -9,10 +9,4 @@ RUN pip install --no-cache-dir . && rm pyproject.toml
RUN pip install --no-cache-dir mangum==0.17.0
COPY pr_agent/ ${LAMBDA_TASK_ROOT}/pr_agent/
FROM base AS github_lambda
CMD ["pr_agent.servers.github_lambda_webhook.lambda_handler"]
FROM base AS gitlab_lambda
CMD ["pr_agent.servers.gitlab_lambda_webhook.lambda_handler"]
FROM github_lambda
CMD ["pr_agent.servers.serverless.serverless"]

View File

@ -19,6 +19,7 @@
</div>
<style>
Untitled
.search-section {
max-width: 800px;
margin: 0 auto;
@ -201,23 +202,7 @@ h1 {
<script>
window.addEventListener('load', function() {
function extractText(responseText) {
try {
console.log('responseText: ', responseText);
const results = JSON.parse(responseText);
const msg = results.message;
if (!msg || msg.trim() === '') {
return "No results found";
}
return msg;
} catch (error) {
console.error('Error parsing results:', error);
throw new Error("Failed parsing response message");
}
}
function displayResults(msg) {
function displayResults(responseText) {
const resultsContainer = document.getElementById('results');
const spinner = document.getElementById('spinner');
const searchContainer = document.querySelector('.search-container');
@ -229,6 +214,8 @@ window.addEventListener('load', function() {
searchContainer.scrollIntoView({ behavior: 'smooth', block: 'start' });
try {
const results = JSON.parse(responseText);
marked.setOptions({
breaks: true,
gfm: true,
@ -236,7 +223,7 @@ window.addEventListener('load', function() {
sanitize: false
});
const htmlContent = marked.parse(msg);
const htmlContent = marked.parse(results.message);
resultsContainer.className = 'markdown-content';
resultsContainer.innerHTML = htmlContent;
@ -255,7 +242,7 @@ window.addEventListener('load', function() {
}, 100);
} catch (error) {
console.error('Error parsing results:', error);
resultsContainer.innerHTML = '<div class="error-message">Cannot process results</div>';
resultsContainer.innerHTML = '<div class="error-message">Error processing results</div>';
}
}
@ -288,24 +275,24 @@ window.addEventListener('load', function() {
body: JSON.stringify(data)
};
//const API_ENDPOINT = 'http://0.0.0.0:3000/api/v1/docs_help';
// const API_ENDPOINT = 'http://0.0.0.0:3000/api/v1/docs_help';
const API_ENDPOINT = 'https://help.merge.qodo.ai/api/v1/docs_help';
const response = await fetch(API_ENDPOINT, options);
const responseText = await response.text();
const msg = extractText(responseText);
if (!response.ok) {
throw new Error(`An error (${response.status}) occurred during search: "${msg}"`);
throw new Error(`HTTP error! status: ${response.status}`);
}
displayResults(msg);
const responseText = await response.text();
displayResults(responseText);
} catch (error) {
spinner.style.display = 'none';
const errorDiv = document.createElement('div');
errorDiv.className = 'error-message';
errorDiv.textContent = error instanceof Error ? error.message : String(error);
resultsContainer.replaceChildren(errorDiv);
resultsContainer.innerHTML = `
<div class="error-message">
An error occurred while searching. Please try again later.
</div>
`;
}
}

View File

@ -4,7 +4,7 @@ With a single-click installation you will gain access to a context-aware chat on
The extension is powered by top code models like Claude 3.7 Sonnet and o4-mini. All the extension's features are free to use on public repositories.
For private repositories, you will need to install [Qodo Merge](https://github.com/apps/qodo-merge-pro){:target="_blank"} in addition to the extension.
For private repositories, you will need to install [Qodo Merge](https://github.com/apps/qodo-merge-pro){:target="_blank"} in addition to the extension (Quick GitHub app setup with a 14-day free trial. No credit card needed).
For a demonstration of how to install Qodo Merge and use it with the Chrome extension, please refer to the tutorial video at the provided [link](https://codium.ai/images/pr_agent/private_repos.mp4){:target="_blank"}.
<img src="https://codium.ai/images/pr_agent/PR-AgentChat.gif" width="768">

View File

@ -1,83 +0,0 @@
# Auto-approval 💎
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
Under specific conditions, Qodo Merge can auto-approve a PR when a manual comment is invoked, or when the PR meets certain criteria.
**To ensure safety, the auto-approval feature is disabled by default.**
To enable auto-approval features, you need to actively set one or both of the following options in a pre-defined _configuration file_:
```toml
[config]
enable_comment_approval = true # For approval via comments
enable_auto_approval = true # For criteria-based auto-approval
```
!!! note "Notes"
- These flags above cannot be set with a command line argument, only in the configuration file, committed to the repository.
- Enabling auto-approval must be a deliberate decision by the repository owner.
## **Approval by commenting**
To enable approval by commenting, set in the configuration file:
```toml
[config]
enable_comment_approval = true
```
After enabling, by commenting on a PR:
```
/review auto_approve
```
Qodo Merge will approve the PR and add a comment with the reason for the approval.
## **Auto-approval when the PR meets certain criteria**
To enable auto-approval based on specific criteria, first, you need to enable the top-level flag:
```toml
[config]
enable_auto_approval = true
```
There are two possible paths leading to this auto-approval - one via the `review` tool, and one via the `improve` tool. Each tool can independently trigger auto-approval.
### Auto-approval via the `review` tool
- **Review effort score criteria**
```toml
[config]
enable_auto_approval = true
auto_approve_for_low_review_effort = X # X is a number between 1 and 5
```
When the [review effort score](https://www.qodo.ai/images/pr_agent/review3.png) is lower than or equal to X, the PR will be auto-approved (unless ticket compliance is enabled and fails, see below).
- **Ticket compliance criteria**
```toml
[config]
enable_auto_approval = true
ensure_ticket_compliance = true # Default is false
```
If `ensure_ticket_compliance` is set to `true`, auto-approval for the `review` toll path will be disabled if no ticket is linked to the PR, or if the PR is not fully compliant with a linked ticket. This ensures that PRs are only auto-approved if their associated tickets are properly resolved.
You can also prevent auto-approval if the PR exceeds the ticket's scope (see [here](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/#configuration-options)).
### Auto-approval via the `improve` tool
PRs can be auto-approved when the `improve` tool doesn't find code suggestions.
To enable this feature, set the following in the configuration file:
```toml
[config]
enable_auto_approval = true
auto_approve_for_no_suggestions = true
```

View File

@ -1,8 +1,3 @@
# Code Validation 💎
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
## Introduction
The Git environment usually represents the final stage before code enters production. Hence, Detecting bugs and issues during the review process is critical.

View File

@ -1,8 +1,5 @@
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
## Overview
## Overview - PR Compression Strategy
There are two scenarios:

View File

@ -1,5 +1,4 @@
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
## TL;DR
Qodo Merge uses an **asymmetric and dynamic context strategy** to improve AI analysis of code changes in pull requests.
It provides more context before changes than after, and dynamically adjusts the context based on code structure (e.g., enclosing functions or classes).

View File

@ -39,40 +39,17 @@ By understanding the reasoning and intent behind modifications, the LLM can offe
Similarly to the `describe` tool, the `review` tool will use the ticket content to provide additional context for the code changes.
In addition, this feature will evaluate how well a Pull Request (PR) adheres to its original purpose/intent as defined by the associated ticket or issue mentioned in the PR description.
Each ticket will be assigned a label (Compliance/Alignment level), Indicates the degree to which the PR fulfills its original purpose:
- Fully Compliant
- Partially Compliant
- Not Compliant
- PR Code Verified
Each ticket will be assigned a label (Compliance/Alignment level), Indicates the degree to which the PR fulfills its original purpose, Options: Fully compliant, Partially compliant or Not compliant.
![Ticket Compliance](https://www.qodo.ai/images/pr_agent/ticket_compliance_review.png){width=768}
A `PR Code Verified` label indicates the PR code meets ticket requirements, but requires additional manual testing beyond the code scope. For example - validating UI display across different environments (Mac, Windows, mobile, etc.).
By default, the tool will automatically validate if the PR complies with the referenced ticket.
If you want to disable this feedback, add the following line to your configuration file:
#### Configuration options
-
By default, the tool will automatically validate if the PR complies with the referenced ticket.
If you want to disable this feedback, add the following line to your configuration file:
```toml
[pr_reviewer]
require_ticket_analysis_review=false
```
-
If you set:
```toml
[pr_reviewer]
check_pr_additional_content=true
```
(default: `false`)
the `review` tool will also validate that the PR code doesn't contain any additional content that is not related to the ticket. If it does, the PR will be labeled at best as `PR Code Verified`, and the `review` tool will provide a comment with the additional unrelated content found in the PR code.
```toml
[pr_reviewer]
require_ticket_analysis_review=false
```
## GitHub Issues Integration
@ -218,7 +195,7 @@ This following steps will help you check if the basic auth is working correctly,
2. run the following Python script (after replacing the placeholders with your actual values):
???- example "Script to validate basic auth"
??? example "Script to validate basic auth"
```python
from jira import JIRA
@ -274,7 +251,7 @@ This following steps will help you check if the token is working correctly, and
2. run the following Python script (after replacing the placeholders with your actual values):
??? example- "Script to validate PAT token"
??? example "Script to validate PAT token"
```python
from jira import JIRA
@ -309,90 +286,13 @@ This following steps will help you check if the token is working correctly, and
print(f"Error fetching JIRA ticket details: {e}")
```
### Multi-JIRA Server Configuration 💎
Qodo Merge supports connecting to multiple JIRA servers using different authentication methods.
=== "Email/Token (Basic Auth)"
Configure multiple servers using Email/Token authentication:
- `jira_servers`: List of JIRA server URLs
- `jira_api_token`: List of API tokens (for Cloud) or passwords (for Data Center)
- `jira_api_email`: List of emails (for Cloud) or usernames (for Data Center)
- `jira_base_url`: Default server for ticket IDs like `PROJ-123`, Each repository can configure (local config file) its own `jira_base_url` to choose which server to use by default.
**Example Configuration:**
```toml
[jira]
# Server URLs
jira_servers = ["https://company.atlassian.net", "https://datacenter.jira.com"]
# API tokens/passwords
jira_api_token = ["cloud_api_token_here", "datacenter_password"]
# Emails/usernames (both required)
jira_api_email = ["user@company.com", "datacenter_username"]
# Default server for ticket IDs
jira_base_url = "https://company.atlassian.net"
```
=== "PAT Auth"
Configure multiple servers using Personal Access Token authentication:
- `jira_servers`: List of JIRA server URLs
- `jira_api_token`: List of PAT tokens
- `jira_api_email`: Not needed (can be omitted or left empty)
- `jira_base_url`: Default server for ticket IDs like `PROJ-123`, Each repository can configure (local config file) its own `jira_base_url` to choose which server to use by default.
**Example Configuration:**
```toml
[jira]
# Server URLs
jira_servers = ["https://server1.jira.com", "https://server2.jira.com"]
# PAT tokens only
jira_api_token = ["pat_token_1", "pat_token_2"]
# Default server for ticket IDs
jira_base_url = "https://server1.jira.com"
```
**Mixed Authentication (Email/Token + PAT):**
```toml
[jira]
jira_servers = ["https://company.atlassian.net", "https://server.jira.com"]
jira_api_token = ["cloud_api_token", "server_pat_token"]
jira_api_email = ["user@company.com", ""] # Empty for PAT
```
=== "Jira Cloud App"
For Jira Cloud instances using App Authentication:
1. Install the Qodo Merge app on each JIRA Cloud instance you want to connect to
2. Set the default server for ticket ID resolution:
```toml
[jira]
jira_base_url = "https://primary-team.atlassian.net"
```
Full URLs (e.g., `https://other-team.atlassian.net/browse/TASK-456`) will automatically use the correct connected instance.
### How to link a PR to a Jira ticket
To integrate with Jira, you can link your PR to a ticket using either of these methods:
**Method 1: Description Reference:**
Include a ticket reference in your PR description, using either the complete URL format `https://<JIRA_ORG>.atlassian.net/browse/ISSUE-123` or the shortened ticket ID `ISSUE-123` (without prefix or suffix for the shortened ID).
Include a ticket reference in your PR description using either the complete URL format https://<JIRA_ORG>.atlassian.net/browse/ISSUE-123 or the shortened ticket ID ISSUE-123.
**Method 2: Branch Name Detection:**
@ -405,7 +305,6 @@ Name your branch with the ticket ID as a prefix (e.g., `ISSUE-123-feature-descri
[jira]
jira_base_url = "https://<JIRA_ORG>.atlassian.net"
```
Where `<JIRA_ORG>` is your Jira organization identifier (e.g., `mycompany` for `https://mycompany.atlassian.net`).
## Linear Integration 💎
@ -440,11 +339,11 @@ Include a ticket reference in your PR description using either:
Name your branch with the ticket ID as a prefix (e.g., `ABC-123-feature-description` or `feature/ABC-123/feature-description`).
!!! note "Linear Base URL"
For shortened ticket IDs or branch detection (method 2), you must configure the Linear base URL in your configuration file under the [linear] section:
For shortened ticket IDs or branch detection (method 2), you must configure the Linear base URL in your configuration file under the [linear] section:
```toml
[linear]
linear_base_url = "https://linear.app/[ORG_ID]"
```
```toml
[linear]
linear_base_url = "https://linear.app/[ORG_ID]"
```
Replace `[ORG_ID]` with your Linear organization identifier.
Replace `[ORG_ID]` with your Linear organization identifier.

View File

@ -1,6 +1,4 @@
# Impact Evaluation 💎
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
# Overview - Impact Evaluation 💎
Demonstrating the return on investment (ROI) of AI-powered initiatives is crucial for modern organizations.
To address this need, Qodo Merge has developed an AI impact measurement tools and metrics, providing advanced analytics to help businesses quantify the tangible benefits of AI adoption in their PR review process.

View File

@ -1,6 +1,6 @@
# Incremental Update 💎
`Supported Git Platforms: GitHub, GitLab (Both cloud & server. For server: Version 17 and above)`
`Supported Git Platforms: GitHub`
## Overview
The Incremental Update feature helps users focus on feedback for their newest changes, making large PRs more manageable.

View File

@ -2,11 +2,10 @@
Qodo Merge utilizes a variety of core abilities to provide a comprehensive and efficient code review experience. These abilities include:
- [Auto approval](https://qodo-merge-docs.qodo.ai/core-abilities/auto_approval/)
- [Auto best practices](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/)
- [Chat on code suggestions](https://qodo-merge-docs.qodo.ai/core-abilities/chat_on_code_suggestions/)
- [Chrome extension](https://qodo-merge-docs.qodo.ai/chrome-extension/)
- [Code validation](https://qodo-merge-docs.qodo.ai/core-abilities/code_validation/) <!-- - [Compression strategy](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/) -->
- [Code validation](https://qodo-merge-docs.qodo.ai/core-abilities/code_validation/)
- [Compression strategy](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/)
- [Dynamic context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic_context/)
- [Fetching ticket context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/)
- [Impact evaluation](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/)

View File

@ -1,4 +1,4 @@
# Interactivity 💎
# Interactivity
`Supported Git Platforms: GitHub, GitLab`

View File

@ -1,6 +1,4 @@
# Local and global metadata injection with multi-stage analysis
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
## Local and global metadata injection with multi-stage analysis
1\.
Qodo Merge initially retrieves for each PR the following data:

View File

@ -27,7 +27,7 @@ In order to enable the RAG feature, add the following lines to your configuratio
enable_rag=true
```
???+ example "RAG Arguments Options"
!!! example "RAG Arguments Options"
<table>
<tr>

View File

@ -1,4 +1,4 @@
`Supported Git Platforms: GitHub, GitLab, Bitbucket`
## TL;DR
Qodo Merge implements a **self-reflection** process where the AI model reflects, scores, and re-ranks its own suggestions, eliminating irrelevant or incorrect ones.
This approach improves the quality and relevance of suggestions, saving users time and enhancing their experience.

View File

@ -1,14 +1,11 @@
# Static Code Analysis 💎
` Supported Git Platforms: GitHub, GitLab, Bitbucket`
## Overview - Static Code Analysis 💎
By combining static code analysis with LLM capabilities, Qodo Merge can provide a comprehensive analysis of the PR code changes on a component level.
It scans the PR code changes, finds all the code components (methods, functions, classes) that changed, and enables to interactively generate tests, docs, code suggestions and similar code search for each component.
!!! note "Language that are currently supported:"
Python, Java, C++, JavaScript, TypeScript, C#, Go.
Python, Java, C++, JavaScript, TypeScript, C#.
## Capabilities

View File

@ -66,7 +66,7 @@ ___
___
??? note "Q: Can Qodo Merge review draft/offline PRs?"
#### Answer:<span style="display:none;">6</span>
#### Answer:<span style="display:none;">5</span>
Yes. While Qodo Merge won't automatically review draft PRs, you can still get feedback by manually requesting it through [online commenting](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#online-usage).
@ -74,7 +74,7 @@ ___
___
??? note "Q: Can the 'Review effort' feedback be calibrated or customized?"
#### Answer:<span style="display:none;">7</span>
#### Answer:<span style="display:none;">5</span>
Yes, you can customize review effort estimates using the `extra_instructions` configuration option (see [documentation](https://qodo-merge-docs.qodo.ai/tools/review/#configuration-options)).

View File

@ -24,54 +24,50 @@ To search the documentation site using natural language:
## Features
PR-Agent and Qodo Merge offer comprehensive pull request functionalities integrated with various git providers:
PR-Agent and Qodo Merge offers extensive pull request functionalities across various git providers:
| | | GitHub | GitLab | Bitbucket | Azure DevOps | Gitea |
| ----- |---------------------------------------------------------------------------------------------------------------------|:------:|:------:|:---------:|:------------:|:-----:|
| [TOOLS](https://qodo-merge-docs.qodo.ai/tools/) | [Describe](https://qodo-merge-docs.qodo.ai/tools/describe/) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [Review](https://qodo-merge-docs.qodo.ai/tools/review/) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [Improve](https://qodo-merge-docs.qodo.ai/tools/improve/) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [Ask](https://qodo-merge-docs.qodo.ai/tools/ask/) | ✅ | ✅ | ✅ | ✅ | |
| | ⮑ [Ask on code lines](https://qodo-merge-docs.qodo.ai/tools/ask/#ask-lines) | ✅ | ✅ | | | |
| | [Help Docs](https://qodo-merge-docs.qodo.ai/tools/help_docs/?h=auto#auto-approval) | ✅ | ✅ | ✅ | | |
| | [Update CHANGELOG](https://qodo-merge-docs.qodo.ai/tools/update_changelog/) | ✅ | ✅ | ✅ | ✅ | |
| | [Add Documentation](https://qodo-merge-docs.qodo.ai/tools/documentation/) 💎 | ✅ | ✅ | | | |
| | [Analyze](https://qodo-merge-docs.qodo.ai/tools/analyze/) 💎 | ✅ | ✅ | | | |
| | [Auto-Approve](https://qodo-merge-docs.qodo.ai/tools/improve/?h=auto#auto-approval) 💎 | ✅ | ✅ | ✅ | | |
| | [CI Feedback](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) 💎 | ✅ | | | | |
| | [Custom Prompt](https://qodo-merge-docs.qodo.ai/tools/custom_prompt/) 💎 | ✅ | ✅ | ✅ | | |
| | [Generate Custom Labels](https://qodo-merge-docs.qodo.ai/tools/custom_labels/) 💎 | ✅ | ✅ | | | |
| | [Generate Tests](https://qodo-merge-docs.qodo.ai/tools/test/) 💎 | ✅ | ✅ | | | |
| | [Implement](https://qodo-merge-docs.qodo.ai/tools/implement/) 💎 | ✅ | ✅ | | | |
| | [Scan Repo Discussions](https://qodo-merge-docs.qodo.ai/tools/scan_repo_discussions/) 💎 | ✅ | | | | |
| | [Similar Code](https://qodo-merge-docs.qodo.ai/tools/similar_code/) 💎 | ✅ | | | | |
| | [Ticket Context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) 💎 | | ✅ | | | |
| | [Utilizing Best Practices](https://qodo-merge-docs.qodo.ai/tools/improve/#best-practices) 💎 | ✅ | ✅ | ✅ | | |
| | [PR Chat](https://qodo-merge-docs.qodo.ai/chrome-extension/features/#pr-chat) 💎 | ✅ | | | | |
| | [Suggestion Tracking](https://qodo-merge-docs.qodo.ai/tools/improve/#suggestion-tracking) 💎 | ✅ | ✅ | | | |
| | | | | | | |
| [USAGE](https://qodo-merge-docs.qodo.ai/usage-guide/) | [CLI](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#local-repo-cli) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [App / webhook](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#github-app) | ✅ | ✅ | ✅ | ✅ | ✅ |
| | [Tagging bot](https://github.com/Codium-ai/pr-agent#try-it-now) | ✅ | | | | |
| | [Actions](https://qodo-merge-docs.qodo.ai/installation/github/#run-as-a-github-action) | ✅ | ✅ | ✅ | ✅ | |
| | | | | | | |
| [CORE](https://qodo-merge-docs.qodo.ai/core-abilities/) | [Adaptive and token-aware file patch fitting](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/) | ✅ | ✅ | ✅ | ✅ | |
| | [Auto Best Practices 💎](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/) | | | | | |
| | [Chat on code suggestions](https://qodo-merge-docs.qodo.ai/core-abilities/chat_on_code_suggestions/) | ✅ | ✅ | | | |
| | [Code Validation 💎](https://qodo-merge-docs.qodo.ai/core-abilities/code_validation/) | ✅ | ✅ | ✅ | ✅ | |
| | [Dynamic context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic_context/) | ✅ | ✅ | ✅ | ✅ | |
| | [Fetching ticket context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) | | ✅ | ✅ | | |
| | [Global and wiki configurations](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/) 💎 | ✅ | ✅ | | | |
| | [Impact Evaluation](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) 💎 | ✅ | ✅ | | | |
| | [Incremental Update 💎](https://qodo-merge-docs.qodo.ai/core-abilities/incremental_update/) | ✅ | | | | |
| | [Interactivity](https://qodo-merge-docs.qodo.ai/core-abilities/interactivity/) | ✅ | ✅ | | | |
| | [Local and global metadata](https://qodo-merge-docs.qodo.ai/core-abilities/metadata/) | ✅ | ✅ | ✅ | ✅ | |
| | [Multiple models support](https://qodo-merge-docs.qodo.ai/usage-guide/changing_a_model/) | ✅ | | | ✅ | |
| | [PR compression](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/) | ✅ | | | ✅ | |
| | [PR interactive actions](https://www.qodo.ai/images/pr_agent/pr-actions.mp4) 💎 | ✅ | ✅ | | | |
| | [RAG context enrichment](https://qodo-merge-docs.qodo.ai/core-abilities/rag_context_enrichment/) | ✅ | | ✅ | | |
| | [Self reflection](https://qodo-merge-docs.qodo.ai/core-abilities/self_reflection/) | ✅ | ✅ | ✅ | ✅ | |
| | [Static code analysis](https://qodo-merge-docs.qodo.ai/core-abilities/static_code_analysis/) 💎 | ✅ | ✅ | | | |
| | | GitHub | GitLab | Bitbucket | Azure DevOps |
| ----- | ------------------------------------------------------------------------------------------------------- |:------:|:------:|:---------:|:------------:|
| TOOLS | [Review](https://qodo-merge-docs.qodo.ai/tools/review/) | ✅ | ✅ | ✅ | ✅ |
| | [Describe](https://qodo-merge-docs.qodo.ai/tools/describe/) | ✅ | ✅ | ✅ | ✅ |
| | [Improve](https://qodo-merge-docs.qodo.ai/tools/improve/) | ✅ | ✅ | ✅ | ✅ |
| | [Ask](https://qodo-merge-docs.qodo.ai/tools/ask/) | ✅ | ✅ | ✅ | ✅ |
| | ⮑ [Ask on code lines](https://qodo-merge-docs.qodo.ai/tools/ask/#ask-lines) | ✅ | ✅ | | |
| | [Update CHANGELOG](https://qodo-merge-docs.qodo.ai/tools/update_changelog/) | ✅ | ✅ | ✅ | |
| | [Help Docs](https://qodo-merge-docs.qodo.ai/tools/help_docs/?h=auto#auto-approval) | ✅ | ✅ | ✅ | |
| | [Ticket Context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) 💎 | ✅ | ✅ | | |
| | [Utilizing Best Practices](https://qodo-merge-docs.qodo.ai/tools/improve/#best-practices) 💎 | ✅ | ✅ | | |
| | [PR Chat](https://qodo-merge-docs.qodo.ai/chrome-extension/features/#pr-chat) 💎 | ✅ | | | |
| | [Suggestion Tracking](https://qodo-merge-docs.qodo.ai/tools/improve/#suggestion-tracking) 💎 | ✅ | | | |
| | [CI Feedback](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) 💎 | ✅ | | | |
| | [PR Documentation](https://qodo-merge-docs.qodo.ai/tools/documentation/) 💎 | ✅ | ✅ | | |
| | [Custom Labels](https://qodo-merge-docs.qodo.ai/tools/custom_labels/) 💎 | ✅ | ✅ | | |
| | [Analyze](https://qodo-merge-docs.qodo.ai/tools/analyze/) 💎 | ✅ | ✅ | | |
| | [Similar Code](https://qodo-merge-docs.qodo.ai/tools/similar_code/) 💎 | ✅ | | | |
| | [Custom Prompt](https://qodo-merge-docs.qodo.ai/tools/custom_prompt/) 💎 | ✅ | ✅ | | |
| | [Test](https://qodo-merge-docs.qodo.ai/tools/test/) 💎 | ✅ | ✅ | | |
| | [Implement](https://qodo-merge-docs.qodo.ai/tools/implement/) 💎 | ✅ | ✅ | ✅ | |
| | [Scan Repo Discussions](https://qodo-merge-docs.qodo.ai/tools/scan_repo_discussions/) 💎 | ✅ | | | |
| | [Auto-Approve](https://qodo-merge-docs.qodo.ai/tools/improve/?h=auto#auto-approval) 💎 | ✅ | ✅ | | |
| | | | | | |
| USAGE | [CLI](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#local-repo-cli) | ✅ | ✅ | ✅ | ✅ |
| | [App / webhook](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#github-app) | ✅ | ✅ | ✅ | ✅ |
| | [Tagging bot](https://github.com/Codium-ai/pr-agent#try-it-now) | ✅ | | | |
| | [Actions](https://qodo-merge-docs.qodo.ai/installation/github/#run-as-a-github-action) | ✅ | ✅ | ✅ | ✅ |
| | | | | | |
| CORE | [PR compression](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/) | ✅ | ✅ | ✅ | ✅ |
| | Adaptive and token-aware file patch fitting | | | | |
| | [Multiple models support](https://qodo-merge-docs.qodo.ai/usage-guide/changing_a_model/) | ✅ | ✅ | | |
| | [Local and global metadata](https://qodo-merge-docs.qodo.ai/core-abilities/metadata/) | ✅ | ✅ | ✅ | ✅ |
| | [Dynamic context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic_context/) | ✅ | ✅ | ✅ | ✅ |
| | [Self reflection](https://qodo-merge-docs.qodo.ai/core-abilities/self_reflection/) | ✅ | ✅ | ✅ | |
| | [Static code analysis](https://qodo-merge-docs.qodo.ai/core-abilities/static_code_analysis/) 💎 | ✅ | ✅ | | |
| | [Global and wiki configurations](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/) 💎 | ✅ | ✅ | | |
| | [PR interactive actions](https://www.qodo.ai/images/pr_agent/pr-actions.mp4) 💎 | ✅ | | | |
| | [Impact Evaluation](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) 💎 | ✅ | ✅ | | |
| | [Code Validation 💎](https://qodo-merge-docs.qodo.ai/core-abilities/code_validation/) | ✅ | ✅ | ✅ | ✅ |
| | [Auto Best Practices 💎](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/) | ✅ | | | |
| | [Incremental Update 💎](https://qodo-merge-docs.qodo.ai/core-abilities/incremental_update/) | ✅ | | | |
!!! note "💎 means Qodo Merge only"
All along the documentation, 💎 marks a feature available only in [Qodo Merge](https://www.codium.ai/pricing/){:target="_blank"}, and not in the open-source version.

View File

@ -1,7 +1,7 @@
## Azure DevOps Pipeline
You can use a pre-built Action Docker image to run PR-Agent as an Azure devops pipeline.
Add the following file to your repository under `azure-pipelines.yml`:
add the following file to your repository under `azure-pipelines.yml`:
```yaml
# Opt out of CI triggers
@ -71,7 +71,7 @@ git_provider="azure"
```
Azure DevOps provider supports [PAT token](https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=Windows) or [DefaultAzureCredential](https://learn.microsoft.com/en-us/azure/developer/python/sdk/authentication-overview#authentication-in-server-environments) authentication.
PAT is faster to create, but has built-in expiration date, and will use the user identity for API calls.
PAT is faster to create, but has build in expiration date, and will use the user identity for API calls.
Using DefaultAzureCredential you can use managed identity or Service principle, which are more secure and will create separate ADO user identity (via AAD) to the agent.
If PAT was chosen, you can assign the value in .secrets.toml.

View File

@ -50,7 +50,7 @@ git_provider="bitbucket_server"
and pass the Pull request URL:
```shell
python cli.py --pr_url https://git.on-prem-instance-of-bitbucket.com/projects/PROJECT/repos/REPO/pull-requests/1 review
python cli.py --pr_url https://git.onpreminstanceofbitbucket.com/projects/PROJECT/repos/REPO/pull-requests/1 review
```
### Run it as service
@ -63,6 +63,6 @@ docker push codiumai/pr-agent:bitbucket_server_webhook # Push to your Docker re
```
Navigate to `Projects` or `Repositories`, `Settings`, `Webhooks`, `Create Webhook`.
Fill in the name and URL. For Authentication, select 'None'. Select the 'Pull Request Opened' checkbox to receive that event as a webhook.
Fill the name and URL, Authentication None select the Pull Request Opened checkbox to receive that event as webhook.
The URL should end with `/webhook`, for example: https://domain.com/webhook

View File

@ -17,11 +17,12 @@ git clone https://github.com/qodo-ai/pr-agent.git
```
5. Prepare variables and secrets. Skip this step if you plan on setting these as environment variables when running the agent:
- In the configuration file/variables:
- Set `config.git_provider` to "gitea"
- In the secrets file/variables:
- Set your AI model key in the respective section
- In the [Gitea] section, set `personal_access_token` (with token from step 2) and `webhook_secret` (with secret from step 3)
1. In the configuration file/variables:
- Set `config.git_provider` to "gitea"
2. In the secrets file/variables:
- Set your AI model key in the respective section
- In the [Gitea] section, set `personal_access_token` (with token from step 2) and `webhook_secret` (with secret from step 3)
6. Build a Docker image for the app and optionally push it to a Docker repository. We'll use Dockerhub as an example:
@ -38,8 +39,6 @@ GITEA__PERSONAL_ACCESS_TOKEN=<personal_access_token>
GITEA__WEBHOOK_SECRET=<webhook_secret>
GITEA__URL=https://gitea.com # Or self host
OPENAI__KEY=<your_openai_api_key>
GITEA__SKIP_SSL_VERIFICATION=false # or true
GITEA__SSL_CA_CERT=/path/to/cacert.pem
```
8. Create a webhook in your Gitea project. Set the URL to `http[s]://<PR_AGENT_HOSTNAME>/api/v1/gitea_webhooks`, the secret token to the generated secret from step 3, and enable the triggers `push`, `comments` and `merge request events`.

View File

@ -187,15 +187,14 @@ For example: `GITHUB.WEBHOOK_SECRET` --> `GITHUB__WEBHOOK_SECRET`
2. Build a docker image that can be used as a lambda function
```shell
# Note: --target github_lambda is optional as it's the default target
docker buildx build --platform=linux/amd64 . -t codiumai/pr-agent:github_lambda --target github_lambda -f docker/Dockerfile.lambda
docker buildx build --platform=linux/amd64 . -t codiumai/pr-agent:serverless -f docker/Dockerfile.lambda
```
3. Push image to ECR
```shell
docker tag codiumai/pr-agent:github_lambda <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:github_lambda
docker push <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:github_lambda
docker tag codiumai/pr-agent:serverless <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:serverless
docker push <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:serverless
```
4. Create a lambda function that uses the uploaded image. Set the lambda timeout to be at least 3m.
@ -204,28 +203,6 @@ For example: `GITHUB.WEBHOOK_SECRET` --> `GITHUB__WEBHOOK_SECRET`
7. Go back to steps 8-9 of [Method 5](#run-as-a-github-app) with the function url as your Webhook URL.
The Webhook URL would look like `https://<LAMBDA_FUNCTION_URL>/api/v1/github_webhooks`
### Using AWS Secrets Manager
For production Lambda deployments, use AWS Secrets Manager instead of environment variables:
1. Create a secret in AWS Secrets Manager with JSON format like this:
```json
{
"openai.key": "sk-proj-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"github.webhook_secret": "your-webhook-secret-from-step-2",
"github.private_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEA...\n-----END RSA PRIVATE KEY-----"
}
```
2. Add IAM permission `secretsmanager:GetSecretValue` to your Lambda execution role
3. Set these environment variables in your Lambda:
```bash
AWS_SECRETS_MANAGER__SECRET_ARN=arn:aws:secretsmanager:us-east-1:123456789012:secret:pr-agent-secrets-AbCdEf
CONFIG__SECRET_PROVIDER=aws_secrets_manager
```
---
## AWS CodeCommit Setup

View File

@ -46,7 +46,7 @@ Note that if your base branches are not protected, don't set the variables as `p
1. In GitLab create a new user and give it "Reporter" role ("Developer" if using Pro version of the agent) for the intended group or project.
2. For the user from step 1, generate a `personal_access_token` with `api` access.
2. For the user from step 1. generate a `personal_access_token` with `api` access.
3. Generate a random secret for your app, and save it for later (`shared_secret`). For example, you can use:
@ -61,12 +61,12 @@ git clone https://github.com/qodo-ai/pr-agent.git
```
5. Prepare variables and secrets. Skip this step if you plan on setting these as environment variables when running the agent:
1. In the configuration file/variables:
- Set `config.git_provider` to "gitlab"
1. In the configuration file/variables:
- Set `config.git_provider` to "gitlab"
2. In the secrets file/variables:
- Set your AI model key in the respective section
- In the [gitlab] section, set `personal_access_token` (with token from step 2) and `shared_secret` (with secret from step 3)
2. In the secrets file/variables:
- Set your AI model key in the respective section
- In the [gitlab] section, set `personal_access_token` (with token from step 2) and `shared_secret` (with secret from step 3)
6. Build a Docker image for the app and optionally push it to a Docker repository. We'll use Dockerhub as an example:
@ -88,63 +88,3 @@ OPENAI__KEY=<your_openai_api_key>
8. Create a webhook in your GitLab project. Set the URL to `http[s]://<PR_AGENT_HOSTNAME>/webhook`, the secret token to the generated secret from step 3, and enable the triggers `push`, `comments` and `merge request events`.
9. Test your installation by opening a merge request or commenting on a merge request using one of PR Agent's commands.
## Deploy as a Lambda Function
Note that since AWS Lambda env vars cannot have "." in the name, you can replace each "." in an env variable with "__".<br>
For example: `GITLAB.PERSONAL_ACCESS_TOKEN` --> `GITLAB__PERSONAL_ACCESS_TOKEN`
1. Follow steps 1-5 from [Run a GitLab webhook server](#run-a-gitlab-webhook-server).
2. Build a docker image that can be used as a lambda function
```shell
docker buildx build --platform=linux/amd64 . -t codiumai/pr-agent:gitlab_lambda --target gitlab_lambda -f docker/Dockerfile.lambda
```
3. Push image to ECR
```shell
docker tag codiumai/pr-agent:gitlab_lambda <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:gitlab_lambda
docker push <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:gitlab_lambda
```
4. Create a lambda function that uses the uploaded image. Set the lambda timeout to be at least 3m.
5. Configure the lambda function to have a Function URL.
6. In the environment variables of the Lambda function, specify `AZURE_DEVOPS_CACHE_DIR` to a writable location such as /tmp. (see [link](https://github.com/Codium-ai/pr-agent/pull/450#issuecomment-1840242269))
7. Go back to steps 8-9 of [Run a GitLab webhook server](#run-a-gitlab-webhook-server) with the function URL as your Webhook URL.
The Webhook URL would look like `https://<LAMBDA_FUNCTION_URL>/webhook`
### Using AWS Secrets Manager
For production Lambda deployments, use AWS Secrets Manager instead of environment variables:
1. Create individual secrets for each GitLab webhook with this JSON format (e.g., secret name: `project-webhook-secret-001`)
```json
{
"gitlab_token": "glpat-xxxxxxxxxxxxxxxxxxxxxxxx",
"token_name": "project-webhook-001"
}
```
2. Create a main configuration secret for common settings (e.g., secret name: `pr-agent-main-config`)
```json
{
"openai.key": "sk-proj-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}
```
3. Set these environment variables in your Lambda:
```bash
CONFIG__SECRET_PROVIDER=aws_secrets_manager
AWS_SECRETS_MANAGER__SECRET_ARN=arn:aws:secretsmanager:us-east-1:123456789012:secret:pr-agent-main-config-AbCdEf
```
4. In your GitLab webhook configuration, set the **Secret Token** to the **Secret name** created in step 1:
- Example: `project-webhook-secret-001`
**Important**: When using Secrets Manager, GitLab's webhook secret must be the Secrets Manager secret name.
5. Add IAM permission `secretsmanager:GetSecretValue` to your Lambda execution role

View File

@ -12,7 +12,7 @@ To invoke a tool (for example `review`), you can run PR-Agent directly from the
- For GitHub:
```bash
docker run --rm -it -e OPENAI.KEY=<your_openai_key> -e GITHUB.USER_TOKEN=<your_github_token> codiumai/pr-agent:latest --pr_url <pr_url> review
docker run --rm -it -e OPENAI.KEY=<your key> -e GITHUB.USER_TOKEN=<your token> codiumai/pr-agent:latest --pr_url <pr_url> review
```
If you are using GitHub enterprise server, you need to specify the custom url as variable.

View File

@ -1,20 +1,20 @@
Qodo Merge is a versatile application compatible with GitHub, GitLab, and BitBucket, hosted by QodoAI.
See [here](https://qodo-merge-docs.qodo.ai/overview/pr_agent_pro/) for more details about the benefits of using Qodo Merge.
## Usage and Licensing
## Trial Period and Licensing
### Cloud Users
### Cloud Users with Teams Account
Non-paying users will enjoy feedback on up to 75 PRs per git organization per month. Above this limit, PRs will not receive feedback until a new month begins.
A complimentary two-week trial is provided to all new users (with three additional grace usages). When the trial period ends, users will stop receiving feedback from Qodo Merge.
For unlimited access, user licenses (seats) are required. Each user requires an individual seat license.
Following the trial period, user licenses (seats) are required for continued access. Each user requires an individual seat license.
After purchasing seats, the team owner can assign them to specific users through the management portal.
With an assigned seat, users can seamlessly deploy the application across any of their code repositories in a git organization, and receive feedback on all their PRs.
With an assigned seat, users can seamlessly deploy the application across any of their code repositories.
### Enterprise Account
For companies who require an Enterprise account, please [contact](https://www.qodo.ai/contact/#pricing) us to initiate a trial period, and to discuss pricing and licensing options.
For organizations who require an Enterprise account, please [contact](https://www.qodo.ai/contact/#pricing) us to initiate a trial period, and to discuss pricing and licensing options.
## Install Qodo Merge for GitHub
@ -27,9 +27,7 @@ Qodo Merge for GitHub cloud is available for installation through the [GitHub Ma
### GitHub Enterprise Server
To use Qodo Merge on your private GitHub Enterprise Server, you will need to [contact](https://www.qodo.ai/contact/#pricing) Qodo for starting an Enterprise trial.
(Note: The marketplace app is not compatible with GitHub Enterprise Server. Installation requires creating a private GitHub App instead.)
To use Qodo Merge application on your private GitHub Enterprise Server, you will need to [contact](https://www.qodo.ai/contact/#pricing) Qodo for starting an Enterprise trial.
### GitHub Open Source Projects
@ -97,4 +95,4 @@ Open a new merge request or add a MR comment with one of Qodo Merges commands
### GitLab Server
For [limited free usage](https://qodo-merge-docs.qodo.ai/installation/qodo_merge/#cloud-users) on private GitLab Server, the same [installation steps](#gitlab-cloud) as for GitLab Cloud apply. For unlimited usage, you will need to [contact](https://www.qodo.ai/contact/#pricing) Qodo for moving to an Enterprise account.
For a trial period of two weeks on your private GitLab Server, the same [installation steps](#gitlab-cloud) as for GitLab Cloud apply. After the trial period, you will need to [contact](https://www.qodo.ai/contact/#pricing) Qodo for moving to an Enterprise account.

View File

@ -1,12 +1,7 @@
### Overview
[Qodo Merge](https://www.codium.ai/pricing/){:target="_blank"} is a hosted version of the open-source [PR-Agent](https://github.com/Codium-ai/pr-agent){:target="_blank"}.
It is designed for companies and teams that require additional features and capabilities.
Free users receive a quota of 75 monthly PR feedbacks per git organization. Unlimited usage requires a paid subscription. See [details](https://qodo-merge-docs.qodo.ai/installation/qodo_merge/#cloud-users).
Qodo Merge provides the following benefits:
[Qodo Merge](https://www.codium.ai/pricing/){:target="_blank"} is a paid, hosted version of open-source [PR-Agent](https://github.com/Codium-ai/pr-agent){:target="_blank"}. A complimentary two-week trial is offered, followed by a monthly subscription fee.
Qodo Merge is designed for companies and teams that require additional features and capabilities. It provides the following benefits:
1. **Fully managed** - We take care of everything for you - hosting, models, regular updates, and more. Installation is as simple as signing up and adding the Qodo Merge app to your GitHub\GitLab\BitBucket repo.
@ -50,7 +45,7 @@ Here are additional tools that are available only for Qodo Merge users:
### Supported languages
Qodo Merge leverages the world's leading code models, such as Claude 4 Sonnet, o4-mini and Gemini-2.5-Pro.
Qodo Merge leverages the world's leading code models, such as Claude 3.7 Sonnet and o3-mini.
As a result, its primary tools such as `describe`, `review`, and `improve`, as well as the PR-chat feature, support virtually all programming languages.
For specialized commands that require static code analysis, Qodo Merge offers support for specific languages. For more details about features that require static code analysis, please refer to the [documentation](https://qodo-merge-docs.qodo.ai/tools/analyze/#overview).

View File

@ -2,300 +2,200 @@
## Methodology
Qodo Merge PR Benchmark evaluates and compares the performance of Large Language Models (LLMs) in analyzing pull request code and providing meaningful code suggestions.
Our diverse dataset contains 400 pull requests from over 100 repositories, spanning various programming languages and frameworks to reflect real-world scenarios.
Qodo Merge PR Benchmark evaluates and compares the performance of two Large Language Models (LLMs) in analyzing pull request code and providing meaningful code suggestions.
Our diverse dataset comprises of 400 pull requests from over 100 repositories, spanning various programming languages and frameworks to reflect real-world scenarios.
- For each pull request, we have pre-generated suggestions from eleven different top-performing models using the Qodo Merge `improve` tool. The prompt for response generation can be found [here](https://github.com/qodo-ai/pr-agent/blob/main/pr_agent/settings/code_suggestions/pr_code_suggestions_prompts_not_decoupled.toml).
- For each pull request, two distinct LLMs process the same prompt using the Qodo Merge `improve` tool, each generating two sets of responses. The prompt for response generation can be found [here](https://github.com/qodo-ai/pr-agent/blob/main/pr_agent/settings/code_suggestions/pr_code_suggestions_prompts_not_decoupled.toml).
- To benchmark a model, we generate its suggestions for the same pull requests and ask a high-performing judge model to **rank** the new model's output against the pre-generated baseline suggestions. We utilize OpenAI's `o3` model as the judge, though other models have yielded consistent results. The prompt for this ranking judgment is available [here](https://github.com/Codium-ai/pr-agent-settings/tree/main/benchmark).
- Subsequently, a high-performing third model (an AI judge) evaluates the responses from the initial two models to determine the superior one. We utilize OpenAI's `o3` model as the judge, though other models have yielded consistent results. The prompt for this comparative judgment is available [here](https://github.com/Codium-ai/pr-agent-settings/tree/main/benchmark).
- We aggregate ranking outcomes across all pull requests, calculating performance metrics for the evaluated model.
- We also analyze the qualitative feedback from the judge to identify the model's comparative strengths and weaknesses against the established baselines.
- We aggregate comparison outcomes across all the pull requests, calculating the win rate for each model. We also analyze the qualitative feedback (the "why" explanations from the judge) to identify each model's comparative strengths and weaknesses.
This approach provides not just a quantitative score but also a detailed analysis of each model's strengths and weaknesses.
A list of the models used for generating the baseline suggestions, and example results, can be found in the [Appendix](#appendix-example-results).
- For each model we build a "Model Card", comparing it against others. To ensure full transparency and enable community scrutiny, we also share the raw code suggestions generated by each model, and the judge's specific feedback. See example for the full output [here](https://github.com/Codium-ai/pr-agent-settings/blob/main/benchmark/sonnet_37_vs_gemini-2.5-pro-preview-05-06.md)
[//]: # (Note that this benchmark focuses on quality: the ability of an LLM to process complex pull request with multiple files and nuanced task to produce high-quality code suggestions.)
Note that this benchmark focuses on quality: the ability of an LLM to process complex pull request with multiple files and nuanced task to produce high-quality code suggestions.
Other factors like speed, cost, and availability, while also relevant for model selection, are outside this benchmark's scope.
[//]: # (Other factors like speed, cost, and availability, while also relevant for model selection, are outside this benchmark's scope. We do specify the thinking budget used by each model, which can be a factor in the model's performance.)
## TL;DR
[//]: # ()
Here's a summary of the win rates based on the benchmark:
## PR Benchmark Results
[//]: # (| Model A | Model B | Model A Win Rate | Model B Win Rate |)
[//]: # (|:-------------------------------|:-------------------------------|:----------------:|:----------------:|)
[//]: # (| Gemini-2.5-pro-preview-05-06 | GPT-4.1 | 70.4% | 29.6% |)
[//]: # (| Gemini-2.5-pro-preview-05-06 | Sonnet 3.7 | 78.1% | 21.9% |)
[//]: # (| GPT-4.1 | Sonnet 3.7 | 61.0% | 39.0% |)
<table>
<thead>
<tr>
<th style="text-align:left;">Model Name</th>
<th style="text-align:left;">Version (Date)</th>
<th style="text-align:left;">Thinking budget tokens</th>
<th style="text-align:center;">Score</th>
</tr>
<th style="text-align:left;">Model A</th>
<th style="text-align:left;">Model B</th>
<th style="text-align:center;">Model A Win Rate</th> <th style="text-align:center;">Model B Win Rate</th> </tr>
</thead>
<tbody>
<tr>
<td style="text-align:left;">o3</td>
<td style="text-align:left;">2025-04-16</td>
<td style="text-align:left;">'medium' (<a href="https://ai.google.dev/gemini-api/docs/openai">8000</a>)</td>
<td style="text-align:center;"><b>62.5</b></td>
</tr>
<td style="text-align:left;">Gemini-2.5-pro-preview-05-06</td>
<td style="text-align:left;">GPT-4.1</td>
<td style="text-align:center; color: #1E8449;"><b>70.4%</b></td> <td style="text-align:center; color: #D8000C;"><b>29.6%</b></td> </tr>
<tr>
<td style="text-align:left;">o4-mini</td>
<td style="text-align:left;">2025-04-16</td>
<td style="text-align:left;">'medium' (<a href="https://ai.google.dev/gemini-api/docs/openai">8000</a>)</td>
<td style="text-align:center;"><b>57.7</b></td>
</tr>
<td style="text-align:left;">Gemini-2.5-pro-preview-05-06</td>
<td style="text-align:left;">Sonnet 3.7</td>
<td style="text-align:center; color: #1E8449;"><b>78.1%</b></td> <td style="text-align:center; color: #D8000C;"><b>21.9%</b></td> </tr>
<tr>
<td style="text-align:left;">Gemini-2.5-pro</td>
<td style="text-align:left;">2025-06-05</td>
<td style="text-align:left;">4096</td>
<td style="text-align:center;"><b>56.3</b></td>
</tr>
<td style="text-align:left;">Gemini-2.5-pro-preview-05-06</td>
<td style="text-align:left;">Gemini-2.5-flash-preview-04-17</td>
<td style="text-align:center; color: #1E8449;"><b>73.0%</b></td> <td style="text-align:center; color: #D8000C;"><b>27.0%</b></td> </tr>
<tr>
<td style="text-align:left;">Gemini-2.5-pro</td>
<td style="text-align:left;">2025-06-05</td>
<td style="text-align:left;">1024</td>
<td style="text-align:center;"><b>44.3</b></td>
</tr>
<td style="text-align:left;">Gemini-2.5-flash-preview-04-17</td>
<td style="text-align:left;">GPT-4.1</td>
<td style="text-align:center; color: #1E8449;"><b>54.6%</b></td> <td style="text-align:center; color: #D8000C;"><b>45.4%</b></td> </tr>
<tr>
<td style="text-align:left;">Claude-4-sonnet</td>
<td style="text-align:left;">2025-05-14</td>
<td style="text-align:left;">4096</td>
<td style="text-align:center;"><b>39.7</b></td>
</tr>
<tr>
<td style="text-align:left;">Claude-4-sonnet</td>
<td style="text-align:left;">2025-05-14</td>
<td style="text-align:left;"></td>
<td style="text-align:center;"><b>39.0</b></td>
</tr>
<tr>
<td style="text-align:left;">Codex-mini</td>
<td style="text-align:left;">2025-06-20</td>
<td style="text-align:left;"><a href="https://platform.openai.com/docs/models/codex-mini-latest">unknown</a></td>
<td style="text-align:center;"><b>37.2</b></td>
</tr>
<tr>
<td style="text-align:left;">Gemini-2.5-flash</td>
<td style="text-align:left;">2025-04-17</td>
<td style="text-align:left;"></td>
<td style="text-align:center;"><b>33.5</b></td>
</tr>
<tr>
<td style="text-align:left;">Claude-4-opus-20250514</td>
<td style="text-align:left;">2025-05-14</td>
<td style="text-align:left;"></td>
<td style="text-align:center;"><b>32.8</b></td>
</tr>
<tr>
<td style="text-align:left;">Claude-3.7-sonnet</td>
<td style="text-align:left;">2025-02-19</td>
<td style="text-align:left;"></td>
<td style="text-align:center;"><b>32.4</b></td>
</tr>
<td style="text-align:left;">Gemini-2.5-flash-preview-04-17</td>
<td style="text-align:left;">Sonnet 3.7</td>
<td style="text-align:center; color: #1E8449;"><b>60.6%</b></td> <td style="text-align:center; color: #D8000C;"><b>39.4%</b></td> </tr>
<tr>
<td style="text-align:left;">GPT-4.1</td>
<td style="text-align:left;">2025-04-14</td>
<td style="text-align:left;"></td>
<td style="text-align:center;"><b>26.5</b></td>
</tr>
<td style="text-align:left;">Sonnet 3.7</td>
<td style="text-align:center; color: #1E8449;"><b>61.0%</b></td> <td style="text-align:center; color: #D8000C;"><b>39.0%</b></td> </tr>
</tbody>
</table>
## Results Analysis
### O3
## Gemini-2.5-pro-preview-05-06 - Model Card
Final score: **62.5**
### Comparison against GPT-4.1
strengths:
![Comparison](https://codium.ai/images/qodo_merge_benchmark/gpt-4.1_vs_gemini-2.5-pro-preview-05-06_judge_o3.png){width=768}
- **High precision & compliance:** Generally respects task rules (limits, “added lines” scope, YAML schema) and avoids false-positive advice, often returning an empty list when appropriate.
- **Clear, actionable output:** Suggestions are concise, well-explained and include correct before/after patches, so reviewers can apply them directly.
- **Good critical-bug detection rate:** Frequently spots compile-breakers or obvious runtime faults (nil / NPE, overflow, race, wrong selector, etc.), putting it at least on par with many peers.
- **Consistent formatting:** Produces syntactically valid YAML with correct labels, making automated consumption easy.
#### Analysis Summary
weaknesses:
Model 'Gemini-2.5-pro-preview-05-06' is generally more useful thanks to wider and more accurate bug detection and concrete patches, but it sacrifices compliance discipline and sometimes oversteps the task rules. Model 'GPT-4.1' is safer and highly rule-abiding, yet often too timid—missing many genuine issues and providing limited insight. An ideal reviewer would combine 'GPT-4.1 restraint with 'Gemini-2.5-pro-preview-05-06' thoroughness.
- **Narrow coverage:** Tends to stop after 1-2 issues; regularly misses additional critical defects that better answers catch, so it is seldom the top-ranked review.
- **Occasional inaccuracies:** A few replies introduce new bugs, give partial/duplicate fixes, or (rarely) violate rules (e.g., import suggestions), hurting trust.
- **Conservative bias:** Prefers silence over risk; while this keeps precision high, it lowers recall and overall usefulness on larger diffs.
- **Little added insight:** Rarely offers broader context, optimisations or holistic improvements, causing it to rank only mid-tier in many comparisons.
#### Detailed Analysis
### O4 Mini ('medium' thinking tokens)
Gemini-2.5-pro-preview-05-06 strengths:
Final score: **57.7**
- better_bug_coverage: Detects and explains more critical issues, winning in ~70 % of comparisons and achieving a higher average score.
- actionable_fixes: Supplies clear code snippets, correct language labels, and often multiple coherent suggestions per diff.
- deeper_reasoning: Shows stronger grasp of logic, edge cases, and cross-file implications, leading to broader, high-impact reviews.
strengths:
Gemini-2.5-pro-preview-05-06 weaknesses:
- **Good rule adherence:** Most answers respect the “new-lines only”, 3-suggestion, and YAML-schema limits, and frequently choose the safe empty list when the diff truly adds no critical bug.
- **Clear, minimal patches:** When the model does spot a defect it usually supplies terse, valid before/after snippets and short, targeted explanations, making fixes easy to read and apply.
- **Language & domain breadth:** Demonstrates competence across many ecosystems (C/C++, Java, TS/JS, Go, Rust, Python, Bash, Markdown, YAML, SQL, CSS, translation files, etc.) and can detect both compile-time and runtime mistakes.
- **Often competitive:** In a sizeable minority of cases the model ties for best or near-best answer, occasionally being the only response to catch a subtle crash or build blocker.
- guideline_violations: More prone to over-eager advice—non-critical tweaks, touching unchanged code, suggesting new imports, or minor format errors.
- occasional_overreach: Some fixes are speculative or risky, potentially introducing new bugs.
- redundant_or_duplicate: At times repeats the same point or exceeds the required brevity.
weaknesses:
- **High miss rate:** A large share of examples show the model returning an empty list or only minor advice while other reviewers catch clear, high-impact bugs—indicative of weak defect-detection recall.
- **False or harmful fixes:** Several answers introduce new compilation errors, propose out-of-scope changes, or violate explicit rules (e.g., adding imports, version bumps, touching untouched lines), reducing trustworthiness.
- **Shallow coverage:** Even when it identifies one real issue it often stops there, missing additional critical problems found by stronger peers; breadth and depth are inconsistent.
### Comparison against Sonnet 3.7
### Gemini-2.5 Pro (4096 thinking tokens)
![Comparison](https://codium.ai/images/qodo_merge_benchmark/sonnet_37_vs_gemini-2.5-pro-preview-05-06_judge_o3.png){width=768}
Final score: **56.3**
#### Analysis Summary
strengths:
Model 'Gemini-2.5-pro-preview-05-06' is the stronger reviewer—more frequently identifies genuine, high-impact bugs and provides well-formed, actionable fixes. Model 'Sonnet 3.7' is safer against false positives and tends to be concise but often misses important defects or offers low-value or incorrect suggestions.
- **High formatting compliance:** The model almost always produces valid YAML, respects the three-suggestion limit, and supplies clear before/after code snippets and short rationales.
- **Good “first-bug” detection:** It frequently notices the single most obvious regression (crash, compile error, nil/NPE risk, wrong path, etc.) and gives a minimal, correct patch—often judged “on-par” with other solid answers.
- **Clear, concise writing:** Explanations are brief yet understandable for reviewers; fixes are scoped to the changed lines and rarely include extraneous context.
- **Low rate of harmful fixes:** Truly dangerous or build-breaking advice is rare; most mistakes are omissions rather than wrong code.
See raw results [here](https://github.com/Codium-ai/pr-agent-settings/blob/main/benchmark/sonnet_37_vs_gemini-2.5-pro-preview-05-06.md)
weaknesses:
- **Limited breadth of review:** The model regularly stops after the first or second issue, missing additional critical problems that stronger answers surface, so it is often out-ranked by more comprehensive peers.
- **Occasional guideline violations:** A noticeable minority of answers touch unchanged lines, exceed the 3-item cap, suggest adding imports, or drop the required YAML wrapper, leading to automatic downgrades.
- **False positives / speculative fixes:** In several cases it flags non-issues (style, performance, redundant code) or supplies debatable “improvements”, lowering precision and sometimes breaching the “critical bugs only” rule.
- **Inconsistent error coverage:** For certain domains (build scripts, schema files, test code) it either returns an empty list when real regressions exist or proposes cosmetic edits, indicating gaps in specialised knowledge.
#### Detailed Analysis
### Claude-4 Sonnet (4096 thinking tokens)
Gemini-2.5-pro-preview-05-06 strengths:
Final score: **39.7**
- higher_accuracy_and_coverage: finds real critical bugs and supplies actionable patches in most examples (better in 78 % of cases).
- guideline_awareness: usually respects new-lines-only scope, ≤3 suggestions, proper YAML, and stays silent when no issues exist.
- detailed_reasoning_and_patches: explanations tie directly to the diff and fixes are concrete, often catching multiple related defects that 'Sonnet 3.7' overlooks.
strengths:
Gemini-2.5-pro-preview-05-06 weaknesses:
- **High guideline & format compliance:** Almost always returns valid YAML, keeps ≤ 3 suggestions, avoids forbidden import/boiler-plate changes and provides clear before/after snippets.
- **Good pinpoint accuracy on single issues:** Frequently spots at least one real critical bug and proposes a concise, technically correct fix that compiles/runs.
- **Clarity & brevity of patches:** Explanations are short, actionable, and focused on changed lines, making the advice easy for reviewers to apply.
- occasional_rule_violations: sometimes proposes new imports, package-version changes, or edits outside the added lines.
- overzealous_suggestions: may add speculative or stylistic fixes that exceed the “critical” scope, or mis-label severity.
- sporadic_technical_slips: a few patches contain minor coding errors, oversized snippets, or duplicate/contradicting advice.
weaknesses:
## GPT-4.1 - Model Card
- **Low coverage / recall:** Regularly surfaces only one minor issue (or none) while missing other, often more severe, problems caught by peer models.
- **High “empty-list” rate:** In many diffs the model returns no suggestions even when clear critical bugs exist, offering zero reviewer value.
- **Occasional incorrect or harmful fixes:** A non-trivial number of suggestions are speculative, contradict code intent, or would break compilation/runtime; sometimes duplicates or contradicts itself.
- **Inconsistent severity labelling & duplication:** Repeats the same point in multiple slots, marks cosmetic edits as “critical”, or leaves `improved_code` identical to original.
### Comparison against Sonnet 3.7
![Comparison](https://codium.ai/images/qodo_merge_benchmark/gpt-4.1_vs_sonnet_3.7_judge_o3.png){width=768}
### Claude-4 Sonnet
#### Analysis Summary
Final score: **39.0**
Model 'GPT-4.1' is safer and more compliant, preferring silence over speculation, which yields fewer rule breaches and false positives but misses some real bugs.
Model 'Sonnet 3.7' is more adventurous and often uncovers important issues that 'GPT-4.1' ignores, yet its aggressive style leads to frequent guideline violations and a higher proportion of incorrect or non-critical advice.
strengths:
See raw results [here](https://github.com/Codium-ai/pr-agent-settings/blob/main/benchmark/gpt-4.1_vs_sonnet_3.7_judge_o3.md)
- **Consistently well-formatted & rule-compliant output:** Almost every answer follows the required YAML schema, keeps within the 3-suggestion limit, and returns an empty list when no issues are found, showing good instruction following.
- **Actionable, code-level patches:** When it does spot a defect the model usually supplies clear, minimal diffs or replacement snippets that compile / run, making the fix easy to apply.
#### Detailed Analysis
- **Decent hit-rate on “obvious” bugs:** The model reliably catches the most blatant syntax errors, null-checks, enum / cast problems, and other first-order issues, so it often ties or slightly beats weaker baseline replies.
GPT-4.1 strengths:
- Strong guideline adherence: usually stays strictly on `+` lines, avoids non-critical or stylistic advice, and rarely suggests forbidden imports; often outputs an empty list when no real bug exists.
- Lower false-positive rate: suggestions are more accurate and seldom introduce new bugs; fixes compile more reliably.
- Good schema discipline: YAML is almost always well-formed and fields are populated correctly.
weaknesses:
GPT-4.1 weaknesses:
- Misses bugs: often returns an empty list even when a clear critical issue is present, so coverage is narrower.
- Sparse feedback: when it does comment, it tends to give fewer suggestions and sometimes lacks depth or completeness.
- Occasional metadata/slip-ups (wrong language tags, overly broad code spans), though less harmful than Sonnet 3.7 errors.
- **Shallow coverage:** It frequently stops after one easy bug and overlooks additional, equally-critical problems that stronger reviewers find, leaving significant risks unaddressed.
### Comparison against Gemini-2.5-pro-preview-05-06
- **False positives & harmful fixes:** In a noticeable minority of cases it misdiagnoses code, suggests changes that break compilation or behaviour, or flags non-issues, sometimes making its output worse than doing nothing.
![Comparison](https://codium.ai/images/qodo_merge_benchmark/gpt-4.1_vs_gemini-2.5-pro-preview-05-06_judge_o3.png){width=768}
- **Drifts into non-critical or out-of-scope advice:** The model regularly proposes style tweaks, documentation edits, or changes to unchanged lines, violating the “critical new-code only” requirement.
#### Analysis Summary
Model 'Gemini-2.5-pro-preview-05-06' is generally more useful thanks to wider and more accurate bug detection and concrete patches, but it sacrifices compliance discipline and sometimes oversteps the task rules. Model 'GPT-4.1' is safer and highly rule-abiding, yet often too timid—missing many genuine issues and providing limited insight. An ideal reviewer would combine 'GPT-4.1 restraint with 'Gemini-2.5-pro-preview-05-06' thoroughness.
### Gemini-2.5 Flash
#### Detailed Analysis
strengths:
GPT-4.1 strengths:
- strict_compliance: Usually sticks to the “critical bugs only / new + lines only” rule, so outputs rarely violate task constraints.
- low_risk: Conservative behaviour avoids harmful or speculative fixes; safer when no obvious issue exists.
- concise_formatting: Tends to produce minimal, correctly-structured YAML without extra noise.
- **High precision / low false-positive rate:** The model often stays silent or gives a single, well-justified fix, so when it does speak the suggestion is usually correct and seldom touches unchanged lines, keeping guideline compliance high.
- **Good guideline awareness:** YAML structure is consistently valid; suggestions rarely exceed the 3-item limit and generally restrict themselves to newly-added lines.
- **Clear, concise patches:** When a defect is found, the model produces short rationales and tidy “improved_code” blocks that reviewers can apply directly.
- **Risk-averse behaviour pays off in “no-bug” PRs:** In examples where the diff truly contained no critical issue, the models empty output ranked above peers that offered speculative or stylistic advice.
GPT-4.1 weaknesses:
- under_detection: Frequently returns an empty list even when real bugs are present, missing ~70 % of the time.
- shallow_analysis: When it does suggest fixes, coverage is narrow and technical depth is limited, sometimes with wrong language tags or minor format slips.
- occasional_inaccuracy: A few suggestions are unfounded or duplicate, and rare guideline breaches (e.g., import advice) still occur.
weaknesses:
- **Very low recall / shallow coverage:** In a large majority of cases it gives 0-1 suggestions and misses other evident, critical bugs highlighted by peer models, leading to inferior rankings.
- **Occasional incorrect or harmful fixes:** A noticeable subset of answers propose changes that break functionality or misunderstand the code (e.g. bad constant, wrong header logic, speculative rollbacks).
- **Non-actionable placeholders:** Some “improved_code” sections contain comments or “…” rather than real patches, reducing practical value.
## Sonnet 3.7 - Model Card
### GPT-4.1
### Comparison against GPT-4.1
Final score: **26.5**
![Comparison](https://codium.ai/images/qodo_merge_benchmark/gpt-4.1_vs_sonnet_3.7_judge_o3.png){width=768}
strengths:
#### Analysis Summary
- **Consistent format & guideline obedience:** Output is almost always valid YAML, within the 3-suggestion limit, and rarely touches lines not prefixed with “+”.
- **Low false-positive rate:** When no real defect exists, the model correctly returns an empty list instead of inventing speculative fixes, avoiding the “noise” many baseline answers add.
- **Clear, concise patches when it does act:** In the minority of cases where it detects a bug (e.g., ex-13, 46, 212), the fix is usually correct, minimal, and easy to apply.
Model 'GPT-4.1' is safer and more compliant, preferring silence over speculation, which yields fewer rule breaches and false positives but misses some real bugs.
Model 'Sonnet 3.7' is more adventurous and often uncovers important issues that 'GPT-4.1' ignores, yet its aggressive style leads to frequent guideline violations and a higher proportion of incorrect or non-critical advice.
weaknesses:
See raw results [here](https://github.com/Codium-ai/pr-agent-settings/blob/main/benchmark/gpt-4.1_vs_sonnet_3.7_judge_o3.md)
- **Very low recall / coverage:** In a large majority of examples it outputs an empty list or only 1 trivial suggestion while obvious critical issues remain unfixed; it systematically misses circular bugs, null-checks, schema errors, etc.
- **Shallow analysis:** Even when it finds one problem it seldom looks deeper, so more severe or additional bugs in the same diff are left unaddressed.
- **Occasional technical inaccuracies:** A noticeable subset of suggestions are wrong (mis-ordered assertions, harmful Bash `set` change, false dangling-reference claims) or carry metadata errors (mis-labeling files as “python”).
- **Repetitive / derivative fixes:** Many outputs duplicate earlier simplistic ideas (e.g., single null-check) without new insight, showing limited reasoning breadth.
#### Detailed Analysis
### OpenAI codex-mini
'Sonnet 3.7' strengths:
- Better bug discovery breadth: more willing to dive into logic and spot critical problems that 'GPT-4.1' overlooks; often supplies multiple, detailed fixes.
- Richer explanations & patches: gives fuller context and, when correct, proposes more functional or user-friendly solutions.
- Generally correct language/context tagging and targeted code snippets.
final score: **37.2**
'Sonnet 3.7' weaknesses:
- Guideline violations: frequently flags non-critical issues, edits untouched code, or recommends adding imports, breaching task rules.
- Higher error rate: suggestions are more speculative and sometimes introduce new defects or duplicate work already done.
- Occasional schema or formatting mistakes (missing list value, duplicated suggestions), reducing reliability.
strengths:
- **Can spot high-impact defects:** When it “locks on”, codex-mini often identifies the main runtime or security regression (e.g., race-conditions, logic inversions, blocking I/O, resource leaks) and proposes a minimal, direct patch that compiles and respects neighbouring style.
- **Produces concise, scoped fixes:** Valid answers usually stay within the allowed 3-suggestion limit, reference only the added lines, and contain clear before/after snippets that reviewers can apply verbatim.
- **Occasional broad coverage:** In a minority of cases the model catches multiple independent issues (logic + tests + docs) and outperforms every baseline answer, showing good contextual understanding of heterogeneous diffs.
### Comparison against Gemini-2.5-pro-preview-05-06
weaknesses:
![Comparison](https://codium.ai/images/qodo_merge_benchmark/sonnet_37_vs_gemini-2.5-pro-preview-05-06_judge_o3.png){width=768}
- **Output instability / format errors:** A very large share of responses are unusable—plain refusals, shell commands, or malformed/empty YAML—indicating brittle adherence to the required schema and tanking overall usefulness.
- **Critical-miss rate:** Even when the format is correct the model frequently overlooks the single most serious bug the diff introduces, instead focusing on stylistic nits or speculative refactors.
- **Introduces new problems:** Several suggestions add unsupported APIs, undeclared variables, wrong types, or break compilation, hurting trust in the recommendations.
- **Rule violations:** It often edits lines outside the diff, exceeds the 3-suggestion cap, or labels cosmetic tweaks as “critical”, showing inconsistent guideline compliance.
#### Analysis Summary
### Claude-4 Opus
final score: **32.8**
strengths:
- **Format & rule adherence:** Almost always returns valid YAML, stays within the ≤3-suggestion limit, and usually restricts edits to newly-added lines, so its output is easy to apply automatically.
- **Concise, focused patches:** When it does find a real bug it gives short, well-scoped explanations plus minimal diff snippets, often outperforming verbose baselines in clarity.
- **Able to catch subtle edge-cases:** In several examples it detected overflow, race-condition or enum-mismatch issues that many other models missed, showing solid codeanalysis capability.
weaknesses:
- **Low recall / narrow coverage:** In a large share of the 399 examples the model produced an empty list or only one minor tip while more serious defects were present, causing it to be rated inferior to most baselines.
- **Frequent incorrect or no-op fixes:** It sometimes supplies identical “before/after” code, flags non-issues, or suggests changes that would break compilation or logic, reducing reviewer trust.
- **Shaky guideline consistency:** Although generally compliant, it still occasionally violates rules (touches unchanged lines, offers stylistic advice, adds imports) and duplicates suggestions, indicating unstable internal checks.
## Appendix - Example Results
Some examples of benchmarked PRs and their results:
- [Example 1](https://www.qodo.ai/images/qodo_merge_benchmark/example_results1.html)
- [Example 2](https://www.qodo.ai/images/qodo_merge_benchmark/example_results2.html)
- [Example 3](https://www.qodo.ai/images/qodo_merge_benchmark/example_results3.html)
- [Example 4](https://www.qodo.ai/images/qodo_merge_benchmark/example_results4.html)
### Models Used for Benchmarking
The following models were used for generating the benchmark baseline:
```markdown
(1) anthropic_sonnet_3.7_v1:0
(2) claude-4-opus-20250514
(3) claude-4-sonnet-20250514
(4) claude-4-sonnet-20250514_thinking_2048
(5) gemini-2.5-flash-preview-04-17
(6) gemini-2.5-pro-preview-05-06
(7) gemini-2.5-pro-preview-06-05_1024
(8) gemini-2.5-pro-preview-06-05_4096
(9) gpt-4.1
(10) o3
(11) o4-mini_medium
```
Model 'Gemini-2.5-pro-preview-05-06' is the stronger reviewer—more frequently identifies genuine, high-impact bugs and provides well-formed, actionable fixes. Model 'Sonnet 3.7' is safer against false positives and tends to be concise but often misses important defects or offers low-value or incorrect suggestions.
See raw results [here](https://github.com/Codium-ai/pr-agent-settings/blob/main/benchmark/sonnet_37_vs_gemini-2.5-pro-preview-05-06.md)

View File

@ -1,21 +1,22 @@
# Recent Updates and Future Roadmap
`Page last updated: 2025-07-01`
`Page last updated: 2025-05-11`
This page summarizes recent enhancements to Qodo Merge (last three months).
It also outlines our development roadmap for the upcoming three months. Please note that the roadmap is subject to change, and features may be adjusted, added, or reprioritized.
=== "Recent Updates"
- **Receiving Qodo Merge feedback locally**: You can receive automatic feedback from Qodo Merge on your local IDE after each commit. ([Learn more](https://github.com/qodo-ai/agents/tree/main/agents/qodo-merge-post-commit)).
- **Mermaid Diagrams**: Qodo Merge now generates by default Mermaid diagrams for PRs, providing a visual representation of code changes. ([Learn more](https://qodo-merge-docs.qodo.ai/tools/describe/#sequence-diagram-support))
- **Best Practices Hierarchy**: Introducing support for structured best practices, such as for folders in monorepos or a unified best practice file for a group of repositories. ([Learn more](https://qodo-merge-docs.qodo.ai/tools/improve/#global-hierarchical-best-practices))
- **Simplified Free Tier**: Qodo Merge now offers a simplified free tier with a monthly limit of 75 PR reviews per organization, replacing the previous two-week trial. ([Learn more](https://qodo-merge-docs.qodo.ai/installation/qodo_merge/#cloud-users))
- **CLI Endpoint**: A new Qodo Merge endpoint that accepts a lists of before/after code changes, executes Qodo Merge commands, and return the results. Currently available for enterprise customers. Contact [Qodo](https://www.qodo.ai/contact/) for more information.
- **Linear tickets support**: Qodo Merge now supports Linear tickets. ([Learn more](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/#linear-integration))
- **Smart Update**: Upon PR updates, Qodo Merge will offer tailored code suggestions, addressing both the entire PR and the specific incremental changes since the last feedback ([Learn more](https://qodo-merge-docs.qodo.ai/core-abilities/incremental_update/))
- **Qodo Merge Pull Request Benchmark** - evaluating the performance of LLMs in analyzing pull request code ([Learn more](https://qodo-merge-docs.qodo.ai/pr_benchmark/))
- **Chat on Suggestions**: Users can now chat with Qodo Merge code suggestions ([Learn more](https://qodo-merge-docs.qodo.ai/tools/improve/#chat-on-code-suggestions))
- **Scan Repo Discussions Tool**: A new tool that analyzes past code discussions to generate a `best_practices.md` file, distilling key insights and recommendations. ([Learn more](https://qodo-merge-docs.qodo.ai/tools/scan_repo_discussions/))
- **Enhanced Models**: Qodo Merge now defaults to a combination of top models (Claude Sonnet 3.7 and Gemini 2.5 Pro) and incorporates dedicated code validation logic for improved results. ([Details 1](https://qodo-merge-docs.qodo.ai/usage-guide/qodo_merge_models/), [Details 2](https://qodo-merge-docs.qodo.ai/core-abilities/code_validation/))
- **Chrome Extension Update**: Qodo Merge Chrome extension now supports single-tenant users. ([Learn more](https://qodo-merge-docs.qodo.ai/chrome-extension/options/#configuration-options/))
- **Installation Metrics**: Upon installation, Qodo Merge analyzes past PRs for key metrics (e.g., time to merge, time to first reviewer feedback), enabling pre/post-installation comparison to calculate ROI.
=== "Future Roadmap"
- **Enhanced `review` tool**: Enhancing the `review` tool validate compliance across multiple categories including security, tickets, and custom best practices.
- **Smarter context retrieval**: Leverage AST and LSP analysis to gather relevant context from across the entire repository.
- **Enhanced portal experience**: Improved user experience in the Qodo Merge portal with new options and capabilities.
- **Smart Update**: Upon PR updates, Qodo Merge will offer tailored code suggestions, addressing both the entire PR and the specific incremental changes since the last feedback.
- **CLI Endpoint**: A new Qodo Merge endpoint will accept lists of before/after code changes, execute Qodo Merge commands, and return the results.
- **Simplified Free Tier**: We plan to transition from a two-week free trial to a free tier offering a limited number of suggestions per month per organization.
- **Best Practices Hierarchy**: Introducing support for structured best practices, such as for folders in monorepos or a unified best practice file for a group of repositories.

View File

@ -17,4 +17,4 @@ An example result:
![Analyze 1](https://codium.ai/images/pr_agent/analyze_1.png){width=750}
!!! note "Language that are currently supported:"
Python, Java, C++, JavaScript, TypeScript, C#, Go.
Python, Java, C++, JavaScript, TypeScript, C#.

View File

@ -47,45 +47,33 @@ publish_labels = true
## Preserving the original user description
By default, Qodo Merge tries to preserve your original PR description by placing it above the generated content.
By default, Qodo Merge preserves your original PR description by placing it above the generated content.
This requires including your description during the initial PR creation.
Be aware that if you edit the description while the automated tool is running, a race condition may occur, potentially causing your original description to be lost.
"Qodo removed the original description from the PR. Why"?
From our experience, there are two possible reasons:
- If you edit the description _while_ the automated tool is running, a race condition may occur, potentially causing your original description to be lost. Hence, create a description before launching the PR.
- When _updating_ PR descriptions, the `/describe` tool considers everything above the "PR Type" field as user content and will preserve it.
When updating PR descriptions, the `/describe` tool considers everything above the "PR Type" field as user content and will preserve it.
Everything below this marker is treated as previously auto-generated content and will be replaced.
![Describe comment](https://codium.ai/images/pr_agent/pr_description_user_description.png){width=512}
## Sequence Diagram Support
The `/describe` tool includes a Mermaid sequence diagram showing component/function interactions.
### Sequence Diagram Support
When the `enable_pr_diagram` option is enabled in your configuration, the `/describe` tool will include a `Mermaid` sequence diagram in the PR description.
This option is enabled by default via the `pr_description.enable_pr_diagram` param.
This diagram represents interactions between components/functions based on the diff content.
### How to enable
[//]: # (### How to enable\disable)
In your configuration:
[//]: # ()
[//]: # (In your configuration:)
[//]: # ()
[//]: # (```)
[//]: # (toml)
[//]: # ([pr_description])
[//]: # (enable_pr_diagram = true)
[//]: # (```)
```
toml
[pr_description]
enable_pr_diagram = true
```
## Configuration options
???+ example "Possible configurations"
!!! example "Possible configurations"
<table>
<tr>
@ -129,8 +117,8 @@ This option is enabled by default via the `pr_description.enable_pr_diagram` par
<td>If set to true, the file list in the "Changes walkthrough" section will be collapsible. If set to "adaptive", the file list will be collapsible only if there are more than 8 files. Default is "adaptive".</td>
</tr>
<tr>
<td><b>enable_large_pr_handling 💎</b></td>
<td>If set to true, in case of a large PR the tool will make several calls to the AI and combine them to be able to cover more files. Default is true.</td>
<td><b>enable_large_pr_handling</b></td>
<td>Pro feature. If set to true, in case of a large PR the tool will make several calls to the AI and combine them to be able to cover more files. Default is true.</td>
</tr>
<tr>
<td><b>enable_help_text</b></td>
@ -138,7 +126,7 @@ This option is enabled by default via the `pr_description.enable_pr_diagram` par
</tr>
<tr>
<td><b>enable_pr_diagram</b></td>
<td>If set to true, the tool will generate a horizontal Mermaid flowchart summarizing the main pull request changes. This field remains empty if not applicable. Default is true.</td>
<td>If set to true, the tool will generate a horizontal Mermaid flowchart summarizing the main pull request changes. This field remains empty if not applicable. Default is false.</td>
</tr>
</table>
@ -182,12 +170,9 @@ pr_agent:summary
## PR Walkthrough:
pr_agent:walkthrough
## PR Diagram:
pr_agent:diagram
```
The marker `pr_agent:type` will be replaced with the PR type, `pr_agent:summary` will be replaced with the PR summary, `pr_agent:walkthrough` will be replaced with the PR walkthrough, and `pr_agent:diagram` will be replaced with the sequence diagram (if enabled).
The marker `pr_agent:type` will be replaced with the PR type, `pr_agent:summary` will be replaced with the PR summary, and `pr_agent:walkthrough` will be replaced with the PR walkthrough.
![Describe markers before](https://codium.ai/images/pr_agent/describe_markers_before.png){width=512}
@ -199,7 +184,6 @@ becomes
- `use_description_markers`: if set to true, the tool will use markers template. It replaces every marker of the form `pr_agent:marker_name` with the relevant content. Default is false.
- `include_generated_by_header`: if set to true, the tool will add a dedicated header: 'Generated by PR Agent at ...' to any automatic content. Default is true.
- `diagram`: if present as a marker, will be replaced by the PR sequence diagram (if enabled).
## Custom labels

View File

@ -46,6 +46,9 @@ pr_commands = [
The `pr_commands` list defines commands that run automatically when a PR is opened.
Since this is under the [github_app] section, it only applies when using the Qodo Merge GitHub App in GitHub environments.
!!! note
By default, /add_docs is not triggered automatically. You must explicitly include it in pr_commands to enable this behavior.
## Configuration options
- `docs_style`: The exact style of the documentation (for python docstring). you can choose between: `google`, `numpy`, `sphinx`, `restructuredtext`, `plain`. Default is `sphinx`.

View File

@ -144,216 +144,84 @@ Use triple quotes to write multi-line instructions. Use bullet points or numbers
> `💎 feature. Platforms supported: GitHub, GitLab, Bitbucket`
Qodo Merge supports both simple and hierarchical best practices configurations to provide guidance to the AI model for generating relevant code suggestions.
Another option to give additional guidance to the AI model is by creating a `best_practices.md` file in your repository's root directory.
This page can contain a list of best practices, coding standards, and guidelines that are specific to your repo/organization.
???- tip "Writing effective best practices files"
The AI model will use this `best_practices.md` file as a reference, and in case the PR code violates any of the guidelines, it will create additional suggestions, with a dedicated label: `Organization
best practice`.
The following guidelines apply to all best practices files:
Example for a Python `best_practices.md` content:
- Write clearly and concisely
- Include brief code examples when helpful with before/after patterns
- Focus on project-specific guidelines that will result in relevant suggestions you actually want to get
- Keep each file relatively short, under 800 lines, since:
- AI models may not process effectively very long documents
- Long files tend to contain generic guidelines already known to AI
- Use pattern-based structure rather than simple bullet points for better clarity
```markdown
## Project best practices
- Make sure that I/O operations are encapsulated in a try-except block
- Use the `logging` module for logging instead of `print` statements
- Use `is` and `is not` to compare with `None`
- Use `if __name__ == '__main__':` to run the code only when the script is executed
- Use `with` statement to open files
...
```
???- tip "Example of a best practices file"
Tips for writing an effective `best_practices.md` file:
Pattern 1: Add proper error handling with try-except blocks around external function calls.
- Write clearly and concisely
- Include brief code examples when helpful
- Focus on project-specific guidelines, that will result in relevant suggestions you actually want to get
- Keep the file relatively short, under 800 lines, since:
- AI models may not process effectively very long documents
- Long files tend to contain generic guidelines already known to AI
Example code before:
To control the number of best practices suggestions generated by the `improve` tool, give the following configuration:
```python
# Some code that might raise an exception
return process_pr_data(data)
```
```toml
[best_practices]
num_best_practice_suggestions = 2
```
Example code after:
#### Local and global best practices
```python
try:
# Some code that might raise an exception
return process_pr_data(data)
except Exception as e:
logger.exception("Failed to process request", extra={"error": e})
```
By default, Qodo Merge will look for a local `best_practices.md` in the root of the relevant local repo.
Pattern 2: Add defensive null/empty checks before accessing object properties or performing operations on potentially null variables to prevent runtime errors.
If you want to enable also a global `best_practices.md` file, set first in the global configuration file:
Example code before:
```toml
[best_practices]
enable_global_best_practices = true
```
```python
def get_pr_code(pr_data):
if "changed_code" in pr_data:
return pr_data.get("changed_code", "")
return ""
```
Then, create a `best_practices.md` file in the root of [global](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/#global-configuration-file) configuration repository, `pr-agent-settings`.
Example code after:
#### Best practices for multiple languages
```python
def get_pr_code(pr_data):
if pr_data is None:
return ""
if "changed_code" in pr_data:
return pr_data.get("changed_code", "")
return ""
```
For a git organization working with multiple programming languages, you can maintain a centralized global `best_practices.md` file containing language-specific guidelines.
When reviewing pull requests, Qodo Merge automatically identifies the programming language and applies the relevant best practices from this file.
#### Local best practices
To do this, structure your `best_practices.md` file using the following format:
For basic usage, create a `best_practices.md` file in your repository's root directory containing a list of best practices, coding standards, and guidelines specific to your repository.
```
# [Python]
...
# [Java]
...
# [JavaScript]
...
```
The AI model will use this `best_practices.md` file as a reference, and in case the PR code violates any of the guidelines, it will create additional suggestions, with a dedicated label: `Organization best practice`.
#### Dedicated label for best practices suggestions
#### Global hierarchical best practices
Best practice suggestions are labeled as `Organization best practice` by default.
To customize this label, modify it in your configuration file:
```toml
[best_practices]
organization_name = "..."
```
For organizations managing multiple repositories with different requirements, Qodo Merge supports a hierarchical best practices system using a dedicated global configuration repository.
And the label will be: `{organization_name} best practice`.
**Supported scenarios:**
#### Example results
1. **Standalone repositories**: Individual repositories can have their own specific best practices tailored to their unique requirements
2. **Groups of repositories**: Repositories can be mapped to shared group-level best practices for consistent standards across similar projects
3. **Monorepos with subprojects**: Large monorepos can have both repository-level and subproject-level best practices, with automatic path-based matching
#### Setting up global hierarchical best practices
1\. Create a new repository named `pr-agent-settings` in your organization/workspace.
2\. Build the folder hierarchy in your `pr-agent-settings` repository, for example:
```bash
pr-agent-settings/
├── metadata.yaml # Maps repos/folders to best practice paths
└── codebase_standards/ # Root for all best practice definitions
├── global/ # Global rules, inherited widely
│ └── best_practices.md
├── groups/ # For groups of repositories
│ ├── frontend_repos/
│ │ └── best_practices.md
│ ├── backend_repos/
│ │ └── best_practices.md
│ └── ...
├── qodo-merge/ # For standalone repositories
│ └── best_practices.md
├── qodo-monorepo/ # For monorepo-specific rules
│ ├── best_practices.md # Root level monorepo rules
│ ├── qodo-github/ # Subproject best practices
│ │ └── best_practices.md
│ └── qodo-gitlab/ # Another subproject
│ └── best_practices.md
└── ... # More repositories
```
3\. Define the metadata file `metadata.yaml` that maps your repositories to their relevant best practices paths, for example:
```yaml
# Standalone repos
qodo-merge:
best_practices_paths:
- "qodo-merge"
# Group-associated repos
repo_b:
best_practices_paths:
- "groups/backend_repos"
# Multi-group repos
repo_c:
best_practices_paths:
- "groups/frontend_repos"
- "groups/backend_repos"
# Monorepo with subprojects
qodo-monorepo:
best_practices_paths:
- "qodo-monorepo"
monorepo_subprojects:
qodo-github:
best_practices_paths:
- "qodo-monorepo/qodo-github"
qodo-gitlab:
best_practices_paths:
- "qodo-monorepo/qodo-gitlab"
```
4\. Set the following configuration in your global configuration file:
```toml
[best_practices]
enable_global_best_practices = true
```
???- info "Best practices priority and fallback behavior"
When global best practices are enabled, Qodo Merge follows this priority order:
1\. **Primary**: Global hierarchical best practices from `pr-agent-settings` repository:
1.1 If the repository is mapped in `metadata.yaml`, it uses the specified paths
1.2 For monorepos, it automatically collects best practices matching PR file paths
1.3 If no mapping exists, it falls back to the global best practices
2\. **Fallback**: Local repository `best_practices.md` file:
2.1 Used when global best practices are not found or configured
2.2 Acts as a safety net for repositories not yet configured in the global system
2.3 Local best practices are completely ignored when global best practices are successfully loaded
???- info "Edge cases and behavior"
- **Missing paths**: If specified paths in `metadata.yaml` don't exist in the file system, those paths are skipped
- **Monorepo subproject matching**: For monorepos, Qodo Merge automatically matches PR file paths against subproject paths to apply relevant best practices
- **Multiple group inheritance**: Repositories can inherit from multiple groups, and all applicable best practices are combined
[//]: # (#### Best practices for multiple languages)
[//]: # ()
[//]: # (For a git organization working with multiple programming languages, you can maintain a centralized global `best_practices.md` file containing language-specific guidelines.)
[//]: # (When reviewing pull requests, Qodo Merge automatically identifies the programming language and applies the relevant best practices from this file.)
[//]: # ()
[//]: # (To do this, structure your `best_practices.md` file using the following format:)
[//]: # ()
[//]: # (```)
[//]: # (# [Python])
[//]: # (...)
[//]: # (# [Java])
[//]: # (...)
[//]: # (# [JavaScript])
[//]: # (...)
[//]: # (```)
???- info "Dedicated label for best practices suggestions"
Best practice suggestions are labeled as `Organization best practice` by default.
To customize this label, modify it in your configuration file:
```toml
[best_practices]
organization_name = "..."
```
And the label will be: `{organization_name} best practice`.
#### Example results
![best_practice](https://codium.ai/images/pr_agent/org_best_practice.png){width=512}
![best_practice](https://codium.ai/images/pr_agent/org_best_practice.png){width=512}
### Auto best practices
@ -437,26 +305,9 @@ dual_publishing_score_threshold = x
Where x represents the minimum score threshold (>=) for suggestions to be presented as committable PR comments in addition to the table. Default is -1 (disabled).
### Controlling suggestions depth
> `💎 feature`
You can control the depth and comprehensiveness of the code suggestions by using the `pr_code_suggestions.suggestions_depth` parameter.
Available options:
- `selective` - Shows only suggestions above a score threshold of 6
- `regular` - Default mode with balanced suggestion coverage
- `exhaustive` - Provides maximum suggestion comprehensiveness
(Alternatively, use numeric values: 1, 2, or 3 respectively)
We recommend starting with `regular` mode, then exploring `exhaustive` mode, which can provide more comprehensive suggestions and enhanced bug detection.
### Self-review
> `💎 feature. Platforms supported: GitHub, GitLab`
> `💎 feature` Platforms supported: GitHub, GitLab
If you set in a configuration file:
@ -500,6 +351,86 @@ code_suggestions_self_review_text = "... (your text here) ..."
To prevent unauthorized approvals, this configuration defaults to false, and cannot be altered through online comments; enabling requires a direct update to the configuration file and a commit to the repository. This ensures that utilizing the feature demands a deliberate documented decision by the repository owner.
### Auto-approval
> `💎 feature. Platforms supported: GitHub, GitLab, Bitbucket`
Under specific conditions, Qodo Merge can auto-approve a PR when a specific comment is invoked, or when the PR meets certain criteria.
**To ensure safety, the auto-approval feature is disabled by default.**
To enable auto-approval features, you need to actively set one or both of the following options in a pre-defined _configuration file_:
```toml
[config]
enable_comment_approval = true # For approval via comments
enable_auto_approval = true # For criteria-based auto-approval
```
!!! note "Notes"
- Note that this specific flag cannot be set with a command line argument, only in the configuration file, committed to the repository.
- Enabling auto-approval must be a deliberate decision by the repository owner.
1\. **Auto-approval by commenting**
To enable auto-approval by commenting, set in the configuration file:
```toml
[config]
enable_comment_approval = true
```
After enabling, by commenting on a PR:
```
/review auto_approve
```
Qodo Merge will automatically approve the PR, and add a comment with the approval.
2\. **Auto-approval when the PR meets certain criteria**
To enable auto-approval based on specific criteria, first, you need to enable the top-level flag:
```toml
[config]
enable_auto_approval = true
```
There are several criteria that can be set for auto-approval:
- **Review effort score**
```toml
[config]
enable_auto_approval = true
auto_approve_for_low_review_effort = X # X is a number between 1 to 5
```
When the [review effort score](https://www.qodo.ai/images/pr_agent/review3.png) is lower or equal to X, the PR will be auto-approved.
___
- **No code suggestions**
```toml
[config]
enable_auto_approval = true
auto_approve_for_no_suggestions = true
```
When no [code suggestions](https://www.qodo.ai/images/pr_agent/code_suggestions_as_comment_closed.png) were found for the PR, the PR will be auto-approved.
___
- **Ticket Compliance**
```toml
[config]
enable_auto_approval = true
ensure_ticket_compliance = true # Default is false
```
If `ensure_ticket_compliance` is set to `true`, auto-approval will be disabled if a ticket is linked to the PR and the ticket is not compliant (e.g., the `review` tool did not mark the PR as fully compliant with the ticket). This ensures that PRs are only auto-approved if their associated tickets are properly resolved.
### How many code suggestions are generated?
@ -523,7 +454,7 @@ Note: Chunking is primarily relevant for large PRs. For most PRs (up to 600 line
## Configuration options
???+ example "General options"
??? example "General options"
<table>
<tr>
@ -538,10 +469,6 @@ Note: Chunking is primarily relevant for large PRs. For most PRs (up to 600 line
<td><b>enable_chat_in_code_suggestions</b></td>
<td>If set to true, QM bot will interact with comments made on code changes it has proposed. Default is true.</td>
</tr>
<tr>
<td><b>suggestions_depth 💎</b></td>
<td> Controls the depth of the suggestions. Can be set to 'selective', 'regular', or 'exhaustive'. Default is 'regular'.</td>
</tr>
<tr>
<td><b>dual_publishing_score_threshold</b></td>
<td>Minimum score threshold for suggestions to be presented as committable PR comments in addition to the table. Default is -1 (disabled).</td>
@ -587,7 +514,7 @@ Note: Chunking is primarily relevant for large PRs. For most PRs (up to 600 line
</tr>
</table>
???+ example "Params for number of suggestions and AI calls"
??? example "Params for number of suggestions and AI calls"
<table>
<tr>

View File

@ -3,23 +3,22 @@
Here is a list of Qodo Merge tools, each with a dedicated page that explains how to use it:
| Tool | Description |
|------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
| ---------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| **[PR Description (`/describe`](./describe.md))** | Automatically generating PR description - title, type, summary, code walkthrough and labels |
| **[PR Review (`/review`](./review.md))** | Adjustable feedback about the PR, possible issues, security concerns, review effort and more |
| **[Code Suggestions (`/improve`](./improve.md))** | Code suggestions for improving the PR |
| **[Question Answering (`/ask ...`](./ask.md))** | Answering free-text questions about the PR, or on specific code lines |
| **[Help (`/help`](./help.md))** | Provides a list of all the available tools. Also enables to trigger them interactively (💎) |
| **[Help Docs (`/help_docs`](./help_docs.md))** | Answer a free-text question based on a git documentation folder. |
| **[Update Changelog (`/update_changelog`](./update_changelog.md))** | Automatically updating the CHANGELOG.md file with the PR changes |
| **[Help (`/help`](./help.md))** | Provides a list of all the available tools. Also enables to trigger them interactively (💎) |
| **💎 [Add Documentation (`/add_docs`](./documentation.md))** | Generates documentation to methods/functions/classes that changed in the PR |
| **💎 [Analyze (`/analyze`](./analyze.md))** | Identify code components that changed in the PR, and enables to interactively generate tests, docs, and code suggestions for each component |
| **💎 [CI Feedback (`/checks ci_job`](./ci_feedback.md))** | Automatically generates feedback and analysis for a failed CI job |
| **💎 [Custom Prompt (`/custom_prompt`](./custom_prompt.md))** | Automatically generates custom suggestions for improving the PR code, based on specific guidelines defined by the user |
| **💎 [Generate Custom Labels (`/generate_labels`](./custom_labels.md))** | Generates custom labels for the PR, based on specific guidelines defined by the user |
| **💎 [Generate Tests (`/test`](./test.md))** | Automatically generates unit tests for a selected component, based on the PR code changes |
| **💎 [Implement (`/implement`](./implement.md))** | Generates implementation code from review suggestions |
| **💎 [Analyze (`/analyze`](./analyze.md))** | Identify code components that changed in the PR, and enables to interactively generate tests, docs, and code suggestions for each component |
| **💎 [Test (`/test`](./test.md))** | generate tests for a selected component, based on the PR code changes |
| **💎 [Custom Prompt (`/custom_prompt`](./custom_prompt.md))** | Automatically generates custom suggestions for improving the PR code, based on specific guidelines defined by the user |
| **💎 [Generate Tests (`/test component_name`](./test.md))** | Automatically generates unit tests for a selected component, based on the PR code changes |
| **💎 [Improve Component (`/improve_component component_name`](./improve_component.md))** | Generates code suggestions for a specific code component that changed in the PR |
| **💎 [CI Feedback (`/checks ci_job`](./ci_feedback.md))** | Automatically generates feedback and analysis for a failed CI job |
| **💎 [Implement (`/implement`](./implement.md))** | Generates implementation code from review suggestions |
| **💎 [Scan Repo Discussions (`/scan_repo_discussions`](./scan_repo_discussions.md))** | Generates `best_practices.md` file based on previous discussions in the repository |
| **💎 [Similar Code (`/similar_code`](./similar_code.md))** | Retrieves the most similar code components from inside the organization's codebase, or from open-source code. |
Note that the tools marked with 💎 are available only for Qodo Merge users.

View File

@ -51,7 +51,7 @@ extra_instructions = "..."
## Configuration options
???+ example "General options"
!!! example "General options"
<table>
<tr>
@ -76,7 +76,7 @@ extra_instructions = "..."
</tr>
</table>
???+ example "Enable\\disable specific sub-sections"
!!! example "Enable\\disable specific sub-sections"
<table>
<tr>
@ -98,11 +98,6 @@ extra_instructions = "..."
<tr>
<td><b>require_security_review</b></td>
<td>If set to true, the tool will add a section that checks if the PR contains a possible security or vulnerability issue. Default is true.</td>
</tr>
<tr>
<td><b>require_todo_scan</b></td>
<td>If set to true, the tool will add a section that lists TODO comments found in the PR code changes. Default is false.
</td>
</tr>
<tr>
<td><b>require_ticket_analysis_review</b></td>
@ -110,7 +105,7 @@ extra_instructions = "..."
</tr>
</table>
???+ example "Adding PR labels"
!!! example "Adding PR labels"
You can enable\disable the `review` tool to add specific labels to the PR:
@ -160,7 +155,7 @@ extra_instructions = "..."
- **`ticket compliance`**: Adds a label indicating code compliance level ("Fully compliant" | "PR Code Verified" | "Partially compliant" | "Not compliant") to any GitHub/Jira/Linea ticket linked in the PR. Controlled by the 'require_ticket_labels' flag (default: false). If 'require_no_ticket_labels' is also enabled, PRs without ticket links will receive a "No ticket found" label.
### Auto-blocking PRs from being merged based on the generated labels
### Blocking PRs from merging based on the generated labels
!!! tip ""

View File

@ -250,15 +250,3 @@ Where the `ignore_pr_authors` is a list of usernames that you want to ignore.
!!! note
There is one specific case where bots will receive an automatic response - when they generated a PR with a _failed test_. In that case, the [`ci_feedback`](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) tool will be invoked.
### Ignoring Generated Files by Language/Framework
To automatically exclude files generated by specific languages or frameworks, you can add the following to your `configuration.toml` file:
```
[config]
ignore_language_framework = ['protobuf', ...]
```
You can view the list of auto-generated file patterns in [`generated_code_ignore.toml`](https://github.com/qodo-ai/pr-agent/blob/main/pr_agent/settings/generated_code_ignore.toml).
Files matching these glob patterns will be automatically excluded from PR Agent analysis.

View File

@ -90,7 +90,7 @@ duplicate_examples=true # will duplicate the examples in the prompt, to help the
api_base = "http://localhost:11434" # or whatever port you're running Ollama on
```
By default, Ollama uses a context window size of 2048 tokens. In most cases this is not enough to cover pr-agent prompt and pull-request diff. Context window size can be overridden with the `OLLAMA_CONTEXT_LENGTH` environment variable. For example, to set the default context length to 8K, use: `OLLAMA_CONTEXT_LENGTH=8192 ollama serve`. More information you can find on the [official ollama faq](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-specify-the-context-window-size).
By default, Ollama uses a context window size of 2048 tokens. In most cases this is not enough to cover pr-agent promt and pull-request diff. Context window size can be overridden with the `OLLAMA_CONTEXT_LENGTH` environment variable. For example, to set the default context length to 8K, use: `OLLAMA_CONTEXT_LENGTH=8192 ollama serve`. More information you can find on the [official ollama faq](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-specify-the-context-window-size).
Please note that the `custom_model_max_tokens` setting should be configured in accordance with the `OLLAMA_CONTEXT_LENGTH`. Failure to do so may result in unexpected model output.
@ -232,14 +232,6 @@ AWS_SECRET_ACCESS_KEY="..."
AWS_REGION_NAME="..."
```
You can also use the new Meta Llama 4 models available on Amazon Bedrock:
```toml
[config] # in configuration.toml
model="bedrock/us.meta.llama4-scout-17b-instruct-v1:0"
fallback_models=["bedrock/us.meta.llama4-maverick-17b-instruct-v1:0"]
```
See [litellm](https://docs.litellm.ai/docs/providers/bedrock#usage) documentation for more information about the environment variables required for Amazon Bedrock.
### DeepSeek

View File

@ -88,7 +88,7 @@ Create a dedicated project to hold a global configuration file that affects all
1. Create a new project with both the name and key: PR_AGENT_SETTINGS.
2. Inside the PR_AGENT_SETTINGS project, create a repository named pr-agent-settings.
3. In this repository, add a `.pr_agent.toml` configuration file—structured similarly to the global configuration file described above.
4. Optionally, you can add organizational-level [global best practices](https://qodo-merge-docs.qodo.ai/tools/improve/#global-hierarchical-best-practices).
4. Optionally, you can add organizational-level [global best practices file](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/#global-configuration-file).
Repositories across your entire Bitbucket organization will inherit the configuration from this file.

View File

@ -1,12 +1,12 @@
The default models used by Qodo Merge (June 2025) are a combination of Claude Sonnet 4 and Gemini 2.5 Pro.
The default models used by Qodo Merge (April 2025) are a combination of Claude Sonnet 3.7 and Gemini 2.5 Pro.
### Selecting a Specific Model
Users can configure Qodo Merge to use only a specific model by editing the [configuration](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/) file.
The models supported by Qodo Merge are:
- `claude-4-sonnet`
- `claude-3-7-sonnet`
- `o4-mini`
- `gpt-4.1`
- `gemini-2.5-pro`

View File

@ -23,32 +23,30 @@ nav:
- Frequently Asked Questions: 'faq/index.md'
- 💎 Qodo Merge Models: 'usage-guide/qodo_merge_models.md'
- Tools:
- 'tools/index.md'
- Describe: 'tools/describe.md'
- Review: 'tools/review.md'
- Improve: 'tools/improve.md'
- Ask: 'tools/ask.md'
- Help: 'tools/help.md'
- Help Docs: 'tools/help_docs.md'
- Update Changelog: 'tools/update_changelog.md'
- 💎 Add Documentation: 'tools/documentation.md'
- 💎 Analyze: 'tools/analyze.md'
- 💎 CI Feedback: 'tools/ci_feedback.md'
- 💎 Custom Prompt: 'tools/custom_prompt.md'
- 💎 Generate Labels: 'tools/custom_labels.md'
- 💎 Generate Tests: 'tools/test.md'
- 💎 Implement: 'tools/implement.md'
- 💎 Improve Components: 'tools/improve_component.md'
- 💎 Scan Repo Discussions: 'tools/scan_repo_discussions.md'
- 💎 Similar Code: 'tools/similar_code.md'
- 'tools/index.md'
- Describe: 'tools/describe.md'
- Review: 'tools/review.md'
- Improve: 'tools/improve.md'
- Ask: 'tools/ask.md'
- Update Changelog: 'tools/update_changelog.md'
- Help Docs: 'tools/help_docs.md'
- Help: 'tools/help.md'
- 💎 Analyze: 'tools/analyze.md'
- 💎 Test: 'tools/test.md'
- 💎 Improve Component: 'tools/improve_component.md'
- 💎 Documentation: 'tools/documentation.md'
- 💎 Custom Labels: 'tools/custom_labels.md'
- 💎 Custom Prompt: 'tools/custom_prompt.md'
- 💎 CI Feedback: 'tools/ci_feedback.md'
- 💎 Similar Code: 'tools/similar_code.md'
- 💎 Implement: 'tools/implement.md'
- 💎 Scan Repo Discussions: 'tools/scan_repo_discussions.md'
- Core Abilities:
- 'core-abilities/index.md'
- Auto approval: 'core-abilities/auto_approval.md'
- Auto best practices: 'core-abilities/auto_best_practices.md'
- Chat on code suggestions: 'core-abilities/chat_on_code_suggestions.md'
- Chrome extension: 'chrome-extension/index.md'
- Code validation: 'core-abilities/code_validation.md'
# - Compression strategy: 'core-abilities/compression_strategy.md'
- Compression strategy: 'core-abilities/compression_strategy.md'
- Dynamic context: 'core-abilities/dynamic_context.md'
- Fetching ticket context: 'core-abilities/fetching_ticket_context.md'
- Impact evaluation: 'core-abilities/impact_evaluation.md'
@ -58,11 +56,11 @@ nav:
- RAG context enrichment: 'core-abilities/rag_context_enrichment.md'
- Self-reflection: 'core-abilities/self_reflection.md'
- Static code analysis: 'core-abilities/static_code_analysis.md'
# - Chrome Extension:
# - Qodo Merge Chrome Extension: 'chrome-extension/index.md'
# - Features: 'chrome-extension/features.md'
# - Data Privacy: 'chrome-extension/data_privacy.md'
# - Options: 'chrome-extension/options.md'
- Chrome Extension:
- Qodo Merge Chrome Extension: 'chrome-extension/index.md'
- Features: 'chrome-extension/features.md'
- Data Privacy: 'chrome-extension/data_privacy.md'
- Options: 'chrome-extension/options.md'
- PR Benchmark:
- PR Benchmark: 'pr_benchmark/index.md'
- Recent Updates:

View File

@ -51,7 +51,7 @@ class PRAgent:
def __init__(self, ai_handler: partial[BaseAiHandler,] = LiteLLMAIHandler):
self.ai_handler = ai_handler # will be initialized in run_action
async def _handle_request(self, pr_url, request, notify=None) -> bool:
async def handle_request(self, pr_url, request, notify=None) -> bool:
# First, apply repo specific settings if exists
apply_repo_settings(pr_url)
@ -117,10 +117,3 @@ class PRAgent:
else:
return False
return True
async def handle_request(self, pr_url, request, notify=None) -> bool:
try:
return await self._handle_request(pr_url, request, notify)
except:
get_logger().exception("Failed to process the command.")
return False

View File

@ -61,24 +61,18 @@ MAX_TOKENS = {
'vertex_ai/gemini-1.5-pro': 1048576,
'vertex_ai/gemini-2.5-pro-preview-03-25': 1048576,
'vertex_ai/gemini-2.5-pro-preview-05-06': 1048576,
'vertex_ai/gemini-2.5-pro-preview-06-05': 1048576,
'vertex_ai/gemini-2.5-pro': 1048576,
'vertex_ai/gemini-1.5-flash': 1048576,
'vertex_ai/gemini-2.0-flash': 1048576,
'vertex_ai/gemini-2.5-flash-preview-04-17': 1048576,
'vertex_ai/gemini-2.5-flash-preview-05-20': 1048576,
'vertex_ai/gemini-2.5-flash': 1048576,
'vertex_ai/gemma2': 8200,
'gemini/gemini-1.5-pro': 1048576,
'gemini/gemini-1.5-flash': 1048576,
'gemini/gemini-2.0-flash': 1048576,
'gemini/gemini-2.5-flash-preview-04-17': 1048576,
'gemini/gemini-2.5-flash-preview-05-20': 1048576,
'gemini/gemini-2.5-flash': 1048576,
'gemini/gemini-2.5-pro-preview-03-25': 1048576,
'gemini/gemini-2.5-pro-preview-05-06': 1048576,
'gemini/gemini-2.5-pro-preview-06-05': 1048576,
'gemini/gemini-2.5-pro': 1048576,
'codechat-bison': 6144,
'codechat-bison-32k': 32000,
'anthropic.claude-instant-v1': 100000,
@ -107,14 +101,9 @@ MAX_TOKENS = {
"bedrock/us.anthropic.claude-3-5-sonnet-20241022-v2:0": 100000,
"bedrock/us.anthropic.claude-3-7-sonnet-20250219-v1:0": 200000,
"bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0": 200000,
"bedrock/apac.anthropic.claude-3-5-sonnet-20241022-v2:0": 100000,
"bedrock/apac.anthropic.claude-3-7-sonnet-20250219-v1:0": 200000,
"bedrock/apac.anthropic.claude-sonnet-4-20250514-v1:0": 200000,
'claude-3-5-sonnet': 100000,
'groq/meta-llama/llama-4-scout-17b-16e-instruct': 131072,
'groq/meta-llama/llama-4-maverick-17b-128e-instruct': 131072,
'bedrock/us.meta.llama4-scout-17b-instruct-v1:0': 128000,
'bedrock/us.meta.llama4-maverick-17b-instruct-v1:0': 128000,
'groq/llama3-8b-8192': 8192,
'groq/llama3-70b-8192': 8192,
'groq/llama-3.1-8b-instant': 8192,

View File

@ -131,7 +131,7 @@ class LiteLLMAIHandler(BaseAiHandler):
self.api_base = openrouter_api_base
litellm.api_base = openrouter_api_base
# Models that only use user message
# Models that only use user meessage
self.user_message_only_models = USER_MESSAGE_ONLY_MODELS
# Model that doesn't support temperature argument
@ -212,7 +212,7 @@ class LiteLLMAIHandler(BaseAiHandler):
return kwargs
def add_litellm_callbacks(self, kwargs) -> dict:
def add_litellm_callbacks(selfs, kwargs) -> dict:
captured_extra = []
def capture_logs(message):

View File

@ -2,7 +2,6 @@ import fnmatch
import re
from pr_agent.config_loader import get_settings
from pr_agent.log import get_logger
def filter_ignored(files, platform = 'github'):
@ -18,17 +17,7 @@ def filter_ignored(files, platform = 'github'):
glob_setting = get_settings().ignore.glob
if isinstance(glob_setting, str): # --ignore.glob=[.*utils.py], --ignore.glob=.*utils.py
glob_setting = glob_setting.strip('[]').split(",")
patterns += translate_globs_to_regexes(glob_setting)
code_generators = get_settings().config.get('ignore_language_framework', [])
if isinstance(code_generators, str):
get_logger().warning("'ignore_language_framework' should be a list. Skipping language framework filtering.")
code_generators = []
for cg in code_generators:
glob_patterns = get_settings().generated_code.get(cg, [])
if isinstance(glob_patterns, str):
glob_patterns = [glob_patterns]
patterns += translate_globs_to_regexes(glob_patterns)
patterns += [fnmatch.translate(glob) for glob in glob_setting]
# compile all valid patterns
compiled_patterns = []
@ -77,11 +66,3 @@ def filter_ignored(files, platform = 'github'):
print(f"Could not filter file list: {e}")
return files
def translate_globs_to_regexes(globs: list):
regexes = []
for pattern in globs:
regexes.append(fnmatch.translate(pattern))
if pattern.startswith("**/"): # cover root-level files
regexes.append(fnmatch.translate(pattern[3:]))
return regexes

View File

@ -1,6 +1,5 @@
from __future__ import annotations
import ast
import copy
import difflib
import hashlib
@ -15,7 +14,7 @@ import traceback
from datetime import datetime
from enum import Enum
from importlib.metadata import PackageNotFoundError, version
from typing import Any, List, Tuple, TypedDict
from typing import Any, List, Tuple
import html2text
import requests
@ -38,31 +37,21 @@ def get_model(model_type: str = "model_weak") -> str:
return get_settings().config.model_reasoning
return get_settings().config.model
class Range(BaseModel):
line_start: int # should be 0-indexed
line_end: int
column_start: int = -1
column_end: int = -1
class ModelType(str, Enum):
REGULAR = "regular"
WEAK = "weak"
REASONING = "reasoning"
class TodoItem(TypedDict):
relevant_file: str
line_range: Tuple[int, int]
content: str
class PRReviewHeader(str, Enum):
REGULAR = "## PR Reviewer Guide"
INCREMENTAL = "## Incremental PR Reviewer Guide"
class ReasoningEffort(str, Enum):
HIGH = "high"
MEDIUM = "medium"
@ -120,7 +109,6 @@ def unique_strings(input_list: List[str]) -> List[str]:
seen.add(item)
return unique_list
def convert_to_markdown_v2(output_data: dict,
gfm_supported: bool = True,
incremental_review=None,
@ -143,7 +131,6 @@ def convert_to_markdown_v2(output_data: dict,
"Focused PR": "",
"Relevant ticket": "🎫",
"Security concerns": "🔒",
"Todo sections": "📝",
"Insights from user's answers": "📝",
"Code feedback": "🤖",
"Estimated effort to review [1-5]": "⏱️",
@ -164,7 +151,6 @@ def convert_to_markdown_v2(output_data: dict,
if gfm_supported:
markdown_text += "<table>\n"
todo_summary = output_data['review'].pop('todo_summary', '')
for key, value in output_data['review'].items():
if value is None or value == '' or value == {} or value == []:
if key.lower() not in ['can_be_split', 'key_issues_to_review']:
@ -223,23 +209,6 @@ def convert_to_markdown_v2(output_data: dict,
markdown_text += f"### {emoji} Security concerns\n\n"
value = emphasize_header(value.strip(), only_markdown=True)
markdown_text += f"{value}\n\n"
elif 'todo sections' in key_nice.lower():
if gfm_supported:
markdown_text += "<tr><td>"
if is_value_no(value):
markdown_text += f"✅&nbsp;<strong>No TODO sections</strong>"
else:
markdown_todo_items = format_todo_items(value, git_provider, gfm_supported)
markdown_text += f"{emoji}&nbsp;<strong>TODO sections</strong>\n<br><br>\n"
markdown_text += markdown_todo_items
markdown_text += "</td></tr>\n"
else:
if is_value_no(value):
markdown_text += f"### ✅ No TODO sections\n\n"
else:
markdown_todo_items = format_todo_items(value, git_provider, gfm_supported)
markdown_text += f"### {emoji} TODO sections\n\n"
markdown_text += markdown_todo_items
elif 'can be split' in key_nice.lower():
if gfm_supported:
markdown_text += f"<tr><td>"
@ -1320,7 +1289,7 @@ def process_description(description_full: str) -> Tuple[str, List]:
pattern_back = r'<details>\s*<summary><strong>(.*?)</strong><dd><code>(.*?)</code>.*?</summary>\s*<hr>\s*(.*?)\n\n\s*(.*?)</details>'
res = re.search(pattern_back, file_data, re.DOTALL)
if not res or res.lastindex != 4:
pattern_back = r'<details>\s*<summary><strong>(.*?)</strong>\s*<dd><code>(.*?)</code>.*?</summary>\s*<hr>\s*(.*?)\s*-\s*(.*?)\s*</details>' # looking for hyphen ('- ')
pattern_back = r'<details>\s*<summary><strong>(.*?)</strong>\s*<dd><code>(.*?)</code>.*?</summary>\s*<hr>\s*(.*?)\s*-\s*(.*?)\s*</details>' # looking for hypen ('- ')
res = re.search(pattern_back, file_data, re.DOTALL)
if res and res.lastindex == 4:
short_filename = res.group(1).strip()
@ -1398,47 +1367,3 @@ def set_file_languages(diff_files) -> List[FilePatchInfo]:
get_logger().exception(f"Failed to set file languages: {e}")
return diff_files
def format_todo_item(todo_item: TodoItem, git_provider, gfm_supported) -> str:
relevant_file = todo_item.get('relevant_file', '').strip()
line_number = todo_item.get('line_number', '')
content = todo_item.get('content', '')
reference_link = git_provider.get_line_link(relevant_file, line_number, line_number)
file_ref = f"{relevant_file} [{line_number}]"
if reference_link:
if gfm_supported:
file_ref = f"<a href='{reference_link}'>{file_ref}</a>"
else:
file_ref = f"[{file_ref}]({reference_link})"
if content:
return f"{file_ref}: {content.strip()}"
else:
# if content is empty, return only the file reference
return file_ref
def format_todo_items(value: list[TodoItem] | TodoItem, git_provider, gfm_supported) -> str:
markdown_text = ""
MAX_ITEMS = 5 # limit the number of items to display
if gfm_supported:
if isinstance(value, list):
markdown_text += "<ul>\n"
if len(value) > MAX_ITEMS:
get_logger().debug(f"Truncating todo items to {MAX_ITEMS} items")
value = value[:MAX_ITEMS]
for todo_item in value:
markdown_text += f"<li>{format_todo_item(todo_item, git_provider, gfm_supported)}</li>\n"
markdown_text += "</ul>\n"
else:
markdown_text += f"<p>{format_todo_item(value, git_provider, gfm_supported)}</p>\n"
else:
if isinstance(value, list):
if len(value) > MAX_ITEMS:
get_logger().debug(f"Truncating todo items to {MAX_ITEMS} items")
value = value[:MAX_ITEMS]
for todo_item in value:
markdown_text += f"- {format_todo_item(todo_item, git_provider, gfm_supported)}\n"
else:
markdown_text += f"- {format_todo_item(value, git_provider, gfm_supported)}\n"
return markdown_text

View File

@ -14,7 +14,6 @@ global_settings = Dynaconf(
settings_files=[join(current_dir, f) for f in [
"settings/configuration.toml",
"settings/ignore.toml",
"settings/generated_code_ignore.toml",
"settings/language_extensions.toml",
"settings/pr_reviewer_prompts.toml",
"settings/pr_questions_prompts.toml",
@ -82,62 +81,3 @@ def _find_pyproject() -> Optional[Path]:
pyproject_path = _find_pyproject()
if pyproject_path is not None:
get_settings().load_file(pyproject_path, env=f'tool.{PR_AGENT_TOML_KEY}')
def apply_secrets_manager_config():
"""
Retrieve configuration from AWS Secrets Manager and override existing settings
"""
try:
# Dynamic imports to avoid circular dependency (secret_providers imports config_loader)
from pr_agent.secret_providers import get_secret_provider
from pr_agent.log import get_logger
secret_provider = get_secret_provider()
if not secret_provider:
return
if (hasattr(secret_provider, 'get_all_secrets') and
get_settings().get("CONFIG.SECRET_PROVIDER") == 'aws_secrets_manager'):
try:
secrets = secret_provider.get_all_secrets()
if secrets:
apply_secrets_to_config(secrets)
get_logger().info("Applied AWS Secrets Manager configuration")
except Exception as e:
get_logger().error(f"Failed to apply AWS Secrets Manager config: {e}")
except Exception as e:
try:
from pr_agent.log import get_logger
get_logger().debug(f"Secret provider not configured: {e}")
except:
# Fail completely silently if log module is not available
pass
def apply_secrets_to_config(secrets: dict):
"""
Apply secret dictionary to configuration
"""
try:
# Dynamic import to avoid potential circular dependency
from pr_agent.log import get_logger
except:
def get_logger():
class DummyLogger:
def debug(self, msg): pass
return DummyLogger()
for key, value in secrets.items():
if '.' in key: # nested key like "openai.key"
parts = key.split('.')
if len(parts) == 2:
section, setting = parts
section_upper = section.upper()
setting_upper = setting.upper()
# Set only when no existing value (prioritize environment variables)
current_value = get_settings().get(f"{section_upper}.{setting_upper}")
if current_value is None or current_value == "":
get_settings().set(f"{section_upper}.{setting_upper}", value)
get_logger().debug(f"Set {section}.{setting} from AWS Secrets Manager")

View File

@ -22,7 +22,6 @@ try:
from azure.devops.connection import Connection
# noinspection PyUnresolvedReferences
from azure.devops.released.git import (Comment, CommentThread, GitPullRequest, GitVersionDescriptor, GitClient, CommentThreadContext, CommentPosition)
from azure.devops.released.work_item_tracking import WorkItemTrackingClient
# noinspection PyUnresolvedReferences
from azure.identity import DefaultAzureCredential
from msrest.authentication import BasicAuthentication
@ -40,7 +39,7 @@ class AzureDevopsProvider(GitProvider):
"Azure DevOps provider is not available. Please install the required dependencies."
)
self.azure_devops_client, self.azure_devops_board_client = self._get_azure_devops_client()
self.azure_devops_client = self._get_azure_devops_client()
self.diff_files = None
self.workspace_slug = None
self.repo_slug = None
@ -351,7 +350,8 @@ class AzureDevopsProvider(GitProvider):
get_logger().debug(f"Skipping publish_comment for temporary comment: {pr_comment}")
return None
comment = Comment(content=pr_comment)
thread = CommentThread(comments=[comment], thread_context=thread_context, status="closed")
# Set status to 'active' to prevent auto-resolve (see CommentThreadStatus docs)
thread = CommentThread(comments=[comment], thread_context=thread_context, status='active')
thread_response = self.azure_devops_client.create_thread(
comment_thread=thread,
project=self.workspace_slug,
@ -567,7 +567,7 @@ class AzureDevopsProvider(GitProvider):
return workspace_slug, repo_slug, pr_number
@staticmethod
def _get_azure_devops_client() -> Tuple[GitClient, WorkItemTrackingClient]:
def _get_azure_devops_client() -> GitClient:
org = get_settings().azure_devops.get("org", None)
pat = get_settings().azure_devops.get("pat", None)
@ -589,12 +589,13 @@ class AzureDevopsProvider(GitProvider):
get_logger().error(f"No PAT found in settings, and Azure Default Authentication failed, error: {e}")
raise
credentials = BasicAuthentication("", auth_token)
credentials = BasicAuthentication("", auth_token)
azure_devops_connection = Connection(base_url=org, creds=credentials)
azure_devops_client = azure_devops_connection.clients.get_git_client()
azure_devops_board_client = azure_devops_connection.clients.get_work_item_tracking_client()
return azure_devops_client, azure_devops_board_client
return azure_devops_client
def _get_repo(self):
if self.repo is None:
@ -636,49 +637,3 @@ class AzureDevopsProvider(GitProvider):
url = self.azure_devops_client.normalized_url + "/" + self.workspace_slug + "/_git/" + self.repo_slug + "/commit/" + last.commit_id
return url
def get_linked_work_items(self) -> list:
"""
Get linked work items from the PR.
"""
try:
work_items = self.azure_devops_client.get_pull_request_work_item_refs(
project=self.workspace_slug,
repository_id=self.repo_slug,
pull_request_id=self.pr_num,
)
ids = [work_item.id for work_item in work_items]
if not work_items:
return []
items = self.get_work_items(ids)
return items
except Exception as e:
get_logger().exception(f"Failed to get linked work items, error: {e}")
return []
def get_work_items(self, work_item_ids: list) -> list:
"""
Get work items by their IDs.
"""
try:
raw_work_items = self.azure_devops_board_client.get_work_items(
project=self.workspace_slug,
ids=work_item_ids,
)
work_items = []
for item in raw_work_items:
work_items.append(
{
"id": item.id,
"title": item.fields.get("System.Title", ""),
"url": item.url,
"body": item.fields.get("System.Description", ""),
"acceptance_criteria": item.fields.get(
"Microsoft.VSTS.Common.AcceptanceCriteria", ""
),
"tags": item.fields.get("System.Tags", "").split("; ") if item.fields.get("System.Tags") else [],
}
)
return work_items
except Exception as e:
get_logger().exception(f"Failed to get work items, error: {e}")
return []

View File

@ -86,7 +86,7 @@ class BitbucketServerProvider(GitProvider):
def get_repo_settings(self):
try:
content = self.bitbucket_client.get_content_of_file(self.workspace_slug, self.repo_slug, ".pr_agent.toml")
content = self.bitbucket_client.get_content_of_file(self.workspace_slug, self.repo_slug, ".pr_agent.toml", self.get_pr_branch())
return content
except Exception as e:

View File

@ -103,7 +103,7 @@ def prepare_repo(url: urllib3.util.Url, project, refspec):
repo_url = (f"{url.scheme}://{url.auth}@{url.host}:{url.port}/{project}")
directory = pathlib.Path(mkdtemp())
clone(repo_url, directory)
clone(repo_url, directory),
fetch(repo_url, refspec, cwd=directory)
checkout(cwd=directory)
return directory

View File

@ -41,12 +41,6 @@ class GiteaProvider(GitProvider):
configuration.host = "{}/api/v1".format(self.base_url)
configuration.api_key['Authorization'] = f'token {gitea_access_token}'
if get_settings().get("GITEA.SKIP_SSL_VERIFICATION", False):
configuration.verify_ssl = False
# Use custom cert (self-signed)
configuration.ssl_ca_cert = get_settings().get("GITEA.SSL_CA_CERT", None)
client = giteapy.ApiClient(configuration)
self.repo_api = RepoApi(client)
self.owner = None
@ -415,7 +409,7 @@ class GiteaProvider(GitProvider):
def _get_file_content_from_base(self, filename: str) -> str:
return self.repo_api.get_file_content(
owner=self.owner,
repo=self.repo,
repo=self.base_ref,
commit_sha=self.base_sha,
filepath=filename
)
@ -423,7 +417,7 @@ class GiteaProvider(GitProvider):
def _get_file_content_from_latest_commit(self, filename: str) -> str:
return self.repo_api.get_file_content(
owner=self.owner,
repo=self.repo,
repo=self.base_ref,
commit_sha=self.last_commit.sha,
filepath=filename
)
@ -477,11 +471,11 @@ class GiteaProvider(GitProvider):
if status == 'added':
edit_type = EDIT_TYPE.ADDED
elif status == 'removed' or status == 'deleted':
elif status == 'removed':
edit_type = EDIT_TYPE.DELETED
elif status == 'renamed':
edit_type = EDIT_TYPE.RENAMED
elif status == 'modified' or status == 'changed':
elif status == 'modified':
edit_type = EDIT_TYPE.MODIFIED
else:
self.logger.error(f"Unknown edit type: {status}")

View File

@ -1,17 +1,16 @@
import difflib
import hashlib
import re
from typing import Optional, Tuple, Any, Union
from urllib.parse import urlparse, parse_qs
from typing import Optional, Tuple
from urllib.parse import urlparse
import gitlab
import requests
from gitlab import GitlabGetError, GitlabAuthenticationError, GitlabCreateError, GitlabUpdateError
from gitlab import GitlabGetError
from pr_agent.algo.types import EDIT_TYPE, FilePatchInfo
from ..algo.file_filter import filter_ignored
from ..algo.git_patch_processing import decode_if_bytes
from ..algo.language_handler import is_valid_file
from ..algo.utils import (clip_tokens,
find_line_number_of_relevant_line_in_file,
@ -113,50 +112,14 @@ class GitLabProvider(GitProvider):
get_logger().error(f"Could not get diff for merge request {self.id_mr}")
raise DiffNotFoundError(f"Could not get diff for merge request {self.id_mr}") from e
def get_pr_file_content(self, file_path: str, branch: str) -> str:
try:
file_obj = self.gl.projects.get(self.id_project).files.get(file_path, branch)
content = file_obj.decode()
return decode_if_bytes(content)
return self.gl.projects.get(self.id_project).files.get(file_path, branch).decode()
except GitlabGetError:
# In case of file creation the method returns GitlabGetError (404 file not found).
# In this case we return an empty string for the diff.
return ''
except Exception as e:
get_logger().warning(f"Error retrieving file {file_path} from branch {branch}: {e}")
return ''
def create_or_update_pr_file(self, file_path: str, branch: str, contents="", message="") -> None:
"""Create or update a file in the GitLab repository."""
try:
project = self.gl.projects.get(self.id_project)
if not message:
action = "Update" if contents else "Create"
message = f"{action} {file_path}"
try:
existing_file = project.files.get(file_path, branch)
existing_file.content = contents
existing_file.save(branch=branch, commit_message=message)
get_logger().debug(f"Updated file {file_path} in branch {branch}")
except GitlabGetError:
project.files.create({
'file_path': file_path,
'branch': branch,
'content': contents,
'commit_message': message
})
get_logger().debug(f"Created file {file_path} in branch {branch}")
except GitlabAuthenticationError as e:
get_logger().error(f"Authentication failed while creating/updating file {file_path} in branch {branch}: {e}")
raise
except (GitlabCreateError, GitlabUpdateError) as e:
get_logger().error(f"Permission denied or validation error for file {file_path} in branch {branch}: {e}")
raise
except Exception as e:
get_logger().exception(f"Unexpected error creating/updating file {file_path} in branch {branch}: {e}")
raise
def get_diff_files(self) -> list[FilePatchInfo]:
"""
@ -204,9 +167,14 @@ class GitLabProvider(GitProvider):
original_file_content_str = ''
new_file_content_str = ''
# Ensure content is properly decoded
original_file_content_str = decode_if_bytes(original_file_content_str)
new_file_content_str = decode_if_bytes(new_file_content_str)
try:
if isinstance(original_file_content_str, bytes):
original_file_content_str = bytes.decode(original_file_content_str, 'utf-8')
if isinstance(new_file_content_str, bytes):
new_file_content_str = bytes.decode(new_file_content_str, 'utf-8')
except UnicodeDecodeError:
get_logger().warning(
f"Cannot decode file {diff['old_path']} or {diff['new_path']} in merge request {self.id_mr}")
edit_type = EDIT_TYPE.MODIFIED
if diff['new_file']:

View File

@ -13,12 +13,5 @@ def get_secret_provider():
return GoogleCloudStorageSecretProvider()
except Exception as e:
raise ValueError(f"Failed to initialize google_cloud_storage secret provider {provider_id}") from e
elif provider_id == 'aws_secrets_manager':
try:
from pr_agent.secret_providers.aws_secrets_manager_provider import \
AWSSecretsManagerProvider
return AWSSecretsManagerProvider()
except Exception as e:
raise ValueError(f"Failed to initialize aws_secrets_manager secret provider {provider_id}") from e
else:
raise ValueError("Unknown SECRET_PROVIDER")

View File

@ -1,57 +0,0 @@
import json
import boto3
from botocore.exceptions import ClientError
from pr_agent.config_loader import get_settings
from pr_agent.log import get_logger
from pr_agent.secret_providers.secret_provider import SecretProvider
class AWSSecretsManagerProvider(SecretProvider):
def __init__(self):
try:
region_name = get_settings().get("aws_secrets_manager.region_name") or \
get_settings().get("aws.AWS_REGION_NAME")
if region_name:
self.client = boto3.client('secretsmanager', region_name=region_name)
else:
self.client = boto3.client('secretsmanager')
self.secret_arn = get_settings().get("aws_secrets_manager.secret_arn")
if not self.secret_arn:
raise ValueError("AWS Secrets Manager ARN is not configured")
except Exception as e:
get_logger().error(f"Failed to initialize AWS Secrets Manager Provider: {e}")
raise e
def get_secret(self, secret_name: str) -> str:
"""
Retrieve individual secret by name (for webhook tokens)
"""
try:
response = self.client.get_secret_value(SecretId=secret_name)
return response['SecretString']
except Exception as e:
get_logger().warning(f"Failed to get secret {secret_name} from AWS Secrets Manager: {e}")
return ""
def get_all_secrets(self) -> dict:
"""
Retrieve all secrets for configuration override
"""
try:
response = self.client.get_secret_value(SecretId=self.secret_arn)
return json.loads(response['SecretString'])
except Exception as e:
get_logger().error(f"Failed to get secrets from AWS Secrets Manager {self.secret_arn}: {e}")
return {}
def store_secret(self, secret_name: str, secret_value: str):
try:
self.client.put_secret_value(
SecretId=secret_name,
SecretString=secret_value
)
except Exception as e:
get_logger().error(f"Failed to store secret {secret_name} in AWS Secrets Manager: {e}")
raise e

View File

@ -1,38 +0,0 @@
{
"name": "Qodo Merge",
"description": "Qodo Merge",
"key": "app_key",
"vendor": {
"name": "Qodo",
"url": "https://qodo.ai"
},
"authentication": {
"type": "jwt"
},
"baseUrl": "base_url",
"lifecycle": {
"installed": "/installed",
"uninstalled": "/uninstalled"
},
"scopes": [
"account",
"repository:write",
"pullrequest:write",
"wiki"
],
"contexts": [
"account"
],
"modules": {
"webhooks": [
{
"event": "*",
"url": "/webhook"
}
]
},
"links": {
"privacy": "https://qodo.ai/privacy-policy",
"terms": "https://qodo.ai/terms"
}
}

View File

@ -30,9 +30,5 @@
"url": "/webhook"
}
]
},
"links": {
"privacy": "https://qodo.ai/privacy-policy",
"terms": "https://qodo.ai/terms"
}
}

View File

@ -80,30 +80,6 @@ async def run_action():
except Exception as e:
get_logger().info(f"github action: failed to apply repo settings: {e}")
# Append the response language in the extra instructions
try:
response_language = get_settings().config.get('response_language', 'en-us')
if response_language.lower() != 'en-us':
get_logger().info(f'User has set the response language to: {response_language}')
lang_instruction_text = f"Your response MUST be written in the language corresponding to locale code: '{response_language}'. This is crucial."
separator_text = "\n======\n\nIn addition, "
for key in get_settings():
setting = get_settings().get(key)
if str(type(setting)) == "<class 'dynaconf.utils.boxing.DynaBox'>":
if key.lower() in ['pr_description', 'pr_code_suggestions', 'pr_reviewer']:
if hasattr(setting, 'extra_instructions'):
extra_instructions = setting.extra_instructions
if lang_instruction_text not in str(extra_instructions):
updated_instructions = (
str(extra_instructions) + separator_text + lang_instruction_text
if extra_instructions else lang_instruction_text
)
setting.extra_instructions = updated_instructions
except Exception as e:
get_logger().info(f"github action: failed to apply language-specific instructions: {e}")
# Handle pull request opened event
if GITHUB_EVENT_NAME == "pull_request" or GITHUB_EVENT_NAME == "pull_request_target":
action = event_payload.get("action")
@ -126,7 +102,7 @@ async def run_action():
auto_improve = get_setting_or_env("GITHUB_ACTION_CONFIG.AUTO_IMPROVE", None)
# Set the configuration for auto actions
get_settings().config.is_auto_command = True # Set the flag to indicate that the command is auto
get_settings().config.is_auto_command = True # Set the flag to indicate that the command is auto
get_settings().pr_description.final_update_message = False # No final update message when auto_describe is enabled
get_logger().info(f"Running auto actions: auto_describe={auto_describe}, auto_review={auto_review}, auto_improve={auto_improve}")

View File

@ -1,27 +0,0 @@
from fastapi import FastAPI
from mangum import Mangum
from starlette.middleware import Middleware
from starlette_context.middleware import RawContextMiddleware
from pr_agent.servers.github_app import router
try:
from pr_agent.config_loader import apply_secrets_manager_config
apply_secrets_manager_config()
except Exception as e:
try:
from pr_agent.log import get_logger
get_logger().debug(f"AWS Secrets Manager initialization failed, falling back to environment variables: {e}")
except:
# Fail completely silently if log module is not available
pass
middleware = [Middleware(RawContextMiddleware)]
app = FastAPI(middleware=middleware)
app.include_router(router)
handler = Mangum(app, lifespan="off")
def lambda_handler(event, context):
return handler(event, context)

View File

@ -1,27 +0,0 @@
from fastapi import FastAPI
from mangum import Mangum
from starlette.middleware import Middleware
from starlette_context.middleware import RawContextMiddleware
from pr_agent.servers.gitlab_webhook import router
try:
from pr_agent.config_loader import apply_secrets_manager_config
apply_secrets_manager_config()
except Exception as e:
try:
from pr_agent.log import get_logger
get_logger().debug(f"AWS Secrets Manager initialization failed, falling back to environment variables: {e}")
except:
# Fail completely silently if log module is not available
pass
middleware = [Middleware(RawContextMiddleware)]
app = FastAPI(middleware=middleware)
app.include_router(router)
handler = Mangum(app, lifespan="off")
def lambda_handler(event, context):
return handler(event, context)

View File

@ -234,9 +234,6 @@ async def gitlab_webhook(background_tasks: BackgroundTasks, request: Request):
get_logger().info(f"Skipping draft MR: {url}")
return JSONResponse(status_code=status.HTTP_200_OK, content=jsonable_encoder({"message": "success"}))
# Apply repo settings before checking push commands or handle_push_trigger
apply_repo_settings(url)
commands_on_push = get_settings().get(f"gitlab.push_commands", {})
handle_push_trigger = get_settings().get(f"gitlab.handle_push_trigger", False)
if not commands_on_push or not handle_push_trigger:
@ -285,8 +282,8 @@ def handle_ask_line(body, data):
question = body.replace('/ask', '').strip()
path = data['object_attributes']['position']['new_path']
side = 'RIGHT' # if line_range_['start']['type'] == 'new' else 'LEFT'
_id = data['object_attributes']["discussion_id"]
get_logger().info("Handling line ")
comment_id = data['object_attributes']["discussion_id"]
get_logger().info("Handling line comment")
body = f"/ask_line --line_start={start_line} --line_end={end_line} --side={side} --file_name={path} --comment_id={comment_id} {question}"
except Exception as e:
get_logger().error(f"Failed to handle ask line comment: {e}")

View File

@ -81,7 +81,6 @@ the tool will replace every marker of the form `pr_agent:marker_name` in the PR
- `type`: the PR type.
- `summary`: the PR summary.
- `walkthrough`: the PR walkthrough.
- `diagram`: the PR sequence diagram (if enabled).
Note that when markers are enabled, if the original PR description does not contain any markers, the tool will not alter the description at all.

View File

@ -0,0 +1,16 @@
from fastapi import FastAPI
from mangum import Mangum
from starlette.middleware import Middleware
from starlette_context.middleware import RawContextMiddleware
from pr_agent.servers.github_app import router
middleware = [Middleware(RawContextMiddleware)]
app = FastAPI(middleware=middleware)
app.include_router(router)
handler = Mangum(app, lifespan="off")
def serverless(event, context):
return handler(event, context)

View File

@ -122,7 +122,3 @@ api_base = ""
AWS_ACCESS_KEY_ID = ""
AWS_SECRET_ACCESS_KEY = ""
AWS_REGION_NAME = ""
[aws_secrets_manager]
secret_arn = "" # The ARN of the AWS Secrets Manager secret containing PR-Agent configuration
region_name = "" # Optional: specific AWS region (defaults to AWS_REGION_NAME or Lambda region)

View File

@ -5,7 +5,7 @@ In addition to evaluating the suggestion correctness and importance, another sub
Examine each suggestion meticulously, assessing its quality, relevance, and accuracy within the context of PR. Keep in mind that the suggestions may vary in their correctness, accuracy and impact.
Consider the following components of each suggestion:
1. 'one_sentence_summary' - A one-liner summary of the suggestion's purpose
1. 'one_sentence_summary' - A one-liner summary summary of the suggestion's purpose
2. 'suggestion_content' - The suggestion content, explaining the proposed modification
3. 'existing_code' - a code snippet from a __new hunk__ section in the PR code diff that the suggestion addresses
4. 'improved_code' - a code snippet demonstrating how the 'existing_code' should be after the suggestion is applied

View File

@ -8,7 +8,7 @@
# models
model="o4-mini"
fallback_models=["gpt-4.1"]
#model_reasoning="o4-mini" # dedicated reasoning model for self-reflection
#model_reasoning="o4-mini" # dedictated reasoning model for self-reflection
#model_weak="gpt-4o" # optional, a weaker model to use for some easier tasks
# CLI
git_provider="github"
@ -39,7 +39,7 @@ allow_dynamic_context=true
max_extra_lines_before_dynamic_context = 10 # will try to include up to 10 extra lines before the hunk in the patch, until we reach an enclosing function or class
patch_extra_lines_before = 5 # Number of extra lines (+3 default ones) to include before each hunk in the patch
patch_extra_lines_after = 1 # Number of extra lines (+3 default ones) to include after each hunk in the patch
secret_provider="" # "" (disabled), "google_cloud_storage", or "aws_secrets_manager" for secure secret management
secret_provider=""
cli_mode=false
ai_disclaimer_title="" # Pro feature, title for a collapsible disclaimer to AI outputs
ai_disclaimer="" # Pro feature, full text for the AI disclaimer
@ -56,7 +56,6 @@ ignore_pr_source_branches = [] # a list of regular expressions of source branche
ignore_pr_labels = [] # labels to ignore from PR agent when an PR is created
ignore_pr_authors = [] # authors to ignore from PR agent when an PR is created
ignore_repositories = [] # a list of regular expressions of repository full names (e.g. "org/repo") to ignore from PR agent processing
ignore_language_framework = [] # a list of code-generation languages or frameworks (e.g. 'protobuf', 'go_gen') whose auto-generated source files will be excluded from analysis
#
is_auto_command = false # will be auto-set to true if the command is triggered by an automation
enable_ai_metadata = false # will enable adding ai metadata
@ -79,10 +78,8 @@ require_tests_review=true
require_estimate_effort_to_review=true
require_can_be_split_review=false
require_security_review=true
require_todo_scan=false
require_ticket_analysis_review=true
# general options
publish_output_no_suggestions=true # Set to "false" if you only need the reviewer's remarks (not labels, not "security audit", etc.) and want to avoid noisy "No major issues detected" comments.
persistent_comment=true
extra_instructions = ""
num_max_findings = 3
@ -107,7 +104,7 @@ enable_pr_type=true
final_update_message = true
enable_help_text=false
enable_help_comment=true
enable_pr_diagram=true # adds a section with a diagram of the PR changes
enable_pr_diagram=false # adds a section with a diagram of the PR changes
# describe as comment
publish_description_as_comment=false
publish_description_as_comment_persistent=true

View File

@ -1,42 +0,0 @@
[generated_code]
# Protocol Buffers
protobuf = [
"**/*.pb.go",
"**/*.pb.cc",
"**/*_pb2.py",
"**/*.pb.swift",
"**/*.pb.rb",
"**/*.pb.php",
"**/*.pb.h"
]
# OpenAPI / Swagger stubs
openapi = [
"**/__generated__/**",
"**/openapi_client/**",
"**/openapi_server/**"
]
swagger = [
"**/swagger.json",
"**/swagger.yaml"
]
# GraphQL codegen
graphql = [
"**/*.graphql.ts",
"**/*.generated.ts",
"**/*.graphql.js"
]
# RPC / gRPC Generators
grpc_python = ["**/*_grpc.py"]
grpc_java = ["**/*Grpc.java"]
grpc_csharp = ["**/*Grpc.cs"]
grpc_typescript = ["**/*_grpc.ts", "**/*_grpc.js"]
# Go code generators
go_gen = [
"**/*_gen.go",
"**/*generated.go"
]

View File

@ -1,12 +1,11 @@
[pr_description_prompt]
system="""You are PR-Reviewer, a language model designed to review a Git Pull Request (PR).
Your task is to provide a full description for the PR content: type, description, title, and files walkthrough.
Your task is to provide a full description for the PR content - type, description, title and files walkthrough.
- Focus on the new PR code (lines starting with '+' in the 'PR Git Diff' section).
- Keep in mind that the 'Previous title', 'Previous description' and 'Commit messages' sections may be partial, simplistic, non-informative or out of date. Hence, compare them to the PR diff code, and use them only as a reference.
- The generated title and description should prioritize the most significant changes.
- If needed, each YAML output should be in block scalar indicator ('|')
- When quoting variables, names or file paths from the code, use backticks (`) instead of single quote (').
- When needed, use '- ' as bullets
{%- if extra_instructions %}
@ -48,7 +47,7 @@ class PRDescription(BaseModel):
description: str = Field(description="summarize the PR changes in up to four bullet points, each up to 8 words. For large PRs, add sub-bullets if needed. Order bullets by importance, with each bullet highlighting a key change group.")
title: str = Field(description="a concise and descriptive title that captures the PR's main theme")
{%- if enable_pr_diagram %}
changes_diagram: str = Field(description="a horizontal diagram that represents the main PR changes, in the format of a valid mermaid LR flowchart. The diagram should be concise and easy to read. Leave empty if no diagram is relevant. To create robust Mermaid diagrams, follow this two-step process: (1) Declare the nodes: nodeID[\"node description\"]. (2) Then define the links: nodeID1 -- \"link text\" --> nodeID2. Node description must always be surrounded with quotation marks.")
changes_diagram: str = Field(description="a horizontal diagram that represents the main PR changes, in the format of a valid mermaid LR flowchart. The diagram should be concise and easy to read. Leave empty if no diagram is relevant. To create robust Mermaid diagrams, follow this two-step process: (1) Declare the nodes: nodeID["node description"]. (2) Then define the links: nodeID1 -- "link text" --> nodeID2 ")
{%- endif %}
{%- if enable_semantic_files_types %}
pr_files: List[FileDescription] = Field(max_items=20, description="a list of all the files that were changed in the PR, and summary of their changes. Each file must be analyzed regardless of change size.")

View File

@ -1,12 +1,12 @@
[pr_help_prompts]
system="""You are Doc-helper, a language models designed to answer questions about a documentation website for an open-soure project called "PR-Agent" (recently renamed to "Qodo Merge").
You will receive a question, and the full documentation website content.
You will recieve a question, and the full documentation website content.
Your goal is to provide the best answer to the question using the documentation provided.
Additional instructions:
- Try to be short and concise in your answers. Try to give examples if needed.
- The main tools of PR-Agent are 'describe', 'review', 'improve'. If there is ambiguity to which tool the user is referring to, prioritize snippets of these tools over others.
- If the question has ambiguity and can relate to different tools or platforms, provide the best answer possible based on what is available, but also state in your answer what additional information would be needed to give a more accurate answer.
- If the question has ambiguity and can relate to different tools or platfroms, provide the best answer possible based on what is available, but also state in your answer what additional information would be needed to give a more accurate answer.
The output must be a YAML object equivalent to type $DocHelper, according to the following Pydantic definitions:

View File

@ -2,7 +2,7 @@
system="""You are PR-Reviewer, a language model designed to review a Git Pull Request (PR).
Given the PR Info and the PR Git Diff, generate 3 short questions about the PR code for the PR author.
The goal of the questions is to help the language model understand the PR better, so the questions should be insightful, informative, non-trivial, and relevant to the PR.
You should prefer asking yes/no questions, or multiple choice questions. Also add at least one open-ended question, but make sure they are not too difficult, and can be answered in a sentence or two.
You should prefer asking yes\\no questions, or multiple choice questions. Also add at least one open-ended question, but make sure they are not too difficult, and can be answered in a sentence or two.
Example output:

View File

@ -37,9 +37,9 @@ __new hunk__
======
- In the format above, the diff is organized into separate '__new hunk__' and '__old hunk__' sections for each code chunk. '__new hunk__' contains the updated code, while '__old hunk__' shows the removed code. If no code was removed in a specific chunk, the __old hunk__ section will be omitted.
- We also added line numbers for the '__new hunk__' code, to help you refer to the code lines in your suggestions. These line numbers are not part of the actual code, and should only be used for reference.
- We also added line numbers for the '__new hunk__' code, to help you refer to the code lines in your suggestions. These line numbers are not part of the actual code, and should only used for reference.
- Code lines are prefixed with symbols ('+', '-', ' '). The '+' symbol indicates new code added in the PR, the '-' symbol indicates code removed in the PR, and the ' ' symbol indicates unchanged code. \
The review should address new code added in the PR code diff (lines starting with '+').
The review should address new code added in the PR code diff (lines starting with '+')
{%- if is_ai_metadata %}
- If available, an AI-generated summary will appear and provide a high-level overview of the file changes. Note that this summary may not be fully accurate or complete.
{%- endif %}
@ -72,13 +72,6 @@ class KeyIssuesComponentLink(BaseModel):
start_line: int = Field(description="The start line that corresponds to this issue in the relevant file")
end_line: int = Field(description="The end line that corresponds to this issue in the relevant file")
{%- if require_todo_scan %}
class TodoSection(BaseModel):
relevant_file: str = Field(description="The full path of the file containing the TODO comment")
line_number: int = Field(description="The line number where the TODO comment starts")
content: str = Field(description="The content of the TODO comment. Only include actual TODO comments within code comments (e.g., comments starting with '#', '//', '/*', '<!--', ...). Remove leading 'TODO' prefixes. If more than 10 words, summarize the TODO comment to a single short sentence up to 10 words.")
{%- endif %}
{%- if related_tickets %}
class TicketCompliance(BaseModel):
@ -100,17 +93,14 @@ class Review(BaseModel):
score: str = Field(description="Rate this PR on a scale of 0-100 (inclusive), where 0 means the worst possible PR code, and 100 means PR code of the highest quality, without any bugs or performance issues, that is ready to be merged immediately and run in production at scale.")
{%- endif %}
{%- if require_tests %}
relevant_tests: str = Field(description="yes/no question: does this PR have relevant tests added or updated ?")
relevant_tests: str = Field(description="yes\\no question: does this PR have relevant tests added or updated ?")
{%- endif %}
{%- if question_str %}
insights_from_user_answers: str = Field(description="shortly summarize the insights you gained from the user's answers to the questions")
{%- endif %}
key_issues_to_review: List[KeyIssuesComponentLink] = Field("A short and diverse list (0-{{ num_max_findings }} issues) of high-priority bugs, problems or performance concerns introduced in the PR code, which the PR reviewer should further focus on and validate during the review process.")
{%- if require_security_review %}
security_concerns: str = Field(description="Does this PR code introduce vulnerabilities such as exposure of sensitive information (e.g., API keys, secrets, passwords), or security concerns like SQL injection, XSS, CSRF, and others ? Answer 'No' (without explaining why) if there are no possible issues. If there are security concerns or issues, start your answer with a short header, such as: 'Sensitive information exposure: ...', 'SQL injection: ...', etc. Explain your answer. Be specific and give examples if possible")
{%- endif %}
{%- if require_todo_scan %}
todo_sections: Union[List[TodoSection], str] = Field(description="A list of TODO comments found in the PR code. Return 'No' (as a string) if there are no TODO comments in the PR")
security_concerns: str = Field(description="Does this PR code introduce possible vulnerabilities such as exposure of sensitive information (e.g., API keys, secrets, passwords), or security concerns like SQL injection, XSS, CSRF, and others ? Answer 'No' (without explaining why) if there are no possible issues. If there are security concerns or issues, start your answer with a short header, such as: 'Sensitive information exposure: ...', 'SQL injection: ...' etc. Explain your answer. Be specific and give examples if possible")
{%- endif %}
{%- if require_can_be_split_review %}
can_be_split: List[SubPR] = Field(min_items=0, max_items=3, description="Can this PR, which contains {{ num_pr_files }} changed files in total, be divided into smaller sub-PRs with distinct tasks that can be reviewed and merged independently, regardless of the order ? Make sure that the sub-PRs are indeed independent, with no code dependencies between them, and that each sub-PR represent a meaningful independent task. Output an empty list if the PR code does not need to be split.")
@ -158,10 +148,6 @@ review:
- ...
security_concerns: |
No
{%- if require_todo_scan %}
todo_sections: |
No
{%- endif %}
{%- if require_can_be_split_review %}
can_be_split:
- relevant_files:
@ -196,13 +182,6 @@ Ticket Description:
{{ ticket.body }}
#####
{%- endif %}
{%- if ticket.requirements %}
Ticket Requirements:
#####
{{ ticket.requirements }}
#####
{%- endif %}
=====
{% endfor %}
{%- endif %}
@ -287,10 +266,6 @@ review:
- ...
security_concerns: |
No
{%- if require_todo_scan %}
todo_sections: |
No
{%- endif %}
{%- if require_can_be_split_review %}
can_be_split:
- relevant_files:

View File

@ -614,13 +614,11 @@ class PRCodeSuggestions:
break
if original_initial_line:
suggested_initial_line = new_code_snippet.splitlines()[0]
original_initial_spaces = len(original_initial_line) - len(original_initial_line.lstrip()) # lstrip works both for spaces and tabs
original_initial_spaces = len(original_initial_line) - len(original_initial_line.lstrip())
suggested_initial_spaces = len(suggested_initial_line) - len(suggested_initial_line.lstrip())
delta_spaces = original_initial_spaces - suggested_initial_spaces
if delta_spaces > 0:
# Detect indentation character from original line
indent_char = '\t' if original_initial_line.startswith('\t') else ' '
new_code_snippet = textwrap.indent(new_code_snippet, delta_spaces * indent_char).rstrip('\n')
new_code_snippet = textwrap.indent(new_code_snippet, delta_spaces * " ").rstrip('\n')
except Exception as e:
get_logger().error(f"Error when dedenting code snippet for file {relevant_file}, error: {e}")

View File

@ -59,7 +59,6 @@ class PRDescription:
# Initialize the variables dictionary
self.COLLAPSIBLE_FILE_LIST_THRESHOLD = get_settings().pr_description.get("collapsible_file_list_threshold", 8)
enable_pr_diagram = get_settings().pr_description.get("enable_pr_diagram", False) and self.git_provider.is_supported("gfm_markdown") # github and gitlab support gfm_markdown
self.vars = {
"title": self.git_provider.pr.title,
"branch": self.git_provider.get_pr_branch(),
@ -74,7 +73,7 @@ class PRDescription:
"related_tickets": "",
"include_file_summary_changes": len(self.git_provider.get_diff_files()) <= self.COLLAPSIBLE_FILE_LIST_THRESHOLD,
"duplicate_prompt_examples": get_settings().config.get("duplicate_prompt_examples", False),
"enable_pr_diagram": enable_pr_diagram,
"enable_pr_diagram": get_settings().pr_description.get("enable_pr_diagram", False),
}
self.user_description = self.git_provider.get_user_description()
@ -538,11 +537,6 @@ class PRDescription:
get_logger().error(f"Failing to process walkthrough {self.pr_id}: {e}")
body = body.replace('pr_agent:walkthrough', "")
# Add support for pr_agent:diagram marker (plain and HTML comment formats)
ai_diagram = self.data.get('changes_diagram')
if ai_diagram:
body = re.sub(r'<!--\s*pr_agent:diagram\s*-->|pr_agent:diagram', ai_diagram, body)
return title, body, walkthrough_gfm, pr_file_changes
def _prepare_pr_answer(self) -> Tuple[str, str, str, List[dict]]:

View File

@ -21,7 +21,7 @@ from pr_agent.servers.help import HelpMessage
#Common code that can be called from similar tools:
def modify_answer_section(ai_response: str) -> str | None:
# Gets the model's answer and relevant sources section, replacing the heading of the answer section with:
# Gets the model's answer and relevant sources section, repacing the heading of the answer section with:
# :bulb: Auto-generated documentation-based answer:
"""
For example: The following input:

View File

@ -87,7 +87,6 @@ class PRReviewer:
"require_estimate_effort_to_review": get_settings().pr_reviewer.require_estimate_effort_to_review,
'require_can_be_split_review': get_settings().pr_reviewer.require_can_be_split_review,
'require_security_review': get_settings().pr_reviewer.require_security_review,
'require_todo_scan': get_settings().pr_reviewer.get("require_todo_scan", False),
'question_str': question_str,
'answer_str': answer_str,
"extra_instructions": get_settings().pr_reviewer.extra_instructions,
@ -159,32 +158,25 @@ class PRReviewer:
pr_review = self._prepare_pr_review()
get_logger().debug(f"PR output", artifact=pr_review)
should_publish = get_settings().config.publish_output and self._should_publish_review_no_suggestions(pr_review)
if not should_publish:
reason = "Review output is not published"
if get_settings().config.publish_output:
reason += ": no major issues detected."
get_logger().info(reason)
if get_settings().config.publish_output:
# publish the review
if get_settings().pr_reviewer.persistent_comment and not self.incremental.is_incremental:
final_update_message = get_settings().pr_reviewer.final_update_message
self.git_provider.publish_persistent_comment(pr_review,
initial_header=f"{PRReviewHeader.REGULAR.value} 🔍",
update_header=True,
final_update_message=final_update_message, )
else:
self.git_provider.publish_comment(pr_review)
self.git_provider.remove_initial_comment()
else:
get_logger().info("Review output is not published")
get_settings().data = {"artifact": pr_review}
return
# publish the review
if get_settings().pr_reviewer.persistent_comment and not self.incremental.is_incremental:
final_update_message = get_settings().pr_reviewer.final_update_message
self.git_provider.publish_persistent_comment(pr_review,
initial_header=f"{PRReviewHeader.REGULAR.value} 🔍",
update_header=True,
final_update_message=final_update_message, )
else:
self.git_provider.publish_comment(pr_review)
self.git_provider.remove_initial_comment()
except Exception as e:
get_logger().error(f"Failed to review PR: {e}")
def _should_publish_review_no_suggestions(self, pr_review: str) -> bool:
return get_settings().pr_reviewer.get('publish_output_no_suggestions', True) or "No major issues detected" not in pr_review
async def _prepare_prediction(self, model: str) -> None:
self.patches_diff = get_pr_diff(self.git_provider,
self.token_handler,

View File

@ -58,7 +58,7 @@ class PRUpdateChangelog:
'config': dict(get_settings().config)}
get_logger().debug("Relevant configs", artifacts=relevant_configs)
# check if the git provider supports pushing changelog changes
# currently only GitHub is supported for pushing changelog changes
if get_settings().pr_update_changelog.push_changelog_changes and not hasattr(
self.git_provider, "create_or_update_pr_file"
):
@ -128,7 +128,6 @@ class PRUpdateChangelog:
existing_content = self.changelog_file
else:
existing_content = ""
if existing_content:
new_file_content = answer + "\n\n" + self.changelog_file
else:
@ -187,18 +186,12 @@ Example:
self.changelog_file = self.git_provider.get_pr_file_content(
"CHANGELOG.md", self.git_provider.get_pr_branch()
)
if isinstance(self.changelog_file, bytes):
self.changelog_file = self.changelog_file.decode('utf-8')
changelog_file_lines = self.changelog_file.splitlines()
changelog_file_lines = changelog_file_lines[:CHANGELOG_LINES]
self.changelog_file_str = "\n".join(changelog_file_lines)
except Exception as e:
get_logger().warning(f"Error getting changelog file: {e}")
except Exception:
self.changelog_file_str = ""
self.changelog_file = ""
return
if not self.changelog_file_str:
self.changelog_file_str = self._get_default_changelog()

View File

@ -3,7 +3,6 @@ import traceback
from pr_agent.config_loader import get_settings
from pr_agent.git_providers import GithubProvider
from pr_agent.git_providers import AzureDevopsProvider
from pr_agent.log import get_logger
# Compile the regex pattern once, outside the function
@ -132,32 +131,6 @@ async def extract_tickets(git_provider):
return tickets_content
elif isinstance(git_provider, AzureDevopsProvider):
tickets_info = git_provider.get_linked_work_items()
tickets_content = []
for ticket in tickets_info:
try:
ticket_body_str = ticket.get("body", "")
if len(ticket_body_str) > MAX_TICKET_CHARACTERS:
ticket_body_str = ticket_body_str[:MAX_TICKET_CHARACTERS] + "..."
tickets_content.append(
{
"ticket_id": ticket.get("id"),
"ticket_url": ticket.get("url"),
"title": ticket.get("title"),
"body": ticket_body_str,
"requirements": ticket.get("acceptance_criteria", ""),
"labels": ", ".join(ticket.get("labels", [])),
}
)
except Exception as e:
get_logger().error(
f"Error processing Azure DevOps ticket: {e}",
artifact={"traceback": traceback.format_exc()},
)
return tickets_content
except Exception as e:
get_logger().error(f"Error extracting tickets error= {e}",
artifact={"traceback": traceback.format_exc()})

View File

@ -1,10 +1,10 @@
[build-system]
requires = ["setuptools>=61.0", "wheel"]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[project]
name = "pr-agent"
version = "0.3.0"
version = "0.2.7"
authors = [{ name = "QodoAI", email = "tal.r@qodo.ai" }]
@ -16,7 +16,7 @@ description = "QodoAI PR-Agent aims to help efficiently review and handle pull r
readme = "README.md"
requires-python = ">=3.12"
keywords = ["AI", "Agents", "Pull Request", "Automation", "Code Review"]
license = { file = "LICENSE" }
license = "Apache-2.0"
classifiers = [
"Intended Audience :: Developers",
@ -34,6 +34,7 @@ dependencies = { file = ["requirements.txt"] }
[tool.setuptools]
include-package-data = true
license-files = ["LICENSE"]
[tool.setuptools.packages.find]
where = ["."]

View File

@ -1,92 +0,0 @@
import pytest
from pr_agent.servers.github_app import handle_new_pr_opened
from pr_agent.tools.pr_add_docs import PRAddDocs
from pr_agent.agent.pr_agent import PRAgent
from pr_agent.config_loader import get_settings
from pr_agent.identity_providers.identity_provider import Eligibility
from pr_agent.identity_providers import get_identity_provider
@pytest.mark.asyncio
@pytest.mark.parametrize(
"action,draft,state,should_run",
[
("opened", False, "open", True),
("edited", False, "open", False),
("opened", True, "open", False),
("opened", False, "closed", False),
],
)
async def test_add_docs_trigger(monkeypatch, action, draft, state, should_run):
# Mock settings to enable the "/add_docs" auto-command on PR opened
settings = get_settings()
settings.github_app.pr_commands = ["/add_docs"]
settings.github_app.handle_pr_actions = ["opened"]
# Define a FakeGitProvider for both apply_repo_settings and PRAddDocs
class FakeGitProvider:
def __init__(self, pr_url, *args, **kwargs):
self.pr = type("pr", (), {"title": "Test PR"})()
self.get_pr_branch = lambda: "test-branch"
self.get_pr_description = lambda: "desc"
self.get_languages = lambda: ["Python"]
self.get_files = lambda: []
self.get_commit_messages = lambda: "msg"
self.publish_comment = lambda *args, **kwargs: None
self.remove_initial_comment = lambda: None
self.publish_code_suggestions = lambda suggestions: True
self.diff_files = []
self.get_repo_settings = lambda: {}
# Patch Git provider lookups
monkeypatch.setattr(
"pr_agent.git_providers.utils.get_git_provider_with_context",
lambda pr_url: FakeGitProvider(pr_url),
)
monkeypatch.setattr(
"pr_agent.tools.pr_add_docs.get_git_provider",
lambda: FakeGitProvider,
)
# Ensure identity provider always eligible
monkeypatch.setattr(
get_identity_provider().__class__,
"verify_eligibility",
lambda *args, **kwargs: Eligibility.ELIGIBLE,
)
# Spy on PRAddDocs.run()
ran = {"flag": False}
async def fake_run(self):
ran["flag"] = True
monkeypatch.setattr(PRAddDocs, "run", fake_run)
# Build minimal PR payload
body = {
"action": action,
"pull_request": {
"url": "https://example.com/fake/pr",
"state": state,
"draft": draft,
},
}
log_context = {}
# Invoke the PR-open handler
agent = PRAgent()
await handle_new_pr_opened(
body=body,
event="pull_request",
sender="tester",
sender_id="123",
action=action,
log_context=log_context,
agent=agent,
)
assert ran["flag"] is should_run, (
f"Expected run() to be {'called' if should_run else 'skipped'}"
f" for action={action!r}, draft={draft}, state={state!r}"
)

View File

@ -1,89 +0,0 @@
import json
import pytest
from unittest.mock import MagicMock, patch
from botocore.exceptions import ClientError
from pr_agent.secret_providers.aws_secrets_manager_provider import AWSSecretsManagerProvider
class TestAWSSecretsManagerProvider:
def _provider(self):
"""Create provider following existing pattern"""
with patch('pr_agent.secret_providers.aws_secrets_manager_provider.get_settings') as mock_get_settings, \
patch('pr_agent.secret_providers.aws_secrets_manager_provider.boto3.client') as mock_boto3_client:
settings = MagicMock()
settings.get.side_effect = lambda k, d=None: {
'aws_secrets_manager.secret_arn': 'arn:aws:secretsmanager:us-east-1:123456789012:secret:test-secret',
'aws_secrets_manager.region_name': 'us-east-1',
'aws.AWS_REGION_NAME': 'us-east-1'
}.get(k, d)
settings.aws_secrets_manager.secret_arn = 'arn:aws:secretsmanager:us-east-1:123456789012:secret:test-secret'
mock_get_settings.return_value = settings
# Mock boto3 client
mock_client = MagicMock()
mock_boto3_client.return_value = mock_client
provider = AWSSecretsManagerProvider()
provider.client = mock_client # Set client directly for testing
return provider, mock_client
# Positive test cases
def test_get_secret_success(self):
provider, mock_client = self._provider()
mock_client.get_secret_value.return_value = {'SecretString': 'test-secret-value'}
result = provider.get_secret('test-secret-name')
assert result == 'test-secret-value'
mock_client.get_secret_value.assert_called_once_with(SecretId='test-secret-name')
def test_get_all_secrets_success(self):
provider, mock_client = self._provider()
secret_data = {'openai.key': 'sk-test', 'github.webhook_secret': 'webhook-secret'}
mock_client.get_secret_value.return_value = {'SecretString': json.dumps(secret_data)}
result = provider.get_all_secrets()
assert result == secret_data
# Negative test cases (following Google Cloud Storage pattern)
def test_get_secret_failure(self):
provider, mock_client = self._provider()
mock_client.get_secret_value.side_effect = Exception("AWS error")
result = provider.get_secret('nonexistent-secret')
assert result == "" # Confirm empty string is returned
def test_get_all_secrets_failure(self):
provider, mock_client = self._provider()
mock_client.get_secret_value.side_effect = Exception("AWS error")
result = provider.get_all_secrets()
assert result == {} # Confirm empty dictionary is returned
def test_store_secret_update_existing(self):
provider, mock_client = self._provider()
mock_client.update_secret.return_value = {}
provider.store_secret('test-secret', 'test-value')
mock_client.put_secret_value.assert_called_once_with(
SecretId='test-secret',
SecretString='test-value'
)
def test_init_failure_invalid_config(self):
with patch('pr_agent.secret_providers.aws_secrets_manager_provider.get_settings') as mock_get_settings:
settings = MagicMock()
settings.aws_secrets_manager.secret_arn = None # Configuration error
mock_get_settings.return_value = settings
with pytest.raises(Exception):
AWSSecretsManagerProvider()
def test_store_secret_failure(self):
provider, mock_client = self._provider()
mock_client.put_secret_value.side_effect = Exception("AWS error")
with pytest.raises(Exception):
provider.store_secret('test-secret', 'test-value')

View File

@ -1,120 +0,0 @@
import pytest
from unittest.mock import MagicMock, patch
from pr_agent.config_loader import apply_secrets_manager_config, apply_secrets_to_config
class TestConfigLoaderSecrets:
def test_apply_secrets_manager_config_success(self):
with patch('pr_agent.secret_providers.get_secret_provider') as mock_get_provider, \
patch('pr_agent.config_loader.apply_secrets_to_config') as mock_apply_secrets, \
patch('pr_agent.config_loader.get_settings') as mock_get_settings:
# Mock secret provider
mock_provider = MagicMock()
mock_provider.get_all_secrets.return_value = {'openai.key': 'sk-test'}
mock_get_provider.return_value = mock_provider
# Mock settings
settings = MagicMock()
settings.get.return_value = "aws_secrets_manager"
mock_get_settings.return_value = settings
apply_secrets_manager_config()
mock_apply_secrets.assert_called_once_with({'openai.key': 'sk-test'})
def test_apply_secrets_manager_config_no_provider(self):
with patch('pr_agent.secret_providers.get_secret_provider') as mock_get_provider:
mock_get_provider.return_value = None
# Confirm no exception is raised
apply_secrets_manager_config()
def test_apply_secrets_manager_config_not_aws(self):
with patch('pr_agent.secret_providers.get_secret_provider') as mock_get_provider, \
patch('pr_agent.config_loader.get_settings') as mock_get_settings:
# Mock Google Cloud Storage provider
mock_provider = MagicMock()
mock_get_provider.return_value = mock_provider
# Mock settings (Google Cloud Storage)
settings = MagicMock()
settings.get.return_value = "google_cloud_storage"
mock_get_settings.return_value = settings
# Confirm execution is skipped for non-AWS Secrets Manager
apply_secrets_manager_config()
# Confirm get_all_secrets is not called
assert not hasattr(mock_provider, 'get_all_secrets') or \
not mock_provider.get_all_secrets.called
def test_apply_secrets_to_config_nested_keys(self):
with patch('pr_agent.config_loader.get_settings') as mock_get_settings:
settings = MagicMock()
settings.get.return_value = None # No existing value
settings.set = MagicMock()
mock_get_settings.return_value = settings
secrets = {
'openai.key': 'sk-test',
'github.webhook_secret': 'webhook-secret'
}
apply_secrets_to_config(secrets)
# Confirm settings are applied correctly
settings.set.assert_any_call('OPENAI.KEY', 'sk-test')
settings.set.assert_any_call('GITHUB.WEBHOOK_SECRET', 'webhook-secret')
def test_apply_secrets_to_config_existing_value_preserved(self):
with patch('pr_agent.config_loader.get_settings') as mock_get_settings:
settings = MagicMock()
settings.get.return_value = "existing-value" # Existing value present
settings.set = MagicMock()
mock_get_settings.return_value = settings
secrets = {'openai.key': 'sk-test'}
apply_secrets_to_config(secrets)
# Confirm settings are not overridden when existing value present
settings.set.assert_not_called()
def test_apply_secrets_to_config_single_key(self):
with patch('pr_agent.config_loader.get_settings') as mock_get_settings:
settings = MagicMock()
settings.get.return_value = None
settings.set = MagicMock()
mock_get_settings.return_value = settings
secrets = {'simple_key': 'simple_value'}
apply_secrets_to_config(secrets)
# Confirm non-dot notation keys are ignored
settings.set.assert_not_called()
def test_apply_secrets_to_config_multiple_dots(self):
with patch('pr_agent.config_loader.get_settings') as mock_get_settings:
settings = MagicMock()
settings.get.return_value = None
settings.set = MagicMock()
mock_get_settings.return_value = settings
secrets = {'section.subsection.key': 'value'}
apply_secrets_to_config(secrets)
# Confirm keys with multiple dots are ignored
settings.set.assert_not_called()
def test_apply_secrets_manager_config_exception_handling(self):
with patch('pr_agent.secret_providers.get_secret_provider') as mock_get_provider:
mock_get_provider.side_effect = Exception("Provider error")
# Confirm processing continues even when exception occurs
apply_secrets_manager_config() # Confirm no exception is raised

View File

@ -80,53 +80,3 @@ class TestIgnoreFilter:
filtered_files = filter_ignored(files)
assert filtered_files == expected, f"Expected {[file.filename for file in expected]}, but got {[file.filename for file in filtered_files]}."
def test_language_framework_ignores(self, monkeypatch):
"""
Test files are ignored based on language/framework mapping (e.g., protobuf).
"""
monkeypatch.setattr(global_settings.config, 'ignore_language_framework', ['protobuf', 'go_gen'])
files = [
type('', (object,), {'filename': 'main.go'})(),
type('', (object,), {'filename': 'dir1/service.pb.go'})(),
type('', (object,), {'filename': 'dir1/dir/data_pb2.py'})(),
type('', (object,), {'filename': 'file.py'})(),
type('', (object,), {'filename': 'dir2/file_gen.go'})(),
type('', (object,), {'filename': 'file.generated.go'})()
]
expected = [
files[0],
files[3]
]
filtered = filter_ignored(files)
assert filtered == expected, (
f"Expected {[f.filename for f in expected]}, "
f"but got {[f.filename for f in filtered]}"
)
def test_skip_invalid_ignore_language_framework(self, monkeypatch):
"""
Test skipping of generated code filtering when ignore_language_framework is not a list
"""
monkeypatch.setattr(global_settings.config, 'ignore_language_framework', 'protobuf')
files = [
type('', (object,), {'filename': 'main.go'})(),
type('', (object,), {'filename': 'file.py'})(),
type('', (object,), {'filename': 'dir1/service.pb.go'})(),
type('', (object,), {'filename': 'file_pb2.py'})()
]
expected = [
files[0],
files[1],
files[2],
files[3]
]
filtered = filter_ignored(files)
assert filtered == expected, (
f"Expected {[f.filename for f in expected]}, "
f"but got {[f.filename for f in filtered]}"
)

View File

@ -1,147 +0,0 @@
import pytest
from unittest.mock import MagicMock, patch
from pr_agent.git_providers.gitlab_provider import GitLabProvider
from gitlab import Gitlab
from gitlab.v4.objects import Project, ProjectFile
from gitlab.exceptions import GitlabGetError
class TestGitLabProvider:
"""Test suite for GitLab provider functionality."""
@pytest.fixture
def mock_gitlab_client(self):
client = MagicMock()
return client
@pytest.fixture
def mock_project(self):
project = MagicMock()
return project
@pytest.fixture
def gitlab_provider(self, mock_gitlab_client, mock_project):
with patch('pr_agent.git_providers.gitlab_provider.gitlab.Gitlab', return_value=mock_gitlab_client), \
patch('pr_agent.git_providers.gitlab_provider.get_settings') as mock_settings:
mock_settings.return_value.get.side_effect = lambda key, default=None: {
"GITLAB.URL": "https://gitlab.com",
"GITLAB.PERSONAL_ACCESS_TOKEN": "fake_token"
}.get(key, default)
mock_gitlab_client.projects.get.return_value = mock_project
provider = GitLabProvider("https://gitlab.com/test/repo/-/merge_requests/1")
provider.gl = mock_gitlab_client
provider.id_project = "test/repo"
return provider
def test_get_pr_file_content_success(self, gitlab_provider, mock_project):
mock_file = MagicMock(ProjectFile)
mock_file.decode.return_value = "# Changelog\n\n## v1.0.0\n- Initial release"
mock_project.files.get.return_value = mock_file
content = gitlab_provider.get_pr_file_content("CHANGELOG.md", "main")
assert content == "# Changelog\n\n## v1.0.0\n- Initial release"
mock_project.files.get.assert_called_once_with("CHANGELOG.md", "main")
mock_file.decode.assert_called_once()
def test_get_pr_file_content_with_bytes(self, gitlab_provider, mock_project):
mock_file = MagicMock(ProjectFile)
mock_file.decode.return_value = b"# Changelog\n\n## v1.0.0\n- Initial release"
mock_project.files.get.return_value = mock_file
content = gitlab_provider.get_pr_file_content("CHANGELOG.md", "main")
assert content == "# Changelog\n\n## v1.0.0\n- Initial release"
mock_project.files.get.assert_called_once_with("CHANGELOG.md", "main")
def test_get_pr_file_content_file_not_found(self, gitlab_provider, mock_project):
mock_project.files.get.side_effect = GitlabGetError("404 Not Found")
content = gitlab_provider.get_pr_file_content("CHANGELOG.md", "main")
assert content == ""
mock_project.files.get.assert_called_once_with("CHANGELOG.md", "main")
def test_get_pr_file_content_other_exception(self, gitlab_provider, mock_project):
mock_project.files.get.side_effect = Exception("Network error")
content = gitlab_provider.get_pr_file_content("CHANGELOG.md", "main")
assert content == ""
def test_create_or_update_pr_file_create_new(self, gitlab_provider, mock_project):
mock_project.files.get.side_effect = GitlabGetError("404 Not Found")
mock_file = MagicMock()
mock_project.files.create.return_value = mock_file
new_content = "# Changelog\n\n## v1.1.0\n- New feature"
commit_message = "Add CHANGELOG.md"
gitlab_provider.create_or_update_pr_file(
"CHANGELOG.md", "feature-branch", new_content, commit_message
)
mock_project.files.get.assert_called_once_with("CHANGELOG.md", "feature-branch")
mock_project.files.create.assert_called_once_with({
'file_path': 'CHANGELOG.md',
'branch': 'feature-branch',
'content': new_content,
'commit_message': commit_message,
})
def test_create_or_update_pr_file_update_existing(self, gitlab_provider, mock_project):
mock_file = MagicMock(ProjectFile)
mock_file.decode.return_value = "# Old changelog content"
mock_project.files.get.return_value = mock_file
new_content = "# New changelog content"
commit_message = "Update CHANGELOG.md"
gitlab_provider.create_or_update_pr_file(
"CHANGELOG.md", "feature-branch", new_content, commit_message
)
mock_project.files.get.assert_called_once_with("CHANGELOG.md", "feature-branch")
mock_file.content = new_content
mock_file.save.assert_called_once_with(branch="feature-branch", commit_message=commit_message)
def test_create_or_update_pr_file_update_exception(self, gitlab_provider, mock_project):
mock_project.files.get.side_effect = Exception("Network error")
with pytest.raises(Exception):
gitlab_provider.create_or_update_pr_file(
"CHANGELOG.md", "feature-branch", "content", "message"
)
def test_has_create_or_update_pr_file_method(self, gitlab_provider):
assert hasattr(gitlab_provider, "create_or_update_pr_file")
assert callable(getattr(gitlab_provider, "create_or_update_pr_file"))
def test_method_signature_compatibility(self, gitlab_provider):
import inspect
sig = inspect.signature(gitlab_provider.create_or_update_pr_file)
params = list(sig.parameters.keys())
expected_params = ['file_path', 'branch', 'contents', 'message']
assert params == expected_params
@pytest.mark.parametrize("content,expected", [
("simple text", "simple text"),
(b"bytes content", "bytes content"),
("", ""),
(b"", ""),
("unicode: café", "unicode: café"),
(b"unicode: caf\xc3\xa9", "unicode: café"),
])
def test_content_encoding_handling(self, gitlab_provider, mock_project, content, expected):
mock_file = MagicMock(ProjectFile)
mock_file.decode.return_value = content
mock_project.files.get.return_value = mock_file
result = gitlab_provider.get_pr_file_content("test.md", "main")
assert result == expected

View File

@ -1,247 +0,0 @@
import pytest
from unittest.mock import MagicMock, patch, AsyncMock
from pr_agent.tools.pr_update_changelog import PRUpdateChangelog
class TestPRUpdateChangelog:
"""Test suite for the PR Update Changelog functionality."""
@pytest.fixture
def mock_git_provider(self):
"""Create a mock git provider."""
provider = MagicMock()
provider.get_pr_branch.return_value = "feature-branch"
provider.get_pr_file_content.return_value = ""
provider.pr.title = "Test PR"
provider.get_pr_description.return_value = "Test description"
provider.get_commit_messages.return_value = "fix: test commit"
provider.get_languages.return_value = {"Python": 80, "JavaScript": 20}
provider.get_files.return_value = ["test.py", "test.js"]
return provider
@pytest.fixture
def mock_ai_handler(self):
"""Create a mock AI handler."""
handler = MagicMock()
handler.chat_completion = AsyncMock(return_value=("Test changelog entry", "stop"))
return handler
@pytest.fixture
def changelog_tool(self, mock_git_provider, mock_ai_handler):
"""Create a PRUpdateChangelog instance with mocked dependencies."""
with patch('pr_agent.tools.pr_update_changelog.get_git_provider', return_value=lambda url: mock_git_provider), \
patch('pr_agent.tools.pr_update_changelog.get_main_pr_language', return_value="Python"), \
patch('pr_agent.tools.pr_update_changelog.get_settings') as mock_settings:
# Configure mock settings
mock_settings.return_value.pr_update_changelog.push_changelog_changes = False
mock_settings.return_value.pr_update_changelog.extra_instructions = ""
mock_settings.return_value.pr_update_changelog_prompt.system = "System prompt"
mock_settings.return_value.pr_update_changelog_prompt.user = "User prompt"
mock_settings.return_value.config.temperature = 0.2
tool = PRUpdateChangelog("https://gitlab.com/test/repo/-/merge_requests/1", ai_handler=lambda: mock_ai_handler)
return tool
def test_get_changelog_file_with_existing_content(self, changelog_tool, mock_git_provider):
"""Test retrieving existing changelog content."""
# Arrange
existing_content = "# Changelog\n\n## v1.0.0\n- Initial release\n- Bug fixes"
mock_git_provider.get_pr_file_content.return_value = existing_content
# Act
changelog_tool._get_changelog_file()
# Assert
assert changelog_tool.changelog_file == existing_content
assert "# Changelog" in changelog_tool.changelog_file_str
def test_get_changelog_file_with_no_existing_content(self, changelog_tool, mock_git_provider):
"""Test handling when no changelog file exists."""
# Arrange
mock_git_provider.get_pr_file_content.return_value = ""
# Act
changelog_tool._get_changelog_file()
# Assert
assert changelog_tool.changelog_file == ""
assert "Example:" in changelog_tool.changelog_file_str # Default template
def test_get_changelog_file_with_bytes_content(self, changelog_tool, mock_git_provider):
"""Test handling when git provider returns bytes instead of string."""
# Arrange
content_bytes = b"# Changelog\n\n## v1.0.0\n- Initial release"
mock_git_provider.get_pr_file_content.return_value = content_bytes
# Act
changelog_tool._get_changelog_file()
# Assert
assert isinstance(changelog_tool.changelog_file, str)
assert changelog_tool.changelog_file == "# Changelog\n\n## v1.0.0\n- Initial release"
def test_get_changelog_file_with_exception(self, changelog_tool, mock_git_provider):
"""Test handling exceptions during file retrieval."""
# Arrange
mock_git_provider.get_pr_file_content.side_effect = Exception("Network error")
# Act
changelog_tool._get_changelog_file()
# Assert
assert changelog_tool.changelog_file == ""
assert changelog_tool.changelog_file_str == "" # Exception should result in empty string, no default template
def test_prepare_changelog_update_with_existing_content(self, changelog_tool):
"""Test preparing changelog update when existing content exists."""
# Arrange
changelog_tool.prediction = "## v1.1.0\n- New feature\n- Bug fix"
changelog_tool.changelog_file = "# Changelog\n\n## v1.0.0\n- Initial release"
changelog_tool.commit_changelog = True
# Act
new_content, answer = changelog_tool._prepare_changelog_update()
# Assert
assert new_content.startswith("## v1.1.0\n- New feature\n- Bug fix\n\n")
assert "# Changelog\n\n## v1.0.0\n- Initial release" in new_content
assert answer == "## v1.1.0\n- New feature\n- Bug fix"
def test_prepare_changelog_update_without_existing_content(self, changelog_tool):
"""Test preparing changelog update when no existing content."""
# Arrange
changelog_tool.prediction = "## v1.0.0\n- Initial release"
changelog_tool.changelog_file = ""
changelog_tool.commit_changelog = True
# Act
new_content, answer = changelog_tool._prepare_changelog_update()
# Assert
assert new_content == "## v1.0.0\n- Initial release"
assert answer == "## v1.0.0\n- Initial release"
def test_prepare_changelog_update_no_commit(self, changelog_tool):
"""Test preparing changelog update when not committing."""
# Arrange
changelog_tool.prediction = "## v1.1.0\n- New feature"
changelog_tool.changelog_file = ""
changelog_tool.commit_changelog = False
# Act
new_content, answer = changelog_tool._prepare_changelog_update()
# Assert
assert new_content == "## v1.1.0\n- New feature"
assert "to commit the new content" in answer
@pytest.mark.asyncio
async def test_run_without_push_support(self, changelog_tool, mock_git_provider):
"""Test running changelog update when git provider doesn't support pushing."""
# Arrange
delattr(mock_git_provider, 'create_or_update_pr_file') # Remove the method
changelog_tool.commit_changelog = True
with patch('pr_agent.tools.pr_update_changelog.get_settings') as mock_settings:
mock_settings.return_value.pr_update_changelog.push_changelog_changes = True
mock_settings.return_value.config.publish_output = True
# Act
await changelog_tool.run()
# Assert
mock_git_provider.publish_comment.assert_called_once()
assert "not currently supported" in str(mock_git_provider.publish_comment.call_args)
@pytest.mark.asyncio
async def test_run_with_push_support(self, changelog_tool, mock_git_provider):
"""Test running changelog update when git provider supports pushing."""
# Arrange
mock_git_provider.create_or_update_pr_file = MagicMock()
changelog_tool.commit_changelog = True
changelog_tool.prediction = "## v1.1.0\n- New feature"
with patch('pr_agent.tools.pr_update_changelog.get_settings') as mock_settings, \
patch('pr_agent.tools.pr_update_changelog.retry_with_fallback_models') as mock_retry, \
patch('pr_agent.tools.pr_update_changelog.sleep'):
mock_settings.return_value.pr_update_changelog.push_changelog_changes = True
mock_settings.return_value.pr_update_changelog.get.return_value = True
mock_settings.return_value.config.publish_output = True
mock_settings.return_value.config.git_provider = "gitlab"
mock_retry.return_value = None
# Act
await changelog_tool.run()
# Assert
mock_git_provider.create_or_update_pr_file.assert_called_once()
call_args = mock_git_provider.create_or_update_pr_file.call_args
assert call_args[1]['file_path'] == 'CHANGELOG.md'
assert call_args[1]['branch'] == 'feature-branch'
def test_push_changelog_update(self, changelog_tool, mock_git_provider):
"""Test the push changelog update functionality."""
# Arrange
mock_git_provider.create_or_update_pr_file = MagicMock()
mock_git_provider.get_pr_branch.return_value = "feature-branch"
new_content = "# Updated changelog content"
answer = "Changes made"
with patch('pr_agent.tools.pr_update_changelog.get_settings') as mock_settings, \
patch('pr_agent.tools.pr_update_changelog.sleep'):
mock_settings.return_value.pr_update_changelog.get.return_value = True
# Act
changelog_tool._push_changelog_update(new_content, answer)
# Assert
mock_git_provider.create_or_update_pr_file.assert_called_once_with(
file_path="CHANGELOG.md",
branch="feature-branch",
contents=new_content,
message="[skip ci] Update CHANGELOG.md"
)
def test_gitlab_provider_method_detection(self, changelog_tool, mock_git_provider):
"""Test that the tool correctly detects GitLab provider method availability."""
# Arrange
mock_git_provider.create_or_update_pr_file = MagicMock()
# Act & Assert
assert hasattr(mock_git_provider, "create_or_update_pr_file")
@pytest.mark.parametrize("existing_content,new_entry,expected_order", [
(
"# Changelog\n\n## v1.0.0\n- Old feature",
"## v1.1.0\n- New feature",
["v1.1.0", "v1.0.0"]
),
(
"",
"## v1.0.0\n- Initial release",
["v1.0.0"]
),
(
"Some existing content",
"## v1.0.0\n- New entry",
["v1.0.0", "Some existing content"]
),
])
def test_changelog_order_preservation(self, changelog_tool, existing_content, new_entry, expected_order):
"""Test that changelog entries are properly ordered (newest first)."""
# Arrange
changelog_tool.prediction = new_entry
changelog_tool.changelog_file = existing_content
changelog_tool.commit_changelog = True
# Act
new_content, _ = changelog_tool._prepare_changelog_update()
# Assert
for i, expected in enumerate(expected_order[:-1]):
current_pos = new_content.find(expected)
next_pos = new_content.find(expected_order[i + 1])
assert current_pos < next_pos, f"Expected {expected} to come before {expected_order[i + 1]}"

View File

@ -1,69 +0,0 @@
import pytest
from unittest.mock import MagicMock, patch
from pr_agent.secret_providers import get_secret_provider
class TestSecretProviderFactory:
def test_get_secret_provider_none_when_not_configured(self):
with patch('pr_agent.secret_providers.get_settings') as mock_get_settings:
settings = MagicMock()
settings.get.return_value = None
mock_get_settings.return_value = settings
result = get_secret_provider()
assert result is None
def test_get_secret_provider_google_cloud_storage(self):
with patch('pr_agent.secret_providers.get_settings') as mock_get_settings:
settings = MagicMock()
settings.get.return_value = "google_cloud_storage"
settings.config.secret_provider = "google_cloud_storage"
mock_get_settings.return_value = settings
with patch('pr_agent.secret_providers.google_cloud_storage_secret_provider.GoogleCloudStorageSecretProvider') as MockProvider:
mock_instance = MagicMock()
MockProvider.return_value = mock_instance
result = get_secret_provider()
assert result is mock_instance
MockProvider.assert_called_once()
def test_get_secret_provider_aws_secrets_manager(self):
with patch('pr_agent.secret_providers.get_settings') as mock_get_settings:
settings = MagicMock()
settings.get.return_value = "aws_secrets_manager"
settings.config.secret_provider = "aws_secrets_manager"
mock_get_settings.return_value = settings
with patch('pr_agent.secret_providers.aws_secrets_manager_provider.AWSSecretsManagerProvider') as MockProvider:
mock_instance = MagicMock()
MockProvider.return_value = mock_instance
result = get_secret_provider()
assert result is mock_instance
MockProvider.assert_called_once()
def test_get_secret_provider_unknown_provider(self):
with patch('pr_agent.secret_providers.get_settings') as mock_get_settings:
settings = MagicMock()
settings.get.return_value = "unknown_provider"
settings.config.secret_provider = "unknown_provider"
mock_get_settings.return_value = settings
with pytest.raises(ValueError, match="Unknown SECRET_PROVIDER"):
get_secret_provider()
def test_get_secret_provider_initialization_error(self):
with patch('pr_agent.secret_providers.get_settings') as mock_get_settings:
settings = MagicMock()
settings.get.return_value = "aws_secrets_manager"
settings.config.secret_provider = "aws_secrets_manager"
mock_get_settings.return_value = settings
with patch('pr_agent.secret_providers.aws_secrets_manager_provider.AWSSecretsManagerProvider') as MockProvider:
MockProvider.side_effect = Exception("Initialization failed")
with pytest.raises(ValueError, match="Failed to initialize aws_secrets_manager secret provider"):
get_secret_provider()