mirror of
https://github.com/qodo-ai/pr-agent.git
synced 2025-07-15 18:20:37 +08:00
Merge remote-tracking branch 'origin/main' into ok/gitlab_webhook
This commit is contained in:
@ -1,80 +0,0 @@
|
||||
## Configuration
|
||||
|
||||
The different tools and sub-tools used by CodiumAI PR-Agent are adjustable via the **[configuration file](pr_agent/settings/configuration.toml)**
|
||||
|
||||
### Working from CLI
|
||||
When running from source (CLI), your local configuration file will be initially used.
|
||||
|
||||
Example for invoking the 'review' tools via the CLI:
|
||||
|
||||
```
|
||||
python cli.py --pr-url=<pr_url> review
|
||||
```
|
||||
In addition to general configurations, the 'review' tool will use parameters from the `[pr_reviewer]` section (every tool has a dedicated section in the configuration file).
|
||||
|
||||
Note that you can print results locally, without publishing them, by setting in `configuration.toml`:
|
||||
|
||||
```
|
||||
[config]
|
||||
publish_output=true
|
||||
verbosity_level=2
|
||||
```
|
||||
This is useful for debugging or experimenting with the different tools.
|
||||
|
||||
### Working from GitHub App (pre-built repo)
|
||||
When running PR-Agent from GitHub App, the default configuration file (`configuration.toml`) will be loaded.
|
||||
|
||||
#### GitHub app default tools
|
||||
The `[github_app]` section of the configuration file defines GitHub app specific configurations.
|
||||
The important parameter is `pr_commands`, which is a list of tools that will be run automatically when a new PR is opened:
|
||||
```
|
||||
[github_app]
|
||||
pr_commands = [
|
||||
"/describe --pr_description.add_original_user_description=true --pr_description.keep_original_user_title=true",
|
||||
"/auto_review",
|
||||
]
|
||||
```
|
||||
This means that when a new PR is opened, PR-Agent will run the `describe` and `auto_review` tools.
|
||||
For the describe tool, the `add_original_user_description` and `keep_original_user_title` parameters will be set to `true`.
|
||||
|
||||
However, you can override the default tool parameters by uploading a local configuration file called `.pr_agent.toml` to the root of your repo.
|
||||
For example, if your local `.pr_agent.toml` file contains:
|
||||
```
|
||||
[pr_description]
|
||||
add_original_user_description = false
|
||||
keep_original_user_title = false
|
||||
```
|
||||
Then when a new PR is opened, PR-Agent will run the `describe` tool with the above parameters.
|
||||
|
||||
Note that a local `.pr_agent.toml` file enables you to edit and customize the default parameters of any tool, not just the ones that are run automatically.
|
||||
|
||||
#### Online usage
|
||||
For online usage (calling tools by comments on a PR like `/ask ...`), just add `--config_path=<value>` to any command, to edit a specific configuration value.
|
||||
For example if you want to edit `pr_reviewer` configurations, you can run:
|
||||
```
|
||||
/review --pr_reviewer.extra_instructions="..." --pr_reviewer.require_score_review=false ...
|
||||
```
|
||||
Any configuration value in `configuration.toml` file can be similarly edited.
|
||||
|
||||
|
||||
### General configuration walkthrough
|
||||
|
||||
#### Changing a model
|
||||
See [here](pr_agent/algo/__init__.py) for the list of available models.
|
||||
|
||||
To use Llama2 model, for example, set:
|
||||
```
|
||||
[config]
|
||||
model = "replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1"
|
||||
[replicate]
|
||||
key = ...
|
||||
```
|
||||
(you can obtain a Llama2 key from [here](https://replicate.com/replicate/llama-2-70b-chat/api))
|
||||
|
||||
Also review the [AiHandler](pr_agent/algo/ai_handler.py) file for instruction how to set keys for other models.
|
||||
|
||||
#### Extra instructions
|
||||
All PR-Agent tools have a parameter called `extra_instructions`, that enables to add free-text extra instructions. Example usage:
|
||||
```
|
||||
/update_changelog --pr_update_changelog.extra_instructions="Make sure to update also the version ..."
|
||||
```
|
78
INSTALL.md
78
INSTALL.md
@ -9,8 +9,8 @@ To get started with PR-Agent quickly, you first need to acquire two tokens:
|
||||
There are several ways to use PR-Agent:
|
||||
|
||||
- [Method 1: Use Docker image (no installation required)](INSTALL.md#method-1-use-docker-image-no-installation-required)
|
||||
- [Method 2: Run as a GitHub Action](INSTALL.md#method-2-run-as-a-github-action)
|
||||
- [Method 3: Run from source](INSTALL.md#method-3-run-from-source)
|
||||
- [Method 2: Run from source](INSTALL.md#method-2-run-from-source)
|
||||
- [Method 3: Run as a GitHub Action](INSTALL.md#method-3-run-as-a-github-action)
|
||||
- [Method 4: Run as a polling server](INSTALL.md#method-4-run-as-a-polling-server)
|
||||
- [Method 5: Run as a GitHub App](INSTALL.md#method-5-run-as-a-github-app)
|
||||
- [Method 6: Deploy as a Lambda Function](INSTALL.md#method-6---deploy-as-a-lambda-function)
|
||||
@ -55,7 +55,41 @@ Possible questions you can ask include:
|
||||
|
||||
---
|
||||
|
||||
### Method 2: Run as a GitHub Action
|
||||
### Method 2: Run from source
|
||||
|
||||
1. Clone this repository:
|
||||
|
||||
```
|
||||
git clone https://github.com/Codium-ai/pr-agent.git
|
||||
```
|
||||
|
||||
2. Install the requirements in your favorite virtual environment:
|
||||
|
||||
```
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
3. Copy the secrets template file and fill in your OpenAI key and your GitHub user token:
|
||||
|
||||
```
|
||||
cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets.toml
|
||||
chmod 600 pr_agent/settings/.secrets.toml
|
||||
# Edit .secrets.toml file
|
||||
```
|
||||
|
||||
4. Add the pr_agent folder to your PYTHONPATH, then run the cli.py script:
|
||||
|
||||
```
|
||||
export PYTHONPATH=[$PYTHONPATH:]<PATH to pr_agent folder>
|
||||
python pr_agent/cli.py --pr_url <pr_url> /review
|
||||
python pr_agent/cli.py --pr_url <pr_url> /ask <your question>
|
||||
python pr_agent/cli.py --pr_url <pr_url> /describe
|
||||
python pr_agent/cli.py --pr_url <pr_url> /improve
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Method 3: Run as a GitHub Action
|
||||
|
||||
You can use our pre-built Github Action Docker image to run PR-Agent as a Github Action.
|
||||
|
||||
@ -114,7 +148,7 @@ The GITHUB_TOKEN secret is automatically created by GitHub.
|
||||
3. Merge this change to your main branch.
|
||||
When you open your next PR, you should see a comment from `github-actions` bot with a review of your PR, and instructions on how to use the rest of the tools.
|
||||
|
||||
4. You may configure PR-Agent by adding environment variables under the env section corresponding to any configurable property in the [configuration](./CONFIGURATION.md) file. Some examples:
|
||||
4. You may configure PR-Agent by adding environment variables under the env section corresponding to any configurable property in the [configuration](./Usage.md) file. Some examples:
|
||||
```yaml
|
||||
env:
|
||||
# ... previous environment values
|
||||
@ -125,40 +159,6 @@ When you open your next PR, you should see a comment from `github-actions` bot w
|
||||
|
||||
---
|
||||
|
||||
### Method 3: Run from source
|
||||
|
||||
1. Clone this repository:
|
||||
|
||||
```
|
||||
git clone https://github.com/Codium-ai/pr-agent.git
|
||||
```
|
||||
|
||||
2. Install the requirements in your favorite virtual environment:
|
||||
|
||||
```
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
3. Copy the secrets template file and fill in your OpenAI key and your GitHub user token:
|
||||
|
||||
```
|
||||
cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets.toml
|
||||
chmod 600 pr_agent/settings/.secrets.toml
|
||||
# Edit .secrets.toml file
|
||||
```
|
||||
|
||||
4. Add the pr_agent folder to your PYTHONPATH, then run the cli.py script:
|
||||
|
||||
```
|
||||
export PYTHONPATH=[$PYTHONPATH:]<PATH to pr_agent folder>
|
||||
python pr_agent/cli.py --pr_url <pr_url> review
|
||||
python pr_agent/cli.py --pr_url <pr_url> ask <your question>
|
||||
python pr_agent/cli.py --pr_url <pr_url> describe
|
||||
python pr_agent/cli.py --pr_url <pr_url> improve
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Method 4: Run as a polling server
|
||||
Request reviews by tagging your Github user on a PR
|
||||
|
||||
@ -253,7 +253,7 @@ docker push codiumai/pr-agent:github_app # Push to your Docker repository
|
||||
|
||||
> **Note:** When running PR-Agent from GitHub App, the default configuration file (configuration.toml) will be loaded.<br>
|
||||
> However, you can override the default tool parameters by uploading a local configuration file<br>
|
||||
> For more information please check out [CONFIGURATION.md](CONFIGURATION.md#working-from-github-app-pre-built-repo)
|
||||
> For more information please check out [CONFIGURATION.md](Usage.md#working-from-github-app-pre-built-repo)
|
||||
---
|
||||
|
||||
### Method 6 - Deploy as a Lambda Function
|
||||
|
150
README.md
150
README.md
@ -17,17 +17,20 @@ Making pull requests less painful with an AI agent
|
||||
|
||||
CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull requests faster and more efficiently. It automatically analyzes the pull request and can provide several types of PR feedback:
|
||||
|
||||
**Auto-Description**: Automatically generating [PR description](https://github.com/Codium-ai/pr-agent/pull/229#issue-1860711415) - title, type, summary, code walkthrough and labels.
|
||||
**Auto Description (/describe)**: Automatically generating [PR description](https://github.com/Codium-ai/pr-agent/pull/229#issue-1860711415) - title, type, summary, code walkthrough and labels.
|
||||
\
|
||||
**Auto Review**: [Adjustable feedback](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695022908) about the PR main theme, type, relevant tests, security issues, score, and various suggestions for the PR content.
|
||||
**Auto Review (/review)**: [Adjustable feedback](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695022908) about the PR main theme, type, relevant tests, security issues, score, and various suggestions for the PR content.
|
||||
\
|
||||
**Question Answering**: Answering [free-text questions](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021332) about the PR.
|
||||
**Question Answering (/ask ...)**: Answering [free-text questions](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021332) about the PR.
|
||||
\
|
||||
**Code Suggestions**: [Committable code suggestions](https://github.com/Codium-ai/pr-agent/pull/229#discussion_r1306919276) for improving the PR.
|
||||
**Code Suggestions (/improve)**: [Committable code suggestions](https://github.com/Codium-ai/pr-agent/pull/229#discussion_r1306919276) for improving the PR.
|
||||
\
|
||||
**Update Changelog**: Automatically updating the CHANGELOG.md file with the [PR changes](https://github.com/Codium-ai/pr-agent/pull/168#discussion_r1282077645).
|
||||
**Update Changelog (/update_changelog)**: Automatically updating the CHANGELOG.md file with the [PR changes](https://github.com/Codium-ai/pr-agent/pull/168#discussion_r1282077645).
|
||||
|
||||
<h3>Example results:</h2>
|
||||
|
||||
See the [usage guide](./Usage.md) for instructions how to run the different tools from [CLI](./Usage.md#working-from-a-local-repo-cli), or by [online usage](./Usage.md#online-usage).
|
||||
|
||||
<h3>Example results:</h3>
|
||||
</div>
|
||||
<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1687561986">/describe:</a></h4>
|
||||
<div align="center">
|
||||
@ -35,90 +38,101 @@ CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull
|
||||
<img src="https://www.codium.ai/images/describe-2.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021901">/review:</a></h4>
|
||||
<div align="center">
|
||||
<p float="center">
|
||||
<img src="https://www.codium.ai/images/review-2.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
<h4><a href="https://github.com/Codium-ai/pr-agent/pull/78#issuecomment-1639739496">/reflect_and_review:</a></h4>
|
||||
<div align="center">
|
||||
<p float="center">
|
||||
<img src="https://www.codium.ai/images/reflect_and_review.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695020538">/ask:</a></h4>
|
||||
<div align="center">
|
||||
<p float="center">
|
||||
<img src="https://www.codium.ai/images/ask-2.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695024952">/improve:</a></h4>
|
||||
<div align="center">
|
||||
<p float="center">
|
||||
<img src="https://www.codium.ai/images/improve-2.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
|
||||
[//]: # (<h4><a href="https://github.com/Codium-ai/pr-agent/pull/78#issuecomment-1639739496">/reflect_and_review:</a></h4>)
|
||||
|
||||
[//]: # (<div align="center">)
|
||||
|
||||
[//]: # (<p float="center">)
|
||||
|
||||
[//]: # (<img src="https://www.codium.ai/images/reflect_and_review.gif" width="800">)
|
||||
|
||||
[//]: # (</p>)
|
||||
|
||||
[//]: # (</div>)
|
||||
|
||||
[//]: # (<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695020538">/ask:</a></h4>)
|
||||
|
||||
[//]: # (<div align="center">)
|
||||
|
||||
[//]: # (<p float="center">)
|
||||
|
||||
[//]: # (<img src="https://www.codium.ai/images/ask-2.gif" width="800">)
|
||||
|
||||
[//]: # (</p>)
|
||||
|
||||
[//]: # (</div>)
|
||||
|
||||
[//]: # (<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695024952">/improve:</a></h4>)
|
||||
|
||||
[//]: # (<div align="center">)
|
||||
|
||||
[//]: # (<p float="center">)
|
||||
|
||||
[//]: # (<img src="https://www.codium.ai/images/improve-2.gif" width="800">)
|
||||
|
||||
[//]: # (</p>)
|
||||
|
||||
[//]: # (</div>)
|
||||
<div align="left">
|
||||
|
||||
|
||||
## Table of Contents
|
||||
- [Overview](#overview)
|
||||
- [Try it now](#try-it-now)
|
||||
- [Installation](#installation)
|
||||
- [Configuration](./CONFIGURATION.md)
|
||||
- [Usage guide](./Usage.md)
|
||||
- [How it works](#how-it-works)
|
||||
- [Why use PR-Agent](#why-use-pr-agent)
|
||||
- [Roadmap](#roadmap)
|
||||
- [Similar projects](#similar-projects)
|
||||
</div>
|
||||
|
||||
|
||||
## Overview
|
||||
`PR-Agent` offers extensive pull request functionalities across various git providers:
|
||||
| | | GitHub | Gitlab | Bitbucket | CodeCommit |
|
||||
|-------|---------------------------------------------|:------:|:------:|:---------:|:----------:|
|
||||
| TOOLS | Review | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | ⮑ Inline review | :white_check_mark: | :white_check_mark: | | |
|
||||
| | Ask | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Auto-Description | :white_check_mark: | :white_check_mark: | | :white_check_mark: |
|
||||
| | Improve Code | :white_check_mark: | :white_check_mark: | | |
|
||||
| | ⮑ Extended | :white_check_mark: | :white_check_mark: | | |
|
||||
| | Reflect and Review | :white_check_mark: | | | |
|
||||
| | Update CHANGELOG.md | :white_check_mark: | | | |
|
||||
| | | | | | |
|
||||
| USAGE | CLI | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | App / webhook | :white_check_mark: | :white_check_mark: | | |
|
||||
| | Tagging bot | :white_check_mark: | | | |
|
||||
| | Actions | :white_check_mark: | | | |
|
||||
| | | | | | |
|
||||
| CORE | PR compression | :white_check_mark: | :white_check_mark: | :white_check_mark: | |
|
||||
| | Repo language prioritization | :white_check_mark: | :white_check_mark: | :white_check_mark: | |
|
||||
| | Adaptive and token-aware<br />file patch fitting | :white_check_mark: | :white_check_mark: | :white_check_mark: | |
|
||||
| | Multiple models support | :white_check_mark: | :white_check_mark: | :white_check_mark: | |
|
||||
| | Incremental PR Review | :white_check_mark: | | | |
|
||||
| | | GitHub | Gitlab | Bitbucket | CodeCommit | Azure DevOps | Gerrit |
|
||||
|-------|---------------------------------------------|:------:|:------:|:---------:|:----------:|:----------:|:----------:|
|
||||
| TOOLS | Review | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Ask | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Auto-Description | :white_check_mark: | :white_check_mark: | | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Improve Code | :white_check_mark: | :white_check_mark: | | :white_check_mark: | | :white_check_mark: |
|
||||
| | ⮑ Extended | :white_check_mark: | :white_check_mark: | | :white_check_mark: | | :white_check_mark: |
|
||||
| | Reflect and Review | :white_check_mark: | | | | :white_check_mark: | :white_check_mark: |
|
||||
| | Update CHANGELOG.md | :white_check_mark: | | | | | |
|
||||
| | | | | | | |
|
||||
| USAGE | CLI | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | App / webhook | :white_check_mark: | :white_check_mark: | | | |
|
||||
| | Tagging bot | :white_check_mark: | | | | |
|
||||
| | Actions | :white_check_mark: | | | | |
|
||||
| | Web server | | | | | | :white_check_mark: |
|
||||
| | | | | | | |
|
||||
| CORE | PR compression | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Repo language prioritization | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Adaptive and token-aware<br />file patch fitting | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Multiple models support | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Incremental PR Review | :white_check_mark: | | | | | |
|
||||
|
||||
Examples for invoking the different tools via the CLI:
|
||||
- **Review**: python cli.py --pr_url=<pr_url> review
|
||||
- **Describe**: python cli.py --pr_url=<pr_url> describe
|
||||
- **Improve**: python cli.py --pr_url=<pr_url> improve
|
||||
- **Ask**: python cli.py --pr_url=<pr_url> ask "Write me a poem about this PR"
|
||||
- **Reflect**: python cli.py --pr_url=<pr_url> reflect
|
||||
- **Update Changelog**: python cli.py --pr_url=<pr_url> update_changelog
|
||||
|
||||
"<pr_url>" is the url of the relevant PR (for example: https://github.com/Codium-ai/pr-agent/pull/50).
|
||||
|
||||
In the [configuration](./CONFIGURATION.md) file you can select your git provider (GitHub, Gitlab, Bitbucket), and further configure the different tools.
|
||||
Options that are available in the configuration file can be specified at run time when calling actions. Two examples:
|
||||
- /review --pr_reviewer.extra_instructions="focus on the file: ..."
|
||||
- /describe --pr_description.add_original_user_description=false -pr_description.extra_instructions="make sure to mention: ..."
|
||||
Review the **[usage guide](./Usage.md)** section for detailed instructions how to use the different tools, select the relevant git provider (GitHub, Gitlab, Bitbucket,...), and adjust the configuration file to your needs.
|
||||
|
||||
## Try it now
|
||||
|
||||
Try GPT-4 powered PR-Agent on your public GitHub repository for free. Just mention `@CodiumAI-Agent` and add the desired command in any PR comment! The agent will generate a response based on your command.
|
||||
You can try GPT-4 powered PR-Agent, on your public GitHub repository, instantly. Just mention `@CodiumAI-Agent` and add the desired command in any PR comment. The agent will generate a response based on your command.
|
||||
For example, add a comment to any pull request with the following text:
|
||||
```
|
||||
@CodiumAI-Agent /review
|
||||
```
|
||||
and the agent will respond with a review of your PR
|
||||
|
||||

|
||||
|
||||
To set up your own PR-Agent, see the [Installation](#installation) section
|
||||
|
||||
To set up your own PR-Agent, see the [Installation](#installation) section below.
|
||||
|
||||
---
|
||||
|
||||
@ -132,8 +146,8 @@ To get started with PR-Agent quickly, you first need to acquire two tokens:
|
||||
There are several ways to use PR-Agent:
|
||||
|
||||
- [Method 1: Use Docker image (no installation required)](INSTALL.md#method-1-use-docker-image-no-installation-required)
|
||||
- [Method 2: Run as a GitHub Action](INSTALL.md#method-2-run-as-a-github-action)
|
||||
- [Method 3: Run from source](INSTALL.md#method-3-run-from-source)
|
||||
- [Method 2: Run from source](INSTALL.md#method-2-run-from-source)
|
||||
- [Method 3: Run as a GitHub Action](INSTALL.md#method-3-run-as-a-github-action)
|
||||
- [Method 4: Run as a polling server](INSTALL.md#method-4-run-as-a-polling-server)
|
||||
- Request reviews by tagging your GitHub user on a PR
|
||||
- [Method 5: Run as a GitHub App](INSTALL.md#method-5-run-as-a-github-app)
|
||||
@ -157,7 +171,7 @@ Here are some advantages of PR-Agent:
|
||||
|
||||
- We emphasize **real-life practical usage**. Each tool (review, improve, ask, ...) has a single GPT-4 call, no more. We feel that this is critical for realistic team usage - obtaining an answer quickly (~30 seconds) and affordably.
|
||||
- Our [PR Compression strategy](./PR_COMPRESSION.md) is a core ability that enables to effectively tackle both short and long PRs.
|
||||
- Our JSON prompting strategy enables to have **modular, customizable tools**. For example, the '/review' tool categories can be controlled via the [configuration](./CONFIGURATION.md) file. Adding additional categories is easy and accessible.
|
||||
- Our JSON prompting strategy enables to have **modular, customizable tools**. For example, the '/review' tool categories can be controlled via the [configuration](pr_agent/settings/configuration.toml) file. Adding additional categories is easy and accessible.
|
||||
- We support **multiple git providers** (GitHub, Gitlab, Bitbucket, CodeCommit), **multiple ways** to use the tool (CLI, GitHub Action, GitHub App, Docker, ...), and **multiple models** (GPT-4, GPT-3.5, Anthropic, Cohere, Llama2).
|
||||
- We are open-source, and welcome contributions from the community.
|
||||
|
||||
@ -184,4 +198,4 @@ Here are some advantages of PR-Agent:
|
||||
- [Aider - GPT powered coding in your terminal](https://github.com/paul-gauthier/aider)
|
||||
- [openai-pr-reviewer](https://github.com/coderabbitai/openai-pr-reviewer)
|
||||
- [CodeReview BOT](https://github.com/anc95/ChatGPT-CodeReview)
|
||||
- [AI-Maintainer](https://github.com/merwanehamadi/AI-Maintainer)
|
||||
- [AI-Maintainer](https://github.com/merwanehamadi/AI-Maintainer)
|
182
Usage.md
Normal file
182
Usage.md
Normal file
@ -0,0 +1,182 @@
|
||||
## Usage guide
|
||||
|
||||
### Table of Contents
|
||||
- [Introduction](#introduction)
|
||||
- [Working from a local repo (CLI)](#working-from-a-local-repo-cli)
|
||||
- [Online usage](#online-usage)
|
||||
- [Working with GitHub App](#working-with-github-app)
|
||||
- [Working with GitHub Action](#working-with-github-action)
|
||||
- [Appendix - additional configurations walkthrough](#appendix---additional-configurations-walkthrough)
|
||||
|
||||
### Introduction
|
||||
|
||||
There are 3 basic ways to invoke CodiumAI PR-Agent:
|
||||
1. Locally running a CLI command
|
||||
2. Online usage - by [commenting](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021901) on a PR
|
||||
3. Enabling PR-Agent tools to run automatically when a new PR is opened
|
||||
|
||||
See the [installation guide](/INSTALL.md) for instructions on how to setup your own PR-Agent.
|
||||
|
||||
Specifically, CLI commands can be issued by invoking a pre-built [docker image](/INSTALL.md#running-from-source), or by invoking a [locally cloned repo](INSTALL.md#method-2-run-from-source).
|
||||
|
||||
For online usage, you will need to setup either a [GitHub App](INSTALL.md#method-5-run-as-a-github-app), or a [GitHub Action](INSTALL.md#method-3-run-as-a-github-action).
|
||||
GitHub App and GitHub Action also enable to run PR-Agent specific tool automatically when a new PR is opened.
|
||||
|
||||
|
||||
#### The configuration file
|
||||
The different tools and sub-tools used by CodiumAI PR-Agent are adjustable via the **[configuration file](pr_agent/settings/configuration.toml)**.
|
||||
In addition to general configuration options, each tool has its own configurations. For example, the `review` tool will use parameters from the [pr_reviewer](/pr_agent/settings/configuration.toml#L16) section in the configuration file.
|
||||
|
||||
**git provider:**
|
||||
The [git_provider](pr_agent/settings/configuration.toml#L4) field in the configuration file determines the GIT provider that will be used by PR-Agent. Currently, the following providers are supported:
|
||||
`
|
||||
"github", "gitlab", "azure", "codecommit", "local"
|
||||
`
|
||||
|
||||
[//]: # (** online usage:**)
|
||||
|
||||
[//]: # (Options that are available in the configuration file can be specified at run time when calling actions. Two examples:)
|
||||
|
||||
[//]: # (```)
|
||||
|
||||
[//]: # (- /review --pr_reviewer.extra_instructions="focus on the file: ...")
|
||||
|
||||
[//]: # (- /describe --pr_description.add_original_user_description=false -pr_description.extra_instructions="make sure to mention: ...")
|
||||
|
||||
[//]: # (```)
|
||||
|
||||
### Working from a local repo (CLI)
|
||||
When running from your local repo (CLI), your local configuration file will be used.
|
||||
|
||||
Examples for invoking the different tools via the CLI:
|
||||
|
||||
- **Review**: `python cli.py --pr_url=<pr_url> /review`
|
||||
- **Describe**: `python cli.py --pr_url=<pr_url> /describe`
|
||||
- **Improve**: `python cli.py --pr_url=<pr_url> /improve`
|
||||
- **Ask**: `python cli.py --pr_url=<pr_url> /ask "Write me a poem about this PR"`
|
||||
- **Reflect**: `python cli.py --pr_url=<pr_url> /reflect`
|
||||
- **Update Changelog**: `python cli.py --pr_url=<pr_url> /update_changelog`
|
||||
|
||||
`<pr_url>` is the url of the relevant PR (for example: https://github.com/Codium-ai/pr-agent/pull/50).
|
||||
|
||||
**Notes:**
|
||||
|
||||
(1) in addition to editing your local configuration file, you can also change any configuration value by adding it to the command line:
|
||||
```
|
||||
python cli.py --pr_url=<pr_url> /review --pr_reviewer.extra_instructions="focus on the file: ..."
|
||||
```
|
||||
|
||||
(2) You can print results locally, without publishing them, by setting in `configuration.toml`:
|
||||
```
|
||||
[config]
|
||||
publish_output=true
|
||||
verbosity_level=2
|
||||
```
|
||||
This is useful for debugging or experimenting with the different tools.
|
||||
|
||||
|
||||
### Online usage
|
||||
|
||||
Online usage means invoking PR-Agent tools by [comments](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021901) on a PR.
|
||||
Commands for invoking the different tools via comments:
|
||||
|
||||
- **Review**: `/review`
|
||||
- **Describe**: `/describe`
|
||||
- **Improve**: `/improve`
|
||||
- **Ask**: `/ask "..."`
|
||||
- **Reflect**: `/reflect`
|
||||
- **Update Changelog**: `/update_changelog`
|
||||
|
||||
|
||||
To edit a specific configuration value, just add `--config_path=<value>` to any command.
|
||||
For example if you want to edit the `review` tool configurations, you can run:
|
||||
```
|
||||
/review --pr_reviewer.extra_instructions="..." --pr_reviewer.require_score_review=false
|
||||
```
|
||||
Any configuration value in [configuration file](pr_agent/settings/configuration.toml) file can be similarly edited.
|
||||
|
||||
|
||||
### Working with GitHub App
|
||||
When running PR-Agent from [GitHub App](INSTALL.md#method-5-run-as-a-github-app), the default configurations from a pre-built repo will be initially loaded.
|
||||
|
||||
#### GitHub app automatic tools
|
||||
The [github_app](pr_agent/settings/configuration.toml#L56) section defines GitHub app specific configurations.
|
||||
An important parameter is `pr_commands`, which is a list of tools that will be **run automatically when a new PR is opened**:
|
||||
```
|
||||
[github_app]
|
||||
pr_commands = [
|
||||
"/describe --pr_description.add_original_user_description=true --pr_description.keep_original_user_title=true",
|
||||
"/auto_review",
|
||||
]
|
||||
```
|
||||
This means that when a new PR is opened, PR-Agent will run the `describe` and `auto_review` tools.
|
||||
For the describe tool, the `add_original_user_description` and `keep_original_user_title` parameters will be set to true.
|
||||
|
||||
However, you can override the default tool parameters by uploading a local configuration file called `.pr_agent.toml` to the root of your repo.
|
||||
For example, if your local `.pr_agent.toml` file contains:
|
||||
```
|
||||
[pr_description]
|
||||
add_original_user_description = false
|
||||
keep_original_user_title = false
|
||||
```
|
||||
When a new PR is opened, PR-Agent will run the `describe` tool with the above parameters.
|
||||
|
||||
Note that a local `.pr_agent.toml` file enables you to edit and customize the default parameters of any tool, not just the ones that are run automatically.
|
||||
|
||||
#### Editing the prompts
|
||||
The prompts for the various PR-Agent tools are defined in the `pr_agent/settings` folder.
|
||||
|
||||
In practice, the prompts are loaded and stored as a standard setting object.
|
||||
Hence, editing them is similar to editing any other configuration value - just place the relevant key in `.pr_agent.toml`file, and override the default value.
|
||||
|
||||
For example, if you want to edit the prompts of the [describe](./pr_agent/settings/pr_description_prompts.toml) tool, you can add the following to your `.pr_agent.toml` file:
|
||||
```
|
||||
[pr_description_prompt]
|
||||
system="""
|
||||
...
|
||||
"""
|
||||
user="""
|
||||
...
|
||||
"""
|
||||
```
|
||||
Note that the new prompt will need to generate an output compatible with the relevant [post-process function](./pr_agent/tools/pr_description.py#L137).
|
||||
|
||||
### Working with GitHub Action
|
||||
TBD
|
||||
|
||||
### Appendix - additional configurations walkthrough
|
||||
|
||||
#### Changing a model
|
||||
See [here](pr_agent/algo/__init__.py) for the list of available models.
|
||||
|
||||
To use Llama2 model, for example, set:
|
||||
```
|
||||
[config]
|
||||
model = "replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1"
|
||||
[replicate]
|
||||
key = ...
|
||||
```
|
||||
(you can obtain a Llama2 key from [here](https://replicate.com/replicate/llama-2-70b-chat/api))
|
||||
|
||||
Also review the [AiHandler](pr_agent/algo/ai_handler.py) file for instruction how to set keys for other models.
|
||||
|
||||
#### Extra instructions
|
||||
All PR-Agent tools have a parameter called `extra_instructions`, that enables to add free-text extra instructions. Example usage:
|
||||
```
|
||||
/update_changelog --pr_update_changelog.extra_instructions="Make sure to update also the version ..."
|
||||
```
|
||||
|
||||
#### Azure DevOps provider
|
||||
To use Azure DevOps provider use the following settings in configuration.toml:
|
||||
```
|
||||
[config]
|
||||
git_provider="azure"
|
||||
use_repo_settings_file=false
|
||||
```
|
||||
|
||||
And use the following settings (you have to replace the values) in .secrets.toml:
|
||||
```
|
||||
[azure_devops]
|
||||
org = "https://dev.azure.com/YOUR_ORGANIZATION/"
|
||||
pat = "YOUR_PAT_TOKEN"
|
||||
```
|
@ -5,6 +5,8 @@ from pr_agent.git_providers.github_provider import GithubProvider
|
||||
from pr_agent.git_providers.gitlab_provider import GitLabProvider
|
||||
from pr_agent.git_providers.local_git_provider import LocalGitProvider
|
||||
from pr_agent.git_providers.azuredevops_provider import AzureDevopsProvider
|
||||
from pr_agent.git_providers.gerrit_provider import GerritProvider
|
||||
|
||||
|
||||
_GIT_PROVIDERS = {
|
||||
'github': GithubProvider,
|
||||
@ -12,7 +14,8 @@ _GIT_PROVIDERS = {
|
||||
'bitbucket': BitbucketProvider,
|
||||
'azure': AzureDevopsProvider,
|
||||
'codecommit': CodeCommitProvider,
|
||||
'local' : LocalGitProvider
|
||||
'local' : LocalGitProvider,
|
||||
'gerrit': GerritProvider,
|
||||
}
|
||||
|
||||
def get_git_provider():
|
||||
|
@ -90,7 +90,11 @@ class CodeCommitClient:
|
||||
):
|
||||
differences.extend(page.get("differences", []))
|
||||
except botocore.exceptions.ClientError as e:
|
||||
raise ValueError(f"Failed to retrieve differences from CodeCommit PR #{self.pr_num}") from e
|
||||
if e.response["Error"]["Code"] == 'RepositoryDoesNotExistException':
|
||||
raise ValueError(f"CodeCommit cannot retrieve differences: Repository does not exist: {repo_name}") from e
|
||||
raise ValueError(f"CodeCommit cannot retrieve differences for {source_commit}..{destination_commit}") from e
|
||||
except Exception as e:
|
||||
raise ValueError(f"CodeCommit cannot retrieve differences for {source_commit}..{destination_commit}") from e
|
||||
|
||||
output = []
|
||||
for json in differences:
|
||||
@ -122,6 +126,8 @@ class CodeCommitClient:
|
||||
try:
|
||||
response = self.boto_client.get_file(repositoryName=repo_name, commitSpecifier=sha_hash, filePath=file_path)
|
||||
except botocore.exceptions.ClientError as e:
|
||||
if e.response["Error"]["Code"] == 'RepositoryDoesNotExistException':
|
||||
raise ValueError(f"CodeCommit cannot retrieve PR: Repository does not exist: {repo_name}") from e
|
||||
# if the file does not exist, but is flagged as optional, then return an empty string
|
||||
if optional and e.response["Error"]["Code"] == 'FileDoesNotExistException':
|
||||
return ""
|
||||
@ -133,11 +139,12 @@ class CodeCommitClient:
|
||||
|
||||
return response.get("fileContent", "")
|
||||
|
||||
def get_pr(self, pr_number: int):
|
||||
def get_pr(self, repo_name: str, pr_number: int):
|
||||
"""
|
||||
Get a information about a CodeCommit PR.
|
||||
|
||||
Args:
|
||||
- repo_name: Name of the repository
|
||||
- pr_number: The PR number you are requesting
|
||||
|
||||
Returns:
|
||||
@ -155,6 +162,8 @@ class CodeCommitClient:
|
||||
except botocore.exceptions.ClientError as e:
|
||||
if e.response["Error"]["Code"] == 'PullRequestDoesNotExistException':
|
||||
raise ValueError(f"CodeCommit cannot retrieve PR: PR number does not exist: {pr_number}") from e
|
||||
if e.response["Error"]["Code"] == 'RepositoryDoesNotExistException':
|
||||
raise ValueError(f"CodeCommit cannot retrieve PR: Repository does not exist: {repo_name}") from e
|
||||
raise ValueError(f"CodeCommit cannot retrieve PR: {pr_number}: boto client error") from e
|
||||
except Exception as e:
|
||||
raise ValueError(f"CodeCommit cannot retrieve PR: {pr_number}") from e
|
||||
@ -201,7 +210,7 @@ class CodeCommitClient:
|
||||
except Exception as e:
|
||||
raise ValueError(f"Error calling publish_description") from e
|
||||
|
||||
def publish_comment(self, repo_name: str, pr_number: int, destination_commit: str, source_commit: str, comment: str):
|
||||
def publish_comment(self, repo_name: str, pr_number: int, destination_commit: str, source_commit: str, comment: str, annotation_file: str = None, annotation_line: int = None):
|
||||
"""
|
||||
Publish a comment to a pull request
|
||||
|
||||
@ -210,7 +219,13 @@ class CodeCommitClient:
|
||||
- pr_number: number of the pull request
|
||||
- destination_commit: The commit hash you want to merge into (the "before" hash) (usually on the main or master branch)
|
||||
- source_commit: The commit hash of the code you are adding (the "after" branch)
|
||||
- pr_comment: comment
|
||||
- comment: The comment you want to publish
|
||||
- annotation_file: The file you want to annotate (optional)
|
||||
- annotation_line: The line number you want to annotate (optional)
|
||||
|
||||
Comment annotations for CodeCommit are different than GitHub.
|
||||
CodeCommit only designates the starting line number for the comment.
|
||||
It does not support the ending line number to highlight a range of lines.
|
||||
|
||||
Returns:
|
||||
- None
|
||||
@ -223,13 +238,30 @@ class CodeCommitClient:
|
||||
self._connect_boto_client()
|
||||
|
||||
try:
|
||||
self.boto_client.post_comment_for_pull_request(
|
||||
pullRequestId=str(pr_number),
|
||||
repositoryName=repo_name,
|
||||
beforeCommitId=destination_commit,
|
||||
afterCommitId=source_commit,
|
||||
content=comment,
|
||||
)
|
||||
# If the comment has code annotations,
|
||||
# then set the file path and line number in the location dictionary
|
||||
if annotation_file and annotation_line:
|
||||
self.boto_client.post_comment_for_pull_request(
|
||||
pullRequestId=str(pr_number),
|
||||
repositoryName=repo_name,
|
||||
beforeCommitId=destination_commit,
|
||||
afterCommitId=source_commit,
|
||||
content=comment,
|
||||
location={
|
||||
"filePath": annotation_file,
|
||||
"filePosition": annotation_line,
|
||||
"relativeFileVersion": "AFTER",
|
||||
},
|
||||
)
|
||||
else:
|
||||
# The comment does not have code annotations
|
||||
self.boto_client.post_comment_for_pull_request(
|
||||
pullRequestId=str(pr_number),
|
||||
repositoryName=repo_name,
|
||||
beforeCommitId=destination_commit,
|
||||
afterCommitId=source_commit,
|
||||
content=comment,
|
||||
)
|
||||
except botocore.exceptions.ClientError as e:
|
||||
if e.response["Error"]["Code"] == 'RepositoryDoesNotExistException':
|
||||
raise ValueError(f"Repository does not exist: {repo_name}") from e
|
||||
|
@ -180,10 +180,37 @@ class CodeCommitProvider(GitProvider):
|
||||
comment=pr_comment,
|
||||
)
|
||||
except Exception as e:
|
||||
raise ValueError(f"CodeCommit Cannot post comment for PR: {self.pr_num}") from e
|
||||
raise ValueError(f"CodeCommit Cannot publish comment for PR: {self.pr_num}") from e
|
||||
|
||||
def publish_code_suggestions(self, code_suggestions: list) -> bool:
|
||||
return [""] # not implemented yet
|
||||
counter = 1
|
||||
for suggestion in code_suggestions:
|
||||
# Verify that each suggestion has the required keys
|
||||
if not all(key in suggestion for key in ["body", "relevant_file", "relevant_lines_start"]):
|
||||
logging.warning(f"Skipping code suggestion #{counter}: Each suggestion must have 'body', 'relevant_file', 'relevant_lines_start' keys")
|
||||
continue
|
||||
|
||||
# Publish the code suggestion to CodeCommit
|
||||
try:
|
||||
logging.debug(f"Code Suggestion #{counter} in file: {suggestion['relevant_file']}: {suggestion['relevant_lines_start']}")
|
||||
self.codecommit_client.publish_comment(
|
||||
repo_name=self.repo_name,
|
||||
pr_number=self.pr_num,
|
||||
destination_commit=self.pr.destination_commit,
|
||||
source_commit=self.pr.source_commit,
|
||||
comment=suggestion["body"],
|
||||
annotation_file=suggestion["relevant_file"],
|
||||
annotation_line=suggestion["relevant_lines_start"],
|
||||
)
|
||||
except Exception as e:
|
||||
raise ValueError(f"CodeCommit Cannot publish code suggestions for PR: {self.pr_num}") from e
|
||||
|
||||
counter += 1
|
||||
|
||||
# The calling function passes in a list of code suggestions, and this function publishes each suggestion one at a time.
|
||||
# If we were to return False here, the calling function will attempt to publish the same list of code suggestions again, one at a time.
|
||||
# Since this function publishes the suggestions one at a time anyway, we always return True here to avoid the retry.
|
||||
return True
|
||||
|
||||
def publish_labels(self, labels):
|
||||
return [""] # not implemented yet
|
||||
@ -195,6 +222,7 @@ class CodeCommitProvider(GitProvider):
|
||||
return "" # not implemented yet
|
||||
|
||||
def publish_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
|
||||
# https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/post_comment_for_compared_commit.html
|
||||
raise NotImplementedError("CodeCommit provider does not support publishing inline comments yet")
|
||||
|
||||
def create_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
|
||||
@ -255,9 +283,11 @@ class CodeCommitProvider(GitProvider):
|
||||
return self.codecommit_client.get_file(self.repo_name, settings_filename, self.pr.source_commit, optional=True)
|
||||
|
||||
def add_eyes_reaction(self, issue_comment_id: int) -> Optional[int]:
|
||||
logging.info("CodeCommit provider does not support eyes reaction yet")
|
||||
return True
|
||||
|
||||
def remove_reaction(self, issue_comment_id: int, reaction_id: int) -> bool:
|
||||
logging.info("CodeCommit provider does not support removing reactions yet")
|
||||
return True
|
||||
|
||||
@staticmethod
|
||||
@ -315,16 +345,16 @@ class CodeCommitProvider(GitProvider):
|
||||
return re.match(r"^[a-z]{2}-(gov-)?[a-z]+-\d\.console\.aws\.amazon\.com$", hostname) is not None
|
||||
|
||||
def _get_pr(self):
|
||||
response = self.codecommit_client.get_pr(self.pr_num)
|
||||
response = self.codecommit_client.get_pr(self.repo_name, self.pr_num)
|
||||
|
||||
if len(response.targets) == 0:
|
||||
raise ValueError(f"No files found in CodeCommit PR: {self.pr_num}")
|
||||
|
||||
# TODO: implement support for multiple commits in one CodeCommit PR
|
||||
# for now, we are only using the first commit in the PR
|
||||
# TODO: implement support for multiple targets in one CodeCommit PR
|
||||
# for now, we are only using the first target in the PR
|
||||
if len(response.targets) > 1:
|
||||
logging.warning(
|
||||
"Multiple commits in one PR is not supported for CodeCommit yet. Continuing, using the first commit only..."
|
||||
"Multiple targets in one PR is not supported for CodeCommit yet. Continuing, using the first target only..."
|
||||
)
|
||||
|
||||
# Return our object that mimics PullRequest class from the PyGithub library
|
||||
|
393
pr_agent/git_providers/gerrit_provider.py
Normal file
393
pr_agent/git_providers/gerrit_provider.py
Normal file
@ -0,0 +1,393 @@
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import pathlib
|
||||
import shutil
|
||||
import subprocess
|
||||
import uuid
|
||||
from collections import Counter, namedtuple
|
||||
from pathlib import Path
|
||||
from tempfile import mkdtemp, NamedTemporaryFile
|
||||
|
||||
import requests
|
||||
import urllib3.util
|
||||
from git import Repo
|
||||
|
||||
from pr_agent.config_loader import get_settings
|
||||
from pr_agent.git_providers.git_provider import GitProvider, FilePatchInfo, \
|
||||
EDIT_TYPE
|
||||
from pr_agent.git_providers.local_git_provider import PullRequestMimic
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _call(*command, **kwargs) -> (int, str, str):
|
||||
res = subprocess.run(
|
||||
command,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
check=True,
|
||||
**kwargs,
|
||||
)
|
||||
return res.stdout.decode()
|
||||
|
||||
|
||||
def clone(url, directory):
|
||||
logger.info("Cloning %s to %s", url, directory)
|
||||
stdout = _call('git', 'clone', "--depth", "1", url, directory)
|
||||
logger.info(stdout)
|
||||
|
||||
|
||||
def fetch(url, refspec, cwd):
|
||||
logger.info("Fetching %s %s", url, refspec)
|
||||
stdout = _call(
|
||||
'git', 'fetch', '--depth', '2', url, refspec,
|
||||
cwd=cwd
|
||||
)
|
||||
logger.info(stdout)
|
||||
|
||||
|
||||
def checkout(cwd):
|
||||
logger.info("Checking out")
|
||||
stdout = _call('git', 'checkout', "FETCH_HEAD", cwd=cwd)
|
||||
logger.info(stdout)
|
||||
|
||||
|
||||
def show(*args, cwd=None):
|
||||
logger.info("Show")
|
||||
return _call('git', 'show', *args, cwd=cwd)
|
||||
|
||||
|
||||
def diff(*args, cwd=None):
|
||||
logger.info("Diff")
|
||||
patch = _call('git', 'diff', *args, cwd=cwd)
|
||||
if not patch:
|
||||
logger.warning("No changes found")
|
||||
return
|
||||
return patch
|
||||
|
||||
|
||||
def reset_local_changes(cwd):
|
||||
logger.info("Reset local changes")
|
||||
_call('git', 'checkout', "--force", cwd=cwd)
|
||||
|
||||
|
||||
def add_comment(url: urllib3.util.Url, refspec, message):
|
||||
*_, patchset, changenum = refspec.rsplit("/")
|
||||
message = "'" + message.replace("'", "'\"'\"'") + "'"
|
||||
return _call(
|
||||
"ssh",
|
||||
"-p", str(url.port),
|
||||
f"{url.auth}@{url.host}",
|
||||
"gerrit", "review",
|
||||
"--message", message,
|
||||
# "--code-review", score,
|
||||
f"{patchset},{changenum}",
|
||||
)
|
||||
|
||||
|
||||
def list_comments(url: urllib3.util.Url, refspec):
|
||||
*_, patchset, _ = refspec.rsplit("/")
|
||||
stdout = _call(
|
||||
"ssh",
|
||||
"-p", str(url.port),
|
||||
f"{url.auth}@{url.host}",
|
||||
"gerrit", "query",
|
||||
"--comments",
|
||||
"--current-patch-set", patchset,
|
||||
"--format", "JSON",
|
||||
)
|
||||
change_set, *_ = stdout.splitlines()
|
||||
return json.loads(change_set)["currentPatchSet"]["comments"]
|
||||
|
||||
|
||||
def prepare_repo(url: urllib3.util.Url, project, refspec):
|
||||
repo_url = (f"{url.scheme}://{url.auth}@{url.host}:{url.port}/{project}")
|
||||
|
||||
directory = pathlib.Path(mkdtemp())
|
||||
clone(repo_url, directory),
|
||||
fetch(repo_url, refspec, cwd=directory)
|
||||
checkout(cwd=directory)
|
||||
return directory
|
||||
|
||||
|
||||
def adopt_to_gerrit_message(message):
|
||||
lines = message.splitlines()
|
||||
buf = []
|
||||
for line in lines:
|
||||
line = line.replace("*", "").replace("``", "`")
|
||||
line = line.strip()
|
||||
if line.startswith('#'):
|
||||
buf.append("\n" +
|
||||
line.replace('#', '').removesuffix(":").strip() +
|
||||
":")
|
||||
continue
|
||||
elif line.startswith('-'):
|
||||
buf.append(line.removeprefix('-').strip())
|
||||
continue
|
||||
else:
|
||||
buf.append(line)
|
||||
return "\n".join(buf).strip()
|
||||
|
||||
|
||||
def add_suggestion(src_filename, context: str, start, end: int):
|
||||
with (
|
||||
NamedTemporaryFile("w", delete=False) as tmp,
|
||||
open(src_filename, "r") as src
|
||||
):
|
||||
lines = src.readlines()
|
||||
tmp.writelines(lines[:start - 1])
|
||||
if context:
|
||||
tmp.write(context)
|
||||
tmp.writelines(lines[end:])
|
||||
|
||||
shutil.copy(tmp.name, src_filename)
|
||||
os.remove(tmp.name)
|
||||
|
||||
|
||||
def upload_patch(patch, path):
|
||||
patch_server_endpoint = get_settings().get(
|
||||
'gerrit.patch_server_endpoint')
|
||||
patch_server_token = get_settings().get(
|
||||
'gerrit.patch_server_token')
|
||||
|
||||
response = requests.post(
|
||||
patch_server_endpoint,
|
||||
json={
|
||||
"content": patch,
|
||||
"path": path,
|
||||
},
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {patch_server_token}",
|
||||
}
|
||||
)
|
||||
response.raise_for_status()
|
||||
patch_server_endpoint = patch_server_endpoint.rstrip("/")
|
||||
return patch_server_endpoint + "/" + path
|
||||
|
||||
|
||||
class GerritProvider(GitProvider):
|
||||
|
||||
def __init__(self, key: str, incremental=False):
|
||||
self.project, self.refspec = key.split(':')
|
||||
assert self.project, "Project name is required"
|
||||
assert self.refspec, "Refspec is required"
|
||||
base_url = get_settings().get('gerrit.url')
|
||||
assert base_url, "Gerrit URL is required"
|
||||
user = get_settings().get('gerrit.user')
|
||||
assert user, "Gerrit user is required"
|
||||
|
||||
parsed = urllib3.util.parse_url(base_url)
|
||||
self.parsed_url = urllib3.util.parse_url(
|
||||
f"{parsed.scheme}://{user}@{parsed.host}:{parsed.port}"
|
||||
)
|
||||
|
||||
self.repo_path = prepare_repo(
|
||||
self.parsed_url, self.project, self.refspec
|
||||
)
|
||||
self.repo = Repo(self.repo_path)
|
||||
assert self.repo
|
||||
|
||||
self.pr = PullRequestMimic(self.get_pr_title(), self.get_diff_files())
|
||||
|
||||
def get_pr_title(self):
|
||||
"""
|
||||
Substitutes the branch-name as the PR-mimic title.
|
||||
"""
|
||||
return self.repo.branches[0].name
|
||||
|
||||
def get_issue_comments(self):
|
||||
comments = list_comments(self.parsed_url, self.refspec)
|
||||
Comments = namedtuple('Comments', ['reversed'])
|
||||
Comment = namedtuple('Comment', ['body'])
|
||||
return Comments([Comment(c['message']) for c in reversed(comments)])
|
||||
|
||||
def get_labels(self):
|
||||
raise NotImplementedError(
|
||||
'Getting labels is not implemented for the gerrit provider')
|
||||
|
||||
def add_eyes_reaction(self, issue_comment_id: int):
|
||||
raise NotImplementedError(
|
||||
'Adding reactions is not implemented for the gerrit provider')
|
||||
|
||||
def remove_reaction(self, issue_comment_id: int, reaction_id: int):
|
||||
raise NotImplementedError(
|
||||
'Removing reactions is not implemented for the gerrit provider')
|
||||
|
||||
def get_commit_messages(self):
|
||||
return [self.repo.head.commit.message]
|
||||
|
||||
def get_repo_settings(self):
|
||||
"""
|
||||
TODO: Implement support of .pr_agent.toml
|
||||
"""
|
||||
return ""
|
||||
|
||||
def get_diff_files(self) -> list[FilePatchInfo]:
|
||||
diffs = self.repo.head.commit.diff(
|
||||
self.repo.head.commit.parents[0], # previous commit
|
||||
create_patch=True,
|
||||
R=True
|
||||
)
|
||||
|
||||
diff_files = []
|
||||
for diff_item in diffs:
|
||||
if diff_item.a_blob is not None:
|
||||
original_file_content_str = (
|
||||
diff_item.a_blob.data_stream.read().decode('utf-8')
|
||||
)
|
||||
else:
|
||||
original_file_content_str = "" # empty file
|
||||
if diff_item.b_blob is not None:
|
||||
new_file_content_str = diff_item.b_blob.data_stream.read(). \
|
||||
decode('utf-8')
|
||||
else:
|
||||
new_file_content_str = "" # empty file
|
||||
edit_type = EDIT_TYPE.MODIFIED
|
||||
if diff_item.new_file:
|
||||
edit_type = EDIT_TYPE.ADDED
|
||||
elif diff_item.deleted_file:
|
||||
edit_type = EDIT_TYPE.DELETED
|
||||
elif diff_item.renamed_file:
|
||||
edit_type = EDIT_TYPE.RENAMED
|
||||
diff_files.append(
|
||||
FilePatchInfo(
|
||||
original_file_content_str,
|
||||
new_file_content_str,
|
||||
diff_item.diff.decode('utf-8'),
|
||||
diff_item.b_path,
|
||||
edit_type=edit_type,
|
||||
old_filename=None
|
||||
if diff_item.a_path == diff_item.b_path
|
||||
else diff_item.a_path
|
||||
)
|
||||
)
|
||||
self.diff_files = diff_files
|
||||
return diff_files
|
||||
|
||||
def get_files(self):
|
||||
diff_index = self.repo.head.commit.diff(
|
||||
self.repo.head.commit.parents[0], # previous commit
|
||||
R=True
|
||||
)
|
||||
# Get the list of changed files
|
||||
diff_files = [item.a_path for item in diff_index]
|
||||
return diff_files
|
||||
|
||||
def get_languages(self):
|
||||
"""
|
||||
Calculate percentage of languages in repository. Used for hunk
|
||||
prioritisation.
|
||||
"""
|
||||
# Get all files in repository
|
||||
filepaths = [Path(item.path) for item in
|
||||
self.repo.tree().traverse() if item.type == 'blob']
|
||||
# Identify language by file extension and count
|
||||
lang_count = Counter(
|
||||
ext.lstrip('.') for filepath in filepaths for ext in
|
||||
[filepath.suffix.lower()])
|
||||
# Convert counts to percentages
|
||||
total_files = len(filepaths)
|
||||
lang_percentage = {lang: count / total_files * 100 for lang, count
|
||||
in lang_count.items()}
|
||||
return lang_percentage
|
||||
|
||||
def get_pr_description_full(self):
|
||||
return self.repo.head.commit.message
|
||||
|
||||
def get_user_id(self):
|
||||
return self.repo.head.commit.author.email
|
||||
|
||||
def is_supported(self, capability: str) -> bool:
|
||||
if capability in [
|
||||
# 'get_issue_comments',
|
||||
'create_inline_comment',
|
||||
'publish_inline_comments',
|
||||
'get_labels'
|
||||
]:
|
||||
return False
|
||||
return True
|
||||
|
||||
def split_suggestion(self, msg) -> tuple[str, str]:
|
||||
is_code_context = False
|
||||
description = []
|
||||
context = []
|
||||
for line in msg.splitlines():
|
||||
if line.startswith('```suggestion'):
|
||||
is_code_context = True
|
||||
continue
|
||||
if line.startswith('```'):
|
||||
is_code_context = False
|
||||
continue
|
||||
if is_code_context:
|
||||
context.append(line)
|
||||
else:
|
||||
description.append(
|
||||
line.replace('*', '')
|
||||
)
|
||||
|
||||
return (
|
||||
'\n'.join(description),
|
||||
'\n'.join(context) + '\n' if context else ''
|
||||
)
|
||||
|
||||
def publish_code_suggestions(self, code_suggestions: list):
|
||||
msg = []
|
||||
for suggestion in code_suggestions:
|
||||
description, code = self.split_suggestion(suggestion['body'])
|
||||
add_suggestion(
|
||||
pathlib.Path(self.repo_path) / suggestion["relevant_file"],
|
||||
code,
|
||||
suggestion["relevant_lines_start"],
|
||||
suggestion["relevant_lines_end"],
|
||||
)
|
||||
patch = diff(cwd=self.repo_path)
|
||||
patch_id = uuid.uuid4().hex[0:4]
|
||||
path = "/".join(["codium-ai", self.refspec, patch_id])
|
||||
full_path = upload_patch(patch, path)
|
||||
reset_local_changes(self.repo_path)
|
||||
msg.append(f'* {description}\n{full_path}')
|
||||
|
||||
if msg:
|
||||
add_comment(self.parsed_url, self.refspec, "\n".join(msg))
|
||||
return True
|
||||
|
||||
def publish_comment(self, pr_comment: str, is_temporary: bool = False):
|
||||
if not is_temporary:
|
||||
msg = adopt_to_gerrit_message(pr_comment)
|
||||
add_comment(self.parsed_url, self.refspec, msg)
|
||||
|
||||
def publish_description(self, pr_title: str, pr_body: str):
|
||||
msg = adopt_to_gerrit_message(pr_body)
|
||||
add_comment(self.parsed_url, self.refspec, pr_title + '\n' + msg)
|
||||
|
||||
def publish_inline_comments(self, comments: list[dict]):
|
||||
raise NotImplementedError(
|
||||
'Publishing inline comments is not implemented for the gerrit '
|
||||
'provider')
|
||||
|
||||
def publish_inline_comment(self, body: str, relevant_file: str,
|
||||
relevant_line_in_file: str):
|
||||
raise NotImplementedError(
|
||||
'Publishing inline comments is not implemented for the gerrit '
|
||||
'provider')
|
||||
|
||||
def create_inline_comment(self, body: str, relevant_file: str,
|
||||
relevant_line_in_file: str):
|
||||
raise NotImplementedError(
|
||||
'Creating inline comments is not implemented for the gerrit '
|
||||
'provider')
|
||||
|
||||
def publish_labels(self, labels):
|
||||
# Not applicable to the local git provider,
|
||||
# but required by the interface
|
||||
pass
|
||||
|
||||
def remove_initial_comment(self):
|
||||
# remove repo, cloned in previous steps
|
||||
# shutil.rmtree(self.repo_path)
|
||||
pass
|
||||
|
||||
def get_pr_branch(self):
|
||||
return self.repo.head
|
@ -161,7 +161,6 @@ def get_main_pr_language(languages, files) -> str:
|
||||
most_common_extension == 'scala' and top_language == 'scala' or \
|
||||
most_common_extension == 'kt' and top_language == 'kotlin' or \
|
||||
most_common_extension == 'pl' and top_language == 'perl' or \
|
||||
most_common_extension == 'swift' and top_language == 'swift' or \
|
||||
most_common_extension == top_language:
|
||||
main_language_str = top_language
|
||||
|
||||
|
78
pr_agent/servers/gerrit_server.py
Normal file
78
pr_agent/servers/gerrit_server.py
Normal file
@ -0,0 +1,78 @@
|
||||
import copy
|
||||
import logging
|
||||
import sys
|
||||
from enum import Enum
|
||||
from json import JSONDecodeError
|
||||
|
||||
import uvicorn
|
||||
from fastapi import APIRouter, FastAPI, HTTPException
|
||||
from pydantic import BaseModel
|
||||
from starlette.middleware import Middleware
|
||||
from starlette_context import context
|
||||
from starlette_context.middleware import RawContextMiddleware
|
||||
|
||||
from pr_agent.agent.pr_agent import PRAgent
|
||||
from pr_agent.config_loader import global_settings, get_settings
|
||||
|
||||
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
class Action(str, Enum):
|
||||
review = "review"
|
||||
describe = "describe"
|
||||
ask = "ask"
|
||||
improve = "improve"
|
||||
reflect = "reflect"
|
||||
answer = "answer"
|
||||
|
||||
|
||||
class Item(BaseModel):
|
||||
refspec: str
|
||||
project: str
|
||||
msg: str
|
||||
|
||||
|
||||
@router.post("/api/v1/gerrit/{action}")
|
||||
async def handle_gerrit_request(action: Action, item: Item):
|
||||
logging.debug("Received a Gerrit request")
|
||||
context["settings"] = copy.deepcopy(global_settings)
|
||||
|
||||
if action == Action.ask:
|
||||
if not item.msg:
|
||||
return HTTPException(
|
||||
status_code=400,
|
||||
detail="msg is required for ask command"
|
||||
)
|
||||
await PRAgent().handle_request(
|
||||
f"{item.project}:{item.refspec}",
|
||||
f"/{item.msg.strip()}"
|
||||
)
|
||||
|
||||
|
||||
async def get_body(request):
|
||||
try:
|
||||
body = await request.json()
|
||||
except JSONDecodeError as e:
|
||||
logging.error("Error parsing request body", e)
|
||||
return {}
|
||||
return body
|
||||
|
||||
|
||||
@router.get("/")
|
||||
async def root():
|
||||
return {"status": "ok"}
|
||||
|
||||
|
||||
def start():
|
||||
# to prevent adding help messages with the output
|
||||
get_settings().set("CONFIG.CLI_MODE", True)
|
||||
middleware = [Middleware(RawContextMiddleware)]
|
||||
app = FastAPI(middleware=middleware)
|
||||
app.include_router(router)
|
||||
|
||||
uvicorn.run(app, host="0.0.0.0", port=3000)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
start()
|
@ -84,4 +84,14 @@ polling_interval_seconds = 30
|
||||
[local]
|
||||
# LocalGitProvider settings - uncomment to use paths other than default
|
||||
# description_path= "path/to/description.md"
|
||||
# review_path= "path/to/review.md"
|
||||
# review_path= "path/to/review.md"
|
||||
|
||||
[gerrit]
|
||||
# endpoint to the gerrit service
|
||||
# url = "ssh://gerrit.example.com:29418"
|
||||
# user for gerrit authentication
|
||||
# user = "ai-reviewer"
|
||||
# patch server where patches will be saved
|
||||
# patch_server_endpoint = "http://127.0.0.1:5000/patch"
|
||||
# token to authenticate in the patch server
|
||||
# patch_server_token = ""
|
||||
|
@ -68,12 +68,17 @@ Code suggestions:
|
||||
type: string
|
||||
description: |-
|
||||
a code snippet showing the relevant code lines from a '__new hunk__' section.
|
||||
It must be continuous, correctly formatted and indented, and without line numbers.
|
||||
relevant lines:
|
||||
type: string
|
||||
It must be contiguous, correctly formatted and indented, and without line numbers.
|
||||
relevant lines start:
|
||||
type: integer
|
||||
description: |-
|
||||
the relevant lines from a '__new hunk__' section, in the format of 'start_line-end_line'.
|
||||
For example: '10-15'. They should be derived from the hunk line numbers, and correspond to the 'existing code' snippet above.
|
||||
The relevant line number from a '__new hunk__' section where the suggestion starts (inclusive).
|
||||
Should be derived from the hunk line numbers, and correspond to the 'existing code' snippet above.
|
||||
relevant lines end:
|
||||
type: integer
|
||||
description: |-
|
||||
The relevant line number from a '__new hunk__' section where the suggestion ends (inclusive).
|
||||
Should be derived from the hunk line numbers, and correspond to the 'existing code' snippet above.
|
||||
improved code:
|
||||
type: string
|
||||
description: |-
|
||||
@ -90,7 +95,8 @@ Code suggestions:
|
||||
Add a docstring to func1()
|
||||
existing code: |-
|
||||
def func1():
|
||||
relevant lines: '12-12'
|
||||
relevant lines start: 12
|
||||
relevant lines end: 12
|
||||
improved code: |-
|
||||
...
|
||||
```
|
||||
|
@ -113,11 +113,8 @@ class PRCodeSuggestions:
|
||||
if get_settings().config.verbosity_level >= 2:
|
||||
logging.info(f"suggestion: {d}")
|
||||
relevant_file = d['relevant file'].strip()
|
||||
relevant_lines_str = d['relevant lines'].strip()
|
||||
if ',' in relevant_lines_str: # handling 'relevant lines': '181, 190' or '178-184, 188-194'
|
||||
relevant_lines_str = relevant_lines_str.split(',')[0]
|
||||
relevant_lines_start = int(relevant_lines_str.split('-')[0]) # absolute position
|
||||
relevant_lines_end = int(relevant_lines_str.split('-')[-1])
|
||||
relevant_lines_start = int(d['relevant lines start']) # absolute position
|
||||
relevant_lines_end = int(d['relevant lines end'])
|
||||
content = d['suggestion content']
|
||||
new_code_snippet = d['improved code']
|
||||
|
||||
|
@ -17,3 +17,5 @@ litellm~=0.1.504
|
||||
boto3~=1.28.25
|
||||
google-cloud-storage==2.10.0
|
||||
ujson==5.8.0
|
||||
azure-devops==7.1.0b3
|
||||
msrest==0.7.1
|
@ -125,7 +125,7 @@ class TestCodeCommitProvider:
|
||||
}
|
||||
}
|
||||
|
||||
pr = api.get_pr(321)
|
||||
pr = api.get_pr("my_test_repo", 321)
|
||||
|
||||
assert pr.title == "My PR"
|
||||
assert pr.description == "My PR description"
|
||||
|
Reference in New Issue
Block a user