mirror of
https://github.com/qodo-ai/pr-agent.git
synced 2025-07-05 13:20:39 +08:00
Compare commits
74 Commits
pre_pr
...
ok/gitlab_
Author | SHA1 | Date | |
---|---|---|---|
55b3c3fe5c | |||
1443df7227 | |||
739b63f73b | |||
4a54532b6a | |||
0dbe64e401 | |||
53ce609266 | |||
7584ec84ce | |||
140760c517 | |||
56e9493f7a | |||
958ecf333a | |||
ae3d7067d3 | |||
a49e81d959 | |||
916d7c236e | |||
6343d35616 | |||
0203086aac | |||
0066156aca | |||
544bac7010 | |||
34090b078b | |||
9567199bb2 | |||
1f7a833a54 | |||
990f69a95d | |||
2b8a8ce824 | |||
6585854c85 | |||
98019fe97f | |||
d52c11b907 | |||
e79bcbed93 | |||
690c819479 | |||
630d1d9e03 | |||
20c32375e1 | |||
44b790567b | |||
4d6d6c4812 | |||
7f6493009c | |||
7a6efbcb55 | |||
777c773a90 | |||
f7c698ff54 | |||
1b780c0496 | |||
2e095807b7 | |||
ae98cfe17b | |||
35a6eb2e52 | |||
8b477c694c | |||
1254ad1727 | |||
eeea38dab3 | |||
8983fd9071 | |||
918ae25654 | |||
de39595522 | |||
4c6595148b | |||
02e0f958e7 | |||
be19b64542 | |||
24900305d6 | |||
06d00032df | |||
244cbbd27f | |||
8263bf5f9c | |||
8823d8c0e9 | |||
5cbcef276c | |||
ce9014073c | |||
376c4523dd | |||
e0ca594a69 | |||
48233fde23 | |||
9c05a6b1b5 | |||
da848d7e39 | |||
c6c97ac98a | |||
92e23ff260 | |||
aa03654ffc | |||
85130c0d30 | |||
3c27432f50 | |||
eec62c14dc | |||
ad6dd38fe3 | |||
307b3b4bf7 | |||
8e7e13ab62 | |||
bd085e610a | |||
d64b1f80da | |||
f26264daf1 | |||
edaeb99b43 | |||
ce54a7b79e |
@ -1,57 +0,0 @@
|
||||
## Configuration
|
||||
|
||||
The different tools and sub-tools used by CodiumAI PR-Agent are adjustable via the **[configuration file](pr_agent/settings/configuration.toml)**
|
||||
|
||||
### Working from CLI
|
||||
When running from source (CLI), your local configuration file will be initially used.
|
||||
|
||||
Example for invoking the 'review' tools via the CLI:
|
||||
|
||||
```
|
||||
python cli.py --pr-url=<pr_url> review
|
||||
```
|
||||
In addition to general configurations, the 'review' tool will use parameters from the `[pr_reviewer]` section (every tool has a dedicated section in the configuration file).
|
||||
|
||||
Note that you can print results locally, without publishing them, by setting in `configuration.toml`:
|
||||
|
||||
```
|
||||
[config]
|
||||
publish_output=true
|
||||
verbosity_level=2
|
||||
```
|
||||
This is useful for debugging or experimenting with the different tools.
|
||||
|
||||
### Working from pre-built repo (GitHub Action/GitHub App/Docker)
|
||||
When running PR-Agent from a pre-built repo, the default configuration file will be loaded.
|
||||
|
||||
To edit the configuration, you have two options:
|
||||
1. Place a local configuration file in the root of your local repo. The local file will be used instead of the default one.
|
||||
2. For online usage, just add `--config_path=<value>` to you command, to edit a specific configuration value.
|
||||
For example if you want to edit `pr_reviewer` configurations, you can run:
|
||||
```
|
||||
/review --pr_reviewer.extra_instructions="..." --pr_reviewer.require_score_review=false ...
|
||||
```
|
||||
|
||||
Any configuration value in `configuration.toml` file can be similarly edited.
|
||||
|
||||
### General configuration parameters
|
||||
|
||||
#### Changing a model
|
||||
See [here](pr_agent/algo/__init__.py) for the list of available models.
|
||||
|
||||
To use Llama2 model, for example, set:
|
||||
```
|
||||
[config]
|
||||
model = "replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1"
|
||||
[replicate]
|
||||
key = ...
|
||||
```
|
||||
(you can obtain a Llama2 key from [here](https://replicate.com/replicate/llama-2-70b-chat/api))
|
||||
|
||||
Also review the [AiHandler](pr_agent/algo/ai_handler.py) file for instruction how to set keys for other models.
|
||||
|
||||
#### Extra instructions
|
||||
All PR-Agent tools have a parameter called `extra_instructions`, that enables to add free-text extra instructions. Example usage:
|
||||
```
|
||||
/update_changelog --pr_update_changelog.extra_instructions="Make sure to update also the version ..."
|
||||
```
|
@ -2,7 +2,8 @@ FROM python:3.10 as base
|
||||
|
||||
WORKDIR /app
|
||||
ADD pyproject.toml .
|
||||
RUN pip install . && rm pyproject.toml
|
||||
ADD requirements.txt .
|
||||
RUN pip install . && rm pyproject.toml requirements.txt
|
||||
ENV PYTHONPATH=/app
|
||||
ADD pr_agent pr_agent
|
||||
ADD github_action/entrypoint.sh /
|
||||
|
104
INSTALL.md
104
INSTALL.md
@ -9,8 +9,8 @@ To get started with PR-Agent quickly, you first need to acquire two tokens:
|
||||
There are several ways to use PR-Agent:
|
||||
|
||||
- [Method 1: Use Docker image (no installation required)](INSTALL.md#method-1-use-docker-image-no-installation-required)
|
||||
- [Method 2: Run as a GitHub Action](INSTALL.md#method-2-run-as-a-github-action)
|
||||
- [Method 3: Run from source](INSTALL.md#method-3-run-from-source)
|
||||
- [Method 2: Run from source](INSTALL.md#method-2-run-from-source)
|
||||
- [Method 3: Run as a GitHub Action](INSTALL.md#method-3-run-as-a-github-action)
|
||||
- [Method 4: Run as a polling server](INSTALL.md#method-4-run-as-a-polling-server)
|
||||
- [Method 5: Run as a GitHub App](INSTALL.md#method-5-run-as-a-github-app)
|
||||
- [Method 6: Deploy as a Lambda Function](INSTALL.md#method-6---deploy-as-a-lambda-function)
|
||||
@ -55,7 +55,41 @@ Possible questions you can ask include:
|
||||
|
||||
---
|
||||
|
||||
### Method 2: Run as a GitHub Action
|
||||
### Method 2: Run from source
|
||||
|
||||
1. Clone this repository:
|
||||
|
||||
```
|
||||
git clone https://github.com/Codium-ai/pr-agent.git
|
||||
```
|
||||
|
||||
2. Install the requirements in your favorite virtual environment:
|
||||
|
||||
```
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
3. Copy the secrets template file and fill in your OpenAI key and your GitHub user token:
|
||||
|
||||
```
|
||||
cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets.toml
|
||||
chmod 600 pr_agent/settings/.secrets.toml
|
||||
# Edit .secrets.toml file
|
||||
```
|
||||
|
||||
4. Add the pr_agent folder to your PYTHONPATH, then run the cli.py script:
|
||||
|
||||
```
|
||||
export PYTHONPATH=[$PYTHONPATH:]<PATH to pr_agent folder>
|
||||
python pr_agent/cli.py --pr_url <pr_url> /review
|
||||
python pr_agent/cli.py --pr_url <pr_url> /ask <your question>
|
||||
python pr_agent/cli.py --pr_url <pr_url> /describe
|
||||
python pr_agent/cli.py --pr_url <pr_url> /improve
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Method 3: Run as a GitHub Action
|
||||
|
||||
You can use our pre-built Github Action Docker image to run PR-Agent as a Github Action.
|
||||
|
||||
@ -114,7 +148,7 @@ The GITHUB_TOKEN secret is automatically created by GitHub.
|
||||
3. Merge this change to your main branch.
|
||||
When you open your next PR, you should see a comment from `github-actions` bot with a review of your PR, and instructions on how to use the rest of the tools.
|
||||
|
||||
4. You may configure PR-Agent by adding environment variables under the env section corresponding to any configurable property in the [configuration](./CONFIGURATION.md) file. Some examples:
|
||||
4. You may configure PR-Agent by adding environment variables under the env section corresponding to any configurable property in the [configuration](./Usage.md) file. Some examples:
|
||||
```yaml
|
||||
env:
|
||||
# ... previous environment values
|
||||
@ -125,40 +159,6 @@ When you open your next PR, you should see a comment from `github-actions` bot w
|
||||
|
||||
---
|
||||
|
||||
### Method 3: Run from source
|
||||
|
||||
1. Clone this repository:
|
||||
|
||||
```
|
||||
git clone https://github.com/Codium-ai/pr-agent.git
|
||||
```
|
||||
|
||||
2. Install the requirements in your favorite virtual environment:
|
||||
|
||||
```
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
3. Copy the secrets template file and fill in your OpenAI key and your GitHub user token:
|
||||
|
||||
```
|
||||
cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets.toml
|
||||
chmod 600 pr_agent/settings/.secrets.toml
|
||||
# Edit .secrets.toml file
|
||||
```
|
||||
|
||||
4. Add the pr_agent folder to your PYTHONPATH, then run the cli.py script:
|
||||
|
||||
```
|
||||
export PYTHONPATH=[$PYTHONPATH:]<PATH to pr_agent folder>
|
||||
python pr_agent/cli.py --pr_url <pr_url> review
|
||||
python pr_agent/cli.py --pr_url <pr_url> ask <your question>
|
||||
python pr_agent/cli.py --pr_url <pr_url> describe
|
||||
python pr_agent/cli.py --pr_url <pr_url> improve
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Method 4: Run as a polling server
|
||||
Request reviews by tagging your Github user on a PR
|
||||
|
||||
@ -251,6 +251,9 @@ docker push codiumai/pr-agent:github_app # Push to your Docker repository
|
||||
|
||||
9. Install the app by navigating to the "Install App" tab and selecting your desired repositories.
|
||||
|
||||
> **Note:** When running PR-Agent from GitHub App, the default configuration file (configuration.toml) will be loaded.<br>
|
||||
> However, you can override the default tool parameters by uploading a local configuration file<br>
|
||||
> For more information please check out [CONFIGURATION.md](Usage.md#working-from-github-app-pre-built-repo)
|
||||
---
|
||||
|
||||
### Method 6 - Deploy as a Lambda Function
|
||||
@ -309,7 +312,9 @@ Example IAM permissions to that user to allow access to CodeCommit:
|
||||
"codecommit:Get*",
|
||||
"codecommit:List*",
|
||||
"codecommit:PostComment*",
|
||||
"codecommit:PutCommentReaction"
|
||||
"codecommit:PutCommentReaction",
|
||||
"codecommit:UpdatePullRequestDescription",
|
||||
"codecommit:UpdatePullRequestTitle"
|
||||
],
|
||||
"Resource": "*"
|
||||
}
|
||||
@ -338,6 +343,27 @@ PYTHONPATH="/PATH/TO/PROJECTS/pr-agent" python pr_agent/cli.py \
|
||||
review
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Method 8 - Run a GitLab webhook server
|
||||
|
||||
1. From the GitLab workspace or group, create an access token. Enable the "api" scope only.
|
||||
2. Generate a random secret for your app, and save it for later. For example, you can use:
|
||||
|
||||
```
|
||||
WEBHOOK_SECRET=$(python -c "import secrets; print(secrets.token_hex(10))")
|
||||
```
|
||||
3. Follow the instructions to build the Docker image, setup a secrets file and deploy on your own server from [Method 5](#method-5-run-as-a-github-app).
|
||||
4. In the secrets file, fill in the following:
|
||||
- Your OpenAI key.
|
||||
- In the [gitlab] section, fill in personal_access_token and shared_secret. The access token can be a personal access token, or a group or project access token.
|
||||
- Set deployment_type to 'gitlab' in [configuration.toml](./pr_agent/settings/configuration.toml)
|
||||
5. Create a webhook in GitLab. Set the URL to the URL of your app's server. Set the secret token to the generated secret from step 2.
|
||||
In the "Trigger" section, check the ‘comments’ and ‘merge request events’ boxes.
|
||||
6. Test your installation by opening a merge request or commenting or a merge request using one of CodiumAI's commands.
|
||||
|
||||
---
|
||||
|
||||
### Appendix - **Debugging LLM API Calls**
|
||||
If you're testing your codium/pr-agent server, and need to see if calls were made successfully + the exact call logs, you can use the [LiteLLM Debugger tool](https://docs.litellm.ai/docs/debugging/hosted_debugging).
|
||||
|
||||
|
@ -1 +0,0 @@
|
||||
recursive-include pr_agent/settings/ *.toml
|
148
README.md
148
README.md
@ -17,17 +17,20 @@ Making pull requests less painful with an AI agent
|
||||
|
||||
CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull requests faster and more efficiently. It automatically analyzes the pull request and can provide several types of PR feedback:
|
||||
|
||||
**Auto-Description**: Automatically generating [PR description](https://github.com/Codium-ai/pr-agent/pull/229#issue-1860711415) - title, type, summary, code walkthrough and labels.
|
||||
**Auto Description (/describe)**: Automatically generating [PR description](https://github.com/Codium-ai/pr-agent/pull/229#issue-1860711415) - title, type, summary, code walkthrough and labels.
|
||||
\
|
||||
**Auto Review**: [Adjustable feedback](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695022908) about the PR main theme, type, relevant tests, security issues, score, and various suggestions for the PR content.
|
||||
**Auto Review (/review)**: [Adjustable feedback](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695022908) about the PR main theme, type, relevant tests, security issues, score, and various suggestions for the PR content.
|
||||
\
|
||||
**Question Answering**: Answering [free-text questions](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021332) about the PR.
|
||||
**Question Answering (/ask ...)**: Answering [free-text questions](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021332) about the PR.
|
||||
\
|
||||
**Code Suggestions**: [Committable code suggestions](https://github.com/Codium-ai/pr-agent/pull/229#discussion_r1306919276) for improving the PR.
|
||||
**Code Suggestions (/improve)**: [Committable code suggestions](https://github.com/Codium-ai/pr-agent/pull/229#discussion_r1306919276) for improving the PR.
|
||||
\
|
||||
**Update Changelog**: Automatically updating the CHANGELOG.md file with the [PR changes](https://github.com/Codium-ai/pr-agent/pull/168#discussion_r1282077645).
|
||||
**Update Changelog (/update_changelog)**: Automatically updating the CHANGELOG.md file with the [PR changes](https://github.com/Codium-ai/pr-agent/pull/168#discussion_r1282077645).
|
||||
|
||||
<h3>Example results:</h2>
|
||||
|
||||
See the [usage guide](./Usage.md) for instructions how to run the different tools from [CLI](./Usage.md#working-from-a-local-repo-cli), or by [online usage](./Usage.md#online-usage).
|
||||
|
||||
<h3>Example results:</h3>
|
||||
</div>
|
||||
<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1687561986">/describe:</a></h4>
|
||||
<div align="center">
|
||||
@ -35,87 +38,101 @@ CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull
|
||||
<img src="https://www.codium.ai/images/describe-2.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021901">/review:</a></h4>
|
||||
<div align="center">
|
||||
<p float="center">
|
||||
<img src="https://www.codium.ai/images/review-2.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
<h4><a href="https://github.com/Codium-ai/pr-agent/pull/78#issuecomment-1639739496">/reflect_and_review:</a></h4>
|
||||
<div align="center">
|
||||
<p float="center">
|
||||
<img src="https://www.codium.ai/images/reflect_and_review.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695020538">/ask:</a></h4>
|
||||
<div align="center">
|
||||
<p float="center">
|
||||
<img src="https://www.codium.ai/images/ask-2.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695024952">/improve:</a></h4>
|
||||
<div align="center">
|
||||
<p float="center">
|
||||
<img src="https://www.codium.ai/images/improve-2.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
|
||||
[//]: # (<h4><a href="https://github.com/Codium-ai/pr-agent/pull/78#issuecomment-1639739496">/reflect_and_review:</a></h4>)
|
||||
|
||||
[//]: # (<div align="center">)
|
||||
|
||||
[//]: # (<p float="center">)
|
||||
|
||||
[//]: # (<img src="https://www.codium.ai/images/reflect_and_review.gif" width="800">)
|
||||
|
||||
[//]: # (</p>)
|
||||
|
||||
[//]: # (</div>)
|
||||
|
||||
[//]: # (<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695020538">/ask:</a></h4>)
|
||||
|
||||
[//]: # (<div align="center">)
|
||||
|
||||
[//]: # (<p float="center">)
|
||||
|
||||
[//]: # (<img src="https://www.codium.ai/images/ask-2.gif" width="800">)
|
||||
|
||||
[//]: # (</p>)
|
||||
|
||||
[//]: # (</div>)
|
||||
|
||||
[//]: # (<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695024952">/improve:</a></h4>)
|
||||
|
||||
[//]: # (<div align="center">)
|
||||
|
||||
[//]: # (<p float="center">)
|
||||
|
||||
[//]: # (<img src="https://www.codium.ai/images/improve-2.gif" width="800">)
|
||||
|
||||
[//]: # (</p>)
|
||||
|
||||
[//]: # (</div>)
|
||||
<div align="left">
|
||||
|
||||
|
||||
## Table of Contents
|
||||
- [Overview](#overview)
|
||||
- [Try it now](#try-it-now)
|
||||
- [Installation](#installation)
|
||||
- [Configuration](./CONFIGURATION.md)
|
||||
- [Usage guide](./Usage.md)
|
||||
- [How it works](#how-it-works)
|
||||
- [Why use PR-Agent](#why-use-pr-agent)
|
||||
- [Roadmap](#roadmap)
|
||||
- [Similar projects](#similar-projects)
|
||||
</div>
|
||||
|
||||
|
||||
## Overview
|
||||
`PR-Agent` offers extensive pull request functionalities across various git providers:
|
||||
| | | GitHub | Gitlab | Bitbucket | CodeCommit |
|
||||
|-------|---------------------------------------------|:------:|:------:|:---------:|:----------:|
|
||||
| TOOLS | Review | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | ⮑ Inline review | :white_check_mark: | :white_check_mark: | | |
|
||||
| | Ask | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Auto-Description | :white_check_mark: | :white_check_mark: | | |
|
||||
| | Improve Code | :white_check_mark: | :white_check_mark: | | |
|
||||
| | ⮑ Extended | :white_check_mark: | :white_check_mark: | | |
|
||||
| | Reflect and Review | :white_check_mark: | | | |
|
||||
| | Update CHANGELOG.md | :white_check_mark: | | | |
|
||||
| | | | | | |
|
||||
| USAGE | CLI | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | App / webhook | :white_check_mark: | :white_check_mark: | | |
|
||||
| | Tagging bot | :white_check_mark: | | | |
|
||||
| | Actions | :white_check_mark: | | | |
|
||||
| | | | | | |
|
||||
| CORE | PR compression | :white_check_mark: | :white_check_mark: | :white_check_mark: | |
|
||||
| | Repo language prioritization | :white_check_mark: | :white_check_mark: | :white_check_mark: | |
|
||||
| | Adaptive and token-aware<br />file patch fitting | :white_check_mark: | :white_check_mark: | :white_check_mark: | |
|
||||
| | Multiple models support | :white_check_mark: | :white_check_mark: | :white_check_mark: | |
|
||||
| | Incremental PR Review | :white_check_mark: | | | |
|
||||
| | | GitHub | Gitlab | Bitbucket | CodeCommit | Azure DevOps | Gerrit |
|
||||
|-------|---------------------------------------------|:------:|:------:|:---------:|:----------:|:----------:|:----------:|
|
||||
| TOOLS | Review | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Ask | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Auto-Description | :white_check_mark: | :white_check_mark: | | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Improve Code | :white_check_mark: | :white_check_mark: | | :white_check_mark: | | :white_check_mark: |
|
||||
| | ⮑ Extended | :white_check_mark: | :white_check_mark: | | :white_check_mark: | | :white_check_mark: |
|
||||
| | Reflect and Review | :white_check_mark: | | | | :white_check_mark: | :white_check_mark: |
|
||||
| | Update CHANGELOG.md | :white_check_mark: | | | | | |
|
||||
| | | | | | | |
|
||||
| USAGE | CLI | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | App / webhook | :white_check_mark: | :white_check_mark: | | | |
|
||||
| | Tagging bot | :white_check_mark: | | | | |
|
||||
| | Actions | :white_check_mark: | | | | |
|
||||
| | Web server | | | | | | :white_check_mark: |
|
||||
| | | | | | | |
|
||||
| CORE | PR compression | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Repo language prioritization | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Adaptive and token-aware<br />file patch fitting | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Multiple models support | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Incremental PR Review | :white_check_mark: | | | | | |
|
||||
|
||||
Examples for invoking the different tools via the CLI:
|
||||
- **Review**: python cli.py --pr_url=<pr_url> review
|
||||
- **Describe**: python cli.py --pr_url=<pr_url> describe
|
||||
- **Improve**: python cli.py --pr_url=<pr_url> improve
|
||||
- **Ask**: python cli.py --pr_url=<pr_url> ask "Write me a poem about this PR"
|
||||
- **Reflect**: python cli.py --pr_url=<pr_url> reflect
|
||||
- **Update Changelog**: python cli.py --pr_url=<pr_url> update_changelog
|
||||
|
||||
"<pr_url>" is the url of the relevant PR (for example: https://github.com/Codium-ai/pr-agent/pull/50).
|
||||
|
||||
In the [configuration](./CONFIGURATION.md) file you can select your git provider (GitHub, Gitlab, Bitbucket), and further configure the different tools.
|
||||
Review the **[usage guide](./Usage.md)** section for detailed instructions how to use the different tools, select the relevant git provider (GitHub, Gitlab, Bitbucket,...), and adjust the configuration file to your needs.
|
||||
|
||||
## Try it now
|
||||
|
||||
Try GPT-4 powered PR-Agent on your public GitHub repository for free. Just mention `@CodiumAI-Agent` and add the desired command in any PR comment! The agent will generate a response based on your command.
|
||||
You can try GPT-4 powered PR-Agent, on your public GitHub repository, instantly. Just mention `@CodiumAI-Agent` and add the desired command in any PR comment. The agent will generate a response based on your command.
|
||||
For example, add a comment to any pull request with the following text:
|
||||
```
|
||||
@CodiumAI-Agent /review
|
||||
```
|
||||
and the agent will respond with a review of your PR
|
||||
|
||||

|
||||
|
||||
To set up your own PR-Agent, see the [Installation](#installation) section
|
||||
|
||||
To set up your own PR-Agent, see the [Installation](#installation) section below.
|
||||
|
||||
---
|
||||
|
||||
@ -129,14 +146,15 @@ To get started with PR-Agent quickly, you first need to acquire two tokens:
|
||||
There are several ways to use PR-Agent:
|
||||
|
||||
- [Method 1: Use Docker image (no installation required)](INSTALL.md#method-1-use-docker-image-no-installation-required)
|
||||
- [Method 2: Run as a GitHub Action](INSTALL.md#method-2-run-as-a-github-action)
|
||||
- [Method 3: Run from source](INSTALL.md#method-3-run-from-source)
|
||||
- [Method 2: Run from source](INSTALL.md#method-2-run-from-source)
|
||||
- [Method 3: Run as a GitHub Action](INSTALL.md#method-3-run-as-a-github-action)
|
||||
- [Method 4: Run as a polling server](INSTALL.md#method-4-run-as-a-polling-server)
|
||||
- Request reviews by tagging your GitHub user on a PR
|
||||
- [Method 5: Run as a GitHub App](INSTALL.md#method-5-run-as-a-github-app)
|
||||
- Allowing you to automate the review process on your private or public repositories
|
||||
- [Method 6: Deploy as a Lambda Function](INSTALL.md#method-6---deploy-as-a-lambda-function)
|
||||
- [Method 7: AWS CodeCommit](INSTALL.md#method-7---aws-codecommit-setup)
|
||||
- [Method 8: Run a GitLab webhook server](INSTALL.md#method-8---run-a-gitlab-webhook-server)
|
||||
|
||||
## How it works
|
||||
|
||||
@ -154,7 +172,7 @@ Here are some advantages of PR-Agent:
|
||||
|
||||
- We emphasize **real-life practical usage**. Each tool (review, improve, ask, ...) has a single GPT-4 call, no more. We feel that this is critical for realistic team usage - obtaining an answer quickly (~30 seconds) and affordably.
|
||||
- Our [PR Compression strategy](./PR_COMPRESSION.md) is a core ability that enables to effectively tackle both short and long PRs.
|
||||
- Our JSON prompting strategy enables to have **modular, customizable tools**. For example, the '/review' tool categories can be controlled via the [configuration](./CONFIGURATION.md) file. Adding additional categories is easy and accessible.
|
||||
- Our JSON prompting strategy enables to have **modular, customizable tools**. For example, the '/review' tool categories can be controlled via the [configuration](pr_agent/settings/configuration.toml) file. Adding additional categories is easy and accessible.
|
||||
- We support **multiple git providers** (GitHub, Gitlab, Bitbucket, CodeCommit), **multiple ways** to use the tool (CLI, GitHub Action, GitHub App, Docker, ...), and **multiple models** (GPT-4, GPT-3.5, Anthropic, Cohere, Llama2).
|
||||
- We are open-source, and welcome contributions from the community.
|
||||
|
||||
@ -177,7 +195,7 @@ Here are some advantages of PR-Agent:
|
||||
|
||||
## Similar Projects
|
||||
|
||||
- [CodiumAI - Meaningful tests for busy devs](https://github.com/Codium-ai/codiumai-vscode-release)
|
||||
- [CodiumAI - Meaningful tests for busy devs](https://github.com/Codium-ai/codiumai-vscode-release) (although various capabilities are much more advanced in the CodiumAI IDE plugins)
|
||||
- [Aider - GPT powered coding in your terminal](https://github.com/paul-gauthier/aider)
|
||||
- [openai-pr-reviewer](https://github.com/coderabbitai/openai-pr-reviewer)
|
||||
- [CodeReview BOT](https://github.com/anc95/ChatGPT-CodeReview)
|
||||
|
182
Usage.md
Normal file
182
Usage.md
Normal file
@ -0,0 +1,182 @@
|
||||
## Usage guide
|
||||
|
||||
### Table of Contents
|
||||
- [Introduction](#introduction)
|
||||
- [Working from a local repo (CLI)](#working-from-a-local-repo-cli)
|
||||
- [Online usage](#online-usage)
|
||||
- [Working with GitHub App](#working-with-github-app)
|
||||
- [Working with GitHub Action](#working-with-github-action)
|
||||
- [Appendix - additional configurations walkthrough](#appendix---additional-configurations-walkthrough)
|
||||
|
||||
### Introduction
|
||||
|
||||
There are 3 basic ways to invoke CodiumAI PR-Agent:
|
||||
1. Locally running a CLI command
|
||||
2. Online usage - by [commenting](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021901) on a PR
|
||||
3. Enabling PR-Agent tools to run automatically when a new PR is opened
|
||||
|
||||
See the [installation guide](/INSTALL.md) for instructions on how to setup your own PR-Agent.
|
||||
|
||||
Specifically, CLI commands can be issued by invoking a pre-built [docker image](/INSTALL.md#running-from-source), or by invoking a [locally cloned repo](INSTALL.md#method-2-run-from-source).
|
||||
|
||||
For online usage, you will need to setup either a [GitHub App](INSTALL.md#method-5-run-as-a-github-app), or a [GitHub Action](INSTALL.md#method-3-run-as-a-github-action).
|
||||
GitHub App and GitHub Action also enable to run PR-Agent specific tool automatically when a new PR is opened.
|
||||
|
||||
|
||||
#### The configuration file
|
||||
The different tools and sub-tools used by CodiumAI PR-Agent are adjustable via the **[configuration file](pr_agent/settings/configuration.toml)**.
|
||||
In addition to general configuration options, each tool has its own configurations. For example, the `review` tool will use parameters from the [pr_reviewer](/pr_agent/settings/configuration.toml#L16) section in the configuration file.
|
||||
|
||||
**git provider:**
|
||||
The [git_provider](pr_agent/settings/configuration.toml#L4) field in the configuration file determines the GIT provider that will be used by PR-Agent. Currently, the following providers are supported:
|
||||
`
|
||||
"github", "gitlab", "azure", "codecommit", "local"
|
||||
`
|
||||
|
||||
[//]: # (** online usage:**)
|
||||
|
||||
[//]: # (Options that are available in the configuration file can be specified at run time when calling actions. Two examples:)
|
||||
|
||||
[//]: # (```)
|
||||
|
||||
[//]: # (- /review --pr_reviewer.extra_instructions="focus on the file: ...")
|
||||
|
||||
[//]: # (- /describe --pr_description.add_original_user_description=false -pr_description.extra_instructions="make sure to mention: ...")
|
||||
|
||||
[//]: # (```)
|
||||
|
||||
### Working from a local repo (CLI)
|
||||
When running from your local repo (CLI), your local configuration file will be used.
|
||||
|
||||
Examples for invoking the different tools via the CLI:
|
||||
|
||||
- **Review**: `python cli.py --pr_url=<pr_url> /review`
|
||||
- **Describe**: `python cli.py --pr_url=<pr_url> /describe`
|
||||
- **Improve**: `python cli.py --pr_url=<pr_url> /improve`
|
||||
- **Ask**: `python cli.py --pr_url=<pr_url> /ask "Write me a poem about this PR"`
|
||||
- **Reflect**: `python cli.py --pr_url=<pr_url> /reflect`
|
||||
- **Update Changelog**: `python cli.py --pr_url=<pr_url> /update_changelog`
|
||||
|
||||
`<pr_url>` is the url of the relevant PR (for example: https://github.com/Codium-ai/pr-agent/pull/50).
|
||||
|
||||
**Notes:**
|
||||
|
||||
(1) in addition to editing your local configuration file, you can also change any configuration value by adding it to the command line:
|
||||
```
|
||||
python cli.py --pr_url=<pr_url> /review --pr_reviewer.extra_instructions="focus on the file: ..."
|
||||
```
|
||||
|
||||
(2) You can print results locally, without publishing them, by setting in `configuration.toml`:
|
||||
```
|
||||
[config]
|
||||
publish_output=true
|
||||
verbosity_level=2
|
||||
```
|
||||
This is useful for debugging or experimenting with the different tools.
|
||||
|
||||
|
||||
### Online usage
|
||||
|
||||
Online usage means invoking PR-Agent tools by [comments](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021901) on a PR.
|
||||
Commands for invoking the different tools via comments:
|
||||
|
||||
- **Review**: `/review`
|
||||
- **Describe**: `/describe`
|
||||
- **Improve**: `/improve`
|
||||
- **Ask**: `/ask "..."`
|
||||
- **Reflect**: `/reflect`
|
||||
- **Update Changelog**: `/update_changelog`
|
||||
|
||||
|
||||
To edit a specific configuration value, just add `--config_path=<value>` to any command.
|
||||
For example if you want to edit the `review` tool configurations, you can run:
|
||||
```
|
||||
/review --pr_reviewer.extra_instructions="..." --pr_reviewer.require_score_review=false
|
||||
```
|
||||
Any configuration value in [configuration file](pr_agent/settings/configuration.toml) file can be similarly edited.
|
||||
|
||||
|
||||
### Working with GitHub App
|
||||
When running PR-Agent from [GitHub App](INSTALL.md#method-5-run-as-a-github-app), the default configurations from a pre-built repo will be initially loaded.
|
||||
|
||||
#### GitHub app automatic tools
|
||||
The [github_app](pr_agent/settings/configuration.toml#L56) section defines GitHub app specific configurations.
|
||||
An important parameter is `pr_commands`, which is a list of tools that will be **run automatically when a new PR is opened**:
|
||||
```
|
||||
[github_app]
|
||||
pr_commands = [
|
||||
"/describe --pr_description.add_original_user_description=true --pr_description.keep_original_user_title=true",
|
||||
"/auto_review",
|
||||
]
|
||||
```
|
||||
This means that when a new PR is opened, PR-Agent will run the `describe` and `auto_review` tools.
|
||||
For the describe tool, the `add_original_user_description` and `keep_original_user_title` parameters will be set to true.
|
||||
|
||||
However, you can override the default tool parameters by uploading a local configuration file called `.pr_agent.toml` to the root of your repo.
|
||||
For example, if your local `.pr_agent.toml` file contains:
|
||||
```
|
||||
[pr_description]
|
||||
add_original_user_description = false
|
||||
keep_original_user_title = false
|
||||
```
|
||||
When a new PR is opened, PR-Agent will run the `describe` tool with the above parameters.
|
||||
|
||||
Note that a local `.pr_agent.toml` file enables you to edit and customize the default parameters of any tool, not just the ones that are run automatically.
|
||||
|
||||
#### Editing the prompts
|
||||
The prompts for the various PR-Agent tools are defined in the `pr_agent/settings` folder.
|
||||
|
||||
In practice, the prompts are loaded and stored as a standard setting object.
|
||||
Hence, editing them is similar to editing any other configuration value - just place the relevant key in `.pr_agent.toml`file, and override the default value.
|
||||
|
||||
For example, if you want to edit the prompts of the [describe](./pr_agent/settings/pr_description_prompts.toml) tool, you can add the following to your `.pr_agent.toml` file:
|
||||
```
|
||||
[pr_description_prompt]
|
||||
system="""
|
||||
...
|
||||
"""
|
||||
user="""
|
||||
...
|
||||
"""
|
||||
```
|
||||
Note that the new prompt will need to generate an output compatible with the relevant [post-process function](./pr_agent/tools/pr_description.py#L137).
|
||||
|
||||
### Working with GitHub Action
|
||||
TBD
|
||||
|
||||
### Appendix - additional configurations walkthrough
|
||||
|
||||
#### Changing a model
|
||||
See [here](pr_agent/algo/__init__.py) for the list of available models.
|
||||
|
||||
To use Llama2 model, for example, set:
|
||||
```
|
||||
[config]
|
||||
model = "replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1"
|
||||
[replicate]
|
||||
key = ...
|
||||
```
|
||||
(you can obtain a Llama2 key from [here](https://replicate.com/replicate/llama-2-70b-chat/api))
|
||||
|
||||
Also review the [AiHandler](pr_agent/algo/ai_handler.py) file for instruction how to set keys for other models.
|
||||
|
||||
#### Extra instructions
|
||||
All PR-Agent tools have a parameter called `extra_instructions`, that enables to add free-text extra instructions. Example usage:
|
||||
```
|
||||
/update_changelog --pr_update_changelog.extra_instructions="Make sure to update also the version ..."
|
||||
```
|
||||
|
||||
#### Azure DevOps provider
|
||||
To use Azure DevOps provider use the following settings in configuration.toml:
|
||||
```
|
||||
[config]
|
||||
git_provider="azure"
|
||||
use_repo_settings_file=false
|
||||
```
|
||||
|
||||
And use the following settings (you have to replace the values) in .secrets.toml:
|
||||
```
|
||||
[azure_devops]
|
||||
org = "https://dev.azure.com/YOUR_ORGANIZATION/"
|
||||
pat = "YOUR_PAT_TOKEN"
|
||||
```
|
@ -18,6 +18,10 @@ FROM base as github_polling
|
||||
ADD pr_agent pr_agent
|
||||
CMD ["python", "pr_agent/servers/github_polling.py"]
|
||||
|
||||
FROM base as gitlab_webhook
|
||||
ADD pr_agent pr_agent
|
||||
CMD ["python", "pr_agent/servers/gitlab_webhook.py"]
|
||||
|
||||
FROM base as test
|
||||
ADD requirements-dev.txt .
|
||||
RUN pip install -r requirements-dev.txt && rm requirements-dev.txt
|
||||
|
@ -1,17 +1,19 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import difflib
|
||||
import logging
|
||||
import re
|
||||
import traceback
|
||||
from typing import Callable, List, Tuple
|
||||
from typing import Any, Callable, List, Tuple
|
||||
|
||||
from github import RateLimitExceededException
|
||||
|
||||
from pr_agent.algo import MAX_TOKENS
|
||||
from pr_agent.algo.git_patch_processing import convert_to_hunks_with_lines_numbers, extend_patch, handle_patch_deletions
|
||||
from pr_agent.algo.language_handler import sort_files_by_main_languages
|
||||
from pr_agent.algo.token_handler import TokenHandler
|
||||
from pr_agent.algo.token_handler import TokenHandler, get_token_encoder
|
||||
from pr_agent.config_loader import get_settings
|
||||
from pr_agent.git_providers.git_provider import GitProvider
|
||||
from pr_agent.git_providers.git_provider import FilePatchInfo, GitProvider
|
||||
|
||||
DELETED_FILES_ = "Deleted files:\n"
|
||||
|
||||
@ -245,6 +247,99 @@ def _get_all_deployments(all_models: List[str]) -> List[str]:
|
||||
return all_deployments
|
||||
|
||||
|
||||
def find_line_number_of_relevant_line_in_file(diff_files: List[FilePatchInfo],
|
||||
relevant_file: str,
|
||||
relevant_line_in_file: str) -> Tuple[int, int]:
|
||||
"""
|
||||
Find the line number and absolute position of a relevant line in a file.
|
||||
|
||||
Args:
|
||||
diff_files (List[FilePatchInfo]): A list of FilePatchInfo objects representing the patches of files.
|
||||
relevant_file (str): The name of the file where the relevant line is located.
|
||||
relevant_line_in_file (str): The content of the relevant line.
|
||||
|
||||
Returns:
|
||||
Tuple[int, int]: A tuple containing the line number and absolute position of the relevant line in the file.
|
||||
"""
|
||||
position = -1
|
||||
absolute_position = -1
|
||||
re_hunk_header = re.compile(
|
||||
r"^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@[ ]?(.*)")
|
||||
|
||||
for file in diff_files:
|
||||
if file.filename.strip() == relevant_file:
|
||||
patch = file.patch
|
||||
patch_lines = patch.splitlines()
|
||||
|
||||
# try to find the line in the patch using difflib, with some margin of error
|
||||
matches_difflib: list[str | Any] = difflib.get_close_matches(relevant_line_in_file,
|
||||
patch_lines, n=3, cutoff=0.93)
|
||||
if len(matches_difflib) == 1 and matches_difflib[0].startswith('+'):
|
||||
relevant_line_in_file = matches_difflib[0]
|
||||
|
||||
delta = 0
|
||||
start1, size1, start2, size2 = 0, 0, 0, 0
|
||||
for i, line in enumerate(patch_lines):
|
||||
if line.startswith('@@'):
|
||||
delta = 0
|
||||
match = re_hunk_header.match(line)
|
||||
start1, size1, start2, size2 = map(int, match.groups()[:4])
|
||||
elif not line.startswith('-'):
|
||||
delta += 1
|
||||
|
||||
if relevant_line_in_file in line and line[0] != '-':
|
||||
position = i
|
||||
absolute_position = start2 + delta - 1
|
||||
break
|
||||
|
||||
if position == -1 and relevant_line_in_file[0] == '+':
|
||||
no_plus_line = relevant_line_in_file[1:].lstrip()
|
||||
for i, line in enumerate(patch_lines):
|
||||
if line.startswith('@@'):
|
||||
delta = 0
|
||||
match = re_hunk_header.match(line)
|
||||
start1, size1, start2, size2 = map(int, match.groups()[:4])
|
||||
elif not line.startswith('-'):
|
||||
delta += 1
|
||||
|
||||
if no_plus_line in line and line[0] != '-':
|
||||
# The model might add a '+' to the beginning of the relevant_line_in_file even if originally
|
||||
# it's a context line
|
||||
position = i
|
||||
absolute_position = start2 + delta - 1
|
||||
break
|
||||
return position, absolute_position
|
||||
|
||||
|
||||
def clip_tokens(text: str, max_tokens: int) -> str:
|
||||
"""
|
||||
Clip the number of tokens in a string to a maximum number of tokens.
|
||||
|
||||
Args:
|
||||
text (str): The string to clip.
|
||||
max_tokens (int): The maximum number of tokens allowed in the string.
|
||||
|
||||
Returns:
|
||||
str: The clipped string.
|
||||
"""
|
||||
if not text:
|
||||
return text
|
||||
|
||||
try:
|
||||
encoder = get_token_encoder()
|
||||
num_input_tokens = len(encoder.encode(text))
|
||||
if num_input_tokens <= max_tokens:
|
||||
return text
|
||||
num_chars = len(text)
|
||||
chars_per_token = num_chars / num_input_tokens
|
||||
num_output_chars = int(chars_per_token * max_tokens)
|
||||
clipped_text = text[:num_output_chars]
|
||||
return clipped_text
|
||||
except Exception as e:
|
||||
logging.warning(f"Failed to clip tokens: {e}")
|
||||
return text
|
||||
|
||||
|
||||
def get_pr_multi_diffs(git_provider: GitProvider,
|
||||
token_handler: TokenHandler,
|
||||
model: str,
|
||||
|
@ -21,7 +21,7 @@ class TokenHandler:
|
||||
method.
|
||||
"""
|
||||
|
||||
def __init__(self, vars: dict, system, user):
|
||||
def __init__(self, pr, vars: dict, system, user):
|
||||
"""
|
||||
Initializes the TokenHandler object.
|
||||
|
||||
@ -32,9 +32,9 @@ class TokenHandler:
|
||||
- user: The user string.
|
||||
"""
|
||||
self.encoder = get_token_encoder()
|
||||
self.prompt_tokens = self._get_system_user_tokens(self.encoder, vars, system, user)
|
||||
self.prompt_tokens = self._get_system_user_tokens(pr, self.encoder, vars, system, user)
|
||||
|
||||
def _get_system_user_tokens(self, encoder, vars: dict, system, user):
|
||||
def _get_system_user_tokens(self, pr, encoder, vars: dict, system, user):
|
||||
"""
|
||||
Calculates the number of tokens in the system and user strings.
|
||||
|
||||
|
@ -5,24 +5,14 @@ import json
|
||||
import logging
|
||||
import re
|
||||
import textwrap
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
from typing import Any, List, Tuple, Optional
|
||||
from typing import Any, List
|
||||
|
||||
import yaml
|
||||
from starlette_context import context
|
||||
|
||||
from pr_agent.algo.token_handler import get_token_encoder
|
||||
from pr_agent.config_loader import get_settings, global_settings
|
||||
|
||||
|
||||
class EDIT_TYPE(Enum):
|
||||
ADDED = 1
|
||||
DELETED = 2
|
||||
MODIFIED = 3
|
||||
RENAMED = 4
|
||||
|
||||
def get_setting(key: str) -> Any:
|
||||
try:
|
||||
key = key.upper()
|
||||
@ -304,108 +294,3 @@ def try_fix_yaml(review_text: str) -> dict:
|
||||
except:
|
||||
pass
|
||||
return data
|
||||
|
||||
def clip_tokens(text: str, max_tokens: int) -> str:
|
||||
"""
|
||||
Clip the number of tokens in a string to a maximum number of tokens.
|
||||
|
||||
Args:
|
||||
text (str): The string to clip.
|
||||
max_tokens (int): The maximum number of tokens allowed in the string.
|
||||
|
||||
Returns:
|
||||
str: The clipped string.
|
||||
"""
|
||||
if not text:
|
||||
return text
|
||||
|
||||
try:
|
||||
encoder = get_token_encoder()
|
||||
num_input_tokens = len(encoder.encode(text))
|
||||
if num_input_tokens <= max_tokens:
|
||||
return text
|
||||
num_chars = len(text)
|
||||
chars_per_token = num_chars / num_input_tokens
|
||||
num_output_chars = int(chars_per_token * max_tokens)
|
||||
clipped_text = text[:num_output_chars]
|
||||
return clipped_text
|
||||
except Exception as e:
|
||||
logging.warning(f"Failed to clip tokens: {e}")
|
||||
return text
|
||||
|
||||
|
||||
@dataclass
|
||||
class FilePatchInfo:
|
||||
base_file: str
|
||||
head_file: str
|
||||
patch: str
|
||||
filename: str
|
||||
tokens: int = -1
|
||||
edit_type: EDIT_TYPE = EDIT_TYPE.MODIFIED
|
||||
old_filename: str = None
|
||||
language: Optional[str] = None
|
||||
|
||||
|
||||
def find_line_number_of_relevant_line_in_file(diff_files: List[FilePatchInfo],
|
||||
relevant_file: str,
|
||||
relevant_line_in_file: str) -> Tuple[int, int]:
|
||||
"""
|
||||
Find the line number and absolute position of a relevant line in a file.
|
||||
|
||||
Args:
|
||||
diff_files (List[FilePatchInfo]): A list of FilePatchInfo objects representing the patches of files.
|
||||
relevant_file (str): The name of the file where the relevant line is located.
|
||||
relevant_line_in_file (str): The content of the relevant line.
|
||||
|
||||
Returns:
|
||||
Tuple[int, int]: A tuple containing the line number and absolute position of the relevant line in the file.
|
||||
"""
|
||||
position = -1
|
||||
absolute_position = -1
|
||||
re_hunk_header = re.compile(
|
||||
r"^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@[ ]?(.*)")
|
||||
|
||||
for file in diff_files:
|
||||
if file.filename.strip() == relevant_file:
|
||||
patch = file.patch
|
||||
patch_lines = patch.splitlines()
|
||||
|
||||
# try to find the line in the patch using difflib, with some margin of error
|
||||
matches_difflib: list[str | Any] = difflib.get_close_matches(relevant_line_in_file,
|
||||
patch_lines, n=3, cutoff=0.93)
|
||||
if len(matches_difflib) == 1 and matches_difflib[0].startswith('+'):
|
||||
relevant_line_in_file = matches_difflib[0]
|
||||
|
||||
delta = 0
|
||||
start1, size1, start2, size2 = 0, 0, 0, 0
|
||||
for i, line in enumerate(patch_lines):
|
||||
if line.startswith('@@'):
|
||||
delta = 0
|
||||
match = re_hunk_header.match(line)
|
||||
start1, size1, start2, size2 = map(int, match.groups()[:4])
|
||||
elif not line.startswith('-'):
|
||||
delta += 1
|
||||
|
||||
if relevant_line_in_file in line and line[0] != '-':
|
||||
position = i
|
||||
absolute_position = start2 + delta - 1
|
||||
break
|
||||
|
||||
if position == -1 and relevant_line_in_file[0] == '+':
|
||||
no_plus_line = relevant_line_in_file[1:].lstrip()
|
||||
for i, line in enumerate(patch_lines):
|
||||
if line.startswith('@@'):
|
||||
delta = 0
|
||||
match = re_hunk_header.match(line)
|
||||
start1, size1, start2, size2 = map(int, match.groups()[:4])
|
||||
elif not line.startswith('-'):
|
||||
delta += 1
|
||||
|
||||
if no_plus_line in line and line[0] != '-':
|
||||
# The model might add a '+' to the beginning of the relevant_line_in_file even if originally
|
||||
# it's a context line
|
||||
position = i
|
||||
absolute_position = start2 + delta - 1
|
||||
break
|
||||
return position, absolute_position
|
||||
|
||||
|
@ -5,6 +5,8 @@ from pr_agent.git_providers.github_provider import GithubProvider
|
||||
from pr_agent.git_providers.gitlab_provider import GitLabProvider
|
||||
from pr_agent.git_providers.local_git_provider import LocalGitProvider
|
||||
from pr_agent.git_providers.azuredevops_provider import AzureDevopsProvider
|
||||
from pr_agent.git_providers.gerrit_provider import GerritProvider
|
||||
|
||||
|
||||
_GIT_PROVIDERS = {
|
||||
'github': GithubProvider,
|
||||
@ -12,7 +14,8 @@ _GIT_PROVIDERS = {
|
||||
'bitbucket': BitbucketProvider,
|
||||
'azure': AzureDevopsProvider,
|
||||
'codecommit': CodeCommitProvider,
|
||||
'local' : LocalGitProvider
|
||||
'local' : LocalGitProvider,
|
||||
'gerrit': GerritProvider,
|
||||
}
|
||||
|
||||
def get_git_provider():
|
||||
|
@ -13,9 +13,11 @@ try:
|
||||
except ImportError:
|
||||
AZURE_DEVOPS_AVAILABLE = False
|
||||
|
||||
from ..algo.pr_processing import clip_tokens
|
||||
from ..config_loader import get_settings
|
||||
from ..algo.utils import load_large_diff, FilePatchInfo, EDIT_TYPE, clip_tokens
|
||||
from ..algo.utils import load_large_diff
|
||||
from ..algo.language_handler import is_valid_file
|
||||
from .git_provider import EDIT_TYPE, FilePatchInfo
|
||||
|
||||
|
||||
class AzureDevopsProvider:
|
||||
|
@ -8,8 +8,7 @@ from atlassian.bitbucket import Cloud
|
||||
from starlette_context import context
|
||||
|
||||
from ..config_loader import get_settings
|
||||
from .git_provider import GitProvider
|
||||
from ..algo.utils import FilePatchInfo
|
||||
from .git_provider import FilePatchInfo, GitProvider
|
||||
|
||||
|
||||
class BitbucketProvider(GitProvider):
|
||||
|
@ -1,9 +1,6 @@
|
||||
try: # Allow this module to be imported without requiring boto3
|
||||
import boto3
|
||||
import botocore
|
||||
except ModuleNotFoundError:
|
||||
boto3 = None
|
||||
botocore = None
|
||||
import boto3
|
||||
import botocore
|
||||
|
||||
|
||||
class CodeCommitDifferencesResponse:
|
||||
"""
|
||||
@ -67,7 +64,7 @@ class CodeCommitClient:
|
||||
"""
|
||||
Get the differences between two commits in CodeCommit.
|
||||
|
||||
Parameters:
|
||||
Args:
|
||||
- repo_name: Name of the repository
|
||||
- destination_commit: Commit hash you want to merge into (the "before" hash) (usually on the main or master branch)
|
||||
- source_commit: Commit hash of the code you are adding (the "after" branch)
|
||||
@ -76,8 +73,8 @@ class CodeCommitClient:
|
||||
- List of CodeCommitDifferencesResponse objects
|
||||
|
||||
Boto3 Documentation:
|
||||
aws codecommit get-differences
|
||||
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_differences.html
|
||||
- aws codecommit get-differences
|
||||
- https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_differences.html
|
||||
"""
|
||||
if self.boto_client is None:
|
||||
self._connect_boto_client()
|
||||
@ -93,7 +90,11 @@ class CodeCommitClient:
|
||||
):
|
||||
differences.extend(page.get("differences", []))
|
||||
except botocore.exceptions.ClientError as e:
|
||||
raise ValueError(f"Failed to retrieve differences from CodeCommit PR #{self.pr_num}") from e
|
||||
if e.response["Error"]["Code"] == 'RepositoryDoesNotExistException':
|
||||
raise ValueError(f"CodeCommit cannot retrieve differences: Repository does not exist: {repo_name}") from e
|
||||
raise ValueError(f"CodeCommit cannot retrieve differences for {source_commit}..{destination_commit}") from e
|
||||
except Exception as e:
|
||||
raise ValueError(f"CodeCommit cannot retrieve differences for {source_commit}..{destination_commit}") from e
|
||||
|
||||
output = []
|
||||
for json in differences:
|
||||
@ -104,7 +105,7 @@ class CodeCommitClient:
|
||||
"""
|
||||
Retrieve a file from CodeCommit.
|
||||
|
||||
Parameters:
|
||||
Args:
|
||||
- repo_name: Name of the repository
|
||||
- file_path: Path to the file you are retrieving
|
||||
- sha_hash: Commit hash of the file you are retrieving
|
||||
@ -113,8 +114,8 @@ class CodeCommitClient:
|
||||
- File contents
|
||||
|
||||
Boto3 Documentation:
|
||||
aws codecommit get_file
|
||||
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_file.html
|
||||
- aws codecommit get_file
|
||||
- https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_file.html
|
||||
"""
|
||||
if not file_path:
|
||||
return ""
|
||||
@ -125,6 +126,8 @@ class CodeCommitClient:
|
||||
try:
|
||||
response = self.boto_client.get_file(repositoryName=repo_name, commitSpecifier=sha_hash, filePath=file_path)
|
||||
except botocore.exceptions.ClientError as e:
|
||||
if e.response["Error"]["Code"] == 'RepositoryDoesNotExistException':
|
||||
raise ValueError(f"CodeCommit cannot retrieve PR: Repository does not exist: {repo_name}") from e
|
||||
# if the file does not exist, but is flagged as optional, then return an empty string
|
||||
if optional and e.response["Error"]["Code"] == 'FileDoesNotExistException':
|
||||
return ""
|
||||
@ -136,19 +139,20 @@ class CodeCommitClient:
|
||||
|
||||
return response.get("fileContent", "")
|
||||
|
||||
def get_pr(self, pr_number: int):
|
||||
def get_pr(self, repo_name: str, pr_number: int):
|
||||
"""
|
||||
Get a information about a CodeCommit PR.
|
||||
|
||||
Parameters:
|
||||
Args:
|
||||
- repo_name: Name of the repository
|
||||
- pr_number: The PR number you are requesting
|
||||
|
||||
Returns:
|
||||
- CodeCommitPullRequestResponse object
|
||||
|
||||
Boto3 Documentation:
|
||||
aws codecommit get_pull_request
|
||||
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_pull_request.html
|
||||
- aws codecommit get_pull_request
|
||||
- https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_pull_request.html
|
||||
"""
|
||||
if self.boto_client is None:
|
||||
self._connect_boto_client()
|
||||
@ -158,6 +162,8 @@ class CodeCommitClient:
|
||||
except botocore.exceptions.ClientError as e:
|
||||
if e.response["Error"]["Code"] == 'PullRequestDoesNotExistException':
|
||||
raise ValueError(f"CodeCommit cannot retrieve PR: PR number does not exist: {pr_number}") from e
|
||||
if e.response["Error"]["Code"] == 'RepositoryDoesNotExistException':
|
||||
raise ValueError(f"CodeCommit cannot retrieve PR: Repository does not exist: {repo_name}") from e
|
||||
raise ValueError(f"CodeCommit cannot retrieve PR: {pr_number}: boto client error") from e
|
||||
except Exception as e:
|
||||
raise ValueError(f"CodeCommit cannot retrieve PR: {pr_number}") from e
|
||||
@ -167,35 +173,95 @@ class CodeCommitClient:
|
||||
|
||||
return CodeCommitPullRequestResponse(response.get("pullRequest", {}))
|
||||
|
||||
def publish_comment(self, repo_name: str, pr_number: int, destination_commit: str, source_commit: str, comment: str):
|
||||
def publish_description(self, pr_number: int, pr_title: str, pr_body: str):
|
||||
"""
|
||||
Publish a comment to a pull request
|
||||
Set the title and description on a pull request
|
||||
|
||||
Parameters:
|
||||
- repo_name: name of the repository
|
||||
- pr_number: number of the pull request
|
||||
- destination_commit: The commit hash you want to merge into (the "before" hash) (usually on the main or master branch)
|
||||
- source_commit: The commit hash of the code you are adding (the "after" branch)
|
||||
- pr_comment: comment
|
||||
Args:
|
||||
- pr_number: the AWS CodeCommit pull request number
|
||||
- pr_title: title of the pull request
|
||||
- pr_body: body of the pull request
|
||||
|
||||
Returns:
|
||||
- None
|
||||
|
||||
Boto3 Documentation:
|
||||
aws codecommit post_comment_for_pull_request
|
||||
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/post_comment_for_pull_request.html
|
||||
- aws codecommit update_pull_request_title
|
||||
- https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/update_pull_request_title.html
|
||||
- aws codecommit update_pull_request_description
|
||||
- https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/update_pull_request_description.html
|
||||
"""
|
||||
if self.boto_client is None:
|
||||
self._connect_boto_client()
|
||||
|
||||
try:
|
||||
self.boto_client.post_comment_for_pull_request(
|
||||
pullRequestId=str(pr_number),
|
||||
repositoryName=repo_name,
|
||||
beforeCommitId=destination_commit,
|
||||
afterCommitId=source_commit,
|
||||
content=comment,
|
||||
)
|
||||
self.boto_client.update_pull_request_title(pullRequestId=str(pr_number), title=pr_title)
|
||||
self.boto_client.update_pull_request_description(pullRequestId=str(pr_number), description=pr_body)
|
||||
except botocore.exceptions.ClientError as e:
|
||||
if e.response["Error"]["Code"] == 'PullRequestDoesNotExistException':
|
||||
raise ValueError(f"PR number does not exist: {pr_number}") from e
|
||||
if e.response["Error"]["Code"] == 'InvalidTitleException':
|
||||
raise ValueError(f"Invalid title for PR number: {pr_number}") from e
|
||||
if e.response["Error"]["Code"] == 'InvalidDescriptionException':
|
||||
raise ValueError(f"Invalid description for PR number: {pr_number}") from e
|
||||
if e.response["Error"]["Code"] == 'PullRequestAlreadyClosedException':
|
||||
raise ValueError(f"PR is already closed: PR number: {pr_number}") from e
|
||||
raise ValueError(f"Boto3 client error calling publish_description") from e
|
||||
except Exception as e:
|
||||
raise ValueError(f"Error calling publish_description") from e
|
||||
|
||||
def publish_comment(self, repo_name: str, pr_number: int, destination_commit: str, source_commit: str, comment: str, annotation_file: str = None, annotation_line: int = None):
|
||||
"""
|
||||
Publish a comment to a pull request
|
||||
|
||||
Args:
|
||||
- repo_name: name of the repository
|
||||
- pr_number: number of the pull request
|
||||
- destination_commit: The commit hash you want to merge into (the "before" hash) (usually on the main or master branch)
|
||||
- source_commit: The commit hash of the code you are adding (the "after" branch)
|
||||
- comment: The comment you want to publish
|
||||
- annotation_file: The file you want to annotate (optional)
|
||||
- annotation_line: The line number you want to annotate (optional)
|
||||
|
||||
Comment annotations for CodeCommit are different than GitHub.
|
||||
CodeCommit only designates the starting line number for the comment.
|
||||
It does not support the ending line number to highlight a range of lines.
|
||||
|
||||
Returns:
|
||||
- None
|
||||
|
||||
Boto3 Documentation:
|
||||
- aws codecommit post_comment_for_pull_request
|
||||
- https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/post_comment_for_pull_request.html
|
||||
"""
|
||||
if self.boto_client is None:
|
||||
self._connect_boto_client()
|
||||
|
||||
try:
|
||||
# If the comment has code annotations,
|
||||
# then set the file path and line number in the location dictionary
|
||||
if annotation_file and annotation_line:
|
||||
self.boto_client.post_comment_for_pull_request(
|
||||
pullRequestId=str(pr_number),
|
||||
repositoryName=repo_name,
|
||||
beforeCommitId=destination_commit,
|
||||
afterCommitId=source_commit,
|
||||
content=comment,
|
||||
location={
|
||||
"filePath": annotation_file,
|
||||
"filePosition": annotation_line,
|
||||
"relativeFileVersion": "AFTER",
|
||||
},
|
||||
)
|
||||
else:
|
||||
# The comment does not have code annotations
|
||||
self.boto_client.post_comment_for_pull_request(
|
||||
pullRequestId=str(pr_number),
|
||||
repositoryName=repo_name,
|
||||
beforeCommitId=destination_commit,
|
||||
afterCommitId=source_commit,
|
||||
content=comment,
|
||||
)
|
||||
except botocore.exceptions.ClientError as e:
|
||||
if e.response["Error"]["Code"] == 'RepositoryDoesNotExistException':
|
||||
raise ValueError(f"Repository does not exist: {repo_name}") from e
|
||||
|
@ -1,14 +1,16 @@
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
from collections import Counter
|
||||
from typing import List, Optional, Tuple
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from pr_agent.git_providers.codecommit_client import CodeCommitClient
|
||||
|
||||
from ..algo.language_handler import is_valid_file, language_extension_map
|
||||
from ..algo.utils import EDIT_TYPE, FilePatchInfo, load_large_diff
|
||||
from .git_provider import GitProvider
|
||||
from ..algo.pr_processing import clip_tokens
|
||||
from ..algo.utils import load_large_diff
|
||||
from ..config_loader import get_settings
|
||||
from .git_provider import EDIT_TYPE, FilePatchInfo, GitProvider, IncrementalPR
|
||||
from pr_agent.git_providers.codecommit_client import CodeCommitClient
|
||||
|
||||
|
||||
class PullRequestCCMimic:
|
||||
@ -152,26 +154,63 @@ class CodeCommitProvider(GitProvider):
|
||||
return self.diff_files
|
||||
|
||||
def publish_description(self, pr_title: str, pr_body: str):
|
||||
return "" # not implemented yet
|
||||
try:
|
||||
self.codecommit_client.publish_description(
|
||||
pr_number=self.pr_num,
|
||||
pr_title=pr_title,
|
||||
pr_body=CodeCommitProvider._add_additional_newlines(pr_body),
|
||||
)
|
||||
except Exception as e:
|
||||
raise ValueError(f"CodeCommit Cannot publish description for PR: {self.pr_num}") from e
|
||||
|
||||
def publish_comment(self, pr_comment: str, is_temporary: bool = False):
|
||||
if is_temporary:
|
||||
logging.info(pr_comment)
|
||||
return
|
||||
|
||||
pr_comment = CodeCommitProvider._remove_markdown_html(pr_comment)
|
||||
pr_comment = CodeCommitProvider._add_additional_newlines(pr_comment)
|
||||
|
||||
try:
|
||||
self.codecommit_client.publish_comment(
|
||||
repo_name=self.repo_name,
|
||||
pr_number=str(self.pr_num),
|
||||
pr_number=self.pr_num,
|
||||
destination_commit=self.pr.destination_commit,
|
||||
source_commit=self.pr.source_commit,
|
||||
comment=pr_comment,
|
||||
)
|
||||
except Exception as e:
|
||||
raise ValueError(f"CodeCommit Cannot post comment for PR: {self.pr_num}") from e
|
||||
raise ValueError(f"CodeCommit Cannot publish comment for PR: {self.pr_num}") from e
|
||||
|
||||
def publish_code_suggestions(self, code_suggestions: list) -> bool:
|
||||
return [""] # not implemented yet
|
||||
counter = 1
|
||||
for suggestion in code_suggestions:
|
||||
# Verify that each suggestion has the required keys
|
||||
if not all(key in suggestion for key in ["body", "relevant_file", "relevant_lines_start"]):
|
||||
logging.warning(f"Skipping code suggestion #{counter}: Each suggestion must have 'body', 'relevant_file', 'relevant_lines_start' keys")
|
||||
continue
|
||||
|
||||
# Publish the code suggestion to CodeCommit
|
||||
try:
|
||||
logging.debug(f"Code Suggestion #{counter} in file: {suggestion['relevant_file']}: {suggestion['relevant_lines_start']}")
|
||||
self.codecommit_client.publish_comment(
|
||||
repo_name=self.repo_name,
|
||||
pr_number=self.pr_num,
|
||||
destination_commit=self.pr.destination_commit,
|
||||
source_commit=self.pr.source_commit,
|
||||
comment=suggestion["body"],
|
||||
annotation_file=suggestion["relevant_file"],
|
||||
annotation_line=suggestion["relevant_lines_start"],
|
||||
)
|
||||
except Exception as e:
|
||||
raise ValueError(f"CodeCommit Cannot publish code suggestions for PR: {self.pr_num}") from e
|
||||
|
||||
counter += 1
|
||||
|
||||
# The calling function passes in a list of code suggestions, and this function publishes each suggestion one at a time.
|
||||
# If we were to return False here, the calling function will attempt to publish the same list of code suggestions again, one at a time.
|
||||
# Since this function publishes the suggestions one at a time anyway, we always return True here to avoid the retry.
|
||||
return True
|
||||
|
||||
def publish_labels(self, labels):
|
||||
return [""] # not implemented yet
|
||||
@ -183,6 +222,7 @@ class CodeCommitProvider(GitProvider):
|
||||
return "" # not implemented yet
|
||||
|
||||
def publish_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
|
||||
# https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/post_comment_for_compared_commit.html
|
||||
raise NotImplementedError("CodeCommit provider does not support publishing inline comments yet")
|
||||
|
||||
def create_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
|
||||
@ -199,7 +239,7 @@ class CodeCommitProvider(GitProvider):
|
||||
Returns a dictionary of languages, containing the percentage of each language used in the PR.
|
||||
|
||||
Returns:
|
||||
dict: A dictionary where each key is a language name and the corresponding value is the percentage of that language in the PR.
|
||||
- dict: A dictionary where each key is a language name and the corresponding value is the percentage of that language in the PR.
|
||||
"""
|
||||
commit_files = self.get_files()
|
||||
filenames = [ item.filename for item in commit_files ]
|
||||
@ -243,18 +283,29 @@ class CodeCommitProvider(GitProvider):
|
||||
return self.codecommit_client.get_file(self.repo_name, settings_filename, self.pr.source_commit, optional=True)
|
||||
|
||||
def add_eyes_reaction(self, issue_comment_id: int) -> Optional[int]:
|
||||
logging.info("CodeCommit provider does not support eyes reaction yet")
|
||||
return True
|
||||
|
||||
def remove_reaction(self, issue_comment_id: int, reaction_id: int) -> bool:
|
||||
logging.info("CodeCommit provider does not support removing reactions yet")
|
||||
return True
|
||||
|
||||
@staticmethod
|
||||
def _parse_pr_url(pr_url: str) -> Tuple[str, int]:
|
||||
"""
|
||||
Parse the CodeCommit PR URL and return the repository name and PR number.
|
||||
|
||||
Args:
|
||||
- pr_url: the full AWS CodeCommit pull request URL
|
||||
|
||||
Returns:
|
||||
- Tuple[str, int]: A tuple containing the repository name and PR number.
|
||||
"""
|
||||
# Example PR URL:
|
||||
# https://us-east-1.console.aws.amazon.com/codesuite/codecommit/repositories/__MY_REPO__/pull-requests/123456"
|
||||
parsed_url = urlparse(pr_url)
|
||||
|
||||
if "us-east-1.console.aws.amazon.com" not in parsed_url.netloc:
|
||||
if not CodeCommitProvider._is_valid_codecommit_hostname(parsed_url.netloc):
|
||||
raise ValueError(f"The provided URL is not a valid CodeCommit URL: {pr_url}")
|
||||
|
||||
path_parts = parsed_url.path.strip("/").split("/")
|
||||
@ -277,17 +328,33 @@ class CodeCommitProvider(GitProvider):
|
||||
|
||||
return repo_name, pr_number
|
||||
|
||||
@staticmethod
|
||||
def _is_valid_codecommit_hostname(hostname: str) -> bool:
|
||||
"""
|
||||
Check if the provided hostname is a valid AWS CodeCommit hostname.
|
||||
|
||||
This is not an exhaustive check of AWS region names,
|
||||
but instead uses a regex to check for matching AWS region patterns.
|
||||
|
||||
Args:
|
||||
- hostname: the hostname to check
|
||||
|
||||
Returns:
|
||||
- bool: True if the hostname is valid, False otherwise.
|
||||
"""
|
||||
return re.match(r"^[a-z]{2}-(gov-)?[a-z]+-\d\.console\.aws\.amazon\.com$", hostname) is not None
|
||||
|
||||
def _get_pr(self):
|
||||
response = self.codecommit_client.get_pr(self.pr_num)
|
||||
response = self.codecommit_client.get_pr(self.repo_name, self.pr_num)
|
||||
|
||||
if len(response.targets) == 0:
|
||||
raise ValueError(f"No files found in CodeCommit PR: {self.pr_num}")
|
||||
|
||||
# TODO: implement support for multiple commits in one CodeCommit PR
|
||||
# for now, we are only using the first commit in the PR
|
||||
# TODO: implement support for multiple targets in one CodeCommit PR
|
||||
# for now, we are only using the first target in the PR
|
||||
if len(response.targets) > 1:
|
||||
logging.warning(
|
||||
"Multiple commits in one PR is not supported for CodeCommit yet. Continuing, using the first commit only..."
|
||||
"Multiple targets in one PR is not supported for CodeCommit yet. Continuing, using the first target only..."
|
||||
)
|
||||
|
||||
# Return our object that mimics PullRequest class from the PyGithub library
|
||||
@ -305,13 +372,52 @@ class CodeCommitProvider(GitProvider):
|
||||
return "" # not implemented yet
|
||||
|
||||
@staticmethod
|
||||
def _get_edit_type(codecommit_change_type):
|
||||
def _add_additional_newlines(body: str) -> str:
|
||||
"""
|
||||
Replace single newlines in a PR body with double newlines.
|
||||
|
||||
CodeCommit Markdown does not seem to render as well as GitHub Markdown,
|
||||
so we add additional newlines to the PR body to make it more readable in CodeCommit.
|
||||
|
||||
Args:
|
||||
- body: the PR body
|
||||
|
||||
Returns:
|
||||
- str: the PR body with the double newlines added
|
||||
"""
|
||||
return re.sub(r'(?<!\n)\n(?!\n)', '\n\n', body)
|
||||
|
||||
@staticmethod
|
||||
def _remove_markdown_html(comment: str) -> str:
|
||||
"""
|
||||
Remove the HTML tags from a PR comment.
|
||||
|
||||
CodeCommit Markdown does not seem to render as well as GitHub Markdown,
|
||||
so we remove the HTML tags from the PR comment to make it more readable in CodeCommit.
|
||||
|
||||
Args:
|
||||
- comment: the PR comment
|
||||
|
||||
Returns:
|
||||
- str: the PR comment with the HTML tags removed
|
||||
"""
|
||||
comment = comment.replace("<details>", "")
|
||||
comment = comment.replace("</details>", "")
|
||||
comment = comment.replace("<summary>", "")
|
||||
comment = comment.replace("</summary>", "")
|
||||
return comment
|
||||
|
||||
@staticmethod
|
||||
def _get_edit_type(codecommit_change_type: str):
|
||||
"""
|
||||
Convert the CodeCommit change type string to the EDIT_TYPE enum.
|
||||
The CodeCommit change type string is returned from the get_differences SDK method.
|
||||
|
||||
Args:
|
||||
- codecommit_change_type: the CodeCommit change type string
|
||||
|
||||
Returns:
|
||||
An EDIT_TYPE enum representing the modified, added, deleted, or renamed file in the PR diff.
|
||||
- An EDIT_TYPE enum representing the modified, added, deleted, or renamed file in the PR diff.
|
||||
"""
|
||||
t = codecommit_change_type.upper()
|
||||
edit_type = None
|
||||
@ -332,6 +438,12 @@ class CodeCommitProvider(GitProvider):
|
||||
The returned extensions will include the dot "." prefix,
|
||||
to accommodate for the dots in the existing language_extension_map settings.
|
||||
Filenames with no extension will return an empty string for the extension.
|
||||
|
||||
Args:
|
||||
- filenames: a list of filenames
|
||||
|
||||
Returns:
|
||||
- list: A list of file extensions, including the dot "." prefix.
|
||||
"""
|
||||
extensions = []
|
||||
for filename in filenames:
|
||||
@ -348,6 +460,12 @@ class CodeCommitProvider(GitProvider):
|
||||
Return a dictionary containing the programming language name (as the key),
|
||||
and the percentage that language is used (as the value),
|
||||
given a list of file extensions.
|
||||
|
||||
Args:
|
||||
- extensions: a list of file extensions
|
||||
|
||||
Returns:
|
||||
- dict: A dictionary where each key is a language name and the corresponding value is the percentage of that language in the PR.
|
||||
"""
|
||||
total_files = len(extensions)
|
||||
if total_files == 0:
|
||||
|
393
pr_agent/git_providers/gerrit_provider.py
Normal file
393
pr_agent/git_providers/gerrit_provider.py
Normal file
@ -0,0 +1,393 @@
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import pathlib
|
||||
import shutil
|
||||
import subprocess
|
||||
import uuid
|
||||
from collections import Counter, namedtuple
|
||||
from pathlib import Path
|
||||
from tempfile import mkdtemp, NamedTemporaryFile
|
||||
|
||||
import requests
|
||||
import urllib3.util
|
||||
from git import Repo
|
||||
|
||||
from pr_agent.config_loader import get_settings
|
||||
from pr_agent.git_providers.git_provider import GitProvider, FilePatchInfo, \
|
||||
EDIT_TYPE
|
||||
from pr_agent.git_providers.local_git_provider import PullRequestMimic
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _call(*command, **kwargs) -> (int, str, str):
|
||||
res = subprocess.run(
|
||||
command,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
check=True,
|
||||
**kwargs,
|
||||
)
|
||||
return res.stdout.decode()
|
||||
|
||||
|
||||
def clone(url, directory):
|
||||
logger.info("Cloning %s to %s", url, directory)
|
||||
stdout = _call('git', 'clone', "--depth", "1", url, directory)
|
||||
logger.info(stdout)
|
||||
|
||||
|
||||
def fetch(url, refspec, cwd):
|
||||
logger.info("Fetching %s %s", url, refspec)
|
||||
stdout = _call(
|
||||
'git', 'fetch', '--depth', '2', url, refspec,
|
||||
cwd=cwd
|
||||
)
|
||||
logger.info(stdout)
|
||||
|
||||
|
||||
def checkout(cwd):
|
||||
logger.info("Checking out")
|
||||
stdout = _call('git', 'checkout', "FETCH_HEAD", cwd=cwd)
|
||||
logger.info(stdout)
|
||||
|
||||
|
||||
def show(*args, cwd=None):
|
||||
logger.info("Show")
|
||||
return _call('git', 'show', *args, cwd=cwd)
|
||||
|
||||
|
||||
def diff(*args, cwd=None):
|
||||
logger.info("Diff")
|
||||
patch = _call('git', 'diff', *args, cwd=cwd)
|
||||
if not patch:
|
||||
logger.warning("No changes found")
|
||||
return
|
||||
return patch
|
||||
|
||||
|
||||
def reset_local_changes(cwd):
|
||||
logger.info("Reset local changes")
|
||||
_call('git', 'checkout', "--force", cwd=cwd)
|
||||
|
||||
|
||||
def add_comment(url: urllib3.util.Url, refspec, message):
|
||||
*_, patchset, changenum = refspec.rsplit("/")
|
||||
message = "'" + message.replace("'", "'\"'\"'") + "'"
|
||||
return _call(
|
||||
"ssh",
|
||||
"-p", str(url.port),
|
||||
f"{url.auth}@{url.host}",
|
||||
"gerrit", "review",
|
||||
"--message", message,
|
||||
# "--code-review", score,
|
||||
f"{patchset},{changenum}",
|
||||
)
|
||||
|
||||
|
||||
def list_comments(url: urllib3.util.Url, refspec):
|
||||
*_, patchset, _ = refspec.rsplit("/")
|
||||
stdout = _call(
|
||||
"ssh",
|
||||
"-p", str(url.port),
|
||||
f"{url.auth}@{url.host}",
|
||||
"gerrit", "query",
|
||||
"--comments",
|
||||
"--current-patch-set", patchset,
|
||||
"--format", "JSON",
|
||||
)
|
||||
change_set, *_ = stdout.splitlines()
|
||||
return json.loads(change_set)["currentPatchSet"]["comments"]
|
||||
|
||||
|
||||
def prepare_repo(url: urllib3.util.Url, project, refspec):
|
||||
repo_url = (f"{url.scheme}://{url.auth}@{url.host}:{url.port}/{project}")
|
||||
|
||||
directory = pathlib.Path(mkdtemp())
|
||||
clone(repo_url, directory),
|
||||
fetch(repo_url, refspec, cwd=directory)
|
||||
checkout(cwd=directory)
|
||||
return directory
|
||||
|
||||
|
||||
def adopt_to_gerrit_message(message):
|
||||
lines = message.splitlines()
|
||||
buf = []
|
||||
for line in lines:
|
||||
line = line.replace("*", "").replace("``", "`")
|
||||
line = line.strip()
|
||||
if line.startswith('#'):
|
||||
buf.append("\n" +
|
||||
line.replace('#', '').removesuffix(":").strip() +
|
||||
":")
|
||||
continue
|
||||
elif line.startswith('-'):
|
||||
buf.append(line.removeprefix('-').strip())
|
||||
continue
|
||||
else:
|
||||
buf.append(line)
|
||||
return "\n".join(buf).strip()
|
||||
|
||||
|
||||
def add_suggestion(src_filename, context: str, start, end: int):
|
||||
with (
|
||||
NamedTemporaryFile("w", delete=False) as tmp,
|
||||
open(src_filename, "r") as src
|
||||
):
|
||||
lines = src.readlines()
|
||||
tmp.writelines(lines[:start - 1])
|
||||
if context:
|
||||
tmp.write(context)
|
||||
tmp.writelines(lines[end:])
|
||||
|
||||
shutil.copy(tmp.name, src_filename)
|
||||
os.remove(tmp.name)
|
||||
|
||||
|
||||
def upload_patch(patch, path):
|
||||
patch_server_endpoint = get_settings().get(
|
||||
'gerrit.patch_server_endpoint')
|
||||
patch_server_token = get_settings().get(
|
||||
'gerrit.patch_server_token')
|
||||
|
||||
response = requests.post(
|
||||
patch_server_endpoint,
|
||||
json={
|
||||
"content": patch,
|
||||
"path": path,
|
||||
},
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {patch_server_token}",
|
||||
}
|
||||
)
|
||||
response.raise_for_status()
|
||||
patch_server_endpoint = patch_server_endpoint.rstrip("/")
|
||||
return patch_server_endpoint + "/" + path
|
||||
|
||||
|
||||
class GerritProvider(GitProvider):
|
||||
|
||||
def __init__(self, key: str, incremental=False):
|
||||
self.project, self.refspec = key.split(':')
|
||||
assert self.project, "Project name is required"
|
||||
assert self.refspec, "Refspec is required"
|
||||
base_url = get_settings().get('gerrit.url')
|
||||
assert base_url, "Gerrit URL is required"
|
||||
user = get_settings().get('gerrit.user')
|
||||
assert user, "Gerrit user is required"
|
||||
|
||||
parsed = urllib3.util.parse_url(base_url)
|
||||
self.parsed_url = urllib3.util.parse_url(
|
||||
f"{parsed.scheme}://{user}@{parsed.host}:{parsed.port}"
|
||||
)
|
||||
|
||||
self.repo_path = prepare_repo(
|
||||
self.parsed_url, self.project, self.refspec
|
||||
)
|
||||
self.repo = Repo(self.repo_path)
|
||||
assert self.repo
|
||||
|
||||
self.pr = PullRequestMimic(self.get_pr_title(), self.get_diff_files())
|
||||
|
||||
def get_pr_title(self):
|
||||
"""
|
||||
Substitutes the branch-name as the PR-mimic title.
|
||||
"""
|
||||
return self.repo.branches[0].name
|
||||
|
||||
def get_issue_comments(self):
|
||||
comments = list_comments(self.parsed_url, self.refspec)
|
||||
Comments = namedtuple('Comments', ['reversed'])
|
||||
Comment = namedtuple('Comment', ['body'])
|
||||
return Comments([Comment(c['message']) for c in reversed(comments)])
|
||||
|
||||
def get_labels(self):
|
||||
raise NotImplementedError(
|
||||
'Getting labels is not implemented for the gerrit provider')
|
||||
|
||||
def add_eyes_reaction(self, issue_comment_id: int):
|
||||
raise NotImplementedError(
|
||||
'Adding reactions is not implemented for the gerrit provider')
|
||||
|
||||
def remove_reaction(self, issue_comment_id: int, reaction_id: int):
|
||||
raise NotImplementedError(
|
||||
'Removing reactions is not implemented for the gerrit provider')
|
||||
|
||||
def get_commit_messages(self):
|
||||
return [self.repo.head.commit.message]
|
||||
|
||||
def get_repo_settings(self):
|
||||
"""
|
||||
TODO: Implement support of .pr_agent.toml
|
||||
"""
|
||||
return ""
|
||||
|
||||
def get_diff_files(self) -> list[FilePatchInfo]:
|
||||
diffs = self.repo.head.commit.diff(
|
||||
self.repo.head.commit.parents[0], # previous commit
|
||||
create_patch=True,
|
||||
R=True
|
||||
)
|
||||
|
||||
diff_files = []
|
||||
for diff_item in diffs:
|
||||
if diff_item.a_blob is not None:
|
||||
original_file_content_str = (
|
||||
diff_item.a_blob.data_stream.read().decode('utf-8')
|
||||
)
|
||||
else:
|
||||
original_file_content_str = "" # empty file
|
||||
if diff_item.b_blob is not None:
|
||||
new_file_content_str = diff_item.b_blob.data_stream.read(). \
|
||||
decode('utf-8')
|
||||
else:
|
||||
new_file_content_str = "" # empty file
|
||||
edit_type = EDIT_TYPE.MODIFIED
|
||||
if diff_item.new_file:
|
||||
edit_type = EDIT_TYPE.ADDED
|
||||
elif diff_item.deleted_file:
|
||||
edit_type = EDIT_TYPE.DELETED
|
||||
elif diff_item.renamed_file:
|
||||
edit_type = EDIT_TYPE.RENAMED
|
||||
diff_files.append(
|
||||
FilePatchInfo(
|
||||
original_file_content_str,
|
||||
new_file_content_str,
|
||||
diff_item.diff.decode('utf-8'),
|
||||
diff_item.b_path,
|
||||
edit_type=edit_type,
|
||||
old_filename=None
|
||||
if diff_item.a_path == diff_item.b_path
|
||||
else diff_item.a_path
|
||||
)
|
||||
)
|
||||
self.diff_files = diff_files
|
||||
return diff_files
|
||||
|
||||
def get_files(self):
|
||||
diff_index = self.repo.head.commit.diff(
|
||||
self.repo.head.commit.parents[0], # previous commit
|
||||
R=True
|
||||
)
|
||||
# Get the list of changed files
|
||||
diff_files = [item.a_path for item in diff_index]
|
||||
return diff_files
|
||||
|
||||
def get_languages(self):
|
||||
"""
|
||||
Calculate percentage of languages in repository. Used for hunk
|
||||
prioritisation.
|
||||
"""
|
||||
# Get all files in repository
|
||||
filepaths = [Path(item.path) for item in
|
||||
self.repo.tree().traverse() if item.type == 'blob']
|
||||
# Identify language by file extension and count
|
||||
lang_count = Counter(
|
||||
ext.lstrip('.') for filepath in filepaths for ext in
|
||||
[filepath.suffix.lower()])
|
||||
# Convert counts to percentages
|
||||
total_files = len(filepaths)
|
||||
lang_percentage = {lang: count / total_files * 100 for lang, count
|
||||
in lang_count.items()}
|
||||
return lang_percentage
|
||||
|
||||
def get_pr_description_full(self):
|
||||
return self.repo.head.commit.message
|
||||
|
||||
def get_user_id(self):
|
||||
return self.repo.head.commit.author.email
|
||||
|
||||
def is_supported(self, capability: str) -> bool:
|
||||
if capability in [
|
||||
# 'get_issue_comments',
|
||||
'create_inline_comment',
|
||||
'publish_inline_comments',
|
||||
'get_labels'
|
||||
]:
|
||||
return False
|
||||
return True
|
||||
|
||||
def split_suggestion(self, msg) -> tuple[str, str]:
|
||||
is_code_context = False
|
||||
description = []
|
||||
context = []
|
||||
for line in msg.splitlines():
|
||||
if line.startswith('```suggestion'):
|
||||
is_code_context = True
|
||||
continue
|
||||
if line.startswith('```'):
|
||||
is_code_context = False
|
||||
continue
|
||||
if is_code_context:
|
||||
context.append(line)
|
||||
else:
|
||||
description.append(
|
||||
line.replace('*', '')
|
||||
)
|
||||
|
||||
return (
|
||||
'\n'.join(description),
|
||||
'\n'.join(context) + '\n' if context else ''
|
||||
)
|
||||
|
||||
def publish_code_suggestions(self, code_suggestions: list):
|
||||
msg = []
|
||||
for suggestion in code_suggestions:
|
||||
description, code = self.split_suggestion(suggestion['body'])
|
||||
add_suggestion(
|
||||
pathlib.Path(self.repo_path) / suggestion["relevant_file"],
|
||||
code,
|
||||
suggestion["relevant_lines_start"],
|
||||
suggestion["relevant_lines_end"],
|
||||
)
|
||||
patch = diff(cwd=self.repo_path)
|
||||
patch_id = uuid.uuid4().hex[0:4]
|
||||
path = "/".join(["codium-ai", self.refspec, patch_id])
|
||||
full_path = upload_patch(patch, path)
|
||||
reset_local_changes(self.repo_path)
|
||||
msg.append(f'* {description}\n{full_path}')
|
||||
|
||||
if msg:
|
||||
add_comment(self.parsed_url, self.refspec, "\n".join(msg))
|
||||
return True
|
||||
|
||||
def publish_comment(self, pr_comment: str, is_temporary: bool = False):
|
||||
if not is_temporary:
|
||||
msg = adopt_to_gerrit_message(pr_comment)
|
||||
add_comment(self.parsed_url, self.refspec, msg)
|
||||
|
||||
def publish_description(self, pr_title: str, pr_body: str):
|
||||
msg = adopt_to_gerrit_message(pr_body)
|
||||
add_comment(self.parsed_url, self.refspec, pr_title + '\n' + msg)
|
||||
|
||||
def publish_inline_comments(self, comments: list[dict]):
|
||||
raise NotImplementedError(
|
||||
'Publishing inline comments is not implemented for the gerrit '
|
||||
'provider')
|
||||
|
||||
def publish_inline_comment(self, body: str, relevant_file: str,
|
||||
relevant_line_in_file: str):
|
||||
raise NotImplementedError(
|
||||
'Publishing inline comments is not implemented for the gerrit '
|
||||
'provider')
|
||||
|
||||
def create_inline_comment(self, body: str, relevant_file: str,
|
||||
relevant_line_in_file: str):
|
||||
raise NotImplementedError(
|
||||
'Creating inline comments is not implemented for the gerrit '
|
||||
'provider')
|
||||
|
||||
def publish_labels(self, labels):
|
||||
# Not applicable to the local git provider,
|
||||
# but required by the interface
|
||||
pass
|
||||
|
||||
def remove_initial_comment(self):
|
||||
# remove repo, cloned in previous steps
|
||||
# shutil.rmtree(self.repo_path)
|
||||
pass
|
||||
|
||||
def get_pr_branch(self):
|
||||
return self.repo.head
|
@ -1,10 +1,28 @@
|
||||
import logging
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass
|
||||
|
||||
# enum EDIT_TYPE (ADDED, DELETED, MODIFIED, RENAMED)
|
||||
from enum import Enum
|
||||
from typing import Optional
|
||||
|
||||
from pr_agent.algo.utils import FilePatchInfo
|
||||
|
||||
class EDIT_TYPE(Enum):
|
||||
ADDED = 1
|
||||
DELETED = 2
|
||||
MODIFIED = 3
|
||||
RENAMED = 4
|
||||
|
||||
|
||||
@dataclass
|
||||
class FilePatchInfo:
|
||||
base_file: str
|
||||
head_file: str
|
||||
patch: str
|
||||
filename: str
|
||||
tokens: int = -1
|
||||
edit_type: EDIT_TYPE = EDIT_TYPE.MODIFIED
|
||||
old_filename: str = None
|
||||
|
||||
|
||||
class GitProvider(ABC):
|
||||
@ -68,11 +86,11 @@ class GitProvider(ABC):
|
||||
def get_pr_description_full(self) -> str:
|
||||
pass
|
||||
|
||||
def get_pr_description(self) -> str:
|
||||
def get_pr_description(self, *, full: bool = True) -> str:
|
||||
from pr_agent.config_loader import get_settings
|
||||
from pr_agent.algo.utils import clip_tokens
|
||||
from pr_agent.algo.pr_processing import clip_tokens
|
||||
max_tokens = get_settings().get("CONFIG.MAX_DESCRIPTION_TOKENS", None)
|
||||
description = self.get_pr_description_full()
|
||||
description = self.get_pr_description_full() if full else self.get_user_description()
|
||||
if max_tokens:
|
||||
return clip_tokens(description, max_tokens)
|
||||
return description
|
||||
@ -143,7 +161,6 @@ def get_main_pr_language(languages, files) -> str:
|
||||
most_common_extension == 'scala' and top_language == 'scala' or \
|
||||
most_common_extension == 'kt' and top_language == 'kotlin' or \
|
||||
most_common_extension == 'pl' and top_language == 'perl' or \
|
||||
most_common_extension == 'swift' and top_language == 'swift' or \
|
||||
most_common_extension == top_language:
|
||||
main_language_str = top_language
|
||||
|
||||
|
@ -9,9 +9,10 @@ from github import AppAuthentication, Auth, Github, GithubException, Reaction
|
||||
from retry import retry
|
||||
from starlette_context import context
|
||||
|
||||
from .git_provider import GitProvider, IncrementalPR
|
||||
from .git_provider import FilePatchInfo, GitProvider, IncrementalPR
|
||||
from ..algo.language_handler import is_valid_file
|
||||
from ..algo.utils import load_large_diff, clip_tokens, find_line_number_of_relevant_line_in_file, FilePatchInfo
|
||||
from ..algo.utils import load_large_diff
|
||||
from ..algo.pr_processing import find_line_number_of_relevant_line_in_file, clip_tokens
|
||||
from ..config_loader import get_settings
|
||||
from ..servers.utils import RateLimitExceeded
|
||||
|
||||
|
@ -7,9 +7,10 @@ import gitlab
|
||||
from gitlab import GitlabGetError
|
||||
|
||||
from ..algo.language_handler import is_valid_file
|
||||
from ..algo.utils import load_large_diff, clip_tokens, EDIT_TYPE, FilePatchInfo
|
||||
from ..algo.pr_processing import clip_tokens
|
||||
from ..algo.utils import load_large_diff
|
||||
from ..config_loader import get_settings
|
||||
from .git_provider import GitProvider
|
||||
from .git_provider import EDIT_TYPE, FilePatchInfo, GitProvider
|
||||
|
||||
logger = logging.getLogger()
|
||||
|
||||
|
@ -1,79 +0,0 @@
|
||||
import itertools
|
||||
from collections import Counter
|
||||
from typing import List, Optional
|
||||
|
||||
from pr_agent.algo.utils import FilePatchInfo
|
||||
from pr_agent.git_providers.git_provider import GitProvider
|
||||
|
||||
|
||||
class InMemoryProvider(GitProvider):
|
||||
def __init__(self, head_branch: str, target_branch: str, files: List[FilePatchInfo]):
|
||||
self.head_branch = head_branch
|
||||
self.target_branch = target_branch
|
||||
self.files = files
|
||||
|
||||
def is_supported(self, capability: str) -> bool:
|
||||
pass
|
||||
|
||||
def get_files(self) -> list[FilePatchInfo]:
|
||||
return self.files
|
||||
|
||||
def get_diff_files(self) -> list[FilePatchInfo]:
|
||||
return self.get_files()
|
||||
|
||||
def publish_description(self, pr_title: str, pr_body: str):
|
||||
pass
|
||||
|
||||
def publish_comment(self, pr_comment: str, is_temporary: bool = False):
|
||||
pass
|
||||
|
||||
def publish_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
|
||||
pass
|
||||
|
||||
def create_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
|
||||
pass
|
||||
|
||||
def publish_inline_comments(self, comments: list[dict]):
|
||||
pass
|
||||
|
||||
def publish_code_suggestions(self, code_suggestions: list) -> bool:
|
||||
pass
|
||||
|
||||
def publish_labels(self, labels):
|
||||
pass
|
||||
|
||||
def get_labels(self):
|
||||
pass
|
||||
|
||||
def remove_initial_comment(self):
|
||||
pass
|
||||
|
||||
def get_languages(self):
|
||||
language_count = Counter(file.language for file in self.files)
|
||||
return dict(language_count)
|
||||
|
||||
def get_pr_branch(self):
|
||||
pass
|
||||
|
||||
def get_user_id(self):
|
||||
pass
|
||||
|
||||
def get_pr_description_full(self) -> str:
|
||||
pass
|
||||
|
||||
def get_issue_comments(self):
|
||||
pass
|
||||
|
||||
def get_repo_settings(self):
|
||||
pass
|
||||
|
||||
def add_eyes_reaction(self, issue_comment_id: int) -> Optional[int]:
|
||||
pass
|
||||
|
||||
def remove_reaction(self, issue_comment_id: int, reaction_id: int) -> bool:
|
||||
pass
|
||||
|
||||
def get_commit_messages(self):
|
||||
pass
|
||||
|
||||
|
@ -6,8 +6,7 @@ from typing import List
|
||||
from git import Repo
|
||||
|
||||
from pr_agent.config_loader import _find_repository_root, get_settings
|
||||
from pr_agent.git_providers.git_provider import GitProvider
|
||||
from pr_agent.algo.utils import EDIT_TYPE, FilePatchInfo
|
||||
from pr_agent.git_providers.git_provider import EDIT_TYPE, FilePatchInfo, GitProvider
|
||||
|
||||
|
||||
class PullRequestMimic:
|
||||
|
78
pr_agent/servers/gerrit_server.py
Normal file
78
pr_agent/servers/gerrit_server.py
Normal file
@ -0,0 +1,78 @@
|
||||
import copy
|
||||
import logging
|
||||
import sys
|
||||
from enum import Enum
|
||||
from json import JSONDecodeError
|
||||
|
||||
import uvicorn
|
||||
from fastapi import APIRouter, FastAPI, HTTPException
|
||||
from pydantic import BaseModel
|
||||
from starlette.middleware import Middleware
|
||||
from starlette_context import context
|
||||
from starlette_context.middleware import RawContextMiddleware
|
||||
|
||||
from pr_agent.agent.pr_agent import PRAgent
|
||||
from pr_agent.config_loader import global_settings, get_settings
|
||||
|
||||
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
class Action(str, Enum):
|
||||
review = "review"
|
||||
describe = "describe"
|
||||
ask = "ask"
|
||||
improve = "improve"
|
||||
reflect = "reflect"
|
||||
answer = "answer"
|
||||
|
||||
|
||||
class Item(BaseModel):
|
||||
refspec: str
|
||||
project: str
|
||||
msg: str
|
||||
|
||||
|
||||
@router.post("/api/v1/gerrit/{action}")
|
||||
async def handle_gerrit_request(action: Action, item: Item):
|
||||
logging.debug("Received a Gerrit request")
|
||||
context["settings"] = copy.deepcopy(global_settings)
|
||||
|
||||
if action == Action.ask:
|
||||
if not item.msg:
|
||||
return HTTPException(
|
||||
status_code=400,
|
||||
detail="msg is required for ask command"
|
||||
)
|
||||
await PRAgent().handle_request(
|
||||
f"{item.project}:{item.refspec}",
|
||||
f"/{item.msg.strip()}"
|
||||
)
|
||||
|
||||
|
||||
async def get_body(request):
|
||||
try:
|
||||
body = await request.json()
|
||||
except JSONDecodeError as e:
|
||||
logging.error("Error parsing request body", e)
|
||||
return {}
|
||||
return body
|
||||
|
||||
|
||||
@router.get("/")
|
||||
async def root():
|
||||
return {"status": "ok"}
|
||||
|
||||
|
||||
def start():
|
||||
# to prevent adding help messages with the output
|
||||
get_settings().set("CONFIG.CLI_MODE", True)
|
||||
middleware = [Middleware(RawContextMiddleware)]
|
||||
app = FastAPI(middleware=middleware)
|
||||
app.include_router(router)
|
||||
|
||||
uvicorn.run(app, host="0.0.0.0", port=3000)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
start()
|
@ -12,6 +12,7 @@ from starlette_context import context
|
||||
from starlette_context.middleware import RawContextMiddleware
|
||||
|
||||
from pr_agent.agent.pr_agent import PRAgent
|
||||
from pr_agent.algo.utils import update_settings_from_args
|
||||
from pr_agent.config_loader import get_settings, global_settings
|
||||
from pr_agent.git_providers import get_git_provider
|
||||
from pr_agent.servers.utils import verify_signature
|
||||
@ -97,6 +98,7 @@ async def handle_request(body: Dict[str, Any], event: str):
|
||||
api_url = body["comment"]["pull_request_url"]
|
||||
else:
|
||||
return {}
|
||||
logging.info(body)
|
||||
logging.info(f"Handling comment because of event={event} and action={action}")
|
||||
comment_id = body.get("comment", {}).get("id")
|
||||
provider = get_git_provider()(pr_url=api_url)
|
||||
@ -123,8 +125,14 @@ async def handle_request(body: Dict[str, Any], event: str):
|
||||
return {}
|
||||
logging.info(f"Performing review because of event={event} and action={action}")
|
||||
for command in get_settings().github_app.pr_commands:
|
||||
logging.info(f"Performing command: {command}")
|
||||
await agent.handle_request(api_url, command)
|
||||
split_command = command.split(" ")
|
||||
command = split_command[0]
|
||||
args = split_command[1:]
|
||||
other_args = update_settings_from_args(args)
|
||||
new_command = ' '.join([command] + other_args)
|
||||
logging.info(body)
|
||||
logging.info(f"Performing command: {new_command}")
|
||||
await agent.handle_request(api_url, new_command)
|
||||
|
||||
logging.info("event or action does not require handling")
|
||||
return {}
|
||||
|
@ -1,21 +1,51 @@
|
||||
import copy
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
|
||||
import uvicorn
|
||||
from fastapi import APIRouter, FastAPI, Request, status
|
||||
from fastapi.encoders import jsonable_encoder
|
||||
from fastapi.responses import JSONResponse
|
||||
from starlette.background import BackgroundTasks
|
||||
from starlette.middleware import Middleware
|
||||
from starlette_context import context
|
||||
from starlette_context.middleware import RawContextMiddleware
|
||||
|
||||
from pr_agent.agent.pr_agent import PRAgent
|
||||
from pr_agent.config_loader import get_settings
|
||||
from pr_agent.config_loader import get_settings, global_settings
|
||||
from pr_agent.secret_providers import get_secret_provider
|
||||
|
||||
app = FastAPI()
|
||||
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
|
||||
router = APIRouter()
|
||||
|
||||
secret_provider = get_secret_provider() if get_settings().get("CONFIG.SECRET_PROVIDER") else None
|
||||
|
||||
|
||||
@router.post("/webhook")
|
||||
async def gitlab_webhook(background_tasks: BackgroundTasks, request: Request):
|
||||
if request.headers.get("X-Gitlab-Token") and secret_provider:
|
||||
request_token = request.headers.get("X-Gitlab-Token")
|
||||
secret = secret_provider.get_secret(request_token)
|
||||
try:
|
||||
secret_dict = json.loads(secret)
|
||||
gitlab_token = secret_dict["gitlab_token"]
|
||||
context["settings"] = copy.deepcopy(global_settings)
|
||||
context["settings"].gitlab.personal_access_token = gitlab_token
|
||||
except Exception as e:
|
||||
logging.error(f"Failed to validate secret {request_token}: {e}")
|
||||
return JSONResponse(status_code=status.HTTP_401_UNAUTHORIZED, content=jsonable_encoder({"message": "unauthorized"}))
|
||||
elif get_settings().get("GITLAB.SHARED_SECRET"):
|
||||
secret = get_settings().get("GITLAB.SHARED_SECRET")
|
||||
if not request.headers.get("X-Gitlab-Token") == secret:
|
||||
return JSONResponse(status_code=status.HTTP_401_UNAUTHORIZED, content=jsonable_encoder({"message": "unauthorized"}))
|
||||
else:
|
||||
return JSONResponse(status_code=status.HTTP_401_UNAUTHORIZED, content=jsonable_encoder({"message": "unauthorized"}))
|
||||
gitlab_token = get_settings().get("GITLAB.PERSONAL_ACCESS_TOKEN", None)
|
||||
if not gitlab_token:
|
||||
return JSONResponse(status_code=status.HTTP_401_UNAUTHORIZED, content=jsonable_encoder({"message": "unauthorized"}))
|
||||
data = await request.json()
|
||||
logging.info(json.dumps(data))
|
||||
if data.get('object_kind') == 'merge_request' and data['object_attributes'].get('action') in ['open', 'reopen']:
|
||||
logging.info(f"A merge request has been opened: {data['object_attributes'].get('title')}")
|
||||
url = data['object_attributes'].get('url')
|
||||
@ -28,16 +58,18 @@ async def gitlab_webhook(background_tasks: BackgroundTasks, request: Request):
|
||||
background_tasks.add_task(PRAgent().handle_request, url, body)
|
||||
return JSONResponse(status_code=status.HTTP_200_OK, content=jsonable_encoder({"message": "success"}))
|
||||
|
||||
|
||||
@router.get("/")
|
||||
async def root():
|
||||
return {"status": "ok"}
|
||||
|
||||
def start():
|
||||
gitlab_url = get_settings().get("GITLAB.URL", None)
|
||||
if not gitlab_url:
|
||||
raise ValueError("GITLAB.URL is not set")
|
||||
gitlab_token = get_settings().get("GITLAB.PERSONAL_ACCESS_TOKEN", None)
|
||||
if not gitlab_token:
|
||||
raise ValueError("GITLAB.PERSONAL_ACCESS_TOKEN is not set")
|
||||
get_settings().config.git_provider = "gitlab"
|
||||
|
||||
app = FastAPI()
|
||||
middleware = [Middleware(RawContextMiddleware)]
|
||||
app = FastAPI(middleware=middleware)
|
||||
app.include_router(router)
|
||||
|
||||
uvicorn.run(app, host="0.0.0.0", port=3000)
|
||||
|
@ -85,3 +85,13 @@ polling_interval_seconds = 30
|
||||
# LocalGitProvider settings - uncomment to use paths other than default
|
||||
# description_path= "path/to/description.md"
|
||||
# review_path= "path/to/review.md"
|
||||
|
||||
[gerrit]
|
||||
# endpoint to the gerrit service
|
||||
# url = "ssh://gerrit.example.com:29418"
|
||||
# user for gerrit authentication
|
||||
# user = "ai-reviewer"
|
||||
# patch server where patches will be saved
|
||||
# patch_server_endpoint = "http://127.0.0.1:5000/patch"
|
||||
# token to authenticate in the patch server
|
||||
# patch_server_token = ""
|
||||
|
@ -68,12 +68,17 @@ Code suggestions:
|
||||
type: string
|
||||
description: |-
|
||||
a code snippet showing the relevant code lines from a '__new hunk__' section.
|
||||
It must be continuous, correctly formatted and indented, and without line numbers.
|
||||
relevant lines:
|
||||
type: string
|
||||
It must be contiguous, correctly formatted and indented, and without line numbers.
|
||||
relevant lines start:
|
||||
type: integer
|
||||
description: |-
|
||||
the relevant lines from a '__new hunk__' section, in the format of 'start_line-end_line'.
|
||||
For example: '10-15'. They should be derived from the hunk line numbers, and correspond to the 'existing code' snippet above.
|
||||
The relevant line number from a '__new hunk__' section where the suggestion starts (inclusive).
|
||||
Should be derived from the hunk line numbers, and correspond to the 'existing code' snippet above.
|
||||
relevant lines end:
|
||||
type: integer
|
||||
description: |-
|
||||
The relevant line number from a '__new hunk__' section where the suggestion ends (inclusive).
|
||||
Should be derived from the hunk line numbers, and correspond to the 'existing code' snippet above.
|
||||
improved code:
|
||||
type: string
|
||||
description: |-
|
||||
@ -90,7 +95,8 @@ Code suggestions:
|
||||
Add a docstring to func1()
|
||||
existing code: |-
|
||||
def func1():
|
||||
relevant lines: '12-12'
|
||||
relevant lines start: 12
|
||||
relevant lines end: 12
|
||||
improved code: |-
|
||||
...
|
||||
```
|
||||
|
@ -42,7 +42,9 @@ class PRCodeSuggestions:
|
||||
"extra_instructions": get_settings().pr_code_suggestions.extra_instructions,
|
||||
"commit_messages_str": self.git_provider.get_commit_messages(),
|
||||
}
|
||||
self.token_handler = TokenHandler(self.vars, get_settings().pr_code_suggestions_prompt.system,
|
||||
self.token_handler = TokenHandler(self.git_provider.pr,
|
||||
self.vars,
|
||||
get_settings().pr_code_suggestions_prompt.system,
|
||||
get_settings().pr_code_suggestions_prompt.user)
|
||||
|
||||
async def run(self):
|
||||
@ -111,11 +113,8 @@ class PRCodeSuggestions:
|
||||
if get_settings().config.verbosity_level >= 2:
|
||||
logging.info(f"suggestion: {d}")
|
||||
relevant_file = d['relevant file'].strip()
|
||||
relevant_lines_str = d['relevant lines'].strip()
|
||||
if ',' in relevant_lines_str: # handling 'relevant lines': '181, 190' or '178-184, 188-194'
|
||||
relevant_lines_str = relevant_lines_str.split(',')[0]
|
||||
relevant_lines_start = int(relevant_lines_str.split('-')[0]) # absolute position
|
||||
relevant_lines_end = int(relevant_lines_str.split('-')[-1])
|
||||
relevant_lines_start = int(d['relevant lines start']) # absolute position
|
||||
relevant_lines_end = int(d['relevant lines end'])
|
||||
content = d['suggestion content']
|
||||
new_code_snippet = d['improved code']
|
||||
|
||||
|
@ -36,7 +36,7 @@ class PRDescription:
|
||||
self.vars = {
|
||||
"title": self.git_provider.pr.title,
|
||||
"branch": self.git_provider.get_pr_branch(),
|
||||
"description": self.git_provider.get_pr_description(),
|
||||
"description": self.git_provider.get_pr_description(full=False),
|
||||
"language": self.main_pr_language,
|
||||
"diff": "", # empty diff for initial calculation
|
||||
"extra_instructions": get_settings().pr_description.extra_instructions,
|
||||
@ -46,8 +46,12 @@ class PRDescription:
|
||||
self.user_description = self.git_provider.get_user_description()
|
||||
|
||||
# Initialize the token handler
|
||||
self.token_handler = TokenHandler(self.vars, get_settings().pr_description_prompt.system,
|
||||
get_settings().pr_description_prompt.user)
|
||||
self.token_handler = TokenHandler(
|
||||
self.git_provider.pr,
|
||||
self.vars,
|
||||
get_settings().pr_description_prompt.system,
|
||||
get_settings().pr_description_prompt.user,
|
||||
)
|
||||
|
||||
# Initialize patches_diff and prediction attributes
|
||||
self.patches_diff = None
|
||||
|
@ -26,7 +26,9 @@ class PRInformationFromUser:
|
||||
"diff": "", # empty diff for initial calculation
|
||||
"commit_messages_str": self.git_provider.get_commit_messages(),
|
||||
}
|
||||
self.token_handler = TokenHandler(self.vars, get_settings().pr_information_from_user_prompt.system,
|
||||
self.token_handler = TokenHandler(self.git_provider.pr,
|
||||
self.vars,
|
||||
get_settings().pr_information_from_user_prompt.system,
|
||||
get_settings().pr_information_from_user_prompt.user)
|
||||
self.patches_diff = None
|
||||
self.prediction = None
|
||||
|
@ -29,7 +29,9 @@ class PRQuestions:
|
||||
"questions": self.question_str,
|
||||
"commit_messages_str": self.git_provider.get_commit_messages(),
|
||||
}
|
||||
self.token_handler = TokenHandler(self.vars, get_settings().pr_questions_prompt.system,
|
||||
self.token_handler = TokenHandler(self.git_provider.pr,
|
||||
self.vars,
|
||||
get_settings().pr_questions_prompt.system,
|
||||
get_settings().pr_questions_prompt.user)
|
||||
self.patches_diff = None
|
||||
self.prediction = None
|
||||
|
@ -9,7 +9,8 @@ from jinja2 import Environment, StrictUndefined
|
||||
from yaml import SafeLoader
|
||||
|
||||
from pr_agent.algo.ai_handler import AiHandler
|
||||
from pr_agent.algo.pr_processing import get_pr_diff, retry_with_fallback_models
|
||||
from pr_agent.algo.pr_processing import get_pr_diff, retry_with_fallback_models, \
|
||||
find_line_number_of_relevant_line_in_file, clip_tokens
|
||||
from pr_agent.algo.token_handler import TokenHandler
|
||||
from pr_agent.algo.utils import convert_to_markdown, try_fix_json, try_fix_yaml, load_yaml
|
||||
from pr_agent.config_loader import get_settings
|
||||
@ -65,8 +66,12 @@ class PRReviewer:
|
||||
"commit_messages_str": self.git_provider.get_commit_messages(),
|
||||
}
|
||||
|
||||
self.token_handler = TokenHandler(self.vars, get_settings().pr_review_prompt.system,
|
||||
get_settings().pr_review_prompt.user)
|
||||
self.token_handler = TokenHandler(
|
||||
self.git_provider.pr,
|
||||
self.vars,
|
||||
get_settings().pr_review_prompt.system,
|
||||
get_settings().pr_review_prompt.user
|
||||
)
|
||||
|
||||
def parse_args(self, args: List[str]) -> None:
|
||||
"""
|
||||
@ -212,8 +217,8 @@ class PRReviewer:
|
||||
markdown_text = convert_to_markdown(data)
|
||||
user = self.git_provider.get_user_id()
|
||||
|
||||
# Add help text if not in CLI§ mode
|
||||
if not get_settings().get("CONFIG.CLI§_MODE", False):
|
||||
# Add help text if not in CLI mode
|
||||
if not get_settings().get("CONFIG.CLI_MODE", False):
|
||||
markdown_text += "\n### How to use\n"
|
||||
if user and '[bot]' not in user:
|
||||
markdown_text += bot_help_text(user)
|
||||
|
@ -40,7 +40,9 @@ class PRUpdateChangelog:
|
||||
"extra_instructions": get_settings().pr_update_changelog.extra_instructions,
|
||||
"commit_messages_str": self.git_provider.get_commit_messages(),
|
||||
}
|
||||
self.token_handler = TokenHandler(self.vars, get_settings().pr_update_changelog_prompt.system,
|
||||
self.token_handler = TokenHandler(self.git_provider.pr,
|
||||
self.vars,
|
||||
get_settings().pr_update_changelog_prompt.system,
|
||||
get_settings().pr_update_changelog_prompt.user)
|
||||
|
||||
async def run(self):
|
||||
|
@ -35,12 +35,12 @@ dependencies = {file = ["requirements.txt"]}
|
||||
"Homepage" = "https://github.com/Codium-ai/pr-agent"
|
||||
|
||||
[tool.setuptools]
|
||||
include-package-data = true
|
||||
include-package-data = false
|
||||
license-files = ["LICENSE"]
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["."]
|
||||
include = ["pr_agent", "pr_agent.*"]
|
||||
include = ["pr_agent"]
|
||||
|
||||
[project.scripts]
|
||||
pr-agent = "pr_agent.cli:run"
|
||||
|
@ -1,19 +1,21 @@
|
||||
dynaconf~=3.1.12
|
||||
fastapi~=0.103.0
|
||||
PyGithub~=1.59.0
|
||||
retry~=0.9.2
|
||||
openai~=0.27.8
|
||||
Jinja2~=3.1.2
|
||||
tiktoken~=0.4.0
|
||||
uvicorn~=0.22.0
|
||||
python-gitlab~=3.15.0
|
||||
dynaconf==3.1.12
|
||||
fastapi==0.99.0
|
||||
PyGithub==1.59.*
|
||||
retry==0.9.2
|
||||
openai==0.27.8
|
||||
Jinja2==3.1.2
|
||||
tiktoken==0.4.0
|
||||
uvicorn==0.22.0
|
||||
python-gitlab==3.15.0
|
||||
pytest~=7.4.0
|
||||
aiohttp~=3.8.4
|
||||
atlassian-python-api~=3.39.0
|
||||
atlassian-python-api==3.39.0
|
||||
GitPython~=3.1.32
|
||||
PyYAML~=6.0
|
||||
starlette-context~=0.3.6
|
||||
litellm~=0.1.445
|
||||
PyYAML==6.0
|
||||
starlette-context==0.3.6
|
||||
litellm~=0.1.504
|
||||
boto3~=1.28.25
|
||||
google-cloud-storage~=2.10.0
|
||||
ujson~=5.8.0
|
||||
google-cloud-storage==2.10.0
|
||||
ujson==5.8.0
|
||||
azure-devops==7.1.0b3
|
||||
msrest==0.7.1
|
@ -125,7 +125,7 @@ class TestCodeCommitProvider:
|
||||
}
|
||||
}
|
||||
|
||||
pr = api.get_pr(321)
|
||||
pr = api.get_pr("my_test_repo", 321)
|
||||
|
||||
assert pr.title == "My PR"
|
||||
assert pr.description == "My PR description"
|
||||
|
@ -1,7 +1,7 @@
|
||||
import pytest
|
||||
from pr_agent.git_providers.codecommit_provider import CodeCommitFile
|
||||
from pr_agent.git_providers.codecommit_provider import CodeCommitProvider
|
||||
from pr_agent.algo.utils import EDIT_TYPE
|
||||
from pr_agent.git_providers.git_provider import EDIT_TYPE
|
||||
|
||||
|
||||
class TestCodeCommitFile:
|
||||
@ -26,11 +26,48 @@ class TestCodeCommitFile:
|
||||
|
||||
class TestCodeCommitProvider:
|
||||
def test_parse_pr_url(self):
|
||||
# Test that the _parse_pr_url() function can extract the repo name and PR number from a CodeCommit URL
|
||||
url = "https://us-east-1.console.aws.amazon.com/codesuite/codecommit/repositories/my_test_repo/pull-requests/321"
|
||||
repo_name, pr_number = CodeCommitProvider._parse_pr_url(url)
|
||||
assert repo_name == "my_test_repo"
|
||||
assert pr_number == 321
|
||||
|
||||
def test_is_valid_codecommit_hostname(self):
|
||||
# Test the various AWS regions
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("af-south-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("ap-east-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("ap-northeast-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("ap-northeast-2.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("ap-northeast-3.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("ap-south-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("ap-south-2.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("ap-southeast-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("ap-southeast-2.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("ap-southeast-3.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("ap-southeast-4.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("ca-central-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("eu-central-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("eu-central-2.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("eu-north-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("eu-south-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("eu-south-2.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("eu-west-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("eu-west-2.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("eu-west-3.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("il-central-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("me-central-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("me-south-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("sa-east-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("us-east-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("us-east-2.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("us-gov-east-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("us-gov-west-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("us-west-1.console.aws.amazon.com")
|
||||
assert CodeCommitProvider._is_valid_codecommit_hostname("us-west-2.console.aws.amazon.com")
|
||||
# Test non-AWS regions
|
||||
assert not CodeCommitProvider._is_valid_codecommit_hostname("no-such-region.console.aws.amazon.com")
|
||||
assert not CodeCommitProvider._is_valid_codecommit_hostname("console.aws.amazon.com")
|
||||
|
||||
# Test that an error is raised when an invalid CodeCommit URL is provided to the set_pr() method of the CodeCommitProvider class.
|
||||
# Generated by CodiumAI
|
||||
def test_invalid_codecommit_url(self):
|
||||
@ -106,6 +143,7 @@ class TestCodeCommitProvider:
|
||||
assert percentages == {}
|
||||
|
||||
def test_get_edit_type(self):
|
||||
# Test that the _get_edit_type() function can convert a CodeCommit letter to an EDIT_TYPE enum
|
||||
assert CodeCommitProvider._get_edit_type("A") == EDIT_TYPE.ADDED
|
||||
assert CodeCommitProvider._get_edit_type("D") == EDIT_TYPE.DELETED
|
||||
assert CodeCommitProvider._get_edit_type("M") == EDIT_TYPE.MODIFIED
|
||||
@ -117,3 +155,18 @@ class TestCodeCommitProvider:
|
||||
assert CodeCommitProvider._get_edit_type("r") == EDIT_TYPE.RENAMED
|
||||
|
||||
assert CodeCommitProvider._get_edit_type("X") is None
|
||||
|
||||
def test_add_additional_newlines(self):
|
||||
# a short string to test adding double newlines
|
||||
input = "abc\ndef\n\n___\nghi\njkl\nmno\n\npqr\n"
|
||||
expect = "abc\n\ndef\n\n___\n\nghi\n\njkl\n\nmno\n\npqr\n\n"
|
||||
assert CodeCommitProvider._add_additional_newlines(input) == expect
|
||||
# a test example from a real PR
|
||||
input = "## PR Type:\nEnhancement\n\n___\n## PR Description:\nThis PR introduces a new feature to the script, allowing users to filter servers by name.\n\n___\n## PR Main Files Walkthrough:\n`foo`: The foo script has been updated to include a new command line option `-f` or `--filter`.\n`bar`: The bar script has been updated to list stopped servers.\n"
|
||||
expect = "## PR Type:\n\nEnhancement\n\n___\n\n## PR Description:\n\nThis PR introduces a new feature to the script, allowing users to filter servers by name.\n\n___\n\n## PR Main Files Walkthrough:\n\n`foo`: The foo script has been updated to include a new command line option `-f` or `--filter`.\n\n`bar`: The bar script has been updated to list stopped servers.\n\n"
|
||||
assert CodeCommitProvider._add_additional_newlines(input) == expect
|
||||
|
||||
def test_remove_markdown_html(self):
|
||||
input = "## PR Feedback\n<details><summary>Code feedback:</summary>\nfile foo\n</summary>\n"
|
||||
expect = "## PR Feedback\nCode feedback:\nfile foo\n\n"
|
||||
assert CodeCommitProvider._remove_markdown_html(input) == expect
|
@ -1,6 +1,8 @@
|
||||
|
||||
# Generated by CodiumAI
|
||||
from pr_agent.algo.utils import FilePatchInfo, find_line_number_of_relevant_line_in_file
|
||||
from pr_agent.git_providers.git_provider import FilePatchInfo
|
||||
from pr_agent.algo.pr_processing import find_line_number_of_relevant_line_in_file
|
||||
|
||||
|
||||
import pytest
|
||||
|
||||
|
Reference in New Issue
Block a user