Compare commits
117 Commits
enhancemen
...
ok/update_
Author | SHA1 | Date | |
---|---|---|---|
3b27c834a4 | |||
5bc2ef1eff | |||
2f558006bf | |||
370520df51 | |||
2e832b8fb4 | |||
a47fa342cb | |||
bdf7eff7cd | |||
dc67e6a66e | |||
77f243b7ab | |||
c507785475 | |||
5c5015b267 | |||
3efe08d619 | |||
2e36fce4eb | |||
d6d4427545 | |||
5d45632247 | |||
90c045e3d0 | |||
7f0a96d8f7 | |||
8fb9affef3 | |||
6c42a471e1 | |||
f2b74b6970 | |||
ffd11aeffc | |||
05e4e09dfc | |||
13092118dc | |||
7d108992fc | |||
e5a8ed205e | |||
90f97b0226 | |||
9e0f5f0ccc | |||
87ea0176b9 | |||
62f08f4ec4 | |||
fe0058f25f | |||
6d2673f39d | |||
b3a1d456b2 | |||
f77a5f6929 | |||
fdeae9c209 | |||
a994ec1427 | |||
e5259e2f5c | |||
978348240b | |||
4d92e7d9c2 | |||
6f1b418b25 | |||
51e08c3c2b | |||
4c29ff2db1 | |||
5fbaa4366f | |||
aee08ebbfe | |||
6ad8df6be7 | |||
539edcad3c | |||
b7172df700 | |||
768bd40ad8 | |||
ea27c63f13 | |||
c866288b0a | |||
8ae3c60670 | |||
f8f415eb75 | |||
24583b05f7 | |||
fa421fd169 | |||
e0ae5c945e | |||
865888e4e8 | |||
3b7cfe7bc5 | |||
262f9dddbc | |||
fa706b6e96 | |||
ff51ab0946 | |||
7884aa2348 | |||
8f3520807c | |||
fa90b242e3 | |||
2dfd34bd61 | |||
48f569bef0 | |||
a20fb9cc0c | |||
c58e1f90e7 | |||
d363f148f0 | |||
cbf96a2e67 | |||
4d87c3ec6a | |||
c13c52d733 | |||
dbf8142fe0 | |||
bacf6c96c2 | |||
c9d49da8f7 | |||
7b22edac60 | |||
fc309f69b9 | |||
7efb5cf74e | |||
8e200197c5 | |||
fe98f67e08 | |||
0b1edd9716 | |||
e638dc075c | |||
559b160886 | |||
571b8769ac | |||
e4bd2148ce | |||
1637bd8774 | |||
ce33582d3d | |||
bc6b592fd9 | |||
24ae6b966f | |||
f4de3d2899 | |||
4cacb07ec2 | |||
2371a9b041 | |||
5b7403ae80 | |||
e979b8643d | |||
05b4f167a3 | |||
2c4245e023 | |||
d54ee252ee | |||
85eec0b98c | |||
41a988d99a | |||
448da3d481 | |||
b030299547 | |||
5bdbfda1e2 | |||
047cfb21f3 | |||
35a2497a38 | |||
99630f83c2 | |||
1757f2707c | |||
66c44d715c | |||
8f7855013a | |||
e200be4e57 | |||
d0b734bc91 | |||
399d5c5c5d | |||
1b88049cb0 | |||
0304bf05c1 | |||
94173cbb06 | |||
75447280e4 | |||
5edff8b7e4 | |||
487351d343 | |||
0a4a604c28 | |||
973cb2de1c |
@ -1,2 +1,3 @@
|
||||
venv/
|
||||
pr_agent/settings/.secrets.toml
|
||||
pr_agent/settings/.secrets.toml
|
||||
pics/
|
2
.github/workflows/review.yaml
vendored
@ -11,6 +11,6 @@ jobs:
|
||||
uses: Codium-ai/pr-agent@main
|
||||
env:
|
||||
OPENAI_KEY: ${{ secrets.OPENAI_KEY }}
|
||||
OPENAI_ORG: ${{ secrets.OPENAI_ORG }}
|
||||
OPENAI_ORG: ${{ secrets.OPENAI_ORG }} # optional
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
|
18
CONFIGURATION.md
Normal file
@ -0,0 +1,18 @@
|
||||
## Configuration
|
||||
|
||||
The different tools and sub-tools used by CodiumAI pr-agent are easily configurable via the configuration file: `/pr-agent/settings/configuration.toml`.
|
||||
##### Git Provider:
|
||||
You can select your git_provider with the flag `git_provider` in the `config` section
|
||||
|
||||
##### PR Reviewer:
|
||||
|
||||
You can enable/disable the different PR Reviewer abilities with the following flags (`pr_reviewer` section):
|
||||
```
|
||||
require_focused_review=true
|
||||
require_tests_review=true
|
||||
require_security_review=true
|
||||
```
|
||||
You can contol the number of suggestions returned by the PR Reviewer with the following flag:
|
||||
```inline_code_comments=3```
|
||||
And enable/disable the inline code suggestions with the following flag:
|
||||
```inline_code_comments=true```
|
217
INSTALL.md
Normal file
@ -0,0 +1,217 @@
|
||||
|
||||
## Installation
|
||||
|
||||
---
|
||||
|
||||
#### Method 1: Use Docker image (no installation required)
|
||||
|
||||
To request a review for a PR, or ask a question about a PR, you can run directly from the Docker image. Here's how:
|
||||
|
||||
1. To request a review for a PR, run the following command:
|
||||
|
||||
```
|
||||
docker run --rm -it -e OPENAI.KEY=<your key> -e GITHUB.USER_TOKEN=<your token> codiumai/pr-agent --pr_url <pr_url> review
|
||||
```
|
||||
|
||||
2. To ask a question about a PR, run the following command:
|
||||
|
||||
```
|
||||
docker run --rm -it -e OPENAI.KEY=<your key> -e GITHUB.USER_TOKEN=<your token> codiumai/pr-agent --pr_url <pr_url> ask "<your question>"
|
||||
```
|
||||
|
||||
Possible questions you can ask include:
|
||||
|
||||
- What is the main theme of this PR?
|
||||
- Is the PR ready for merge?
|
||||
- What are the main changes in this PR?
|
||||
- Should this PR be split into smaller parts?
|
||||
- Can you compose a rhymed song about this PR?
|
||||
|
||||
---
|
||||
|
||||
#### Method 2: Run as a GitHub Action
|
||||
|
||||
You can use our pre-built Github Action Docker image to run PR-Agent as a Github Action.
|
||||
|
||||
1. Add the following file to your repository under `.github/workflows/pr_agent.yml`:
|
||||
|
||||
```yaml
|
||||
on:
|
||||
pull_request:
|
||||
issue_comment:
|
||||
jobs:
|
||||
pr_agent_job:
|
||||
runs-on: ubuntu-latest
|
||||
name: Run pr agent on every pull request, respond to user comments
|
||||
steps:
|
||||
- name: PR Agent action step
|
||||
id: pragent
|
||||
uses: Codium-ai/pr-agent@main
|
||||
env:
|
||||
OPENAI_KEY: ${{ secrets.OPENAI_KEY }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
```
|
||||
|
||||
2. Add the following secret to your repository under `Settings > Secrets`:
|
||||
|
||||
```
|
||||
OPENAI_KEY: <your key>
|
||||
```
|
||||
|
||||
The GITHUB_TOKEN secret is automatically created by GitHub.
|
||||
|
||||
3. Merge this change to your main branch.
|
||||
When you open your next PR, you should see a comment from `github-actions` bot with a review of your PR, and instructions on how to use the rest of the tools.
|
||||
|
||||
4. You may configure PR-Agent by adding environment variables under the env section corresponding to any configurable property in the [configuration](./CONFIGURATION.md) file. Some examples:
|
||||
```yaml
|
||||
env:
|
||||
# ... previous environment values
|
||||
OPENAI.ORG: "<Your organization name under your OpenAI account>"
|
||||
PR_REVIEWER.REQUIRE_TESTS_REVIEW: "false" # Disable tests review
|
||||
PR_CODE_SUGGESTIONS.NUM_CODE_SUGGESTIONS: 6 # Increase number of code suggestions
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Method 3: Run from source
|
||||
|
||||
1. Clone this repository:
|
||||
|
||||
```
|
||||
git clone https://github.com/Codium-ai/pr-agent.git
|
||||
```
|
||||
|
||||
2. Install the requirements in your favorite virtual environment:
|
||||
|
||||
```
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
3. Copy the secrets template file and fill in your OpenAI key and your GitHub user token:
|
||||
|
||||
```
|
||||
cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets.toml
|
||||
# Edit .secrets.toml file
|
||||
```
|
||||
|
||||
4. Run the appropriate Python scripts from the scripts folder:
|
||||
|
||||
```
|
||||
python pr_agent/cli.py --pr_url <pr_url> review
|
||||
python pr_agent/cli.py --pr_url <pr_url> ask <your question>
|
||||
python pr_agent/cli.py --pr_url <pr_url> describe
|
||||
python pr_agent/cli.py --pr_url <pr_url> improve
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Method 4: Run as a polling server
|
||||
Request reviews by tagging your Github user on a PR
|
||||
|
||||
Follow steps 1-3 of method 2.
|
||||
Run the following command to start the server:
|
||||
|
||||
```
|
||||
python pr_agent/servers/github_polling.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Method 5: Run as a GitHub App
|
||||
Allowing you to automate the review process on your private or public repositories.
|
||||
|
||||
1. Create a GitHub App from the [Github Developer Portal](https://docs.github.com/en/developers/apps/creating-a-github-app).
|
||||
|
||||
- Set the following permissions:
|
||||
- Pull requests: Read & write
|
||||
- Issue comment: Read & write
|
||||
- Metadata: Read-only
|
||||
- Set the following events:
|
||||
- Issue comment
|
||||
- Pull request
|
||||
|
||||
2. Generate a random secret for your app, and save it for later. For example, you can use:
|
||||
|
||||
```
|
||||
WEBHOOK_SECRET=$(python -c "import secrets; print(secrets.token_hex(10))")
|
||||
```
|
||||
|
||||
3. Acquire the following pieces of information from your app's settings page:
|
||||
|
||||
- App private key (click "Generate a private key" and save the file)
|
||||
- App ID
|
||||
|
||||
4. Clone this repository:
|
||||
|
||||
```
|
||||
git clone https://github.com/Codium-ai/pr-agent.git
|
||||
```
|
||||
|
||||
5. Copy the secrets template file and fill in the following:
|
||||
```
|
||||
cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets.toml
|
||||
# Edit .secrets.toml file
|
||||
```
|
||||
- Your OpenAI key.
|
||||
- Copy your app's private key to the private_key field.
|
||||
- Copy your app's ID to the app_id field.
|
||||
- Copy your app's webhook secret to the webhook_secret field.
|
||||
- Set deployment_type to 'app' in [configuration.toml](./pr_agent/settings/configuration.toml)
|
||||
|
||||
> The .secrets.toml file is not copied to the Docker image by default, and is only used for local development.
|
||||
> If you want to use the .secrets.toml file in your Docker image, you can add remove it from the .dockerignore file.
|
||||
> In most production environments, you would inject the secrets file as environment variables or as mounted volumes.
|
||||
> For example, in order to inject a secrets file as a volume in a Kubernetes environment you can update your pod spec to include the following,
|
||||
> assuming you have a secret named `pr-agent-settings` with a key named `.secrets.toml`:
|
||||
```
|
||||
volumes:
|
||||
- name: settings-volume
|
||||
secret:
|
||||
secretName: pr-agent-settings
|
||||
// ...
|
||||
containers:
|
||||
// ...
|
||||
volumeMounts:
|
||||
- mountPath: /app/pr_agent/settings_prod
|
||||
name: settings-volume
|
||||
```
|
||||
|
||||
> Another option is to set the secrets as environment variables in your deployment environment, for example `OPENAI.KEY` and `GITHUB.USER_TOKEN`.
|
||||
|
||||
6. Build a Docker image for the app and optionally push it to a Docker repository. We'll use Dockerhub as an example:
|
||||
|
||||
```
|
||||
docker build . -t codiumai/pr-agent:github_app --target github_app -f docker/Dockerfile
|
||||
docker push codiumai/pr-agent:github_app # Push to your Docker repository
|
||||
```
|
||||
|
||||
7. Host the app using a server, serverless function, or container environment. Alternatively, for development and
|
||||
debugging, you may use tools like smee.io to forward webhooks to your local machine.
|
||||
You can check [Deploy as a Lambda Function](#deploy-as-a-lambda-function)
|
||||
|
||||
8. Go back to your app's settings, and set the following:
|
||||
|
||||
- Webhook URL: The URL of your app's server or the URL of the smee.io channel.
|
||||
- Webhook secret: The secret you generated earlier.
|
||||
|
||||
9. Install the app by navigating to the "Install App" tab and selecting your desired repositories.
|
||||
|
||||
---
|
||||
|
||||
#### Deploy as a Lambda Function
|
||||
|
||||
1. Follow steps 1-5 of [Method 5](#method-5-run-as-a-github-app).
|
||||
2. Build a docker image that can be used as a lambda function
|
||||
```shell
|
||||
docker buildx build --platform=linux/amd64 . -t codiumai/pr-agent:serverless -f docker/Dockerfile.lambda
|
||||
```
|
||||
3. Push image to ECR
|
||||
```shell
|
||||
docker tag codiumai/pr-agent:serverless <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:serverless
|
||||
docker push <AWS_ACCOUNT>.dkr.ecr.<AWS_REGION>.amazonaws.com/codiumai/pr-agent:serverless
|
||||
```
|
||||
4. Create a lambda function that uses the uploaded image. Set the lambda timeout to be at least 3m.
|
||||
5. Configure the lambda function to have a Function URL.
|
||||
6. Go back to steps 8-9 of [Method 5](#method-5-run-as-a-github-app) with the function url as your Webhook URL.
|
||||
The Webhook URL would look like `https://<LAMBDA_FUNCTION_URL>/api/v1/github_webhooks`
|
@ -39,4 +39,4 @@ We use [tiktoken](https://github.com/openai/tiktoken) to tokenize the patches af
|
||||
4. If we haven't reached the max token length, add the `deleted files` to the prompt until the prompt reaches the max token length (hard stop), skip the rest of the patches.
|
||||
|
||||
### Example
|
||||

|
||||

|
306
README.md
@ -10,26 +10,42 @@
|
||||
[](https://github.com/Codium-ai/pr-agent/blob/main/LICENSE)
|
||||
[](https://discord.com/channels/1057273017547378788/1126104260430528613)
|
||||
</div>
|
||||
<div align="left">
|
||||
<div style="text-align:left;">
|
||||
|
||||
CodiumAI `pr-agent` is an open-source tool aiming to help developers review PRs faster and more efficiently. It automatically analyzes the PR and can provide several types of feedback:
|
||||
CodiumAI `PR-Agent` is an open-source tool aiming to help developers review PRs faster and more efficiently. It automatically analyzes the PR and can provide several types of feedback:
|
||||
|
||||
**Auto-Description**: Automatically generating PR description - name, type, summary, and code walkthrough.
|
||||
\
|
||||
**PR Review**: Feedback about the PR main theme, type, relevant tests, security issues, focused, and various suggestions for the PR content.
|
||||
**PR Review**: Feedback about the PR main theme, type, relevant tests, security issues, focused PR, and various suggestions for the PR content.
|
||||
\
|
||||
**Question Answering**: Answering free-text questions about the PR.
|
||||
\
|
||||
**Code Suggestion**: Committable code suggestions for improving the PR.
|
||||
|
||||
Example results:
|
||||
<h3>Example results:</h2>
|
||||
</div>
|
||||
<h4>/describe:</h4>
|
||||
<div align="center">
|
||||
<p float="center">
|
||||
<img src="./pics/pr_reviewer_1.png" width="800">
|
||||
<img src="https://codium.ai/images/describe.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
<h4>/review:</h4>
|
||||
<div align="center">
|
||||
<p float="center">
|
||||
<img src="./pics/pr_code_suggestions.png" width="800">
|
||||
<img src="https://codium.ai/images/review.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
<h4>/ask:</h4>
|
||||
<div align="center">
|
||||
<p float="center">
|
||||
<img src="https://codium.ai/images/ask.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
<h4>/improve:</h4>
|
||||
<div align="center">
|
||||
<p float="center">
|
||||
<img src="https://codium.ai/images/improve.gif" width="800">
|
||||
</p>
|
||||
</div>
|
||||
<div align="left">
|
||||
@ -38,7 +54,7 @@ Example results:
|
||||
- [Overview](#overview)
|
||||
- [Quickstart](#quickstart)
|
||||
- [Usage and tools](#usage-and-tools)
|
||||
- [Configuration](#configuration)
|
||||
- [Configuration](./CONFIGURATION.md)
|
||||
- [How it works](#how-it-works)
|
||||
- [Roadmap](#roadmap)
|
||||
- [Similar projects](#similar-projects)
|
||||
@ -46,265 +62,91 @@ Example results:
|
||||
|
||||
## Live demo
|
||||
|
||||
Experience GPT-4 powered PR review on your public GitHub repository with our hosted pr-agent. To try it, just mention `@CodiumAI-Agent` in any PR comment! The agent will generate a PR review in response.
|
||||
Experience GPT-4 powered PR review on your public GitHub repository with our hosted PR-Agent. To try it, just mention `@CodiumAI-Agent` and add the desired command in any PR comment! The agent will generate a response based on your command.
|
||||
|
||||

|
||||

|
||||
|
||||
To set up your own pr-agent, see the [Quickstart](#Quickstart) section
|
||||
To set up your own PR-Agent, see the [Quickstart](#Quickstart) section
|
||||
|
||||
---
|
||||
## Overview
|
||||
`pr-agent` offers extensive pull request functionalities across various git providers:
|
||||
| | | Github | Gitlab | Bitbucket |
|
||||
|-------|---------------------------------------------|--------|--------|-----------|
|
||||
| TOOLS | Review | ✓ | ✓ | ✓ |
|
||||
| | ⮑ Inline review | ✓ | ✓ | |
|
||||
| | Ask | ✓ | ✓ | |
|
||||
| | Auto-Description | ✓ | | |
|
||||
| | Improve Code | ✓ | | |
|
||||
`PR-Agent` offers extensive pull request functionalities across various git providers:
|
||||
| | | GitHub | Gitlab | Bitbucket |
|
||||
|-------|---------------------------------------------|:------:|:------:|:---------:|
|
||||
| TOOLS | Review | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | ⮑ Inline review | :white_check_mark: | :white_check_mark: | |
|
||||
| | Ask | :white_check_mark: | :white_check_mark: | |
|
||||
| | Auto-Description | :white_check_mark: | :white_check_mark: | |
|
||||
| | Improve Code | :white_check_mark: | :white_check_mark: | |
|
||||
| | Reflect and Review | :white_check_mark: | | |
|
||||
| | | | | |
|
||||
| USAGE | CLI | ✓ | ✓ | ✓ |
|
||||
| | Tagging bot | ✓ | ✓ | |
|
||||
| | Actions | ✓ | | |
|
||||
| USAGE | CLI | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Tagging bot | :white_check_mark: | :white_check_mark: | |
|
||||
| | Actions | :white_check_mark: | | |
|
||||
| | | | | |
|
||||
| CORE | PR compression | ✓ | ✓ | ✓ |
|
||||
| | Repo language prioritization | ✓ | ✓ | ✓ |
|
||||
| | Adaptive and token-aware<br />file patch fitting | ✓ | ✓ | ✓ |
|
||||
| CORE | PR compression | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Repo language prioritization | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| | Adaptive and token-aware<br />file patch fitting | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
|
||||
Examples for invoking the different tools via the [CLI](#quickstart):
|
||||
- **Review**: python cli.py --pr-url=<pr_url> review
|
||||
- **Describe**: python cli.py --pr-url=<pr_url> describe
|
||||
- **Improve**: python cli.py --pr-url=<pr_url> improve
|
||||
- **Ask**: python cli.py --pr-url=<pr_url> ask "Write me a poem about this PR"
|
||||
- **Reflect**: python cli.py --pr-url=<pr_url> reflect
|
||||
|
||||
"<pr_url>" is the url of the relevant PR (for example: https://github.com/Codium-ai/pr-agent/pull/50).
|
||||
|
||||
In the [configuration](./CONFIGURATION.md) file you can select your git provider (GitHub, Gitlab, Bitbucket), and further configure the different tools.
|
||||
|
||||
## Quickstart
|
||||
|
||||
To get started with pr-agent quickly, you first need to acquire two tokens:
|
||||
To get started with PR-Agent quickly, you first need to acquire two tokens:
|
||||
|
||||
1. An OpenAI key from [here](https://platform.openai.com/), with access to GPT-4.
|
||||
2. A GitHub personal access token (classic) with the repo scope.
|
||||
|
||||
There are several ways to use pr-agent. Let's start with the simplest one:
|
||||
There are several ways to use PR-Agent. Let's start with the simplest one:
|
||||
|
||||
---
|
||||
|
||||
#### Method 1: Use Docker image (no installation required)
|
||||
## Install
|
||||
Here are several ways to install and run PR-Agent:
|
||||
|
||||
To request a review for a PR, or ask a question about a PR, you can run directly from the Docker image. Here's how:
|
||||
|
||||
1. To request a review for a PR, run the following command:
|
||||
|
||||
```
|
||||
docker run --rm -it -e OPENAI.KEY=<your key> -e GITHUB.USER_TOKEN=<your token> codiumai/pr-agent --pr_url <pr url>
|
||||
```
|
||||
|
||||
2. To ask a question about a PR, run the following command:
|
||||
|
||||
```
|
||||
docker run --rm -it -e OPENAI.KEY=<your key> -e GITHUB.USER_TOKEN=<your token> codiumai/pr-agent --pr_url <pr url> --question "<your question>"
|
||||
```
|
||||
|
||||
Possible questions you can ask include:
|
||||
|
||||
- What is the main theme of this PR?
|
||||
- Is the PR ready for merge?
|
||||
- What are the main changes in this PR?
|
||||
- Should this PR be split into smaller parts?
|
||||
- Can you compose a rhymed song about this PR.
|
||||
|
||||
---
|
||||
|
||||
#### Method 2: Run from source
|
||||
|
||||
1. Clone this repository:
|
||||
|
||||
```
|
||||
git clone https://github.com/Codium-ai/pr-agent.git
|
||||
```
|
||||
|
||||
2. Install the requirements in your favorite virtual environment:
|
||||
|
||||
```
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
3. Copy the secrets template file and fill in your OpenAI key and your GitHub user token:
|
||||
|
||||
```
|
||||
cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets.toml
|
||||
# Edit .secrets.toml file
|
||||
```
|
||||
|
||||
4. Run the appropriate Python scripts from the scripts folder:
|
||||
|
||||
```
|
||||
python pr_agent/cli.py --pr_url <pr url>
|
||||
python pr_agent/cli.py --pr_url <pr url> --question "<your question>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Method 3: Method 3: Run as a polling server; request reviews by tagging your Github user on a PR
|
||||
|
||||
Follow steps 1-3 of method 2.
|
||||
Run the following command to start the server:
|
||||
|
||||
```
|
||||
python pr_agent/servers/github_polling.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Method 4: Run as a Github App, allowing you to automate the review process on your private or public repositories.
|
||||
|
||||
1. Create a GitHub App from the [Github Developer Portal](https://docs.github.com/en/developers/apps/creating-a-github-app).
|
||||
|
||||
- Set the following permissions:
|
||||
- Pull requests: Read & write
|
||||
- Issue comment: Read & write
|
||||
- Metadata: Read-only
|
||||
- Set the following events:
|
||||
- Issue comment
|
||||
- Pull request
|
||||
|
||||
2. Generate a random secret for your app, and save it for later. For example, you can use:
|
||||
|
||||
```
|
||||
WEBHOOK_SECRET=$(python -c "import secrets; print(secrets.token_hex(10))")
|
||||
```
|
||||
|
||||
3. Acquire the following pieces of information from your app's settings page:
|
||||
|
||||
- App private key (click "Generate a private key", and save the file)
|
||||
- App ID
|
||||
|
||||
4. Clone this repository:
|
||||
|
||||
```
|
||||
git clone https://github.com/Codium-ai/pr-agent.git
|
||||
```
|
||||
|
||||
5. Copy the secrets template file and fill in the following:
|
||||
- Your OpenAI key.
|
||||
- Set deployment_type to 'app'
|
||||
- Copy your app's private key to the private_key field.
|
||||
- Copy your app's ID to the app_id field.
|
||||
- Copy your app's webhook secret to the webhook_secret field.
|
||||
|
||||
```
|
||||
cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets.toml
|
||||
# Edit .secrets.toml file
|
||||
```
|
||||
|
||||
6. Build a Docker image for the app and optionally push it to a Docker repository. We'll use Dockerhub as an example:
|
||||
|
||||
```
|
||||
docker build . -t codiumai/pr-agent:github_app --target github_app -f docker/Dockerfile
|
||||
docker push codiumai/pr-agent:github_app # Push to your Docker repository
|
||||
```
|
||||
|
||||
7. Host the app using a server, serverless function, or container environment. Alternatively, for development and
|
||||
debugging, you may use tools like smee.io to forward webhooks to your local machine.
|
||||
|
||||
8. Go back to your app's settings, set the following:
|
||||
|
||||
- Webhook URL: The URL of your app's server, or the URL of the smee.io channel.
|
||||
- Webhook secret: The secret you generated earlier.
|
||||
|
||||
9. Install the app by navigating to the "Install App" tab, and selecting your desired repositories.
|
||||
|
||||
---
|
||||
- [Method 1: Use Docker image (no installation required)](INSTALL.md#method-1-use-docker-image-no-installation-required)
|
||||
- [Method 2: Run as a GitHub Action](INSTALL.md#method-2-run-as-a-github-action)
|
||||
- [Method 3: Run from source](INSTALL.md#method-3-run-from-source)
|
||||
- [Method 4: Run as a polling server](INSTALL.md#method-4-run-as-a-polling-server)
|
||||
- Request reviews by tagging your GitHub user on a PR
|
||||
- [Method 5: Run as a GitHub App](INSTALL.md#method-5-run-as-a-github-app)
|
||||
- Allowing you to automate the review process on your private or public repositories
|
||||
|
||||
## Usage and Tools
|
||||
|
||||
CodiumAI pr-agent provides two types of interactions ("tools"): `"PR Reviewer"` and `"PR Q&A"`.
|
||||
**PR-Agent** provides five types of interactions ("tools"): `"PR Reviewer"`, `"PR Q&A"`, `"PR Description"`, `"PR Code Sueggestions"` and `"PR Reflect and Review"`.
|
||||
|
||||
- The "PR Reviewer" tool automatically analyzes PRs, and provides different types of feedbacks.
|
||||
- The "PR Reviewer" tool automatically analyzes PRs, and provides various types of feedback.
|
||||
- The "PR Q&A" tool answers free-text questions about the PR.
|
||||
|
||||
### PR Reviewer
|
||||
|
||||
Here is a quick overview of the different sub-tools of PR Reviewer:
|
||||
|
||||
- PR Analysis
|
||||
- Summarize main theme
|
||||
- PR type classification
|
||||
- Is the PR covered by relevant tests
|
||||
- Is this a focused PR
|
||||
- Are there security concerns
|
||||
- PR Feedback
|
||||
- General PR suggestions
|
||||
- Code suggestions
|
||||
|
||||
This is how a typical output of the PR Reviewer looks like:
|
||||
|
||||
---
|
||||
|
||||
#### PR Analysis
|
||||
|
||||
- 🎯 **Main theme:** Adding language extension handler and token handler
|
||||
- 📌 **Type of PR:** Enhancement
|
||||
- 🧪 **Relevant tests added:** No
|
||||
- ✨ **Focused PR:** Yes, the PR is focused on adding two new handlers for language extension and token counting.
|
||||
- 🔒 **Security concerns:** No, the PR does not introduce possible security concerns or issues.
|
||||
|
||||
#### PR Feedback
|
||||
|
||||
- 💡 **General PR suggestions:** The PR is generally well-structured and the code is clean. However, it would be beneficial to add some tests to ensure the new handlers work as expected. Also, consider adding docstrings to the new functions and classes to improve code readability and maintainability.
|
||||
|
||||
- 🤖 **Code suggestions:**
|
||||
|
||||
- **relevant file:** pr_agent/algo/language_handler.py
|
||||
|
||||
**suggestion content:** Consider using a set instead of a list for 'bad_extensions' as checking membership in a set is faster than in a list. [medium]
|
||||
|
||||
- **relevant file:** pr_agent/algo/language_handler.py
|
||||
|
||||
**suggestion content:** In the 'filter_bad_extensions' function, you are splitting the filename on '.' and taking the last element to get the extension. This might not work as expected if the filename contains multiple '.' characters. Consider using 'os.path.splitext' to get the file extension more reliably. [important]
|
||||
|
||||
---
|
||||
|
||||
### PR Q&A
|
||||
|
||||
This tool answers free-text questions about the PR. This is how a typical output of the PR Q&A looks like:
|
||||
|
||||
**Question**: summarize for me the PR in 4 bullet points
|
||||
|
||||
**Answer**:
|
||||
|
||||
- The PR introduces a new feature to sort files by their main languages. It uses a mapping of programming languages to their file extensions to achieve this.
|
||||
- It also introduces a filter to exclude files with certain extensions, deemed as 'bad extensions', from the sorting process.
|
||||
- The PR modifies the `get_pr_diff` function in `pr_processing.py` to use the new sorting function. It also refactors the code to move the PR pruning logic into a separate function.
|
||||
- A new `TokenHandler` class is introduced in `token_handler.py` to handle token counting operations. This class is initialized with a PR, variables, system, and user, and provides methods to get system and user tokens and to count tokens in a patch.
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
The different tools and sub-tools used by CodiumAI pr-agent are easily configurable via the configuration file: `/settings/configuration.toml`.
|
||||
|
||||
#### Enabling/disabling sub-tools:
|
||||
|
||||
You can enable/disable the different PR Reviewer sub-sections with the following flags:
|
||||
|
||||
```
|
||||
require_focused_review=true
|
||||
require_tests_review=true
|
||||
require_security_review=true
|
||||
```
|
||||
- The "PR Description" tool automatically sets the PR Title and body.
|
||||
- The "PR Code Suggestion" tool provide inline code suggestions for the PR that can be applied and committed.
|
||||
- The "PR Reflect and Review" tool first initiates a dialog with the user and asks them to reflect on the PR, and then provides a review.
|
||||
|
||||
## How it works
|
||||
|
||||

|
||||

|
||||
|
||||
Check out the [PR Compression strategy](./PR_COMPRESSION.md) page for more details on how we convert a code diff to a manageable LLM prompt
|
||||
|
||||
## Roadmap
|
||||
|
||||
- [ ] Support open-source models, as a replacement for openai models. Note that a minimal requirement for each open-source model is to have 8k+ context, and good support for generating json as an output
|
||||
- [ ] Support other Git providers, such as Gitlab and Bitbucket.
|
||||
- [ ] Support open-source models, as a replacement for openai models. (Note - a minimal requirement for each open-source model is to have 8k+ context, and good support for generating json as an output)
|
||||
- [x] Support other Git providers, such as Gitlab and Bitbucket.
|
||||
- [ ] Develop additional logics for handling large PRs, and compressing git patches
|
||||
- [ ] Dedicated tools and sub-tools for specific programming languages (Python, Javascript, Java, C++, etc)
|
||||
- [ ] Add additional context to the prompt. For example, repo (or relevant files) summarization, with tools such a [ctags](https://github.com/universal-ctags/ctags)
|
||||
- [ ] Adding more tools. Possible directions:
|
||||
- [ ] Code Quality
|
||||
- [ ] Coding Style
|
||||
- [x] PR description
|
||||
- [x] Inline code suggestions
|
||||
- [x] Reflect and review
|
||||
- [ ] Enforcing CONTRIBUTING.md guidelines
|
||||
- [ ] Performance (are there any performance issues)
|
||||
- [ ] Documentation (is the PR properly documented)
|
||||
- [ ] Rank the PR importance
|
||||
@ -314,6 +156,6 @@ Check out the [PR Compression strategy](./PR_COMPRESSION.md) page for more detai
|
||||
|
||||
- [CodiumAI - Meaningful tests for busy devs](https://github.com/Codium-ai/codiumai-vscode-release)
|
||||
- [Aider - GPT powered coding in your terminal](https://github.com/paul-gauthier/aider)
|
||||
- [GPT-Engineer](https://github.com/AntonOsika/gpt-engineer)
|
||||
- [openai-pr-reviewer](https://github.com/coderabbitai/openai-pr-reviewer)
|
||||
- [CodeReview BOT](https://github.com/anc95/ChatGPT-CodeReview)
|
||||
- [AI-Maintainer](https://github.com/merwanehamadi/AI-Maintainer)
|
||||
|
12
docker/Dockerfile.lambda
Normal file
@ -0,0 +1,12 @@
|
||||
FROM public.ecr.aws/lambda/python:3.10
|
||||
|
||||
RUN yum update -y && \
|
||||
yum install -y gcc python3-devel && \
|
||||
yum clean all
|
||||
|
||||
ADD requirements.txt .
|
||||
RUN pip install -r requirements.txt && rm requirements.txt
|
||||
RUN pip install mangum==16.0.0
|
||||
COPY pr_agent/ ${LAMBDA_TASK_ROOT}/pr_agent/
|
||||
|
||||
CMD ["pr_agent.servers.serverless.serverless"]
|
BIN
pics/.DS_Store
vendored
Before Width: | Height: | Size: 346 KiB |
Before Width: | Height: | Size: 19 KiB After Width: | Height: | Size: 20 KiB |
Before Width: | Height: | Size: 20 KiB After Width: | Height: | Size: 21 KiB |
Before Width: | Height: | Size: 260 KiB |
Before Width: | Height: | Size: 1.4 MiB |
Before Width: | Height: | Size: 413 KiB |
Before Width: | Height: | Size: 335 KiB |
Before Width: | Height: | Size: 193 KiB |
Before Width: | Height: | Size: 161 KiB |
Before Width: | Height: | Size: 185 KiB |
Before Width: | Height: | Size: 162 KiB |
@ -1,7 +1,9 @@
|
||||
import re
|
||||
|
||||
from pr_agent.config_loader import settings
|
||||
from pr_agent.tools.pr_code_suggestions import PRCodeSuggestions
|
||||
from pr_agent.tools.pr_description import PRDescription
|
||||
from pr_agent.tools.pr_information_from_user import PRInformationFromUser
|
||||
from pr_agent.tools.pr_questions import PRQuestions
|
||||
from pr_agent.tools.pr_reviewer import PRReviewer
|
||||
|
||||
@ -11,8 +13,13 @@ class PRAgent:
|
||||
pass
|
||||
|
||||
async def handle_request(self, pr_url, request) -> bool:
|
||||
if any(cmd in request for cmd in ["/review", "/review_pr"]):
|
||||
await PRReviewer(pr_url).review()
|
||||
if any(cmd in request for cmd in ["/answer"]):
|
||||
await PRReviewer(pr_url, is_answer=True).review()
|
||||
elif any(cmd in request for cmd in ["/review", "/review_pr", "/reflect_and_review"]):
|
||||
if settings.pr_reviewer.ask_and_reflect or "/reflect_and_review" in request:
|
||||
await PRInformationFromUser(pr_url).generate_questions()
|
||||
else:
|
||||
await PRReviewer(pr_url).review()
|
||||
elif any(cmd in request for cmd in ["/describe", "/describe_pr"]):
|
||||
await PRDescription(pr_url).describe()
|
||||
elif any(cmd in request for cmd in ["/improve", "/improve_code"]):
|
||||
|
@ -158,7 +158,7 @@ def convert_to_hunks_with_lines_numbers(patch: str, file) -> str:
|
||||
patch_with_lines_str += f"{start2 + i} {line_new}\n"
|
||||
if old_content_lines:
|
||||
patch_with_lines_str += '--old hunk--\n'
|
||||
for i, line_old in enumerate(old_content_lines):
|
||||
for line_old in old_content_lines:
|
||||
patch_with_lines_str += f"{line_old}\n"
|
||||
new_content_lines = []
|
||||
old_content_lines = []
|
||||
@ -179,7 +179,7 @@ def convert_to_hunks_with_lines_numbers(patch: str, file) -> str:
|
||||
patch_with_lines_str += f"{start2 + i} {line_new}\n"
|
||||
if old_content_lines:
|
||||
patch_with_lines_str += '\n--old hunk--\n'
|
||||
for i, line_old in enumerate(old_content_lines):
|
||||
for line_old in old_content_lines:
|
||||
patch_with_lines_str += f"{line_old}\n"
|
||||
|
||||
return patch_with_lines_str.strip()
|
||||
|
@ -64,7 +64,11 @@ bad_extensions = [
|
||||
|
||||
|
||||
def filter_bad_extensions(files):
|
||||
return [f for f in files if f.filename.split('.')[-1] not in bad_extensions]
|
||||
return [f for f in files if is_valid_file(f.filename)]
|
||||
|
||||
|
||||
def is_valid_file(filename):
|
||||
return filename.split('.')[-1] not in bad_extensions
|
||||
|
||||
|
||||
def sort_files_by_main_languages(languages: Dict, files: list):
|
||||
|
@ -4,8 +4,7 @@ import difflib
|
||||
import logging
|
||||
from typing import Any, Tuple, Union
|
||||
|
||||
from pr_agent.algo.git_patch_processing import extend_patch, handle_patch_deletions, \
|
||||
convert_to_hunks_with_lines_numbers
|
||||
from pr_agent.algo.git_patch_processing import convert_to_hunks_with_lines_numbers, extend_patch, handle_patch_deletions
|
||||
from pr_agent.algo.language_handler import sort_files_by_main_languages
|
||||
from pr_agent.algo.token_handler import TokenHandler
|
||||
from pr_agent.config_loader import settings
|
||||
@ -30,10 +29,10 @@ def get_pr_diff(git_provider: Union[GithubProvider, Any], token_handler: TokenHa
|
||||
global PATCH_EXTRA_LINES
|
||||
PATCH_EXTRA_LINES = 0
|
||||
|
||||
git_provider.pr.diff_files = list(git_provider.get_diff_files())
|
||||
diff_files = list(git_provider.get_diff_files())
|
||||
|
||||
# get pr languages
|
||||
pr_languages = sort_files_by_main_languages(git_provider.get_languages(), git_provider.pr.diff_files)
|
||||
pr_languages = sort_files_by_main_languages(git_provider.get_languages(), diff_files)
|
||||
|
||||
# generate a standard diff string, with patch extension
|
||||
patches_extended, total_tokens = pr_generate_extended_diff(pr_languages, token_handler,
|
||||
|
@ -17,6 +17,7 @@ def convert_to_markdown(output_data: dict) -> str:
|
||||
"Focused PR": "✨",
|
||||
"Security concerns": "🔒",
|
||||
"General PR suggestions": "💡",
|
||||
"Insights from user's answers": "📝",
|
||||
"Code suggestions": "🤖"
|
||||
}
|
||||
|
||||
@ -61,18 +62,24 @@ def parse_code_suggestion(code_suggestions: dict) -> str:
|
||||
return markdown_text
|
||||
|
||||
|
||||
def try_fix_json(review, max_iter=10):
|
||||
def try_fix_json(review, max_iter=10, code_suggestions=False):
|
||||
if review.endswith("}"):
|
||||
return fix_json_escape_char(review)
|
||||
# Try to fix JSON if it is broken/incomplete: parse until the last valid code suggestion
|
||||
data = {}
|
||||
if code_suggestions:
|
||||
closing_bracket = "]}"
|
||||
else:
|
||||
closing_bracket = "]}}"
|
||||
if review.rfind("'Code suggestions': [") > 0 or review.rfind('"Code suggestions": [') > 0:
|
||||
last_code_suggestion_ind = [m.end() for m in re.finditer(r"\}\s*,", review)][-1] - 1
|
||||
valid_json = False
|
||||
iter_count = 0
|
||||
while last_code_suggestion_ind > 0 and not valid_json and iter_count < max_iter:
|
||||
try:
|
||||
data = json.loads(review[:last_code_suggestion_ind] + "]}}")
|
||||
data = json.loads(review[:last_code_suggestion_ind] + closing_bracket)
|
||||
valid_json = True
|
||||
review = review[:last_code_suggestion_ind].strip() + "]}}"
|
||||
review = review[:last_code_suggestion_ind].strip() + closing_bracket
|
||||
except json.decoder.JSONDecodeError:
|
||||
review = review[:last_code_suggestion_ind]
|
||||
# Use regular expression to find the last occurrence of "}," with any number of whitespaces or newlines
|
||||
@ -82,3 +89,17 @@ def try_fix_json(review, max_iter=10):
|
||||
logging.error("Unable to decode JSON response from AI")
|
||||
data = {}
|
||||
return data
|
||||
|
||||
|
||||
def fix_json_escape_char(json_message=None):
|
||||
try:
|
||||
result = json.loads(json_message)
|
||||
except Exception as e:
|
||||
# Find the offending character index:
|
||||
idx_to_replace = int(str(e).split(' ')[-1].replace(')', ''))
|
||||
# Remove the offending character:
|
||||
json_message = list(json_message)
|
||||
json_message[idx_to_replace] = ' '
|
||||
new_message = ''.join(json_message)
|
||||
return fix_json_escape_char(json_message=new_message)
|
||||
return result
|
||||
|
@ -5,27 +5,37 @@ import os
|
||||
|
||||
from pr_agent.tools.pr_code_suggestions import PRCodeSuggestions
|
||||
from pr_agent.tools.pr_description import PRDescription
|
||||
from pr_agent.tools.pr_information_from_user import PRInformationFromUser
|
||||
from pr_agent.tools.pr_questions import PRQuestions
|
||||
from pr_agent.tools.pr_reviewer import PRReviewer
|
||||
|
||||
|
||||
def run():
|
||||
def run(args=None):
|
||||
parser = argparse.ArgumentParser(description='AI based pull request analyzer', usage="""\
|
||||
Usage: cli.py --pr-url <URL on supported git hosting service> <command> [<args>].
|
||||
For example:
|
||||
- cli.py --pr-url=... review
|
||||
- cli.py --pr-url=... describe
|
||||
- cli.py --pr-url=... improve
|
||||
- cli.py --pr-url=... ask "write me a poem about this PR"
|
||||
- cli.py --pr-url=... reflect
|
||||
|
||||
Supported commands:
|
||||
review / review_pr - Add a review that includes a summary of the PR and specific suggestions for improvement.
|
||||
ask / ask_question [question] - Ask a question about the PR.
|
||||
describe / describe_pr - Modify the PR title and description based on the PR's contents.
|
||||
improve / improve_code - Suggest improvements to the code in the PR as pull request comments ready to commit.
|
||||
reflect - Ask the PR author questions about the PR.
|
||||
""")
|
||||
parser.add_argument('--pr_url', type=str, help='The URL of the PR to review', required=True)
|
||||
parser.add_argument('command', type=str, help='The', choices=['review', 'review_pr',
|
||||
'ask', 'ask_question',
|
||||
'describe', 'describe_pr',
|
||||
'improve', 'improve_code'], default='review')
|
||||
'improve', 'improve_code',
|
||||
'reflect', 'review_after_reflect'],
|
||||
default='review')
|
||||
parser.add_argument('rest', nargs=argparse.REMAINDER, default=[])
|
||||
args = parser.parse_args()
|
||||
args = parser.parse_args(args)
|
||||
logging.basicConfig(level=os.environ.get("LOGLEVEL", "INFO"))
|
||||
command = args.command.lower()
|
||||
if command in ['ask', 'ask_question']:
|
||||
@ -49,6 +59,14 @@ improve / improve_code - Suggest improvements to the code in the PR as pull requ
|
||||
print(f"Reviewing PR: {args.pr_url}")
|
||||
reviewer = PRReviewer(args.pr_url, cli_mode=True)
|
||||
asyncio.run(reviewer.review())
|
||||
elif command in ['reflect']:
|
||||
print(f"Asking the PR author questions: {args.pr_url}")
|
||||
reviewer = PRInformationFromUser(args.pr_url)
|
||||
asyncio.run(reviewer.generate_questions())
|
||||
elif command in ['review_after_reflect']:
|
||||
print(f"Processing author's answers and sending review: {args.pr_url}")
|
||||
reviewer = PRReviewer(args.pr_url, cli_mode=True, is_answer=True)
|
||||
asyncio.run(reviewer.review())
|
||||
else:
|
||||
print(f"Unknown command: {command}")
|
||||
parser.print_help()
|
||||
|
@ -13,6 +13,7 @@ settings = Dynaconf(
|
||||
"settings/pr_questions_prompts.toml",
|
||||
"settings/pr_description_prompts.toml",
|
||||
"settings/pr_code_suggestions_prompts.toml",
|
||||
"settings/pr_information_from_user_prompts.toml",
|
||||
"settings_prod/.secrets.toml"
|
||||
]]
|
||||
)
|
||||
|
@ -1,10 +1,12 @@
|
||||
from pr_agent.config_loader import settings
|
||||
from pr_agent.git_providers.bitbucket_provider import BitbucketProvider
|
||||
from pr_agent.git_providers.github_provider import GithubProvider
|
||||
from pr_agent.git_providers.gitlab_provider import GitLabProvider
|
||||
|
||||
_GIT_PROVIDERS = {
|
||||
'github': GithubProvider,
|
||||
'gitlab': GitLabProvider,
|
||||
'bitbucket': BitbucketProvider,
|
||||
}
|
||||
|
||||
def get_git_provider():
|
||||
|
122
pr_agent/git_providers/bitbucket_provider.py
Normal file
@ -0,0 +1,122 @@
|
||||
import logging
|
||||
from typing import Optional, Tuple
|
||||
from urllib.parse import urlparse
|
||||
|
||||
import requests
|
||||
from atlassian.bitbucket import Cloud
|
||||
|
||||
from pr_agent.config_loader import settings
|
||||
|
||||
from .git_provider import FilePatchInfo
|
||||
|
||||
|
||||
class BitbucketProvider:
|
||||
def __init__(self, pr_url: Optional[str] = None):
|
||||
s = requests.Session()
|
||||
s.headers['Authorization'] = f'Bearer {settings.get("BITBUCKET.BEARER_TOKEN", None)}'
|
||||
self.bitbucket_client = Cloud(session=s)
|
||||
|
||||
self.workspace_slug = None
|
||||
self.repo_slug = None
|
||||
self.repo = None
|
||||
self.pr_num = None
|
||||
self.pr = None
|
||||
self.temp_comments = []
|
||||
if pr_url:
|
||||
self.set_pr(pr_url)
|
||||
|
||||
def is_supported(self, capability: str) -> bool:
|
||||
if capability in ['get_issue_comments', 'create_inline_comment', 'publish_inline_comments']:
|
||||
return False
|
||||
return True
|
||||
|
||||
def set_pr(self, pr_url: str):
|
||||
self.workspace_slug, self.repo_slug, self.pr_num = self._parse_pr_url(pr_url)
|
||||
self.pr = self._get_pr()
|
||||
|
||||
def get_files(self):
|
||||
return [diff.new.path for diff in self.pr.diffstat()]
|
||||
|
||||
def get_diff_files(self) -> list[FilePatchInfo]:
|
||||
diffs = self.pr.diffstat()
|
||||
diff_split = ['diff --git%s' % x for x in self.pr.diff().split('diff --git') if x.strip()]
|
||||
|
||||
diff_files = []
|
||||
for index, diff in enumerate(diffs):
|
||||
original_file_content_str = self._get_pr_file_content(diff.old.get_data('links'))
|
||||
new_file_content_str = self._get_pr_file_content(diff.new.get_data('links'))
|
||||
diff_files.append(FilePatchInfo(original_file_content_str, new_file_content_str,
|
||||
diff_split[index], diff.new.path))
|
||||
return diff_files
|
||||
|
||||
def publish_comment(self, pr_comment: str, is_temporary: bool = False):
|
||||
comment = self.pr.comment(pr_comment)
|
||||
if is_temporary:
|
||||
self.temp_comments.append(comment['id'])
|
||||
|
||||
def remove_initial_comment(self):
|
||||
try:
|
||||
for comment in self.temp_comments:
|
||||
self.pr.delete(f'comments/{comment}')
|
||||
except Exception as e:
|
||||
logging.exception(f"Failed to remove temp comments, error: {e}")
|
||||
|
||||
def publish_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
|
||||
pass
|
||||
|
||||
def create_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
|
||||
raise NotImplementedError("Bitbucket provider does not support creating inline comments yet")
|
||||
|
||||
def publish_inline_comments(self, comments: list[dict]):
|
||||
raise NotImplementedError("Bitbucket provider does not support publishing inline comments yet")
|
||||
|
||||
def get_title(self):
|
||||
return self.pr.title
|
||||
|
||||
def get_languages(self):
|
||||
languages = {self._get_repo().get_data('language'): 0}
|
||||
return languages
|
||||
|
||||
def get_pr_branch(self):
|
||||
return self.pr.source_branch
|
||||
|
||||
def get_pr_description(self):
|
||||
return self.pr.description
|
||||
|
||||
def get_user_id(self):
|
||||
return 0
|
||||
|
||||
def get_issue_comments(self):
|
||||
raise NotImplementedError("Bitbucket provider does not support issue comments yet")
|
||||
|
||||
@staticmethod
|
||||
def _parse_pr_url(pr_url: str) -> Tuple[str, int]:
|
||||
parsed_url = urlparse(pr_url)
|
||||
|
||||
if 'bitbucket.org' not in parsed_url.netloc:
|
||||
raise ValueError("The provided URL is not a valid GitHub URL")
|
||||
|
||||
path_parts = parsed_url.path.strip('/').split('/')
|
||||
|
||||
if len(path_parts) < 4 or path_parts[2] != 'pull-requests':
|
||||
raise ValueError("The provided URL does not appear to be a Bitbucket PR URL")
|
||||
|
||||
workspace_slug = path_parts[0]
|
||||
repo_slug = path_parts[1]
|
||||
try:
|
||||
pr_number = int(path_parts[3])
|
||||
except ValueError as e:
|
||||
raise ValueError("Unable to convert PR number to integer") from e
|
||||
|
||||
return workspace_slug, repo_slug, pr_number
|
||||
|
||||
def _get_repo(self):
|
||||
if self.repo is None:
|
||||
self.repo = self.bitbucket_client.workspaces.get(self.workspace_slug).repositories.get(self.repo_slug)
|
||||
return self.repo
|
||||
|
||||
def _get_pr(self):
|
||||
return self._get_repo().pullrequests.get(self.pr_num)
|
||||
|
||||
def _get_pr_file_content(self, remote_link: str):
|
||||
return ""
|
@ -3,12 +3,15 @@ from dataclasses import dataclass
|
||||
|
||||
# enum EDIT_TYPE (ADDED, DELETED, MODIFIED, RENAMED)
|
||||
from enum import Enum
|
||||
|
||||
|
||||
class EDIT_TYPE(Enum):
|
||||
ADDED = 1
|
||||
DELETED = 2
|
||||
MODIFIED = 3
|
||||
RENAMED = 4
|
||||
|
||||
|
||||
@dataclass
|
||||
class FilePatchInfo:
|
||||
base_file: str
|
||||
@ -21,6 +24,10 @@ class FilePatchInfo:
|
||||
|
||||
|
||||
class GitProvider(ABC):
|
||||
@abstractmethod
|
||||
def is_supported(self, capability: str) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_diff_files(self) -> list[FilePatchInfo]:
|
||||
pass
|
||||
@ -37,6 +44,14 @@ class GitProvider(ABC):
|
||||
def publish_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def create_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def publish_inline_comments(self, comments: list[dict]):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def publish_code_suggestion(self, body: str, relevant_file: str,
|
||||
relevant_lines_start: int, relevant_lines_end: int):
|
||||
@ -62,6 +77,10 @@ class GitProvider(ABC):
|
||||
def get_pr_description(self):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_issue_comments(self):
|
||||
pass
|
||||
|
||||
|
||||
def get_main_pr_language(languages, files) -> str:
|
||||
"""
|
||||
|
@ -3,11 +3,12 @@ from datetime import datetime
|
||||
from typing import Optional, Tuple
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from github import AppAuthentication, Github
|
||||
from github import AppAuthentication, Github, Auth
|
||||
|
||||
from pr_agent.config_loader import settings
|
||||
|
||||
from .git_provider import FilePatchInfo, GitProvider
|
||||
from ..algo.language_handler import is_valid_file
|
||||
|
||||
|
||||
class GithubProvider(GitProvider):
|
||||
@ -23,6 +24,9 @@ class GithubProvider(GitProvider):
|
||||
self.set_pr(pr_url)
|
||||
self.last_commit_id = list(self.pr.get_commits())[-1]
|
||||
|
||||
def is_supported(self, capability: str) -> bool:
|
||||
return True
|
||||
|
||||
def set_pr(self, pr_url: str):
|
||||
self.repo, self.pr_num = self._parse_pr_url(pr_url)
|
||||
self.pr = self._get_pr()
|
||||
@ -34,9 +38,10 @@ class GithubProvider(GitProvider):
|
||||
files = self.pr.get_files()
|
||||
diff_files = []
|
||||
for file in files:
|
||||
original_file_content_str = self._get_pr_file_content(file, self.pr.base.sha)
|
||||
new_file_content_str = self._get_pr_file_content(file, self.pr.head.sha)
|
||||
diff_files.append(FilePatchInfo(original_file_content_str, new_file_content_str, file.patch, file.filename))
|
||||
if is_valid_file(file.filename):
|
||||
original_file_content_str = self._get_pr_file_content(file, self.pr.base.sha)
|
||||
new_file_content_str = self._get_pr_file_content(file, self.pr.head.sha)
|
||||
diff_files.append(FilePatchInfo(original_file_content_str, new_file_content_str, file.patch, file.filename))
|
||||
self.diff_files = diff_files
|
||||
return diff_files
|
||||
|
||||
@ -54,6 +59,9 @@ class GithubProvider(GitProvider):
|
||||
self.pr.comments_list.append(response)
|
||||
|
||||
def publish_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
|
||||
self.publish_inline_comments([self.create_inline_comment(body, relevant_file, relevant_line_in_file)])
|
||||
|
||||
def create_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
|
||||
self.diff_files = self.diff_files if self.diff_files else self.get_diff_files()
|
||||
position = -1
|
||||
for file in self.diff_files:
|
||||
@ -64,7 +72,7 @@ class GithubProvider(GitProvider):
|
||||
if relevant_line_in_file in line:
|
||||
position = i
|
||||
break
|
||||
elif relevant_line_in_file[0] == '+' and relevant_line_in_file[1:] in line:
|
||||
elif relevant_line_in_file[0] == '+' and relevant_line_in_file[1:].lstrip() in line:
|
||||
# The model often adds a '+' to the beginning of the relevant_line_in_file even if originally
|
||||
# it's a context line
|
||||
position = i
|
||||
@ -72,9 +80,16 @@ class GithubProvider(GitProvider):
|
||||
if position == -1:
|
||||
if settings.config.verbosity_level >= 2:
|
||||
logging.info(f"Could not find position for {relevant_file} {relevant_line_in_file}")
|
||||
subject_type = "FILE"
|
||||
else:
|
||||
path = relevant_file.strip()
|
||||
self.pr.create_review_comment(body=body, commit_id=self.last_commit_id, path=path, position=position)
|
||||
subject_type = "LINE"
|
||||
path = relevant_file.strip()
|
||||
# placeholder for future API support (already supported in single inline comment)
|
||||
# return dict(body=body, path=path, position=position, subject_type=subject_type)
|
||||
return dict(body=body, path=path, position=position) if subject_type == "LINE" else {}
|
||||
|
||||
def publish_inline_comments(self, comments: list[dict]):
|
||||
self.pr.create_review(commit=self.last_commit_id, comments=comments)
|
||||
|
||||
def publish_code_suggestion(self, body: str,
|
||||
relevant_file: str,
|
||||
@ -134,9 +149,6 @@ class GithubProvider(GitProvider):
|
||||
def get_title(self):
|
||||
return self.pr.title
|
||||
|
||||
def get_description(self):
|
||||
return self.pr.body
|
||||
|
||||
def get_languages(self):
|
||||
languages = self._get_repo().get_languages()
|
||||
return languages
|
||||
@ -164,6 +176,9 @@ class GithubProvider(GitProvider):
|
||||
notifications = self.github_client.get_user().get_notifications(since=since)
|
||||
return notifications
|
||||
|
||||
def get_issue_comments(self):
|
||||
return self.pr.get_issue_comments()
|
||||
|
||||
@staticmethod
|
||||
def _parse_pr_url(pr_url: str) -> Tuple[str, int]:
|
||||
parsed_url = urlparse(pr_url)
|
||||
@ -215,7 +230,7 @@ class GithubProvider(GitProvider):
|
||||
raise ValueError(
|
||||
"GitHub token is required when using user deployment. See: "
|
||||
"https://github.com/Codium-ai/pr-agent#method-2-run-from-source") from e
|
||||
return Github(token)
|
||||
return Github(auth=Auth.Token(token))
|
||||
|
||||
def _get_repo(self):
|
||||
return self.github_client.get_repo(self.repo)
|
||||
|
@ -4,10 +4,12 @@ from typing import Optional, Tuple
|
||||
from urllib.parse import urlparse
|
||||
|
||||
import gitlab
|
||||
from gitlab import GitlabGetError
|
||||
|
||||
from pr_agent.config_loader import settings
|
||||
|
||||
from .git_provider import FilePatchInfo, GitProvider, EDIT_TYPE
|
||||
from .git_provider import EDIT_TYPE, FilePatchInfo, GitProvider
|
||||
from ..algo.language_handler import is_valid_file
|
||||
|
||||
|
||||
class GitLabProvider(GitProvider):
|
||||
@ -28,6 +30,13 @@ class GitLabProvider(GitProvider):
|
||||
self.diff_files = None
|
||||
self.temp_comments = []
|
||||
self._set_merge_request(merge_request_url)
|
||||
self.RE_HUNK_HEADER = re.compile(
|
||||
r"^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@[ ]?(.*)")
|
||||
|
||||
def is_supported(self, capability: str) -> bool:
|
||||
if capability in ['get_issue_comments', 'create_inline_comment', 'publish_inline_comments']:
|
||||
return False
|
||||
return True
|
||||
|
||||
@property
|
||||
def pr(self):
|
||||
@ -40,31 +49,39 @@ class GitLabProvider(GitProvider):
|
||||
self.last_diff = self.mr.diffs.list()[-1]
|
||||
|
||||
def _get_pr_file_content(self, file_path: str, branch: str) -> str:
|
||||
return self.gl.projects.get(self.id_project).files.get(file_path, branch).decode()
|
||||
try:
|
||||
return self.gl.projects.get(self.id_project).files.get(file_path, branch).decode()
|
||||
except GitlabGetError:
|
||||
# In case of file creation the method returns GitlabGetError (404 file not found).
|
||||
# In this case we return an empty string for the diff.
|
||||
return ''
|
||||
|
||||
def get_diff_files(self) -> list[FilePatchInfo]:
|
||||
diffs = self.mr.changes()['changes']
|
||||
diff_files = []
|
||||
for diff in diffs:
|
||||
original_file_content_str = self._get_pr_file_content(diff['old_path'], self.mr.target_branch)
|
||||
new_file_content_str = self._get_pr_file_content(diff['new_path'], self.mr.source_branch)
|
||||
edit_type = EDIT_TYPE.MODIFIED
|
||||
if diff['new_file']:
|
||||
edit_type = EDIT_TYPE.ADDED
|
||||
elif diff['deleted_file']:
|
||||
edit_type = EDIT_TYPE.DELETED
|
||||
elif diff['renamed_file']:
|
||||
edit_type = EDIT_TYPE.RENAMED
|
||||
try:
|
||||
original_file_content_str = bytes.decode(original_file_content_str, 'utf-8')
|
||||
new_file_content_str = bytes.decode(new_file_content_str, 'utf-8')
|
||||
except UnicodeDecodeError:
|
||||
logging.warning(
|
||||
f"Cannot decode file {diff['old_path']} or {diff['new_path']} in merge request {self.id_mr}")
|
||||
diff_files.append(
|
||||
FilePatchInfo(original_file_content_str, new_file_content_str, diff['diff'], diff['new_path'],
|
||||
edit_type=edit_type,
|
||||
old_filename=None if diff['old_path'] == diff['new_path'] else diff['old_path']))
|
||||
if is_valid_file(diff['new_path']):
|
||||
original_file_content_str = self._get_pr_file_content(diff['old_path'], self.mr.target_branch)
|
||||
new_file_content_str = self._get_pr_file_content(diff['new_path'], self.mr.source_branch)
|
||||
edit_type = EDIT_TYPE.MODIFIED
|
||||
if diff['new_file']:
|
||||
edit_type = EDIT_TYPE.ADDED
|
||||
elif diff['deleted_file']:
|
||||
edit_type = EDIT_TYPE.DELETED
|
||||
elif diff['renamed_file']:
|
||||
edit_type = EDIT_TYPE.RENAMED
|
||||
try:
|
||||
if isinstance(original_file_content_str, bytes):
|
||||
original_file_content_str = bytes.decode(original_file_content_str, 'utf-8')
|
||||
if isinstance(new_file_content_str, bytes):
|
||||
new_file_content_str = bytes.decode(new_file_content_str, 'utf-8')
|
||||
except UnicodeDecodeError:
|
||||
logging.warning(
|
||||
f"Cannot decode file {diff['old_path']} or {diff['new_path']} in merge request {self.id_mr}")
|
||||
diff_files.append(
|
||||
FilePatchInfo(original_file_content_str, new_file_content_str, diff['diff'], diff['new_path'],
|
||||
edit_type=edit_type,
|
||||
old_filename=None if diff['old_path'] == diff['new_path'] else diff['old_path']))
|
||||
self.diff_files = diff_files
|
||||
return diff_files
|
||||
|
||||
@ -72,8 +89,12 @@ class GitLabProvider(GitProvider):
|
||||
return [change['new_path'] for change in self.mr.changes()['changes']]
|
||||
|
||||
def publish_description(self, pr_title: str, pr_body: str):
|
||||
logging.exception("Not implemented yet")
|
||||
pass
|
||||
try:
|
||||
self.mr.title = pr_title
|
||||
self.mr.description = pr_body
|
||||
self.mr.save()
|
||||
except Exception as e:
|
||||
logging.exception(f"Could not update merge request {self.id_mr} description: {e}")
|
||||
|
||||
def publish_comment(self, mr_comment: str, is_temporary: bool = False):
|
||||
comment = self.mr.notes.create({'body': mr_comment})
|
||||
@ -84,25 +105,32 @@ class GitLabProvider(GitProvider):
|
||||
self.diff_files = self.diff_files if self.diff_files else self.get_diff_files()
|
||||
edit_type, found, source_line_no, target_file, target_line_no = self.search_line(relevant_file,
|
||||
relevant_line_in_file)
|
||||
self.send_inline_comment(body, edit_type, found, relevant_file, relevant_line_in_file, source_line_no,
|
||||
target_file, target_line_no)
|
||||
|
||||
def create_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
|
||||
raise NotImplementedError("Gitlab provider does not support creating inline comments yet")
|
||||
|
||||
def create_inline_comment(self, comments: list[dict]):
|
||||
raise NotImplementedError("Gitlab provider does not support publishing inline comments yet")
|
||||
|
||||
def send_inline_comment(self, body, edit_type, found, relevant_file, relevant_line_in_file, source_line_no,
|
||||
target_file, target_line_no):
|
||||
if not found:
|
||||
logging.info(f"Could not find position for {relevant_file} {relevant_line_in_file}")
|
||||
else:
|
||||
if edit_type == 'addition':
|
||||
position = target_line_no - 1
|
||||
else:
|
||||
position = source_line_no - 1
|
||||
d = self.last_diff
|
||||
pos_obj = {'position_type': 'text',
|
||||
'new_path': target_file.filename,
|
||||
'old_path': target_file.old_filename if target_file.old_filename else target_file.filename,
|
||||
'base_sha': d.base_commit_sha, 'start_sha': d.start_commit_sha, 'head_sha': d.head_commit_sha}
|
||||
'new_path': target_file.filename,
|
||||
'old_path': target_file.old_filename if target_file.old_filename else target_file.filename,
|
||||
'base_sha': d.base_commit_sha, 'start_sha': d.start_commit_sha, 'head_sha': d.head_commit_sha}
|
||||
if edit_type == 'deletion':
|
||||
pos_obj['old_line'] = position
|
||||
pos_obj['old_line'] = source_line_no - 1
|
||||
elif edit_type == 'addition':
|
||||
pos_obj['new_line'] = position
|
||||
pos_obj['new_line'] = target_line_no - 1
|
||||
else:
|
||||
pos_obj['new_line'] = position
|
||||
pos_obj['old_line'] = position
|
||||
pos_obj['new_line'] = target_line_no - 1
|
||||
pos_obj['old_line'] = source_line_no - 1
|
||||
self.mr.discussions.create({'body': body,
|
||||
'position': pos_obj})
|
||||
|
||||
@ -110,47 +138,67 @@ class GitLabProvider(GitProvider):
|
||||
relevant_file: str,
|
||||
relevant_lines_start: int,
|
||||
relevant_lines_end: int):
|
||||
raise "not implemented yet for gitlab"
|
||||
self.diff_files = self.diff_files if self.diff_files else self.get_diff_files()
|
||||
target_file = None
|
||||
for file in self.diff_files:
|
||||
if file.filename == relevant_file:
|
||||
if file.filename == relevant_file:
|
||||
target_file = file
|
||||
break
|
||||
range = relevant_lines_end - relevant_lines_start + 1
|
||||
body = body.replace('```suggestion', f'```suggestion:-0+{range}')
|
||||
|
||||
lines = target_file.head_file.splitlines()
|
||||
relevant_line_in_file = lines[relevant_lines_start - 1]
|
||||
edit_type, found, source_line_no, target_file, target_line_no = self.find_in_file(target_file,
|
||||
relevant_line_in_file)
|
||||
self.send_inline_comment(body, edit_type, found, relevant_file, relevant_line_in_file, source_line_no,
|
||||
target_file, target_line_no)
|
||||
|
||||
def search_line(self, relevant_file, relevant_line_in_file):
|
||||
RE_HUNK_HEADER = re.compile(
|
||||
r"^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@[ ]?(.*)")
|
||||
target_file = None
|
||||
source_line_no = 0
|
||||
target_line_no = 0
|
||||
found = False
|
||||
|
||||
edit_type = self.get_edit_type(relevant_line_in_file)
|
||||
for file in self.diff_files:
|
||||
if file.filename == relevant_file:
|
||||
target_file = file
|
||||
patch = file.patch
|
||||
patch_lines = patch.splitlines()
|
||||
for i, line in enumerate(patch_lines):
|
||||
if line.startswith('@@'):
|
||||
match = RE_HUNK_HEADER.match(line)
|
||||
if not match:
|
||||
continue
|
||||
start_old, size_old, start_new, size_new, _ = match.groups()
|
||||
source_line_no = int(start_old)
|
||||
target_line_no = int(start_new)
|
||||
continue
|
||||
if line.startswith('-'):
|
||||
source_line_no += 1
|
||||
elif line.startswith('+'):
|
||||
target_line_no += 1
|
||||
elif line.startswith(' '):
|
||||
source_line_no += 1
|
||||
target_line_no += 1
|
||||
if relevant_line_in_file in line:
|
||||
found = True
|
||||
edit_type = self.get_edit_type(line)
|
||||
break
|
||||
elif relevant_line_in_file[0] == '+' and relevant_line_in_file[1:] in line:
|
||||
# The model often adds a '+' to the beginning of the relevant_line_in_file even if originally
|
||||
# it's a context line
|
||||
found = True
|
||||
edit_type = self.get_edit_type(line)
|
||||
break
|
||||
edit_type, found, source_line_no, target_file, target_line_no = self.find_in_file(file,
|
||||
relevant_line_in_file)
|
||||
return edit_type, found, source_line_no, target_file, target_line_no
|
||||
|
||||
def find_in_file(self, file, relevant_line_in_file):
|
||||
edit_type = 'context'
|
||||
source_line_no = 0
|
||||
target_line_no = 0
|
||||
found = False
|
||||
target_file = file
|
||||
patch = file.patch
|
||||
patch_lines = patch.splitlines()
|
||||
for line in patch_lines:
|
||||
if line.startswith('@@'):
|
||||
match = self.RE_HUNK_HEADER.match(line)
|
||||
if not match:
|
||||
continue
|
||||
start_old, size_old, start_new, size_new, _ = match.groups()
|
||||
source_line_no = int(start_old)
|
||||
target_line_no = int(start_new)
|
||||
continue
|
||||
if line.startswith('-'):
|
||||
source_line_no += 1
|
||||
elif line.startswith('+'):
|
||||
target_line_no += 1
|
||||
elif line.startswith(' '):
|
||||
source_line_no += 1
|
||||
target_line_no += 1
|
||||
if relevant_line_in_file in line:
|
||||
found = True
|
||||
edit_type = self.get_edit_type(line)
|
||||
break
|
||||
elif relevant_line_in_file[0] == '+' and relevant_line_in_file[1:].lstrip() in line:
|
||||
# The model often adds a '+' to the beginning of the relevant_line_in_file even if originally
|
||||
# it's a context line
|
||||
found = True
|
||||
edit_type = self.get_edit_type(line)
|
||||
break
|
||||
return edit_type, found, source_line_no, target_file, target_line_no
|
||||
|
||||
def get_edit_type(self, relevant_line_in_file):
|
||||
@ -171,9 +219,6 @@ class GitLabProvider(GitProvider):
|
||||
def get_title(self):
|
||||
return self.mr.title
|
||||
|
||||
def get_description(self):
|
||||
return self.mr.description
|
||||
|
||||
def get_languages(self):
|
||||
languages = self.gl.projects.get(self.id_project).languages()
|
||||
return languages
|
||||
@ -184,6 +229,9 @@ class GitLabProvider(GitProvider):
|
||||
def get_pr_description(self):
|
||||
return self.mr.description
|
||||
|
||||
def get_issue_comments(self):
|
||||
raise NotImplementedError("GitLab provider does not support issue comments yet")
|
||||
|
||||
def _parse_merge_request_url(self, merge_request_url: str) -> Tuple[int, int]:
|
||||
parsed_url = urlparse(merge_request_url)
|
||||
|
||||
|
@ -1,12 +1,9 @@
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
|
||||
from pr_agent.agent.pr_agent import PRAgent
|
||||
from pr_agent.config_loader import settings
|
||||
from pr_agent.tools.pr_code_suggestions import PRCodeSuggestions
|
||||
from pr_agent.tools.pr_description import PRDescription
|
||||
from pr_agent.tools.pr_questions import PRQuestions
|
||||
from pr_agent.tools.pr_reviewer import PRReviewer
|
||||
|
||||
|
||||
@ -53,20 +50,7 @@ async def run_action():
|
||||
pr_url = event_payload.get("issue", {}).get("pull_request", {}).get("url", None)
|
||||
if pr_url:
|
||||
body = comment_body.strip().lower()
|
||||
if any(cmd in body for cmd in ["/review", "/review_pr"]):
|
||||
await PRReviewer(pr_url).review()
|
||||
elif any(cmd in body for cmd in ["/describe", "/describe_pr"]):
|
||||
await PRDescription(pr_url).describe()
|
||||
elif any(cmd in body for cmd in ["/improve", "/improve_code"]):
|
||||
await PRCodeSuggestions(pr_url).suggest()
|
||||
elif any(cmd in body for cmd in ["/ask", "/ask_question"]):
|
||||
pattern = r'(/ask|/ask_question)\s*(.*)'
|
||||
matches = re.findall(pattern, comment_body, re.IGNORECASE)
|
||||
if matches:
|
||||
question = matches[0][1]
|
||||
await PRQuestions(pr_url, question).answer()
|
||||
else:
|
||||
print(f"Unknown command: {body}")
|
||||
await PRAgent().handle_request(pr_url, body)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
@ -1,6 +1,5 @@
|
||||
import asyncio
|
||||
import logging
|
||||
import re
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
|
||||
@ -10,10 +9,6 @@ from pr_agent.agent.pr_agent import PRAgent
|
||||
from pr_agent.config_loader import settings
|
||||
from pr_agent.git_providers import get_git_provider
|
||||
from pr_agent.servers.help import bot_help_text
|
||||
from pr_agent.tools.pr_code_suggestions import PRCodeSuggestions
|
||||
from pr_agent.tools.pr_description import PRDescription
|
||||
from pr_agent.tools.pr_questions import PRQuestions
|
||||
from pr_agent.tools.pr_reviewer import PRReviewer
|
||||
|
||||
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
|
||||
NOTIFICATION_URL = "https://api.github.com/notifications"
|
||||
@ -94,7 +89,7 @@ async def polling_loop():
|
||||
success = await agent.handle_request(pr_url, rest_of_comment)
|
||||
if not success:
|
||||
git_provider.set_pr(pr_url)
|
||||
git_provider.publish_comment("### How to user PR-Agent\n" +
|
||||
git_provider.publish_comment("### How to use PR-Agent\n" +
|
||||
bot_help_text(user_id))
|
||||
|
||||
elif response.status != 304:
|
||||
@ -103,5 +98,6 @@ async def polling_loop():
|
||||
except Exception as e:
|
||||
logging.error(f"Exception during processing of a notification: {e}")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
asyncio.run(polling_loop())
|
||||
|
@ -1,14 +1,13 @@
|
||||
commands_text = "> /review - Ask for a new review after your update the PR\n" \
|
||||
"> /describe - Modify the PR title and description based " \
|
||||
"on the PR's contents.\n" \
|
||||
"> /improve - Suggest improvements to the code in the PR as pull " \
|
||||
"request comments ready to commit.\n" \
|
||||
"> /ask <QUESTION> - Ask a question about the PR.\n"
|
||||
commands_text = "> /review - Request a review of the latest update to the PR.\n" \
|
||||
"> /describe - Modify the PR title and description based on the contents of the PR.\n" \
|
||||
"> /improve - Suggest improvements to the code in the PR. " \
|
||||
"These will be provided as pull request comments, ready to commit.\n" \
|
||||
"> /ask <QUESTION> - Pose a question about the PR.\n"
|
||||
|
||||
|
||||
def bot_help_text(user: str):
|
||||
return f"> Tag me in a comment '@{user}' and add one of the following commands:\n" + commands_text
|
||||
|
||||
|
||||
actions_help_text = "> Add a comment to to invoke PR-Agent, use one of the following commands:\n" + \
|
||||
actions_help_text = "> To invoke the PR-Agent, add a comment using one of the following commands:\n" + \
|
||||
commands_text
|
||||
|
18
pr_agent/servers/serverless.py
Normal file
@ -0,0 +1,18 @@
|
||||
import logging
|
||||
|
||||
from fastapi import FastAPI
|
||||
from mangum import Mangum
|
||||
|
||||
from pr_agent.servers.github_app import router
|
||||
|
||||
logger = logging.getLogger()
|
||||
logger.setLevel(logging.DEBUG)
|
||||
|
||||
app = FastAPI()
|
||||
app.include_router(router)
|
||||
|
||||
handler = Mangum(app, lifespan="off")
|
||||
|
||||
|
||||
def serverless(event, context):
|
||||
return handler(event, context)
|
@ -32,3 +32,6 @@ webhook_secret = "<WEBHOOK SECRET>" # Optional, may be commented out.
|
||||
# Gitlab personal access token
|
||||
personal_access_token = ""
|
||||
|
||||
[bitbucket]
|
||||
# Bitbucket personal bearer token
|
||||
bearer_token = ""
|
||||
|
@ -1,7 +1,7 @@
|
||||
[config]
|
||||
model="gpt-4-0613"
|
||||
git_provider="github"
|
||||
publish_review=true
|
||||
publish_output=true
|
||||
verbosity_level=0 # 0,1,2
|
||||
|
||||
[pr_reviewer]
|
||||
@ -10,6 +10,10 @@ require_tests_review=true
|
||||
require_security_review=true
|
||||
num_code_suggestions=3
|
||||
inline_code_comments = true
|
||||
ask_and_reflect=false
|
||||
|
||||
[pr_description]
|
||||
publish_description_as_comment=false
|
||||
|
||||
[pr_questions]
|
||||
|
||||
|
@ -1,12 +1,12 @@
|
||||
[pr_code_suggestions_prompt]
|
||||
system="""You are a language model called CodiumAI-PR-Code-Reviewer.
|
||||
Your task is to provide provide meaningfull non-trivial code suggestions to improve the new code in a PR (the '+' lines).
|
||||
Your task is to provide meaningfull non-trivial code suggestions to improve the new code in a PR (the '+' lines).
|
||||
- Try to give important suggestions like fixing code problems, issues and bugs. As a second priority, provide suggestions for meaningfull code improvements, like performance, vulnerability, modularity, and best practices.
|
||||
- Suggestions should refer only to the 'new hunk' code, and focus on improving the new added code lines, with '+'.
|
||||
- Provide the exact line number range (inclusive) for each issue.
|
||||
- Assume there is additional code in the relevant file that is not included in the diff.
|
||||
- Provide up to {{ num_code_suggestions }} code suggestions.
|
||||
- Make sure not to provide suggestion repeating modifications already implemented in the new PR code (the '+' lines).
|
||||
- Make sure not to provide suggestions repeating modifications already implemented in the new PR code (the '+' lines).
|
||||
- Don't output line numbers in the 'improved code' snippets.
|
||||
|
||||
You must use the following JSON schema to format your answer:
|
||||
|
34
pr_agent/settings/pr_information_from_user_prompts.toml
Normal file
@ -0,0 +1,34 @@
|
||||
[pr_information_from_user_prompt]
|
||||
system="""You are CodiumAI-PR-Reviewer, a language model designed to review git pull requests.
|
||||
Given the PR Info and the PR Git Diff, generate 3 short questions about the PR code for the PR author.
|
||||
The goal of the questions is to help the language model understand the PR better, so the questions should be insightful, informative, non-trivial, and relevant to the PR.
|
||||
You should prefer asking yes\\no questions, or multiple choice questions. Also add at least one open-ended question, but make sure they are not too difficult, and can be answered in a sentence or two.
|
||||
|
||||
|
||||
Example output:
|
||||
'
|
||||
Questions to better understand the PR:
|
||||
1) ...
|
||||
2) ...
|
||||
...
|
||||
'
|
||||
"""
|
||||
|
||||
user="""PR Info:
|
||||
Title: '{{title}}'
|
||||
Branch: '{{branch}}'
|
||||
Description: '{{description}}'
|
||||
{%- if language %}
|
||||
Main language: {{language}}
|
||||
{%- endif %}
|
||||
|
||||
|
||||
The PR Git Diff:
|
||||
```
|
||||
{{diff}}
|
||||
```
|
||||
Note that lines in the diff body are prefixed with a symbol that represents the type of change: '-' for deletions, '+' for additions, and ' ' (a space) for unchanged lines
|
||||
|
||||
|
||||
Response:
|
||||
"""
|
@ -1,9 +1,9 @@
|
||||
[pr_questions_prompt]
|
||||
system="""You are CodiumAI-PR-Reviewer, a language model designed to review git pull requests.
|
||||
Your task is to answer questions about the new PR code (the '+' lines), and provide feedback.
|
||||
Be informative, constructive, and give examples. Try to be as specific as possible, and don't avoid answering the questions.
|
||||
Be informative, constructive, and give examples. Try to be as specific as possible.
|
||||
Don't avoid answering the questions. You must answer the questions, as best as you can, without adding unrelated content.
|
||||
Make sure not to repeat modifications already implemented in the new PR code (the '+' lines).
|
||||
Answer only the questions, and don't add unrelated content.
|
||||
"""
|
||||
|
||||
user="""PR Info:
|
||||
|
@ -2,8 +2,11 @@
|
||||
system="""You are CodiumAI-PR-Reviewer, a language model designed to review git pull requests.
|
||||
Your task is to provide constructive and concise feedback for the PR, and also provide meaningfull code suggestions to improve the new PR code (the '+' lines).
|
||||
- Provide up to {{ num_code_suggestions }} code suggestions.
|
||||
{%- if num_code_suggestions > 0 %}
|
||||
- Try to focus on important suggestions like fixing code problems, issues and bugs. As a second priority, provide suggestions for meaningfull code improvements, like performance, vulnerability, modularity, and best practices.
|
||||
- Make sure not to provide suggestion repeating modifications already implemented in the new PR code (the '+' lines).
|
||||
- Suggestions should focus on improving the new added code lines.
|
||||
- Make sure not to provide suggestions repeating modifications already implemented in the new PR code (the '+' lines).
|
||||
{%- endif %}
|
||||
|
||||
You must use the following JSON schema to format your answer:
|
||||
```json
|
||||
@ -23,6 +26,12 @@ You must use the following JSON schema to format your answer:
|
||||
"description": "yes\\no question: does this PR have relevant tests ?"
|
||||
},
|
||||
{%- endif %}
|
||||
{%- if question_str %}
|
||||
"Insights from user's answer": {
|
||||
"type": "string",
|
||||
"description": "shortly summarize the insights you gained from the user's answers to the questions"
|
||||
},
|
||||
{%- endif %}
|
||||
{%- if require_focused %}
|
||||
"Focused PR": {
|
||||
"type": "string",
|
||||
@ -35,6 +44,7 @@ You must use the following JSON schema to format your answer:
|
||||
"type": "string",
|
||||
"description": "General suggestions and feedback for the contributors and maintainers of this PR. May include important suggestions for the overall structure, primary purpose, best practices, critical bugs, and other aspects of the PR. Explain your suggestions."
|
||||
},
|
||||
{%- if num_code_suggestions > 0 %}
|
||||
"Code suggestions": {
|
||||
"type": "array",
|
||||
"maxItems": {{ num_code_suggestions }},
|
||||
@ -54,6 +64,7 @@ You must use the following JSON schema to format your answer:
|
||||
}
|
||||
}
|
||||
},
|
||||
{%- endif %}
|
||||
{%- if require_security %}
|
||||
"Security concerns": {
|
||||
"type": "string",
|
||||
@ -82,6 +93,7 @@ Example output:
|
||||
"PR Feedback":
|
||||
{
|
||||
"General PR suggestions": "..., `xxx`...",
|
||||
{%- if num_code_suggestions > 0 %}
|
||||
"Code suggestions": [
|
||||
{
|
||||
"relevant file": "directory/xxx.py",
|
||||
@ -90,6 +102,7 @@ Example output:
|
||||
},
|
||||
...
|
||||
]
|
||||
{%- endif %}
|
||||
{%- if require_security %},
|
||||
"Security concerns": "No, because ..."
|
||||
{%- endif %}
|
||||
@ -108,6 +121,16 @@ Description: '{{description}}'
|
||||
Main language: {{language}}
|
||||
{%- endif %}
|
||||
|
||||
{%- if question_str %}
|
||||
######
|
||||
Here are questions to better understand the PR. Use the answers to provide better feedback.
|
||||
|
||||
{{question_str|trim}}
|
||||
|
||||
User answers:
|
||||
{{answer_str|trim}}
|
||||
######
|
||||
{%- endif %}
|
||||
|
||||
The PR Git Diff:
|
||||
```
|
||||
|
@ -8,9 +8,9 @@ from jinja2 import Environment, StrictUndefined
|
||||
from pr_agent.algo.ai_handler import AiHandler
|
||||
from pr_agent.algo.pr_processing import get_pr_diff
|
||||
from pr_agent.algo.token_handler import TokenHandler
|
||||
from pr_agent.algo.utils import convert_to_markdown, try_fix_json
|
||||
from pr_agent.algo.utils import try_fix_json
|
||||
from pr_agent.config_loader import settings
|
||||
from pr_agent.git_providers import get_git_provider, GithubProvider
|
||||
from pr_agent.git_providers import BitbucketProvider, get_git_provider
|
||||
from pr_agent.git_providers.git_provider import get_main_pr_language
|
||||
|
||||
|
||||
@ -39,10 +39,10 @@ class PRCodeSuggestions:
|
||||
settings.pr_code_suggestions_prompt.user)
|
||||
|
||||
async def suggest(self):
|
||||
assert type(self.git_provider) == GithubProvider, "Only Github is supported for now"
|
||||
assert type(self.git_provider) != BitbucketProvider, "Bitbucket is not supported for now"
|
||||
|
||||
logging.info('Generating code suggestions for PR...')
|
||||
if settings.config.publish_review:
|
||||
if settings.config.publish_output:
|
||||
self.git_provider.publish_comment("Preparing review...", is_temporary=True)
|
||||
logging.info('Getting PR diff...')
|
||||
|
||||
@ -56,13 +56,12 @@ class PRCodeSuggestions:
|
||||
self.prediction = await self._get_prediction()
|
||||
logging.info('Preparing PR review...')
|
||||
data = self._prepare_pr_code_suggestions()
|
||||
if settings.config.publish_review:
|
||||
if settings.config.publish_output:
|
||||
logging.info('Pushing PR review...')
|
||||
self.git_provider.remove_initial_comment()
|
||||
logging.info('Pushing inline code comments...')
|
||||
self.push_inline_code_suggestions(data)
|
||||
|
||||
|
||||
async def _get_prediction(self):
|
||||
variables = copy.deepcopy(self.vars)
|
||||
variables["diff"] = self.patches_diff # update diff
|
||||
@ -86,7 +85,7 @@ class PRCodeSuggestions:
|
||||
except json.decoder.JSONDecodeError:
|
||||
if settings.config.verbosity_level >= 2:
|
||||
logging.info(f"Could not parse json response: {review}")
|
||||
data = try_fix_json(review)
|
||||
data = try_fix_json(review, code_suggestions=True)
|
||||
return data
|
||||
|
||||
def push_inline_code_suggestions(self, data):
|
||||
@ -98,12 +97,12 @@ class PRCodeSuggestions:
|
||||
relevant_lines_start = int(relevant_lines_str.split('-')[0]) # absolute position
|
||||
relevant_lines_end = int(relevant_lines_str.split('-')[-1])
|
||||
content = d['suggestion content']
|
||||
existing_code_snippet = d['existing code']
|
||||
new_code_snippet = d['improved code']
|
||||
|
||||
if new_code_snippet:
|
||||
try: # dedent code snippet
|
||||
self.diff_files = self.git_provider.diff_files if self.git_provider.diff_files else self.git_provider.get_diff_files()
|
||||
self.diff_files = self.git_provider.diff_files if self.git_provider.diff_files \
|
||||
else self.git_provider.get_diff_files()
|
||||
original_initial_line = None
|
||||
for file in self.diff_files:
|
||||
if file.filename.strip() == relevant_file:
|
||||
@ -121,7 +120,7 @@ class PRCodeSuggestions:
|
||||
logging.info(f"Could not dedent code snippet for file {relevant_file}, error: {e}")
|
||||
|
||||
body = f"**Suggestion:** {content}\n```suggestion\n" + new_code_snippet + "\n```"
|
||||
success = self.git_provider.publish_code_suggestion(body=body,
|
||||
self.git_provider.publish_code_suggestion(body=body,
|
||||
relevant_file=relevant_file,
|
||||
relevant_lines_start=relevant_lines_start,
|
||||
relevant_lines_end=relevant_lines_end)
|
||||
|
@ -7,7 +7,6 @@ from jinja2 import Environment, StrictUndefined
|
||||
from pr_agent.algo.ai_handler import AiHandler
|
||||
from pr_agent.algo.pr_processing import get_pr_diff
|
||||
from pr_agent.algo.token_handler import TokenHandler
|
||||
from pr_agent.algo.utils import convert_to_markdown
|
||||
from pr_agent.config_loader import settings
|
||||
from pr_agent.git_providers import get_git_provider
|
||||
from pr_agent.git_providers.git_provider import get_main_pr_language
|
||||
@ -23,7 +22,7 @@ class PRDescription:
|
||||
self.vars = {
|
||||
"title": self.git_provider.pr.title,
|
||||
"branch": self.git_provider.get_pr_branch(),
|
||||
"description": self.git_provider.get_description(),
|
||||
"description": self.git_provider.get_pr_description(),
|
||||
"language": self.main_pr_language,
|
||||
"diff": "", # empty diff for initial calculation
|
||||
}
|
||||
@ -36,17 +35,20 @@ class PRDescription:
|
||||
|
||||
async def describe(self):
|
||||
logging.info('Generating a PR description...')
|
||||
if settings.config.publish_review:
|
||||
if settings.config.publish_output:
|
||||
self.git_provider.publish_comment("Preparing pr description...", is_temporary=True)
|
||||
logging.info('Getting PR diff...')
|
||||
self.patches_diff = get_pr_diff(self.git_provider, self.token_handler)
|
||||
logging.info('Getting AI prediction...')
|
||||
self.prediction = await self._get_prediction()
|
||||
logging.info('Preparing answer...')
|
||||
pr_title, pr_body = self._prepare_pr_answer()
|
||||
if settings.config.publish_review:
|
||||
pr_title, pr_body, markdown_text = self._prepare_pr_answer()
|
||||
if settings.config.publish_output:
|
||||
logging.info('Pushing answer...')
|
||||
self.git_provider.publish_description(pr_title, pr_body)
|
||||
if settings.pr_description.publish_description_as_comment:
|
||||
self.git_provider.publish_comment(markdown_text)
|
||||
else:
|
||||
self.git_provider.publish_description(pr_title, pr_body)
|
||||
self.git_provider.remove_initial_comment()
|
||||
return ""
|
||||
|
||||
@ -66,10 +68,11 @@ class PRDescription:
|
||||
|
||||
def _prepare_pr_answer(self):
|
||||
data = json.loads(self.prediction)
|
||||
markdown_text = ""
|
||||
for key, value in data.items():
|
||||
markdown_text += f"## {key}\n\n"
|
||||
markdown_text += f"{value}\n\n"
|
||||
pr_body = ""
|
||||
# for key, value in data.items():
|
||||
# markdown_text += f"## {key}\n\n"
|
||||
# markdown_text += f"{value}\n\n"
|
||||
title = data['PR Title']
|
||||
del data['PR Title']
|
||||
for key, value in data.items():
|
||||
@ -80,4 +83,4 @@ class PRDescription:
|
||||
pr_body += f"**{value}**\n\n___\n"
|
||||
if settings.config.verbosity_level >= 2:
|
||||
logging.info(f"title:\n{title}\n{pr_body}")
|
||||
return title, pr_body
|
||||
return title, pr_body, markdown_text
|
||||
|
71
pr_agent/tools/pr_information_from_user.py
Normal file
@ -0,0 +1,71 @@
|
||||
import copy
|
||||
import logging
|
||||
|
||||
from jinja2 import Environment, StrictUndefined
|
||||
|
||||
from pr_agent.algo.ai_handler import AiHandler
|
||||
from pr_agent.algo.pr_processing import get_pr_diff
|
||||
from pr_agent.algo.token_handler import TokenHandler
|
||||
from pr_agent.config_loader import settings
|
||||
from pr_agent.git_providers import get_git_provider
|
||||
from pr_agent.git_providers.git_provider import get_main_pr_language
|
||||
|
||||
|
||||
class PRInformationFromUser:
|
||||
def __init__(self, pr_url: str):
|
||||
self.git_provider = get_git_provider()(pr_url)
|
||||
self.main_pr_language = get_main_pr_language(
|
||||
self.git_provider.get_languages(), self.git_provider.get_files()
|
||||
)
|
||||
self.ai_handler = AiHandler()
|
||||
self.vars = {
|
||||
"title": self.git_provider.pr.title,
|
||||
"branch": self.git_provider.get_pr_branch(),
|
||||
"description": self.git_provider.get_pr_description(),
|
||||
"language": self.main_pr_language,
|
||||
"diff": "", # empty diff for initial calculation
|
||||
}
|
||||
self.token_handler = TokenHandler(self.git_provider.pr,
|
||||
self.vars,
|
||||
settings.pr_information_from_user_prompt.system,
|
||||
settings.pr_information_from_user_prompt.user)
|
||||
self.patches_diff = None
|
||||
self.prediction = None
|
||||
|
||||
async def generate_questions(self):
|
||||
logging.info('Generating question to the user...')
|
||||
if settings.config.publish_output:
|
||||
self.git_provider.publish_comment("Preparing questions...", is_temporary=True)
|
||||
logging.info('Getting PR diff...')
|
||||
self.patches_diff = get_pr_diff(self.git_provider, self.token_handler)
|
||||
logging.info('Getting AI prediction...')
|
||||
self.prediction = await self._get_prediction()
|
||||
logging.info('Preparing questions...')
|
||||
pr_comment = self._prepare_pr_answer()
|
||||
if settings.config.publish_output:
|
||||
logging.info('Pushing questions...')
|
||||
self.git_provider.publish_comment(pr_comment)
|
||||
self.git_provider.remove_initial_comment()
|
||||
return ""
|
||||
|
||||
async def _get_prediction(self):
|
||||
variables = copy.deepcopy(self.vars)
|
||||
variables["diff"] = self.patches_diff # update diff
|
||||
environment = Environment(undefined=StrictUndefined)
|
||||
system_prompt = environment.from_string(settings.pr_information_from_user_prompt.system).render(variables)
|
||||
user_prompt = environment.from_string(settings.pr_information_from_user_prompt.user).render(variables)
|
||||
if settings.config.verbosity_level >= 2:
|
||||
logging.info(f"\nSystem prompt:\n{system_prompt}")
|
||||
logging.info(f"\nUser prompt:\n{user_prompt}")
|
||||
model = settings.config.model
|
||||
response, finish_reason = await self.ai_handler.chat_completion(model=model, temperature=0.2,
|
||||
system=system_prompt, user=user_prompt)
|
||||
return response
|
||||
|
||||
def _prepare_pr_answer(self) -> str:
|
||||
model_output = self.prediction.strip()
|
||||
if settings.config.verbosity_level >= 2:
|
||||
logging.info(f"answer_str:\n{model_output}")
|
||||
answer_str = f"{model_output}\n\n Please respond to the questions above in the following format:\n\n" +\
|
||||
"\n>/answer\n>1) ...\n>2) ...\n>...\n"
|
||||
return answer_str
|
@ -22,7 +22,7 @@ class PRQuestions:
|
||||
self.vars = {
|
||||
"title": self.git_provider.pr.title,
|
||||
"branch": self.git_provider.get_pr_branch(),
|
||||
"description": self.git_provider.get_description(),
|
||||
"description": self.git_provider.get_pr_description(),
|
||||
"language": self.main_pr_language,
|
||||
"diff": "", # empty diff for initial calculation
|
||||
"questions": self.question_str,
|
||||
@ -36,7 +36,7 @@ class PRQuestions:
|
||||
|
||||
async def answer(self):
|
||||
logging.info('Answering a PR question...')
|
||||
if settings.config.publish_review:
|
||||
if settings.config.publish_output:
|
||||
self.git_provider.publish_comment("Preparing answer...", is_temporary=True)
|
||||
logging.info('Getting PR diff...')
|
||||
self.patches_diff = get_pr_diff(self.git_provider, self.token_handler)
|
||||
@ -44,7 +44,7 @@ class PRQuestions:
|
||||
self.prediction = await self._get_prediction()
|
||||
logging.info('Preparing answer...')
|
||||
pr_comment = self._prepare_pr_answer()
|
||||
if settings.config.publish_review:
|
||||
if settings.config.publish_output:
|
||||
logging.info('Pushing answer...')
|
||||
self.git_provider.publish_comment(pr_comment)
|
||||
self.git_provider.remove_initial_comment()
|
||||
|
@ -11,16 +11,20 @@ from pr_agent.algo.utils import convert_to_markdown, try_fix_json
|
||||
from pr_agent.config_loader import settings
|
||||
from pr_agent.git_providers import get_git_provider
|
||||
from pr_agent.git_providers.git_provider import get_main_pr_language
|
||||
from pr_agent.servers.help import bot_help_text, actions_help_text
|
||||
from pr_agent.servers.help import actions_help_text, bot_help_text
|
||||
|
||||
|
||||
class PRReviewer:
|
||||
def __init__(self, pr_url: str, cli_mode=False):
|
||||
def __init__(self, pr_url: str, cli_mode=False, is_answer: bool = False):
|
||||
|
||||
self.git_provider = get_git_provider()(pr_url)
|
||||
self.main_language = get_main_pr_language(
|
||||
self.git_provider.get_languages(), self.git_provider.get_files()
|
||||
)
|
||||
self.is_answer = is_answer
|
||||
if self.is_answer and not self.git_provider.is_supported("get_issue_comments"):
|
||||
raise Exception(f"Answer mode is not supported for {settings.config.git_provider} for now")
|
||||
answer_str, question_str = self._get_user_answers()
|
||||
self.ai_handler = AiHandler()
|
||||
self.patches_diff = None
|
||||
self.prediction = None
|
||||
@ -35,6 +39,9 @@ class PRReviewer:
|
||||
"require_security": settings.pr_reviewer.require_security_review,
|
||||
"require_focused": settings.pr_reviewer.require_focused_review,
|
||||
'num_code_suggestions': settings.pr_reviewer.num_code_suggestions,
|
||||
#
|
||||
'question_str': question_str,
|
||||
'answer_str': answer_str,
|
||||
}
|
||||
self.token_handler = TokenHandler(self.git_provider.pr,
|
||||
self.vars,
|
||||
@ -43,15 +50,15 @@ class PRReviewer:
|
||||
|
||||
async def review(self):
|
||||
logging.info('Reviewing PR...')
|
||||
if settings.config.publish_review:
|
||||
self.git_provider.publish_comment("Preparing review...", is_temporary=True)
|
||||
if settings.config.publish_output:
|
||||
self.git_provider.publish_comment("Preparing review...", is_temporary=True)
|
||||
logging.info('Getting PR diff...')
|
||||
self.patches_diff = get_pr_diff(self.git_provider, self.token_handler)
|
||||
logging.info('Getting AI prediction...')
|
||||
self.prediction = await self._get_prediction()
|
||||
logging.info('Preparing PR review...')
|
||||
pr_comment = self._prepare_pr_review()
|
||||
if settings.config.publish_review:
|
||||
if settings.config.publish_output:
|
||||
logging.info('Pushing PR review...')
|
||||
self.git_provider.publish_comment(pr_comment)
|
||||
self.git_provider.remove_initial_comment()
|
||||
@ -89,8 +96,16 @@ class PRReviewer:
|
||||
del data['PR Feedback']['Security concerns']
|
||||
data['PR Analysis']['Security concerns'] = val
|
||||
|
||||
if settings.config.git_provider == 'github' and settings.pr_reviewer.inline_code_comments:
|
||||
del data['PR Feedback']['Code suggestions']
|
||||
if settings.config.git_provider != 'bitbucket' and \
|
||||
settings.pr_reviewer.inline_code_comments and \
|
||||
'Code suggestions' in data['PR Feedback']:
|
||||
# keeping only code suggestions that can't be submitted as inline comments
|
||||
data['PR Feedback']['Code suggestions'] = [
|
||||
d for d in data['PR Feedback']['Code suggestions']
|
||||
if any(key not in d for key in ('relevant file', 'relevant line in file', 'suggestion content'))
|
||||
]
|
||||
if not data['PR Feedback']['Code suggestions']:
|
||||
del data['PR Feedback']['Code suggestions']
|
||||
|
||||
markdown_text = convert_to_markdown(data)
|
||||
user = self.git_provider.get_user_id()
|
||||
@ -107,15 +122,43 @@ class PRReviewer:
|
||||
return markdown_text
|
||||
|
||||
def _publish_inline_code_comments(self):
|
||||
if settings.pr_reviewer.num_code_suggestions == 0:
|
||||
return
|
||||
|
||||
review = self.prediction.strip()
|
||||
try:
|
||||
data = json.loads(review)
|
||||
except json.decoder.JSONDecodeError:
|
||||
data = try_fix_json(review)
|
||||
|
||||
comments = []
|
||||
for d in data['PR Feedback']['Code suggestions']:
|
||||
relevant_file = d['relevant file'].strip()
|
||||
relevant_line_in_file = d['relevant line in file'].strip()
|
||||
content = d['suggestion content']
|
||||
relevant_file = d.get('relevant file', '').strip()
|
||||
relevant_line_in_file = d.get('relevant line in file', '').strip()
|
||||
content = d.get('suggestion content', '')
|
||||
if not relevant_file or not relevant_line_in_file or not content:
|
||||
logging.info("Skipping inline comment with missing file/line/content")
|
||||
continue
|
||||
|
||||
self.git_provider.publish_inline_comment(content, relevant_file, relevant_line_in_file)
|
||||
if self.git_provider.is_supported("create_inline_comment"):
|
||||
comment = self.git_provider.create_inline_comment(content, relevant_file, relevant_line_in_file)
|
||||
if comment:
|
||||
comments.append(comment)
|
||||
else:
|
||||
self.git_provider.publish_inline_comment(content, relevant_file, relevant_line_in_file)
|
||||
|
||||
if comments:
|
||||
self.git_provider.publish_inline_comments(comments)
|
||||
|
||||
def _get_user_answers(self):
|
||||
answer_str = question_str = ""
|
||||
if self.is_answer:
|
||||
discussion_messages = self.git_provider.get_issue_comments()
|
||||
for message in discussion_messages.reversed:
|
||||
if "Questions to better understand the PR:" in message.body:
|
||||
question_str = message.body
|
||||
elif '/answer' in message.body:
|
||||
answer_str = message.body
|
||||
if answer_str and question_str:
|
||||
break
|
||||
return question_str, answer_str
|
||||
|
@ -1,6 +1,6 @@
|
||||
dynaconf==3.1.12
|
||||
fastapi==0.99.0
|
||||
PyGithub==1.58.2
|
||||
PyGithub==1.59.*
|
||||
retry==0.9.2
|
||||
openai==0.27.8
|
||||
Jinja2==3.1.2
|
||||
@ -9,3 +9,4 @@ uvicorn==0.22.0
|
||||
python-gitlab==3.15.0
|
||||
pytest~=7.4.0
|
||||
aiohttp~=3.8.4
|
||||
atlassian-python-api==3.39.0
|
||||
|
@ -1,13 +1,12 @@
|
||||
# Generated by CodiumAI
|
||||
|
||||
from pr_agent.algo.utils import try_fix_json
|
||||
|
||||
|
||||
import pytest
|
||||
|
||||
class TestTryFixJson:
|
||||
# Tests that JSON with complete 'Code suggestions' section returns expected output
|
||||
def test_incomplete_code_suggestions(self):
|
||||
review = '{"PR Analysis": {"Main theme": "xxx", "Type of PR": "Bug fix"}, "PR Feedback": {"General PR suggestions": "..., `xxx`...", "Code suggestions": [{"relevant file": "xxx.py", "suggestion content": "xxx [important]"}, {"suggestion number": 2, "relevant file": "yyy.py", "suggestion content": "yyy [incomp...'
|
||||
review = '{"PR Analysis": {"Main theme": "xxx", "Type of PR": "Bug fix"}, "PR Feedback": {"General PR suggestions": "..., `xxx`...", "Code suggestions": [{"relevant file": "xxx.py", "suggestion content": "xxx [important]"}, {"suggestion number": 2, "relevant file": "yyy.py", "suggestion content": "yyy [incomp...' # noqa: E501
|
||||
expected_output = {
|
||||
'PR Analysis': {
|
||||
'Main theme': 'xxx',
|
||||
@ -26,7 +25,7 @@ class TestTryFixJson:
|
||||
assert try_fix_json(review) == expected_output
|
||||
|
||||
def test_incomplete_code_suggestions_new_line(self):
|
||||
review = '{"PR Analysis": {"Main theme": "xxx", "Type of PR": "Bug fix"}, "PR Feedback": {"General PR suggestions": "..., `xxx`...", "Code suggestions": [{"relevant file": "xxx.py", "suggestion content": "xxx [important]"} \n\t, {"suggestion number": 2, "relevant file": "yyy.py", "suggestion content": "yyy [incomp...'
|
||||
review = '{"PR Analysis": {"Main theme": "xxx", "Type of PR": "Bug fix"}, "PR Feedback": {"General PR suggestions": "..., `xxx`...", "Code suggestions": [{"relevant file": "xxx.py", "suggestion content": "xxx [important]"} \n\t, {"suggestion number": 2, "relevant file": "yyy.py", "suggestion content": "yyy [incomp...' # noqa: E501
|
||||
expected_output = {
|
||||
'PR Analysis': {
|
||||
'Main theme': 'xxx',
|
||||
@ -45,7 +44,7 @@ class TestTryFixJson:
|
||||
assert try_fix_json(review) == expected_output
|
||||
|
||||
def test_incomplete_code_suggestions_many_close_brackets(self):
|
||||
review = '{"PR Analysis": {"Main theme": "xxx", "Type of PR": "Bug fix"}, "PR Feedback": {"General PR suggestions": "..., `xxx`...", "Code suggestions": [{"relevant file": "xxx.py", "suggestion content": "xxx [important]"} \n, {"suggestion number": 2, "relevant file": "yyy.py", "suggestion content": "yyy }, [}\n ,incomp.} ,..'
|
||||
review = '{"PR Analysis": {"Main theme": "xxx", "Type of PR": "Bug fix"}, "PR Feedback": {"General PR suggestions": "..., `xxx`...", "Code suggestions": [{"relevant file": "xxx.py", "suggestion content": "xxx [important]"} \n, {"suggestion number": 2, "relevant file": "yyy.py", "suggestion content": "yyy }, [}\n ,incomp.} ,..' # noqa: E501
|
||||
expected_output = {
|
||||
'PR Analysis': {
|
||||
'Main theme': 'xxx',
|
||||
@ -64,7 +63,7 @@ class TestTryFixJson:
|
||||
assert try_fix_json(review) == expected_output
|
||||
|
||||
def test_incomplete_code_suggestions_relevant_file(self):
|
||||
review = '{"PR Analysis": {"Main theme": "xxx", "Type of PR": "Bug fix"}, "PR Feedback": {"General PR suggestions": "..., `xxx`...", "Code suggestions": [{"relevant file": "xxx.py", "suggestion content": "xxx [important]"}, {"suggestion number": 2, "relevant file": "yyy.p'
|
||||
review = '{"PR Analysis": {"Main theme": "xxx", "Type of PR": "Bug fix"}, "PR Feedback": {"General PR suggestions": "..., `xxx`...", "Code suggestions": [{"relevant file": "xxx.py", "suggestion content": "xxx [important]"}, {"suggestion number": 2, "relevant file": "yyy.p' # noqa: E501
|
||||
expected_output = {
|
||||
'PR Analysis': {
|
||||
'Main theme': 'xxx',
|
||||
|