Add Gitlab webhook installation documentation

Gerrit support

CONFIGURATION.md

@@ -1,57 +0,0 @@
-## Configuration
-
-The different tools and sub-tools used by CodiumAI PR-Agent are adjustable via the **[configuration file](pr_agent/settings/configuration.toml)**
-
-### Working from CLI
-When running from source (CLI), your local configuration file will be initially used.
-
-Example for invoking the 'review' tools via the CLI: 
-
-```
-python cli.py --pr-url=<pr_url>  review
-```
-In addition to general configurations, the 'review' tool will use parameters from the `[pr_reviewer]` section (every tool has a dedicated section in the configuration file).
-
-Note that you can print results locally, without publishing them, by setting in `configuration.toml`:
-
-```
-[config]
-publish_output=true
-verbosity_level=2
-```
-This is useful for debugging or experimenting with the different tools.
-
-### Working from pre-built repo (GitHub Action/GitHub App/Docker)
-When running PR-Agent from a pre-built repo, the default configuration file will be loaded.
-
-To edit the configuration, you have two options: 
-1. Place a local configuration file in the root of your local repo. The local file will be used instead of the default one.
-2. For online usage, just add `--config_path=<value>` to you command, to edit a specific configuration value.
-For example if you want to edit `pr_reviewer` configurations, you can run:
-```
-/review --pr_reviewer.extra_instructions="..." --pr_reviewer.require_score_review=false ...
-```
-
-Any configuration value in `configuration.toml` file can be similarly edited.
-
-### General configuration parameters
-
-#### Changing a model
-See [here](pr_agent/algo/__init__.py) for the list of available models.
-
-To use Llama2 model, for example, set:
-```
-[config]
-model = "replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1"
-[replicate]
-key = ...
-```
-(you can obtain a Llama2 key from [here](https://replicate.com/replicate/llama-2-70b-chat/api))
-
-Also review the [AiHandler](pr_agent/algo/ai_handler.py) file for instruction how to set keys for other models.
-
-#### Extra instructions
-All PR-Agent tools have a parameter called `extra_instructions`, that enables to add free-text extra instructions. Example usage:
-```
-/update_changelog --pr_update_changelog.extra_instructions="Make sure to update also the version ..."
-```

Dockerfile.github_action

@@ -2,7 +2,8 @@ FROM python:3.10 as base
 
 WORKDIR /app
 ADD pyproject.toml .
-RUN pip install . && rm pyproject.toml
+ADD requirements.txt .
+RUN pip install . && rm pyproject.toml requirements.txt
 ENV PYTHONPATH=/app
 ADD pr_agent pr_agent
 ADD github_action/entrypoint.sh /

INSTALL.md

@@ -9,8 +9,8 @@ To get started with PR-Agent quickly, you first need to acquire two tokens:
 There are several ways to use PR-Agent:
 
 - [Method 1: Use Docker image (no installation required)](INSTALL.md#method-1-use-docker-image-no-installation-required)
-- [Method 2: Run as a GitHub Action](INSTALL.md#method-2-run-as-a-github-action)
+- [Method 2: Run from source](INSTALL.md#method-2-run-from-source)
-- [Method 3: Run from source](INSTALL.md#method-3-run-from-source)
+- [Method 3: Run as a GitHub Action](INSTALL.md#method-3-run-as-a-github-action)
 - [Method 4: Run as a polling server](INSTALL.md#method-4-run-as-a-polling-server)
 - [Method 5: Run as a GitHub App](INSTALL.md#method-5-run-as-a-github-app)
 - [Method 6: Deploy as a Lambda Function](INSTALL.md#method-6---deploy-as-a-lambda-function)
@@ -55,7 +55,41 @@ Possible questions you can ask include:
 
 ---
 
-### Method 2: Run as a GitHub Action
+### Method 2: Run from source
+
+1. Clone this repository:
+
+```
+git clone https://github.com/Codium-ai/pr-agent.git
+```
+
+2. Install the requirements in your favorite virtual environment:
+
+```
+pip install -r requirements.txt
+```
+
+3. Copy the secrets template file and fill in your OpenAI key and your GitHub user token:
+
+```
+cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets.toml
+chmod 600 pr_agent/settings/.secrets.toml
+# Edit .secrets.toml file
+```
+
+4. Add the pr_agent folder to your PYTHONPATH, then run the cli.py script:
+
+```
+export PYTHONPATH=[$PYTHONPATH:]<PATH to pr_agent folder>
+python pr_agent/cli.py --pr_url <pr_url> /review
+python pr_agent/cli.py --pr_url <pr_url> /ask <your question>
+python pr_agent/cli.py --pr_url <pr_url> /describe
+python pr_agent/cli.py --pr_url <pr_url> /improve
+```
+
+---
+
+### Method 3: Run as a GitHub Action
 
 You can use our pre-built Github Action Docker image to run PR-Agent as a Github Action. 
 
@@ -114,7 +148,7 @@ The GITHUB_TOKEN secret is automatically created by GitHub.
 3. Merge this change to your main branch. 
 When you open your next PR, you should see a comment from `github-actions` bot with a review of your PR, and instructions on how to use the rest of the tools.
 
-4. You may configure PR-Agent by adding environment variables under the env section corresponding to any configurable property in the [configuration](./CONFIGURATION.md) file. Some examples:
+4. You may configure PR-Agent by adding environment variables under the env section corresponding to any configurable property in the [configuration](./Usage.md) file. Some examples:
 ```yaml
       env:
         # ... previous environment values
@@ -125,40 +159,6 @@ When you open your next PR, you should see a comment from `github-actions` bot w
 
 ---
 
-### Method 3: Run from source
-
-1. Clone this repository:
-
-```
-git clone https://github.com/Codium-ai/pr-agent.git
-```
-
-2. Install the requirements in your favorite virtual environment:
-
-```
-pip install -r requirements.txt
-```
-
-3. Copy the secrets template file and fill in your OpenAI key and your GitHub user token:
-
-```
-cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets.toml
-chmod 600 pr_agent/settings/.secrets.toml
-# Edit .secrets.toml file
-```
-
-4. Add the pr_agent folder to your PYTHONPATH, then run the cli.py script:
-
-```
-export PYTHONPATH=[$PYTHONPATH:]<PATH to pr_agent folder>
-python pr_agent/cli.py --pr_url <pr_url> review
-python pr_agent/cli.py --pr_url <pr_url> ask <your question>
-python pr_agent/cli.py --pr_url <pr_url> describe
-python pr_agent/cli.py --pr_url <pr_url> improve
-```
-
----
-
 ### Method 4: Run as a polling server
 Request reviews by tagging your Github user on a PR
 
@@ -251,6 +251,9 @@ docker push codiumai/pr-agent:github_app  # Push to your Docker repository
 
 9. Install the app by navigating to the "Install App" tab and selecting your desired repositories.
 
+> **Note:** When running PR-Agent from GitHub App, the default configuration file (configuration.toml) will be loaded.<br>
+> However, you can override the default tool parameters by uploading a local configuration file<br>
+> For more information please check out [CONFIGURATION.md](Usage.md#working-from-github-app-pre-built-repo)
 ---
 
 ### Method 6 - Deploy as a Lambda Function
@@ -309,7 +312,9 @@ Example IAM permissions to that user to allow access to CodeCommit:
                 "codecommit:Get*",
                 "codecommit:List*",
                 "codecommit:PostComment*",
-                "codecommit:PutCommentReaction"
+                "codecommit:PutCommentReaction",
+                "codecommit:UpdatePullRequestDescription",
+                "codecommit:UpdatePullRequestTitle"                
             ],
             "Resource": "*"
         }
@@ -338,6 +343,27 @@ PYTHONPATH="/PATH/TO/PROJECTS/pr-agent" python pr_agent/cli.py \
   review
 ```
 
+---
+
+### Method 8 - Run a GitLab webhook server
+
+1. From the GitLab workspace or group, create an access token. Enable the "api" scope only.
+2. Generate a random secret for your app, and save it for later. For example, you can use:
+
+```
+WEBHOOK_SECRET=$(python -c "import secrets; print(secrets.token_hex(10))")
+```
+3. Follow the instructions to build the Docker image, setup a secrets file and deploy on your own server from [Method 5](#method-5-run-as-a-github-app).
+4. In the secrets file, fill in the following:
+    - Your OpenAI key.
+    - In the [gitlab] section, fill in personal_access_token and shared_secret. The access token can be a personal access token, or a group or project access token.
+    - Set deployment_type to 'gitlab' in [configuration.toml](./pr_agent/settings/configuration.toml)
+5. Create a webhook in GitLab. Set the URL to the URL of your app's server. Set the secret token to the generated secret from step 2. 
+In the "Trigger" section, check the ‘comments’ and ‘merge request events’ boxes. 
+6. Test your installation by opening a merge request or commenting or a merge request using one of CodiumAI's commands.
+
+---
+
 ### Appendix - **Debugging LLM API Calls**  
 If you're testing your codium/pr-agent server, and need to see if calls were made successfully + the exact call logs, you can use the [LiteLLM Debugger tool](https://docs.litellm.ai/docs/debugging/hosted_debugging). 
 

MANIFEST.in

@@ -1 +0,0 @@
-recursive-include pr_agent/settings/ *.toml

README.md

@@ -17,17 +17,20 @@ Making pull requests less painful with an AI agent
 
 CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull requests faster and more efficiently. It automatically analyzes the pull request and can provide several types of PR feedback:
 
-**Auto-Description**: Automatically generating [PR description](https://github.com/Codium-ai/pr-agent/pull/229#issue-1860711415) - title, type, summary, code walkthrough and labels.
+**Auto Description (/describe)**: Automatically generating [PR description](https://github.com/Codium-ai/pr-agent/pull/229#issue-1860711415) - title, type, summary, code walkthrough and labels.
 \
-**Auto Review**: [Adjustable feedback](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695022908) about the PR main theme, type, relevant tests, security issues, score, and various suggestions for the PR content.
+**Auto Review (/review)**: [Adjustable feedback](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695022908) about the PR main theme, type, relevant tests, security issues, score, and various suggestions for the PR content.
 \
-**Question Answering**: Answering [free-text questions](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021332) about the PR.
+**Question Answering (/ask ...)**: Answering [free-text questions](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021332) about the PR.
 \
-**Code Suggestions**: [Committable code suggestions](https://github.com/Codium-ai/pr-agent/pull/229#discussion_r1306919276) for improving the PR.
+**Code Suggestions (/improve)**: [Committable code suggestions](https://github.com/Codium-ai/pr-agent/pull/229#discussion_r1306919276) for improving the PR.
 \
-**Update Changelog**: Automatically updating the CHANGELOG.md file with the [PR changes](https://github.com/Codium-ai/pr-agent/pull/168#discussion_r1282077645).
+**Update Changelog (/update_changelog)**: Automatically updating the CHANGELOG.md file with the [PR changes](https://github.com/Codium-ai/pr-agent/pull/168#discussion_r1282077645).
 
-<h3>Example results:</h2>
+
+See the [usage guide](./Usage.md) for instructions how to run the different tools from [CLI](./Usage.md#working-from-a-local-repo-cli), or by [online usage](./Usage.md#online-usage).
+
+<h3>Example results:</h3>
 </div>
 <h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1687561986">/describe:</a></h4>
 <div align="center">
@@ -35,87 +38,101 @@ CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull
 <img src="https://www.codium.ai/images/describe-2.gif" width="800">
 </p>
 </div>
+
 <h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021901">/review:</a></h4>
 <div align="center">
 <p float="center">
 <img src="https://www.codium.ai/images/review-2.gif" width="800">
 </p>
 </div>
-<h4><a href="https://github.com/Codium-ai/pr-agent/pull/78#issuecomment-1639739496">/reflect_and_review:</a></h4>
+
-<div align="center">
+[//]: # (<h4><a href="https://github.com/Codium-ai/pr-agent/pull/78#issuecomment-1639739496">/reflect_and_review:</a></h4>)
-<p float="center">
+
-<img src="https://www.codium.ai/images/reflect_and_review.gif" width="800">
+[//]: # (<div align="center">)
-</p>
+
-</div>
+[//]: # (<p float="center">)
-<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695020538">/ask:</a></h4>
+
-<div align="center">
+[//]: # (<img src="https://www.codium.ai/images/reflect_and_review.gif" width="800">)
-<p float="center">
+
-<img src="https://www.codium.ai/images/ask-2.gif" width="800">
+[//]: # (</p>)
-</p>
+
-</div>
+[//]: # (</div>)
-<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695024952">/improve:</a></h4>
+
-<div align="center">
+[//]: # (<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695020538">/ask:</a></h4>)
-<p float="center">
+
-<img src="https://www.codium.ai/images/improve-2.gif" width="800">
+[//]: # (<div align="center">)
-</p>
+
-</div>
+[//]: # (<p float="center">)
+
+[//]: # (<img src="https://www.codium.ai/images/ask-2.gif" width="800">)
+
+[//]: # (</p>)
+
+[//]: # (</div>)
+
+[//]: # (<h4><a href="https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695024952">/improve:</a></h4>)
+
+[//]: # (<div align="center">)
+
+[//]: # (<p float="center">)
+
+[//]: # (<img src="https://www.codium.ai/images/improve-2.gif" width="800">)
+
+[//]: # (</p>)
+
+[//]: # (</div>)
 <div align="left">
 
-
+## Table of Contents
 - [Overview](#overview)
 - [Try it now](#try-it-now)
 - [Installation](#installation)
-- [Configuration](./CONFIGURATION.md)
+- [Usage guide](./Usage.md)
 - [How it works](#how-it-works)
 - [Why use PR-Agent](#why-use-pr-agent)
 - [Roadmap](#roadmap)
-- [Similar projects](#similar-projects)
 </div>
 
 
 ## Overview
 `PR-Agent` offers extensive pull request functionalities across various git providers:
-|       |                                             | GitHub | Gitlab | Bitbucket | CodeCommit |
+|       |                                             | GitHub | Gitlab | Bitbucket | CodeCommit | Azure DevOps | Gerrit |
-|-------|---------------------------------------------|:------:|:------:|:---------:|:----------:|
+|-------|---------------------------------------------|:------:|:------:|:---------:|:----------:|:----------:|:----------:|
-| TOOLS | Review                                      |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:       |   :white_check_mark:    |
+| TOOLS | Review                                      |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:       |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:    |
-|       | ⮑ Inline review                             |   :white_check_mark:    |   :white_check_mark:    |           |          |
+|       | Ask                                         |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:          |   :white_check_mark:          | :white_check_mark: |  :white_check_mark:    |
-|       | Ask                                         |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:          |
+|       | Auto-Description                            |   :white_check_mark:    |   :white_check_mark:    |           |   :white_check_mark:    |   :white_check_mark:    | :white_check_mark:    |
-|       | Auto-Description                            |   :white_check_mark:    |   :white_check_mark:    |           |          |
+|       | Improve Code                                |   :white_check_mark:    |   :white_check_mark:    |           |   :white_check_mark:    |          |    :white_check_mark:    |
-|       | Improve Code                                |   :white_check_mark:    |   :white_check_mark:    |           |          |
+|       | ⮑ Extended                             |   :white_check_mark:    |   :white_check_mark:    |           |   :white_check_mark:    |          | :white_check_mark:    |
-|       | ⮑ Extended                             |   :white_check_mark:    |   :white_check_mark:    |           |          |
+|       | Reflect and Review                          |   :white_check_mark:    |                         |           |          |   :white_check_mark:    |    :white_check_mark:    |
-|       | Reflect and Review                          |   :white_check_mark:    |                         |           |          |
+|       | Update CHANGELOG.md                         |   :white_check_mark:    |                         |           |          |          |       |
-|       | Update CHANGELOG.md                         |   :white_check_mark:    |                         |           |          |
+|       |                                             |        |        |      |      |      |
-|       |                                             |        |        |      |      |
+| USAGE | CLI                                         |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:       |   :white_check_mark:    |   :white_check_mark:    |
-| USAGE | CLI                                         |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:       |   :white_check_mark:    |
+|       | App / webhook                               |   :white_check_mark:    |   :white_check_mark:    |           |          |          |
-|       | App / webhook                               |   :white_check_mark:    |   :white_check_mark:    |           |          |
+|       | Tagging bot                                 |   :white_check_mark:    |        |           |          |          |
-|       | Tagging bot                                 |   :white_check_mark:    |        |           |          |
+|       | Actions                                     |   :white_check_mark:    |        |           |          |          |
-|       | Actions                                     |   :white_check_mark:    |        |           |          |
+|       | Web server                                  |       |        |           |          |          |  :white_check_mark:   |
-|       |                                             |        |        |      |      |
+|       |                                             |        |        |      |      |      |
-| CORE  | PR compression                              |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:       |          |
+| CORE  | PR compression                              |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:       | :white_check_mark: |   :white_check_mark:       | :white_check_mark:       |
-|       | Repo language prioritization                |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:       |          |
+|       | Repo language prioritization                |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:       | :white_check_mark: |   :white_check_mark:       | :white_check_mark:       |
-|       | Adaptive and token-aware<br />file patch fitting |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:       |          |
+|       | Adaptive and token-aware<br />file patch fitting |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:       | :white_check_mark: |   :white_check_mark:       | :white_check_mark:       |
-|       | Multiple models support |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:       |          |
+|       | Multiple models support |   :white_check_mark:    |   :white_check_mark:    |   :white_check_mark:       | :white_check_mark: |   :white_check_mark:       | :white_check_mark:       |
-|       | Incremental PR Review |   :white_check_mark:    |      |      |      |
+|       | Incremental PR Review |   :white_check_mark:    |      |      |      |      |      |
 
-Examples for invoking the different tools via the CLI:
+Review the **[usage guide](./Usage.md)** section for detailed instructions how to use the different tools, select the relevant git provider (GitHub, Gitlab, Bitbucket,...), and adjust the configuration file to your needs.
-- **Review**:       python cli.py --pr_url=<pr_url>  review
-- **Describe**:     python cli.py --pr_url=<pr_url>  describe
-- **Improve**:      python cli.py --pr_url=<pr_url>  improve
-- **Ask**:          python cli.py --pr_url=<pr_url>  ask "Write me a poem about this PR"
-- **Reflect**:      python cli.py --pr_url=<pr_url>  reflect
-- **Update Changelog**:      python cli.py --pr_url=<pr_url>  update_changelog
-
-"<pr_url>" is the url of the relevant PR (for example: https://github.com/Codium-ai/pr-agent/pull/50).
-
-In the [configuration](./CONFIGURATION.md) file you can select your git provider (GitHub, Gitlab, Bitbucket), and further configure the different tools.
 
 ## Try it now
 
-Try GPT-4 powered PR-Agent on your public GitHub repository for free. Just mention `@CodiumAI-Agent` and add the desired command in any PR comment! The agent will generate a response based on your command.
+You can try GPT-4 powered PR-Agent, on your public GitHub repository, instantly. Just mention `@CodiumAI-Agent` and add the desired command in any PR comment. The agent will generate a response based on your command.
+For example, add a comment to any pull request with the following text:
+```
+@CodiumAI-Agent /review
+```
+and the agent will respond with a review of your PR
 
 ![Review generation process](https://www.codium.ai/images/demo-2.gif)
 
-To set up your own PR-Agent, see the [Installation](#installation) section
+
+To set up your own PR-Agent, see the [Installation](#installation) section below.
 
 ---
 
@@ -129,14 +146,15 @@ To get started with PR-Agent quickly, you first need to acquire two tokens:
 There are several ways to use PR-Agent:
 
 - [Method 1: Use Docker image (no installation required)](INSTALL.md#method-1-use-docker-image-no-installation-required)
-- [Method 2: Run as a GitHub Action](INSTALL.md#method-2-run-as-a-github-action)
+- [Method 2: Run from source](INSTALL.md#method-2-run-from-source)
-- [Method 3: Run from source](INSTALL.md#method-3-run-from-source)
+- [Method 3: Run as a GitHub Action](INSTALL.md#method-3-run-as-a-github-action)
 - [Method 4: Run as a polling server](INSTALL.md#method-4-run-as-a-polling-server)
   - Request reviews by tagging your GitHub user on a PR
 - [Method 5: Run as a GitHub App](INSTALL.md#method-5-run-as-a-github-app)
   - Allowing you to automate the review process on your private or public repositories
 - [Method 6: Deploy as a Lambda Function](INSTALL.md#method-6---deploy-as-a-lambda-function)
 - [Method 7: AWS CodeCommit](INSTALL.md#method-7---aws-codecommit-setup)
+- [Method 8: Run a GitLab webhook server](INSTALL.md#method-8---run-a-gitlab-webhook-server)
 
 ## How it works
 
@@ -154,7 +172,7 @@ Here are some advantages of PR-Agent:
 
 - We emphasize **real-life practical usage**. Each tool (review, improve, ask, ...) has a single GPT-4 call, no more. We feel that this is critical for realistic team usage - obtaining an answer quickly (~30 seconds) and affordably.
 - Our [PR Compression strategy](./PR_COMPRESSION.md)  is a core ability that enables to effectively tackle both short and long PRs.
-- Our JSON prompting strategy enables to have **modular, customizable tools**. For example, the '/review' tool categories can be controlled via the [configuration](./CONFIGURATION.md) file. Adding additional categories is easy and accessible.
+- Our JSON prompting strategy enables to have **modular, customizable tools**. For example, the '/review' tool categories can be controlled via the [configuration](pr_agent/settings/configuration.toml) file. Adding additional categories is easy and accessible.
 - We support **multiple git providers** (GitHub, Gitlab, Bitbucket, CodeCommit), **multiple ways** to use the tool (CLI, GitHub Action, GitHub App, Docker, ...), and **multiple models** (GPT-4, GPT-3.5, Anthropic, Cohere, Llama2).
 - We are open-source, and welcome contributions from the community.
 
@@ -177,8 +195,8 @@ Here are some advantages of PR-Agent:
 
 ## Similar Projects
 
-- [CodiumAI - Meaningful tests for busy devs](https://github.com/Codium-ai/codiumai-vscode-release)
+- [CodiumAI - Meaningful tests for busy devs](https://github.com/Codium-ai/codiumai-vscode-release) (although various capabilities are much more advanced in the CodiumAI IDE plugins)
 - [Aider - GPT powered coding in your terminal](https://github.com/paul-gauthier/aider)
 - [openai-pr-reviewer](https://github.com/coderabbitai/openai-pr-reviewer)
 - [CodeReview BOT](https://github.com/anc95/ChatGPT-CodeReview)
-- [AI-Maintainer](https://github.com/merwanehamadi/AI-Maintainer)
+- [AI-Maintainer](https://github.com/merwanehamadi/AI-Maintainer)

Usage.md

@@ -0,0 +1,182 @@
+## Usage guide
+
+### Table of Contents
+- [Introduction](#introduction)
+- [Working from a local repo (CLI)](#working-from-a-local-repo-cli)
+- [Online usage](#online-usage)
+- [Working with GitHub App](#working-with-github-app)
+- [Working with GitHub Action](#working-with-github-action)
+- [Appendix - additional configurations walkthrough](#appendix---additional-configurations-walkthrough)
+
+### Introduction
+
+There are 3 basic ways to invoke CodiumAI PR-Agent:
+1. Locally running a CLI command
+2. Online usage - by [commenting](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021901) on a PR
+3. Enabling PR-Agent tools to run automatically when a new PR is opened
+
+See the [installation guide](/INSTALL.md) for instructions on how to setup your own PR-Agent.
+
+Specifically, CLI commands can be issued by invoking a pre-built [docker image](/INSTALL.md#running-from-source), or by invoking a [locally cloned repo](INSTALL.md#method-2-run-from-source).
+
+For online usage, you will need to setup either a [GitHub App](INSTALL.md#method-5-run-as-a-github-app), or a [GitHub Action](INSTALL.md#method-3-run-as-a-github-action).
+GitHub App and GitHub Action also enable to run PR-Agent specific tool automatically when a new PR is opened.
+
+
+#### The configuration file
+The different tools and sub-tools used by CodiumAI PR-Agent are adjustable via the **[configuration file](pr_agent/settings/configuration.toml)**.
+In addition to general configuration options, each tool has its own configurations. For example, the `review` tool will use parameters from the [pr_reviewer](/pr_agent/settings/configuration.toml#L16) section in the configuration file.
+
+**git provider:**
+The [git_provider](pr_agent/settings/configuration.toml#L4) field in the configuration file determines the GIT provider that will be used by PR-Agent. Currently, the following providers are supported:
+`
+"github", "gitlab", "azure", "codecommit", "local"
+`
+
+[//]: # (** online usage:**)
+
+[//]: # (Options that are available in the configuration file can be specified at run time when calling actions. Two examples:)
+
+[//]: # (```)
+
+[//]: # (- /review --pr_reviewer.extra_instructions="focus on the file: ...")
+
+[//]: # (- /describe --pr_description.add_original_user_description=false -pr_description.extra_instructions="make sure to mention: ...")
+
+[//]: # (```)
+
+### Working from a local repo (CLI)
+When running from your local repo (CLI), your local configuration file will be used.
+
+Examples for invoking the different tools via the CLI:
+
+- **Review**:       `python cli.py --pr_url=<pr_url>  /review`
+- **Describe**:     `python cli.py --pr_url=<pr_url>  /describe`
+- **Improve**:      `python cli.py --pr_url=<pr_url>  /improve`
+- **Ask**:          `python cli.py --pr_url=<pr_url>  /ask "Write me a poem about this PR"`
+- **Reflect**:      `python cli.py --pr_url=<pr_url>  /reflect`
+- **Update Changelog**:      `python cli.py --pr_url=<pr_url>  /update_changelog`
+
+`<pr_url>` is the url of the relevant PR (for example: https://github.com/Codium-ai/pr-agent/pull/50).
+
+**Notes:**
+
+(1) in addition to editing your local configuration file, you can also change any configuration value by adding it to the command line:
+```
+python cli.py --pr_url=<pr_url>  /review --pr_reviewer.extra_instructions="focus on the file: ..."
+```
+
+(2) You can print results locally, without publishing them, by setting in `configuration.toml`:
+```
+[config]
+publish_output=true
+verbosity_level=2
+```
+This is useful for debugging or experimenting with the different tools.
+
+
+### Online usage
+
+Online usage means invoking PR-Agent tools by [comments](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021901) on a PR.
+Commands for invoking the different tools via comments:
+
+- **Review**:       `/review`
+- **Describe**:     `/describe`
+- **Improve**:      `/improve`
+- **Ask**:          `/ask "..."`
+- **Reflect**:      `/reflect`
+- **Update Changelog**:      `/update_changelog`
+
+
+To edit a specific configuration value, just add `--config_path=<value>` to any command.
+For example if you want to edit the `review` tool configurations, you can run:
+```
+/review --pr_reviewer.extra_instructions="..." --pr_reviewer.require_score_review=false
+```
+Any configuration value in [configuration file](pr_agent/settings/configuration.toml) file can be similarly edited.
+
+
+### Working with GitHub App
+When running PR-Agent from [GitHub App](INSTALL.md#method-5-run-as-a-github-app), the default configurations from a pre-built repo will be initially loaded.
+
+#### GitHub app automatic tools
+The [github_app](pr_agent/settings/configuration.toml#L56) section defines GitHub app specific configurations. 
+An important parameter is `pr_commands`, which is a list of tools that will be **run automatically when a new PR is opened**:
+```
+[github_app]
+pr_commands = [
+    "/describe --pr_description.add_original_user_description=true --pr_description.keep_original_user_title=true",
+    "/auto_review",
+]
+```
+This means that when a new PR is opened, PR-Agent will run the `describe` and `auto_review` tools.
+For the describe tool, the `add_original_user_description` and `keep_original_user_title` parameters will be set to true.
+
+However, you can override the default tool parameters by uploading a local configuration file called `.pr_agent.toml` to the root of your repo.
+For example, if your local `.pr_agent.toml` file contains:
+```
+[pr_description]
+add_original_user_description = false
+keep_original_user_title = false
+```
+When a new PR is opened, PR-Agent will run the `describe` tool with the above parameters.
+
+Note that a local `.pr_agent.toml` file enables you to edit and customize the default parameters of any tool, not just the ones that are run automatically.
+
+#### Editing the prompts
+The prompts for the various PR-Agent tools are defined in the `pr_agent/settings` folder.
+
+In practice, the prompts are loaded and stored as a standard setting object. 
+Hence, editing them is similar to editing any other configuration value - just place the relevant key in `.pr_agent.toml`file, and override the default value.
+
+For example, if you want to edit the prompts of the [describe](./pr_agent/settings/pr_description_prompts.toml) tool, you can add the following to your `.pr_agent.toml` file:
+```
+[pr_description_prompt]
+system="""
+...
+"""
+user="""
+...
+"""
+```
+Note that the new prompt will need to generate an output compatible with the relevant [post-process function](./pr_agent/tools/pr_description.py#L137).
+
+### Working with GitHub Action
+TBD
+
+### Appendix - additional configurations walkthrough
+
+#### Changing a model
+See [here](pr_agent/algo/__init__.py) for the list of available models.
+
+To use Llama2 model, for example, set:
+```
+[config]
+model = "replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1"
+[replicate]
+key = ...
+```
+(you can obtain a Llama2 key from [here](https://replicate.com/replicate/llama-2-70b-chat/api))
+
+Also review the [AiHandler](pr_agent/algo/ai_handler.py) file for instruction how to set keys for other models.
+
+#### Extra instructions
+All PR-Agent tools have a parameter called `extra_instructions`, that enables to add free-text extra instructions. Example usage:
+```
+/update_changelog --pr_update_changelog.extra_instructions="Make sure to update also the version ..."
+```
+
+#### Azure DevOps provider
+To use Azure DevOps provider use the following settings in configuration.toml:
+```
+[config]
+git_provider="azure"
+use_repo_settings_file=false
+```
+
+And use the following settings (you have to replace the values) in .secrets.toml:
+```
+[azure_devops]
+org = "https://dev.azure.com/YOUR_ORGANIZATION/"
+pat = "YOUR_PAT_TOKEN"
+```

docker/Dockerfile

@@ -18,6 +18,10 @@ FROM base as github_polling
 ADD pr_agent pr_agent
 CMD ["python", "pr_agent/servers/github_polling.py"]
 
+FROM base as gitlab_webhook
+ADD pr_agent pr_agent
+CMD ["python", "pr_agent/servers/gitlab_webhook.py"]
+
 FROM base as test
 ADD requirements-dev.txt .
 RUN pip install -r requirements-dev.txt && rm requirements-dev.txt

pr_agent/algo/pr_processing.py

@@ -1,17 +1,19 @@
 from __future__ import annotations
 
+import difflib
 import logging
+import re
 import traceback
-from typing import Callable, List, Tuple
+from typing import Any, Callable, List, Tuple
 
 from github import RateLimitExceededException
 
 from pr_agent.algo import MAX_TOKENS
 from pr_agent.algo.git_patch_processing import convert_to_hunks_with_lines_numbers, extend_patch, handle_patch_deletions
 from pr_agent.algo.language_handler import sort_files_by_main_languages
-from pr_agent.algo.token_handler import TokenHandler
+from pr_agent.algo.token_handler import TokenHandler, get_token_encoder
 from pr_agent.config_loader import get_settings
-from pr_agent.git_providers.git_provider import GitProvider
+from pr_agent.git_providers.git_provider import FilePatchInfo, GitProvider
 
 DELETED_FILES_ = "Deleted files:\n"
 
@@ -245,6 +247,99 @@ def _get_all_deployments(all_models: List[str]) -> List[str]:
     return all_deployments
 
 
+def find_line_number_of_relevant_line_in_file(diff_files: List[FilePatchInfo],
+                                              relevant_file: str,
+                                              relevant_line_in_file: str) -> Tuple[int, int]:
+    """
+    Find the line number and absolute position of a relevant line in a file.
+
+    Args:
+        diff_files (List[FilePatchInfo]): A list of FilePatchInfo objects representing the patches of files.
+        relevant_file (str): The name of the file where the relevant line is located.
+        relevant_line_in_file (str): The content of the relevant line.
+
+    Returns:
+        Tuple[int, int]: A tuple containing the line number and absolute position of the relevant line in the file.
+    """
+    position = -1
+    absolute_position = -1
+    re_hunk_header = re.compile(
+        r"^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@[ ]?(.*)")
+
+    for file in diff_files:
+        if file.filename.strip() == relevant_file:
+            patch = file.patch
+            patch_lines = patch.splitlines()
+
+            # try to find the line in the patch using difflib, with some margin of error
+            matches_difflib: list[str | Any] = difflib.get_close_matches(relevant_line_in_file,
+                                                                         patch_lines, n=3, cutoff=0.93)
+            if len(matches_difflib) == 1 and matches_difflib[0].startswith('+'):
+                relevant_line_in_file = matches_difflib[0]
+
+            delta = 0
+            start1, size1, start2, size2 = 0, 0, 0, 0
+            for i, line in enumerate(patch_lines):
+                if line.startswith('@@'):
+                    delta = 0
+                    match = re_hunk_header.match(line)
+                    start1, size1, start2, size2 = map(int, match.groups()[:4])
+                elif not line.startswith('-'):
+                    delta += 1
+
+                if relevant_line_in_file in line and line[0] != '-':
+                    position = i
+                    absolute_position = start2 + delta - 1
+                    break
+
+            if position == -1 and relevant_line_in_file[0] == '+':
+                no_plus_line = relevant_line_in_file[1:].lstrip()
+                for i, line in enumerate(patch_lines):
+                    if line.startswith('@@'):
+                        delta = 0
+                        match = re_hunk_header.match(line)
+                        start1, size1, start2, size2 = map(int, match.groups()[:4])
+                    elif not line.startswith('-'):
+                        delta += 1
+
+                    if no_plus_line in line and line[0] != '-':
+                        # The model might add a '+' to the beginning of the relevant_line_in_file even if originally
+                        # it's a context line
+                        position = i
+                        absolute_position = start2 + delta - 1
+                        break
+    return position, absolute_position
+
+
+def clip_tokens(text: str, max_tokens: int) -> str:
+    """
+    Clip the number of tokens in a string to a maximum number of tokens.
+
+    Args:
+        text (str): The string to clip.
+        max_tokens (int): The maximum number of tokens allowed in the string.
+
+    Returns:
+        str: The clipped string.
+    """
+    if not text:
+        return text
+
+    try:
+        encoder = get_token_encoder()
+        num_input_tokens = len(encoder.encode(text))
+        if num_input_tokens <= max_tokens:
+            return text
+        num_chars = len(text)
+        chars_per_token = num_chars / num_input_tokens
+        num_output_chars = int(chars_per_token * max_tokens)
+        clipped_text = text[:num_output_chars]
+        return clipped_text
+    except Exception as e:
+        logging.warning(f"Failed to clip tokens: {e}")
+        return text
+
+
 def get_pr_multi_diffs(git_provider: GitProvider,
                        token_handler: TokenHandler,
                        model: str,

pr_agent/algo/token_handler.py

@@ -21,7 +21,7 @@ class TokenHandler:
       method.
     """
 
-    def __init__(self, vars: dict, system, user):
+    def __init__(self, pr, vars: dict, system, user):
         """
         Initializes the TokenHandler object.
 
@@ -32,9 +32,9 @@ class TokenHandler:
         - user: The user string.
         """
         self.encoder = get_token_encoder()
-        self.prompt_tokens = self._get_system_user_tokens(self.encoder, vars, system, user)
+        self.prompt_tokens = self._get_system_user_tokens(pr, self.encoder, vars, system, user)
 
-    def _get_system_user_tokens(self, encoder, vars: dict, system, user):
+    def _get_system_user_tokens(self, pr, encoder, vars: dict, system, user):
         """
         Calculates the number of tokens in the system and user strings.
 

pr_agent/algo/utils.py

@@ -5,24 +5,14 @@ import json
 import logging
 import re
 import textwrap
-from dataclasses import dataclass
 from datetime import datetime
-from enum import Enum
+from typing import Any, List
-from typing import Any, List, Tuple, Optional
 
 import yaml
 from starlette_context import context
-
-from pr_agent.algo.token_handler import get_token_encoder
 from pr_agent.config_loader import get_settings, global_settings
 
 
-class EDIT_TYPE(Enum):
-    ADDED = 1
-    DELETED = 2
-    MODIFIED = 3
-    RENAMED = 4
-
 def get_setting(key: str) -> Any:
     try:
         key = key.upper()
@@ -304,108 +294,3 @@ def try_fix_yaml(review_text: str) -> dict:
         except:
             pass
     return data
-
-def clip_tokens(text: str, max_tokens: int) -> str:
-    """
-    Clip the number of tokens in a string to a maximum number of tokens.
-
-    Args:
-        text (str): The string to clip.
-        max_tokens (int): The maximum number of tokens allowed in the string.
-
-    Returns:
-        str: The clipped string.
-    """
-    if not text:
-        return text
-
-    try:
-        encoder = get_token_encoder()
-        num_input_tokens = len(encoder.encode(text))
-        if num_input_tokens <= max_tokens:
-            return text
-        num_chars = len(text)
-        chars_per_token = num_chars / num_input_tokens
-        num_output_chars = int(chars_per_token * max_tokens)
-        clipped_text = text[:num_output_chars]
-        return clipped_text
-    except Exception as e:
-        logging.warning(f"Failed to clip tokens: {e}")
-        return text
-
-
-@dataclass
-class FilePatchInfo:
-    base_file: str
-    head_file: str
-    patch: str
-    filename: str
-    tokens: int = -1
-    edit_type: EDIT_TYPE = EDIT_TYPE.MODIFIED
-    old_filename: str = None
-    language: Optional[str] = None
-
-
-def find_line_number_of_relevant_line_in_file(diff_files: List[FilePatchInfo],
-                                              relevant_file: str,
-                                              relevant_line_in_file: str) -> Tuple[int, int]:
-    """
-    Find the line number and absolute position of a relevant line in a file.
-
-    Args:
-        diff_files (List[FilePatchInfo]): A list of FilePatchInfo objects representing the patches of files.
-        relevant_file (str): The name of the file where the relevant line is located.
-        relevant_line_in_file (str): The content of the relevant line.
-
-    Returns:
-        Tuple[int, int]: A tuple containing the line number and absolute position of the relevant line in the file.
-    """
-    position = -1
-    absolute_position = -1
-    re_hunk_header = re.compile(
-        r"^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@[ ]?(.*)")
-
-    for file in diff_files:
-        if file.filename.strip() == relevant_file:
-            patch = file.patch
-            patch_lines = patch.splitlines()
-
-            # try to find the line in the patch using difflib, with some margin of error
-            matches_difflib: list[str | Any] = difflib.get_close_matches(relevant_line_in_file,
-                                                                         patch_lines, n=3, cutoff=0.93)
-            if len(matches_difflib) == 1 and matches_difflib[0].startswith('+'):
-                relevant_line_in_file = matches_difflib[0]
-
-            delta = 0
-            start1, size1, start2, size2 = 0, 0, 0, 0
-            for i, line in enumerate(patch_lines):
-                if line.startswith('@@'):
-                    delta = 0
-                    match = re_hunk_header.match(line)
-                    start1, size1, start2, size2 = map(int, match.groups()[:4])
-                elif not line.startswith('-'):
-                    delta += 1
-
-                if relevant_line_in_file in line and line[0] != '-':
-                    position = i
-                    absolute_position = start2 + delta - 1
-                    break
-
-            if position == -1 and relevant_line_in_file[0] == '+':
-                no_plus_line = relevant_line_in_file[1:].lstrip()
-                for i, line in enumerate(patch_lines):
-                    if line.startswith('@@'):
-                        delta = 0
-                        match = re_hunk_header.match(line)
-                        start1, size1, start2, size2 = map(int, match.groups()[:4])
-                    elif not line.startswith('-'):
-                        delta += 1
-
-                    if no_plus_line in line and line[0] != '-':
-                        # The model might add a '+' to the beginning of the relevant_line_in_file even if originally
-                        # it's a context line
-                        position = i
-                        absolute_position = start2 + delta - 1
-                        break
-    return position, absolute_position
-

pr_agent/git_providers/__init__.py

@@ -5,6 +5,8 @@ from pr_agent.git_providers.github_provider import GithubProvider
 from pr_agent.git_providers.gitlab_provider import GitLabProvider
 from pr_agent.git_providers.local_git_provider import LocalGitProvider
 from pr_agent.git_providers.azuredevops_provider import AzureDevopsProvider
+from pr_agent.git_providers.gerrit_provider import GerritProvider
+
 
 _GIT_PROVIDERS = {
     'github': GithubProvider,
@@ -12,7 +14,8 @@ _GIT_PROVIDERS = {
     'bitbucket': BitbucketProvider,
     'azure': AzureDevopsProvider,
     'codecommit': CodeCommitProvider,
-    'local' : LocalGitProvider
+    'local' : LocalGitProvider,
+    'gerrit': GerritProvider,
 }
 
 def get_git_provider():

pr_agent/git_providers/azuredevops_provider.py

@@ -13,9 +13,11 @@ try:
 except ImportError:
     AZURE_DEVOPS_AVAILABLE = False
 
+from ..algo.pr_processing import clip_tokens
 from ..config_loader import get_settings
-from ..algo.utils import load_large_diff, FilePatchInfo, EDIT_TYPE, clip_tokens
+from ..algo.utils import load_large_diff
 from ..algo.language_handler import is_valid_file
+from .git_provider import EDIT_TYPE, FilePatchInfo
 
 
 class AzureDevopsProvider:

pr_agent/git_providers/bitbucket_provider.py

@@ -8,8 +8,7 @@ from atlassian.bitbucket import Cloud
 from starlette_context import context
 
 from ..config_loader import get_settings
-from .git_provider import GitProvider
+from .git_provider import FilePatchInfo, GitProvider
-from ..algo.utils import FilePatchInfo
 
 
 class BitbucketProvider(GitProvider):

pr_agent/git_providers/codecommit_client.py

@@ -1,9 +1,6 @@
-try:  # Allow this module to be imported without requiring boto3
+import boto3
-    import boto3
+import botocore
-    import botocore
+
-except ModuleNotFoundError:
-    boto3 = None
-    botocore = None
 
 class CodeCommitDifferencesResponse:
     """
@@ -67,7 +64,7 @@ class CodeCommitClient:
         """
         Get the differences between two commits in CodeCommit.
 
-        Parameters:
+        Args:
         - repo_name: Name of the repository
         - destination_commit: Commit hash you want to merge into (the "before" hash) (usually on the main or master branch)
         - source_commit: Commit hash of the code you are adding (the "after" branch)
@@ -76,8 +73,8 @@ class CodeCommitClient:
         - List of CodeCommitDifferencesResponse objects
 
         Boto3 Documentation:
-        aws codecommit get-differences
+        - aws codecommit get-differences
-        https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_differences.html
+        - https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_differences.html
         """
         if self.boto_client is None:
             self._connect_boto_client()
@@ -93,7 +90,11 @@ class CodeCommitClient:
             ):
                 differences.extend(page.get("differences", []))
         except botocore.exceptions.ClientError as e:
-            raise ValueError(f"Failed to retrieve differences from CodeCommit PR #{self.pr_num}") from e
+            if e.response["Error"]["Code"] == 'RepositoryDoesNotExistException':
+                raise ValueError(f"CodeCommit cannot retrieve differences: Repository does not exist: {repo_name}") from e
+            raise ValueError(f"CodeCommit cannot retrieve differences for {source_commit}..{destination_commit}") from e
+        except Exception as e:
+            raise ValueError(f"CodeCommit cannot retrieve differences for {source_commit}..{destination_commit}") from e
 
         output = []
         for json in differences:
@@ -104,7 +105,7 @@ class CodeCommitClient:
         """
         Retrieve a file from CodeCommit.
 
-        Parameters:
+        Args:
         - repo_name: Name of the repository
         - file_path: Path to the file you are retrieving
         - sha_hash: Commit hash of the file you are retrieving
@@ -113,8 +114,8 @@ class CodeCommitClient:
         - File contents
 
         Boto3 Documentation:
-        aws codecommit get_file
+        - aws codecommit get_file
-        https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_file.html
+        - https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_file.html
         """
         if not file_path:
             return ""
@@ -125,6 +126,8 @@ class CodeCommitClient:
         try:
             response = self.boto_client.get_file(repositoryName=repo_name, commitSpecifier=sha_hash, filePath=file_path)
         except botocore.exceptions.ClientError as e:
+            if e.response["Error"]["Code"] == 'RepositoryDoesNotExistException':
+                raise ValueError(f"CodeCommit cannot retrieve PR: Repository does not exist: {repo_name}") from e
             # if the file does not exist, but is flagged as optional, then return an empty string
             if optional and e.response["Error"]["Code"] == 'FileDoesNotExistException':
                 return ""
@@ -136,19 +139,20 @@ class CodeCommitClient:
 
         return response.get("fileContent", "")
 
-    def get_pr(self, pr_number: int):
+    def get_pr(self, repo_name: str, pr_number: int):
         """
         Get a information about a CodeCommit PR.
 
-        Parameters:
+        Args:
+        - repo_name: Name of the repository
         - pr_number: The PR number you are requesting
 
         Returns:
         - CodeCommitPullRequestResponse object
 
         Boto3 Documentation:
-        aws codecommit get_pull_request
+        - aws codecommit get_pull_request
-        https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_pull_request.html
+        - https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_pull_request.html
         """
         if self.boto_client is None:
             self._connect_boto_client()
@@ -158,6 +162,8 @@ class CodeCommitClient:
         except botocore.exceptions.ClientError as e:
             if e.response["Error"]["Code"] == 'PullRequestDoesNotExistException':
                 raise ValueError(f"CodeCommit cannot retrieve PR: PR number does not exist: {pr_number}") from e
+            if e.response["Error"]["Code"] == 'RepositoryDoesNotExistException':
+                raise ValueError(f"CodeCommit cannot retrieve PR: Repository does not exist: {repo_name}") from e
             raise ValueError(f"CodeCommit cannot retrieve PR: {pr_number}: boto client error") from e
         except Exception as e:
             raise ValueError(f"CodeCommit cannot retrieve PR: {pr_number}") from e
@@ -167,35 +173,95 @@ class CodeCommitClient:
 
         return CodeCommitPullRequestResponse(response.get("pullRequest", {}))
 
-    def publish_comment(self, repo_name: str, pr_number: int, destination_commit: str, source_commit: str, comment: str):
+    def publish_description(self, pr_number: int, pr_title: str, pr_body: str):
         """
-        Publish a comment to a pull request
+        Set the title and description on a pull request
 
-        Parameters:
+        Args:
-        - repo_name: name of the repository
+        - pr_number: the AWS CodeCommit pull request number
-        - pr_number: number of the pull request
+        - pr_title: title of the pull request
-        - destination_commit: The commit hash you want to merge into (the "before" hash) (usually on the main or master branch)
+        - pr_body: body of the pull request
-        - source_commit: The commit hash of the code you are adding (the "after" branch)
-        - pr_comment: comment
 
         Returns:
         - None
 
         Boto3 Documentation:
-        aws codecommit post_comment_for_pull_request
+        - aws codecommit update_pull_request_title
-        https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/post_comment_for_pull_request.html
+        - https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/update_pull_request_title.html
+        - aws codecommit update_pull_request_description
+        - https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/update_pull_request_description.html
         """
         if self.boto_client is None:
             self._connect_boto_client()
 
         try:
-            self.boto_client.post_comment_for_pull_request(
+            self.boto_client.update_pull_request_title(pullRequestId=str(pr_number), title=pr_title)
-                pullRequestId=str(pr_number),
+            self.boto_client.update_pull_request_description(pullRequestId=str(pr_number), description=pr_body)
-                repositoryName=repo_name,
+        except botocore.exceptions.ClientError as e:
-                beforeCommitId=destination_commit,
+            if e.response["Error"]["Code"] == 'PullRequestDoesNotExistException':
-                afterCommitId=source_commit,
+                raise ValueError(f"PR number does not exist: {pr_number}") from e
-                content=comment,
+            if e.response["Error"]["Code"] == 'InvalidTitleException':
-            )
+                raise ValueError(f"Invalid title for PR number: {pr_number}") from e
+            if e.response["Error"]["Code"] == 'InvalidDescriptionException':
+                raise ValueError(f"Invalid description for PR number: {pr_number}") from e
+            if e.response["Error"]["Code"] == 'PullRequestAlreadyClosedException':
+                raise ValueError(f"PR is already closed: PR number: {pr_number}") from e
+            raise ValueError(f"Boto3 client error calling publish_description") from e
+        except Exception as e:
+            raise ValueError(f"Error calling publish_description") from e
+
+    def publish_comment(self, repo_name: str, pr_number: int, destination_commit: str, source_commit: str, comment: str, annotation_file: str = None, annotation_line: int = None):
+        """
+        Publish a comment to a pull request
+
+        Args:
+        - repo_name: name of the repository
+        - pr_number: number of the pull request
+        - destination_commit: The commit hash you want to merge into (the "before" hash) (usually on the main or master branch)
+        - source_commit: The commit hash of the code you are adding (the "after" branch)
+        - comment: The comment you want to publish
+        - annotation_file: The file you want to annotate (optional)
+        - annotation_line: The line number you want to annotate (optional)
+
+        Comment annotations for CodeCommit are different than GitHub.
+        CodeCommit only designates the starting line number for the comment.
+        It does not support the ending line number to highlight a range of lines.
+
+        Returns:
+        - None
+
+        Boto3 Documentation:
+        - aws codecommit post_comment_for_pull_request
+        - https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/post_comment_for_pull_request.html
+        """
+        if self.boto_client is None:
+            self._connect_boto_client()
+
+        try:
+            # If the comment has code annotations,
+            # then set the file path and line number in the location dictionary
+            if annotation_file and annotation_line:
+                self.boto_client.post_comment_for_pull_request(
+                    pullRequestId=str(pr_number),
+                    repositoryName=repo_name,
+                    beforeCommitId=destination_commit,
+                    afterCommitId=source_commit,
+                    content=comment,
+                    location={
+                        "filePath": annotation_file,
+                        "filePosition": annotation_line,
+                        "relativeFileVersion": "AFTER",
+                    },
+                )
+            else:
+                # The comment does not have code annotations
+                self.boto_client.post_comment_for_pull_request(
+                    pullRequestId=str(pr_number),
+                    repositoryName=repo_name,
+                    beforeCommitId=destination_commit,
+                    afterCommitId=source_commit,
+                    content=comment,
+                )
         except botocore.exceptions.ClientError as e:
             if e.response["Error"]["Code"] == 'RepositoryDoesNotExistException':
                 raise ValueError(f"Repository does not exist: {repo_name}") from e

pr_agent/git_providers/codecommit_provider.py

@@ -1,14 +1,16 @@
 import logging
 import os
+import re
 from collections import Counter
 from typing import List, Optional, Tuple
 from urllib.parse import urlparse
 
-from pr_agent.git_providers.codecommit_client import CodeCommitClient
-
 from ..algo.language_handler import is_valid_file, language_extension_map
-from ..algo.utils import EDIT_TYPE, FilePatchInfo, load_large_diff
+from ..algo.pr_processing import clip_tokens
-from .git_provider import GitProvider
+from ..algo.utils import load_large_diff
+from ..config_loader import get_settings
+from .git_provider import EDIT_TYPE, FilePatchInfo, GitProvider, IncrementalPR
+from pr_agent.git_providers.codecommit_client import CodeCommitClient
 
 
 class PullRequestCCMimic:
@@ -152,26 +154,63 @@ class CodeCommitProvider(GitProvider):
         return self.diff_files
 
     def publish_description(self, pr_title: str, pr_body: str):
-        return ""  # not implemented yet
+        try:
+            self.codecommit_client.publish_description(
+                pr_number=self.pr_num,
+                pr_title=pr_title,
+                pr_body=CodeCommitProvider._add_additional_newlines(pr_body),
+            )
+        except Exception as e:
+            raise ValueError(f"CodeCommit Cannot publish description for PR: {self.pr_num}") from e    
 
     def publish_comment(self, pr_comment: str, is_temporary: bool = False):
         if is_temporary:
             logging.info(pr_comment)
             return
 
+        pr_comment = CodeCommitProvider._remove_markdown_html(pr_comment)
+        pr_comment = CodeCommitProvider._add_additional_newlines(pr_comment)
+
         try:
             self.codecommit_client.publish_comment(
                 repo_name=self.repo_name,
-                pr_number=str(self.pr_num),
+                pr_number=self.pr_num,
                 destination_commit=self.pr.destination_commit,
                 source_commit=self.pr.source_commit,
                 comment=pr_comment,
             )
         except Exception as e:
-            raise ValueError(f"CodeCommit Cannot post comment for PR: {self.pr_num}") from e
+            raise ValueError(f"CodeCommit Cannot publish comment for PR: {self.pr_num}") from e
 
     def publish_code_suggestions(self, code_suggestions: list) -> bool:
-        return [""]  # not implemented yet
+        counter = 1
+        for suggestion in code_suggestions:
+            # Verify that each suggestion has the required keys
+            if not all(key in suggestion for key in ["body", "relevant_file", "relevant_lines_start"]):
+                logging.warning(f"Skipping code suggestion #{counter}: Each suggestion must have 'body', 'relevant_file', 'relevant_lines_start' keys")
+                continue
+       
+            # Publish the code suggestion to CodeCommit
+            try:
+                logging.debug(f"Code Suggestion #{counter} in file: {suggestion['relevant_file']}: {suggestion['relevant_lines_start']}")
+                self.codecommit_client.publish_comment(
+                    repo_name=self.repo_name,
+                    pr_number=self.pr_num,
+                    destination_commit=self.pr.destination_commit,
+                    source_commit=self.pr.source_commit,
+                    comment=suggestion["body"],
+                    annotation_file=suggestion["relevant_file"],
+                    annotation_line=suggestion["relevant_lines_start"],
+                )
+            except Exception as e:
+                raise ValueError(f"CodeCommit Cannot publish code suggestions for PR: {self.pr_num}") from e
+            
+            counter += 1
+
+        # The calling function passes in a list of code suggestions, and this function publishes each suggestion one at a time.
+        # If we were to return False here, the calling function will attempt to publish the same list of code suggestions again, one at a time.
+        # Since this function publishes the suggestions one at a time anyway, we always return True here to avoid the retry.        
+        return True
 
     def publish_labels(self, labels):
         return [""]  # not implemented yet
@@ -183,6 +222,7 @@ class CodeCommitProvider(GitProvider):
         return ""  # not implemented yet
 
     def publish_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
+        # https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/post_comment_for_compared_commit.html
         raise NotImplementedError("CodeCommit provider does not support publishing inline comments yet")
 
     def create_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
@@ -199,7 +239,7 @@ class CodeCommitProvider(GitProvider):
         Returns a dictionary of languages, containing the percentage of each language used in the PR.
 
         Returns:
-            dict: A dictionary where each key is a language name and the corresponding value is the percentage of that language in the PR.
+        - dict: A dictionary where each key is a language name and the corresponding value is the percentage of that language in the PR.
         """
         commit_files = self.get_files()
         filenames = [ item.filename for item in commit_files ]
@@ -243,18 +283,29 @@ class CodeCommitProvider(GitProvider):
         return self.codecommit_client.get_file(self.repo_name, settings_filename, self.pr.source_commit, optional=True)
 
     def add_eyes_reaction(self, issue_comment_id: int) -> Optional[int]:
+        logging.info("CodeCommit provider does not support eyes reaction yet")
         return True
 
     def remove_reaction(self, issue_comment_id: int, reaction_id: int) -> bool:
+        logging.info("CodeCommit provider does not support removing reactions yet")
         return True
 
     @staticmethod
     def _parse_pr_url(pr_url: str) -> Tuple[str, int]:
+        """
+        Parse the CodeCommit PR URL and return the repository name and PR number.
+
+        Args:
+        - pr_url: the full AWS CodeCommit pull request URL
+
+        Returns:
+        - Tuple[str, int]: A tuple containing the repository name and PR number.
+        """
         # Example PR URL:
         # https://us-east-1.console.aws.amazon.com/codesuite/codecommit/repositories/__MY_REPO__/pull-requests/123456"
         parsed_url = urlparse(pr_url)
 
-        if "us-east-1.console.aws.amazon.com" not in parsed_url.netloc:
+        if not CodeCommitProvider._is_valid_codecommit_hostname(parsed_url.netloc):
             raise ValueError(f"The provided URL is not a valid CodeCommit URL: {pr_url}")
 
         path_parts = parsed_url.path.strip("/").split("/")
@@ -277,17 +328,33 @@ class CodeCommitProvider(GitProvider):
 
         return repo_name, pr_number
 
+    @staticmethod
+    def _is_valid_codecommit_hostname(hostname: str) -> bool:
+        """
+        Check if the provided hostname is a valid AWS CodeCommit hostname.
+
+        This is not an exhaustive check of AWS region names, 
+        but instead uses a regex to check for matching AWS region patterns.
+
+        Args:
+        - hostname: the hostname to check
+
+        Returns:
+        - bool: True if the hostname is valid, False otherwise.
+        """
+        return re.match(r"^[a-z]{2}-(gov-)?[a-z]+-\d\.console\.aws\.amazon\.com$", hostname) is not None
+
     def _get_pr(self):
-        response = self.codecommit_client.get_pr(self.pr_num)
+        response = self.codecommit_client.get_pr(self.repo_name, self.pr_num)
 
         if len(response.targets) == 0:
             raise ValueError(f"No files found in CodeCommit PR: {self.pr_num}")
 
-        # TODO: implement support for multiple commits in one CodeCommit PR
+        # TODO: implement support for multiple targets in one CodeCommit PR
-        #       for now, we are only using the first commit in the PR
+        #       for now, we are only using the first target in the PR
         if len(response.targets) > 1:
             logging.warning(
-                "Multiple commits in one PR is not supported for CodeCommit yet. Continuing, using the first commit only..."
+                "Multiple targets in one PR is not supported for CodeCommit yet. Continuing, using the first target only..."
             )
 
         # Return our object that mimics PullRequest class from the PyGithub library
@@ -305,13 +372,52 @@ class CodeCommitProvider(GitProvider):
         return ""  # not implemented yet
 
     @staticmethod
-    def _get_edit_type(codecommit_change_type):
+    def _add_additional_newlines(body: str) -> str:
+        """
+        Replace single newlines in a PR body with double newlines.
+
+        CodeCommit Markdown does not seem to render as well as GitHub Markdown,
+        so we add additional newlines to the PR body to make it more readable in CodeCommit.
+
+        Args:
+        - body: the PR body
+
+        Returns:
+        - str: the PR body with the double newlines added
+        """
+        return re.sub(r'(?<!\n)\n(?!\n)', '\n\n', body)
+
+    @staticmethod
+    def _remove_markdown_html(comment: str) -> str:
+        """
+        Remove the HTML tags from a PR comment.
+
+        CodeCommit Markdown does not seem to render as well as GitHub Markdown,
+        so we remove the HTML tags from the PR comment to make it more readable in CodeCommit.
+
+        Args:
+        - comment: the PR comment
+
+        Returns:
+        - str: the PR comment with the HTML tags removed
+        """
+        comment = comment.replace("<details>", "")
+        comment = comment.replace("</details>", "")
+        comment = comment.replace("<summary>", "")
+        comment = comment.replace("</summary>", "")
+        return comment
+
+    @staticmethod
+    def _get_edit_type(codecommit_change_type: str):
         """
         Convert the CodeCommit change type string to the EDIT_TYPE enum.
         The CodeCommit change type string is returned from the get_differences SDK method.
 
+        Args:
+        - codecommit_change_type: the CodeCommit change type string
+
         Returns:
-            An EDIT_TYPE enum representing the modified, added, deleted, or renamed file in the PR diff.
+        - An EDIT_TYPE enum representing the modified, added, deleted, or renamed file in the PR diff.
         """
         t = codecommit_change_type.upper()
         edit_type = None
@@ -332,6 +438,12 @@ class CodeCommitProvider(GitProvider):
         The returned extensions will include the dot "." prefix,
         to accommodate for the dots in the existing language_extension_map settings.
         Filenames with no extension will return an empty string for the extension.
+
+        Args:
+        - filenames: a list of filenames
+
+        Returns:
+        - list: A list of file extensions, including the dot "." prefix.
         """
         extensions = []
         for filename in filenames:
@@ -348,6 +460,12 @@ class CodeCommitProvider(GitProvider):
         Return a dictionary containing the programming language name (as the key),
         and the percentage that language is used (as the value),
         given a list of file extensions.
+
+        Args:
+        - extensions: a list of file extensions
+
+        Returns:
+        - dict: A dictionary where each key is a language name and the corresponding value is the percentage of that language in the PR.
         """
         total_files = len(extensions)
         if total_files == 0:

pr_agent/git_providers/gerrit_provider.py

@@ -0,0 +1,393 @@
+import json
+import logging
+import os
+import pathlib
+import shutil
+import subprocess
+import uuid
+from collections import Counter, namedtuple
+from pathlib import Path
+from tempfile import mkdtemp, NamedTemporaryFile
+
+import requests
+import urllib3.util
+from git import Repo
+
+from pr_agent.config_loader import get_settings
+from pr_agent.git_providers.git_provider import GitProvider, FilePatchInfo, \
+    EDIT_TYPE
+from pr_agent.git_providers.local_git_provider import PullRequestMimic
+
+logger = logging.getLogger(__name__)
+
+
+def _call(*command, **kwargs) -> (int, str, str):
+    res = subprocess.run(
+        command,
+        stdout=subprocess.PIPE,
+        stderr=subprocess.PIPE,
+        check=True,
+        **kwargs,
+    )
+    return res.stdout.decode()
+
+
+def clone(url, directory):
+    logger.info("Cloning %s to %s", url, directory)
+    stdout = _call('git', 'clone', "--depth", "1", url, directory)
+    logger.info(stdout)
+
+
+def fetch(url, refspec, cwd):
+    logger.info("Fetching %s %s", url, refspec)
+    stdout = _call(
+        'git', 'fetch', '--depth', '2', url, refspec,
+        cwd=cwd
+    )
+    logger.info(stdout)
+
+
+def checkout(cwd):
+    logger.info("Checking out")
+    stdout = _call('git', 'checkout', "FETCH_HEAD", cwd=cwd)
+    logger.info(stdout)
+
+
+def show(*args, cwd=None):
+    logger.info("Show")
+    return _call('git', 'show', *args, cwd=cwd)
+
+
+def diff(*args, cwd=None):
+    logger.info("Diff")
+    patch = _call('git', 'diff', *args, cwd=cwd)
+    if not patch:
+        logger.warning("No changes found")
+        return
+    return patch
+
+
+def reset_local_changes(cwd):
+    logger.info("Reset local changes")
+    _call('git', 'checkout', "--force", cwd=cwd)
+
+
+def add_comment(url: urllib3.util.Url, refspec, message):
+    *_, patchset, changenum = refspec.rsplit("/")
+    message = "'" + message.replace("'", "'\"'\"'") + "'"
+    return _call(
+        "ssh",
+        "-p", str(url.port),
+        f"{url.auth}@{url.host}",
+        "gerrit", "review",
+        "--message", message,
+        # "--code-review", score,
+        f"{patchset},{changenum}",
+    )
+
+
+def list_comments(url: urllib3.util.Url, refspec):
+    *_, patchset, _ = refspec.rsplit("/")
+    stdout = _call(
+        "ssh",
+        "-p", str(url.port),
+        f"{url.auth}@{url.host}",
+        "gerrit", "query",
+        "--comments",
+        "--current-patch-set", patchset,
+        "--format", "JSON",
+    )
+    change_set, *_ = stdout.splitlines()
+    return json.loads(change_set)["currentPatchSet"]["comments"]
+
+
+def prepare_repo(url: urllib3.util.Url, project, refspec):
+    repo_url = (f"{url.scheme}://{url.auth}@{url.host}:{url.port}/{project}")
+
+    directory = pathlib.Path(mkdtemp())
+    clone(repo_url, directory),
+    fetch(repo_url, refspec, cwd=directory)
+    checkout(cwd=directory)
+    return directory
+
+
+def adopt_to_gerrit_message(message):
+    lines = message.splitlines()
+    buf = []
+    for line in lines:
+        line = line.replace("*", "").replace("``", "`")
+        line = line.strip()
+        if line.startswith('#'):
+            buf.append("\n" +
+                       line.replace('#', '').removesuffix(":").strip() +
+                       ":")
+            continue
+        elif line.startswith('-'):
+            buf.append(line.removeprefix('-').strip())
+            continue
+        else:
+            buf.append(line)
+    return "\n".join(buf).strip()
+
+
+def add_suggestion(src_filename, context: str, start, end: int):
+    with (
+        NamedTemporaryFile("w", delete=False) as tmp,
+        open(src_filename, "r") as src
+    ):
+        lines = src.readlines()
+        tmp.writelines(lines[:start - 1])
+        if context:
+            tmp.write(context)
+        tmp.writelines(lines[end:])
+
+    shutil.copy(tmp.name, src_filename)
+    os.remove(tmp.name)
+
+
+def upload_patch(patch, path):
+    patch_server_endpoint = get_settings().get(
+        'gerrit.patch_server_endpoint')
+    patch_server_token = get_settings().get(
+        'gerrit.patch_server_token')
+
+    response = requests.post(
+        patch_server_endpoint,
+        json={
+            "content": patch,
+            "path": path,
+        },
+        headers={
+            "Content-Type": "application/json",
+            "Authorization": f"Bearer {patch_server_token}",
+        }
+    )
+    response.raise_for_status()
+    patch_server_endpoint = patch_server_endpoint.rstrip("/")
+    return patch_server_endpoint + "/" + path
+
+
+class GerritProvider(GitProvider):
+
+    def __init__(self, key: str, incremental=False):
+        self.project, self.refspec = key.split(':')
+        assert self.project, "Project name is required"
+        assert self.refspec, "Refspec is required"
+        base_url = get_settings().get('gerrit.url')
+        assert base_url, "Gerrit URL is required"
+        user = get_settings().get('gerrit.user')
+        assert user, "Gerrit user is required"
+
+        parsed = urllib3.util.parse_url(base_url)
+        self.parsed_url = urllib3.util.parse_url(
+            f"{parsed.scheme}://{user}@{parsed.host}:{parsed.port}"
+        )
+
+        self.repo_path = prepare_repo(
+            self.parsed_url, self.project, self.refspec
+        )
+        self.repo = Repo(self.repo_path)
+        assert self.repo
+
+        self.pr = PullRequestMimic(self.get_pr_title(), self.get_diff_files())
+
+    def get_pr_title(self):
+        """
+        Substitutes the branch-name as the PR-mimic title.
+        """
+        return self.repo.branches[0].name
+
+    def get_issue_comments(self):
+        comments = list_comments(self.parsed_url, self.refspec)
+        Comments = namedtuple('Comments', ['reversed'])
+        Comment = namedtuple('Comment', ['body'])
+        return Comments([Comment(c['message']) for c in reversed(comments)])
+
+    def get_labels(self):
+        raise NotImplementedError(
+            'Getting labels is not implemented for the gerrit provider')
+
+    def add_eyes_reaction(self, issue_comment_id: int):
+        raise NotImplementedError(
+            'Adding reactions is not implemented for the gerrit provider')
+
+    def remove_reaction(self, issue_comment_id: int, reaction_id: int):
+        raise NotImplementedError(
+            'Removing reactions is not implemented for the gerrit provider')
+
+    def get_commit_messages(self):
+        return [self.repo.head.commit.message]
+
+    def get_repo_settings(self):
+        """
+        TODO: Implement support of .pr_agent.toml
+        """
+        return ""
+
+    def get_diff_files(self) -> list[FilePatchInfo]:
+        diffs = self.repo.head.commit.diff(
+            self.repo.head.commit.parents[0],  # previous commit
+            create_patch=True,
+            R=True
+        )
+
+        diff_files = []
+        for diff_item in diffs:
+            if diff_item.a_blob is not None:
+                original_file_content_str = (
+                    diff_item.a_blob.data_stream.read().decode('utf-8')
+                )
+            else:
+                original_file_content_str = ""  # empty file
+            if diff_item.b_blob is not None:
+                new_file_content_str = diff_item.b_blob.data_stream.read(). \
+                    decode('utf-8')
+            else:
+                new_file_content_str = ""  # empty file
+            edit_type = EDIT_TYPE.MODIFIED
+            if diff_item.new_file:
+                edit_type = EDIT_TYPE.ADDED
+            elif diff_item.deleted_file:
+                edit_type = EDIT_TYPE.DELETED
+            elif diff_item.renamed_file:
+                edit_type = EDIT_TYPE.RENAMED
+            diff_files.append(
+                FilePatchInfo(
+                    original_file_content_str,
+                    new_file_content_str,
+                    diff_item.diff.decode('utf-8'),
+                    diff_item.b_path,
+                    edit_type=edit_type,
+                    old_filename=None
+                    if diff_item.a_path == diff_item.b_path
+                    else diff_item.a_path
+                )
+            )
+        self.diff_files = diff_files
+        return diff_files
+
+    def get_files(self):
+        diff_index = self.repo.head.commit.diff(
+            self.repo.head.commit.parents[0],  # previous commit
+            R=True
+        )
+        # Get the list of changed files
+        diff_files = [item.a_path for item in diff_index]
+        return diff_files
+
+    def get_languages(self):
+        """
+        Calculate percentage of languages in repository. Used for hunk
+        prioritisation.
+        """
+        # Get all files in repository
+        filepaths = [Path(item.path) for item in
+                     self.repo.tree().traverse() if item.type == 'blob']
+        # Identify language by file extension and count
+        lang_count = Counter(
+            ext.lstrip('.') for filepath in filepaths for ext in
+            [filepath.suffix.lower()])
+        # Convert counts to percentages
+        total_files = len(filepaths)
+        lang_percentage = {lang: count / total_files * 100 for lang, count
+                           in lang_count.items()}
+        return lang_percentage
+
+    def get_pr_description_full(self):
+        return self.repo.head.commit.message
+
+    def get_user_id(self):
+        return self.repo.head.commit.author.email
+
+    def is_supported(self, capability: str) -> bool:
+        if capability in [
+            # 'get_issue_comments',
+            'create_inline_comment',
+            'publish_inline_comments',
+            'get_labels'
+        ]:
+            return False
+        return True
+
+    def split_suggestion(self, msg) -> tuple[str, str]:
+        is_code_context = False
+        description = []
+        context = []
+        for line in msg.splitlines():
+            if line.startswith('```suggestion'):
+                is_code_context = True
+                continue
+            if line.startswith('```'):
+                is_code_context = False
+                continue
+            if is_code_context:
+                context.append(line)
+            else:
+                description.append(
+                    line.replace('*', '')
+                )
+
+        return (
+            '\n'.join(description),
+            '\n'.join(context) + '\n' if context else ''
+        )
+
+    def publish_code_suggestions(self, code_suggestions: list):
+        msg = []
+        for suggestion in code_suggestions:
+            description, code = self.split_suggestion(suggestion['body'])
+            add_suggestion(
+                pathlib.Path(self.repo_path) / suggestion["relevant_file"],
+                code,
+                suggestion["relevant_lines_start"],
+                suggestion["relevant_lines_end"],
+            )
+            patch = diff(cwd=self.repo_path)
+            patch_id = uuid.uuid4().hex[0:4]
+            path = "/".join(["codium-ai", self.refspec, patch_id])
+            full_path = upload_patch(patch, path)
+            reset_local_changes(self.repo_path)
+            msg.append(f'* {description}\n{full_path}')
+
+        if msg:
+            add_comment(self.parsed_url, self.refspec, "\n".join(msg))
+            return True
+
+    def publish_comment(self, pr_comment: str, is_temporary: bool = False):
+        if not is_temporary:
+            msg = adopt_to_gerrit_message(pr_comment)
+            add_comment(self.parsed_url, self.refspec, msg)
+
+    def publish_description(self, pr_title: str, pr_body: str):
+        msg = adopt_to_gerrit_message(pr_body)
+        add_comment(self.parsed_url, self.refspec, pr_title + '\n' + msg)
+
+    def publish_inline_comments(self, comments: list[dict]):
+        raise NotImplementedError(
+            'Publishing inline comments is not implemented for the gerrit '
+            'provider')
+
+    def publish_inline_comment(self, body: str, relevant_file: str,
+                               relevant_line_in_file: str):
+        raise NotImplementedError(
+            'Publishing inline comments is not implemented for the gerrit '
+            'provider')
+
+    def create_inline_comment(self, body: str, relevant_file: str,
+                              relevant_line_in_file: str):
+        raise NotImplementedError(
+            'Creating inline comments is not implemented for the gerrit '
+            'provider')
+
+    def publish_labels(self, labels):
+        # Not applicable to the local git provider,
+        # but required by the interface
+        pass
+
+    def remove_initial_comment(self):
+        # remove repo, cloned in previous steps
+        # shutil.rmtree(self.repo_path)
+        pass
+
+    def get_pr_branch(self):
+        return self.repo.head

pr_agent/git_providers/git_provider.py

@@ -1,10 +1,28 @@
 import logging
 from abc import ABC, abstractmethod
+from dataclasses import dataclass
 
 # enum EDIT_TYPE (ADDED, DELETED, MODIFIED, RENAMED)
+from enum import Enum
 from typing import Optional
 
-from pr_agent.algo.utils import FilePatchInfo
+
+class EDIT_TYPE(Enum):
+    ADDED = 1
+    DELETED = 2
+    MODIFIED = 3
+    RENAMED = 4
+
+
+@dataclass
+class FilePatchInfo:
+    base_file: str
+    head_file: str
+    patch: str
+    filename: str
+    tokens: int = -1
+    edit_type: EDIT_TYPE = EDIT_TYPE.MODIFIED
+    old_filename: str = None
 
 
 class GitProvider(ABC):
@@ -68,11 +86,11 @@ class GitProvider(ABC):
     def get_pr_description_full(self) -> str:
         pass
 
-    def get_pr_description(self) -> str:
+    def get_pr_description(self, *, full: bool = True) -> str:
         from pr_agent.config_loader import get_settings
-        from pr_agent.algo.utils import clip_tokens
+        from pr_agent.algo.pr_processing import clip_tokens
         max_tokens = get_settings().get("CONFIG.MAX_DESCRIPTION_TOKENS", None)
-        description = self.get_pr_description_full()
+        description = self.get_pr_description_full() if full else self.get_user_description()
         if max_tokens:
             return clip_tokens(description, max_tokens)
         return description
@@ -143,7 +161,6 @@ def get_main_pr_language(languages, files) -> str:
                 most_common_extension == 'scala' and top_language == 'scala' or \
                 most_common_extension == 'kt' and top_language == 'kotlin' or \
                 most_common_extension == 'pl' and top_language == 'perl' or \
-                most_common_extension == 'swift' and top_language == 'swift' or \
                 most_common_extension == top_language:
             main_language_str = top_language
 

pr_agent/git_providers/github_provider.py

@@ -9,9 +9,10 @@ from github import AppAuthentication, Auth, Github, GithubException, Reaction
 from retry import retry
 from starlette_context import context
 
-from .git_provider import GitProvider, IncrementalPR
+from .git_provider import FilePatchInfo, GitProvider, IncrementalPR
 from ..algo.language_handler import is_valid_file
-from ..algo.utils import load_large_diff, clip_tokens, find_line_number_of_relevant_line_in_file, FilePatchInfo
+from ..algo.utils import load_large_diff
+from ..algo.pr_processing import find_line_number_of_relevant_line_in_file, clip_tokens
 from ..config_loader import get_settings
 from ..servers.utils import RateLimitExceeded
 

pr_agent/git_providers/gitlab_provider.py

@@ -7,9 +7,10 @@ import gitlab
 from gitlab import GitlabGetError
 
 from ..algo.language_handler import is_valid_file
-from ..algo.utils import load_large_diff, clip_tokens, EDIT_TYPE, FilePatchInfo
+from ..algo.pr_processing import clip_tokens
+from ..algo.utils import load_large_diff
 from ..config_loader import get_settings
-from .git_provider import GitProvider
+from .git_provider import EDIT_TYPE, FilePatchInfo, GitProvider
 
 logger = logging.getLogger()
 

pr_agent/git_providers/in_memory_provider.py

@@ -1,79 +0,0 @@
-import itertools
-from collections import Counter
-from typing import List, Optional
-
-from pr_agent.algo.utils import FilePatchInfo
-from pr_agent.git_providers.git_provider import GitProvider
-
-
-class InMemoryProvider(GitProvider):
-    def __init__(self, head_branch: str, target_branch: str, files: List[FilePatchInfo]):
-        self.head_branch = head_branch
-        self.target_branch = target_branch
-        self.files = files
-
-    def is_supported(self, capability: str) -> bool:
-        pass
-
-    def get_files(self) -> list[FilePatchInfo]:
-        return self.files
-
-    def get_diff_files(self) -> list[FilePatchInfo]:
-        return self.get_files()
-
-    def publish_description(self, pr_title: str, pr_body: str):
-        pass
-
-    def publish_comment(self, pr_comment: str, is_temporary: bool = False):
-        pass
-
-    def publish_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
-        pass
-
-    def create_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
-        pass
-
-    def publish_inline_comments(self, comments: list[dict]):
-        pass
-
-    def publish_code_suggestions(self, code_suggestions: list) -> bool:
-        pass
-
-    def publish_labels(self, labels):
-        pass
-
-    def get_labels(self):
-        pass
-
-    def remove_initial_comment(self):
-        pass
-
-    def get_languages(self):
-        language_count = Counter(file.language for file in self.files)
-        return dict(language_count)
-
-    def get_pr_branch(self):
-        pass
-
-    def get_user_id(self):
-        pass
-
-    def get_pr_description_full(self) -> str:
-        pass
-
-    def get_issue_comments(self):
-        pass
-
-    def get_repo_settings(self):
-        pass
-
-    def add_eyes_reaction(self, issue_comment_id: int) -> Optional[int]:
-        pass
-
-    def remove_reaction(self, issue_comment_id: int, reaction_id: int) -> bool:
-        pass
-
-    def get_commit_messages(self):
-        pass
-
-

pr_agent/git_providers/local_git_provider.py

@@ -6,8 +6,7 @@ from typing import List
 from git import Repo
 
 from pr_agent.config_loader import _find_repository_root, get_settings
-from pr_agent.git_providers.git_provider import GitProvider
+from pr_agent.git_providers.git_provider import EDIT_TYPE, FilePatchInfo, GitProvider
-from pr_agent.algo.utils import EDIT_TYPE, FilePatchInfo
 
 
 class PullRequestMimic:

pr_agent/servers/gerrit_server.py

@@ -0,0 +1,78 @@
+import copy
+import logging
+import sys
+from enum import Enum
+from json import JSONDecodeError
+
+import uvicorn
+from fastapi import APIRouter, FastAPI, HTTPException
+from pydantic import BaseModel
+from starlette.middleware import Middleware
+from starlette_context import context
+from starlette_context.middleware import RawContextMiddleware
+
+from pr_agent.agent.pr_agent import PRAgent
+from pr_agent.config_loader import global_settings, get_settings
+
+logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
+router = APIRouter()
+
+
+class Action(str, Enum):
+    review = "review"
+    describe = "describe"
+    ask = "ask"
+    improve = "improve"
+    reflect = "reflect"
+    answer = "answer"
+
+
+class Item(BaseModel):
+    refspec: str
+    project: str
+    msg: str
+
+
+@router.post("/api/v1/gerrit/{action}")
+async def handle_gerrit_request(action: Action, item: Item):
+    logging.debug("Received a Gerrit request")
+    context["settings"] = copy.deepcopy(global_settings)
+
+    if action == Action.ask:
+        if not item.msg:
+            return HTTPException(
+                status_code=400,
+                detail="msg is required for ask command"
+            )
+    await PRAgent().handle_request(
+        f"{item.project}:{item.refspec}",
+        f"/{item.msg.strip()}"
+    )
+
+
+async def get_body(request):
+    try:
+        body = await request.json()
+    except JSONDecodeError as e:
+        logging.error("Error parsing request body", e)
+        return {}
+    return body
+
+
+@router.get("/")
+async def root():
+    return {"status": "ok"}
+
+
+def start():
+    # to prevent adding help messages with the output
+    get_settings().set("CONFIG.CLI_MODE", True)
+    middleware = [Middleware(RawContextMiddleware)]
+    app = FastAPI(middleware=middleware)
+    app.include_router(router)
+
+    uvicorn.run(app, host="0.0.0.0", port=3000)
+
+
+if __name__ == '__main__':
+    start()

pr_agent/servers/github_app.py

@@ -12,6 +12,7 @@ from starlette_context import context
 from starlette_context.middleware import RawContextMiddleware
 
 from pr_agent.agent.pr_agent import PRAgent
+from pr_agent.algo.utils import update_settings_from_args
 from pr_agent.config_loader import get_settings, global_settings
 from pr_agent.git_providers import get_git_provider
 from pr_agent.servers.utils import verify_signature
@@ -97,6 +98,7 @@ async def handle_request(body: Dict[str, Any], event: str):
             api_url = body["comment"]["pull_request_url"]
         else:
             return {}
+        logging.info(body)
         logging.info(f"Handling comment because of event={event} and action={action}")
         comment_id = body.get("comment", {}).get("id")
         provider = get_git_provider()(pr_url=api_url)
@@ -123,8 +125,14 @@ async def handle_request(body: Dict[str, Any], event: str):
                     return {}
             logging.info(f"Performing review because of event={event} and action={action}")
             for command in get_settings().github_app.pr_commands:
-                logging.info(f"Performing command: {command}")
+                split_command = command.split(" ")
-                await agent.handle_request(api_url, command)
+                command = split_command[0]
+                args = split_command[1:]
+                other_args = update_settings_from_args(args)
+                new_command = ' '.join([command] + other_args)
+                logging.info(body)
+                logging.info(f"Performing command: {new_command}")
+                await agent.handle_request(api_url, new_command)
 
     logging.info("event or action does not require handling")
     return {}

pr_agent/servers/gitlab_webhook.py

@@ -1,21 +1,51 @@
+import copy
+import json
 import logging
+import sys
 
 import uvicorn
 from fastapi import APIRouter, FastAPI, Request, status
 from fastapi.encoders import jsonable_encoder
 from fastapi.responses import JSONResponse
 from starlette.background import BackgroundTasks
+from starlette.middleware import Middleware
+from starlette_context import context
+from starlette_context.middleware import RawContextMiddleware
 
 from pr_agent.agent.pr_agent import PRAgent
-from pr_agent.config_loader import get_settings
+from pr_agent.config_loader import get_settings, global_settings
+from pr_agent.secret_providers import get_secret_provider
 
-app = FastAPI()
+logging.basicConfig(stream=sys.stdout, level=logging.INFO)
 router = APIRouter()
 
+secret_provider = get_secret_provider() if get_settings().get("CONFIG.SECRET_PROVIDER") else None
+
 
 @router.post("/webhook")
 async def gitlab_webhook(background_tasks: BackgroundTasks, request: Request):
+    if request.headers.get("X-Gitlab-Token") and secret_provider:
+        request_token = request.headers.get("X-Gitlab-Token")
+        secret = secret_provider.get_secret(request_token)
+        try:
+            secret_dict = json.loads(secret)
+            gitlab_token = secret_dict["gitlab_token"]
+            context["settings"] = copy.deepcopy(global_settings)
+            context["settings"].gitlab.personal_access_token = gitlab_token
+        except Exception as e:
+            logging.error(f"Failed to validate secret {request_token}: {e}")
+            return JSONResponse(status_code=status.HTTP_401_UNAUTHORIZED, content=jsonable_encoder({"message": "unauthorized"}))
+    elif get_settings().get("GITLAB.SHARED_SECRET"):
+        secret = get_settings().get("GITLAB.SHARED_SECRET")
+        if not request.headers.get("X-Gitlab-Token") == secret:
+            return JSONResponse(status_code=status.HTTP_401_UNAUTHORIZED, content=jsonable_encoder({"message": "unauthorized"}))
+    else:
+        return JSONResponse(status_code=status.HTTP_401_UNAUTHORIZED, content=jsonable_encoder({"message": "unauthorized"}))
+    gitlab_token = get_settings().get("GITLAB.PERSONAL_ACCESS_TOKEN", None)
+    if not gitlab_token:
+        return JSONResponse(status_code=status.HTTP_401_UNAUTHORIZED, content=jsonable_encoder({"message": "unauthorized"}))
     data = await request.json()
+    logging.info(json.dumps(data))
     if data.get('object_kind') == 'merge_request' and data['object_attributes'].get('action') in ['open', 'reopen']:
         logging.info(f"A merge request has been opened: {data['object_attributes'].get('title')}")
         url = data['object_attributes'].get('url')
@@ -28,16 +58,18 @@ async def gitlab_webhook(background_tasks: BackgroundTasks, request: Request):
             background_tasks.add_task(PRAgent().handle_request, url, body)
     return JSONResponse(status_code=status.HTTP_200_OK, content=jsonable_encoder({"message": "success"}))
 
+
+@router.get("/")
+async def root():
+    return {"status": "ok"}
+
 def start():
     gitlab_url = get_settings().get("GITLAB.URL", None)
     if not gitlab_url:
         raise ValueError("GITLAB.URL is not set")
-    gitlab_token = get_settings().get("GITLAB.PERSONAL_ACCESS_TOKEN", None)
-    if not gitlab_token:
-        raise ValueError("GITLAB.PERSONAL_ACCESS_TOKEN is not set")
     get_settings().config.git_provider = "gitlab"
-
+    middleware = [Middleware(RawContextMiddleware)]
-    app = FastAPI()
+    app = FastAPI(middleware=middleware)
     app.include_router(router)
 
     uvicorn.run(app, host="0.0.0.0", port=3000)

pr_agent/settings/configuration.toml

@@ -84,4 +84,14 @@ polling_interval_seconds = 30
 [local]
 # LocalGitProvider settings - uncomment to use paths other than default
 # description_path= "path/to/description.md"
-# review_path= "path/to/review.md"
+# review_path= "path/to/review.md"
+
+[gerrit]
+# endpoint to the gerrit service
+# url = "ssh://gerrit.example.com:29418"
+# user for gerrit authentication
+# user = "ai-reviewer"
+# patch server where patches will be saved
+# patch_server_endpoint = "http://127.0.0.1:5000/patch"
+# token to authenticate in the patch server
+# patch_server_token = ""

pr_agent/settings/pr_code_suggestions_prompts.toml

@@ -68,12 +68,17 @@ Code suggestions:
       type: string
       description: |-
         a code snippet showing the relevant code lines from a '__new hunk__' section.
-        It must be continuous, correctly formatted and indented, and without line numbers.
+        It must be contiguous, correctly formatted and indented, and without line numbers.
-    relevant lines:
+    relevant lines start:
-      type: string
+      type: integer
       description: |-
-        the relevant lines from a '__new hunk__' section, in the format of 'start_line-end_line'.
+        The relevant line number from a '__new hunk__' section where the suggestion starts (inclusive).
-        For example: '10-15'. They should be derived from the hunk line numbers, and correspond to the 'existing code' snippet above.
+        Should be derived from the hunk line numbers, and correspond to the 'existing code' snippet above.
+    relevant lines end:
+      type: integer
+      description: |-
+        The relevant line number from a '__new hunk__' section where the suggestion ends (inclusive).
+        Should be derived from the hunk line numbers, and correspond to the 'existing code' snippet above.
     improved code:
       type: string
       description: |-
@@ -90,7 +95,8 @@ Code suggestions:
         Add a docstring to func1()
     existing code: |-
         def func1():
-    relevant lines: '12-12'
+    relevant lines start: 12
+    relevant lines end: 12
     improved code: |-
         ...
 ```

pr_agent/tools/pr_code_suggestions.py

@@ -42,7 +42,9 @@ class PRCodeSuggestions:
             "extra_instructions": get_settings().pr_code_suggestions.extra_instructions,
             "commit_messages_str": self.git_provider.get_commit_messages(),
         }
-        self.token_handler = TokenHandler(self.vars, get_settings().pr_code_suggestions_prompt.system,
+        self.token_handler = TokenHandler(self.git_provider.pr,
+                                          self.vars,
+                                          get_settings().pr_code_suggestions_prompt.system,
                                           get_settings().pr_code_suggestions_prompt.user)
 
     async def run(self):
@@ -111,11 +113,8 @@ class PRCodeSuggestions:
                 if get_settings().config.verbosity_level >= 2:
                     logging.info(f"suggestion: {d}")
                 relevant_file = d['relevant file'].strip()
-                relevant_lines_str = d['relevant lines'].strip()
+                relevant_lines_start = int(d['relevant lines start'])  # absolute position
-                if ',' in relevant_lines_str:  # handling 'relevant lines': '181, 190' or '178-184, 188-194'
+                relevant_lines_end = int(d['relevant lines end'])
-                    relevant_lines_str = relevant_lines_str.split(',')[0]
-                relevant_lines_start = int(relevant_lines_str.split('-')[0])  # absolute position
-                relevant_lines_end = int(relevant_lines_str.split('-')[-1])
                 content = d['suggestion content']
                 new_code_snippet = d['improved code']
 

pr_agent/tools/pr_description.py

@@ -36,7 +36,7 @@ class PRDescription:
         self.vars = {
             "title": self.git_provider.pr.title,
             "branch": self.git_provider.get_pr_branch(),
-            "description": self.git_provider.get_pr_description(),
+            "description": self.git_provider.get_pr_description(full=False),
             "language": self.main_pr_language,
             "diff": "",  # empty diff for initial calculation
             "extra_instructions": get_settings().pr_description.extra_instructions,
@@ -46,8 +46,12 @@ class PRDescription:
         self.user_description = self.git_provider.get_user_description()
     
         # Initialize the token handler
-        self.token_handler = TokenHandler(self.vars, get_settings().pr_description_prompt.system,
+        self.token_handler = TokenHandler(
-                                          get_settings().pr_description_prompt.user)
+            self.git_provider.pr,
+            self.vars,
+            get_settings().pr_description_prompt.system,
+            get_settings().pr_description_prompt.user,
+        )
     
         # Initialize patches_diff and prediction attributes
         self.patches_diff = None

pr_agent/tools/pr_information_from_user.py

@@ -26,7 +26,9 @@ class PRInformationFromUser:
             "diff": "",  # empty diff for initial calculation
             "commit_messages_str": self.git_provider.get_commit_messages(),
         }
-        self.token_handler = TokenHandler(self.vars, get_settings().pr_information_from_user_prompt.system,
+        self.token_handler = TokenHandler(self.git_provider.pr,
+                                          self.vars,
+                                          get_settings().pr_information_from_user_prompt.system,
                                           get_settings().pr_information_from_user_prompt.user)
         self.patches_diff = None
         self.prediction = None

pr_agent/tools/pr_questions.py

@@ -29,7 +29,9 @@ class PRQuestions:
             "questions": self.question_str,
             "commit_messages_str": self.git_provider.get_commit_messages(),
         }
-        self.token_handler = TokenHandler(self.vars, get_settings().pr_questions_prompt.system,
+        self.token_handler = TokenHandler(self.git_provider.pr,
+                                          self.vars,
+                                          get_settings().pr_questions_prompt.system,
                                           get_settings().pr_questions_prompt.user)
         self.patches_diff = None
         self.prediction = None

pr_agent/tools/pr_reviewer.py

@@ -9,7 +9,8 @@ from jinja2 import Environment, StrictUndefined
 from yaml import SafeLoader
 
 from pr_agent.algo.ai_handler import AiHandler
-from pr_agent.algo.pr_processing import get_pr_diff, retry_with_fallback_models
+from pr_agent.algo.pr_processing import get_pr_diff, retry_with_fallback_models, \
+    find_line_number_of_relevant_line_in_file, clip_tokens
 from pr_agent.algo.token_handler import TokenHandler
 from pr_agent.algo.utils import convert_to_markdown, try_fix_json, try_fix_yaml, load_yaml
 from pr_agent.config_loader import get_settings
@@ -65,8 +66,12 @@ class PRReviewer:
             "commit_messages_str": self.git_provider.get_commit_messages(),
         }
 
-        self.token_handler = TokenHandler(self.vars, get_settings().pr_review_prompt.system,
+        self.token_handler = TokenHandler(
-                                          get_settings().pr_review_prompt.user)
+            self.git_provider.pr,
+            self.vars,
+            get_settings().pr_review_prompt.system,
+            get_settings().pr_review_prompt.user
+        )
 
     def parse_args(self, args: List[str]) -> None:
         """
@@ -212,8 +217,8 @@ class PRReviewer:
         markdown_text = convert_to_markdown(data)
         user = self.git_provider.get_user_id()
 
-        # Add help text if not in CLI§ mode
+        # Add help text if not in CLI mode
-        if not get_settings().get("CONFIG.CLI§_MODE", False):
+        if not get_settings().get("CONFIG.CLI_MODE", False):
             markdown_text += "\n### How to use\n"
             if user and '[bot]' not in user:
                 markdown_text += bot_help_text(user)

pr_agent/tools/pr_update_changelog.py

@@ -40,7 +40,9 @@ class PRUpdateChangelog:
             "extra_instructions": get_settings().pr_update_changelog.extra_instructions,
             "commit_messages_str": self.git_provider.get_commit_messages(),
         }
-        self.token_handler = TokenHandler(self.vars, get_settings().pr_update_changelog_prompt.system,
+        self.token_handler = TokenHandler(self.git_provider.pr,
+                                          self.vars,
+                                          get_settings().pr_update_changelog_prompt.system,
                                           get_settings().pr_update_changelog_prompt.user)
 
     async def run(self):

pyproject.toml

@@ -35,12 +35,12 @@ dependencies = {file = ["requirements.txt"]}
 "Homepage" = "https://github.com/Codium-ai/pr-agent"
 
 [tool.setuptools]
-include-package-data = true
+include-package-data = false
 license-files = ["LICENSE"]
 
 [tool.setuptools.packages.find]
 where = ["."]
-include = ["pr_agent", "pr_agent.*"]
+include = ["pr_agent"]
 
 [project.scripts]
 pr-agent = "pr_agent.cli:run"

requirements.txt

@@ -1,19 +1,21 @@
-dynaconf~=3.1.12
+dynaconf==3.1.12
-fastapi~=0.103.0
+fastapi==0.99.0
-PyGithub~=1.59.0
+PyGithub==1.59.*
-retry~=0.9.2
+retry==0.9.2
-openai~=0.27.8
+openai==0.27.8
-Jinja2~=3.1.2
+Jinja2==3.1.2
-tiktoken~=0.4.0
+tiktoken==0.4.0
-uvicorn~=0.22.0
+uvicorn==0.22.0
-python-gitlab~=3.15.0
+python-gitlab==3.15.0
 pytest~=7.4.0
 aiohttp~=3.8.4
-atlassian-python-api~=3.39.0
+atlassian-python-api==3.39.0
 GitPython~=3.1.32
-PyYAML~=6.0
+PyYAML==6.0
-starlette-context~=0.3.6
+starlette-context==0.3.6
-litellm~=0.1.445
+litellm~=0.1.504
 boto3~=1.28.25
-google-cloud-storage~=2.10.0
+google-cloud-storage==2.10.0
-ujson~=5.8.0
+ujson==5.8.0
+azure-devops==7.1.0b3
+msrest==0.7.1

tests/unittest/test_codecommit_client.py

@@ -125,7 +125,7 @@ class TestCodeCommitProvider:
             }
         }
 
-        pr = api.get_pr(321)
+        pr = api.get_pr("my_test_repo", 321)
 
         assert pr.title == "My PR"
         assert pr.description == "My PR description"

tests/unittest/test_codecommit_provider.py

@@ -1,7 +1,7 @@
 import pytest
 from pr_agent.git_providers.codecommit_provider import CodeCommitFile
 from pr_agent.git_providers.codecommit_provider import CodeCommitProvider
-from pr_agent.algo.utils import EDIT_TYPE
+from pr_agent.git_providers.git_provider import EDIT_TYPE
 
 
 class TestCodeCommitFile:
@@ -26,11 +26,48 @@ class TestCodeCommitFile:
 
 class TestCodeCommitProvider:
     def test_parse_pr_url(self):
+        # Test that the _parse_pr_url() function can extract the repo name and PR number from a CodeCommit URL
         url = "https://us-east-1.console.aws.amazon.com/codesuite/codecommit/repositories/my_test_repo/pull-requests/321"
         repo_name, pr_number = CodeCommitProvider._parse_pr_url(url)
         assert repo_name == "my_test_repo"
         assert pr_number == 321
 
+    def test_is_valid_codecommit_hostname(self):
+        # Test the various AWS regions
+        assert CodeCommitProvider._is_valid_codecommit_hostname("af-south-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("ap-east-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("ap-northeast-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("ap-northeast-2.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("ap-northeast-3.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("ap-south-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("ap-south-2.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("ap-southeast-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("ap-southeast-2.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("ap-southeast-3.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("ap-southeast-4.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("ca-central-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("eu-central-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("eu-central-2.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("eu-north-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("eu-south-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("eu-south-2.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("eu-west-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("eu-west-2.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("eu-west-3.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("il-central-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("me-central-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("me-south-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("sa-east-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("us-east-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("us-east-2.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("us-gov-east-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("us-gov-west-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("us-west-1.console.aws.amazon.com")
+        assert CodeCommitProvider._is_valid_codecommit_hostname("us-west-2.console.aws.amazon.com")
+        # Test non-AWS regions
+        assert not CodeCommitProvider._is_valid_codecommit_hostname("no-such-region.console.aws.amazon.com")
+        assert not CodeCommitProvider._is_valid_codecommit_hostname("console.aws.amazon.com")
+
     # Test that an error is raised when an invalid CodeCommit URL is provided to the set_pr() method of the CodeCommitProvider class.
     # Generated by CodiumAI
     def test_invalid_codecommit_url(self):
@@ -106,6 +143,7 @@ class TestCodeCommitProvider:
         assert percentages == {}
 
     def test_get_edit_type(self):
+        # Test that the _get_edit_type() function can convert a CodeCommit letter to an EDIT_TYPE enum
         assert CodeCommitProvider._get_edit_type("A") == EDIT_TYPE.ADDED
         assert CodeCommitProvider._get_edit_type("D") == EDIT_TYPE.DELETED
         assert CodeCommitProvider._get_edit_type("M") == EDIT_TYPE.MODIFIED
@@ -117,3 +155,18 @@ class TestCodeCommitProvider:
         assert CodeCommitProvider._get_edit_type("r") == EDIT_TYPE.RENAMED
 
         assert CodeCommitProvider._get_edit_type("X") is None
+
+    def test_add_additional_newlines(self):
+        # a short string to test adding double newlines
+        input = "abc\ndef\n\n___\nghi\njkl\nmno\n\npqr\n"
+        expect = "abc\n\ndef\n\n___\n\nghi\n\njkl\n\nmno\n\npqr\n\n"
+        assert CodeCommitProvider._add_additional_newlines(input) == expect
+        # a test example from a real PR
+        input = "## PR Type:\nEnhancement\n\n___\n## PR Description:\nThis PR introduces a new feature to the script, allowing users to filter servers by name.\n\n___\n## PR Main Files Walkthrough:\n`foo`: The foo script has been updated to include a new command line option `-f` or `--filter`.\n`bar`: The bar script has been updated to list stopped servers.\n"
+        expect = "## PR Type:\n\nEnhancement\n\n___\n\n## PR Description:\n\nThis PR introduces a new feature to the script, allowing users to filter servers by name.\n\n___\n\n## PR Main Files Walkthrough:\n\n`foo`: The foo script has been updated to include a new command line option `-f` or `--filter`.\n\n`bar`: The bar script has been updated to list stopped servers.\n\n"
+        assert CodeCommitProvider._add_additional_newlines(input) == expect
+
+    def test_remove_markdown_html(self):
+        input = "## PR Feedback\n<details><summary>Code feedback:</summary>\nfile foo\n</summary>\n"
+        expect = "## PR Feedback\nCode feedback:\nfile foo\n\n"
+        assert CodeCommitProvider._remove_markdown_html(input) == expect

tests/unittest/test_find_line_number_of_relevant_line_in_file.py

@@ -1,6 +1,8 @@
 
 # Generated by CodiumAI
-from pr_agent.algo.utils import FilePatchInfo, find_line_number_of_relevant_line_in_file
+from pr_agent.git_providers.git_provider import FilePatchInfo
+from pr_agent.algo.pr_processing import find_line_number_of_relevant_line_in_file
+
 
 import pytest