Add Azure OpenAI support

Remove installation_id from cli

.dockerignore

@@ -1 +1,2 @@
-venv/
+venv/
+pr_agent/settings/.secrets.toml

PR_COMPRESSION.md

@@ -0,0 +1,42 @@
+# Git Patch Logic
+There are two scenarios:
+1. The PR is small enough to fit in a single prompt (including system and user prompt)
+2. The PR is too large to fit in a single prompt (including system and user prompt)
+
+For both scenarios, we first use the following strategy
+#### Repo language prioritization strategy
+
+We prioritize the languages of the repo based on the following criteria:
+1. Exclude binary files and non code files (e.g. images, pdfs, etc)
+2. Given the main languages used in the repo
+2. We sort the PR files by the most common languages in the repo (in descending order): 
+   * ```[[file.py, file2.py],[file3.js, file4.jsx],[readme.md]]```
+   
+
+## Small PR
+In this case, we can fit the entire PR in a single prompt:
+1. Exclude binary files and non code files (e.g. images, pdfs, etc)
+2. We Expand the surrounding context of each patch to 6 lines above and below the patch
+## Large PR
+
+### Motivation
+Pull Requests can be very long and contain a lot of information with varying degree of relevance to the pr-agent.
+We want to be able to pack as much information as possible in a single LMM prompt, while keeping the information relevant to the pr-agent.
+
+
+
+#### PR compression strategy
+We prioritize additions over deletions:
+ - Combine all deleted files into a single list (`deleted files`)
+ - File patches are a list of hunks, remove all hunks of type deletion-only from the hunks in the file patch
+####  Adaptive and token-aware file patch fitting
+We use [tiktoken](https://github.com/openai/tiktoken) to tokenize the patches after the modifications described above, and we use the following strategy to fit the patches into the prompt:
+1. Withing each language we sort the files by the number of tokens in the file (in descending order):
+   * ```[[file2.py, file.py],[file4.jsx, file3.js],[readme.md]]```
+2. Iterate through the patches in the order described above
+2. Add the patches to the prompt until the prompt reaches a certain buffer from the max token length
+3. If there are still patches left, add the remaining patches as a list called `other modified files` to the prompt until the prompt reaches the max token length (hard stop), skip the rest of the patches.
+4. If we haven't reached the max token length, add the `deleted files` to the prompt until the prompt reaches the max token length (hard stop), skip the rest of the patches.
+
+### Example
+![](./pics/git_patch_logic.png)

README.md

@@ -1,51 +1,66 @@
 <div align="center">
 
-# 🛡️ CodiumAI PR-Agent
+<div align="center">
+
+<img src="./pics/logo-dark.png#gh-dark-mode-only" width="250"/>
+<img src="./pics/logo-light.png#gh-light-mode-only" width="250"/>
+
+</div>
+
 [![GitHub license](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://github.com/Codium-ai/pr-agent/blob/main/LICENSE)
 [![Discord](https://badgen.net/badge/icon/discord?icon=discord&label&color=purple)](https://discord.com/channels/1057273017547378788/1126104260430528613)
 
-CodiumAI `PR-Agent` is an open-source tool that helps developers review PRs faster and more efficiently. 
-It automatically analyzes the PR, and provides feedback and suggestions, and can answer questions. 
-It is powered by GPT-4, and is based on the [CodiumAI](https://github.com/Codium-ai/) platform.
+CodiumAI `pr-agent` is an open-source tool aiming to help developers review PRs faster and more efficiently. It automatically analyzes the PR, provides feedback and suggestions, and can answer free-text questions.
+
 </div>
 
+- [Live demo](#live-demo)
+- [Quickstart](#Quickstart)
+- [Usage and tools](#usage-and-tools)
+- [Configuration](#Configuration)
+- [How it works](#how-it-works)
+- [Roadmap](#roadmap)
+- [Similar projects](#similar-projects)
 
+## Live demo
 
-* [Quickstart](#Quickstart)
-* [Configuration](#Configuration)
-* [Usage and Tools](#usage-and-tools)
-* [Roadmap](#roadmap)
-* [Similar projects](#similar-projects)
+Experience GPT-4 powered PR review on your public GitHub repository with our hosted pr-agent. To try it, just mention `@CodiumAI-Agent` in any PR comment! The agent will generate a PR review in response.
 
+![Review generation process](./pics/pr-agent-review-process1.gif)
 
-## Quickstart
-
-To get started with PR-Agent quickly, you first need to acquire two tokens:
-1. An OpenAI key from [here](https://platform.openai.com/), with access to GPT-4.
-2. A GitHub personal access token (classic) with the repo scope.
-
-There are several ways to use PR-Agent. Let's start with the simplest one:
+To set up your own pr-agent, see the [Quickstart](#Quickstart) section
 
 ---
 
-### Method 1: Use Docker image (no installation required)
+## Quickstart
 
-To request a review for a PR, or ask a question about a PR, you can run the appropriate
-Python scripts from the scripts folder. Here's how:
+To get started with pr-agent quickly, you first need to acquire two tokens:
+
+1. An OpenAI key from [here](https://platform.openai.com/), with access to GPT-4.
+2. A GitHub personal access token (classic) with the repo scope.
+
+There are several ways to use pr-agent. Let's start with the simplest one:
+
+---
+
+#### Method 1: Use Docker image (no installation required)
+
+To request a review for a PR, or ask a question about a PR, you can run directly from the Docker image. Here's how:
 
 1. To request a review for a PR, run the following command:
+
 ```
 docker run --rm -it -e OPENAI.KEY=<your key> -e GITHUB.USER_TOKEN=<your token> codiumai/pr-agent --pr_url <pr url>
 ```
 
----
-
 2. To ask a question about a PR, run the following command:
+
 ```
 docker run --rm -it -e OPENAI.KEY=<your key> -e GITHUB.USER_TOKEN=<your token> codiumai/pr-agent --pr_url <pr url> --question "<your question>"
 ```
 
 Possible questions you can ask include:
+
 - What is the main theme of this PR?
 - Is the PR ready for merge?
 - What are the main changes in this PR?
@@ -54,25 +69,29 @@ Possible questions you can ask include:
 
 ---
 
-### Method 2: Run from source
+#### Method 2: Run from source
 
 1. Clone this repository:
+
 ```
 git clone https://github.com/Codium-ai/pr-agent.git
 ```
 
 2. Install the requirements in your favorite virtual environment:
+
 ```
 pip install -r requirements.txt
 ```
 
 3. Copy the secrets template file and fill in your OpenAI key and your GitHub user token:
+
 ```
-cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets
-# Edit .secrets file
+cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets.toml
+# Edit .secrets.toml file
 ```
 
 4. Run the appropriate Python scripts from the scripts folder:
+
 ```
 python pr_agent/cli.py --pr_url <pr url>
 python pr_agent/cli.py --pr_url <pr url> --question "<your question>"
@@ -80,19 +99,21 @@ python pr_agent/cli.py --pr_url <pr url> --question "<your question>"
 
 ---
 
-### Method 3: Method 3: Run as a polling server; request reviews by tagging your Github user on a PR
+#### Method 3: Method 3: Run as a polling server; request reviews by tagging your Github user on a PR
 
 Follow steps 1-3 of method 2.
 Run the following command to start the server:
+
 ```
 python pr_agent/servers/github_polling.py
 ```
 
 ---
 
-### Method 4: Run as a Github App, allowing you to automate the review process on your private or public repositories.
+#### Method 4: Run as a Github App, allowing you to automate the review process on your private or public repositories.
 
 1. Create a GitHub App from the [Github Developer Portal](https://docs.github.com/en/developers/apps/creating-a-github-app).
+
    - Set the following permissions:
      - Pull requests: Read & write
      - Issue comment: Read & write
@@ -102,15 +123,18 @@ python pr_agent/servers/github_polling.py
      - Pull request
 
 2. Generate a random secret for your app, and save it for later. For example, you can use:
+
 ```
 WEBHOOK_SECRET=$(python -c "import secrets; print(secrets.token_hex(10))")
 ```
 
 3. Acquire the following pieces of information from your app's settings page:
+
    - App private key (click "Generate a private key", and save the file)
    - App ID
 
 4. Clone this repository:
+
 ```
 git clone https://github.com/Codium-ai/pr-agent.git
 ```
@@ -121,21 +145,24 @@ git clone https://github.com/Codium-ai/pr-agent.git
    - Copy your app's private key to the private_key field.
    - Copy your app's ID to the app_id field.
    - Copy your app's webhook secret to the webhook_secret field.
+
 ```
-cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets
-# Edit .secrets file
+cp pr_agent/settings/.secrets_template.toml pr_agent/settings/.secrets.toml
+# Edit .secrets.toml file
 ```
 
 6. Build a Docker image for the app and optionally push it to a Docker repository. We'll use Dockerhub as an example:
+
 ```
 docker build . -t codiumai/pr-agent:github_app --target github_app -f docker/Dockerfile
 docker push codiumai/pr-agent:github_app  # Push to your Docker repository
 ```
 
-7. Host the app using a server, serverless function, or container environment. Alternatively, for development and 
-   debugging, you may use tools like smee.io to forward webhooks to your local machine. 
+7. Host the app using a server, serverless function, or container environment. Alternatively, for development and
+   debugging, you may use tools like smee.io to forward webhooks to your local machine.
 
 8. Go back to your app's settings, set the following:
+
    - Webhook URL: The URL of your app's server, or the URL of the smee.io channel.
    - Webhook secret: The secret you generated earlier.
 
@@ -144,11 +171,14 @@ docker push codiumai/pr-agent:github_app  # Push to your Docker repository
 ---
 
 ## Usage and Tools
-CodiumAI PR-Agent provides two types of interactions ("tools"): `"PR Reviewer"` and `"PR Q&A"`.
+
+CodiumAI pr-agent provides two types of interactions ("tools"): `"PR Reviewer"` and `"PR Q&A"`.
+
 - The "PR Reviewer" tool automatically analyzes PRs, and provides different types of feedbacks.
 - The "PR Q&A" tool answers free-text questions about the PR.
 
 ### PR Reviewer
+
 Here is a quick overview of the different sub-tools of PR Reviewer:
 
 - PR Analysis
@@ -156,48 +186,49 @@ Here is a quick overview of the different sub-tools of PR Reviewer:
   - PR description and title
   - PR type classification
   - Is the PR covered by relevant tests
-  - Is the PR minimal and focused
+  - Is this a focused PR
+  - Are there security concerns
 - PR Feedback
   - General PR suggestions
   - Code suggestions
-  - Security concerns
 
 This is how a typical output of the PR Reviewer looks like:
 
 ---
+
 #### PR Analysis
 
 - 🎯 **Main theme:** Adding language extension handler and token handler
 - 🔍 **Description and title:** Yes
 - 📌 **Type of PR:** Enhancement
 - 🧪 **Relevant tests added:** No
-- ✨ **Minimal and focused:** Yes, the PR is focused on adding two new handlers for language extension and token counting.
+- ✨ **Focused PR:** Yes, the PR is focused on adding two new handlers for language extension and token counting.
+- 🔒 **Security concerns:** No, the PR does not introduce possible security concerns or issues.
+
 #### PR Feedback
 
 - 💡 **General PR suggestions:** The PR is generally well-structured and the code is clean. However, it would be beneficial to add some tests to ensure the new handlers work as expected. Also, consider adding docstrings to the new functions and classes to improve code readability and maintainability.
 
 - 🤖 **Code suggestions:**
 
-- **suggestion 1:**
   - **relevant file:** pr_agent/algo/language_handler.py
-  - **suggestion content:** Consider using a set instead of a list for 'bad_extensions' as checking membership in a set is faster than in a list. [medium]
 
-- **suggestion 2:**
+    **suggestion content:** Consider using a set instead of a list for 'bad_extensions' as checking membership in a set is faster than in a list. [medium]
+
   - **relevant file:** pr_agent/algo/language_handler.py
-  - **suggestion content:** In the 'filter_bad_extensions' function, you are splitting the filename on '.' and taking the last element to get the extension. This might not work as expected if the filename contains multiple '.' characters. Consider using 'os.path.splitext' to get the file extension more reliably. [important]
 
-- 🔒 **Security concerns:** No, the PR does not introduce possible security concerns or issues.
+    **suggestion content:** In the 'filter_bad_extensions' function, you are splitting the filename on '.' and taking the last element to get the extension. This might not work as expected if the filename contains multiple '.' characters. Consider using 'os.path.splitext' to get the file extension more reliably. [important]
 
 ---
 
-
 ### PR Q&A
+
 This tool answers free-text questions about the PR. This is how a typical output of the PR Q&A looks like:
 
----
 **Question**: summarize for me the PR in 4 bullet points
 
-**Answer**: 
+**Answer**:
+
 - The PR introduces a new feature to sort files by their main languages. It uses a mapping of programming languages to their file extensions to achieve this.
 - It also introduces a filter to exclude files with certain extensions, deemed as 'bad extensions', from the sorting process.
 - The PR modifies the `get_pr_diff` function in `pr_processing.py` to use the new sorting function. It also refactors the code to move the PR pruning logic into a separate function.
@@ -206,57 +237,66 @@ This tool answers free-text questions about the PR. This is how a typical output
 ---
 
 ## Configuration
-The different tools and sub-tools used by CodiumAI PR-Agent are easily configurable via the configuration file: `/settings/configuration.toml`.
+
+The different tools and sub-tools used by CodiumAI pr-agent are easily configurable via the configuration file: `/settings/configuration.toml`.
+
 #### Enabling/disabling sub-tools:
-You can enable/disable the different PR Reviewer sub-sections  with the following flags:
+
+You can enable/disable the different PR Reviewer sub-sections with the following flags:
+
 ```
-require_minimal_and_focused_review=true
+require_focused_review=true
 require_tests_review=true
 require_security_review=true
 ```
+
 #### Code Suggestions configuration:
+
 There are also configuration options to control different aspects of the `code suggestions` feature.
 The number of suggestions provided can be controlled by adjusting the following parameter:
+
 ```
 num_code_suggestions=4
 ```
+
 You can also enable more verbose and informative mode of code suggestions:
+
 ```
 extended_code_suggestions=false
-``` 
+```
+
 This is a comparison of the regular and extended code suggestions modes:
 
----
-Example for regular suggestion:
-
-
-- **suggestion 1:**
-  - **relevant file:** sql.py
-  - **suggestion content:** Remove hardcoded sensitive information like username and password. Use environment variables or a secure method to store these values. [important]
----
+- **relevant file:** sql.py
+- **suggestion content:** Remove hardcoded sensitive information like username and password. Use environment variables or a secure method to store these values. [important]
 
 Example for extended suggestion:
 
+- **relevant file:** sql.py
+- **suggestion content:** Remove hardcoded sensitive information (username and password) [important]
+- **why:** Hardcoding sensitive information is a security risk. It's better to use environment variables or a secure way to store these values.
+- **code example:**
+  - **before code:**
+    ```
+    user = "root",
+    password = "Mysql@123",
+    ```
+  - **after code:**
+    ```
+    user = os.getenv('DB_USER'),
+    password = os.getenv('DB_PASSWORD'),
+    ```
 
-- **suggestion 1:**
-  - **relevant file:** sql.py
-  - **suggestion content:** Remove hardcoded sensitive information (username and password) [important]
-  - **why:** Hardcoding sensitive information is a security risk. It's better to use environment variables or a secure way to store these values.
-  - **code example:**
-    - **before code:**
-        ```
-        user = "root",
-        password = "Mysql@123",
-        ```
-    - **after code:**
-        ```
-        user = os.getenv('DB_USER'),
-        password = os.getenv('DB_PASSWORD'),
-        ```
 ---
 
+## How it works
+
+![PR-Agent Tools](./pics/pr_agent_overview.png)
+
+Check out the [PR Compression strategy](./PR_COMPRESSION.md) page for more details on how we convert a code diff to a manageable LLM prompt
 
 ## Roadmap
+
 - [ ] Support open-source models, as a replacement for openai models. Note that a minimal requirement for each open-source model is to have 8k+ context, and good support for generating json as an output
 - [ ] Support other Git providers, such as Gitlab and Bitbucket.
 - [ ] Develop additional logics for handling large PRs, and compressing git patches
@@ -271,7 +311,9 @@ Example for extended suggestion:
   - [ ] ...
 
 ## Similar Projects
+
 - [CodiumAI - Meaningful tests for busy devs](https://github.com/Codium-ai/codiumai-vscode-release)
 - [Aider - GPT powered coding in your terminal](https://github.com/paul-gauthier/aider)
 - [GPT-Engineer](https://github.com/AntonOsika/gpt-engineer)
 - [CodeReview BOT](https://github.com/anc95/ChatGPT-CodeReview)
+- [AI-Maintainer](https://github.com/merwanehamadi/AI-Maintainer)

pr_agent/agent/pr_agent.py

@@ -1,20 +1,24 @@
 import re
-from typing import Optional
 
 from pr_agent.tools.pr_questions import PRQuestions
 from pr_agent.tools.pr_reviewer import PRReviewer
 
 
 class PRAgent:
-    def __init__(self, installation_id: Optional[int] = None):
-        self.installation_id = installation_id
+    def __init__(self):
+        pass
 
     async def handle_request(self, pr_url, request):
-        if 'please review' in request.lower():
-            reviewer = PRReviewer(pr_url, self.installation_id)
+        if 'please review' in request.lower() or 'review' == request.lower().strip() or len(request) == 0:
+            reviewer = PRReviewer(pr_url)
             await reviewer.review()
 
-        elif 'please answer' in request.lower():
-            question = re.split(r'(?i)please answer', request)[1].strip()
-            answerer = PRQuestions(pr_url, question, self.installation_id)
+        else:
+            if "please answer" in request.lower():
+                question = re.split(r'(?i)please answer', request)[1].strip()
+            elif request.lower().strip().startswith("answer"):
+                question = re.split(r'(?i)answer', request)[1].strip()
+            else:
+                question = request
+            answerer = PRQuestions(pr_url, question)
             await answerer.answer()

pr_agent/algo/ai_handler.py

@@ -14,6 +14,13 @@ class AiHandler:
             openai.api_key = settings.openai.key
             if settings.get("OPENAI.ORG", None):
                 openai.organization = settings.openai.org
+            self.deployment_id = settings.get("OPENAI.DEPLOYMENT_ID", None)
+            if settings.get("OPENAI.API_TYPE", None):
+                openai.api_type = settings.openai.api_type
+            if settings.get("OPENAI.API_VERSION", None):
+                openai.engine = settings.openai.api_version
+            if settings.get("OPENAI.API_BASE", None):
+                openai.api_base = settings.openai.api_base
         except AttributeError as e:
             raise ValueError("OpenAI key is required") from e
 
@@ -23,6 +30,7 @@ class AiHandler:
         try:
             response = await openai.ChatCompletion.acreate(
                             model=model,
+                            deployment_id=self.deployment_id,
                             messages=[
                                 {"role": "system", "content": system},
                                 {"role": "user", "content": user}

pr_agent/algo/git_patch_processing.py

@@ -96,7 +96,7 @@ def handle_patch_deletions(patch: str, original_file_content_str: str,
         # logic for handling deleted files - don't show patch, just show that the file was deleted
         if settings.config.verbosity_level > 0:
             logging.info(f"Processing file: {file_name}, minimizing deletion file")
-        patch = "File was deleted\n"
+        patch = None # file was deleted
     else:
         patch_lines = patch.splitlines()
         patch_new = omit_deletion_hunks(patch_lines)

pr_agent/algo/language_handler.py

@@ -58,7 +58,8 @@ bad_extensions = [
     'woff2',
     'xz',
     'zip',
-    'zst'
+    'zst',
+    'snap'
 ]
 
 
@@ -92,7 +93,7 @@ def sort_files_by_main_languages(languages: Dict, files: list):
     for ext in main_extensions:
         main_extensions_flat.extend(ext)
 
-    for extensions, lang in zip(main_extensions, languages_sorted_list):
+    for extensions, lang in zip(main_extensions, languages_sorted_list):  # noqa: B905
         tmp = []
         for file in files_filtered:
             extension_str = f".{file.filename.split('.')[-1]}"

pr_agent/algo/pr_processing.py

@@ -2,7 +2,7 @@ from __future__ import annotations
 
 import difflib
 import logging
-from typing import Any, Dict, Tuple
+from typing import Any, Tuple, Union
 
 from pr_agent.algo.git_patch_processing import extend_patch, handle_patch_deletions
 from pr_agent.algo.language_handler import sort_files_by_main_languages
@@ -10,11 +10,16 @@ from pr_agent.algo.token_handler import TokenHandler
 from pr_agent.config_loader import settings
 from pr_agent.git_providers import GithubProvider
 
-OUTPUT_BUFFER_TOKENS = 800
+DELETED_FILES_ = "Deleted files:\n"
+
+MORE_MODIFIED_FILES_ = "More modified files:\n"
+
+OUTPUT_BUFFER_TOKENS_SOFT_THRESHOLD = 1000
+OUTPUT_BUFFER_TOKENS_HARD_THRESHOLD = 600
 PATCH_EXTRA_LINES = 3
 
 
-def get_pr_diff(git_provider: [GithubProvider, Any], token_handler: TokenHandler) -> str:
+def get_pr_diff(git_provider: Union[GithubProvider, Any], token_handler: TokenHandler) -> str:
     """
     Returns a string with the diff of the PR.
     If needed, apply diff minimization techniques to reduce the number of tokens
@@ -28,12 +33,20 @@ def get_pr_diff(git_provider: [GithubProvider, Any], token_handler: TokenHandler
     patches_extended, total_tokens = pr_generate_extended_diff(pr_languages, token_handler)
 
     # if we are under the limit, return the full diff
-    if total_tokens + OUTPUT_BUFFER_TOKENS < token_handler.limit:
+    if total_tokens + OUTPUT_BUFFER_TOKENS_SOFT_THRESHOLD < token_handler.limit:
         return "\n".join(patches_extended)
 
     # if we are over the limit, start pruning
-    patches_compressed = pr_generate_compressed_diff(pr_languages, token_handler)
-    return "\n".join(patches_compressed)
+    patches_compressed, modified_file_names, deleted_file_names = pr_generate_compressed_diff(pr_languages,
+                                                                                              token_handler)
+    final_diff = "\n".join(patches_compressed)
+    if modified_file_names:
+        modified_list_str = MORE_MODIFIED_FILES_ + "\n".join(modified_file_names)
+        final_diff = final_diff + "\n\n" + modified_list_str
+    if deleted_file_names:
+        deleted_list_str = DELETED_FILES_ + "\n".join(deleted_file_names)
+        final_diff = final_diff + "\n\n" + deleted_list_str
+    return final_diff
 
 
 def pr_generate_extended_diff(pr_languages: list, token_handler: TokenHandler) -> \
@@ -67,7 +80,7 @@ def pr_generate_extended_diff(pr_languages: list, token_handler: TokenHandler) -
     return patches_extended, total_tokens
 
 
-def pr_generate_compressed_diff(top_langs: list, token_handler: TokenHandler) -> list:
+def pr_generate_compressed_diff(top_langs: list, token_handler: TokenHandler) -> Tuple[list, list, list]:
     # Apply Diff Minimization techniques to reduce the number of tokens:
     # 0. Start from the largest diff patch to smaller ones
     # 1. Don't use extend context lines around diff
@@ -76,7 +89,8 @@ def pr_generate_compressed_diff(top_langs: list, token_handler: TokenHandler) ->
     # 4. Minimize all remaining files when you reach token limit
 
     patches = []
-
+    modified_files_list = []
+    deleted_files_list = []
     # sort each one of the languages in top_langs by the number of tokens in the diff
     sorted_files = []
     for lang in top_langs:
@@ -94,25 +108,40 @@ def pr_generate_compressed_diff(top_langs: list, token_handler: TokenHandler) ->
         # removing delete-only hunks
         patch = handle_patch_deletions(patch, original_file_content_str,
                                        new_file_content_str, file.filename)
+        if patch is None:
+            if not deleted_files_list:
+                total_tokens += token_handler.count_tokens(DELETED_FILES_)
+            deleted_files_list.append(file.filename)
+            total_tokens += token_handler.count_tokens(file.filename) + 1
+            continue
         new_patch_tokens = token_handler.count_tokens(patch)
 
-        if total_tokens > token_handler.limit - OUTPUT_BUFFER_TOKENS // 2:
+        # Hard Stop, no more tokens
+        if total_tokens > token_handler.limit - OUTPUT_BUFFER_TOKENS_HARD_THRESHOLD:
             logging.warning(f"File was fully skipped, no more tokens: {file.filename}.")
-            continue  # Hard Stop, no more tokens
-        if total_tokens + new_patch_tokens > token_handler.limit - OUTPUT_BUFFER_TOKENS:
+            continue
+
+        # If the patch is too large, just show the file name
+        if total_tokens + new_patch_tokens > token_handler.limit - OUTPUT_BUFFER_TOKENS_SOFT_THRESHOLD:
             # Current logic is to skip the patch if it's too large
             # TODO: Option for alternative logic to remove hunks from the patch to reduce the number of tokens
             #  until we meet the requirements
             if settings.config.verbosity_level >= 2:
                 logging.warning(f"Patch too large, minimizing it, {file.filename}")
-            patch = "File was modified"
+            if not modified_files_list:
+                total_tokens += token_handler.count_tokens(MORE_MODIFIED_FILES_)
+            modified_files_list.append(file.filename)
+            total_tokens += token_handler.count_tokens(file.filename) + 1
+            continue
+
         if patch:
             patch_final = f"## {file.filename}\n\n{patch}\n"
             patches.append(patch_final)
             total_tokens += token_handler.count_tokens(patch_final)
             if settings.config.verbosity_level >= 2:
                 logging.info(f"Tokens: {total_tokens}, last filename: {file.filename}")
-    return patches
+
+    return patches, modified_files_list, deleted_files_list
 
 
 def load_large_diff(file, new_file_content_str: str, original_file_content_str: str, patch: str) -> str:

pr_agent/algo/token_handler.py

@@ -21,4 +21,4 @@ class TokenHandler:
         return system_prompt_tokens + user_prompt_tokens
 
     def count_tokens(self, patch: str) -> int:
-        return len(self.encoder.encode(patch))
+        return len(self.encoder.encode(patch, disallowed_special=()))

pr_agent/algo/utils.py

@@ -1,5 +1,8 @@
 from __future__ import annotations
 
+import json
+import logging
+import re
 import textwrap
 
 
@@ -12,7 +15,7 @@ def convert_to_markdown(output_data: dict) -> str:
         "Type of PR": "📌",
         "Relevant tests added": "🧪",
         "Unrelated changes": "⚠️",
-        "Minimal and focused": "✨",
+        "Focused PR": "✨",
         "Security concerns": "🔒",
         "General PR suggestions": "💡",
         "Code suggestions": "🤖"
@@ -51,9 +54,35 @@ def parse_code_suggestion(code_suggestions: dict) -> str:
                 markdown_text += f"    - **{code_key}:**\n{code_str_indented}\n"
         else:
             if "suggestion number" in sub_key.lower():
-                markdown_text += f"- **suggestion {sub_value}:**\n"  # prettier formatting
+                # markdown_text += f"- **suggestion {sub_value}:**\n"  # prettier formatting
+                pass
+            elif "relevant file" in sub_key.lower():
+                markdown_text += f"\n  - **{sub_key}:** {sub_value}\n"
             else:
-                markdown_text += f"  - **{sub_key}:** {sub_value}\n"
+                markdown_text += f"   **{sub_key}:** {sub_value}\n"
+
     markdown_text += "\n"
     return markdown_text
 
+
+def try_fix_json(review, max_iter=10):
+    # Try to fix JSON if it is broken/incomplete: parse until the last valid code suggestion
+    data = {}
+    if review.rfind("'Code suggestions': [") > 0 or review.rfind('"Code suggestions": [') > 0:
+        last_code_suggestion_ind = [m.end() for m in re.finditer(r"\}\s*,", review)][-1] - 1
+        valid_json = False
+        iter_count = 0
+        while last_code_suggestion_ind > 0 and not valid_json and iter_count < max_iter:
+            try:
+                data = json.loads(review[:last_code_suggestion_ind] + "]}}")
+                valid_json = True
+                review = review[:last_code_suggestion_ind].strip() + "]}}"
+            except json.decoder.JSONDecodeError:
+                review = review[:last_code_suggestion_ind]
+                # Use regular expression to find the last occurrence of "}," with any number of whitespaces or newlines
+                last_code_suggestion_ind = [m.end() for m in re.finditer(r"\}\s*,", review)][-1] - 1
+                iter_count += 1
+        if not valid_json:
+            logging.error("Unable to decode JSON response from AI")
+            data = {}
+    return data

pr_agent/cli.py

@@ -15,11 +15,11 @@ def run():
     logging.basicConfig(level=os.environ.get("LOGLEVEL", "INFO"))
     if args.question:
         print(f"Question: {args.question} about PR {args.pr_url}")
-        reviewer = PRQuestions(args.pr_url, args.question, None)
+        reviewer = PRQuestions(args.pr_url, args.question)
         asyncio.run(reviewer.answer())
     else:
         print(f"Reviewing PR: {args.pr_url}")
-        reviewer = PRReviewer(args.pr_url, None)
+        reviewer = PRReviewer(args.pr_url, cli_mode=True)
         asyncio.run(reviewer.review())
 
 

pr_agent/config_loader.py

@@ -5,10 +5,12 @@ from dynaconf import Dynaconf
 current_dir = dirname(abspath(__file__))
 settings = Dynaconf(
     envvar_prefix=False,
+    merge_enabled=True,
     settings_files=[join(current_dir, f) for f in [
          "settings/.secrets.toml",
          "settings/configuration.toml",
          "settings/pr_reviewer_prompts.toml",
-         "settings/pr_questions_prompts.toml"
+         "settings/pr_questions_prompts.toml",
+         "settings_prod/.secrets.toml"
         ]]
 )

pr_agent/git_providers/__init__.py

@@ -1,15 +1,17 @@
 from pr_agent.config_loader import settings
 from pr_agent.git_providers.github_provider import GithubProvider
+from pr_agent.git_providers.gitlab_provider import GitLabProvider
 
 _GIT_PROVIDERS = {
-    'github': GithubProvider
+    'github': GithubProvider,
+    'gitlab': GitLabProvider,
 }
 
 def get_git_provider():
     try:
         provider_id = settings.config.git_provider
     except AttributeError as e:
-        raise ValueError("github_provider is a required attribute in the configuration file") from e
+        raise ValueError("git_provider is a required attribute in the configuration file") from e
     if provider_id not in _GIT_PROVIDERS:
         raise ValueError(f"Unknown git provider: {provider_id}")
     return _GIT_PROVIDERS[provider_id]

pr_agent/git_providers/git_provider.py

@@ -0,0 +1,82 @@
+from abc import ABC, abstractmethod
+from dataclasses import dataclass
+
+
+@dataclass
+class FilePatchInfo:
+    base_file: str
+    head_file: str
+    patch: str
+    filename: str
+    tokens: int = -1
+
+
+class GitProvider(ABC):
+    @abstractmethod
+    def get_diff_files(self) -> list[FilePatchInfo]:
+        pass
+
+    @abstractmethod
+    def publish_comment(self, pr_comment: str, is_temporary: bool = False):
+        pass
+
+    @abstractmethod
+    def remove_initial_comment(self):
+        pass
+
+    @abstractmethod
+    def get_languages(self):
+        pass
+
+    @abstractmethod
+    def get_pr_branch(self):
+        pass
+
+    @abstractmethod
+    def get_user_id(self):
+        pass
+
+    @abstractmethod
+    def get_pr_description(self):
+        pass
+
+
+def get_main_pr_language(languages, files) -> str:
+    """
+    Get the main language of the commit. Return an empty string if cannot determine.
+    """
+    main_language_str = ""
+    try:
+        top_language = max(languages, key=languages.get).lower()
+
+        # validate that the specific commit uses the main language
+        extension_list = []
+        for file in files:
+            extension_list.append(file.filename.rsplit('.')[-1])
+
+        # get the most common extension
+        most_common_extension = max(set(extension_list), key=extension_list.count)
+
+        # look for a match. TBD: add more languages, do this systematically
+        if most_common_extension == 'py' and top_language == 'python' or \
+                most_common_extension == 'js' and top_language == 'javascript' or \
+                most_common_extension == 'ts' and top_language == 'typescript' or \
+                most_common_extension == 'go' and top_language == 'go' or \
+                most_common_extension == 'java' and top_language == 'java' or \
+                most_common_extension == 'c' and top_language == 'c' or \
+                most_common_extension == 'cpp' and top_language == 'c++' or \
+                most_common_extension == 'cs' and top_language == 'c#' or \
+                most_common_extension == 'swift' and top_language == 'swift' or \
+                most_common_extension == 'php' and top_language == 'php' or \
+                most_common_extension == 'rb' and top_language == 'ruby' or \
+                most_common_extension == 'rs' and top_language == 'rust' or \
+                most_common_extension == 'scala' and top_language == 'scala' or \
+                most_common_extension == 'kt' and top_language == 'kotlin' or \
+                most_common_extension == 'pl' and top_language == 'perl' or \
+                most_common_extension == 'swift' and top_language == 'swift':
+            main_language_str = top_language
+
+    except Exception:
+        pass
+
+    return main_language_str

pr_agent/git_providers/github_provider.py

@@ -1,29 +1,23 @@
 import logging
-from collections import namedtuple
-from dataclasses import dataclass
 from datetime import datetime
 from typing import Optional, Tuple
 from urllib.parse import urlparse
 
-from github import AppAuthentication, File, Github
+from github import AppAuthentication, Github
 
 from pr_agent.config_loader import settings
 
-@dataclass
-class FilePatchInfo:
-    base_file: str
-    head_file: str
-    patch: str
-    filename: str
-    tokens: int = -1
+from .git_provider import FilePatchInfo
+
 
 class GithubProvider:
-    def __init__(self, pr_url: Optional[str] = None, installation_id: Optional[int] = None):
-        self.installation_id = installation_id
+    def __init__(self, pr_url: Optional[str] = None):
+        self.installation_id = settings.get("GITHUB.INSTALLATION_ID")
         self.github_client = self._get_github_client()
         self.repo = None
         self.pr_num = None
         self.pr = None
+        self.github_user_id = None
         if pr_url:
             self.set_pr(pr_url)
 
@@ -31,6 +25,9 @@ class GithubProvider:
         self.repo, self.pr_num = self._parse_pr_url(pr_url)
         self.pr = self._get_pr()
 
+    def get_files(self):
+        return self.pr.get_files()
+
     def get_diff_files(self) -> list[FilePatchInfo]:
         files = self.pr.get_files()
         diff_files = []
@@ -42,6 +39,8 @@ class GithubProvider:
 
     def publish_comment(self, pr_comment: str, is_temporary: bool = False):
         response = self.pr.create_issue_comment(pr_comment)
+        if hasattr(response, "user") and hasattr(response.user, "login"):
+            self.github_user_id = response.user.login
         response.is_temporary = is_temporary
         if not hasattr(self.pr, 'comments_list'):
             self.pr.comments_list = []
@@ -62,53 +61,23 @@ class GithubProvider:
         return self.pr.body
 
     def get_languages(self):
-        return self._get_repo().get_languages()
-
-    def get_main_pr_language(self) -> str:
-        """
-        Get the main language of the commit. Return an empty string if cannot determine.
-        """
-        main_language_str = ""
-        try:
-            languages = self.get_languages()
-            top_language = max(languages, key=languages.get).lower()
-
-            # validate that the specific commit uses the main language
-            extension_list = []
-            files = self.pr.get_files()
-            for file in files:
-                extension_list.append(file.filename.rsplit('.')[-1])
-
-            # get the most common extension
-            most_common_extension = max(set(extension_list), key=extension_list.count)
-
-            # look for a match. TBD: add more languages, do this systematically
-            if most_common_extension == 'py' and top_language == 'python' or \
-                    most_common_extension == 'js' and top_language == 'javascript' or \
-                    most_common_extension == 'ts' and top_language == 'typescript' or \
-                    most_common_extension == 'go' and top_language == 'go' or \
-                    most_common_extension == 'java' and top_language == 'java' or \
-                    most_common_extension == 'c' and top_language == 'c' or \
-                    most_common_extension == 'cpp' and top_language == 'c++' or \
-                    most_common_extension == 'cs' and top_language == 'c#' or \
-                    most_common_extension == 'swift' and top_language == 'swift' or \
-                    most_common_extension == 'php' and top_language == 'php' or \
-                    most_common_extension == 'rb' and top_language == 'ruby' or \
-                    most_common_extension == 'rs' and top_language == 'rust' or \
-                    most_common_extension == 'scala' and top_language == 'scala' or \
-                    most_common_extension == 'kt' and top_language == 'kotlin' or \
-                    most_common_extension == 'pl' and top_language == 'perl' or \
-                    most_common_extension == 'swift' and top_language == 'swift':
-                main_language_str = top_language
-
-        except Exception:
-            pass
-
-        return main_language_str
+        languages = self._get_repo().get_languages()
+        return languages
 
     def get_pr_branch(self):
         return self.pr.head.ref
 
+    def get_pr_description(self):
+        return self.pr.body
+
+    def get_user_id(self):
+        if not self.github_user_id:
+            try:
+                self.github_user_id = self.github_client.get_user().login
+            except Exception as e:
+                logging.exception(f"Failed to get user id, error: {e}")
+        return self.github_user_id
+
     def get_notifications(self, since: datetime):
         deployment_type = settings.get("GITHUB.DEPLOYMENT_TYPE", "user")
 

pr_agent/git_providers/gitlab_provider.py

@@ -0,0 +1,92 @@
+import logging
+from typing import Optional, Tuple
+from urllib.parse import urlparse
+
+import gitlab
+
+from pr_agent.config_loader import settings
+
+from .git_provider import FilePatchInfo, GitProvider
+
+
+class GitLabProvider(GitProvider):
+    def __init__(self, merge_request_url: Optional[str] = None):
+        gitlab_url = settings.get("GITLAB.URL", None)
+        if not gitlab_url:
+            raise ValueError("GitLab URL is not set in the config file")
+        gitlab_access_token = settings.get("GITLAB.PERSONAL_ACCESS_TOKEN", None)
+        if not gitlab_access_token:
+            raise ValueError("GitLab personal access token is not set in the config file")
+        self.gl = gitlab.Gitlab(
+            gitlab_url,
+            gitlab_access_token
+        )
+        self.id_project = None
+        self.id_mr = None
+        self.mr = None
+        self.temp_comments = []
+        self._set_merge_request(merge_request_url)
+
+    @property
+    def pr(self):
+        '''The GitLab terminology is merge request (MR) instead of pull request (PR)'''
+        return self.mr
+
+    def _set_merge_request(self, merge_request_url: str):
+        self.id_project, self.id_mr = self._parse_merge_request_url(merge_request_url)
+        self.mr = self._get_merge_request()
+
+    def get_diff_files(self) -> list[FilePatchInfo]:
+        diffs = self.mr.changes()['changes']
+        diff_files = [FilePatchInfo("", "", diff['diff'], diff['new_path']) for diff in diffs]
+        return diff_files
+
+    def get_files(self):
+        return [change['new_path'] for change in self.mr.changes()['changes']]
+
+    def publish_comment(self, mr_comment: str, is_temporary: bool = False):
+        comment = self.mr.notes.create({'body': mr_comment})
+        if is_temporary:
+            self.temp_comments.append(comment)
+
+    def remove_initial_comment(self):
+        try:
+            for comment in self.temp_comments:
+                comment.delete()
+        except Exception as e:
+            logging.exception(f"Failed to remove temp comments, error: {e}")
+
+    def get_title(self):
+        return self.mr.title
+
+    def get_description(self):
+        return self.mr.description
+
+    def get_languages(self):
+        languages = self.gl.projects.get(self.id_project).languages()
+        return languages
+
+    def get_pr_branch(self):
+        return self.mr.source_branch
+
+    def get_pr_description(self):
+        return self.mr.description
+
+    def _parse_merge_request_url(self, merge_request_url: str) -> Tuple[int, int]:
+        parsed_url = urlparse(merge_request_url)
+
+        path_parts = parsed_url.path.strip('/').split('/')
+        if path_parts[-2] != 'merge_requests':
+            raise ValueError("The provided URL does not appear to be a GitLab merge request URL")
+
+        try:
+            mr_id = int(path_parts[-1])
+        except ValueError as e:
+            raise ValueError("Unable to convert merge request ID to integer") from e
+
+        # Gitlab supports access by both project numeric ID as well as 'namespace/project_name'
+        return "/".join(path_parts[:2]), mr_id
+
+    def _get_merge_request(self):
+        mr = self.gl.projects.get(self.id_project).mergerequests.get(self.id_mr)
+        return mr

pr_agent/servers/github_app.py

@@ -35,12 +35,13 @@ async def handle_github_webhooks(request: Request, response: Response):
 async def handle_request(body):
     action = body.get("action", None)
     installation_id = body.get("installation", {}).get("id", None)
-    agent = PRAgent(installation_id)
+    settings.set("GITHUB.INSTALLATION_ID", installation_id)
+    agent = PRAgent()
     if action == 'created':
         if "comment" not in body:
             return {}
         comment_body = body.get("comment", {}).get("body", None)
-        if "says 'Please" in comment_body:
+        if 'sender' in body and 'login' in body['sender'] and 'bot' in body['sender']['login']:
             return {}
         if "issue" not in body and "pull_request" not in body["issue"]:
             return {}
@@ -66,8 +67,8 @@ async def root():
 
 
 def start():
-    if settings.get("GITHUB.DEPLOYMENT_TYPE", "user") != "app":
-        raise Exception("Please set deployment type to app in .secrets.toml file")
+    # Override the deployment type to app
+    settings.set("GITHUB.DEPLOYMENT_TYPE", "app")
     app = FastAPI()
     app.include_router(router)
 

pr_agent/servers/github_polling.py

@@ -7,6 +7,7 @@ import aiohttp
 
 from pr_agent.agent.pr_agent import PRAgent
 from pr_agent.config_loader import settings
+from pr_agent.git_providers import get_git_provider
 
 logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
 NOTIFICATION_URL = "https://api.github.com/notifications"
@@ -19,8 +20,11 @@ def now() -> str:
 
 
 async def polling_loop():
+    handled_ids = set()
     since = [now()]
     last_modified = [None]
+    git_provider = get_git_provider()()
+    user_id = git_provider.get_user_id()
     try:
         deployment_type = settings.github.deployment_type
         token = settings.github.user_token
@@ -33,41 +37,59 @@ async def polling_loop():
         raise ValueError("User token must be set to get notifications")
     async with aiohttp.ClientSession() as session:
         while True:
-            headers = {
-                "Accept": "application/vnd.github.v3+json",
-                "Authorization": f"Bearer {token}"
-            }
-            params = {
-                "participating": "true"
-            }
-            if since[0]:
-                params["since"] = since[0]
-            if last_modified[0]:
-                headers["If-Modified-Since"] = last_modified[0]
-            async with session.get(NOTIFICATION_URL, headers=headers, params=params) as response:
-                if response.status == 200:
-                    if 'Last-Modified' in response.headers:
-                        last_modified[0] = response.headers['Last-Modified']
-                        since[0] = None
-                    notifications = await response.json()
-                    for notification in notifications:
-                        if 'reason' in notification and notification['reason'] == 'mention':
-                            if 'subject' in notification and notification['subject']['type'] == 'PullRequest':
-                                pr_url = notification['subject']['url']
-                                latest_comment = notification['subject']['latest_comment_url']
-                                async with session.get(latest_comment, headers=headers) as comment_response:
-                                    if comment_response.status == 200:
-                                        comment = await comment_response.json()
-                                        comment_body = comment['body'] if 'body' in comment else ''
-                                        commenter_github_user = comment['user']['login'] if 'user' in comment else ''
-                                        logging.info(f"Commenter: {commenter_github_user}\nComment: {comment_body}")
-                                        if comment_body.strip().startswith("@"):
+            try:
+                await asyncio.sleep(5)
+                headers = {
+                    "Accept": "application/vnd.github.v3+json",
+                    "Authorization": f"Bearer {token}"
+                }
+                params = {
+                    "participating": "true"
+                }
+                if since[0]:
+                    params["since"] = since[0]
+                if last_modified[0]:
+                    headers["If-Modified-Since"] = last_modified[0]
+                async with session.get(NOTIFICATION_URL, headers=headers, params=params) as response:
+                    if response.status == 200:
+                        if 'Last-Modified' in response.headers:
+                            last_modified[0] = response.headers['Last-Modified']
+                            since[0] = None
+                        notifications = await response.json()
+                        if not notifications:
+                            continue
+                        for notification in notifications:
+                            handled_ids.add(notification['id'])
+                            if 'reason' in notification and notification['reason'] == 'mention':
+                                if 'subject' in notification and notification['subject']['type'] == 'PullRequest':
+                                    pr_url = notification['subject']['url']
+                                    latest_comment = notification['subject']['latest_comment_url']
+                                    async with session.get(latest_comment, headers=headers) as comment_response:
+                                        if comment_response.status == 200:
+                                            comment = await comment_response.json()
+                                            if 'id' in comment:
+                                                if comment['id'] in handled_ids:
+                                                    continue
+                                                else:
+                                                    handled_ids.add(comment['id'])
+                                            if 'user' in comment and 'login' in comment['user']:
+                                                if comment['user']['login'] == user_id:
+                                                    continue
+                                            comment_body = comment['body'] if 'body' in comment else ''
+                                            commenter_github_user = comment['user']['login'] \
+                                                if 'user' in comment else ''
+                                            logging.info(f"Commenter: {commenter_github_user}\nComment: {comment_body}")
+                                            user_tag = "@" + user_id
+                                            if user_tag not in comment_body:
+                                                continue
+                                            rest_of_comment = comment_body.split(user_tag)[1].strip()
                                             agent = PRAgent()
-                                            await agent.handle_request(pr_url, comment_body)
-                elif response.status != 304:
-                    print(f"Failed to fetch notifications. Status code: {response.status}")
+                                            await agent.handle_request(pr_url, rest_of_comment)
+                    elif response.status != 304:
+                        print(f"Failed to fetch notifications. Status code: {response.status}")
 
-            await asyncio.sleep(5)
+            except Exception as e:
+                logging.error(f"Exception during processing of a notification: {e}")
 
 if __name__ == '__main__':
     asyncio.run(polling_loop())

pr_agent/servers/gitlab_polling.py

@@ -0,0 +1,64 @@
+import asyncio
+import time
+
+import gitlab
+
+from pr_agent.agent.pr_agent import PRAgent
+from pr_agent.config_loader import settings
+
+gl = gitlab.Gitlab(
+    settings.get("GITLAB.URL"),
+    private_token=settings.get("GITLAB.PERSONAL_ACCESS_TOKEN")
+)
+
+# Set the list of projects to monitor
+projects_to_monitor = settings.get("GITLAB.PROJECTS_TO_MONITOR")
+magic_word = settings.get("GITLAB.MAGIC_WORD")
+
+# Hold the previous seen comments
+previous_comments = set()
+
+
+def check_comments():
+    print('Polling')
+    new_comments = {}
+    for project in projects_to_monitor:
+        project = gl.projects.get(project)
+        merge_requests = project.mergerequests.list(state='opened')
+        for mr in merge_requests:
+            notes = mr.notes.list(get_all=True)
+            for note in notes:
+                if note.id not in previous_comments and note.body.startswith(magic_word):
+                    new_comments[note.id] = dict(
+                        body=note.body[len(magic_word):],
+                        project=project.name,
+                        mr=mr
+                    )
+                    previous_comments.add(note.id)
+                    print(f"New comment in project {project.name}, merge request {mr.title}: {note.body}")
+
+    return new_comments
+
+
+def handle_new_comments(new_comments):
+    print('Handling new comments')
+    agent = PRAgent()
+    for _, comment in new_comments.items():
+        print(f"Handling comment: {comment['body']}")
+        asyncio.run(agent.handle_request(comment['mr'].web_url, comment['body']))
+
+
+def run():
+    assert settings.get('CONFIG.GIT_PROVIDER') == 'gitlab', 'This script is only for GitLab'
+    # Initial run to populate previous_comments
+    check_comments()
+
+    # Run the check every minute
+    while True:
+        time.sleep(settings.get("GITLAB.POLLING_INTERVAL_SECONDS"))
+        new_comments = check_comments()
+        if new_comments:
+            handle_new_comments(new_comments)
+
+if __name__ == '__main__':
+    run()

pr_agent/settings/.secrets_template.toml

@@ -1,5 +1,5 @@
 # QUICKSTART:
-# Copy this file to .secrets in the same folder.
+# Copy this file to .secrets.toml in the same folder.
 # The minimum workable settings - set openai.key to your API key.
 # Set github.deployment_type to "user" and github.user_token to your GitHub personal access token.
 # This will allow you to run the CLI scripts in the scripts/ folder and the github_polling server.
@@ -9,11 +9,13 @@
 [openai]
 key = "<API_KEY>"  # Acquire through https://platform.openai.com
 org = "<ORGANIZATION>"  # Optional, may be commented out.
+# Uncomment the following for Azure OpenAI
+#api_type = "azure"
+#api_version = '2023-05-15'  # Check Azure documentation for the current API version
+#api_base = "<API_BASE>"  # The base URL for your Azure OpenAI resource. e.g. "https://<your resource name>.openai.azure.com"
+#deployment_id = "<DEPLOYMENT_ID>"  # The deployment name you chose when you deployed the engine
 
 [github]
-# The type of deployment to create. Valid values are 'app' or 'user'.
-deployment_type = "user"
-
 # ---- Set the following only for deployment type == "user"
 user_token = "<TOKEN>"  # A GitHub personal access token with 'repo' scope.
 
@@ -25,3 +27,8 @@ private_key = """\
 """
 app_id = 123456  # The GitHub App ID, replace with your own.
 webhook_secret = "<WEBHOOK SECRET>"  # Optional, may be commented out.
+
+[gitlab]
+# Gitlab personal access token
+personal_access_token = ""
+

pr_agent/settings/configuration.toml

@@ -5,11 +5,27 @@ publish_review=true
 verbosity_level=0  # 0,1,2
 
 [pr_reviewer]
-require_minimal_and_focused_review=true
+require_focused_review=true
 require_tests_review=true
 require_security_review=true
 extended_code_suggestions=false
 num_code_suggestions=4
 
+[pr_questions]
 
-[pr_questions]
+[github]
+# The type of deployment to create. Valid values are 'app' or 'user'.
+deployment_type = "user"
+
+[gitlab]
+# URL to the gitlab service
+gitlab_url = "https://gitlab.com"
+
+# Polling (either project id or namespace/project_name) syntax can be used
+projects_to_monitor = ['org_name/repo_name']
+
+# Polling trigger
+magic_word = "AutoReview"
+
+# Polling interval
+polling_interval_seconds = 30

pr_agent/settings/pr_questions_prompts.toml

@@ -3,6 +3,7 @@ system="""You are CodiumAI-PR-Reviewer, a language model designed to review git
 Your task is to answer questions about the new PR code (the '+' lines), and provide feedback.
 Be informative, constructive, and give examples. Try to be as specific as possible, and don't avoid answering the questions.
 Make sure not to repeat modifications already implemented in the new PR code (the '+' lines).
+Answer only the questions, and don't add unrelated content.
 """
 
 user="""PR Info:

pr_agent/settings/pr_reviewer_prompts.toml

@@ -30,10 +30,10 @@ You must use the following JSON schema to format your answer:
       "description": "yes\\no question: does this PR have relevant tests ?"
     },
 {%- endif %}
-{%- if require_minimal_and_focused %}
-    "Minimal and focused": {
+{%- if require_focused %}
+    "Focused PR": {
       "type": "string",
-      "description": "is this PR as minimal and focused as possible, with all code changes centered around a single coherent theme, described in the PR description and title ?" explain your answer"
+      "description": "Is this a focused PR, in the sense that it has a clear and coherent title and description, and all PR code diff changes are properly derived from the title and description? Explain your response."
     }
   },
 {%- endif %}
@@ -106,8 +106,8 @@ Example output:
 {%- if require_tests %}
         "Relevant tests added": "No",
 {%- endif %}
-{%- if require_minimal_and_focused %}
-        "Minimal and focused": "No, because ..."
+{%- if require_focused %}
+        "Focused PR": "yes\\no, because ..."
 {%- endif %}
     },
     "PR Feedback":

pr_agent/tools/pr_questions.py

@@ -1,6 +1,5 @@
 import copy
 import logging
-from typing import Optional
 
 from jinja2 import Environment, StrictUndefined
 
@@ -9,21 +8,23 @@ from pr_agent.algo.pr_processing import get_pr_diff
 from pr_agent.algo.token_handler import TokenHandler
 from pr_agent.config_loader import settings
 from pr_agent.git_providers import get_git_provider
+from pr_agent.git_providers.git_provider import get_main_pr_language
 
 
 class PRQuestions:
-    def __init__(self, pr_url: str, question_str: str, installation_id: Optional[int] = None):
-        self.git_provider = get_git_provider()(pr_url, installation_id)
-        self.main_pr_language = self.git_provider.get_main_pr_language()
-        self.installation_id = installation_id
+    def __init__(self, pr_url: str, question_str: str):
+        self.git_provider = get_git_provider()(pr_url)
+        self.main_pr_language = get_main_pr_language(
+            self.git_provider.get_languages(), self.git_provider.get_files()
+        )
         self.ai_handler = AiHandler()
         self.question_str = question_str
         self.vars = {
             "title": self.git_provider.pr.title,
             "branch": self.git_provider.get_pr_branch(),
-            "description": self.git_provider.pr.body,
-            "language": self.git_provider.get_main_pr_language(),
-            "diff": "", # empty diff for initial calculation
+            "description": self.git_provider.get_description(),
+            "language": self.main_pr_language,
+            "diff": "",  # empty diff for initial calculation
             "questions": self.question_str,
         }
         self.token_handler = TokenHandler(self.git_provider.pr,
@@ -64,6 +65,8 @@ class PRQuestions:
         return response
 
     def _prepare_pr_answer(self) -> str:
-        answer_str = f"Questions: {self.question_str}\n\n"
-        answer_str += f"Answer: {self.prediction.strip()}\n\n"
+        answer_str = f"Question: {self.question_str}\n\n"
+        answer_str += f"Answer:\n{self.prediction.strip()}\n\n"
+        if settings.config.verbosity_level >= 2:
+            logging.info(f"answer_str:\n{answer_str}")
         return answer_str

pr_agent/tools/pr_reviewer.py

@@ -1,36 +1,38 @@
 import copy
 import json
 import logging
-from typing import Optional
 
 from jinja2 import Environment, StrictUndefined
 
 from pr_agent.algo.ai_handler import AiHandler
 from pr_agent.algo.pr_processing import get_pr_diff
 from pr_agent.algo.token_handler import TokenHandler
-from pr_agent.algo.utils import convert_to_markdown
+from pr_agent.algo.utils import convert_to_markdown, try_fix_json
 from pr_agent.config_loader import settings
 from pr_agent.git_providers import get_git_provider
+from pr_agent.git_providers.git_provider import get_main_pr_language
 
 
 class PRReviewer:
-    def __init__(self, pr_url: str, installation_id: Optional[int] = None):
+    def __init__(self, pr_url: str, cli_mode=False):
 
-        self.git_provider = get_git_provider()(pr_url, installation_id)
-        self.main_language = self.git_provider.get_main_pr_language()
-        self.installation_id = installation_id
+        self.git_provider = get_git_provider()(pr_url)
+        self.main_language = get_main_pr_language(
+            self.git_provider.get_languages(), self.git_provider.get_files()
+        )
         self.ai_handler = AiHandler()
         self.patches_diff = None
         self.prediction = None
+        self.cli_mode = cli_mode
         self.vars = {
             "title": self.git_provider.pr.title,
             "branch": self.git_provider.get_pr_branch(),
-            "description": self.git_provider.pr.body,
-            "language": self.git_provider.get_main_pr_language(),
+            "description": self.git_provider.get_pr_description(),
+            "language": self.main_language,
             "diff": "",  # empty diff for initial calculation
             "require_tests": settings.pr_reviewer.require_tests_review,
             "require_security": settings.pr_reviewer.require_security_review,
-            "require_minimal_and_focused": settings.pr_reviewer.require_minimal_and_focused_review,
+            "require_focused": settings.pr_reviewer.require_focused_review,
             'extended_code_suggestions': settings.pr_reviewer.extended_code_suggestions,
             'num_code_suggestions': settings.pr_reviewer.num_code_suggestions,
         }
@@ -67,11 +69,7 @@ class PRReviewer:
         model = settings.config.model
         response, finish_reason = await self.ai_handler.chat_completion(model=model, temperature=0.2,
                                                                         system=system_prompt, user=user_prompt)
-        try:
-            json.loads(response)
-        except json.decoder.JSONDecodeError:
-            logging.warning("Could not decode JSON")
-            response = {}
+
         return response
 
     def _prepare_pr_review(self) -> str:
@@ -79,11 +77,30 @@ class PRReviewer:
         try:
             data = json.loads(review)
         except json.decoder.JSONDecodeError:
-            logging.error("Unable to decode JSON response from AI")
-            data = {}
+            data = try_fix_json(review)
+
+        # reordering for nicer display
+        if 'PR Feedback' in data:
+            if 'Security concerns' in data['PR Feedback']:
+                val = data['PR Feedback']['Security concerns']
+                del data['PR Feedback']['Security concerns']
+                data['PR Analysis']['Security concerns'] = val
+
         markdown_text = convert_to_markdown(data)
-        markdown_text += "\nAdd a comment that says 'Please review' to ask for a new review after you update the PR.\n"
-        markdown_text += "Add a comment that says 'Please answer <QUESTION...>' to ask a question about this PR.\n"
+        user = self.git_provider.get_user_id()
+
+        if not self.cli_mode:
+            markdown_text += "\n### How to use\n"
+            if user and '[bot]' not in user:
+                markdown_text += f"> Tag me in a comment '@{user}' to ask for a new review after you update the PR.\n"
+                markdown_text += "> You can also tag me and ask any question, " \
+                                 f"for example '@{user} is the PR ready for merge?'"
+            else:
+                markdown_text += "> Add a comment that says 'review' to ask for a new review " \
+                                 "after you update the PR.\n"
+                markdown_text += "> You can also add a comment that says 'answer QUESTION', " \
+                                 "for example 'answer is the PR ready for merge?'"
+
         if settings.config.verbosity_level >= 2:
             logging.info(f"Markdown response:\n{markdown_text}")
-        return markdown_text
+        return markdown_text

requirements.txt

@@ -6,3 +6,6 @@ openai==0.27.8
 Jinja2==3.1.2
 tiktoken==0.4.0
 uvicorn==0.22.0
+python-gitlab==3.15.0
+pytest~=7.4.0
+aiohttp~=3.8.4

tests/unit/test_convert_to_markdown.py

@@ -50,7 +50,7 @@ class TestConvertToMarkdown:
             'Type of PR': 'Test type',
             'Relevant tests added': 'no',
             'Unrelated changes': 'n/a',  # won't be included in the output
-            'Minimal and focused': 'Yes',
+            'Focused PR': 'Yes',
             'General PR suggestions': 'general suggestion...',
             'Code suggestions': [
                 {
@@ -74,12 +74,11 @@ class TestConvertToMarkdown:
 - 🔍 **Description and title:** Test description
 - 📌 **Type of PR:** Test type
 - 🧪 **Relevant tests added:** no
-- ✨ **Minimal and focused:** Yes
+- ✨ **Focused PR:** Yes
 - 💡 **General PR suggestions:** general suggestion...
 
 - 🤖 **Code suggestions:**
 
-- **suggestion 1:**
   - **Code example:**
     - **Before:**
         ```
@@ -90,7 +89,6 @@ class TestConvertToMarkdown:
         Code after
         ```
 
-- **suggestion 2:**
   - **Code example:**
     - **Before:**
         ```
@@ -116,7 +114,7 @@ class TestConvertToMarkdown:
             'Type of PR': {},
             'Relevant tests added': {},
             'Unrelated changes': {},
-            'Minimal and focused': {},
+            'Focused PR': {},
             'General PR suggestions': {},
             'Code suggestions': {}
         }

tests/unit/test_fix_output.py

@@ -0,0 +1,91 @@
+# Generated by CodiumAI
+from pr_agent.algo.utils import try_fix_json
+
+
+import pytest
+
+class TestTryFixJson:
+    # Tests that JSON with complete 'Code suggestions' section returns expected output
+    def test_incomplete_code_suggestions(self):
+        review = '{"PR Analysis": {"Main theme": "xxx", "Description and title": "Yes", "Type of PR": "Bug fix"}, "PR Feedback": {"General PR suggestions": "..., `xxx`...", "Code suggestions": [{"suggestion number": 1, "relevant file": "xxx.py", "suggestion content": "xxx [important]"}, {"suggestion number": 2, "relevant file": "yyy.py", "suggestion content": "yyy [incomp...'
+        expected_output = {
+            'PR Analysis': {
+                'Main theme': 'xxx',
+                'Description and title': 'Yes',
+                'Type of PR': 'Bug fix'
+            },
+            'PR Feedback': {
+                'General PR suggestions': '..., `xxx`...',
+                'Code suggestions': [
+                    {
+                        'suggestion number': 1,
+                        'relevant file': 'xxx.py',
+                        'suggestion content': 'xxx [important]'
+                    }
+                ]
+            }
+        }
+        assert try_fix_json(review) == expected_output
+
+    def test_incomplete_code_suggestions_new_line(self):
+        review = '{"PR Analysis": {"Main theme": "xxx", "Description and title": "Yes", "Type of PR": "Bug fix"}, "PR Feedback": {"General PR suggestions": "..., `xxx`...", "Code suggestions": [{"suggestion number": 1, "relevant file": "xxx.py", "suggestion content": "xxx [important]"} \n\t, {"suggestion number": 2, "relevant file": "yyy.py", "suggestion content": "yyy [incomp...'
+        expected_output = {
+            'PR Analysis': {
+                'Main theme': 'xxx',
+                'Description and title': 'Yes',
+                'Type of PR': 'Bug fix'
+            },
+            'PR Feedback': {
+                'General PR suggestions': '..., `xxx`...',
+                'Code suggestions': [
+                    {
+                        'suggestion number': 1,
+                        'relevant file': 'xxx.py',
+                        'suggestion content': 'xxx [important]'
+                    }
+                ]
+            }
+        }
+        assert try_fix_json(review) == expected_output
+
+    def test_incomplete_code_suggestions_many_close_brackets(self):
+        review = '{"PR Analysis": {"Main theme": "xxx", "Description and title": "Yes", "Type of PR": "Bug fix"}, "PR Feedback": {"General PR suggestions": "..., `xxx`...", "Code suggestions": [{"suggestion number": 1, "relevant file": "xxx.py", "suggestion content": "xxx [important]"} \n, {"suggestion number": 2, "relevant file": "yyy.py", "suggestion content": "yyy }, [}\n ,incomp.}  ,..'
+        expected_output = {
+            'PR Analysis': {
+                'Main theme': 'xxx',
+                'Description and title': 'Yes',
+                'Type of PR': 'Bug fix'
+            },
+            'PR Feedback': {
+                'General PR suggestions': '..., `xxx`...',
+                'Code suggestions': [
+                    {
+                        'suggestion number': 1,
+                        'relevant file': 'xxx.py',
+                        'suggestion content': 'xxx [important]'
+                    }
+                ]
+            }
+        }
+        assert try_fix_json(review) == expected_output
+
+    def test_incomplete_code_suggestions_relevant_file(self):
+        review = '{"PR Analysis": {"Main theme": "xxx", "Description and title": "Yes", "Type of PR": "Bug fix"}, "PR Feedback": {"General PR suggestions": "..., `xxx`...", "Code suggestions": [{"suggestion number": 1, "relevant file": "xxx.py", "suggestion content": "xxx [important]"}, {"suggestion number": 2, "relevant file": "yyy.p'
+        expected_output = {
+            'PR Analysis': {
+                'Main theme': 'xxx',
+                'Description and title': 'Yes',
+                'Type of PR': 'Bug fix'
+            },
+            'PR Feedback': {
+                'General PR suggestions': '..., `xxx`...',
+                'Code suggestions': [
+                    {
+                        'suggestion number': 1,
+                        'relevant file': 'xxx.py',
+                        'suggestion content': 'xxx [important]'
+                    }
+                ]
+            }
+        }
+        assert try_fix_json(review) == expected_output

tests/unit/test_handle_patch_deletions.py

@@ -62,7 +62,7 @@ class TestHandlePatchDeletions:
         new_file_content_str = ''
         file_name = 'file.py'
         assert handle_patch_deletions(patch, original_file_content_str, new_file_content_str,
-                                      file_name) == 'File was deleted\n'
+                                      file_name) is None
 
     # Tests that handle_patch_deletions returns the original patch when patch and patch_new are equal
     def test_handle_patch_deletions_edge_case_patch_and_patch_new_are_equal(self):

tests/unit/test_language_handler.py

@@ -1,15 +1,15 @@
 
 # Generated by CodiumAI
+
 from pr_agent.algo.language_handler import sort_files_by_main_languages
 
-
-import pytest
-
 """
 Code Analysis
 
 Objective:
-The objective of the function is to sort a list of files by their main language, putting the files that are in the main language first and the rest of the files after. It takes in a dictionary of languages and their sizes, and a list of files.
+The objective of the function is to sort a list of files by their main language, putting the files that are in the main 
+language first and the rest of the files after. It takes in a dictionary of languages and their sizes, and a list of 
+files.
 
 Inputs:
 - languages: a dictionary containing the languages and their sizes
@@ -33,6 +33,8 @@ Additional aspects:
 - The function uses the filter_bad_extensions function to filter out files with bad extensions
 - The function uses a rest_files dictionary to store the files that do not belong to any of the main extensions
 """
+
+
 class TestSortFilesByMainLanguages:
     # Tests that files are sorted by main language, with files in main language first and the rest after
     def test_happy_path_sort_files_by_main_languages(self):
@@ -118,4 +120,4 @@ class TestSortFilesByMainLanguages:
             {'language': 'C++', 'files': [files[2], files[7]]},
             {'language': 'Other', 'files': []}
         ]
-        assert sort_files_by_main_languages(languages, files) == expected_output
+        assert sort_files_by_main_languages(languages, files) == expected_output

tests/unit/test_parse_code_suggestion.py

@@ -47,7 +47,7 @@ class TestParseCodeSuggestion:
             "Suggestion number": "one",
             "Description": "This is a suggestion"
         }
-        expected_output = "- **suggestion one:**\n  - **Description:** This is a suggestion\n\n"
+        expected_output = "   **Description:** This is a suggestion\n\n"
         assert parse_code_suggestion(input_data) == expected_output
 
     # Tests that function returns correct output when 'before' or 'after' key has a non-string value
@@ -70,7 +70,7 @@ class TestParseCodeSuggestion:
             'before': 'Before 1',
             'after': 'After 1'
         }
-        expected_output = "- **suggestion 1:**\n  - **suggestion:** Suggestion 1\n  - **description:** Description 1\n  - **before:** Before 1\n  - **after:** After 1\n\n"  # noqa: E501
+        expected_output = "   **suggestion:** Suggestion 1\n   **description:** Description 1\n   **before:** Before 1\n   **after:** After 1\n\n"  # noqa: E501
         assert parse_code_suggestion(code_suggestions) == expected_output
 
     # Tests that function returns correct output when input dictionary has 'code example' key
@@ -84,5 +84,5 @@ class TestParseCodeSuggestion:
                 'after': 'After 2'
             }
         }
-        expected_output = "- **suggestion 2:**\n  - **suggestion:** Suggestion 2\n  - **description:** Description 2\n  - **code example:**\n    - **before:**\n        ```\n        Before 2\n        ```\n    - **after:**\n        ```\n        After 2\n        ```\n\n"  # noqa: E501
+        expected_output = "   **suggestion:** Suggestion 2\n   **description:** Description 2\n  - **code example:**\n    - **before:**\n        ```\n        Before 2\n        ```\n    - **after:**\n        ```\n        After 2\n        ```\n\n"  # noqa: E501
         assert parse_code_suggestion(code_suggestions) == expected_output