fix improve

Enhance glob pattern handling and logging in file filtering

.github/workflows/pr-agent-review.yaml

@@ -26,5 +26,7 @@ jobs:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           PINECONE.API_KEY: ${{ secrets.PINECONE_API_KEY }}
           PINECONE.ENVIRONMENT: ${{ secrets.PINECONE_ENVIRONMENT }}
+          GITHUB_ACTION.AUTO_REVIEW: 'true'
+          GITHUB_ACTION.AUTO_IMPROVE: 'true'
 
 

.pr_agent.toml

@@ -0,0 +1,2 @@
+[pr_reviewer]
+enable_review_labels_effort = true

INSTALL.md

@@ -101,6 +101,7 @@ python3 -m pr_agent.cli --pr_url <pr_url> ask <your question>
 python3 -m pr_agent.cli --pr_url <pr_url> describe
 python3 -m pr_agent.cli --pr_url <pr_url> improve
 python3 -m pr_agent.cli --pr_url <pr_url> add_docs
+python3 -m pr_agent.cli --pr_url <pr_url> generate_labels
 python3 -m pr_agent.cli --issue_url <issue_url> similar_issue
 ...
 ```
@@ -409,10 +410,12 @@ BITBUCKET_BEARER_TOKEN: <your token>
 
 You can get a Bitbucket token for your repository by following Repository Settings -> Security -> Access Tokens.
 
+Note that comments on a PR are not supported in Bitbucket Pipeline.
 
-### Run on a hosted Bitbucket app
 
-Please contact <support@codium.ai> if you're interested in a hosted BitBucket app solution that provides full functionality including PR reviews and comment handling. It's based on the [bitbucket_app.py](https://github.com/Codium-ai/pr-agent/blob/main/pr_agent/git_providers/bitbucket_provider.py) implmentation.
+### Run using CodiumAI-hosted Bitbucket app
+
+Please contact <support@codium.ai> or visit [CodiumAI pricing page](https://www.codium.ai/pricing/) if you're interested in a hosted BitBucket app solution that provides full functionality including PR reviews and comment handling. It's based on the [bitbucket_app.py](https://github.com/Codium-ai/pr-agent/blob/main/pr_agent/git_providers/bitbucket_provider.py) implementation.
 
 
 =======

RELEASE_NOTES.md

@@ -1,3 +1,25 @@
+## [Version 0.10] - 2023-11-15
+- codiumai/pr-agent:0.10
+- codiumai/pr-agent:0.10-github_app
+- codiumai/pr-agent:0.10-bitbucket-app
+- codiumai/pr-agent:0.10-gitlab_webhook
+- codiumai/pr-agent:0.10-github_polling
+- codiumai/pr-agent:0.10-github_action
+
+### Added::Algo
+- Review tool now works with [persistent comments](https://github.com/Codium-ai/pr-agent/pull/451) by default
+- Bitbucket now publishes review suggestions with [code links](https://github.com/Codium-ai/pr-agent/pull/428)
+- Enabling to limit [max number of tokens](https://github.com/Codium-ai/pr-agent/pull/437/files)
+- Support ['gpt-4-1106-preview'](https://github.com/Codium-ai/pr-agent/pull/437/files) model
+- Support for Google's [Vertex AI](https://github.com/Codium-ai/pr-agent/pull/436)
+- Implementing [thresholds](https://github.com/Codium-ai/pr-agent/pull/423) for incremental PR reviews
+- Decoupled custom labels from [PR type](https://github.com/Codium-ai/pr-agent/pull/431)
+
+### Fixed
+- Fixed bug in [parsing quotes](https://github.com/Codium-ai/pr-agent/pull/446) in CLI
+- Preserve [user-added labels](https://github.com/Codium-ai/pr-agent/pull/433) in pull requests
+- Bug fixes in GitLab and BitBucket
+
 ## [Version 0.9] - 2023-10-29
 - codiumai/pr-agent:0.9
 - codiumai/pr-agent:0.9-github_app

Usage.md

@@ -32,12 +32,19 @@ The [Tools Guide](./docs/TOOLS_GUIDE.md) provides a detailed description of the
 #### Ignoring files from analysis
 In some cases, you may want to exclude specific files or directories from the analysis performed by CodiumAI PR-Agent. This can be useful, for example, when you have files that are generated automatically or files that shouldn't be reviewed, like vendored code.
 
-To ignore files or directories, edit the **[ignore.toml](/pr_agent/settings/ignore.toml)** configuration file. This setting is also exposed the following environment variables:
+To ignore files or directories, edit the **[ignore.toml](/pr_agent/settings/ignore.toml)** configuration file. This setting also exposes the following environment variables:
 
  - `IGNORE.GLOB`
  - `IGNORE.REGEX`
 
-See [dynaconf envvars documentation](https://www.dynaconf.com/envvars/).
+For example, to ignore python files in a PR with online usage, comment on a PR:
+`/review --ignore.glob=['*.py']`
+
+To ignore python files in all PRs, set in a configuration file:
+```
+[ignore]
+glob = ['*.py']
+```
 
 #### git provider
 The [git_provider](pr_agent/settings/configuration.toml#L4) field in the configuration file determines the GIT provider that will be used by PR-Agent. Currently, the following providers are supported:
@@ -59,7 +66,7 @@ The [git_provider](pr_agent/settings/configuration.toml#L4) field in the configu
 
 ### Working from a local repo (CLI)
 When running from your local repo (CLI), your local configuration file will be used.
-Examples for invoking the different tools via the CLI:
+Examples of invoking the different tools via the CLI:
 
 - **Review**:       `python -m pr_agent.cli --pr_url=<pr_url>  review`
 - **Describe**:     `python -m pr_agent.cli --pr_url=<pr_url>  describe`
@@ -83,7 +90,7 @@ python -m pr_agent.cli --pr_url=<pr_url>  /review --pr_reviewer.extra_instructio
 publish_output=true
 verbosity_level=2
 ```
-This is useful for debugging or experimenting with the different tools.
+This is useful for debugging or experimenting with different tools.
 
 
 ### Online usage
@@ -100,17 +107,17 @@ Commands for invoking the different tools via comments:
 
 
 To edit a specific configuration value, just add `--config_path=<value>` to any command.
-For example if you want to edit the `review` tool configurations, you can run:
+For example, if you want to edit the `review` tool configurations, you can run:
 ```
 /review --pr_reviewer.extra_instructions="..." --pr_reviewer.require_score_review=false
 ```
-Any configuration value in [configuration file](pr_agent/settings/configuration.toml) file can be similarly edited. comment `/config` to see the list of available configurations.
+Any configuration value in [configuration file](pr_agent/settings/configuration.toml) file can be similarly edited. Comment `/config` to see the list of available configurations.
 
 
 ### Working with GitHub App
 When running PR-Agent from GitHub App, the default [configuration file](pr_agent/settings/configuration.toml) from a pre-built docker will be initially loaded.
 
-By uploading a local `.pr_agent.toml` file, you can edit and customize any configuration parameter.
+By uploading a local `.pr_agent.toml` file to the root of the repo's main branch, you can edit and customize any configuration parameter.
 
 For example, if you set in `.pr_agent.toml`:
 
@@ -119,7 +126,7 @@ For example, if you set in `.pr_agent.toml`:
 num_code_suggestions=1
 ```
 
-Than you will overwrite the default number of code suggestions to be 1.
+Then you will overwrite the default number of code suggestions to 1.
 
 #### GitHub app automatic tools
 The [github_app](pr_agent/settings/configuration.toml#L76) section defines GitHub app-specific configurations.  
@@ -133,7 +140,7 @@ The GitHub app can respond to the following actions on a PR:
 4. `review_requested` - Specifically requesting review (in the PR reviewers list) from the `github-actions[bot]` user
 
 The configuration parameter `handle_pr_actions` defines the list of actions for which the GitHub app will trigger the PR-Agent.  
-The configuration parameter `pr_commands` defines the list of tools that will be **run automatically** when one of the above action happens (e.g. a new PR is opened):
+The configuration parameter `pr_commands` defines the list of tools that will be **run automatically** when one of the above actions happens (e.g., a new PR is opened):
 ```
 [github_app]
 handle_pr_actions = ['opened', 'reopened', 'ready_for_review', 'review_requested']
@@ -173,11 +180,11 @@ push_commands = [
     "/auto_review -i --pr_reviewer.remove_previous_review_comment=true",
 ]
 ```
-The means that when new code is pused to the PR, the PR-Agent will run the `describe` and incremental `auto_review` tools.  
+This means that when new code is pushed to the PR, the PR-Agent will run the `describe` and incremental `auto_review` tools.  
 For the describe tool, the `add_original_user_description` and `keep_original_user_title` parameters will be set to true.  
 For the `auto_review` tool, it will run in incremental mode, and the `remove_previous_review_comment` parameter will be set to true.
 
-Much like the configurations for `pr_commands`, you can override the default tool paramteres by uploading a local configuration file to the root of your repo.
+Much like the configurations for `pr_commands`, you can override the default tool parameters by uploading a local configuration file to the root of your repo.
 
 #### Editing the prompts
 The prompts for the various PR-Agent tools are defined in the `pr_agent/settings` folder.
@@ -303,6 +310,24 @@ key = ...
 
 Also review the [AiHandler](pr_agent/algo/ai_handler.py) file for instruction how to set keys for other models.
 
+#### Vertex AI
+
+To use Google's Vertex AI platform and its associated models (chat-bison/codechat-bison) set:
+
+``` 
+[config] # in configuration.toml
+model = "vertex_ai/codechat-bison"
+fallback_models="vertex_ai/codechat-bison"
+
+[vertexai] # in .secrets.toml
+vertex_project = "my-google-cloud-project"
+vertex_location = ""
+```
+
+Your [application default credentials](https://cloud.google.com/docs/authentication/application-default-credentials) will be used for authentication so there is no need to set explicit credentials in most environments.
+
+If you do want to set explicit credentials then you can use the `GOOGLE_APPLICATION_CREDENTIALS` environment variable set to a path to a json credentials file.
+
 ### Working with large PRs
 
 The default mode of CodiumAI is to have a single call per tool, using GPT-4, which has a token limit of 8000 tokens.

docs/DESCRIBE.md

@@ -27,6 +27,8 @@ Under the section 'pr_description', the [configuration file](./../pr_agent/setti
 
 - `extra_instructions`: Optional extra instructions to the tool. For example: "focus on the changes in the file X. Ignore change in ...".
 - To enable `custom labels`, apply the configuration changes described [here](./GENERATE_CUSTOM_LABELS.md#configuration-changes)
+- `enable_pr_type`: if set to false, it will not show the `PR type` as a text value in the description content. Default is true.
+  
 ### Markers template
 
 markers enable to easily integrate user's content and auto-generated content, with a template-like mechanism.

docs/REVIEW.md

@@ -16,15 +16,22 @@ The `review` tool can also be triggered automatically every time a new PR is ope
 
 Under the section 'pr_reviewer', the [configuration file](./../pr_agent/settings/configuration.toml#L16) contains options to customize the 'review' tool:
 
+#### enable\\disable features
 - `require_focused_review`: if set to true, the tool will add a section - 'is the PR a focused one'. Default is false.
 - `require_score_review`: if set to true, the tool will add a section that scores the PR. Default is false.
 - `require_tests_review`: if set to true, the tool will add a section that checks if the PR contains tests. Default is true.
 - `require_security_review`: if set to true, the tool will add a section that checks if the PR contains security issues. Default is true.
 - `require_estimate_effort_to_review`: if set to true, the tool will add a section that estimates thed effort needed to review the PR. Default is true.
+#### general options
 - `num_code_suggestions`: number of code suggestions provided by the 'review' tool. Default is 4.
 - `inline_code_comments`: if set to true, the tool will publish the code suggestions as comments on the code diff. Default is false.
 - `automatic_review`: if set to false, no automatic reviews will be done. Default is true.
+- `remove_previous_review_comment`: if set to true, the tool will remove the previous review comment before adding a new one. Default is false.
+- `persistent_comment`: if set to true, the review comment will be persistent, meaning that every new review request will edit the previous one. Default is true.
 - `extra_instructions`: Optional extra instructions to the tool. For example: "focus on the changes in the file X. Ignore change in ...".
+#### review labels
+- `enable_review_labels_security`: if set to true, the tool will publish a 'possible security issue' label if it detects a security issue. Default is true.
+- `enable_review_labels_effort`: if set to true, the tool will publish a 'Review effort [1-5]: x' label. Default is false.
 - To enable `custom labels`, apply the configuration changes described [here](./GENERATE_CUSTOM_LABELS.md#configuration-changes) 
 ####  Incremental Mode
 For an incremental review, which only considers changes since the last PR-Agent review, this can be useful when working on the PR in an iterative manner, and you want to focus on the changes since the last review instead of reviewing the entire PR again, the following command can be used:

pr_agent/agent/pr_agent.py

@@ -46,10 +46,13 @@ class PRAgent:
         apply_repo_settings(pr_url)
 
         # Then, apply user specific settings if exists
-        request = request.replace("'", "\\'")
-        lexer = shlex.shlex(request, posix=True)
-        lexer.whitespace_split = True
-        action, *args = list(lexer)
+        if isinstance(request, str):
+            request = request.replace("'", "\\'")
+            lexer = shlex.shlex(request, posix=True)
+            lexer.whitespace_split = True
+            action, *args = list(lexer)
+        else:
+            action, *args = request
         args = update_settings_from_args(args)
 
         action = action.lstrip("/").lower()

pr_agent/algo/__init__.py

@@ -8,9 +8,14 @@ MAX_TOKENS = {
     'gpt-4': 8000,
     'gpt-4-0613': 8000,
     'gpt-4-32k': 32000,
+    'gpt-4-1106-preview': 128000, # 128K, but may be limited by config.max_model_tokens
     'claude-instant-1': 100000,
     'claude-2': 100000,
     'command-nightly': 4096,
     'replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1': 4096,
-    'meta-llama/Llama-2-7b-chat-hf': 4096
+    'meta-llama/Llama-2-7b-chat-hf': 4096,
+    'vertex_ai/codechat-bison': 6144,
+    'vertex_ai/codechat-bison-32k': 32000,
+    'codechat-bison': 6144,
+    'codechat-bison-32k': 32000,
 }

pr_agent/algo/ai_handler.py

@@ -23,39 +23,43 @@ class AiHandler:
         Initializes the OpenAI API key and other settings from a configuration file.
         Raises a ValueError if the OpenAI key is missing.
         """
-        try:
+        self.azure = False
+
+        if get_settings().get("OPENAI.KEY", None):
             openai.api_key = get_settings().openai.key
             litellm.openai_key = get_settings().openai.key
-            if get_settings().get("litellm.use_client"):
-                litellm_token = get_settings().get("litellm.LITELLM_TOKEN")
-                assert litellm_token, "LITELLM_TOKEN is required"
-                os.environ["LITELLM_TOKEN"] = litellm_token
-                litellm.use_client = True
-            self.azure = False
-            if get_settings().get("OPENAI.ORG", None):
-                litellm.organization = get_settings().openai.org
-            if get_settings().get("OPENAI.API_TYPE", None):
-                if get_settings().openai.api_type == "azure":
-                    self.azure = True
-                    litellm.azure_key = get_settings().openai.key
-            if get_settings().get("OPENAI.API_VERSION", None):
-                litellm.api_version = get_settings().openai.api_version
-            if get_settings().get("OPENAI.API_BASE", None):
-                litellm.api_base = get_settings().openai.api_base
-            if get_settings().get("ANTHROPIC.KEY", None):
-                litellm.anthropic_key = get_settings().anthropic.key
-            if get_settings().get("COHERE.KEY", None):
-                litellm.cohere_key = get_settings().cohere.key
-            if get_settings().get("REPLICATE.KEY", None):
-                litellm.replicate_key = get_settings().replicate.key
-            if get_settings().get("REPLICATE.KEY", None):
-                litellm.replicate_key = get_settings().replicate.key
-            if get_settings().get("HUGGINGFACE.KEY", None):
-                litellm.huggingface_key = get_settings().huggingface.key
-                if get_settings().get("HUGGINGFACE.API_BASE", None):
-                    litellm.api_base = get_settings().huggingface.api_base
-        except AttributeError as e:
-            raise ValueError("OpenAI key is required") from e
+        if get_settings().get("litellm.use_client"):
+            litellm_token = get_settings().get("litellm.LITELLM_TOKEN")
+            assert litellm_token, "LITELLM_TOKEN is required"
+            os.environ["LITELLM_TOKEN"] = litellm_token
+            litellm.use_client = True
+        if get_settings().get("OPENAI.ORG", None):
+            litellm.organization = get_settings().openai.org
+        if get_settings().get("OPENAI.API_TYPE", None):
+            if get_settings().openai.api_type == "azure":
+                self.azure = True
+                litellm.azure_key = get_settings().openai.key
+        if get_settings().get("OPENAI.API_VERSION", None):
+            litellm.api_version = get_settings().openai.api_version
+        if get_settings().get("OPENAI.API_BASE", None):
+            litellm.api_base = get_settings().openai.api_base
+        if get_settings().get("ANTHROPIC.KEY", None):
+            litellm.anthropic_key = get_settings().anthropic.key
+        if get_settings().get("COHERE.KEY", None):
+            litellm.cohere_key = get_settings().cohere.key
+        if get_settings().get("REPLICATE.KEY", None):
+            litellm.replicate_key = get_settings().replicate.key
+        if get_settings().get("REPLICATE.KEY", None):
+            litellm.replicate_key = get_settings().replicate.key
+        if get_settings().get("HUGGINGFACE.KEY", None):
+            litellm.huggingface_key = get_settings().huggingface.key
+            if get_settings().get("HUGGINGFACE.API_BASE", None):
+                litellm.api_base = get_settings().huggingface.api_base
+        if get_settings().get("VERTEXAI.VERTEX_PROJECT", None):
+            litellm.vertex_project = get_settings().vertexai.vertex_project
+            litellm.vertex_location = get_settings().get(
+                "VERTEXAI.VERTEX_LOCATION", None
+            )
 
     @property
     def deployment_id(self):

pr_agent/algo/file_filter.py

@@ -11,7 +11,12 @@ def filter_ignored(files):
     try:
         # load regex patterns, and translate glob patterns to regex
         patterns = get_settings().ignore.regex
-        patterns += [fnmatch.translate(glob) for glob in get_settings().ignore.glob]
+        if isinstance(patterns, str):
+            patterns = [patterns]
+        glob_setting = get_settings().ignore.glob
+        if isinstance(glob_setting, str): # --ignore.glob=[.*utils.py], --ignore.glob=.*utils.py
+            glob_setting = glob_setting.strip('[]').split(",")
+        patterns += [fnmatch.translate(glob) for glob in glob_setting]
 
         # compile all valid patterns
         compiled_patterns = []
@@ -23,7 +28,7 @@ def filter_ignored(files):
 
         # keep filenames that _don't_ match the ignore regex
         for r in compiled_patterns:
-            files = [f for f in files if not r.match(f.filename)]
+            files = [f for f in files if (f.filename and not r.match(f.filename))]
 
     except Exception as e:
         print(f"Could not filter file list: {e}")

pr_agent/algo/git_patch_processing.py

@@ -3,6 +3,7 @@ from __future__ import annotations
 import re
 
 from pr_agent.config_loader import get_settings
+from pr_agent.git_providers.git_provider import EDIT_TYPE
 from pr_agent.log import get_logger
 
 
@@ -115,7 +116,7 @@ def omit_deletion_hunks(patch_lines) -> str:
 
 
 def handle_patch_deletions(patch: str, original_file_content_str: str,
-                           new_file_content_str: str, file_name: str) -> str:
+                           new_file_content_str: str, file_name: str, edit_type: EDIT_TYPE = EDIT_TYPE.UNKNOWN) -> str:
     """
     Handle entire file or deletion patches.
 
@@ -132,7 +133,7 @@ def handle_patch_deletions(patch: str, original_file_content_str: str,
         str: The modified patch with deletion hunks omitted.
 
     """
-    if not new_file_content_str:
+    if not new_file_content_str and edit_type != EDIT_TYPE.ADDED:
         # logic for handling deleted files - don't show patch, just show that the file was deleted
         if get_settings().config.verbosity_level > 0:
             get_logger().info(f"Processing file: {file_name}, minimizing deletion file")

pr_agent/algo/pr_processing.py

@@ -7,18 +7,20 @@ from typing import Any, Callable, List, Tuple
 
 from github import RateLimitExceededException
 
-from pr_agent.algo import MAX_TOKENS
 from pr_agent.algo.git_patch_processing import convert_to_hunks_with_lines_numbers, extend_patch, handle_patch_deletions
 from pr_agent.algo.language_handler import sort_files_by_main_languages
 from pr_agent.algo.file_filter import filter_ignored
 from pr_agent.algo.token_handler import TokenHandler, get_token_encoder
+from pr_agent.algo.utils import get_max_tokens
 from pr_agent.config_loader import get_settings
-from pr_agent.git_providers.git_provider import FilePatchInfo, GitProvider
+from pr_agent.git_providers.git_provider import FilePatchInfo, GitProvider, EDIT_TYPE
 from pr_agent.log import get_logger
 
 DELETED_FILES_ = "Deleted files:\n"
 
-MORE_MODIFIED_FILES_ = "More modified files:\n"
+MORE_MODIFIED_FILES_ = "Additional modified files (insufficient token budget to process):\n"
+
+ADDED_FILES_ = "Additional added files (insufficient token budget to process):\n"
 
 OUTPUT_BUFFER_TOKENS_SOFT_THRESHOLD = 1000
 OUTPUT_BUFFER_TOKENS_HARD_THRESHOLD = 600
@@ -64,14 +66,17 @@ def get_pr_diff(git_provider: GitProvider, token_handler: TokenHandler, model: s
         pr_languages, token_handler, add_line_numbers_to_hunks, patch_extra_lines=PATCH_EXTRA_LINES)
 
     # if we are under the limit, return the full diff
-    if total_tokens + OUTPUT_BUFFER_TOKENS_SOFT_THRESHOLD < MAX_TOKENS[model]:
+    if total_tokens + OUTPUT_BUFFER_TOKENS_SOFT_THRESHOLD < get_max_tokens(model):
         return "\n".join(patches_extended)
 
     # if we are over the limit, start pruning
-    patches_compressed, modified_file_names, deleted_file_names = \
+    patches_compressed, modified_file_names, deleted_file_names, added_file_names = \
         pr_generate_compressed_diff(pr_languages, token_handler, model, add_line_numbers_to_hunks)
 
     final_diff = "\n".join(patches_compressed)
+    if added_file_names:
+        added_list_str = ADDED_FILES_ + "\n".join(added_file_names)
+        final_diff = final_diff + "\n\n" + added_list_str
     if modified_file_names:
         modified_list_str = MORE_MODIFIED_FILES_ + "\n".join(modified_file_names)
         final_diff = final_diff + "\n\n" + modified_list_str
@@ -122,7 +127,7 @@ def pr_generate_extended_diff(pr_languages: list,
 
 
 def pr_generate_compressed_diff(top_langs: list, token_handler: TokenHandler, model: str,
-                                convert_hunks_to_line_numbers: bool) -> Tuple[list, list, list]:
+                                convert_hunks_to_line_numbers: bool) -> Tuple[list, list, list, list]:
     """
     Generate a compressed diff string for a pull request, using diff minimization techniques to reduce the number of
     tokens used.
@@ -148,6 +153,7 @@ def pr_generate_compressed_diff(top_langs: list, token_handler: TokenHandler, mo
     """
 
     patches = []
+    added_files_list = []
     modified_files_list = []
     deleted_files_list = []
     # sort each one of the languages in top_langs by the number of tokens in the diff
@@ -165,7 +171,7 @@ def pr_generate_compressed_diff(top_langs: list, token_handler: TokenHandler, mo
 
         # removing delete-only hunks
         patch = handle_patch_deletions(patch, original_file_content_str,
-                                       new_file_content_str, file.filename)
+                                       new_file_content_str, file.filename, file.edit_type)
         if patch is None:
             if not deleted_files_list:
                 total_tokens += token_handler.count_tokens(DELETED_FILES_)
@@ -179,21 +185,26 @@ def pr_generate_compressed_diff(top_langs: list, token_handler: TokenHandler, mo
         new_patch_tokens = token_handler.count_tokens(patch)
 
         # Hard Stop, no more tokens
-        if total_tokens > MAX_TOKENS[model] - OUTPUT_BUFFER_TOKENS_HARD_THRESHOLD:
+        if total_tokens > get_max_tokens(model) - OUTPUT_BUFFER_TOKENS_HARD_THRESHOLD:
             get_logger().warning(f"File was fully skipped, no more tokens: {file.filename}.")
             continue
 
         # If the patch is too large, just show the file name
-        if total_tokens + new_patch_tokens > MAX_TOKENS[model] - OUTPUT_BUFFER_TOKENS_SOFT_THRESHOLD:
+        if total_tokens + new_patch_tokens > get_max_tokens(model) - OUTPUT_BUFFER_TOKENS_SOFT_THRESHOLD:
             # Current logic is to skip the patch if it's too large
             # TODO: Option for alternative logic to remove hunks from the patch to reduce the number of tokens
             #  until we meet the requirements
             if get_settings().config.verbosity_level >= 2:
                 get_logger().warning(f"Patch too large, minimizing it, {file.filename}")
-            if not modified_files_list:
-                total_tokens += token_handler.count_tokens(MORE_MODIFIED_FILES_)
-            modified_files_list.append(file.filename)
-            total_tokens += token_handler.count_tokens(file.filename) + 1
+            if file.edit_type == EDIT_TYPE.ADDED:
+                if not added_files_list:
+                    total_tokens += token_handler.count_tokens(ADDED_FILES_)
+                added_files_list.append(file.filename)
+            else:
+                if not modified_files_list:
+                    total_tokens += token_handler.count_tokens(MORE_MODIFIED_FILES_)
+                modified_files_list.append(file.filename)
+                total_tokens += token_handler.count_tokens(file.filename) + 1
             continue
 
         if patch:
@@ -206,7 +217,7 @@ def pr_generate_compressed_diff(top_langs: list, token_handler: TokenHandler, mo
             if get_settings().config.verbosity_level >= 2:
                 get_logger().info(f"Tokens: {total_tokens}, last filename: {file.filename}")
 
-    return patches, modified_files_list, deleted_files_list
+    return patches, modified_files_list, deleted_files_list, added_files_list
 
 
 async def retry_with_fallback_models(f: Callable):
@@ -271,7 +282,7 @@ def find_line_number_of_relevant_line_in_file(diff_files: List[FilePatchInfo],
         r"^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@[ ]?(.*)")
 
     for file in diff_files:
-        if file.filename.strip() == relevant_file:
+        if file.filename and (file.filename.strip() == relevant_file):
             patch = file.patch
             patch_lines = patch.splitlines()
 
@@ -397,13 +408,13 @@ def get_pr_multi_diffs(git_provider: GitProvider,
             continue
 
         # Remove delete-only hunks
-        patch = handle_patch_deletions(patch, original_file_content_str, new_file_content_str, file.filename)
+        patch = handle_patch_deletions(patch, original_file_content_str, new_file_content_str, file.filename, file.edit_type)
         if patch is None:
             continue
 
         patch = convert_to_hunks_with_lines_numbers(patch, file)
         new_patch_tokens = token_handler.count_tokens(patch)
-        if patch and (total_tokens + new_patch_tokens > MAX_TOKENS[model] - OUTPUT_BUFFER_TOKENS_SOFT_THRESHOLD):
+        if patch and (total_tokens + new_patch_tokens > get_max_tokens(model) - OUTPUT_BUFFER_TOKENS_SOFT_THRESHOLD):
             final_diff = "\n".join(patches)
             final_diff_list.append(final_diff)
             patches = []

pr_agent/algo/utils.py

@@ -9,6 +9,8 @@ from typing import Any, List
 
 import yaml
 from starlette_context import context
+
+from pr_agent.algo import MAX_TOKENS
 from pr_agent.config_loader import get_settings, global_settings
 from pr_agent.log import get_logger
 
@@ -280,26 +282,43 @@ def _fix_key_value(key: str, value: str):
     try:
         value = yaml.safe_load(value)
     except Exception as e:
-        get_logger().error(f"Failed to parse YAML for config override {key}={value}", exc_info=e)
+        get_logger().debug(f"Failed to parse YAML for config override {key}={value}", exc_info=e)
     return key, value
 
 
-def load_yaml(review_text: str) -> dict:
-    review_text = review_text.removeprefix('```yaml').rstrip('`')
+def load_yaml(response_text: str) -> dict:
+    response_text = response_text.removeprefix('```yaml').rstrip('`')
     try:
-        data = yaml.safe_load(review_text)
+        data = yaml.safe_load(response_text)
     except Exception as e:
         get_logger().error(f"Failed to parse AI prediction: {e}")
-        data = try_fix_yaml(review_text)
+        data = try_fix_yaml(response_text)
     return data
 
-def try_fix_yaml(review_text: str) -> dict:
-    review_text_lines = review_text.split('\n')
+def try_fix_yaml(response_text: str) -> dict:
+    response_text_lines = response_text.split('\n')
+
+    keys = ['relevant line:', 'suggestion content:', 'relevant file:']
+    # first fallback - try to convert 'relevant line: ...' to relevant line: |-\n        ...'
+    response_text_lines_copy = response_text_lines.copy()
+    for i in range(0, len(response_text_lines_copy)):
+        for key in keys:
+            if key in response_text_lines_copy[i] and not '|-' in response_text_lines_copy[i]:
+                response_text_lines_copy[i] = response_text_lines_copy[i].replace(f'{key}',
+                                                                                  f'{key} |-\n        ')
+    try:
+        data = yaml.safe_load('\n'.join(response_text_lines_copy))
+        get_logger().info(f"Successfully parsed AI prediction after adding |-\n")
+        return data
+    except:
+        get_logger().info(f"Failed to parse AI prediction after adding |-\n")
+
+    # second fallback - try to remove last lines
     data = {}
-    for i in range(1, len(review_text_lines)):
-        review_text_lines_tmp = '\n'.join(review_text_lines[:-i])
+    for i in range(1, len(response_text_lines)):
+        response_text_lines_tmp = '\n'.join(response_text_lines[:-i])
         try:
-            data = yaml.load(review_text_lines_tmp, Loader=yaml.SafeLoader)
+            data = yaml.safe_load(response_text_lines_tmp,)
             get_logger().info(f"Successfully parsed AI prediction after removing {i} lines")
             break
         except:
@@ -326,18 +345,33 @@ def set_custom_labels(variables):
     variables["custom_labels_examples"] = f"      - {list(labels.keys())[0]}"
 
 
-def get_user_labels(current_labels):
-    ## Only keep labels that has been added by the user
-    if current_labels is None:
-        current_labels = []
-    user_labels = []
-    for label in current_labels:
-        if label in ['Bug fix', 'Tests', 'Refactoring', 'Enhancement', 'Documentation', 'Other']:
-            continue
-        if get_settings().config.enable_custom_labels:
-            if label in get_settings().custom_labels:
+def get_user_labels(current_labels: List[str] = None):
+    """
+    Only keep labels that has been added by the user
+    """
+    try:
+        if current_labels is None:
+            current_labels = []
+        user_labels = []
+        for label in current_labels:
+            if label.lower() in ['bug fix', 'tests', 'refactoring', 'enhancement', 'documentation', 'other']:
                 continue
-        user_labels.append(label)
-    if user_labels:
-        get_logger().info(f"Keeping user labels: {user_labels}")
+            if get_settings().config.enable_custom_labels:
+                if label in get_settings().custom_labels:
+                    continue
+            user_labels.append(label)
+        if user_labels:
+            get_logger().info(f"Keeping user labels: {user_labels}")
+    except Exception as e:
+        get_logger().exception(f"Failed to get user labels: {e}")
+        return current_labels
     return user_labels
+
+
+def get_max_tokens(model):
+    settings = get_settings()
+    max_tokens_model = MAX_TOKENS[model]
+    if settings.config.max_model_tokens:
+        max_tokens_model = min(settings.config.max_model_tokens, max_tokens_model)
+        # get_logger().debug(f"limiting max tokens to {max_tokens_model}")
+    return max_tokens_model

pr_agent/cli.py

@@ -8,6 +8,8 @@ from pr_agent.log import setup_logger
 
 setup_logger()
 
+
+
 def run(inargs=None):
     parser = argparse.ArgumentParser(description='AI based pull request analyzer', usage=
 """\
@@ -51,9 +53,9 @@ For example: 'python cli.py --pr_url=... review --pr_reviewer.extra_instructions
     command = args.command.lower()
     get_settings().set("CONFIG.CLI_MODE", True)
     if args.issue_url:
-        result = asyncio.run(PRAgent().handle_request(args.issue_url, command + " " + " ".join(args.rest)))
+        result = asyncio.run(PRAgent().handle_request(args.issue_url, [command] + args.rest))
     else:
-        result = asyncio.run(PRAgent().handle_request(args.pr_url, command + " " + " ".join(args.rest)))
+        result = asyncio.run(PRAgent().handle_request(args.pr_url, [command] + args.rest))
     if not result:
         parser.print_help()
 

pr_agent/git_providers/bitbucket_provider.py

@@ -9,7 +9,7 @@ from starlette_context import context
 from ..algo.pr_processing import find_line_number_of_relevant_line_in_file
 from ..config_loader import get_settings
 from ..log import get_logger
-from .git_provider import FilePatchInfo, GitProvider
+from .git_provider import FilePatchInfo, GitProvider, EDIT_TYPE
 
 
 class BitbucketProvider(GitProvider):
@@ -132,17 +132,56 @@ class BitbucketProvider(GitProvider):
                 diff.old.get_data("links")
             )
             new_file_content_str = self._get_pr_file_content(diff.new.get_data("links"))
-            diff_files.append(
-                FilePatchInfo(
-                    original_file_content_str,
-                    new_file_content_str,
-                    diff_split[index],
-                    diff.new.path,
-                )
+            file_patch_canonic_structure = FilePatchInfo(
+                original_file_content_str,
+                new_file_content_str,
+                diff_split[index],
+                diff.new.path,
             )
+
+            if diff.data['status'] == 'added':
+                file_patch_canonic_structure.edit_type = EDIT_TYPE.ADDED
+            elif diff.data['status'] == 'removed':
+                file_patch_canonic_structure.edit_type = EDIT_TYPE.DELETED
+            elif diff.data['status'] == 'modified':
+                file_patch_canonic_structure.edit_type = EDIT_TYPE.MODIFIED
+            elif diff.data['status'] == 'renamed':
+                file_patch_canonic_structure.edit_type = EDIT_TYPE.RENAMED
+            diff_files.append(file_patch_canonic_structure)
+
+
         self.diff_files = diff_files
         return diff_files
 
+    def get_latest_commit_url(self):
+        return self.pr.data['source']['commit']['links']['html']['href']
+
+    def get_comment_url(self, comment):
+        return comment.data['links']['html']['href']
+
+    def publish_persistent_comment(self, pr_comment: str, initial_header: str, update_header: bool = True):
+        try:
+            for comment in self.pr.comments():
+                body = comment.raw
+                if initial_header in body:
+                    latest_commit_url = self.get_latest_commit_url()
+                    comment_url = self.get_comment_url(comment)
+                    if update_header:
+                        updated_header = f"{initial_header}\n\n### (review updated until commit {latest_commit_url})\n"
+                        pr_comment_updated = pr_comment.replace(initial_header, updated_header)
+                    else:
+                        pr_comment_updated = pr_comment
+                    get_logger().info(f"Persistent mode- updating comment {comment_url} to latest review message")
+                    d = {"content": {"raw": pr_comment_updated}}
+                    response = comment._update_data(comment.put(None, data=d))
+                    self.publish_comment(
+                        f"**[Persistent review]({comment_url})** updated to latest commit {latest_commit_url}")
+                    return
+        except Exception as e:
+            get_logger().exception(f"Failed to update persistent review, error: {e}")
+            pass
+        self.publish_comment(pr_comment)
+
     def publish_comment(self, pr_comment: str, is_temporary: bool = False):
         comment = self.pr.comment(pr_comment)
         if is_temporary:
@@ -288,6 +327,11 @@ class BitbucketProvider(GitProvider):
             })
 
         response = requests.request("PUT", self.bitbucket_pull_request_api_url, headers=self.headers, data=payload)
+        try:
+            if response.status_code != 200:
+                get_logger().info(f"Failed to update description, error code: {response.status_code}")
+        except:
+            pass
         return response
 
     # bitbucket does not support labels

pr_agent/git_providers/git_provider.py

@@ -13,6 +13,7 @@ class EDIT_TYPE(Enum):
     DELETED = 2
     MODIFIED = 3
     RENAMED = 4
+    UNKNOWN = 5
 
 
 @dataclass
@@ -22,7 +23,7 @@ class FilePatchInfo:
     patch: str
     filename: str
     tokens: int = -1
-    edit_type: EDIT_TYPE = EDIT_TYPE.MODIFIED
+    edit_type: EDIT_TYPE = EDIT_TYPE.UNKNOWN
     old_filename: str = None
 
 
@@ -39,42 +40,10 @@ class GitProvider(ABC):
     def publish_description(self, pr_title: str, pr_body: str):
         pass
 
-    @abstractmethod
-    def publish_comment(self, pr_comment: str, is_temporary: bool = False):
-        pass
-
-    @abstractmethod
-    def publish_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
-        pass
-
-    @abstractmethod
-    def create_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
-        pass
-
-    @abstractmethod
-    def publish_inline_comments(self, comments: list[dict]):
-        pass
-
     @abstractmethod
     def publish_code_suggestions(self, code_suggestions: list) -> bool:
         pass
 
-    @abstractmethod
-    def publish_labels(self, labels):
-        pass
-
-    @abstractmethod
-    def get_labels(self):
-        pass
-
-    @abstractmethod
-    def remove_initial_comment(self):
-        pass
-
-    @abstractmethod
-    def remove_comment(self, comment):
-        pass
-
     @abstractmethod
     def get_languages(self):
         pass
@@ -94,16 +63,16 @@ class GitProvider(ABC):
     def get_pr_description(self, *, full: bool = True) -> str:
         from pr_agent.config_loader import get_settings
         from pr_agent.algo.pr_processing import clip_tokens
-        max_tokens = get_settings().get("CONFIG.MAX_DESCRIPTION_TOKENS", None)
+        max_tokens_description = get_settings().get("CONFIG.MAX_DESCRIPTION_TOKENS", None)
         description = self.get_pr_description_full() if full else self.get_user_description()
-        if max_tokens:
-            return clip_tokens(description, max_tokens)
+        if max_tokens_description:
+            return clip_tokens(description, max_tokens_description)
         return description
 
     def get_user_description(self) -> str:
         description = (self.get_pr_description_full() or "").strip()
         # if the existing description wasn't generated by the pr-agent, just return it as-is
-        if not description.startswith("## PR Type"):
+        if not any(description.startswith(header) for header in ("## PR Type", "## PR Description")):
             return description
         # if the existing description was generated by the pr-agent, but it doesn't contain the user description,
         # return nothing (empty string) because it means there is no user description
@@ -113,11 +82,54 @@ class GitProvider(ABC):
         return description.split("## User Description:", 1)[1].strip()
 
     @abstractmethod
-    def get_issue_comments(self):
+    def get_repo_settings(self):
+        pass
+
+    def get_pr_id(self):
+        return ""
+
+    #### comments operations ####
+    @abstractmethod
+    def publish_comment(self, pr_comment: str, is_temporary: bool = False):
+        pass
+
+    def publish_persistent_comment(self, pr_comment: str, initial_header: str, update_header: bool):
+        self.publish_comment(pr_comment)
+
+    @abstractmethod
+    def publish_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
         pass
 
     @abstractmethod
-    def get_repo_settings(self):
+    def create_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str):
+        pass
+
+    @abstractmethod
+    def publish_inline_comments(self, comments: list[dict]):
+        pass
+
+    @abstractmethod
+    def remove_initial_comment(self):
+        pass
+
+    @abstractmethod
+    def remove_comment(self, comment):
+        pass
+
+    @abstractmethod
+    def get_issue_comments(self):
+        pass
+
+    def get_comment_url(self, comment) -> str:
+        return ""
+
+    #### labels operations ####
+    @abstractmethod
+    def publish_labels(self, labels):
+        pass
+
+    @abstractmethod
+    def get_labels(self):
         pass
 
     @abstractmethod
@@ -128,11 +140,12 @@ class GitProvider(ABC):
     def remove_reaction(self, issue_comment_id: int, reaction_id: int) -> bool:
         pass
 
+    #### commits operations ####
     @abstractmethod
     def get_commit_messages(self):
         pass
 
-    def get_pr_id(self):
+    def get_latest_commit_url(self) -> str:
         return ""
 
 def get_main_pr_language(languages, files) -> str:
@@ -153,6 +166,8 @@ def get_main_pr_language(languages, files) -> str:
         # validate that the specific commit uses the main language
         extension_list = []
         for file in files:
+            if not file:
+                continue
             if isinstance(file, str):
                 file = FilePatchInfo(base_file=None, head_file=None, patch=None, filename=file)
             extension_list.append(file.filename.rsplit('.')[-1])

pr_agent/git_providers/github_provider.py

@@ -13,7 +13,7 @@ from ..algo.utils import load_large_diff
 from ..config_loader import get_settings
 from ..log import get_logger
 from ..servers.utils import RateLimitExceeded
-from .git_provider import FilePatchInfo, GitProvider, IncrementalPR
+from .git_provider import FilePatchInfo, GitProvider, IncrementalPR, EDIT_TYPE
 
 
 class GithubProvider(GitProvider):
@@ -129,7 +129,20 @@ class GithubProvider(GitProvider):
                     if not patch:
                         patch = load_large_diff(file.filename, new_file_content_str, original_file_content_str)
 
-                diff_files.append(FilePatchInfo(original_file_content_str, new_file_content_str, patch, file.filename))
+                if file.status == 'added':
+                    edit_type = EDIT_TYPE.ADDED
+                elif file.status == 'removed':
+                    edit_type = EDIT_TYPE.DELETED
+                elif file.status == 'renamed':
+                    edit_type = EDIT_TYPE.RENAMED
+                elif file.status == 'modified':
+                    edit_type = EDIT_TYPE.MODIFIED
+                else:
+                    get_logger().error(f"Unknown edit type: {file.status}")
+                    edit_type = EDIT_TYPE.UNKNOWN
+                file_patch_canonical_structure = FilePatchInfo(original_file_content_str, new_file_content_str, patch,
+                                                               file.filename, edit_type=edit_type)
+                diff_files.append(file_patch_canonical_structure)
 
             self.diff_files = diff_files
             return diff_files
@@ -141,10 +154,36 @@ class GithubProvider(GitProvider):
     def publish_description(self, pr_title: str, pr_body: str):
         self.pr.edit(title=pr_title, body=pr_body)
 
+    def get_latest_commit_url(self) -> str:
+        return self.last_commit_id.html_url
+
+    def get_comment_url(self, comment) -> str:
+        return comment.html_url
+
+    def publish_persistent_comment(self, pr_comment: str, initial_header: str, update_header: bool = True):
+        prev_comments = list(self.pr.get_issue_comments())
+        for comment in prev_comments:
+            body = comment.body
+            if body.startswith(initial_header):
+                latest_commit_url = self.get_latest_commit_url()
+                comment_url = self.get_comment_url(comment)
+                if update_header:
+                    updated_header = f"{initial_header}\n\n### (review updated until commit {latest_commit_url})\n"
+                    pr_comment_updated = pr_comment.replace(initial_header, updated_header)
+                else:
+                    pr_comment_updated = pr_comment
+                get_logger().info(f"Persistent mode- updating comment {comment_url} to latest review message")
+                response = comment.edit(pr_comment_updated)
+                self.publish_comment(
+                    f"**[Persistent review]({comment_url})** updated to latest commit {latest_commit_url}")
+                return
+        self.publish_comment(pr_comment)
+
     def publish_comment(self, pr_comment: str, is_temporary: bool = False):
         if is_temporary and not get_settings().config.publish_output_progress:
             get_logger().debug(f"Skipping publish_comment for temporary comment: {pr_comment}")
             return
+
         response = self.pr.create_issue_comment(pr_comment)
         if hasattr(response, "user") and hasattr(response.user, "login"):
             self.github_user_id = response.user.login
@@ -366,7 +405,7 @@ class GithubProvider(GitProvider):
                 raise ValueError("GitHub app installation ID is required when using GitHub app deployment")
             auth = AppAuthentication(app_id=app_id, private_key=private_key,
                                      installation_id=self.installation_id)
-            return Github(app_auth=auth)
+            return Github(app_auth=auth, base_url=get_settings().github.base_url)
 
         if deployment_type == 'user':
             try:
@@ -375,7 +414,7 @@ class GithubProvider(GitProvider):
                 raise ValueError(
                     "GitHub token is required when using user deployment. See: "
                     "https://github.com/Codium-ai/pr-agent#method-2-run-from-source") from e
-            return Github(auth=Auth.Token(token))
+            return Github(auth=Auth.Token(token), base_url=get_settings().github.base_url)
 
     def _get_repo(self):
         if hasattr(self, 'repo_obj') and \

pr_agent/git_providers/gitlab_provider.py

@@ -136,6 +136,33 @@ class GitLabProvider(GitProvider):
         except Exception as e:
             get_logger().exception(f"Could not update merge request {self.id_mr} description: {e}")
 
+    def get_latest_commit_url(self):
+        return self.mr.commits().next().web_url
+
+    def get_comment_url(self, comment):
+        return f"{self.mr.web_url}#note_{comment.id}"
+
+    def publish_persistent_comment(self, pr_comment: str, initial_header: str, update_header: bool = True):
+        try:
+            for comment in self.mr.notes.list(get_all=True)[::-1]:
+                if comment.body.startswith(initial_header):
+                    latest_commit_url = self.get_latest_commit_url()
+                    comment_url = self.get_comment_url(comment)
+                    if update_header:
+                        updated_header = f"{initial_header}\n\n### (review updated until commit {latest_commit_url})\n"
+                        pr_comment_updated = pr_comment.replace(initial_header, updated_header)
+                    else:
+                        pr_comment_updated = pr_comment
+                    get_logger().info(f"Persistent mode- updating comment {comment_url} to latest review message")
+                    response = self.mr.notes.update(comment.id, {'body': pr_comment_updated})
+                    self.publish_comment(
+                        f"**[Persistent review]({comment_url})** updated to latest commit {latest_commit_url}")
+                    return
+        except Exception as e:
+            get_logger().exception(f"Failed to update persistent review, error: {e}")
+            pass
+        self.publish_comment(pr_comment)
+
     def publish_comment(self, mr_comment: str, is_temporary: bool = False):
         comment = self.mr.notes.create({'body': mr_comment})
         if is_temporary:

pr_agent/servers/github_action_runner.py

@@ -65,13 +65,13 @@ async def run_action():
         if action in ["opened", "reopened"]:
             pr_url = event_payload.get("pull_request", {}).get("url")
             if pr_url:
-                auto_review = os.environ.get('github_action.auto_review', None)
+                auto_review = get_settings().get('GITHUB_ACTION.AUTO_REVIEW', None)
                 if auto_review is None or (isinstance(auto_review, str) and auto_review.lower() == 'true'):
                     await PRReviewer(pr_url).run()
-                auto_describe = os.environ.get('github_action.auto_describe', None)
+                auto_describe = get_settings().get('GITHUB_ACTION.AUTO_DESCRIBE', None)
                 if isinstance(auto_describe, str) and auto_describe.lower() == 'true':
                     await PRDescription(pr_url).run()
-                auto_improve = os.environ.get('github_action.auto_improve', None)
+                auto_improve = get_settings().get('GITHUB_ACTION.AUTO_IMPROVE', None)
                 if isinstance(auto_improve, str) and auto_improve.lower() == 'true':
                     await PRCodeSuggestions(pr_url).run()
 

pr_agent/servers/gitlab_webhook.py

@@ -38,7 +38,7 @@ async def gitlab_webhook(background_tasks: BackgroundTasks, request: Request):
         try:
             secret_dict = json.loads(secret)
             gitlab_token = secret_dict["gitlab_token"]
-            log_context["sender"] = secret_dict["id"]
+            log_context["sender"] = secret_dict.get("token_name", secret_dict.get("id", "unknown"))
             context["settings"] = copy.deepcopy(global_settings)
             context["settings"].gitlab.personal_access_token = gitlab_token
         except Exception as e:

pr_agent/servers/help.py

@@ -1,12 +1,14 @@
-commands_text = "> **/review [-i]**: Request a review of your Pull Request. For an incremental review, which only " \
-                "considers changes since the last review, include the '-i' option.\n" \
-                "> **/describe**: Modify the PR title and description based on the contents of the PR.\n" \
-                "> **/improve [--extended]**: Suggest improvements to the code in the PR. Extended mode employs several calls, and provides a more thorough feedback. \n" \
-                "> **/ask \\<QUESTION\\>**: Pose a question about the PR.\n" \
-                "> **/update_changelog**: Update the changelog based on the PR's contents.\n\n" \
-                ">To edit any configuration parameter from **configuration.toml**, add --config_path=new_value\n" \
+commands_text = "> **/review**: Request a review of your Pull Request.\n" \
+                "> **/describe**: Update the PR title and description based on the contents of the PR.\n" \
+                "> **/improve [--extended]**: Suggest code improvements. Extended mode provides a higher quality feedback.\n" \
+                "> **/ask \\<QUESTION\\>**: Ask a question about the PR.\n" \
+                "> **/update_changelog**: Update the changelog based on the PR's contents.\n" \
+                "> **/add_docs**: Generate docstring for new components introduced in the PR.\n" \
+                "> **/generate_labels**: Generate labels for the PR based on the PR's contents.\n" \
+                "> see the [tools guide](https://github.com/Codium-ai/pr-agent/blob/main/docs/TOOLS_GUIDE.md) for more details.\n\n" \
+                ">To edit any configuration parameter from the [configuration.toml](https://github.com/Codium-ai/pr-agent/blob/main/pr_agent/settings/configuration.toml), add --config_path=new_value.\n" \
                 ">For example: /review --pr_reviewer.extra_instructions=\"focus on the file: ...\" \n" \
-                ">To list the possible configuration parameters, use the **/config** command.\n" \
+                ">To list the possible configuration parameters, add a **/config** comment.\n" \
 
 
 def bot_help_text(user: str):

pr_agent/servers/serverless.py

@@ -3,10 +3,8 @@ from mangum import Mangum
 from starlette.middleware import Middleware
 from starlette_context.middleware import RawContextMiddleware
 
-from pr_agent.log import setup_logger
 from pr_agent.servers.github_app import router
 
-setup_logger()
 
 middleware = [Middleware(RawContextMiddleware)]
 app = FastAPI(middleware=middleware)

pr_agent/settings/.secrets_template.toml

@@ -36,6 +36,10 @@ api_base = "" # the base url for your huggingface inference endpoint
 [ollama]
 api_base = "" # the base url for your local Llama 2, Code Llama, and other models inference endpoint. Acquire through https://ollama.ai/
 
+[vertexai]
+vertex_project = "" # the google cloud platform project name for your vertexai deployment
+vertex_location = "" # the google cloud platform location for your vertexai deployment
+
 [github]
 # ---- Set the following only for deployment type == "user"
 user_token = ""  # A GitHub personal access token with 'repo' scope.

pr_agent/settings/configuration.toml

@@ -1,5 +1,5 @@
 [config]
-model="gpt-4"
+model="gpt-4" # "gpt-4-1106-preview"
 fallback_models=["gpt-3.5-turbo-16k"]
 git_provider="github"
 publish_output=true
@@ -10,22 +10,29 @@ use_repo_settings_file=true
 ai_timeout=180
 max_description_tokens = 500
 max_commits_tokens = 500
+max_model_tokens = 32000 # Limits the maximum number of tokens that can be used by any model, regardless of the model's default capabilities.
 patch_extra_lines = 3
 secret_provider="google_cloud_storage"
 cli_mode=false
 
 [pr_reviewer] # /review #
+# enable/disable features
 require_focused_review=false
 require_score_review=false
 require_tests_review=true
 require_security_review=true
 require_estimate_effort_to_review=true
+# general options
 num_code_suggestions=4
 inline_code_comments = false
 ask_and_reflect=false
 automatic_review=true
 remove_previous_review_comment=false
+persistent_comment=true
 extra_instructions = ""
+# review labels
+enable_review_labels_security=true
+enable_review_labels_effort=false
 # specific configurations for incremental review (/review -i)
 require_all_thresholds_for_incremental_review=false
 minimal_commits_for_incremental_review=0
@@ -72,6 +79,7 @@ extra_instructions = ""
 # The type of deployment to create. Valid values are 'app' or 'user'.
 deployment_type = "user"
 ratelimit_retries = 5
+base_url = "https://api.github.com"
 
 [github_action]
 # auto_review = true    # set as env var in .github/workflows/pr-agent.yaml

pr_agent/settings/pr_code_suggestions_prompts.toml

@@ -90,16 +90,19 @@ Code suggestions:
 Example output:
 ```yaml
 Code suggestions:
-  - relevant file: |-
-        src/file1.py
-    suggestion content: |-
-        Add a docstring to func1()
-    existing code: |-
-        def func1():
-    relevant lines start: 12
-    relevant lines end: 12
-    improved code: |-
-        ...
+- relevant file: |-
+    src/file1.py
+  suggestion content: |-
+    Add a docstring to func1()
+  existing code: |-
+    def func1():
+  relevant lines start: |-
+    12
+  relevant lines end: |-
+    12
+  improved code: |-
+    ...
+...
 ```
 
 

pr_agent/settings/pr_custom_labels.toml

@@ -16,7 +16,7 @@ You must use the following YAML schema to format your answer:
 PR Type:
   type: array
 {%- if enable_custom_labels %}
-  description: One or more labels that describe the PR type. Don't output the description in the parentheses.
+  description: Labels that are applicable to the Pull Request. Don't output the description in the parentheses. If none of the labels is relevant to the PR, output an empty array.
 {%- endif %}
   items:
     type: string

pr_agent/settings/pr_description_prompts.toml

@@ -30,7 +30,7 @@ PR Type:
 {%- if enable_custom_labels %}
 PR Labels:
   type: array
-  description: One or more labels that describe the PR labels. Don't output the description in the parentheses.
+  description: Labels that are applicable to the Pull Request. Don't output the description in the parentheses. If none of the labels is relevant to the PR, output an empty array.
   items:
     type: string
     enum:

pr_agent/settings/pr_reviewer_prompts.toml

@@ -93,7 +93,7 @@ PR Analysis:
     description: >-
       Estimate, on a scale of 1-5 (inclusive), the time and effort required to review this PR by an experienced and knowledgeable developer. 1 means short and easy review , 5 means long and hard review.
       Take into account the size, complexity, quality, and the needed changes of the PR code diff.
-      Explain your answer shortly (1-2 sentences).
+      Explain your answer shortly (1-2 sentences). Use the format: '1, because ...'
 {%- endif %}
 PR Feedback:
   General suggestions:
@@ -130,7 +130,8 @@ PR Feedback:
   Security concerns:
     type: string
     description: >-
-      yes\\no question: does this PR code introduce possible vulnerabilities such as exposure of sensitive information (e.g., API keys, secrets, passwords), or security concerns like SQL injection, XSS, CSRF, and others ? If answered 'yes', explain your answer briefly.
+      does this PR code introduce possible vulnerabilities such as exposure of sensitive information (e.g., API keys, secrets, passwords), or security concerns like SQL injection, XSS, CSRF, and others ? Answer 'No' if there are no possible issues.
+      Answer 'Yes, because ...' if there are security concerns or issues. Explain your answer shortly.
 {%- endif %}
 ```
 
@@ -152,7 +153,8 @@ PR Analysis:
   Focused PR: no, because ...
 {%- endif %}
 {%- if require_estimate_effort_to_review %}
-  Estimated effort to review [1-5]: 3, because ...
+  Estimated effort to review [1-5]: |-
+    3, because ...
 {%- endif %}
 PR Feedback:
   General PR suggestions: |-

pr_agent/tools/pr_description.py

@@ -158,6 +158,9 @@ class PRDescription:
             user=user_prompt
         )
 
+        if get_settings().config.verbosity_level >= 2:
+            get_logger().info(f"\nAI response:\n{response}")
+
         return response
 
     def _prepare_data(self):

pr_agent/tools/pr_reviewer.py

@@ -10,7 +10,7 @@ from yaml import SafeLoader
 from pr_agent.algo.ai_handler import AiHandler
 from pr_agent.algo.pr_processing import get_pr_diff, retry_with_fallback_models
 from pr_agent.algo.token_handler import TokenHandler
-from pr_agent.algo.utils import convert_to_markdown, load_yaml, try_fix_yaml, set_custom_labels
+from pr_agent.algo.utils import convert_to_markdown, load_yaml, try_fix_yaml, set_custom_labels, get_user_labels
 from pr_agent.config_loader import get_settings
 from pr_agent.git_providers import get_git_provider
 from pr_agent.git_providers.git_provider import IncrementalPR, get_main_pr_language
@@ -117,7 +117,15 @@ class PRReviewer:
             if get_settings().config.publish_output:
                 get_logger().info('Pushing PR review...')
                 previous_review_comment = self._get_previous_review_comment()
-                self.git_provider.publish_comment(pr_comment)
+
+                # publish the review
+                if get_settings().pr_reviewer.persistent_comment and not self.incremental.is_incremental:
+                    self.git_provider.publish_persistent_comment(pr_comment,
+                                                                 initial_header="## PR Analysis",
+                                                                 update_header=True)
+                else:
+                    self.git_provider.publish_comment(pr_comment)
+
                 self.git_provider.remove_initial_comment()
                 if previous_review_comment:
                     self._remove_previous_review_comment(previous_review_comment)
@@ -156,7 +164,6 @@ class PRReviewer:
         variables["diff"] = self.patches_diff  # update diff
 
         environment = Environment(undefined=StrictUndefined)
-        # set_custom_labels(variables)
         system_prompt = environment.from_string(get_settings().pr_review_prompt.system).render(variables)
         user_prompt = environment.from_string(get_settings().pr_review_prompt.user).render(variables)
 
@@ -171,6 +178,9 @@ class PRReviewer:
             user=user_prompt
         )
 
+        if get_settings().config.verbosity_level >= 2:
+            get_logger().info(f"\nAI response:\n{response}")
+
         return response
 
     def _prepare_pr_review(self) -> str:
@@ -239,11 +249,18 @@ class PRReviewer:
         # Add help text if not in CLI mode
         if not get_settings().get("CONFIG.CLI_MODE", False):
             markdown_text += "\n### How to use\n"
+            if self.git_provider.is_supported("gfm_markdown"):
+                markdown_text += "\n**<details><summary> Instructions**</summary>\n"
             bot_user = "[bot]" if get_settings().github_app.override_deployment_type else get_settings().github_app.bot_user
             if user and bot_user not in user:
                 markdown_text += bot_help_text(user)
             else:
                 markdown_text += actions_help_text
+            if self.git_provider.is_supported("gfm_markdown"):
+                markdown_text += "\n</details>\n"
+
+        # Add custom labels from the review prediction (effort, security)
+        self.set_review_labels(data)
 
         # Log markdown response if verbosity level is high
         if get_settings().config.verbosity_level >= 2:
@@ -261,14 +278,7 @@ class PRReviewer:
         if get_settings().pr_reviewer.num_code_suggestions == 0:
             return
 
-        review_text = self.prediction.strip()
-        review_text = review_text.removeprefix('```yaml').rstrip('`')
-        try:
-            data = yaml.load(review_text, Loader=SafeLoader)
-        except Exception as e:
-            get_logger().error(f"Failed to parse AI prediction: {e}")
-            data = try_fix_yaml(review_text)
-
+        data = load_yaml(self.prediction.strip())
         comments: List[str] = []
         for suggestion in data.get('PR Feedback', {}).get('Code feedback', []):
             relevant_file = suggestion.get('relevant file', '').strip()
@@ -365,3 +375,28 @@ class PRReviewer:
             )
             return False
         return True
+
+    def set_review_labels(self, data):
+        if (get_settings().pr_reviewer.enable_review_labels_security or
+                get_settings().pr_reviewer.enable_review_labels_effort):
+            try:
+                review_labels = []
+                if get_settings().pr_reviewer.enable_review_labels_effort:
+                    estimated_effort = data['PR Analysis']['Estimated effort to review [1-5]']
+                    estimated_effort_number = int(estimated_effort.split(',')[0])
+                    if 1 <= estimated_effort_number <= 5: # 1, because ...
+                        review_labels.append(f'Review effort [1-5]: {estimated_effort_number}')
+                if get_settings().pr_reviewer.enable_review_labels_security:
+                    security_concerns = data['PR Analysis']['Security concerns'] # yes, because ...
+                    security_concerns_bool = 'yes' in security_concerns.lower() or 'true' in security_concerns.lower()
+                    if security_concerns_bool:
+                        review_labels.append('Possible security concern')
+
+                if review_labels:
+                    current_labels = self.git_provider.get_labels()
+                    current_labels_filtered = [label for label in current_labels if
+                                               not label.lower().startswith('review effort [1-5]:') and not label.lower().startswith(
+                                                   'possible security concern')]
+                    self.git_provider.publish_labels(review_labels + current_labels_filtered)
+            except Exception as e:
+                get_logger().error(f"Failed to set review labels, error: {e}")

pr_agent/tools/pr_similar_issue.py

@@ -10,6 +10,7 @@ from pydantic import BaseModel, Field
 
 from pr_agent.algo import MAX_TOKENS
 from pr_agent.algo.token_handler import TokenHandler
+from pr_agent.algo.utils import get_max_tokens
 from pr_agent.config_loader import get_settings
 from pr_agent.git_providers import get_git_provider
 from pr_agent.log import get_logger
@@ -197,7 +198,7 @@ class PRSimilarIssue:
             username = issue.user.login
             created_at = str(issue.created_at)
             if len(issue_str) < 8000 or \
-                    self.token_handler.count_tokens(issue_str) < MAX_TOKENS[MODEL]:  # fast reject first
+                    self.token_handler.count_tokens(issue_str) < get_max_tokens(MODEL):  # fast reject first
                 issue_record = Record(
                     id=issue_key + "." + "issue",
                     text=issue_str,

requirements.txt

@@ -13,7 +13,7 @@ atlassian-python-api==3.39.0
 GitPython==3.1.32
 PyYAML==6.0
 starlette-context==0.3.6
-litellm~=0.1.574
+litellm==0.12.5
 boto3==1.28.25
 google-cloud-storage==2.10.0
 ujson==5.8.0
@@ -22,3 +22,4 @@ msrest==0.7.1
 pinecone-client
 pinecone-datasets @ git+https://github.com/mrT23/pinecone-datasets.git@main
 loguru==0.7.2
+google-cloud-aiplatform==1.35.0

tests/unittest/test_load_yaml.py

@@ -2,6 +2,9 @@
 # Generated by CodiumAI
 
 import pytest
+import yaml
+from yaml.scanner import ScannerError
+
 from pr_agent.algo.utils import load_yaml
 
 
@@ -12,7 +15,7 @@ class TestLoadYaml:
         expected_output = {'name': 'John Smith', 'age': 35}
         assert load_yaml(yaml_str) == expected_output
 
-    def test_load_complicated_yaml(self):
+    def test_load_invalid_yaml1(self):
         yaml_str = \
 '''\
 PR Analysis:
@@ -26,7 +29,23 @@ PR Feedback:
   Code feedback:
     - relevant file: pr_agent/settings/pr_description_prompts.toml
       suggestion: Consider using a more descriptive variable name than 'user' for the command prompt. A more descriptive name would make the code more readable and maintainable. [medium]
-      relevant line: 'user="""PR Info:'
+      relevant line: user="""PR Info: aaa
   Security concerns: No'''
-        expected_output = {'PR Analysis': {'Main theme': 'Enhancing the `/describe` command prompt by adding title and description', 'Type of PR': 'Enhancement', 'Relevant tests added': False, 'Focused PR': 'Yes, the PR is focused on enhancing the `/describe` command prompt.'}, 'PR Feedback': {'General suggestions': 'The PR seems to be well-structured and focused on a specific enhancement. However, it would be beneficial to add tests to ensure the new feature works as expected.', 'Code feedback': [{'relevant file': 'pr_agent/settings/pr_description_prompts.toml', 'suggestion': "Consider using a more descriptive variable name than 'user' for the command prompt. A more descriptive name would make the code more readable and maintainable. [medium]", 'relevant line': 'user="""PR Info:'}], 'Security concerns': False}}
+        with pytest.raises(ScannerError):
+            yaml.safe_load(yaml_str)
+
+        expected_output = {'PR Analysis': {'Main theme': 'Enhancing the `/describe` command prompt by adding title and description', 'Type of PR': 'Enhancement', 'Relevant tests added': False, 'Focused PR': 'Yes, the PR is focused on enhancing the `/describe` command prompt.'}, 'PR Feedback': {'General suggestions': 'The PR seems to be well-structured and focused on a specific enhancement. However, it would be beneficial to add tests to ensure the new feature works as expected.', 'Code feedback': [{'relevant file': 'pr_agent/settings/pr_description_prompts.toml', 'suggestion': "Consider using a more descriptive variable name than 'user' for the command prompt. A more descriptive name would make the code more readable and maintainable. [medium]", 'relevant line': 'user="""PR Info: aaa'}], 'Security concerns': False}}
         assert load_yaml(yaml_str) == expected_output
+
+    def test_load_invalid_yaml2(self):
+        yaml_str = '''\
+- relevant file: src/app.py:
+  suggestion content: The print statement is outside inside the if __name__ ==: \
+  '''
+        with pytest.raises(ScannerError):
+            yaml.safe_load(yaml_str)
+
+        expected_output =[{'relevant file': 'src/app.py:',
+  'suggestion content': 'The print statement is outside inside the if __name__ ==:   '}]
+        assert load_yaml(yaml_str) == expected_output
+

tests/unittest/try_fix_yaml.py

@@ -0,0 +1,31 @@
+
+# Generated by CodiumAI
+from pr_agent.algo.utils import try_fix_yaml
+
+
+import pytest
+
+class TestTryFixYaml:
+
+    # The function successfully parses a valid YAML string.
+    def test_valid_yaml(self):
+        review_text = "key: value\n"
+        expected_output = {"key": "value"}
+        assert try_fix_yaml(review_text) == expected_output
+
+    # The function adds '|-' to 'relevant line:' if it is not already present and successfully parses the YAML string.
+    def test_add_relevant_line(self):
+        review_text = "relevant line: value: 3\n"
+        expected_output = {"relevant line": "value: 3"}
+        assert try_fix_yaml(review_text) == expected_output
+
+    # The function removes the last line(s) of the YAML string and successfully parses the YAML string.
+    def test_remove_last_line(self):
+        review_text = "key: value\nextra invalid line\n"
+        expected_output = {"key": "value"}
+        assert try_fix_yaml(review_text) == expected_output
+
+    # The YAML string is empty.
+    def test_empty_yaml_fixed(self):
+        review_text = ""
+        assert try_fix_yaml(review_text) is None