Update

feat: Enable PR description publishing as comment in bitbucket_app.py

INSTALL.md

@@ -206,6 +206,7 @@ Allowing you to automate the review process on your private or public repositori
    - Set the following events:
      - Issue comment
      - Pull request
+     - Push (if you need to enable triggering on PR update)
 
 2. Generate a random secret for your app, and save it for later. For example, you can use:
 

README.md

@@ -21,7 +21,7 @@ Making pull requests less painful with an AI agent
 </div>
 <div style="text-align:left;">
 
-CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull requests faster and more efficiently. It automatically analyzes the pull request and can provide several types of commands:
+CodiumAI `PR-Agent` is an open-source tool for efficient pull request reviewing and handling. It automatically analyzes the pull request and can provide several types of commands:
 
 ‣ **Auto Description ([`/describe`](./docs/DESCRIBE.md))**: Automatically generating PR description - title, type, summary, code walkthrough and labels.
 \
@@ -33,17 +33,17 @@ CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull
 \
 ‣ **Update Changelog ([`/update_changelog`](./docs/UPDATE_CHANGELOG.md))**: Automatically updating the CHANGELOG.md file with the PR changes.
 \
-‣ **Find Similar Issue ([`/similar_issue`](./docs/SIMILAR_ISSUE.md))**: Automatically retrieves and presents similar issues
+‣ **Find Similar Issue ([`/similar_issue`](./docs/SIMILAR_ISSUE.md))**: Automatically retrieves and presents similar issues.
 \
 ‣ **Add Documentation ([`/add_docs`](./docs/ADD_DOCUMENTATION.md))**: Automatically adds documentation to un-documented functions/classes in the PR.
 \
 ‣ **Generate Custom Labels ([`/generate_labels`](./docs/GENERATE_CUSTOM_LABELS.md))**: Automatically suggests custom labels based on the PR code changes.
 
-See the [Installation Guide](./INSTALL.md) for instructions how to install and run the tool on different platforms.
+See the [Installation Guide](./INSTALL.md) for instructions on installing and running the tool on different git platforms.
 
-See the [Usage Guide](./Usage.md) for instructions how to run the different tools from _CLI_, _online usage_, or by _automatically triggering_ them when a new PR is opened.
+See the [Usage Guide](./Usage.md) for running the PR-Agent commands via different interfaces, including _CLI_, _online usage_, or by _automatically triggering_ them when a new PR is opened.
 
-See the [Tools Guide](./docs/TOOLS_GUIDE.md) for detailed description of the different tools.
+See the [Tools Guide](./docs/TOOLS_GUIDE.md) for detailed description of the different tools (tools are run via the commands).
 
 <h3>Example results:</h3>
 </div>
@@ -140,7 +140,7 @@ Review the [usage guide](./Usage.md) section for detailed instructions how to us
 
 ## Try it now
 
-You can try GPT-4 powered PR-Agent, on your public GitHub repository, instantly. Just mention `@CodiumAI-Agent` and add the desired command in any PR comment. The agent will generate a response based on your command.
+Try the GPT-4 powered PR-Agent instantly on _your public GitHub repository_. Just mention `@CodiumAI-Agent` and add the desired command in any PR comment. The agent will generate a response based on your command.
 For example, add a comment to any pull request with the following text:
 ```
 @CodiumAI-Agent /review
@@ -151,6 +151,7 @@ and the agent will respond with a review of your PR
 
 
 To set up your own PR-Agent, see the [Installation](#installation) section below.
+Note that when you set your own PR-Agent or use CodiumAI hosted PR-Agent, there is no need to mention `@CodiumAI-Agent ...`. Instead, directly start with the command, e.g., `/ask ...`.
 
 ---
 

RELEASE_NOTES.md

@@ -1,3 +1,21 @@
+## [Version 0.11] - 2023-12-07
+- codiumai/pr-agent:0.11
+- codiumai/pr-agent:0.11-github_app
+- codiumai/pr-agent:0.11-bitbucket-app
+- codiumai/pr-agent:0.11-gitlab_webhook
+- codiumai/pr-agent:0.11-github_polling
+- codiumai/pr-agent:0.11-github_action
+
+### Added::Algo
+- New section in `/describe` tool - [PR changes walkthrough](https://github.com/Codium-ai/pr-agent/pull/509)
+- Improving PR Agent [prompts](https://github.com/Codium-ai/pr-agent/pull/501)
+- Persistent tools (`/review`, `/describe`) now send an [update message](https://github.com/Codium-ai/pr-agent/pull/499) after finishing
+- Add Amazon Bedrock [support](https://github.com/Codium-ai/pr-agent/pull/483)
+
+### Fixed
+- Update [dependencies](https://github.com/Codium-ai/pr-agent/pull/503) in requirements.txt for Python 3.12
+
+
 ## [Version 0.10] - 2023-11-15
 - codiumai/pr-agent:0.10
 - codiumai/pr-agent:0.10-github_app

Usage.md

@@ -239,6 +239,23 @@ inline_code_comments = true
 
 Each time you invoke a `/review` tool, it will use inline code comments.
 
+#### BitBucket Self-Hosted App automatic tools
+You can configure in your local `.pr_agent.toml` file which tools will **run automatically** when a new PR is opened.
+
+Specifically, set the following values:
+```yaml
+[bitbucket_app]
+auto_review = true    # set as config var in .pr_agent.toml
+auto_describe = true  # set as config var in .pr_agent.toml
+auto_improve = true   # set as config var in .pr_agent.toml
+```
+
+`bitbucket_app.auto_review`, `bitbucket_app.auto_describe` and `bitbucket_app.auto_improve` are used to enable/disable automatic tools.
+If not set, the default option is that only the `review` tool will run automatically when a new PR is opened.
+
+Note that due to limitations of the bitbucket platform, the `auto_describe` tool will be able to publish a PR description only as a comment. 
+In addition, some subsections like `PR changes walkthrough` will not appear, since they require the usage of collapsible sections, which are not supported by bitbucket.
+
 ### Changing a model
 
 See [here](pr_agent/algo/__init__.py) for the list of available models.

docs/DESCRIBE.md

@@ -1,6 +1,6 @@
 # Describe Tool
 
-The `describe` tool scans the PR code changes, and automatically generates PR description - title, type, summary, code walkthrough and labels.
+The `describe` tool scans the PR code changes, and automatically generates PR description - title, type, summary, walkthrough and labels.
 It can be invoked manually by commenting on any PR:
 ```
 /describe
@@ -26,9 +26,14 @@ Under the section 'pr_description', the [configuration file](./../pr_agent/setti
 - `keep_original_user_title`: if set to true, the tool will keep the original PR title, and won't change it. Default is false.
 
 - `extra_instructions`: Optional extra instructions to the tool. For example: "focus on the changes in the file X. Ignore change in ...".
+
 - To enable `custom labels`, apply the configuration changes described [here](./GENERATE_CUSTOM_LABELS.md#configuration-changes)
+
 - `enable_pr_type`: if set to false, it will not show the `PR type` as a text value in the description content. Default is true.
+
 - `final_update_message`: if set to true, it will add a comment message [`PR Description updated to latest commit...`](https://github.com/Codium-ai/pr-agent/pull/499#issuecomment-1837412176) after finishing calling `/describe`. Default is true.
+
+- `enable_semantic_files_types`: if set to true, "PR changes walkthrough" section will be generated. Default is true.
   
 ### Markers template
 

pr_agent/algo/utils.py

@@ -59,14 +59,14 @@ def convert_to_markdown(output_data: dict, gfm_supported: bool=True) -> str:
             if key.lower() == 'code feedback':
                 if gfm_supported:
                     markdown_text += f"\n\n- "
-                    markdown_text += f"<details><summary> { emoji } Code feedback:</summary>\n\n"
+                    markdown_text += f"<details><summary> { emoji } Code feedback:</summary>"
                 else:
                     markdown_text += f"\n\n- **{emoji} Code feedback:**\n\n"
             else:
                 markdown_text += f"- {emoji} **{key}:**\n\n"
-            for item in value:
+            for i, item in enumerate(value):
                 if isinstance(item, dict) and key.lower() == 'code feedback':
-                    markdown_text += parse_code_suggestion(item, gfm_supported)
+                    markdown_text += parse_code_suggestion(item, i, gfm_supported)
                 elif item:
                     markdown_text += f"  - {item}\n"
             if key.lower() == 'code feedback':
@@ -80,7 +80,7 @@ def convert_to_markdown(output_data: dict, gfm_supported: bool=True) -> str:
     return markdown_text
 
 
-def parse_code_suggestion(code_suggestions: dict, gfm_supported: bool=True) -> str:
+def parse_code_suggestion(code_suggestions: dict, i: int = 0, gfm_supported: bool = True) -> str:
     """
     Convert a dictionary of data into markdown format.
 
@@ -91,24 +91,52 @@ def parse_code_suggestion(code_suggestions: dict, gfm_supported: bool=True) -> s
         str: A string containing the markdown formatted text generated from the input dictionary.
     """
     markdown_text = ""
-    for sub_key, sub_value in code_suggestions.items():
-        if isinstance(sub_value, dict):  # "code example"
-            markdown_text += f"  - **{sub_key}:**\n"
-            for code_key, code_value in sub_value.items():  # 'before' and 'after' code
-                code_str = f"```\n{code_value}\n```"
-                code_str_indented = textwrap.indent(code_str, '        ')
-                markdown_text += f"    - **{code_key}:**\n{code_str_indented}\n"
-        else:
-            if "relevant file" in sub_key.lower():
-                markdown_text += f"\n  - **{sub_key}:** {sub_value}  \n"
+    if gfm_supported and 'relevant line' in code_suggestions:
+        if i == 0:
+            markdown_text += "<hr>"
+        markdown_text += '<table>'
+        for sub_key, sub_value in code_suggestions.items():
+            try:
+                if sub_key.lower() == 'relevant file':
+                    relevant_file = sub_value.strip('`').strip('"').strip("'")
+                    markdown_text += f"<tr><td>{sub_key}</td><td>{relevant_file}</td></tr>"
+                    # continue
+                elif sub_key.lower() == 'suggestion':
+                    markdown_text += f"<tr><td>{sub_key} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</td><td><strong>{sub_value}</strong></td></tr>"
+                elif sub_key.lower() == 'relevant line':
+                    markdown_text += f"<tr><td>relevant line</td>"
+                    sub_value_list = sub_value.split('](')
+                    relevant_line = sub_value_list[0].lstrip('`').lstrip('[')
+                    if len(sub_value_list) > 1:
+                        link = sub_value_list[1].rstrip(')').strip('`')
+                        markdown_text += f"<td><a href={link}>{relevant_line}</a></td>"
+                    else:
+                        markdown_text += f"<td>{relevant_line}</td>"
+                    markdown_text += "</tr>"
+            except Exception as e:
+                get_logger().exception(f"Failed to parse code suggestion: {e}")
+                pass
+        markdown_text += '</table>'
+        markdown_text += "<hr>"
+    else:
+        for sub_key, sub_value in code_suggestions.items():
+            if isinstance(sub_value, dict):  # "code example"
+                markdown_text += f"  - **{sub_key}:**\n"
+                for code_key, code_value in sub_value.items():  # 'before' and 'after' code
+                    code_str = f"```\n{code_value}\n```"
+                    code_str_indented = textwrap.indent(code_str, '        ')
+                    markdown_text += f"    - **{code_key}:**\n{code_str_indented}\n"
             else:
-                markdown_text += f"   **{sub_key}:** {sub_value}  \n"
-            if not gfm_supported:
-                if "relevant line" not in sub_key.lower(): # nicer presentation
+                if "relevant file" in sub_key.lower():
+                    markdown_text += f"\n  - **{sub_key}:** {sub_value}  \n"
+                else:
+                    markdown_text += f"   **{sub_key}:** {sub_value}  \n"
+                if not gfm_supported:
+                    if "relevant line" not in sub_key.lower():  # nicer presentation
                         # markdown_text = markdown_text.rstrip('\n') + "\\\n" # works for gitlab
                         markdown_text = markdown_text.rstrip('\n') + "   \n"  # works for gitlab and bitbucker
 
-    markdown_text += "\n"
+        markdown_text += "\n"
     return markdown_text
 
 
@@ -336,7 +364,7 @@ def try_fix_yaml(response_text: str) -> dict:
         pass
 
 
-def set_custom_labels(variables):
+def set_custom_labels(variables, git_provider=None):
     if not get_settings().config.enable_custom_labels:
         return
 
@@ -348,11 +376,8 @@ def set_custom_labels(variables):
         labels_list = f"      - {labels_list}" if labels_list else ""
         variables["custom_labels"] = labels_list
         return
-    #final_labels = ""
-    #for k, v in labels.items():
-    #    final_labels += f"      - {k} ({v['description']})\n"
-    #variables["custom_labels"] = final_labels
-    #variables["custom_labels_examples"] = f"      - {list(labels.keys())[0]}"
+
+    # Set custom labels
     variables["custom_labels_class"] = "class Label(str, Enum):"
     for k, v in labels.items():
         description = v['description'].strip('\n').replace('\n', '\\n')

pr_agent/config_loader.py

@@ -26,6 +26,7 @@ global_settings = Dynaconf(
         "settings/pr_custom_labels.toml",
         "settings/pr_add_docs.toml",
         "settings_prod/.secrets.toml",
+        "settings_prod/.secrets_foo.toml",
         "settings/custom_labels.toml"
     ]]
 )

pr_agent/git_providers/bitbucket_provider.py

@@ -354,5 +354,5 @@ class BitbucketProvider(GitProvider):
         pass
     
     # bitbucket does not support labels
-    def get_labels(self):
+    def get_pr_labels(self):
         pass

pr_agent/git_providers/bitbucket_server_provider.py

@@ -344,7 +344,7 @@ class BitbucketServerProvider(GitProvider):
         pass
     
     # bitbucket does not support labels
-    def get_labels(self):
+    def get_pr_labels(self):
         pass
 
     def _get_pr_comments_url(self):

pr_agent/git_providers/codecommit_provider.py

@@ -216,7 +216,7 @@ class CodeCommitProvider(GitProvider):
     def publish_labels(self, labels):
         return [""]  # not implemented yet
 
-    def get_labels(self):
+    def get_pr_labels(self):
         return [""]  # not implemented yet
 
     def remove_initial_comment(self):

pr_agent/git_providers/gerrit_provider.py

@@ -207,7 +207,7 @@ class GerritProvider(GitProvider):
         Comment = namedtuple('Comment', ['body'])
         return Comments([Comment(c['message']) for c in reversed(comments)])
 
-    def get_labels(self):
+    def get_pr_labels(self):
         raise NotImplementedError(
             'Getting labels is not implemented for the gerrit provider')
 

pr_agent/git_providers/git_provider.py

@@ -26,6 +26,8 @@ class FilePatchInfo:
     tokens: int = -1
     edit_type: EDIT_TYPE = EDIT_TYPE.UNKNOWN
     old_filename: str = None
+    num_plus_lines: int = -1
+    num_minus_lines: int = -1
 
 
 class GitProvider(ABC):
@@ -133,7 +135,10 @@ class GitProvider(ABC):
         pass
 
     @abstractmethod
-    def get_labels(self):
+    def get_pr_labels(self):
+        pass
+
+    def get_repo_labels(self):
         pass
 
     @abstractmethod

pr_agent/git_providers/github_provider.py

@@ -143,8 +143,15 @@ class GithubProvider(GitProvider):
                 else:
                     get_logger().error(f"Unknown edit type: {file.status}")
                     edit_type = EDIT_TYPE.UNKNOWN
+
+                # count number of lines added and removed
+                patch_lines = patch.splitlines(keepends=True)
+                num_plus_lines = len([line for line in patch_lines if line.startswith('+')])
+                num_minus_lines = len([line for line in patch_lines if line.startswith('-')])
                 file_patch_canonical_structure = FilePatchInfo(original_file_content_str, new_file_content_str, patch,
-                                                               file.filename, edit_type=edit_type)
+                                                               file.filename, edit_type=edit_type,
+                                                               num_plus_lines=num_plus_lines,
+                                                               num_minus_lines=num_minus_lines,)
                 diff_files.append(file_patch_canonical_structure)
 
             self.diff_files = diff_files
@@ -454,13 +461,17 @@ class GithubProvider(GitProvider):
         except Exception as e:
             get_logger().exception(f"Failed to publish labels, error: {e}")
 
-    def get_labels(self):
+    def get_pr_labels(self):
         try:
             return [label.name for label in self.pr.labels]
         except Exception as e:
             get_logger().exception(f"Failed to get labels, error: {e}")
             return []
 
+    def get_repo_labels(self):
+        labels = self.repo_obj.get_labels()
+        return [label for label in labels]
+
     def get_commit_messages(self):
         """
         Retrieves the commit messages of a pull request.

pr_agent/git_providers/gitlab_provider.py

@@ -115,12 +115,20 @@ class GitLabProvider(GitProvider):
                 if not patch:
                     patch = load_large_diff(filename, new_file_content_str, original_file_content_str)
 
+
+                # count number of lines added and removed
+                patch_lines = patch.splitlines(keepends=True)
+                num_plus_lines = len([line for line in patch_lines if line.startswith('+')])
+                num_minus_lines = len([line for line in patch_lines if line.startswith('-')])
                 diff_files.append(
                     FilePatchInfo(original_file_content_str, new_file_content_str,
                                   patch=patch,
                                   filename=filename,
                                   edit_type=edit_type,
-                                  old_filename=None if diff['old_path'] == diff['new_path'] else diff['old_path']))
+                                  old_filename=None if diff['old_path'] == diff['new_path'] else diff['old_path'],
+                                  num_plus_lines=num_plus_lines,
+                                  num_minus_lines=num_minus_lines, ))
+
         self.diff_files = diff_files
         return diff_files
 
@@ -203,7 +211,11 @@ class GitLabProvider(GitProvider):
                 pos_obj['new_line'] = target_line_no - 1
                 pos_obj['old_line'] = source_line_no - 1
             get_logger().debug(f"Creating comment in {self.id_mr} with body {body} and position {pos_obj}")
-            self.mr.discussions.create({'body': body, 'position': pos_obj})
+            try:
+                self.mr.discussions.create({'body': body, 'position': pos_obj})
+            except Exception as e:
+                get_logger().debug(
+                    f"Failed to create comment in {self.id_mr} with position {pos_obj} (probably not a '+' line)")
 
     def get_relevant_diff(self, relevant_file: str, relevant_line_in_file: int) -> Optional[dict]:
         changes = self.mr.changes()  # Retrieve the changes for the merge request once
@@ -396,7 +408,7 @@ class GitLabProvider(GitProvider):
     def publish_inline_comments(self, comments: list[dict]):
         pass
 
-    def get_labels(self):
+    def get_pr_labels(self):
         return self.mr.labels
 
     def get_commit_messages(self):

pr_agent/git_providers/local_git_provider.py

@@ -178,5 +178,5 @@ class LocalGitProvider(GitProvider):
     def get_issue_comments(self):
         raise NotImplementedError('Getting issue comments is not implemented for the local git provider')
 
-    def get_labels(self):
+    def get_pr_labels(self):
         raise NotImplementedError('Getting labels is not implemented for the local git provider')

pr_agent/servers/bitbucket_app.py

@@ -16,8 +16,13 @@ from starlette_context.middleware import RawContextMiddleware
 
 from pr_agent.agent.pr_agent import PRAgent
 from pr_agent.config_loader import get_settings, global_settings
+from pr_agent.git_providers.utils import apply_repo_settings
 from pr_agent.log import LoggingFormat, get_logger, setup_logger
 from pr_agent.secret_providers import get_secret_provider
+from pr_agent.servers.github_action_runner import get_setting_or_env, is_true
+from pr_agent.tools.pr_code_suggestions import PRCodeSuggestions
+from pr_agent.tools.pr_description import PRDescription
+from pr_agent.tools.pr_reviewer import PRReviewer
 
 setup_logger(fmt=LoggingFormat.JSON)
 router = APIRouter()
@@ -89,8 +94,20 @@ async def handle_github_webhooks(background_tasks: BackgroundTasks, request: Req
                 pr_url = data["data"]["pullrequest"]["links"]["html"]["href"]
                 log_context["api_url"] = pr_url
                 log_context["event"] = "pull_request"
-                with get_logger().contextualize(**log_context):
-                    await agent.handle_request(pr_url, "review")
+                if pr_url:
+                    with get_logger().contextualize(**log_context):
+                        apply_repo_settings(pr_url)
+                        auto_review = get_setting_or_env("BITBUCKET_APP.AUTO_REVIEW", None)
+                        if auto_review is None or is_true(auto_review):  # by default, auto review is enabled
+                            await PRReviewer(pr_url).run()
+                        auto_improve = get_setting_or_env("BITBUCKET_APP.AUTO_IMPROVE", None)
+                        if is_true(auto_improve):  # by default, auto improve is disabled
+                            await PRCodeSuggestions(pr_url).run()
+                        auto_describe = get_setting_or_env("BITBUCKET_APP.AUTO_DESCRIBE", None)
+                        if is_true(auto_describe):  # by default, auto describe is disabled
+                            await PRDescription(pr_url).run()
+                # with get_logger().contextualize(**log_context):
+                #     await agent.handle_request(pr_url, "review")
             elif event == "pullrequest:comment_created":
                 pr_url = data["data"]["pullrequest"]["links"]["html"]["href"]
                 log_context["api_url"] = pr_url

pr_agent/servers/github_app.py

@@ -125,11 +125,15 @@ async def handle_request(body: Dict[str, Any], event: str):
             await _perform_commands("pr_commands", agent, body, api_url, log_context)
 
     # handle pull_request event with synchronize action - "push trigger" for new commits
-    elif event == 'pull_request' and action == 'synchronize' and get_settings().github_app.handle_push_trigger:
+    elif event == 'pull_request' and action == 'synchronize':
         pull_request, api_url = _check_pull_request_event(action, body, log_context, bot_user)
         if not (pull_request and api_url):
             return {}
 
+        apply_repo_settings(api_url)
+        if not get_settings().github_app.handle_push_trigger:
+            return {}
+
         # TODO: do we still want to get the list of commits to filter bot/merge commits?
         before_sha = body.get("before")
         after_sha = body.get("after")

pr_agent/settings/configuration.toml

@@ -143,6 +143,12 @@ magic_word = "AutoReview"
 # Polling interval
 polling_interval_seconds = 30
 
+[bitbucket_app]
+#auto_review = true    # set as config var in .pr_agent.toml
+#auto_describe = true  # set as config var in .pr_agent.toml
+#auto_improve = true   # set as config var in .pr_agent.toml
+
+
 [local]
 # LocalGitProvider settings - uncomment to use paths other than default
 # description_path= "path/to/description.md"
@@ -170,3 +176,4 @@ max_issues_to_scan = 500
 # fill and place in .secrets.toml
 #api_key = ...
 # environment = "gcp-starter"
+

pr_agent/settings/pr_description_prompts.toml

@@ -33,13 +33,14 @@ class PRType(str, Enum):
 {%- if enable_file_walkthrough %}
 class FileWalkthrough(BaseModel):
     filename: str = Field(description="the relevant file full path")
-    changes_in_file: str = Field(description="minimal and concise description of the changes in the relevant file")
+    changes_in_file: str = Field(description="minimal and concise summary of the changes in the relevant file")
 {%- endif %}
 
 {%- if enable_semantic_files_types %}
-class SemanticFileLabels(BaseModel):
-    label: str = Field(description="a semantic label that represents a type of code changes that occurred in the PR. Possible values (partial list): 'bug fix', 'tests', 'enhancement', 'documentation', 'error handling', 'configuration changes', 'logging', 'dependencies', 'new feature', 'other', ...")
-    files: List[str] = Field(description="a list of file names related to the chosen semantic label. Present the file full path, and nothing else. A file should appear only in a single label, that best describes the type of changes in the file. If you are not sure, prefer a more general label.")
+Class FileDescription(BaseModel):
+    filename: str = Field(description="the relevant file full path")
+    changes_summary: str = Field(description="minimal and concise summary of the changes in the relevant file")
+    label: str = Field(description="a single semantic label that represents a type of code changes that occurred in the File. Possible values (partial list): 'bug fix', 'tests', 'enhancement', 'documentation', 'error handling', 'configuration changes', 'dependencies', 'formatting', 'miscellaneous', ...")
 {%- endif %}
 
 Class PRDescription(BaseModel):
@@ -53,7 +54,7 @@ Class PRDescription(BaseModel):
     main_files_walkthrough: List[FileWalkthrough] = Field(max_items=10)
 {%- endif %}
 {%- if enable_semantic_files_types %}
-    pr_files_labels[List[SemanticFileLabels]] = Field(min_items=2, description="A list of semantic labels that describe the type of changes in the PR files.")
+    pr_files[List[FileDescription]] = Field(max_items=15")
 {%- endif %}
 =====
 
@@ -79,12 +80,12 @@ main_files_walkthrough:
 - ...
 {%- endif %}
 {%- if enable_semantic_files_types %}
-pr_files_labels:
-- label: ...
-  files:
-  - |
+pr_files:
+- filename: |
     ...
-  - |
+  changes_summary: |
+    ...
+  label: |
     ...
 ...
 {%- endif %}

pr_agent/tools/pr_description.py

@@ -30,6 +30,11 @@ class PRDescription:
         )
         self.pr_id = self.git_provider.get_pr_id()
 
+        if get_settings().pr_description.enable_semantic_files_types and not self.git_provider.is_supported(
+                "gfm_markdown"):
+            get_logger().debug(f"Disabling semantic files types for {self.pr_id}")
+            get_settings().pr_description.enable_semantic_files_types = False
+
         # Initialize the AI handler
         self.ai_handler = AiHandler()
     
@@ -81,6 +86,9 @@ class PRDescription:
             else:
                 return None
 
+            if get_settings().pr_description.enable_semantic_files_types:
+                self._prepare_file_labels()
+
             pr_labels = []
             if get_settings().pr_description.publish_labels:
                 pr_labels = self._prepare_labels()
@@ -94,11 +102,12 @@ class PRDescription:
             if get_settings().config.publish_output:
                 get_logger().info(f"Pushing answer {self.pr_id}")
                 if get_settings().pr_description.publish_description_as_comment:
+                    get_logger().info(f"Publishing answer as comment")
                     self.git_provider.publish_comment(full_markdown_description)
                 else:
                     self.git_provider.publish_description(pr_title, pr_body)
                     if get_settings().pr_description.publish_labels and self.git_provider.is_supported("get_labels"):
-                        current_labels = self.git_provider.get_labels()
+                        current_labels = self.git_provider.get_pr_labels()
                         user_labels = get_user_labels(current_labels)
                         self.git_provider.publish_labels(pr_labels + user_labels)
 
@@ -150,7 +159,7 @@ class PRDescription:
         variables["diff"] = self.patches_diff  # update diff
 
         environment = Environment(undefined=StrictUndefined)
-        set_custom_labels(variables)
+        set_custom_labels(variables, self.git_provider)
         system_prompt = environment.from_string(get_settings().pr_description_prompt.system).render(variables)
         user_prompt = environment.from_string(get_settings().pr_description_prompt.user).render(variables)
 
@@ -259,12 +268,13 @@ class PRDescription:
         # except for the items containing the word 'walkthrough'
         pr_body = ""
         for idx, (key, value) in enumerate(self.data.items()):
-            key_publish = key.rstrip(':').replace("_", " ").capitalize()
-            if key == 'pr_files_labels':
-                key_publish = 'PR Changes Analysis'
+            if key == 'pr_files':
+                value = self.file_label_dict
+                key_publish = "PR changes walkthrough"
+            else:
+                key_publish = key.rstrip(':').replace("_", " ").capitalize()
             pr_body += f"## {key_publish}\n"
             if 'walkthrough' in key.lower():
-                # for filename, description in value.items():
                 if self.git_provider.is_supported("gfm_markdown"):
                     pr_body += "<details> <summary>files:</summary>\n\n"
                 for file in value:
@@ -272,48 +282,116 @@ class PRDescription:
                     description = file['changes_in_file']
                     pr_body += f'- `{filename}`: {description}\n'
                 if self.git_provider.is_supported("gfm_markdown"):
-                    pr_body +="</details>\n"
-            elif 'pr_files_labels' in key.lower():
-                pr_body += """\n| | Relevant Files """
-                pr_body += "&nbsp; " * 60
-                pr_body += """|\n|-----------|-------------|\n"""
-                for semantic_label in value:
-                    # for filename, description in value.items():
-                    s_label = semantic_label['label'].strip("'").strip('"')
-                    if self.git_provider.is_supported("gfm_markdown"):
-                        # pr_body += f"<details> <summary>{semantic_label['label']}</summary>\n\n"
-                        pr_body += f"| **{s_label}** | <details><summary>files:</summary><ul>"
-                    else:
-                        pr_body += f"| **{s_label}** | "
-
-                    for file in semantic_label['files']:
-                        filename = file.replace("'", "`")
-                        # description = file['changes_in_file']
-                        # pr_body += f'- `{filename}`\n'
-
-                        # try to add line numbers link to code suggestions
-                        if hasattr(self.git_provider, 'get_line_link'):
-                            filename = filename.strip()
-                            link = self.git_provider.get_line_link(filename, relevant_line_start=-1)
-                            if link:
-                                filename = f"[{filename}]({link})"
-                        if self.git_provider.is_supported("gfm_markdown"):
-                            pr_body += f"<li>{filename}</li>"
-                        else:
-                            pr_body += f"{filename}  &emsp; &emsp;"
-                    if self.git_provider.is_supported("gfm_markdown"):
-                        pr_body += "</ul></details>|\n"
-                    else:
-                        pr_body += "|"
+                    pr_body += "</details>\n"
+            elif 'pr_files' in key.lower():
+                pr_body = self.process_pr_files_prediction(pr_body, value)
             else:
                 # if the value is a list, join its items by comma
                 if isinstance(value, list):
                     value = ', '.join(v for v in value)
                 pr_body += f"{value}\n"
             if idx < len(self.data) - 1:
-                pr_body += "\n___\n"
+                pr_body += "\n\n___\n\n"
 
         if get_settings().config.verbosity_level >= 2:
             get_logger().info(f"title:\n{title}\n{pr_body}")
 
-        return title, pr_body
+        return title, pr_body
+
+    def _prepare_file_labels(self):
+        self.file_label_dict = {}
+        for file in self.data['pr_files']:
+            try:
+                filename = file['filename'].replace("'", "`").replace('"', '`')
+                changes_summary = file['changes_summary']
+                label = file['label']
+                if label not in self.file_label_dict:
+                    self.file_label_dict[label] = []
+                self.file_label_dict[label].append((filename, changes_summary))
+            except Exception as e:
+                get_logger().error(f"Error preparing file label dict {self.pr_id}: {e}")
+                pass
+
+    def process_pr_files_prediction(self, pr_body, value):
+        if not self.git_provider.is_supported("gfm_markdown"):
+            get_logger().info(f"Disabling semantic files types for {self.pr_id} since gfm_markdown is not supported")
+            return pr_body
+        try:
+            pr_body += "<table>"
+            header = f"Relevant files"
+            delta = 65
+            header += "&nbsp; " * delta
+            pr_body += f"""<thead><tr><th></th><th>{header}</th></tr></thead>"""
+            pr_body += """<tbody>"""
+            for semantic_label in value.keys():
+                s_label = semantic_label.strip("'").strip('"')
+                pr_body += f"""<tr><td><strong>{s_label.capitalize()}</strong></td>"""
+                list_tuples = value[semantic_label]
+                pr_body += f"""<td><details><summary>{len(list_tuples)} files</summary><table>"""
+                for filename, file_change_description in list_tuples:
+                    filename = filename.replace("'", "`")
+                    filename_publish = filename.split("/")[-1]
+                    filename_publish = f"{filename_publish}"
+                    if len(filename_publish) < (delta - 5):
+                        filename_publish += "&nbsp; " * ((delta - 5) - len(filename_publish))
+                    diff_plus_minus = ""
+                    diff_files = self.git_provider.diff_files
+                    for f in diff_files:
+                        if f.filename.lower() == filename.lower():
+                            num_plus_lines = f.num_plus_lines
+                            num_minus_lines = f.num_minus_lines
+                            diff_plus_minus += f"+{num_plus_lines}/-{num_minus_lines}"
+                            break
+
+                    # try to add line numbers link to code suggestions
+                    link = ""
+                    if hasattr(self.git_provider, 'get_line_link'):
+                        filename = filename.strip()
+                        link = self.git_provider.get_line_link(filename, relevant_line_start=-1)
+
+                    file_change_description = self._insert_br_after_x_chars(file_change_description, x=(delta - 5))
+                    pr_body += f"""
+<tr>
+  <td>
+    <details>
+      <summary><strong>{filename_publish}</strong></summary>
+      <ul>
+        {filename}<br><br>
+        <strong>{file_change_description}</strong>
+      </ul>
+    </details>
+  </td>
+  <td><a href="{link}"> {diff_plus_minus}</a></td>
+
+</tr>                    
+"""
+                pr_body += """</table></details></td></tr>"""
+            pr_body += """</tr></tbody></table>"""
+
+        except Exception as e:
+            get_logger().error(f"Error processing pr files to markdown {self.pr_id}: {e}")
+            pass
+        return pr_body
+
+    def _insert_br_after_x_chars(self, text, x=70):
+        """
+        Insert <br> into a string after a word that increases its length above x characters.
+        """
+        if len(text) < x:
+            return text
+
+        words = text.split(' ')
+        new_text = ""
+        current_length = 0
+
+        for word in words:
+            # Check if adding this word exceeds x characters
+            if current_length + len(word) > x:
+                new_text += "<br>"  # Insert line break
+                current_length = 0  # Reset counter
+
+            # Add the word to the new text
+            new_text += word + " "
+            current_length += len(word) + 1  # Add 1 for the space
+
+        return new_text.strip()  # Remove trailing space

pr_agent/tools/pr_generate_labels.py

@@ -82,7 +82,7 @@ class PRGenerateLabels:
             if get_settings().config.publish_output:
                 get_logger().info(f"Pushing labels {self.pr_id}")
 
-                current_labels = self.git_provider.get_labels()
+                current_labels = self.git_provider.get_pr_labels()
                 user_labels = get_user_labels(current_labels)
                 pr_labels = pr_labels + user_labels
 
@@ -132,7 +132,7 @@ class PRGenerateLabels:
         variables["diff"] = self.patches_diff  # update diff
 
         environment = Environment(undefined=StrictUndefined)
-        set_custom_labels(variables)
+        set_custom_labels(variables, self.git_provider)
         system_prompt = environment.from_string(get_settings().pr_custom_labels_prompt.system).render(variables)
         user_prompt = environment.from_string(get_settings().pr_custom_labels_prompt.user).render(variables)
 

pr_agent/tools/pr_reviewer.py

@@ -392,7 +392,7 @@ class PRReviewer:
                     if security_concerns_bool:
                         review_labels.append('Possible security concern')
 
-                current_labels = self.git_provider.get_labels()
+                current_labels = self.git_provider.get_pr_labels()
                 current_labels_filtered = [label for label in current_labels if
                                            not label.lower().startswith('review effort [1-5]:') and not label.lower().startswith(
                                                'possible security concern')]

tests/unittest/test_convert_to_markdown.py

@@ -71,7 +71,7 @@ class TestConvertToMarkdown:
 - 📌 **Type of PR:** Test type\n\
 - 🧪 **Relevant tests added:** no\n\
 - ✨ **Focused PR:** Yes\n\
--  **General PR suggestions:** general suggestion...\n\n\n- <details><summary> 🤖 Code feedback:</summary>\n\n  - **Code example:**\n    - **Before:**\n        ```\n        Code before\n        ```\n    - **After:**\n        ```\n        Code after\n        ```\n\n  - **Code example:**\n    - **Before:**\n        ```\n        Code before 2\n        ```\n    - **After:**\n        ```\n        Code after 2\n        ```\n\n</details>\
+-  **General PR suggestions:** general suggestion...\n\n\n- <details><summary> 🤖 Code feedback:</summary>  - **Code example:**\n    - **Before:**\n        ```\n        Code before\n        ```\n    - **After:**\n        ```\n        Code after\n        ```\n\n  - **Code example:**\n    - **Before:**\n        ```\n        Code before 2\n        ```\n    - **After:**\n        ```\n        Code after 2\n        ```\n\n</details>\
 """
         assert convert_to_markdown(input_data).strip() == expected_output.strip()