Update

feat: Enable PR description publishing as comment in bitbucket_app.py

README.md

@@ -21,7 +21,7 @@ Making pull requests less painful with an AI agent
 </div>
 <div style="text-align:left;">
 
-CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull requests faster and more efficiently. It automatically analyzes the pull request and can provide several types of commands:
+CodiumAI `PR-Agent` is an open-source tool for efficient pull request reviewing and handling. It automatically analyzes the pull request and can provide several types of commands:
 
 ‣ **Auto Description ([`/describe`](./docs/DESCRIBE.md))**: Automatically generating PR description - title, type, summary, code walkthrough and labels.
 \
@@ -33,17 +33,17 @@ CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull
 \
 ‣ **Update Changelog ([`/update_changelog`](./docs/UPDATE_CHANGELOG.md))**: Automatically updating the CHANGELOG.md file with the PR changes.
 \
-‣ **Find Similar Issue ([`/similar_issue`](./docs/SIMILAR_ISSUE.md))**: Automatically retrieves and presents similar issues
+‣ **Find Similar Issue ([`/similar_issue`](./docs/SIMILAR_ISSUE.md))**: Automatically retrieves and presents similar issues.
 \
 ‣ **Add Documentation ([`/add_docs`](./docs/ADD_DOCUMENTATION.md))**: Automatically adds documentation to un-documented functions/classes in the PR.
 \
 ‣ **Generate Custom Labels ([`/generate_labels`](./docs/GENERATE_CUSTOM_LABELS.md))**: Automatically suggests custom labels based on the PR code changes.
 
-See the [Installation Guide](./INSTALL.md) for instructions how to install and run the tool on different platforms.
+See the [Installation Guide](./INSTALL.md) for instructions on installing and running the tool on different git platforms.
 
-See the [Usage Guide](./Usage.md) for instructions how to run the different tools from _CLI_, _online usage_, or by _automatically triggering_ them when a new PR is opened.
+See the [Usage Guide](./Usage.md) for running the PR-Agent commands via different interfaces, including _CLI_, _online usage_, or by _automatically triggering_ them when a new PR is opened.
 
-See the [Tools Guide](./docs/TOOLS_GUIDE.md) for detailed description of the different tools.
+See the [Tools Guide](./docs/TOOLS_GUIDE.md) for detailed description of the different tools (tools are run via the commands).
 
 <h3>Example results:</h3>
 </div>
@@ -140,7 +140,7 @@ Review the [usage guide](./Usage.md) section for detailed instructions how to us
 
 ## Try it now
 
-You can try GPT-4 powered PR-Agent, on your public GitHub repository, instantly. Just mention `@CodiumAI-Agent` and add the desired command in any PR comment. The agent will generate a response based on your command.
+Try the GPT-4 powered PR-Agent instantly on _your public GitHub repository_. Just mention `@CodiumAI-Agent` and add the desired command in any PR comment. The agent will generate a response based on your command.
 For example, add a comment to any pull request with the following text:
 ```
 @CodiumAI-Agent /review
@@ -151,6 +151,7 @@ and the agent will respond with a review of your PR
 
 
 To set up your own PR-Agent, see the [Installation](#installation) section below.
+Note that when you set your own PR-Agent or use CodiumAI hosted PR-Agent, there is no need to mention `@CodiumAI-Agent ...`. Instead, directly start with the command, e.g., `/ask ...`.
 
 ---
 

Usage.md

@@ -239,6 +239,23 @@ inline_code_comments = true
 
 Each time you invoke a `/review` tool, it will use inline code comments.
 
+#### BitBucket Self-Hosted App automatic tools
+You can configure in your local `.pr_agent.toml` file which tools will **run automatically** when a new PR is opened.
+
+Specifically, set the following values:
+```yaml
+[bitbucket_app]
+auto_review = true    # set as config var in .pr_agent.toml
+auto_describe = true  # set as config var in .pr_agent.toml
+auto_improve = true   # set as config var in .pr_agent.toml
+```
+
+`bitbucket_app.auto_review`, `bitbucket_app.auto_describe` and `bitbucket_app.auto_improve` are used to enable/disable automatic tools.
+If not set, the default option is that only the `review` tool will run automatically when a new PR is opened.
+
+Note that due to limitations of the bitbucket platform, the `auto_describe` tool will be able to publish a PR description only as a comment. 
+In addition, some subsections like `PR changes walkthrough` will not appear, since they require the usage of collapsible sections, which are not supported by bitbucket.
+
 ### Changing a model
 
 See [here](pr_agent/algo/__init__.py) for the list of available models.

pr_agent/algo/utils.py

@@ -59,14 +59,14 @@ def convert_to_markdown(output_data: dict, gfm_supported: bool=True) -> str:
             if key.lower() == 'code feedback':
                 if gfm_supported:
                     markdown_text += f"\n\n- "
-                    markdown_text += f"<details><summary> { emoji } Code feedback:</summary>\n\n"
+                    markdown_text += f"<details><summary> { emoji } Code feedback:</summary>"
                 else:
                     markdown_text += f"\n\n- **{emoji} Code feedback:**\n\n"
             else:
                 markdown_text += f"- {emoji} **{key}:**\n\n"
-            for item in value:
+            for i, item in enumerate(value):
                 if isinstance(item, dict) and key.lower() == 'code feedback':
-                    markdown_text += parse_code_suggestion(item, gfm_supported)
+                    markdown_text += parse_code_suggestion(item, i, gfm_supported)
                 elif item:
                     markdown_text += f"  - {item}\n"
             if key.lower() == 'code feedback':
@@ -80,7 +80,7 @@ def convert_to_markdown(output_data: dict, gfm_supported: bool=True) -> str:
     return markdown_text
 
 
-def parse_code_suggestion(code_suggestions: dict, gfm_supported: bool=True) -> str:
+def parse_code_suggestion(code_suggestions: dict, i: int = 0, gfm_supported: bool = True) -> str:
     """
     Convert a dictionary of data into markdown format.
 
@@ -91,24 +91,52 @@ def parse_code_suggestion(code_suggestions: dict, gfm_supported: bool=True) -> s
         str: A string containing the markdown formatted text generated from the input dictionary.
     """
     markdown_text = ""
-    for sub_key, sub_value in code_suggestions.items():
-        if isinstance(sub_value, dict):  # "code example"
-            markdown_text += f"  - **{sub_key}:**\n"
-            for code_key, code_value in sub_value.items():  # 'before' and 'after' code
-                code_str = f"```\n{code_value}\n```"
-                code_str_indented = textwrap.indent(code_str, '        ')
-                markdown_text += f"    - **{code_key}:**\n{code_str_indented}\n"
-        else:
-            if "relevant file" in sub_key.lower():
-                markdown_text += f"\n  - **{sub_key}:** {sub_value}  \n"
+    if gfm_supported and 'relevant line' in code_suggestions:
+        if i == 0:
+            markdown_text += "<hr>"
+        markdown_text += '<table>'
+        for sub_key, sub_value in code_suggestions.items():
+            try:
+                if sub_key.lower() == 'relevant file':
+                    relevant_file = sub_value.strip('`').strip('"').strip("'")
+                    markdown_text += f"<tr><td>{sub_key}</td><td>{relevant_file}</td></tr>"
+                    # continue
+                elif sub_key.lower() == 'suggestion':
+                    markdown_text += f"<tr><td>{sub_key} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</td><td><strong>{sub_value}</strong></td></tr>"
+                elif sub_key.lower() == 'relevant line':
+                    markdown_text += f"<tr><td>relevant line</td>"
+                    sub_value_list = sub_value.split('](')
+                    relevant_line = sub_value_list[0].lstrip('`').lstrip('[')
+                    if len(sub_value_list) > 1:
+                        link = sub_value_list[1].rstrip(')').strip('`')
+                        markdown_text += f"<td><a href={link}>{relevant_line}</a></td>"
+                    else:
+                        markdown_text += f"<td>{relevant_line}</td>"
+                    markdown_text += "</tr>"
+            except Exception as e:
+                get_logger().exception(f"Failed to parse code suggestion: {e}")
+                pass
+        markdown_text += '</table>'
+        markdown_text += "<hr>"
+    else:
+        for sub_key, sub_value in code_suggestions.items():
+            if isinstance(sub_value, dict):  # "code example"
+                markdown_text += f"  - **{sub_key}:**\n"
+                for code_key, code_value in sub_value.items():  # 'before' and 'after' code
+                    code_str = f"```\n{code_value}\n```"
+                    code_str_indented = textwrap.indent(code_str, '        ')
+                    markdown_text += f"    - **{code_key}:**\n{code_str_indented}\n"
             else:
-                markdown_text += f"   **{sub_key}:** {sub_value}  \n"
-            if not gfm_supported:
-                if "relevant line" not in sub_key.lower(): # nicer presentation
+                if "relevant file" in sub_key.lower():
+                    markdown_text += f"\n  - **{sub_key}:** {sub_value}  \n"
+                else:
+                    markdown_text += f"   **{sub_key}:** {sub_value}  \n"
+                if not gfm_supported:
+                    if "relevant line" not in sub_key.lower():  # nicer presentation
                         # markdown_text = markdown_text.rstrip('\n') + "\\\n" # works for gitlab
                         markdown_text = markdown_text.rstrip('\n') + "   \n"  # works for gitlab and bitbucker
 
-    markdown_text += "\n"
+        markdown_text += "\n"
     return markdown_text
 
 
@@ -336,7 +364,7 @@ def try_fix_yaml(response_text: str) -> dict:
         pass
 
 
-def set_custom_labels(variables):
+def set_custom_labels(variables, git_provider=None):
     if not get_settings().config.enable_custom_labels:
         return
 
@@ -348,11 +376,8 @@ def set_custom_labels(variables):
         labels_list = f"      - {labels_list}" if labels_list else ""
         variables["custom_labels"] = labels_list
         return
-    #final_labels = ""
-    #for k, v in labels.items():
-    #    final_labels += f"      - {k} ({v['description']})\n"
-    #variables["custom_labels"] = final_labels
-    #variables["custom_labels_examples"] = f"      - {list(labels.keys())[0]}"
+
+    # Set custom labels
     variables["custom_labels_class"] = "class Label(str, Enum):"
     for k, v in labels.items():
         description = v['description'].strip('\n').replace('\n', '\\n')

pr_agent/config_loader.py

@@ -26,6 +26,7 @@ global_settings = Dynaconf(
         "settings/pr_custom_labels.toml",
         "settings/pr_add_docs.toml",
         "settings_prod/.secrets.toml",
+        "settings_prod/.secrets_foo.toml",
         "settings/custom_labels.toml"
     ]]
 )

pr_agent/git_providers/bitbucket_provider.py

@@ -354,5 +354,5 @@ class BitbucketProvider(GitProvider):
         pass
     
     # bitbucket does not support labels
-    def get_labels(self):
+    def get_pr_labels(self):
         pass

pr_agent/git_providers/bitbucket_server_provider.py

@@ -344,7 +344,7 @@ class BitbucketServerProvider(GitProvider):
         pass
     
     # bitbucket does not support labels
-    def get_labels(self):
+    def get_pr_labels(self):
         pass
 
     def _get_pr_comments_url(self):

pr_agent/git_providers/codecommit_provider.py

@@ -216,7 +216,7 @@ class CodeCommitProvider(GitProvider):
     def publish_labels(self, labels):
         return [""]  # not implemented yet
 
-    def get_labels(self):
+    def get_pr_labels(self):
         return [""]  # not implemented yet
 
     def remove_initial_comment(self):

pr_agent/git_providers/gerrit_provider.py

@@ -207,7 +207,7 @@ class GerritProvider(GitProvider):
         Comment = namedtuple('Comment', ['body'])
         return Comments([Comment(c['message']) for c in reversed(comments)])
 
-    def get_labels(self):
+    def get_pr_labels(self):
         raise NotImplementedError(
             'Getting labels is not implemented for the gerrit provider')
 

pr_agent/git_providers/git_provider.py

@@ -135,7 +135,10 @@ class GitProvider(ABC):
         pass
 
     @abstractmethod
-    def get_labels(self):
+    def get_pr_labels(self):
+        pass
+
+    def get_repo_labels(self):
         pass
 
     @abstractmethod

pr_agent/git_providers/github_provider.py

@@ -461,13 +461,17 @@ class GithubProvider(GitProvider):
         except Exception as e:
             get_logger().exception(f"Failed to publish labels, error: {e}")
 
-    def get_labels(self):
+    def get_pr_labels(self):
         try:
             return [label.name for label in self.pr.labels]
         except Exception as e:
             get_logger().exception(f"Failed to get labels, error: {e}")
             return []
 
+    def get_repo_labels(self):
+        labels = self.repo_obj.get_labels()
+        return [label for label in labels]
+
     def get_commit_messages(self):
         """
         Retrieves the commit messages of a pull request.

pr_agent/git_providers/gitlab_provider.py

@@ -211,7 +211,11 @@ class GitLabProvider(GitProvider):
                 pos_obj['new_line'] = target_line_no - 1
                 pos_obj['old_line'] = source_line_no - 1
             get_logger().debug(f"Creating comment in {self.id_mr} with body {body} and position {pos_obj}")
-            self.mr.discussions.create({'body': body, 'position': pos_obj})
+            try:
+                self.mr.discussions.create({'body': body, 'position': pos_obj})
+            except Exception as e:
+                get_logger().debug(
+                    f"Failed to create comment in {self.id_mr} with position {pos_obj} (probably not a '+' line)")
 
     def get_relevant_diff(self, relevant_file: str, relevant_line_in_file: int) -> Optional[dict]:
         changes = self.mr.changes()  # Retrieve the changes for the merge request once
@@ -404,7 +408,7 @@ class GitLabProvider(GitProvider):
     def publish_inline_comments(self, comments: list[dict]):
         pass
 
-    def get_labels(self):
+    def get_pr_labels(self):
         return self.mr.labels
 
     def get_commit_messages(self):

pr_agent/git_providers/local_git_provider.py

@@ -178,5 +178,5 @@ class LocalGitProvider(GitProvider):
     def get_issue_comments(self):
         raise NotImplementedError('Getting issue comments is not implemented for the local git provider')
 
-    def get_labels(self):
+    def get_pr_labels(self):
         raise NotImplementedError('Getting labels is not implemented for the local git provider')

pr_agent/servers/bitbucket_app.py

@@ -16,8 +16,13 @@ from starlette_context.middleware import RawContextMiddleware
 
 from pr_agent.agent.pr_agent import PRAgent
 from pr_agent.config_loader import get_settings, global_settings
+from pr_agent.git_providers.utils import apply_repo_settings
 from pr_agent.log import LoggingFormat, get_logger, setup_logger
 from pr_agent.secret_providers import get_secret_provider
+from pr_agent.servers.github_action_runner import get_setting_or_env, is_true
+from pr_agent.tools.pr_code_suggestions import PRCodeSuggestions
+from pr_agent.tools.pr_description import PRDescription
+from pr_agent.tools.pr_reviewer import PRReviewer
 
 setup_logger(fmt=LoggingFormat.JSON)
 router = APIRouter()
@@ -89,8 +94,20 @@ async def handle_github_webhooks(background_tasks: BackgroundTasks, request: Req
                 pr_url = data["data"]["pullrequest"]["links"]["html"]["href"]
                 log_context["api_url"] = pr_url
                 log_context["event"] = "pull_request"
-                with get_logger().contextualize(**log_context):
-                    await agent.handle_request(pr_url, "review")
+                if pr_url:
+                    with get_logger().contextualize(**log_context):
+                        apply_repo_settings(pr_url)
+                        auto_review = get_setting_or_env("BITBUCKET_APP.AUTO_REVIEW", None)
+                        if auto_review is None or is_true(auto_review):  # by default, auto review is enabled
+                            await PRReviewer(pr_url).run()
+                        auto_improve = get_setting_or_env("BITBUCKET_APP.AUTO_IMPROVE", None)
+                        if is_true(auto_improve):  # by default, auto improve is disabled
+                            await PRCodeSuggestions(pr_url).run()
+                        auto_describe = get_setting_or_env("BITBUCKET_APP.AUTO_DESCRIBE", None)
+                        if is_true(auto_describe):  # by default, auto describe is disabled
+                            await PRDescription(pr_url).run()
+                # with get_logger().contextualize(**log_context):
+                #     await agent.handle_request(pr_url, "review")
             elif event == "pullrequest:comment_created":
                 pr_url = data["data"]["pullrequest"]["links"]["html"]["href"]
                 log_context["api_url"] = pr_url

pr_agent/servers/github_app.py

@@ -125,11 +125,15 @@ async def handle_request(body: Dict[str, Any], event: str):
             await _perform_commands("pr_commands", agent, body, api_url, log_context)
 
     # handle pull_request event with synchronize action - "push trigger" for new commits
-    elif event == 'pull_request' and action == 'synchronize' and get_settings().github_app.handle_push_trigger:
+    elif event == 'pull_request' and action == 'synchronize':
         pull_request, api_url = _check_pull_request_event(action, body, log_context, bot_user)
         if not (pull_request and api_url):
             return {}
 
+        apply_repo_settings(api_url)
+        if not get_settings().github_app.handle_push_trigger:
+            return {}
+
         # TODO: do we still want to get the list of commits to filter bot/merge commits?
         before_sha = body.get("before")
         after_sha = body.get("after")

pr_agent/settings/configuration.toml

@@ -143,6 +143,12 @@ magic_word = "AutoReview"
 # Polling interval
 polling_interval_seconds = 30
 
+[bitbucket_app]
+#auto_review = true    # set as config var in .pr_agent.toml
+#auto_describe = true  # set as config var in .pr_agent.toml
+#auto_improve = true   # set as config var in .pr_agent.toml
+
+
 [local]
 # LocalGitProvider settings - uncomment to use paths other than default
 # description_path= "path/to/description.md"
@@ -170,3 +176,4 @@ max_issues_to_scan = 500
 # fill and place in .secrets.toml
 #api_key = ...
 # environment = "gcp-starter"
+

pr_agent/tools/pr_description.py

@@ -102,11 +102,12 @@ class PRDescription:
             if get_settings().config.publish_output:
                 get_logger().info(f"Pushing answer {self.pr_id}")
                 if get_settings().pr_description.publish_description_as_comment:
+                    get_logger().info(f"Publishing answer as comment")
                     self.git_provider.publish_comment(full_markdown_description)
                 else:
                     self.git_provider.publish_description(pr_title, pr_body)
                     if get_settings().pr_description.publish_labels and self.git_provider.is_supported("get_labels"):
-                        current_labels = self.git_provider.get_labels()
+                        current_labels = self.git_provider.get_pr_labels()
                         user_labels = get_user_labels(current_labels)
                         self.git_provider.publish_labels(pr_labels + user_labels)
 
@@ -158,7 +159,7 @@ class PRDescription:
         variables["diff"] = self.patches_diff  # update diff
 
         environment = Environment(undefined=StrictUndefined)
-        set_custom_labels(variables)
+        set_custom_labels(variables, self.git_provider)
         system_prompt = environment.from_string(get_settings().pr_description_prompt.system).render(variables)
         user_prompt = environment.from_string(get_settings().pr_description_prompt.user).render(variables)
 
@@ -290,7 +291,7 @@ class PRDescription:
                     value = ', '.join(v for v in value)
                 pr_body += f"{value}\n"
             if idx < len(self.data) - 1:
-                pr_body += "\n___\n"
+                pr_body += "\n\n___\n\n"
 
         if get_settings().config.verbosity_level >= 2:
             get_logger().info(f"title:\n{title}\n{pr_body}")
@@ -315,7 +316,6 @@ class PRDescription:
         if not self.git_provider.is_supported("gfm_markdown"):
             get_logger().info(f"Disabling semantic files types for {self.pr_id} since gfm_markdown is not supported")
             return pr_body
-
         try:
             pr_body += "<table>"
             header = f"Relevant files"

pr_agent/tools/pr_generate_labels.py

@@ -82,7 +82,7 @@ class PRGenerateLabels:
             if get_settings().config.publish_output:
                 get_logger().info(f"Pushing labels {self.pr_id}")
 
-                current_labels = self.git_provider.get_labels()
+                current_labels = self.git_provider.get_pr_labels()
                 user_labels = get_user_labels(current_labels)
                 pr_labels = pr_labels + user_labels
 
@@ -132,7 +132,7 @@ class PRGenerateLabels:
         variables["diff"] = self.patches_diff  # update diff
 
         environment = Environment(undefined=StrictUndefined)
-        set_custom_labels(variables)
+        set_custom_labels(variables, self.git_provider)
         system_prompt = environment.from_string(get_settings().pr_custom_labels_prompt.system).render(variables)
         user_prompt = environment.from_string(get_settings().pr_custom_labels_prompt.user).render(variables)
 

pr_agent/tools/pr_reviewer.py

@@ -392,7 +392,7 @@ class PRReviewer:
                     if security_concerns_bool:
                         review_labels.append('Possible security concern')
 
-                current_labels = self.git_provider.get_labels()
+                current_labels = self.git_provider.get_pr_labels()
                 current_labels_filtered = [label for label in current_labels if
                                            not label.lower().startswith('review effort [1-5]:') and not label.lower().startswith(
                                                'possible security concern')]

tests/unittest/test_convert_to_markdown.py

@@ -71,7 +71,7 @@ class TestConvertToMarkdown:
 - 📌 **Type of PR:** Test type\n\
 - 🧪 **Relevant tests added:** no\n\
 - ✨ **Focused PR:** Yes\n\
--  **General PR suggestions:** general suggestion...\n\n\n- <details><summary> 🤖 Code feedback:</summary>\n\n  - **Code example:**\n    - **Before:**\n        ```\n        Code before\n        ```\n    - **After:**\n        ```\n        Code after\n        ```\n\n  - **Code example:**\n    - **Before:**\n        ```\n        Code before 2\n        ```\n    - **After:**\n        ```\n        Code after 2\n        ```\n\n</details>\
+-  **General PR suggestions:** general suggestion...\n\n\n- <details><summary> 🤖 Code feedback:</summary>  - **Code example:**\n    - **Before:**\n        ```\n        Code before\n        ```\n    - **After:**\n        ```\n        Code after\n        ```\n\n  - **Code example:**\n    - **Before:**\n        ```\n        Code before 2\n        ```\n    - **After:**\n        ```\n        Code after 2\n        ```\n\n</details>\
 """
         assert convert_to_markdown(input_data).strip() == expected_output.strip()