Merge pull request #598 from Codium-ai/tr/improve_usage_guide

Enhancements to the 'improve' tool and updates to the related documentation

docs/IMPROVE.md

@@ -10,21 +10,27 @@
     - [A note on code suggestions quality](#a-note-on-code-suggestions-quality)
 
 ## Overview
-The `improve` tool scans the PR code changes, and automatically generates committable suggestions for improving the PR code.
+The `improve` tool scans the PR code changes, and automatically generates suggestions for improving the PR code.
 The tool can be triggered automatically every time a new PR is [opened](https://github.com/Codium-ai/pr-agent/blob/main/Usage.md#github-app-automatic-tools), or it can be invoked manually by commenting on any PR:
 ```
 /improve
 ```
 
-For example:
+### Summarized vs commitable code suggestions
 
-<kbd><img src=https://codium.ai/images/pr_agent/improve_comment.png width="768"></kbd>
+The code suggestions can be presented as a single comment (via `pr_code_suggestions.summarize=true`):
-
+___
----
+<kbd><img src=https://codium.ai/images/pr_agent/code_suggestions_as_comment.png width="768"></kbd>
+___
 
+Or as a separate commitable code comment for each suggestion:
+___
 <kbd><img src=https://codium.ai/images/pr_agent/improve.png width="768"></kbd>
 
 ---
+Note that  a single comment has a significantly smaller PR footprint. We recommend this mode for most cases.
+
+### Extended mode
 
 An extended mode, which does not involve PR Compression and provides more comprehensive suggestions, can be invoked by commenting on any PR:
 ```
@@ -45,7 +51,8 @@ To edit [configurations](./../pr_agent/settings/configuration.toml#L66) related
 - `extra_instructions`: Optional extra instructions to the tool. For example: "focus on the changes in the file X. Ignore change in ...".
 - `rank_suggestions`: if set to true, the tool will rank the suggestions, based on importance. Default is false.
 - `include_improved_code`: if set to true, the tool will include an improved code implementation in the suggestion. Default is true.
-
+- `summarize`: if set to true, the tool will display the suggestions in a single comment. Default is false.
+- `enable_help_text`: if set to true, the tool will display a help text in the comment. Default is true.
 #### params for '/improve --extended' mode
 - `auto_extended_mode`: enable extended mode automatically (no need for the `--extended` option). Default is false.
 - `num_code_suggestions_per_chunk`: number of code suggestions provided by the 'improve' tool, per chunk. Default is 8.
@@ -53,19 +60,6 @@ To edit [configurations](./../pr_agent/settings/configuration.toml#L66) related
 - `max_number_of_calls`: maximum number of chunks. Default is 5.
 - `final_clip_factor`: factor to remove suggestions with low confidence. Default is 0.9.
 
-#### Summarize mode
-In this mode, instead of presenting committable suggestions, the different suggestions will be combined into a single compact comment, with significantly smaller PR footprint.
-
-To invoke the summarize mode, use the following command:
-```
-/improve --pr_code_suggestions.summarize=true
-```
-
-For example:
-
-<kbd><img src=https://codium.ai/images/pr_agent/improved_summerize_open.png width="768"></kbd>
-
-___
 
 ## Usage Tips
 
@@ -87,10 +81,6 @@ Emphasize the following aspects:
 ```
 Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.
 
-### PR footprint - regular vs summarize mode
-The default mode of the `improve` tool provides committable suggestions. This mode as a high PR footprint, since each suggestion is a separate comment you need to resolve.
-If you prefer something more compact, use the [`summarize`](#summarize-mode) mode, which combines all the suggestions into a single comment.
-
 ### A note on code suggestions quality
 
 - While the current AI for code is getting better and better (GPT-4), it's not flawless. Not all the suggestions will be perfect, and a user should not accept all of them automatically.

pr_agent/servers/help.py

@@ -231,3 +231,86 @@ Note that the tool does not have "memory" of previous questions, and answers eac
         output += f"\n\nSee the [ask usage](https://github.com/Codium-ai/pr-agent/blob/main/docs/ASK.md) page for a comprehensive guide on using this tool.\n\n"
 
         return output
+
+
+    @staticmethod
+    def get_improve_usage_guide():
+        output = "**Overview:**\n"
+        output += "The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. "
+        output += "The tool can be triggered [automatically](https://github.com/Codium-ai/pr-agent/blob/main/Usage.md#github-app-automatic-tools) every time a new PR is opened, or can be invoked manually by commenting on a PR.\n"
+        output += """\
+When commenting, to edit [configurations](https://github.com/Codium-ai/pr-agent/blob/main/pr_agent/settings/configuration.toml#L69) related to the improve tool (`pr_code_suggestions` section), use the following template:
+
+```
+/improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...
+```
+
+With a [configuration file](https://github.com/Codium-ai/pr-agent/blob/main/Usage.md#working-with-github-app), use the following template:
+
+```
+[pr_code_suggestions]
+some_config1=...
+some_config2=...
+```
+    
+"""
+        output += "\n\n<table>"
+
+        # automation
+        output += "<tr><td><details> <summary><strong> Enabling\\disabling automation </strong></summary><hr>\n\n"
+        output += """\
+When you first install the app, the [default mode](https://github.com/Codium-ai/pr-agent/blob/main/Usage.md#github-app-automatic-tools) for the improve tool is:
+
+```
+pr_commands = ["/improve --pr_code_suggestions.summarize=true", ...]
+```
+
+meaning the `improve` tool will run automatically on every PR, with summarization enabled. Delete this line to disable the tool from running automatically.
+"""
+        output += "\n\n</details></td></tr>\n\n"
+
+        # extra instructions
+        output += "<tr><td><details> <summary><strong> Utilizing extra instructions</strong></summary><hr>\n\n"
+        output += '''\
+Extra instructions are very important for the `improve` tool, since they enable to guide the model to suggestions that are more relevant to the specific needs of the project.
+
+Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify relevant aspects that you want the model to focus on.
+
+Examples for extra instructions:
+
+```
+[pr_code_suggestions] # /improve #
+extra_instructions="""
+Emphasize the following aspects:
+- Does the code logic cover relevant edge cases?
+- Is the code logic clear and easy to understand?
+- Is the code logic efficient?
+...
+"""
+```
+
+Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.
+    '''
+        output += "\n\n</details></td></tr>\n\n"
+
+        # suggestions quality
+        output += "\n\n<tr><td><details> <summary><strong> A note on code suggestions quality</strong></summary><hr> \n\n"
+        output += """\
+- While the current AI for code is getting better and better (GPT-4), it's not flawless. Not all the suggestions will be perfect, and a user should not accept all of them automatically.
+- Suggestions are not meant to be simplistic. Instead, they aim to give deep feedback and raise questions, ideas and thoughts to the user, who can then use his judgment, experience, and understanding of the code base.
+- Recommended to use the 'extra_instructions' field to guide the model to suggestions that are more relevant to the specific needs of the project.
+- Best quality will be obtained by using 'improve --extended' mode.
+
+"""
+        output += "\n\n</details></td></tr>\n\n"\
+
+        # general
+        output += "\n\n<tr><td><details> <summary><strong> More PR-Agent commands</strong></summary><hr> \n\n"
+        output += HelpMessage.get_general_bot_help_text()
+        output += "\n\n</details></td></tr>\n\n"
+
+        output += "</table>"
+
+        output += f"\n\nSee the [improve usage](https://github.com/Codium-ai/pr-agent/blob/main/docs/IMPROVE.md) page for a more comprehensive guide on using this tool.\n\n"
+
+        return output

pr_agent/settings/configuration.toml

@@ -72,6 +72,7 @@ summarize = false
 include_improved_code = true
 extra_instructions = ""
 rank_suggestions = false
+enable_help_text=true
 # params for '/improve --extended' mode
 auto_extended_mode=false
 num_code_suggestions_per_chunk=8

pr_agent/tools/pr_code_suggestions.py

@@ -13,6 +13,7 @@ from pr_agent.config_loader import get_settings
 from pr_agent.git_providers import get_git_provider
 from pr_agent.git_providers.git_provider import get_main_pr_language
 from pr_agent.log import get_logger
+from pr_agent.servers.help import HelpMessage
 from pr_agent.tools.pr_description import insert_br_after_x_chars
 import difflib
 
@@ -79,9 +80,19 @@ class PRCodeSuggestions:
             if get_settings().config.publish_output:
                 get_logger().info('Pushing PR code suggestions...')
                 self.git_provider.remove_initial_comment()
-                if get_settings().pr_code_suggestions.summarize:
+                if get_settings().pr_code_suggestions.summarize and self.git_provider.is_supported("gfm_markdown"):
                     get_logger().info('Pushing summarize code suggestions...')
-                    self.publish_summarizes_suggestions(data)
+
+                    # generate summarized suggestions
+                    pr_body = self.generate_summarized_suggestions(data)
+
+                    # add usage guide
+                    if get_settings().pr_code_suggestions.enable_help_text:
+                        pr_body += "<hr>\n\n<details> <summary><strong>✨ Usage guide:</strong></summary><hr> \n\n"
+                        pr_body += HelpMessage.get_improve_usage_guide()
+                        pr_body += "\n</details>\n"
+
+                    self.git_provider.publish_comment(pr_body)
                 else:
                     get_logger().info('Pushing inline code suggestions...')
                     self.push_inline_code_suggestions(data)
@@ -298,7 +309,7 @@ class PRCodeSuggestions:
 
         return data_sorted
 
-    def publish_summarizes_suggestions(self, data: Dict):
+    def generate_summarized_suggestions(self, data: Dict) -> str:
         try:
             pr_body = "## PR Code Suggestions\n\n"
 
@@ -379,6 +390,7 @@ class PRCodeSuggestions:
                 # pr_body += "</details>"
                 pr_body += """</td></tr>"""
             pr_body += """</tr></tbody></table>"""
-            self.git_provider.publish_comment(pr_body)
+            return pr_body
         except Exception as e:
             get_logger().info(f"Failed to publish summarized code suggestions, error: {e}")
+            return ""