diff --git a/docs/docs/core-abilities/auto_best_practices.md b/docs/docs/core-abilities/auto_best_practices.md new file mode 100644 index 00000000..f2cac762 --- /dev/null +++ b/docs/docs/core-abilities/auto_best_practices.md @@ -0,0 +1,64 @@ +# Auto Best Practices 💎 +`Supported Git Platforms: GitHub` + +## Overview + +### Finding Code Problems - Exploration Phase + +The Qodo Merge `improve` tool identifies potential issues, problems and bugs in Pull Request (PR) code changes. +Rather than focusing on generic concepts like code style, the tool intelligently analyzes code to detect meaningful problems. + +The analysis is intentionally broad to allow discovery of diverse issues - from subtle bugs to architectural concerns. +This open-ended exploration enables the tool to surface any relevant issues without being constrained to predefined categories. + +### Tracking Implemented Suggestions + +Qodo Merge features a novel [tracking system](https://qodo-merge-docs.qodo.ai/tools/improve/#suggestion-tracking) that automatically detects when PR authors implement AI-generated code suggestions. +All accepted suggestions are documented in a repository-specific wiki page called [`.pr_agent_accepted_suggestions`](https://github.com/qodo-ai/pr-agent/wiki/.pr_agent_accepted_suggestions). + +### Learning and Applying Auto Best Practices + +Monthly, Qodo Merge analyzes the collection of accepted suggestions to generate repository-specific best practices, stored in [`.pr_agent_auto_best_practices`](https://github.com/qodo-ai/pr-agent/wiki/.pr_agent_auto_best_practices) wiki file. +These best practices reflect recurring patterns in accepted code improvements. + +The `improve` tool will incorporate these best practices as an additional analysis layer, checking PR code changes against known patterns of previously accepted improvements. +This creates a two-phase analysis: + +1. Open exploration for general code issues +2. Targeted checking against established best practices - exploiting the knowledge gained from past suggestions + +By keeping these phases decoupled, the tool remains free to discover new or unseen issues and problems, while also learning from past experiences. + +When presenting the suggestions generated by the `improve` tool, Qodo Merge will add a dedicated label for each suggestion generated from the auto best practices - 'Learned Best Practice': + +![Auto best practice suggestion](https://www.qodo.ai/images/pr_agent/auto_best_practices.png){width=512} + + +## Auto Best Practices vs Custom Best Practices + +Teams and companies can also manually define their own [custom best practices](https://qodo-merge-docs.qodo.ai/tools/improve/#best-practices) in Qodo Merge to evaluate PR code changes. + +When custom best practices exist, Qodo Merge will still generate an 'auto best practices' wiki file, though it won't use it in the `improve` tool analysis. +However, this auto-generated file can still serve two valuable purposes: + +1. It can help enhance your custom best practices with additional insights derived from suggestions your team found valuable enough to implement +2. It demonstrates effective patterns for writing AI-friendly best practices + +We recommend regularly reviewing the auto best practices file to refine your custom rules, even when using custom best practices. + +## Relevant configurations + +```toml +[auto_best_practices] +# Disable all auto best practices usage or generation +enable_auto_best_practices = true + +# Disable usage of auto best practices file in the 'improve' tool +utilize_auto_best_practices = true + +# Extra instructions to the auto best practices generation prompt +extra_instructions = "" + +# Max number of patterns to be detected +max_patterns = 5 +``` \ No newline at end of file diff --git a/docs/docs/core-abilities/index.md b/docs/docs/core-abilities/index.md index b8895517..a70c04de 100644 --- a/docs/docs/core-abilities/index.md +++ b/docs/docs/core-abilities/index.md @@ -2,6 +2,7 @@ Qodo Merge utilizes a variety of core abilities to provide a comprehensive and efficient code review experience. These abilities include: - [Fetching ticket context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) +- [Auto best practices](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/) - [Local and global metadata](https://qodo-merge-docs.qodo.ai/core-abilities/metadata/) - [Dynamic context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic_context/) - [Self-reflection](https://qodo-merge-docs.qodo.ai/core-abilities/self_reflection/) diff --git a/docs/docs/tools/improve.md b/docs/docs/tools/improve.md index cfb312a7..58b91a7c 100644 --- a/docs/docs/tools/improve.md +++ b/docs/docs/tools/improve.md @@ -70,7 +70,7 @@ In post-process, Qodo Merge counts the number of suggestions that were implement ## Suggestion tracking 💎 `Platforms supported: GitHub, GitLab` -Qodo Merge employs an novel detection system to automatically [identify](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) AI code suggestions that PR authors have accepted and implemented. +Qodo Merge employs a novel detection system to automatically [identify](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) AI code suggestions that PR authors have accepted and implemented. Accepted suggestions are also automatically documented in a dedicated wiki page called `.pr_agent_accepted_suggestions`, allowing users to track historical changes, assess the tool's effectiveness, and learn from previously implemented recommendations in the repository. An example [result](https://github.com/Codium-ai/pr-agent/wiki/.pr_agent_accepted_suggestions): @@ -192,7 +192,7 @@ And the label will be: `{organization_name} best practice`. ![best_practice](https://codium.ai/images/pr_agent/org_best_practice.png){width=512} -### How to combine `extra instructions` and `best practices` +### Combining `extra instructions` and `best practices` 💎 The `extra instructions` configuration is more related to the `improve` tool prompt. It can be used, for example, to avoid specific suggestions ("Don't suggest to add try-except block", "Ignore changes in toml files", ...) or to emphasize specific aspects or formats ("Answer in Japanese", "Give only short suggestions", ...) @@ -200,6 +200,41 @@ In contrast, the `best_practices.md` file is a general guideline for the way cod Using a combination of both can help the AI model to provide relevant and tailored suggestions. +### Auto best practices 💎 + +>`Platforms supported: GitHub` + +'Auto best practices' is a novel Qodo Merge capability that: + +1. Identifies recurring patterns from accepted suggestions +2. **Automatically** generates [best practices page](https://github.com/qodo-ai/pr-agent/wiki/.pr_agent_auto_best_practices) based on what your team consistently values +3. Applies these learned patterns to future code reviews + +This creates an automatic feedback loop where the system continuously learns from your team's choices to provide increasingly relevant suggestions. +The system maintains two analysis phases: + +- Open exploration for new issues +- Targeted checking against established best practices + +Note that when a [custom best practices](https://qodo-merge-docs.qodo.ai/tools/improve/#best-practices) exist, Qodo Merge will still generate an 'auto best practices' wiki file, though it won't use it in the `improve` tool. +Learn more about utilizing 'auto best practices' in our [detailed guide](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/). + +#### Relevant configurations + +```toml +[auto_best_practices] +# Disable all auto best practices usage or generation +enable_auto_best_practices = true + +# Disable usage of auto best practices file in the 'improve' tool +utilize_auto_best_practices = true + +# Extra instructions to the auto best practices generation prompt +extra_instructions = "" + +# Max number of patterns to be detected +max_patterns = 5 +``` ## Usage Tips diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 42ad14df..a817cf54 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -42,6 +42,7 @@ nav: - Core Abilities: - 'core-abilities/index.md' - Fetching ticket context: 'core-abilities/fetching_ticket_context.md' + - Auto best practices: 'core-abilities/auto_best_practices.md' - Local and global metadata: 'core-abilities/metadata.md' - Dynamic context: 'core-abilities/dynamic_context.md' - Self-reflection: 'core-abilities/self_reflection.md' diff --git a/pr_agent/settings/configuration.toml b/pr_agent/settings/configuration.toml index d285b01a..62649d5e 100644 --- a/pr_agent/settings/configuration.toml +++ b/pr_agent/settings/configuration.toml @@ -307,7 +307,16 @@ number_of_results = 5 [lancedb] uri = "./lancedb" + [best_practices] content = "" +organization_name = "" max_lines_allowed = 800 enable_global_best_practices = false + +[auto_best_practices] +enable_auto_best_practices = true # public - general flag to disable all auto best practices usage +utilize_auto_best_practices = true # public - disable usage of auto best practices in the 'improve' tool +extra_instructions = "" # public - extra instructions to the auto best practices generation prompt +content = "" +max_patterns = 5 # max number of patterns to be detected