docs: add auto best practices documentation and configuration

This commit is contained in:
mrT23
2025-01-26 12:02:27 +02:00
parent 83baac975d
commit 4d52715d25
5 changed files with 112 additions and 2 deletions

View File

@ -0,0 +1,64 @@
# Auto Best Practices 💎
`Supported Git Platforms: GitHub`
## Overview
### Finding Code Problems - Exploration Phase
The Qodo Merge `improve` tool identifies potential issues, problems and bugs in Pull Request (PR) code changes.
Rather than focusing on generic concepts like code style, the tool intelligently analyzes code to detect meaningful problems.
The analysis is intentionally broad to allow discovery of diverse issues - from subtle bugs to architectural concerns.
This open-ended exploration enables the tool to surface any relevant issues without being constrained to predefined categories.
### Tracking Implemented Suggestions
Qodo Merge features a novel [tracking system](https://qodo-merge-docs.qodo.ai/tools/improve/#suggestion-tracking) that automatically detects when PR authors implement AI-generated code suggestions.
All accepted suggestions are documented in a repository-specific wiki page called [`.pr_agent_accepted_suggestions`](https://github.com/qodo-ai/pr-agent/wiki/.pr_agent_accepted_suggestions).
### Learning and Applying Auto Best Practices
Monthly, Qodo Merge analyzes the collection of accepted suggestions to generate repository-specific best practices, stored in [`.pr_agent_auto_best_practices`](https://github.com/qodo-ai/pr-agent/wiki/.pr_agent_auto_best_practices) wiki file.
These best practices reflect recurring patterns in accepted code improvements.
The `improve` tool will incorporate these best practices as an additional analysis layer, checking PR code changes against known patterns of previously accepted improvements.
This creates a two-phase analysis:
1. Open exploration for general code issues
2. Targeted checking against established best practices - exploiting the knowledge gained from past suggestions
By keeping these phases decoupled, the tool remains free to discover new or unseen issues and problems, while also learning from past experiences.
When presenting the suggestions generated by the `improve` tool, Qodo Merge will add a dedicated label for each suggestion generated from the auto best practices - 'Learned Best Practice':
![Auto best practice suggestion](https://www.qodo.ai/images/pr_agent/auto_best_practices.png){width=512}
## Auto Best Practices vs Custom Best Practices
Teams and companies can also manually define their own [custom best practices](https://qodo-merge-docs.qodo.ai/tools/improve/#best-practices) in Qodo Merge to evaluate PR code changes.
When custom best practices exist, Qodo Merge will still generate an 'auto best practices' wiki file, though it won't use it in the `improve` tool analysis.
However, this auto-generated file can still serve two valuable purposes:
1. It can help enhance your custom best practices with additional insights derived from suggestions your team found valuable enough to implement
2. It demonstrates effective patterns for writing AI-friendly best practices
We recommend regularly reviewing the auto best practices file to refine your custom rules, even when using custom best practices.
## Relevant configurations
```toml
[auto_best_practices]
# Disable all auto best practices usage or generation
enable_auto_best_practices = true
# Disable usage of auto best practices file in the 'improve' tool
utilize_auto_best_practices = true
# Extra instructions to the auto best practices generation prompt
extra_instructions = ""
# Max number of patterns to be detected
max_patterns = 5
```

View File

@ -2,6 +2,7 @@
Qodo Merge utilizes a variety of core abilities to provide a comprehensive and efficient code review experience. These abilities include: Qodo Merge utilizes a variety of core abilities to provide a comprehensive and efficient code review experience. These abilities include:
- [Fetching ticket context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/) - [Fetching ticket context](https://qodo-merge-docs.qodo.ai/core-abilities/fetching_ticket_context/)
- [Auto best practices](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/)
- [Local and global metadata](https://qodo-merge-docs.qodo.ai/core-abilities/metadata/) - [Local and global metadata](https://qodo-merge-docs.qodo.ai/core-abilities/metadata/)
- [Dynamic context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic_context/) - [Dynamic context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic_context/)
- [Self-reflection](https://qodo-merge-docs.qodo.ai/core-abilities/self_reflection/) - [Self-reflection](https://qodo-merge-docs.qodo.ai/core-abilities/self_reflection/)

View File

@ -70,7 +70,7 @@ In post-process, Qodo Merge counts the number of suggestions that were implement
## Suggestion tracking 💎 ## Suggestion tracking 💎
`Platforms supported: GitHub, GitLab` `Platforms supported: GitHub, GitLab`
Qodo Merge employs an novel detection system to automatically [identify](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) AI code suggestions that PR authors have accepted and implemented. Qodo Merge employs a novel detection system to automatically [identify](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) AI code suggestions that PR authors have accepted and implemented.
Accepted suggestions are also automatically documented in a dedicated wiki page called `.pr_agent_accepted_suggestions`, allowing users to track historical changes, assess the tool's effectiveness, and learn from previously implemented recommendations in the repository. Accepted suggestions are also automatically documented in a dedicated wiki page called `.pr_agent_accepted_suggestions`, allowing users to track historical changes, assess the tool's effectiveness, and learn from previously implemented recommendations in the repository.
An example [result](https://github.com/Codium-ai/pr-agent/wiki/.pr_agent_accepted_suggestions): An example [result](https://github.com/Codium-ai/pr-agent/wiki/.pr_agent_accepted_suggestions):
@ -192,7 +192,7 @@ And the label will be: `{organization_name} best practice`.
![best_practice](https://codium.ai/images/pr_agent/org_best_practice.png){width=512} ![best_practice](https://codium.ai/images/pr_agent/org_best_practice.png){width=512}
### How to combine `extra instructions` and `best practices` ### Combining `extra instructions` and `best practices` 💎
The `extra instructions` configuration is more related to the `improve` tool prompt. It can be used, for example, to avoid specific suggestions ("Don't suggest to add try-except block", "Ignore changes in toml files", ...) or to emphasize specific aspects or formats ("Answer in Japanese", "Give only short suggestions", ...) The `extra instructions` configuration is more related to the `improve` tool prompt. It can be used, for example, to avoid specific suggestions ("Don't suggest to add try-except block", "Ignore changes in toml files", ...) or to emphasize specific aspects or formats ("Answer in Japanese", "Give only short suggestions", ...)
@ -200,6 +200,41 @@ In contrast, the `best_practices.md` file is a general guideline for the way cod
Using a combination of both can help the AI model to provide relevant and tailored suggestions. Using a combination of both can help the AI model to provide relevant and tailored suggestions.
### Auto best practices 💎
>`Platforms supported: GitHub`
'Auto best practices' is a novel Qodo Merge capability that:
1. Identifies recurring patterns from accepted suggestions
2. **Automatically** generates [best practices page](https://github.com/qodo-ai/pr-agent/wiki/.pr_agent_auto_best_practices) based on what your team consistently values
3. Applies these learned patterns to future code reviews
This creates an automatic feedback loop where the system continuously learns from your team's choices to provide increasingly relevant suggestions.
The system maintains two analysis phases:
- Open exploration for new issues
- Targeted checking against established best practices
Note that when a [custom best practices](https://qodo-merge-docs.qodo.ai/tools/improve/#best-practices) exist, Qodo Merge will still generate an 'auto best practices' wiki file, though it won't use it in the `improve` tool.
Learn more about utilizing 'auto best practices' in our [detailed guide](https://qodo-merge-docs.qodo.ai/core-abilities/auto_best_practices/).
#### Relevant configurations
```toml
[auto_best_practices]
# Disable all auto best practices usage or generation
enable_auto_best_practices = true
# Disable usage of auto best practices file in the 'improve' tool
utilize_auto_best_practices = true
# Extra instructions to the auto best practices generation prompt
extra_instructions = ""
# Max number of patterns to be detected
max_patterns = 5
```
## Usage Tips ## Usage Tips

View File

@ -42,6 +42,7 @@ nav:
- Core Abilities: - Core Abilities:
- 'core-abilities/index.md' - 'core-abilities/index.md'
- Fetching ticket context: 'core-abilities/fetching_ticket_context.md' - Fetching ticket context: 'core-abilities/fetching_ticket_context.md'
- Auto best practices: 'core-abilities/auto_best_practices.md'
- Local and global metadata: 'core-abilities/metadata.md' - Local and global metadata: 'core-abilities/metadata.md'
- Dynamic context: 'core-abilities/dynamic_context.md' - Dynamic context: 'core-abilities/dynamic_context.md'
- Self-reflection: 'core-abilities/self_reflection.md' - Self-reflection: 'core-abilities/self_reflection.md'

View File

@ -307,7 +307,16 @@ number_of_results = 5
[lancedb] [lancedb]
uri = "./lancedb" uri = "./lancedb"
[best_practices] [best_practices]
content = "" content = ""
organization_name = ""
max_lines_allowed = 800 max_lines_allowed = 800
enable_global_best_practices = false enable_global_best_practices = false
[auto_best_practices]
enable_auto_best_practices = true # public - general flag to disable all auto best practices usage
utilize_auto_best_practices = true # public - disable usage of auto best practices in the 'improve' tool
extra_instructions = "" # public - extra instructions to the auto best practices generation prompt
content = ""
max_patterns = 5 # max number of patterns to be detected