mirror of
https://github.com/qodo-ai/pr-agent.git
synced 2025-07-14 01:30:37 +08:00
Compare commits
1 Commits
1331
...
mrT23-patc
| Author | SHA1 | Date | |
|---|---|---|---|
| b9e3e5603b |
@ -43,13 +43,6 @@ Qode Merge PR-Agent aims to help efficiently review and handle pull requests, by
|
||||
|
||||
## News and Updates
|
||||
|
||||
### November 3, 2024
|
||||
|
||||
Meaningful improvement to the quality of code suggestions by separating the code suggestion generation from [line number detection](https://github.com/Codium-ai/pr-agent/pull/1338)
|
||||
|
||||
<kbd></kbd>
|
||||
|
||||
|
||||
### October 27, 2024
|
||||
|
||||
Qodo Merge PR Agent will now automatically document accepted code suggestions in a dedicated wiki page (`.pr_agent_accepted_suggestions`), enabling users to track historical changes, assess the tool's effectiveness, and learn from previously implemented recommendations in the repository.
|
||||
|
||||
@ -46,5 +46,6 @@ This results in a more refined and valuable set of suggestions for the user, sav
|
||||
## Appendix - Relevant Configuration Options
|
||||
```
|
||||
[pr_code_suggestions]
|
||||
self_reflect_on_suggestions = true # Enable self-reflection on code suggestions
|
||||
suggestions_score_threshold = 0 # Filter out suggestions with a score below this threshold (0-10)
|
||||
```
|
||||
@ -3,7 +3,7 @@
|
||||
|
||||
You can use the Bitbucket Pipeline system to run Qodo Merge on every pull request open or update.
|
||||
|
||||
1. Add the following file in your repository bitbucket-pipelines.yml
|
||||
1. Add the following file in your repository bitbucket_pipelines.yml
|
||||
|
||||
```yaml
|
||||
pipelines:
|
||||
|
||||
@ -42,36 +42,21 @@ Note that if your base branches are not protected, don't set the variables as `p
|
||||
|
||||
## Run a GitLab webhook server
|
||||
|
||||
1. From the GitLab workspace or group, create an access token with "Reporter" role ("Developer" if using Pro version of the agent) and "api" scope.
|
||||
1. From the GitLab workspace or group, create an access token. Enable the "api" scope only.
|
||||
|
||||
2. Generate a random secret for your app, and save it for later. For example, you can use:
|
||||
|
||||
```
|
||||
WEBHOOK_SECRET=$(python -c "import secrets; print(secrets.token_hex(10))")
|
||||
```
|
||||
3. Follow the instructions to build the Docker image, setup a secrets file and deploy on your own server from [here](https://qodo-merge-docs.qodo.ai/installation/github/#run-as-a-github-app) steps 4-7.
|
||||
|
||||
3. Clone this repository:
|
||||
4. In the secrets file, fill in the following:
|
||||
- Your OpenAI key.
|
||||
- In the [gitlab] section, fill in personal_access_token and shared_secret. The access token can be a personal access token, or a group or project access token.
|
||||
- Set deployment_type to 'gitlab' in [configuration.toml](https://github.com/Codium-ai/pr-agent/blob/main/pr_agent/settings/configuration.toml)
|
||||
|
||||
```
|
||||
git clone https://github.com/Codium-ai/pr-agent.git
|
||||
```
|
||||
5. Create a webhook in GitLab. Set the URL to ```http[s]://<PR_AGENT_HOSTNAME>/webhook```. Set the secret token to the generated secret from step 2.
|
||||
In the "Trigger" section, check the ‘comments’ and ‘merge request events’ boxes.
|
||||
|
||||
4. Prepare variables and secrets. Skip this step if you plan on settings these as environment variables when running the agent:
|
||||
1. In the configuration file/variables:
|
||||
- Set `deployment_type` to "gitlab"
|
||||
|
||||
2. In the secrets file/variables:
|
||||
- Set your AI model key in the respective section
|
||||
- In the [gitlab] section, set `personal_access_token` (with token from step 1) and `shared_secret` (with secret from step 2)
|
||||
|
||||
|
||||
5. Build a Docker image for the app and optionally push it to a Docker repository. We'll use Dockerhub as an example:
|
||||
```
|
||||
docker build . -t gitlab_pr_agent --target gitlab_webhook -f docker/Dockerfile
|
||||
docker push codiumai/pr-agent:gitlab_webhook # Push to your Docker repository
|
||||
```
|
||||
|
||||
6. Create a webhook in GitLab. Set the URL to ```http[s]://<PR_AGENT_HOSTNAME>/webhook```, the secret token to the generated secret from step 2, and enable the triggers `push`, `comments` and `merge request events`.
|
||||
|
||||
7. Test your installation by opening a merge request or commenting on a merge request using one of CodiumAI's commands.
|
||||
boxes
|
||||
6. Test your installation by opening a merge request or commenting or a merge request using one of CodiumAI's commands.
|
||||
|
||||
@ -279,6 +279,10 @@ Using a combination of both can help the AI model to provide relevant and tailor
|
||||
<td><b>persistent_comment</b></td>
|
||||
<td>If set to true, the improve comment will be persistent, meaning that every new improve request will edit the previous one. Default is false.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>self_reflect_on_suggestions</b></td>
|
||||
<td>If set to true, the improve tool will calculate an importance score for each suggestion [1-10], and sort the suggestion labels group based on this score. Default is true.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>suggestions_score_threshold</b></td>
|
||||
<td> Any suggestion with importance score less than this threshold will be removed. Default is 0. Highly recommend not to set this value above 7-8, since above it may clip relevant suggestions that can be useful. </td>
|
||||
|
||||
@ -160,13 +160,3 @@ ignore_pr_target_branches = ["qa"]
|
||||
|
||||
Where the `ignore_pr_source_branches` and `ignore_pr_target_branches` are lists of regex patterns to match the source and target branches you want to ignore.
|
||||
They are not mutually exclusive, you can use them together or separately.
|
||||
|
||||
|
||||
To allow only specific folders (often needed in large monorepos), set:
|
||||
|
||||
```
|
||||
[config]
|
||||
allow_only_specific_folders=['folder1','folder2']
|
||||
```
|
||||
|
||||
For the configuration above, automatic feedback will only be triggered when the PR changes include files from 'folder1' or 'folder2'
|
||||
|
||||
@ -133,26 +133,9 @@ Your [application default credentials](https://cloud.google.com/docs/authenticat
|
||||
|
||||
If you do want to set explicit credentials, then you can use the `GOOGLE_APPLICATION_CREDENTIALS` environment variable set to a path to a json credentials file.
|
||||
|
||||
### Google AI Studio
|
||||
|
||||
To use [Google AI Studio](https://aistudio.google.com/) models, set the relevant models in the configuration section of the configuration file:
|
||||
|
||||
```toml
|
||||
[config] # in configuration.toml
|
||||
model="google_ai_studio/gemini-1.5-flash"
|
||||
model_turbo="google_ai_studio/gemini-1.5-flash"
|
||||
fallback_models=["google_ai_studio/gemini-1.5-flash"]
|
||||
|
||||
[google_ai_studio] # in .secrets.toml
|
||||
gemini_api_key = "..."
|
||||
```
|
||||
|
||||
If you don't want to set the API key in the .secrets.toml file, you can set the `GOOGLE_AI_STUDIO.GEMINI_API_KEY` environment variable.
|
||||
|
||||
### Anthropic
|
||||
|
||||
To use Anthropic models, set the relevant models in the configuration section of the configuration file:
|
||||
|
||||
```
|
||||
[config]
|
||||
model="anthropic/claude-3-opus-20240229"
|
||||
|
||||
@ -38,8 +38,6 @@ MAX_TOKENS = {
|
||||
'vertex_ai/gemini-1.5-pro': 1048576,
|
||||
'vertex_ai/gemini-1.5-flash': 1048576,
|
||||
'vertex_ai/gemma2': 8200,
|
||||
'gemini/gemini-1.5-pro': 1048576,
|
||||
'gemini/gemini-1.5-flash': 1048576,
|
||||
'codechat-bison': 6144,
|
||||
'codechat-bison-32k': 32000,
|
||||
'anthropic.claude-instant-v1': 100000,
|
||||
|
||||
@ -83,11 +83,6 @@ class LiteLLMAIHandler(BaseAiHandler):
|
||||
litellm.vertex_location = get_settings().get(
|
||||
"VERTEXAI.VERTEX_LOCATION", None
|
||||
)
|
||||
# Google AI Studio
|
||||
# SEE https://docs.litellm.ai/docs/providers/gemini
|
||||
if get_settings().get("GOOGLE_AI_STUDIO.GEMINI_API_KEY", None):
|
||||
os.environ["GEMINI_API_KEY"] = get_settings().google_ai_studio.gemini_api_key
|
||||
|
||||
def prepare_logs(self, response, system, user, resp, finish_reason):
|
||||
response_log = response.dict().copy()
|
||||
response_log['system'] = system
|
||||
|
||||
@ -1,7 +1,6 @@
|
||||
from os import environ
|
||||
from pr_agent.algo.ai_handlers.base_ai_handler import BaseAiHandler
|
||||
import openai
|
||||
from openai import APIError, AsyncOpenAI, RateLimitError, Timeout
|
||||
from openai.error import APIError, RateLimitError, Timeout, TryAgain
|
||||
from retry import retry
|
||||
|
||||
from pr_agent.config_loader import get_settings
|
||||
@ -15,7 +14,7 @@ class OpenAIHandler(BaseAiHandler):
|
||||
# Initialize OpenAIHandler specific attributes here
|
||||
try:
|
||||
super().__init__()
|
||||
environ["OPENAI_API_KEY"] = get_settings().openai.key
|
||||
openai.api_key = get_settings().openai.key
|
||||
if get_settings().get("OPENAI.ORG", None):
|
||||
openai.organization = get_settings().openai.org
|
||||
if get_settings().get("OPENAI.API_TYPE", None):
|
||||
@ -25,7 +24,7 @@ class OpenAIHandler(BaseAiHandler):
|
||||
if get_settings().get("OPENAI.API_VERSION", None):
|
||||
openai.api_version = get_settings().openai.api_version
|
||||
if get_settings().get("OPENAI.API_BASE", None):
|
||||
environ["OPENAI_BASE_URL"] = get_settings().openai.api_base
|
||||
openai.api_base = get_settings().openai.api_base
|
||||
|
||||
except AttributeError as e:
|
||||
raise ValueError("OpenAI key is required") from e
|
||||
@ -37,7 +36,7 @@ class OpenAIHandler(BaseAiHandler):
|
||||
"""
|
||||
return get_settings().get("OPENAI.DEPLOYMENT_ID", None)
|
||||
|
||||
@retry(exceptions=(APIError, Timeout, AttributeError, RateLimitError),
|
||||
@retry(exceptions=(APIError, Timeout, TryAgain, AttributeError, RateLimitError),
|
||||
tries=OPENAI_RETRIES, delay=2, backoff=2, jitter=(1, 3))
|
||||
async def chat_completion(self, model: str, system: str, user: str, temperature: float = 0.2):
|
||||
try:
|
||||
@ -45,19 +44,20 @@ class OpenAIHandler(BaseAiHandler):
|
||||
get_logger().info("System: ", system)
|
||||
get_logger().info("User: ", user)
|
||||
messages = [{"role": "system", "content": system}, {"role": "user", "content": user}]
|
||||
client = AsyncOpenAI()
|
||||
chat_completion = await client.chat.completions.create(
|
||||
|
||||
chat_completion = await openai.ChatCompletion.acreate(
|
||||
model=model,
|
||||
deployment_id=deployment_id,
|
||||
messages=messages,
|
||||
temperature=temperature,
|
||||
)
|
||||
resp = chat_completion.choices[0].message.content
|
||||
finish_reason = chat_completion.choices[0].finish_reason
|
||||
usage = chat_completion.usage
|
||||
resp = chat_completion["choices"][0]['message']['content']
|
||||
finish_reason = chat_completion["choices"][0]["finish_reason"]
|
||||
usage = chat_completion.get("usage")
|
||||
get_logger().info("AI response", response=resp, messages=messages, finish_reason=finish_reason,
|
||||
model=model, usage=usage)
|
||||
return resp, finish_reason
|
||||
except (APIError, Timeout) as e:
|
||||
except (APIError, Timeout, TryAgain) as e:
|
||||
get_logger().error("Error during OpenAI inference: ", e)
|
||||
raise
|
||||
except (RateLimitError) as e:
|
||||
@ -65,4 +65,4 @@ class OpenAIHandler(BaseAiHandler):
|
||||
raise
|
||||
except (Exception) as e:
|
||||
get_logger().error("Unknown error during OpenAI inference: ", e)
|
||||
raise
|
||||
raise TryAgain from e
|
||||
|
||||
@ -43,10 +43,6 @@ class PRReviewHeader(str, Enum):
|
||||
INCREMENTAL = "## Incremental PR Reviewer Guide"
|
||||
|
||||
|
||||
class PRDescriptionHeader(str, Enum):
|
||||
CHANGES_WALKTHROUGH = "### **Changes walkthrough** 📝"
|
||||
|
||||
|
||||
def get_setting(key: str) -> Any:
|
||||
try:
|
||||
key = key.upper()
|
||||
@ -1028,7 +1024,8 @@ def process_description(description_full: str) -> Tuple[str, List]:
|
||||
if not description_full:
|
||||
return "", []
|
||||
|
||||
description_split = description_full.split(PRDescriptionHeader.CHANGES_WALKTHROUGH.value)
|
||||
split_str = "### **Changes walkthrough** 📝"
|
||||
description_split = description_full.split(split_str)
|
||||
base_description_str = description_split[0]
|
||||
changes_walkthrough_str = ""
|
||||
files = []
|
||||
@ -1063,9 +1060,6 @@ def process_description(description_full: str) -> Tuple[str, List]:
|
||||
if not res or res.lastindex != 4:
|
||||
pattern_back = r'<details>\s*<summary><strong>(.*?)</strong><dd><code>(.*?)</code>.*?</summary>\s*<hr>\s*(.*?)\n\n\s*(.*?)</details>'
|
||||
res = re.search(pattern_back, file_data, re.DOTALL)
|
||||
if not res or res.lastindex != 4:
|
||||
pattern_back = r'<details>\s*<summary><strong>(.*?)</strong>\s*<dd><code>(.*?)</code>.*?</summary>\s*<hr>\s*(.*?)\s*-\s*(.*?)\s*</details>' # looking for hypen ('- ')
|
||||
res = re.search(pattern_back, file_data, re.DOTALL)
|
||||
if res and res.lastindex == 4:
|
||||
short_filename = res.group(1).strip()
|
||||
short_summary = res.group(2).strip()
|
||||
|
||||
@ -5,7 +5,7 @@ from urllib.parse import urlparse
|
||||
from ..algo.file_filter import filter_ignored
|
||||
from ..log import get_logger
|
||||
from ..algo.language_handler import is_valid_file
|
||||
from ..algo.utils import clip_tokens, find_line_number_of_relevant_line_in_file, load_large_diff, PRDescriptionHeader
|
||||
from ..algo.utils import clip_tokens, find_line_number_of_relevant_line_in_file, load_large_diff
|
||||
from ..config_loader import get_settings
|
||||
from .git_provider import GitProvider
|
||||
from pr_agent.algo.types import EDIT_TYPE, FilePatchInfo
|
||||
@ -404,7 +404,7 @@ class AzureDevopsProvider(GitProvider):
|
||||
pr_body = pr_body[:ind]
|
||||
|
||||
if len(pr_body) > MAX_PR_DESCRIPTION_AZURE_LENGTH:
|
||||
changes_walkthrough_text = PRDescriptionHeader.CHANGES_WALKTHROUGH.value
|
||||
changes_walkthrough_text = '## **Changes walkthrough**'
|
||||
ind = pr_body.find(changes_walkthrough_text)
|
||||
if ind != -1:
|
||||
pr_body = pr_body[:ind]
|
||||
|
||||
@ -43,9 +43,6 @@ api_base = "" # the base url for your local Llama 2, Code Llama, and other model
|
||||
vertex_project = "" # the google cloud platform project name for your vertexai deployment
|
||||
vertex_location = "" # the google cloud platform location for your vertexai deployment
|
||||
|
||||
[google_ai_studio]
|
||||
gemini_api_key = "" # the google AI Studio API key
|
||||
|
||||
[github]
|
||||
# ---- Set the following only for deployment type == "user"
|
||||
user_token = "" # A GitHub personal access token with 'repo' scope.
|
||||
@ -63,7 +60,6 @@ webhook_secret = "<WEBHOOK SECRET>" # Optional, may be commented out.
|
||||
[gitlab]
|
||||
# Gitlab personal access token
|
||||
personal_access_token = ""
|
||||
shared_secret = "" # webhook secret
|
||||
|
||||
[bitbucket]
|
||||
# For Bitbucket personal/repository bearer token
|
||||
|
||||
@ -7,7 +7,6 @@ fallback_models=["gpt-4o-2024-05-13"]
|
||||
git_provider="github"
|
||||
publish_output=true
|
||||
publish_output_progress=true
|
||||
publish_output_no_suggestions=true
|
||||
verbosity_level=0 # 0,1,2
|
||||
use_extra_bad_extensions=false
|
||||
# Configurations
|
||||
@ -122,6 +121,7 @@ max_history_len=4
|
||||
# enable to apply suggestion 💎
|
||||
apply_suggestions_checkbox=true
|
||||
# suggestions scoring
|
||||
self_reflect_on_suggestions=true
|
||||
suggestions_score_threshold=0 # [0-10]| recommend not to set this value above 8, since above it may clip highly relevant suggestions
|
||||
# params for '/improve --extended' mode
|
||||
auto_extended_mode=true
|
||||
|
||||
@ -14,10 +14,10 @@ The PR code diff will be in the following structured format:
|
||||
|
||||
@@ ... @@ def func1():
|
||||
__new hunk__
|
||||
unchanged code line0 in the PR
|
||||
unchanged code line1 in the PR
|
||||
+new code line2 added in the PR
|
||||
unchanged code line3 in the PR
|
||||
11 unchanged code line0 in the PR
|
||||
12 unchanged code line1 in the PR
|
||||
13 +new code line2 added in the PR
|
||||
14 unchanged code line3 in the PR
|
||||
__old hunk__
|
||||
unchanged code line0
|
||||
unchanged code line1
|
||||
@ -35,6 +35,7 @@ __new hunk__
|
||||
======
|
||||
|
||||
- In the format above, the diff is organized into separate '__new hunk__' and '__old hunk__' sections for each code chunk. '__new hunk__' contains the updated code, while '__old hunk__' shows the removed code. If no code was removed in a specific chunk, the __old hunk__ section will be omitted.
|
||||
- Line numbers were added for the '__new hunk__' sections to help referencing specific lines in the code suggestions. These numbers are for reference only and are not part of the actual code.
|
||||
- Code lines are prefixed with symbols: '+' for new code added in the PR, '-' for code removed, and ' ' for unchanged code.
|
||||
{%- if is_ai_metadata %}
|
||||
- When available, an AI-generated summary will precede each file's diff, with a high-level overview of the changes. Note that this summary may not be fully accurate or complete.
|
||||
@ -43,7 +44,7 @@ __new hunk__
|
||||
|
||||
Specific guidelines for generating code suggestions:
|
||||
- Provide up to {{ num_code_suggestions }} distinct and insightful code suggestions.
|
||||
- Focus solely on enhancing new code introduced in the PR, identified by '+' prefixes in '__new hunk__' sections.
|
||||
- Focus solely on enhancing new code introduced in the PR, identified by '+' prefixes in '__new hunk__' sections (after the line numbers).
|
||||
- Prioritize suggestions that address potential issues, critical problems, and bugs in the PR code. Avoid repeating changes already implemented in the PR. If no pertinent suggestions are applicable, return an empty list.
|
||||
- Don't suggest to add docstring, type hints, or comments, to remove unused imports, or to use more specific exception types.
|
||||
- When referencing variables or names from the code, enclose them in backticks (`). Example: "ensure that `variable_name` is..."
|
||||
@ -66,10 +67,12 @@ class CodeSuggestion(BaseModel):
|
||||
relevant_file: str = Field(description="Full path of the relevant file")
|
||||
language: str = Field(description="Programming language used by the relevant file")
|
||||
suggestion_content: str = Field(description="An actionable suggestion to enhance, improve or fix the new code introduced in the PR. Don't present here actual code snippets, just the suggestion. Be short and concise")
|
||||
existing_code: str = Field(description="A short code snippet from a '__new hunk__' section that the suggestion aims to enhance or fix. Include only complete code lines. Use ellipsis (...) for brevity if needed. This snippet should represent the specific PR code targeted for improvement.")
|
||||
existing_code: str = Field(description="A short code snippet from a '__new hunk__' section that the suggestion aims to enhance or fix. Include only complete code lines, without line numbers. Use ellipsis (...) for brevity if needed. This snippet should represent the specific PR code targeted for improvement.")
|
||||
improved_code: str = Field(description="A refined code snippet that replaces the 'existing_code' snippet after implementing the suggestion.")
|
||||
one_sentence_summary: str = Field(description="A concise, single-sentence overview of the suggested improvement. Focus on the 'what'. Be general, and avoid method or variable names.")
|
||||
label: str = Field(description="A single, descriptive label that best characterizes the suggestion type. Possible labels include 'security', 'possible bug', 'possible issue', 'performance', 'enhancement', 'best practice', 'maintainability', 'typo'. Other relevant labels are also acceptable.")
|
||||
relevant_lines_start: int = Field(description="The relevant line number, from a '__new hunk__' section, where the suggestion starts (inclusive). Should be derived from the hunk line numbers, and correspond to the beginning of the 'existing code' snippet above")
|
||||
relevant_lines_end: int = Field(description="The relevant line number, from a '__new hunk__' section, where the suggestion ends (inclusive). Should be derived from the hunk line numbers, and correspond to the end of the 'existing code' snippet above")
|
||||
label: str = Field(description="A single, descriptive label that best characterizes the suggestion type. Possible labels include 'security', 'possible bug', 'possible issue', 'performance', 'enhancement', 'best practice', 'maintainability'. Other relevant labels are also acceptable.")
|
||||
|
||||
|
||||
class PRCodeSuggestions(BaseModel):
|
||||
@ -92,6 +95,8 @@ code_suggestions:
|
||||
...
|
||||
one_sentence_summary: |
|
||||
...
|
||||
relevant_lines_start: 12
|
||||
relevant_lines_end: 13
|
||||
label: |
|
||||
...
|
||||
```
|
||||
@ -107,7 +112,7 @@ Title: '{{title}}'
|
||||
|
||||
The PR Diff:
|
||||
======
|
||||
{{ diff_no_line_numbers|trim }}
|
||||
{{ diff|trim }}
|
||||
======
|
||||
|
||||
|
||||
|
||||
@ -15,8 +15,8 @@ Be particularly vigilant for suggestions that:
|
||||
- Contradict or ignore parts of the PR's modifications
|
||||
In such cases, assign the suggestion a score of 0.
|
||||
|
||||
Evaluate each valid suggestion by scoring its potential impact on the PR's correctness, quality and functionality.
|
||||
In addition, you should also detect the line numbers in the '__new hunk__' section that correspond to the 'existing_code' snippet.
|
||||
For valid suggestions, your role is to provide an impartial and precise score assessment that accurately reflects each suggestion's potential impact on the PR's correctness, quality and functionality.
|
||||
|
||||
|
||||
Key guidelines for evaluation:
|
||||
- Thoroughly examine both the suggestion content and the corresponding PR code diff. Be vigilant for potential errors in each suggestion, ensuring they are logically sound, accurate, and directly derived from the PR code diff.
|
||||
@ -82,8 +82,6 @@ The output must be a YAML object equivalent to type $PRCodeSuggestionsFeedback,
|
||||
class CodeSuggestionFeedback(BaseModel):
|
||||
suggestion_summary: str = Field(description="Repeated from the input")
|
||||
relevant_file: str = Field(description="Repeated from the input")
|
||||
relevant_lines_start: int = Field(description="The relevant line number, from a '__new hunk__' section, where the suggestion starts (inclusive). Should be derived from the hunk line numbers, and correspond to the beginning of the relevant 'existing code' snippet")
|
||||
relevant_lines_end: int = Field(description="The relevant line number, from a '__new hunk__' section, where the suggestion ends (inclusive). Should be derived from the hunk line numbers, and correspond to the end of the relevant 'existing code' snippet")
|
||||
suggestion_score: int = Field(description="Evaluate the suggestion and assign a score from 0 to 10. Give 0 if the suggestion is wrong. For valid suggestions, score from 1 (lowest impact/importance) to 10 (highest impact/importance).")
|
||||
why: str = Field(description="Briefly explain the score given in 1-2 sentences, focusing on the suggestion's impact, relevance, and accuracy.")
|
||||
|
||||
@ -98,8 +96,6 @@ code_suggestions:
|
||||
- suggestion_summary: |
|
||||
Use a more descriptive variable name here
|
||||
relevant_file: "src/file1.py"
|
||||
relevant_lines_start: 13
|
||||
relevant_lines_end: 14
|
||||
suggestion_score: 6
|
||||
why: |
|
||||
The variable name 't' is not descriptive enough
|
||||
|
||||
@ -1,7 +1,6 @@
|
||||
import asyncio
|
||||
import copy
|
||||
import textwrap
|
||||
import traceback
|
||||
from functools import partial
|
||||
from typing import Dict, List
|
||||
from jinja2 import Environment, StrictUndefined
|
||||
@ -45,7 +44,7 @@ class PRCodeSuggestions:
|
||||
self.is_extended = self._get_is_extended(args or [])
|
||||
except:
|
||||
self.is_extended = False
|
||||
num_code_suggestions = int(get_settings().pr_code_suggestions.num_code_suggestions_per_chunk)
|
||||
num_code_suggestions = get_settings().pr_code_suggestions.num_code_suggestions_per_chunk
|
||||
|
||||
|
||||
self.ai_handler = ai_handler()
|
||||
@ -70,7 +69,6 @@ class PRCodeSuggestions:
|
||||
"description": self.pr_description,
|
||||
"language": self.main_language,
|
||||
"diff": "", # empty diff for initial calculation
|
||||
"diff_no_line_numbers": "", # empty diff for initial calculation
|
||||
"num_code_suggestions": num_code_suggestions,
|
||||
"extra_instructions": get_settings().pr_code_suggestions.extra_instructions,
|
||||
"commit_messages_str": self.git_provider.get_commit_messages(),
|
||||
@ -112,17 +110,15 @@ class PRCodeSuggestions:
|
||||
if not data:
|
||||
data = {"code_suggestions": []}
|
||||
|
||||
if (data is None or 'code_suggestions' not in data or not data['code_suggestions']):
|
||||
pr_body = "## PR Code Suggestions ✨\n\nNo code suggestions found for the PR."
|
||||
if (data is None or 'code_suggestions' not in data or not data['code_suggestions']
|
||||
and get_settings().config.publish_output):
|
||||
get_logger().warning('No code suggestions found for the PR.')
|
||||
if get_settings().config.publish_output and get_settings().config.publish_output_no_suggestions:
|
||||
get_logger().debug(f"PR output", artifact=pr_body)
|
||||
if self.progress_response:
|
||||
self.git_provider.edit_comment(self.progress_response, body=pr_body)
|
||||
else:
|
||||
self.git_provider.publish_comment(pr_body)
|
||||
pr_body = "## PR Code Suggestions ✨\n\nNo code suggestions found for the PR."
|
||||
get_logger().debug(f"PR output", artifact=pr_body)
|
||||
if self.progress_response:
|
||||
self.git_provider.edit_comment(self.progress_response, body=pr_body)
|
||||
else:
|
||||
get_settings().data = {"artifact": ""}
|
||||
self.git_provider.publish_comment(pr_body)
|
||||
return
|
||||
|
||||
if (not self.is_extended and get_settings().pr_code_suggestions.rank_suggestions) or \
|
||||
@ -199,11 +195,8 @@ class PRCodeSuggestions:
|
||||
self.git_provider.remove_comment(self.progress_response)
|
||||
else:
|
||||
get_logger().info('Code suggestions generated for PR, but not published since publish_output is False.')
|
||||
get_settings().data = {"artifact": data}
|
||||
return
|
||||
except Exception as e:
|
||||
get_logger().error(f"Failed to generate code suggestions for PR, error: {e}",
|
||||
artifact={"traceback": traceback.format_exc()})
|
||||
get_logger().error(f"Failed to generate code suggestions for PR, error: {e}")
|
||||
if get_settings().config.publish_output:
|
||||
if self.progress_response:
|
||||
self.progress_response.delete()
|
||||
@ -335,7 +328,7 @@ class PRCodeSuggestions:
|
||||
|
||||
if self.patches_diff:
|
||||
get_logger().debug(f"PR diff", artifact=self.patches_diff)
|
||||
self.prediction = await self._get_prediction(model, self.patches_diff, self.patches_diff_no_line_number)
|
||||
self.prediction = await self._get_prediction(model, self.patches_diff)
|
||||
else:
|
||||
get_logger().warning(f"Empty PR diff")
|
||||
self.prediction = None
|
||||
@ -343,76 +336,42 @@ class PRCodeSuggestions:
|
||||
data = self.prediction
|
||||
return data
|
||||
|
||||
async def _get_prediction(self, model: str, patches_diff: str, patches_diff_no_line_number: str) -> dict:
|
||||
async def _get_prediction(self, model: str, patches_diff: str) -> dict:
|
||||
variables = copy.deepcopy(self.vars)
|
||||
variables["diff"] = patches_diff # update diff
|
||||
variables["diff_no_line_numbers"] = patches_diff_no_line_number # update diff
|
||||
environment = Environment(undefined=StrictUndefined)
|
||||
system_prompt = environment.from_string(self.pr_code_suggestions_prompt_system).render(variables)
|
||||
user_prompt = environment.from_string(get_settings().pr_code_suggestions_prompt.user).render(variables)
|
||||
response, finish_reason = await self.ai_handler.chat_completion(
|
||||
model=model, temperature=get_settings().config.temperature, system=system_prompt, user=user_prompt)
|
||||
if not get_settings().config.publish_output:
|
||||
get_settings().system_prompt = system_prompt
|
||||
get_settings().user_prompt = user_prompt
|
||||
|
||||
# load suggestions from the AI response
|
||||
data = self._prepare_pr_code_suggestions(response)
|
||||
|
||||
# self-reflect on suggestions (mandatory, since line numbers are generated now here)
|
||||
model_reflection = get_settings().config.model
|
||||
response_reflect = await self.self_reflect_on_suggestions(data["code_suggestions"],
|
||||
patches_diff, model=model_reflection)
|
||||
if response_reflect:
|
||||
response_reflect_yaml = load_yaml(response_reflect)
|
||||
code_suggestions_feedback = response_reflect_yaml["code_suggestions"]
|
||||
if len(code_suggestions_feedback) == len(data["code_suggestions"]):
|
||||
for i, suggestion in enumerate(data["code_suggestions"]):
|
||||
try:
|
||||
suggestion["score"] = code_suggestions_feedback[i]["suggestion_score"]
|
||||
suggestion["score_why"] = code_suggestions_feedback[i]["why"]
|
||||
|
||||
if 'relevant_lines_start' not in suggestion:
|
||||
relevant_lines_start = code_suggestions_feedback[i].get('relevant_lines_start', -1)
|
||||
relevant_lines_end = code_suggestions_feedback[i].get('relevant_lines_end', -1)
|
||||
suggestion['relevant_lines_start'] = relevant_lines_start
|
||||
suggestion['relevant_lines_end'] = relevant_lines_end
|
||||
if relevant_lines_start < 0 or relevant_lines_end < 0:
|
||||
suggestion["score"] = 0
|
||||
|
||||
# self-reflect on suggestions
|
||||
if get_settings().pr_code_suggestions.self_reflect_on_suggestions:
|
||||
model_turbo = get_settings().config.model_turbo # use turbo model for self-reflection, since it is an easier task
|
||||
response_reflect = await self.self_reflect_on_suggestions(data["code_suggestions"],
|
||||
patches_diff, model=model_turbo)
|
||||
if response_reflect:
|
||||
response_reflect_yaml = load_yaml(response_reflect)
|
||||
code_suggestions_feedback = response_reflect_yaml["code_suggestions"]
|
||||
if len(code_suggestions_feedback) == len(data["code_suggestions"]):
|
||||
for i, suggestion in enumerate(data["code_suggestions"]):
|
||||
try:
|
||||
if get_settings().config.publish_output:
|
||||
suggestion_statistics_dict = {'score': int(suggestion["score"]),
|
||||
'label': suggestion["label"].lower().strip()}
|
||||
get_logger().info(f"PR-Agent suggestions statistics",
|
||||
statistics=suggestion_statistics_dict, analytics=True)
|
||||
except Exception as e:
|
||||
get_logger().error(f"Failed to log suggestion statistics, error: {e}")
|
||||
pass
|
||||
|
||||
except Exception as e: #
|
||||
get_logger().error(f"Error processing suggestion score {i}",
|
||||
artifact={"suggestion": suggestion,
|
||||
"code_suggestions_feedback": code_suggestions_feedback[i]})
|
||||
suggestion["score"] = 7
|
||||
suggestion["score_why"] = ""
|
||||
|
||||
# if the before and after code is the same, clear one of them
|
||||
try:
|
||||
if suggestion['existing_code'] == suggestion['improved_code']:
|
||||
get_logger().debug(
|
||||
f"edited improved suggestion {i + 1}, because equal to existing code: {suggestion['existing_code']}")
|
||||
if get_settings().pr_code_suggestions.commitable_code_suggestions:
|
||||
suggestion['improved_code'] = "" # we need 'existing_code' to locate the code in the PR
|
||||
else:
|
||||
suggestion['existing_code'] = ""
|
||||
except Exception as e:
|
||||
get_logger().error(f"Error processing suggestion {i + 1}, error: {e}")
|
||||
else:
|
||||
# get_logger().error(f"Could not self-reflect on suggestions. using default score 7")
|
||||
for i, suggestion in enumerate(data["code_suggestions"]):
|
||||
suggestion["score"] = 7
|
||||
suggestion["score_why"] = ""
|
||||
suggestion["score"] = code_suggestions_feedback[i]["suggestion_score"]
|
||||
suggestion["score_why"] = code_suggestions_feedback[i]["why"]
|
||||
except Exception as e: #
|
||||
get_logger().error(f"Error processing suggestion score {i}",
|
||||
artifact={"suggestion": suggestion,
|
||||
"code_suggestions_feedback": code_suggestions_feedback[i]})
|
||||
suggestion["score"] = 7
|
||||
suggestion["score_why"] = ""
|
||||
else:
|
||||
# get_logger().error(f"Could not self-reflect on suggestions. using default score 7")
|
||||
for i, suggestion in enumerate(data["code_suggestions"]):
|
||||
suggestion["score"] = 7
|
||||
suggestion["score_why"] = ""
|
||||
|
||||
return data
|
||||
|
||||
@ -422,10 +381,10 @@ class PRCodeSuggestions:
|
||||
suggestion_truncation_message = get_settings().get("PR_CODE_SUGGESTIONS.SUGGESTION_TRUNCATION_MESSAGE", "")
|
||||
if max_code_suggestion_length > 0:
|
||||
if len(suggestion['improved_code']) > max_code_suggestion_length:
|
||||
get_logger().info(f"Truncated suggestion from {len(suggestion['improved_code'])} "
|
||||
f"characters to {max_code_suggestion_length} characters")
|
||||
suggestion['improved_code'] = suggestion['improved_code'][:max_code_suggestion_length]
|
||||
suggestion['improved_code'] += f"\n{suggestion_truncation_message}"
|
||||
get_logger().info(f"Truncated suggestion from {len(suggestion['improved_code'])} "
|
||||
f"characters to {max_code_suggestion_length} characters")
|
||||
return suggestion
|
||||
|
||||
def _prepare_pr_code_suggestions(self, predictions: str) -> Dict:
|
||||
@ -440,7 +399,8 @@ class PRCodeSuggestions:
|
||||
one_sentence_summary_list = []
|
||||
for i, suggestion in enumerate(data['code_suggestions']):
|
||||
try:
|
||||
needed_keys = ['one_sentence_summary', 'label', 'relevant_file']
|
||||
needed_keys = ['one_sentence_summary', 'label', 'relevant_file', 'relevant_lines_start',
|
||||
'relevant_lines_end']
|
||||
is_valid_keys = True
|
||||
for key in needed_keys:
|
||||
if key not in suggestion:
|
||||
@ -462,6 +422,13 @@ class PRCodeSuggestions:
|
||||
continue
|
||||
|
||||
if ('existing_code' in suggestion) and ('improved_code' in suggestion):
|
||||
if suggestion['existing_code'] == suggestion['improved_code']:
|
||||
get_logger().debug(
|
||||
f"edited improved suggestion {i + 1}, because equal to existing code: {suggestion['existing_code']}")
|
||||
if get_settings().pr_code_suggestions.commitable_code_suggestions:
|
||||
suggestion['improved_code'] = "" # we need 'existing_code' to locate the code in the PR
|
||||
else:
|
||||
suggestion['existing_code'] = ""
|
||||
suggestion = self._truncate_if_needed(suggestion)
|
||||
one_sentence_summary_list.append(suggestion['one_sentence_summary'])
|
||||
suggestion_list.append(suggestion)
|
||||
@ -564,33 +531,9 @@ class PRCodeSuggestions:
|
||||
return True
|
||||
return False
|
||||
|
||||
def remove_line_numbers(self, patches_diff_list: List[str]) -> List[str]:
|
||||
# create a copy of the patches_diff_list, without line numbers for '__new hunk__' sections
|
||||
try:
|
||||
self.patches_diff_list_no_line_numbers = []
|
||||
for patches_diff in self.patches_diff_list:
|
||||
patches_diff_lines = patches_diff.splitlines()
|
||||
for i, line in enumerate(patches_diff_lines):
|
||||
if line.strip():
|
||||
if line[0].isdigit():
|
||||
# find the first letter in the line that starts with a valid letter
|
||||
for j, char in enumerate(line):
|
||||
if not char.isdigit():
|
||||
patches_diff_lines[i] = line[j + 1:]
|
||||
break
|
||||
self.patches_diff_list_no_line_numbers.append('\n'.join(patches_diff_lines))
|
||||
return self.patches_diff_list_no_line_numbers
|
||||
except Exception as e:
|
||||
get_logger().error(f"Error removing line numbers from patches_diff_list, error: {e}")
|
||||
return patches_diff_list
|
||||
|
||||
async def _prepare_prediction_extended(self, model: str) -> dict:
|
||||
self.patches_diff_list = get_pr_multi_diffs(self.git_provider, self.token_handler, model,
|
||||
max_calls=get_settings().pr_code_suggestions.max_number_of_calls)
|
||||
|
||||
# create a copy of the patches_diff_list, without line numbers for '__new hunk__' sections
|
||||
self.patches_diff_list_no_line_numbers = self.remove_line_numbers(self.patches_diff_list)
|
||||
|
||||
if self.patches_diff_list:
|
||||
get_logger().info(f"Number of PR chunk calls: {len(self.patches_diff_list)}")
|
||||
get_logger().debug(f"PR diff:", artifact=self.patches_diff_list)
|
||||
@ -598,14 +541,12 @@ class PRCodeSuggestions:
|
||||
# parallelize calls to AI:
|
||||
if get_settings().pr_code_suggestions.parallel_calls:
|
||||
prediction_list = await asyncio.gather(
|
||||
*[self._get_prediction(model, patches_diff, patches_diff_no_line_numbers) for
|
||||
patches_diff, patches_diff_no_line_numbers in
|
||||
zip(self.patches_diff_list, self.patches_diff_list_no_line_numbers)])
|
||||
*[self._get_prediction(model, patches_diff) for patches_diff in self.patches_diff_list])
|
||||
self.prediction_list = prediction_list
|
||||
else:
|
||||
prediction_list = []
|
||||
for patches_diff, patches_diff_no_line_numbers in zip(self.patches_diff_list, self.patches_diff_list_no_line_numbers):
|
||||
prediction = await self._get_prediction(model, patches_diff, patches_diff_no_line_numbers)
|
||||
for i, patches_diff in enumerate(self.patches_diff_list):
|
||||
prediction = await self._get_prediction(model, patches_diff)
|
||||
prediction_list.append(prediction)
|
||||
|
||||
data = {"code_suggestions": []}
|
||||
@ -614,16 +555,18 @@ class PRCodeSuggestions:
|
||||
score_threshold = max(1, int(get_settings().pr_code_suggestions.suggestions_score_threshold))
|
||||
for i, prediction in enumerate(predictions["code_suggestions"]):
|
||||
try:
|
||||
score = int(prediction.get("score", 1))
|
||||
if score >= score_threshold:
|
||||
data["code_suggestions"].append(prediction)
|
||||
if get_settings().pr_code_suggestions.self_reflect_on_suggestions:
|
||||
score = int(prediction.get("score", 1))
|
||||
if score >= score_threshold:
|
||||
data["code_suggestions"].append(prediction)
|
||||
else:
|
||||
get_logger().info(
|
||||
f"Removing suggestions {i} from call {j}, because score is {score}, and score_threshold is {score_threshold}",
|
||||
artifact=prediction)
|
||||
else:
|
||||
get_logger().info(
|
||||
f"Removing suggestions {i} from call {j}, because score is {score}, and score_threshold is {score_threshold}",
|
||||
artifact=prediction)
|
||||
data["code_suggestions"].append(prediction)
|
||||
except Exception as e:
|
||||
get_logger().error(f"Error getting PR diff for suggestion {i} in call {j}, error: {e}",
|
||||
artifact={"prediction": prediction})
|
||||
get_logger().error(f"Error getting PR diff for suggestion {i} in call {j}, error: {e}")
|
||||
self.data = data
|
||||
else:
|
||||
get_logger().warning(f"Empty PR diff list")
|
||||
@ -674,7 +617,7 @@ class PRCodeSuggestions:
|
||||
if get_settings().pr_code_suggestions.final_clip_factor != 1:
|
||||
max_len = max(
|
||||
len(data_sorted),
|
||||
int(get_settings().pr_code_suggestions.num_code_suggestions_per_chunk),
|
||||
get_settings().pr_code_suggestions.num_code_suggestions_per_chunk,
|
||||
)
|
||||
new_len = int(0.5 + max_len * get_settings().pr_code_suggestions.final_clip_factor)
|
||||
if new_len < len(data_sorted):
|
||||
@ -707,7 +650,10 @@ class PRCodeSuggestions:
|
||||
header = f"Suggestion"
|
||||
delta = 66
|
||||
header += " " * delta
|
||||
pr_body += f"""<thead><tr><td>Category</td><td align=left>{header}</td><td align=center>Score</td></tr>"""
|
||||
if get_settings().pr_code_suggestions.self_reflect_on_suggestions:
|
||||
pr_body += f"""<thead><tr><td>Category</td><td align=left>{header}</td><td align=center>Score</td></tr>"""
|
||||
else:
|
||||
pr_body += f"""<thead><tr><td>Category</td><td align=left>{header}</td></tr>"""
|
||||
pr_body += """<tbody>"""
|
||||
suggestions_labels = dict()
|
||||
# add all suggestions related to each label
|
||||
@ -718,11 +664,12 @@ class PRCodeSuggestions:
|
||||
suggestions_labels[label].append(suggestion)
|
||||
|
||||
# sort suggestions_labels by the suggestion with the highest score
|
||||
suggestions_labels = dict(
|
||||
sorted(suggestions_labels.items(), key=lambda x: max([s['score'] for s in x[1]]), reverse=True))
|
||||
# sort the suggestions inside each label group by score
|
||||
for label, suggestions in suggestions_labels.items():
|
||||
suggestions_labels[label] = sorted(suggestions, key=lambda x: x['score'], reverse=True)
|
||||
if get_settings().pr_code_suggestions.self_reflect_on_suggestions:
|
||||
suggestions_labels = dict(
|
||||
sorted(suggestions_labels.items(), key=lambda x: max([s['score'] for s in x[1]]), reverse=True))
|
||||
# sort the suggestions inside each label group by score
|
||||
for label, suggestions in suggestions_labels.items():
|
||||
suggestions_labels[label] = sorted(suggestions, key=lambda x: x['score'], reverse=True)
|
||||
|
||||
counter_suggestions = 0
|
||||
for label, suggestions in suggestions_labels.items():
|
||||
@ -781,14 +728,16 @@ class PRCodeSuggestions:
|
||||
|
||||
{example_code.rstrip()}
|
||||
"""
|
||||
pr_body += f"<details><summary>Suggestion importance[1-10]: {suggestion['score']}</summary>\n\n"
|
||||
pr_body += f"Why: {suggestion['score_why']}\n\n"
|
||||
pr_body += f"</details>"
|
||||
if get_settings().pr_code_suggestions.self_reflect_on_suggestions:
|
||||
pr_body += f"<details><summary>Suggestion importance[1-10]: {suggestion['score']}</summary>\n\n"
|
||||
pr_body += f"Why: {suggestion['score_why']}\n\n"
|
||||
pr_body += f"</details>"
|
||||
|
||||
pr_body += f"</details>"
|
||||
|
||||
# # add another column for 'score'
|
||||
pr_body += f"</td><td align=center>{suggestion['score']}\n\n"
|
||||
if get_settings().pr_code_suggestions.self_reflect_on_suggestions:
|
||||
pr_body += f"</td><td align=center>{suggestion['score']}\n\n"
|
||||
|
||||
pr_body += f"</td></tr>"
|
||||
counter_suggestions += 1
|
||||
|
||||
@ -12,7 +12,7 @@ from pr_agent.algo.ai_handlers.litellm_ai_handler import LiteLLMAIHandler
|
||||
from pr_agent.algo.pr_processing import get_pr_diff, retry_with_fallback_models, get_pr_diff_multiple_patchs, \
|
||||
OUTPUT_BUFFER_TOKENS_HARD_THRESHOLD
|
||||
from pr_agent.algo.token_handler import TokenHandler
|
||||
from pr_agent.algo.utils import set_custom_labels, PRDescriptionHeader
|
||||
from pr_agent.algo.utils import set_custom_labels
|
||||
from pr_agent.algo.utils import load_yaml, get_user_labels, ModelType, show_relevant_configurations, get_max_tokens, \
|
||||
clip_tokens
|
||||
from pr_agent.config_loader import get_settings
|
||||
@ -501,7 +501,7 @@ extra_file_yaml =
|
||||
pr_body += "</details>\n"
|
||||
elif 'pr_files' in key.lower() and get_settings().pr_description.enable_semantic_files_types:
|
||||
changes_walkthrough, pr_file_changes = self.process_pr_files_prediction(changes_walkthrough, value)
|
||||
changes_walkthrough = f"{PRDescriptionHeader.CHANGES_WALKTHROUGH.value}\n{changes_walkthrough}"
|
||||
changes_walkthrough = f"### **Changes walkthrough** 📝\n{changes_walkthrough}"
|
||||
else:
|
||||
# if the value is a list, join its items by comma
|
||||
if isinstance(value, list):
|
||||
|
||||
@ -4,12 +4,10 @@ atlassian-python-api==3.41.4
|
||||
azure-devops==7.1.0b3
|
||||
azure-identity==1.15.0
|
||||
boto3==1.33.6
|
||||
certifi==2024.8.30
|
||||
dynaconf==3.2.4
|
||||
fastapi==0.111.0
|
||||
GitPython==3.1.41
|
||||
google-cloud-aiplatform==1.38.0
|
||||
google-generativeai==0.8.3
|
||||
google-cloud-storage==2.10.0
|
||||
Jinja2==3.1.2
|
||||
litellm==1.50.2
|
||||
|
||||
Reference in New Issue
Block a user