diff --git a/INSTALL.md b/INSTALL.md index 55c59492..88ad92bb 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -1,9 +1,23 @@ ## Installation +To get started with PR-Agent quickly, you first need to acquire two tokens: + +1. An OpenAI key from [here](https://platform.openai.com/), with access to GPT-4. +2. A GitHub personal access token (classic) with the repo scope. + +There are several ways to use PR-Agent: + +- [Method 1: Use Docker image (no installation required)](INSTALL.md#method-1-use-docker-image-no-installation-required) +- [Method 2: Run as a GitHub Action](INSTALL.md#method-2-run-as-a-github-action) +- [Method 3: Run from source](INSTALL.md#method-3-run-from-source) +- [Method 4: Run as a polling server](INSTALL.md#method-4-run-as-a-polling-server) +- [Method 5: Run as a GitHub App](INSTALL.md#method-5-run-as-a-github-app) +- [Method 6: Deploy as a Lambda Function](INSTALL.md#method-6---deploy-as-a-lambda-function) +- [Method 7: AWS CodeCommit](INSTALL.md#method-7---aws-codecommit-setup) --- -#### Method 1: Use Docker image (no installation required) +### Method 1: Use Docker image (no installation required) To request a review for a PR, or ask a question about a PR, you can run directly from the Docker image. Here's how: @@ -41,7 +55,7 @@ Possible questions you can ask include: --- -#### Method 2: Run as a GitHub Action +### Method 2: Run as a GitHub Action You can use our pre-built Github Action Docker image to run PR-Agent as a Github Action. @@ -111,7 +125,7 @@ When you open your next PR, you should see a comment from `github-actions` bot w --- -#### Method 3: Run from source +### Method 3: Run from source 1. Clone this repository: @@ -143,17 +157,9 @@ python pr_agent/cli.py --pr_url describe python pr_agent/cli.py --pr_url improve ``` -5. **Debugging LLM API Calls** -If you're testing your codium/pr-agent server, and need to see if calls were made successfully + the exact call logs, you can use the [LiteLLM Debugger tool](https://docs.litellm.ai/docs/debugging/hosted_debugging). - -You can do this by setting `litellm_debugger=true` in configuration.toml. Your Logs will be viewable in real-time @ `admin.litellm.ai/`. Set your email in the `.secrets.toml` under 'user_email'. - - - - --- -#### Method 4: Run as a polling server +### Method 4: Run as a polling server Request reviews by tagging your Github user on a PR Follow steps 1-3 of method 2. @@ -165,7 +171,7 @@ python pr_agent/servers/github_polling.py --- -#### Method 5: Run as a GitHub App +### Method 5: Run as a GitHub App Allowing you to automate the review process on your private or public repositories. 1. Create a GitHub App from the [Github Developer Portal](https://docs.github.com/en/developers/apps/creating-a-github-app). @@ -247,7 +253,7 @@ docker push codiumai/pr-agent:github_app # Push to your Docker repository --- -#### Deploy as a Lambda Function +### Method 6 - Deploy as a Lambda Function 1. Follow steps 1-5 of [Method 5](#method-5-run-as-a-github-app). 2. Build a docker image that can be used as a lambda function @@ -266,7 +272,7 @@ docker push codiumai/pr-agent:github_app # Push to your Docker repository --- -#### AWS CodeCommit Setup +### Method 7 - AWS CodeCommit Setup Not all features have been added to CodeCommit yet. As of right now, CodeCommit has been implemented to run the pr-agent CLI on the command line, using AWS credentials stored in environment variables. (More features will be added in the future.) The following is a set of instructions to have pr-agent do a review of your CodeCommit pull request from the command line: @@ -281,7 +287,7 @@ Not all features have been added to CodeCommit yet. As of right now, CodeCommit * Option B: Set `PYTHONPATH` and run the CLI in one command, for example: * `PYTHONPATH="/PATH/TO/PROJECTS/pr-agent python pr_agent/cli.py [--ARGS]` -#### AWS CodeCommit IAM Role Example +##### AWS CodeCommit IAM Role Example Example IAM permissions to that user to allow access to CodeCommit: @@ -311,7 +317,7 @@ Example IAM permissions to that user to allow access to CodeCommit: } ``` -#### AWS CodeCommit Access Key and Secret +##### AWS CodeCommit Access Key and Secret Example setting the Access Key and Secret using environment variables @@ -321,7 +327,7 @@ export AWS_SECRET_ACCESS_KEY="XXXXXXXXXXXXXXXX" export AWS_DEFAULT_REGION="us-east-1" ``` -#### AWS CodeCommit CLI Example +##### AWS CodeCommit CLI Example After you set up AWS CodeCommit using the instructions above, here is an example CLI run that tells pr-agent to **review** a given pull request. (Replace your specific PYTHONPATH and PR URL in the example) @@ -331,3 +337,10 @@ PYTHONPATH="/PATH/TO/PROJECTS/pr-agent" python pr_agent/cli.py \ --pr_url https://us-east-1.console.aws.amazon.com/codesuite/codecommit/repositories/MY_REPO_NAME/pull-requests/321 \ review ``` + +### Appendix - **Debugging LLM API Calls** +If you're testing your codium/pr-agent server, and need to see if calls were made successfully + the exact call logs, you can use the [LiteLLM Debugger tool](https://docs.litellm.ai/docs/debugging/hosted_debugging). + +You can do this by setting `litellm_debugger=true` in configuration.toml. Your Logs will be viewable in real-time @ `admin.litellm.ai/`. Set your email in the `.secrets.toml` under 'user_email'. + + \ No newline at end of file diff --git a/README.md b/README.md index 1b120241..47dca106 100644 --- a/README.md +++ b/README.md @@ -15,45 +15,45 @@ Making pull requests less painful with an AI agent
-CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull requests faster and more efficiently. It automatically analyzes the pull request and can provide several types of feedback: +CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull requests faster and more efficiently. It automatically analyzes the pull request and can provide several types of PR feedback: -**Auto-Description**: Automatically generating PR description - title, type, summary, code walkthrough and PR labels. +**Auto-Description**: Automatically generating [PR description](https://github.com/Codium-ai/pr-agent/pull/229#issue-1860711415) - title, type, summary, code walkthrough and labels. \ -**PR Review**: Adjustable feedback about the PR main theme, type, relevant tests, security issues, focus, score, and various suggestions for the PR content. +**Auto Review**: [Adjustable feedback](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695022908) about the PR main theme, type, relevant tests, security issues, score, and various suggestions for the PR content. \ -**Question Answering**: Answering free-text questions about the PR. +**Question Answering**: Answering [free-text questions](https://github.com/Codium-ai/pr-agent/pull/229#issuecomment-1695021332) about the PR. \ -**Code Suggestions**: Committable code suggestions for improving the PR. +**Code Suggestions**: [Committable code suggestions](https://github.com/Codium-ai/pr-agent/pull/229#discussion_r1306919276) for improving the PR. \ -**Update Changelog**: Automatically updating the CHANGELOG.md file with the PR changes. +**Update Changelog**: Automatically updating the CHANGELOG.md file with the [PR changes](https://github.com/Codium-ai/pr-agent/pull/168#discussion_r1282077645).

Example results:

-

/describe:

+

/describe:

-

/review:

+

/review:

-

/reflect_and_review:

+

/reflect_and_review:

-

/ask:

+

/ask:

-

/improve:

+

/improve:

@@ -82,6 +82,7 @@ CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull | | Ask | :white_check_mark: | :white_check_mark: | :white_check_mark: | | | Auto-Description | :white_check_mark: | :white_check_mark: | | | | | Improve Code | :white_check_mark: | :white_check_mark: | | | +| | ⮑ Extended | :white_check_mark: | :white_check_mark: | | | | | Reflect and Review | :white_check_mark: | | | | | | Update CHANGELOG.md | :white_check_mark: | | | | | | | | | | | @@ -134,7 +135,8 @@ There are several ways to use PR-Agent: - Request reviews by tagging your GitHub user on a PR - [Method 5: Run as a GitHub App](INSTALL.md#method-5-run-as-a-github-app) - Allowing you to automate the review process on your private or public repositories - +- [Method 6: Deploy as a Lambda Function](INSTALL.md#method-6---deploy-as-a-lambda-function) +- [Method 7: AWS CodeCommit](INSTALL.md#method-7---aws-codecommit-setup) ## How it works @@ -160,8 +162,9 @@ Here are some advantages of PR-Agent: ## Roadmap - [x] Support additional models, as a replacement for OpenAI (see [here](https://github.com/Codium-ai/pr-agent/pull/172)) -- [ ] Develop additional logic for handling large PRs +- [x] Develop additional logic for handling large PRs (see [here](https://github.com/Codium-ai/pr-agent/pull/229)) - [ ] Add additional context to the prompt. For example, repo (or relevant files) summarization, with tools such a [ctags](https://github.com/universal-ctags/ctags) +- [ ] PR-Agent for issues, and just for pull requests - [ ] Adding more tools. Possible directions: - [x] PR description - [x] Inline code suggestions diff --git a/docker/Dockerfile b/docker/Dockerfile index 8d28a9ed..4336cacc 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -2,7 +2,8 @@ FROM python:3.10 as base WORKDIR /app ADD pyproject.toml . -RUN pip install . && rm pyproject.toml +ADD requirements.txt . +RUN pip install . && rm pyproject.toml requirements.txt ENV PYTHONPATH=/app FROM base as github_app diff --git a/pr_agent/algo/ai_handler.py b/pr_agent/algo/ai_handler.py index 1a12564b..fcc5f04c 100644 --- a/pr_agent/algo/ai_handler.py +++ b/pr_agent/algo/ai_handler.py @@ -87,8 +87,6 @@ class AiHandler: f"Generating completion with {model}" f"{(' from deployment ' + deployment_id) if deployment_id else ''}" ) - if self.azure: - model = self.azure + "/" + model response = await acompletion( model=model, deployment_id=deployment_id, @@ -97,6 +95,7 @@ class AiHandler: {"role": "user", "content": user} ], temperature=temperature, + azure=self.azure, force_timeout=get_settings().config.ai_timeout ) except (APIError, Timeout, TryAgain) as e: diff --git a/pr_agent/algo/utils.py b/pr_agent/algo/utils.py index 2d7a6d39..31ca2c42 100644 --- a/pr_agent/algo/utils.py +++ b/pr_agent/algo/utils.py @@ -286,7 +286,7 @@ def _fix_key_value(key: str, value: str): def load_yaml(review_text: str) -> dict: review_text = review_text.removeprefix('```yaml').rstrip('`') try: - data = yaml.load(review_text, Loader=yaml.SafeLoader) + data = yaml.safe_load(review_text) except Exception as e: logging.error(f"Failed to parse AI prediction: {e}") data = try_fix_yaml(review_text) diff --git a/pr_agent/git_providers/__init__.py b/pr_agent/git_providers/__init__.py index dddf58c8..376d09f5 100644 --- a/pr_agent/git_providers/__init__.py +++ b/pr_agent/git_providers/__init__.py @@ -4,11 +4,13 @@ from pr_agent.git_providers.codecommit_provider import CodeCommitProvider from pr_agent.git_providers.github_provider import GithubProvider from pr_agent.git_providers.gitlab_provider import GitLabProvider from pr_agent.git_providers.local_git_provider import LocalGitProvider +from pr_agent.git_providers.azuredevops_provider import AzureDevopsProvider _GIT_PROVIDERS = { 'github': GithubProvider, 'gitlab': GitLabProvider, 'bitbucket': BitbucketProvider, + 'azure': AzureDevopsProvider, 'codecommit': CodeCommitProvider, 'local' : LocalGitProvider } diff --git a/pr_agent/git_providers/azuredevops_provider.py b/pr_agent/git_providers/azuredevops_provider.py new file mode 100644 index 00000000..71ae0947 --- /dev/null +++ b/pr_agent/git_providers/azuredevops_provider.py @@ -0,0 +1,269 @@ +import json +import logging +from typing import Optional, Tuple +from urllib.parse import urlparse + +import os + +AZURE_DEVOPS_AVAILABLE = True +try: + from msrest.authentication import BasicAuthentication + from azure.devops.connection import Connection + from azure.devops.v7_1.git.models import Comment, CommentThread, GitVersionDescriptor, GitPullRequest +except ImportError: + AZURE_DEVOPS_AVAILABLE = False + +from ..algo.pr_processing import clip_tokens +from ..config_loader import get_settings +from ..algo.utils import load_large_diff +from ..algo.language_handler import is_valid_file +from .git_provider import EDIT_TYPE, FilePatchInfo + + +class AzureDevopsProvider: + def __init__(self, pr_url: Optional[str] = None, incremental: Optional[bool] = False): + if not AZURE_DEVOPS_AVAILABLE: + raise ImportError("Azure DevOps provider is not available. Please install the required dependencies.") + + self.azure_devops_client = self._get_azure_devops_client() + + self.workspace_slug = None + self.repo_slug = None + self.repo = None + self.pr_num = None + self.pr = None + self.temp_comments = [] + self.incremental = incremental + if pr_url: + self.set_pr(pr_url) + + def is_supported(self, capability: str) -> bool: + if capability in ['get_issue_comments', 'create_inline_comment', 'publish_inline_comments', 'get_labels', 'remove_initial_comment']: + return False + return True + + def set_pr(self, pr_url: str): + self.workspace_slug, self.repo_slug, self.pr_num = self._parse_pr_url(pr_url) + self.pr = self._get_pr() + + def get_repo_settings(self): + try: + contents = self.azure_devops_client.get_item_content(repository_id=self.repo_slug, + project=self.workspace_slug, download=False, + include_content_metadata=False, include_content=True, + path=".pr_agent.toml") + return contents + except Exception as e: + logging.exception("get repo settings error") + return "" + + def get_files(self): + files = [] + for i in self.azure_devops_client.get_pull_request_commits(project=self.workspace_slug, + repository_id=self.repo_slug, + pull_request_id=self.pr_num): + + changes_obj = self.azure_devops_client.get_changes(project=self.workspace_slug, + repository_id=self.repo_slug, commit_id=i.commit_id) + + for c in changes_obj.changes: + files.append(c['item']['path']) + return list(set(files)) + + def get_diff_files(self) -> list[FilePatchInfo]: + try: + base_sha = self.pr.last_merge_target_commit + head_sha = self.pr.last_merge_source_commit + + commits = self.azure_devops_client.get_pull_request_commits(project=self.workspace_slug, + repository_id=self.repo_slug, + pull_request_id=self.pr_num) + + diff_files = [] + diffs = [] + diff_types = {} + + for c in commits: + changes_obj = self.azure_devops_client.get_changes(project=self.workspace_slug, + repository_id=self.repo_slug, commit_id=c.commit_id) + for i in changes_obj.changes: + diffs.append(i['item']['path']) + diff_types[i['item']['path']] = i['changeType'] + + diffs = list(set(diffs)) + + for file in diffs: + if not is_valid_file(file): + continue + + version = GitVersionDescriptor(version=head_sha.commit_id, version_type='commit') + new_file_content_str = self.azure_devops_client.get_item(repository_id=self.repo_slug, + path=file, + project=self.workspace_slug, + version_descriptor=version, + download=False, + include_content=True) + + new_file_content_str = new_file_content_str.content + + edit_type = EDIT_TYPE.MODIFIED + if diff_types[file] == 'add': + edit_type = EDIT_TYPE.ADDED + elif diff_types[file] == 'delete': + edit_type = EDIT_TYPE.DELETED + elif diff_types[file] == 'rename': + edit_type = EDIT_TYPE.RENAMED + + version = GitVersionDescriptor(version=base_sha.commit_id, version_type='commit') + original_file_content_str = self.azure_devops_client.get_item(repository_id=self.repo_slug, + path=file, + project=self.workspace_slug, + version_descriptor=version, + download=False, + include_content=True) + original_file_content_str = original_file_content_str.content + + patch = load_large_diff(file, new_file_content_str, original_file_content_str) + + diff_files.append(FilePatchInfo(original_file_content_str, new_file_content_str, + patch=patch, + filename=file, + edit_type=edit_type)) + + self.diff_files = diff_files + return diff_files + except Exception as e: + print(f"Error: {str(e)}") + return [] + + def publish_comment(self, pr_comment: str, is_temporary: bool = False): + comment = Comment(content=pr_comment) + thread = CommentThread(comments=[comment]) + thread_response = self.azure_devops_client.create_thread(comment_thread=thread, project=self.workspace_slug, + repository_id=self.repo_slug, + pull_request_id=self.pr_num) + if is_temporary: + self.temp_comments.append({'thread_id': thread_response.id, 'comment_id': comment.id}) + + def publish_description(self, pr_title: str, pr_body: str): + try: + updated_pr = GitPullRequest() + updated_pr.title = pr_title + updated_pr.description = pr_body + self.azure_devops_client.update_pull_request(project=self.workspace_slug, + repository_id=self.repo_slug, + pull_request_id=self.pr_num, + git_pull_request_to_update=updated_pr) + except Exception as e: + logging.exception(f"Could not update pull request {self.pr_num} description: {e}") + + def remove_initial_comment(self): + return "" # not implemented yet + + def publish_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str): + raise NotImplementedError("Azure DevOps provider does not support publishing inline comment yet") + + def create_inline_comment(self, body: str, relevant_file: str, relevant_line_in_file: str): + raise NotImplementedError("Azure DevOps provider does not support creating inline comments yet") + + def publish_inline_comments(self, comments: list[dict]): + raise NotImplementedError("Azure DevOps provider does not support publishing inline comments yet") + + def get_title(self): + return self.pr.title + + def get_languages(self): + languages = [] + files = self.azure_devops_client.get_items(project=self.workspace_slug, repository_id=self.repo_slug, + recursion_level="Full", include_content_metadata=True, + include_links=False, download=False) + for f in files: + if f.git_object_type == 'blob': + file_name, file_extension = os.path.splitext(f.path) + languages.append(file_extension[1:]) + + extension_counts = {} + for ext in languages: + if ext != '': + extension_counts[ext] = extension_counts.get(ext, 0) + 1 + + total_extensions = sum(extension_counts.values()) + + extension_percentages = {ext: (count / total_extensions) * 100 for ext, count in extension_counts.items()} + + return extension_percentages + + def get_pr_branch(self): + pr_info = self.azure_devops_client.get_pull_request_by_id(project=self.workspace_slug, + pull_request_id=self.pr_num) + source_branch = pr_info.source_ref_name.split('/')[-1] + return source_branch + + def get_pr_description(self): + max_tokens = get_settings().get("CONFIG.MAX_DESCRIPTION_TOKENS", None) + if max_tokens: + return clip_tokens(self.pr.description, max_tokens) + return self.pr.description + + def get_user_id(self): + return 0 + + def get_issue_comments(self): + raise NotImplementedError("Azure DevOps provider does not support issue comments yet") + + def add_eyes_reaction(self, issue_comment_id: int) -> Optional[int]: + return True + + def remove_reaction(self, issue_comment_id: int, reaction_id: int) -> bool: + return True + + def get_issue_comments(self): + raise NotImplementedError("Azure DevOps provider does not support issue comments yet") + + @staticmethod + def _parse_pr_url(pr_url: str) -> Tuple[str, int]: + parsed_url = urlparse(pr_url) + + if 'azure.com' not in parsed_url.netloc: + raise ValueError("The provided URL is not a valid Azure DevOps URL") + + path_parts = parsed_url.path.strip('/').split('/') + + if len(path_parts) < 6 or path_parts[4] != 'pullrequest': + raise ValueError("The provided URL does not appear to be a Azure DevOps PR URL") + + workspace_slug = path_parts[1] + repo_slug = path_parts[3] + try: + pr_number = int(path_parts[5]) + except ValueError as e: + raise ValueError("Unable to convert PR number to integer") from e + + return workspace_slug, repo_slug, pr_number + + def _get_azure_devops_client(self): + try: + pat = get_settings().azure_devops.pat + org = get_settings().azure_devops.org + except AttributeError as e: + raise ValueError( + "Azure DevOps PAT token is required ") from e + + credentials = BasicAuthentication('', pat) + azure_devops_connection = Connection(base_url=org, creds=credentials) + azure_devops_client = azure_devops_connection.clients.get_git_client() + + return azure_devops_client + + def _get_repo(self): + if self.repo is None: + self.repo = self.azure_devops_client.get_repository(project=self.workspace_slug, + repository_id=self.repo_slug) + return self.repo + + def _get_pr(self): + self.pr = self.azure_devops_client.get_pull_request_by_id(pull_request_id=self.pr_num, project=self.workspace_slug) + return self.pr + + def get_commit_messages(self): + return "" # not implemented yet diff --git a/pr_agent/git_providers/git_provider.py b/pr_agent/git_providers/git_provider.py index 72e1cf07..8eee022b 100644 --- a/pr_agent/git_providers/git_provider.py +++ b/pr_agent/git_providers/git_provider.py @@ -1,3 +1,4 @@ +import logging from abc import ABC, abstractmethod # enum EDIT_TYPE (ADDED, DELETED, MODIFIED, RENAMED) @@ -119,6 +120,8 @@ def get_main_pr_language(languages, files) -> str: # validate that the specific commit uses the main language extension_list = [] for file in files: + if isinstance(file, str): + file = FilePatchInfo(base_file=None, head_file=None, patch=None, filename=file) extension_list.append(file.filename.rsplit('.')[-1]) # get the most common extension @@ -140,10 +143,12 @@ def get_main_pr_language(languages, files) -> str: most_common_extension == 'scala' and top_language == 'scala' or \ most_common_extension == 'kt' and top_language == 'kotlin' or \ most_common_extension == 'pl' and top_language == 'perl' or \ - most_common_extension == 'swift' and top_language == 'swift': + most_common_extension == 'swift' and top_language == 'swift' or \ + most_common_extension == top_language: main_language_str = top_language - except Exception: + except Exception as e: + logging.exception(e) pass return main_language_str diff --git a/pr_agent/settings/pr_code_suggestions_prompts.toml b/pr_agent/settings/pr_code_suggestions_prompts.toml index 4e4b57e5..f60b9cc2 100644 --- a/pr_agent/settings/pr_code_suggestions_prompts.toml +++ b/pr_agent/settings/pr_code_suggestions_prompts.toml @@ -1,8 +1,8 @@ [pr_code_suggestions_prompt] -system="""You are a language model called PR-Code-Reviewer. -Your task is to provide meaningful actionable code suggestions, to improve the new code presented in a PR. +system="""You are a language model called PR-Code-Reviewer, that specializes in suggesting code improvements for Pull Request (PR). +Your task is to provide meaningful and actionable code suggestions, to improve the new code presented in a PR. -Example PR Diff input: +Example for a PR Diff input: ' ## src/file1.py @@ -10,8 +10,8 @@ Example PR Diff input: __new hunk__ 12 code line that already existed in the file... 13 code line that already existed in the file.... -14 +new code line added in the PR -15 code line that already existed in the file... +14 +new code line1 added in the PR +15 +new code line2 added in the PR 16 code line that already existed in the file... __old hunk__ code line that already existed in the file... @@ -31,13 +31,17 @@ __old hunk__ ' Specific instructions: -- Focus on important suggestions like fixing code problems, issues and bugs. As a second priority, provide suggestions for meaningful code improvements, like performance, vulnerability, modularity, and best practices. -- Suggestions should refer only to code from the '__new hunk__' sections, and focus on new lines of code (lines starting with '+'). -- Provide the exact line number range (inclusive) for each issue. -- Assume there is additional relevant code, that is not included in the diff. - Provide up to {{ num_code_suggestions }} code suggestions. -- Avoid making suggestions that have already been implemented in the PR code. For example, if you want to add logs, or change a variable to const, or anything else, make sure it isn't already in the '__new hunk__' code. -- Don't suggest to add docstring or type hints. +- Prioritize suggestions that address major problems, issues and bugs in the code. + As a second priority, suggestions should focus on best practices, code readability, maintainability, enhancments, performance, and other aspects. + Don't suggest to add docstring or type hints. + Try to provide diverse and insightful suggestions. +- Suggestions should refer only to code from the '__new hunk__' sections, and focus on new lines of code (lines starting with '+'). + Avoid making suggestions that have already been implemented in the PR code. For example, if you want to add logs, or change a variable to const, or anything else, make sure it isn't already in the '__new hunk__' code. + For each suggestion, make sure to take into consideration also the context, meaning the lines before and after the relevant code. +- Provide the exact line numbers range (inclusive) for each issue. +- Assume there is additional relevant code, that is not included in the diff. + {%- if extra_instructions %} @@ -45,63 +49,76 @@ Extra instructions from the user: {{ extra_instructions }} {%- endif %} -You must use the following JSON schema to format your answer: -```json -{ - "Code suggestions": { - "type": "array", - "minItems": 1, - "maxItems": {{ num_code_suggestions }}, - "uniqueItems": "true", - "items": { - "relevant file": { - "type": "string", - "description": "the relevant file full path" - }, - "suggestion content": { - "type": "string", - "description": "a concrete suggestion for meaningfully improving the new PR code (lines from the '__new hunk__' sections, starting with '+')." - }, - "existing code": { - "type": "string", - "description": "a code snippet showing the relevant code lines from a '__new hunk__' section. It must be continuous, correctly formatted and indented, and without line numbers." - }, - "relevant lines": { - "type": "string", - "description": "the relevant lines from a '__new hunk__' section, in the format of 'start_line-end_line'. For example: '10-15'. They should be derived from the hunk line numbers, and correspond to the 'existing code' snippet above." - }, - "improved code": { - "type": "string", - "description": "a new code snippet that can be used to replace the relevant lines in '__new hunk__' code. Replacement suggestions should be complete, correctly formatted and indented, and without line numbers." - } - } - } -} +You must use the following YAML schema to format your answer: +```yaml +Code suggestions: + type: array + minItems: 1 + maxItems: {{ num_code_suggestions }} + uniqueItems: true + items: + relevant file: + type: string + description: the relevant file full path + suggestion content: + type: string + description: |- + a concrete suggestion for meaningfully improving the new PR code. + existing code: + type: string + description: |- + a code snippet showing the relevant code lines from a '__new hunk__' section. + It must be continuous, correctly formatted and indented, and without line numbers. + relevant lines: + type: string + description: |- + the relevant lines from a '__new hunk__' section, in the format of 'start_line-end_line'. + For example: '10-15'. They should be derived from the hunk line numbers, and correspond to the 'existing code' snippet above. + improved code: + type: string + description: |- + a new code snippet that can be used to replace the relevant lines in '__new hunk__' code. + Replacement suggestions should be complete, correctly formatted and indented, and without line numbers. ``` -Don't output line numbers in the 'improved code' snippets. +Example output: +```yaml +Code suggestions: + - relevant file: |- + src/file1.py + suggestion content: |- + Add a docstring to func1() + existing code: |- + def func1(): + relevant lines: '12-12' + improved code: |- + ... +``` + + +Each YAML output MUST be after a newline, indented, with block scalar indicator ('|-'). Don't repeat the prompt in the answer, and avoid outputting the 'type' and 'description' fields. """ user="""PR Info: -Title: '{{title}}' -Branch: '{{branch}}' -Description: '{{description}}' -{%- if language %} -Main language: {{language}} -{%- endif %} -{%- if commit_messages_str %} -Commit messages: -{{commit_messages_str}} +Title: '{{title}}' + +Branch: '{{branch}}' + +Description: '{{description}}' + +{%- if language %} + +Main language: {{language}} {%- endif %} The PR Diff: ``` -{{diff}} +{{- diff|trim }} ``` -Response (should be a valid JSON, and nothing else): -```json +Response (should be a valid YAML, and nothing else): +```yaml """ diff --git a/pr_agent/tools/pr_code_suggestions.py b/pr_agent/tools/pr_code_suggestions.py index 2d9bdd33..17322bdf 100644 --- a/pr_agent/tools/pr_code_suggestions.py +++ b/pr_agent/tools/pr_code_suggestions.py @@ -1,16 +1,13 @@ import copy -import json import logging import textwrap -from typing import List - -import yaml +from typing import List, Dict from jinja2 import Environment, StrictUndefined from pr_agent.algo.ai_handler import AiHandler from pr_agent.algo.pr_processing import get_pr_diff, retry_with_fallback_models, get_pr_multi_diffs from pr_agent.algo.token_handler import TokenHandler -from pr_agent.algo.utils import try_fix_json +from pr_agent.algo.utils import load_yaml from pr_agent.config_loader import get_settings from pr_agent.git_providers import BitbucketProvider, get_git_provider from pr_agent.git_providers.git_provider import get_main_pr_language @@ -96,14 +93,11 @@ class PRCodeSuggestions: return response - def _prepare_pr_code_suggestions(self) -> str: + def _prepare_pr_code_suggestions(self) -> Dict: review = self.prediction.strip() - try: - data = json.loads(review) - except json.decoder.JSONDecodeError: - if get_settings().config.verbosity_level >= 2: - logging.info(f"Could not parse json response: {review}") - data = try_fix_json(review, code_suggestions=True) + data = load_yaml(review) + if isinstance(data, list): + data = {'Code suggestions': data} return data def push_inline_code_suggestions(self, data): @@ -225,7 +219,7 @@ class PRCodeSuggestions: response, finish_reason = await self.ai_handler.chat_completion(model=model, system=system_prompt, user=user_prompt) - sort_order = yaml.safe_load(response) + sort_order = load_yaml(response) for s in sort_order['Sort Order']: suggestion_number = s['suggestion number'] importance_order = s['importance order'] diff --git a/pyproject.toml b/pyproject.toml index 8d429668..0e1289f4 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -26,28 +26,10 @@ classifiers = [ "Operating System :: Independent", "Programming Language :: Python :: 3", ] +dynamic = ["dependencies"] -dependencies = [ - "dynaconf==3.1.12", - "fastapi==0.99.0", - "PyGithub==1.59.*", - "retry==0.9.2", - "openai==0.27.8", - "Jinja2==3.1.2", - "tiktoken==0.4.0", - "uvicorn==0.22.0", - "python-gitlab==3.15.0", - "pytest~=7.4.0", - "aiohttp~=3.8.4", - "atlassian-python-api==3.39.0", - "GitPython~=3.1.32", - "starlette-context==0.3.6", - "litellm~=0.1.445", - "PyYAML==6.0", - "boto3~=1.28.25", - "google-cloud-storage==2.10.0", - "ujson==5.8.0" -] +[tool.setuptools.dynamic] +dependencies = {file = ["requirements.txt"]} [project.urls] "Homepage" = "https://github.com/Codium-ai/pr-agent"