From 21a1cc970e02f4f13eecf5581aad75bc883fd240 Mon Sep 17 00:00:00 2001 From: mrT23 Date: Thu, 19 Oct 2023 09:16:20 +0300 Subject: [PATCH] - update readme - minor prompts change --- docs/IMPROVE.md | 12 +++++++++++- docs/REVIEW.md | 12 +++++++++++- pr_agent/settings/pr_reviewer_prompts.toml | 19 +++++++++---------- 3 files changed, 31 insertions(+), 12 deletions(-) diff --git a/docs/IMPROVE.md b/docs/IMPROVE.md index 5c55e26f..fcfca54d 100644 --- a/docs/IMPROVE.md +++ b/docs/IMPROVE.md @@ -31,4 +31,14 @@ Under the section 'pr_code_suggestions', the [configuration file](./../pr_agent/ - `num_code_suggestions_per_chunk`: number of code suggestions provided by the 'improve' tool, per chunk. Default is 8. - `rank_extended_suggestions`: if set to true, the tool will rank the suggestions, based on importance. Default is true. - `max_number_of_calls`: maximum number of chunks. Default is 5. -- `final_clip_factor`: factor to remove suggestions with low confidence. Default is 0.9. \ No newline at end of file +- `final_clip_factor`: factor to remove suggestions with low confidence. Default is 0.9. + + +#### A note on code suggestions quality + +- With current level of AI for code (GPT-4), mistakes can happen. Not all the suggestions will be perfect, and a user should not accept all of them automatically. +Suggestions are not meant to be [simplistic](./../pr_agent/settings/pr_code_suggestions_prompts.toml#L34). Instead, they aim to give deep feedback and raise questions, ideas and thoughts to the user, who can then use his judgment, experience, and understanding of the code base. + +- Recommended to use the 'extra instructions' field to guide the model to suggestions that are more relevant to the specific needs of the project. + +- Best quality will be obtained by using 'improve --extended' mode. \ No newline at end of file diff --git a/docs/REVIEW.md b/docs/REVIEW.md index 71e36707..f0798e8f 100644 --- a/docs/REVIEW.md +++ b/docs/REVIEW.md @@ -43,4 +43,14 @@ The tool will first ask the author questions about the PR, and will guide the re - \ No newline at end of file + + + +#### A note on code suggestions quality + +- With current level of AI for code (GPT-4), mistakes can happen. Not all the suggestions will be perfect, and a user should not accept all of them automatically. +Suggestions are not meant to be [simplistic](./../pr_agent/settings/pr_reviewer_prompts.toml#L29). Instead, they aim to give deep feedback and raise questions, ideas and thoughts to the user, who can then use his judgment, experience, and understanding of the code base. + +- Recommended to use the 'extra instructions' field to guide the model to suggestions that are more relevant to the specific needs of the project. + +- Unlike the 'review', which does a lot of things, the ['improve --extended'](./IMPROVE.md) feature is dedicated to suggestions, and usually gives better suggestions \ No newline at end of file diff --git a/pr_agent/settings/pr_reviewer_prompts.toml b/pr_agent/settings/pr_reviewer_prompts.toml index 657027af..cb49b5d0 100644 --- a/pr_agent/settings/pr_reviewer_prompts.toml +++ b/pr_agent/settings/pr_reviewer_prompts.toml @@ -25,7 +25,7 @@ code line that already existed in the file.... The review should focus on new code added in the PR (lines starting with '+'), and not on code that already existed in the file (lines starting with '-', or without prefix). {%- if num_code_suggestions > 0 %} -- Provide up to {{ num_code_suggestions }} code suggestions. +- Provide up to {{ num_code_suggestions }} code suggestions. Try to provide diverse and insightful suggestions. - Focus on important suggestions like fixing code problems, issues and bugs. As a second priority, provide suggestions for meaningful code improvements, like performance, vulnerability, modularity, and best practices. - Avoid making suggestions that have already been implemented in the PR code. For example, if you want to add logs, or change a variable to const, or anything else, make sure it isn't already in the PR code. - Don't suggest to add docstring, type hints, or comments. @@ -99,10 +99,10 @@ PR Feedback: General suggestions: type: string description: |- - General suggestions and feedback for the contributors and maintainers of - this PR. May include important suggestions for the overall structure, - primary purpose, best practices, critical bugs, and other aspects of the - PR. Don't address PR title and description, or lack of tests. Explain your suggestions. + General suggestions and feedback for the contributors and maintainers of this PR. + May include important suggestions for the overall structure, + primary purpose, best practices, critical bugs, and other aspects of the PR. + Don't address PR title and description, or lack of tests. Explain your suggestions. {%- if num_code_suggestions > 0 %} Code feedback: type: array @@ -115,11 +115,10 @@ PR Feedback: suggestion: type: string description: |- - a concrete suggestion for meaningfully improving the new PR code. Also - describe how, specifically, the suggestion can be applied to new PR - code. Add tags with importance measure that matches each suggestion - ('important' or 'medium'). Do not make suggestions for updating or - adding docstrings, renaming PR title and description, or linter like. + a concrete suggestion for meaningfully improving the new PR code. + Also describe how, specifically, the suggestion can be applied to new PR code. + Add tags with importance measure that matches each suggestion ('important' or 'medium'). + Do not make suggestions for updating or adding docstrings, renaming PR title and description, or linter like. relevant line: type: string description: |-