pr-agent/Usage.md

Usage guide

Table of Contents

• Introduction
• Working from a local repo (CLI)
• Online usage
• Working with GitHub App
• Working with GitHub Action
• Appendix - additional configurations walkthrough

Introduction

There are 3 basic ways to invoke CodiumAI PR-Agent:

1. Locally running a CLI command
2. Online usage - by commenting on a PR
3. Enabling PR-Agent tools to run automatically when a new PR is opened

See the installation guide for instructions on how to setup your own PR-Agent.

Specifically, CLI commands can be issued by invoking a pre-built docker image, or by invoking a locally cloned repo.

For online usage, you will need to setup either a GitHub App, or a GitHub Action. GitHub App and GitHub Action also enable to run PR-Agent specific tool automatically when a new PR is opened.

The configuration file

The different tools and sub-tools used by CodiumAI PR-Agent are adjustable via the configuration file. In addition to general configuration options, each tool has its own configurations. For example, the review tool will use parameters from the pr_reviewer section in the configuration file.

git provider: The git_provider field in the configuration file determines the GIT provider that will be used by PR-Agent. Currently, the following providers are supported: "github", "gitlab", "azure", "codecommit", "local"

Working from a local repo (CLI)

When running from your local repo (CLI), your local configuration file will be used.

Examples for invoking the different tools via the CLI:

• Review: python cli.py --pr_url=<pr_url> /review
• Describe: python cli.py --pr_url=<pr_url> /describe
• Improve: python cli.py --pr_url=<pr_url> /improve
• Ask: python cli.py --pr_url=<pr_url> /ask "Write me a poem about this PR"
• Reflect: python cli.py --pr_url=<pr_url> /reflect
• Update Changelog: python cli.py --pr_url=<pr_url> /update_changelog

<pr_url> is the url of the relevant PR (for example: https://github.com/Codium-ai/pr-agent/pull/50).

Notes:

(1) in addition to editing your local configuration file, you can also change any configuration value by adding it to the command line:

python cli.py --pr_url=<pr_url>  /review --pr_reviewer.extra_instructions="focus on the file: ..."

(2) You can print results locally, without publishing them, by setting in configuration.toml:

[config]
publish_output=true
verbosity_level=2

This is useful for debugging or experimenting with the different tools.

Online usage

Online usage means invoking PR-Agent tools by comments on a PR. Commands for invoking the different tools via comments:

• Review: /review
• Describe: /describe
• Improve: /improve
• Ask: /ask "..."
• Reflect: /reflect
• Update Changelog: /update_changelog

To edit a specific configuration value, just add --config_path=<value> to any command. For example if you want to edit the review tool configurations, you can run:

/review --pr_reviewer.extra_instructions="..." --pr_reviewer.require_score_review=false

Any configuration value in configuration file file can be similarly edited.

Working with GitHub App

When running PR-Agent from GitHub App, the default configurations from a pre-built repo will be initially loaded.

GitHub app automatic tools

The github_app section defines GitHub app specific configurations. An important parameter is pr_commands, which is a list of tools that will be run automatically when a new PR is opened:

[github_app]
pr_commands = [
    "/describe --pr_description.add_original_user_description=true --pr_description.keep_original_user_title=true",
    "/auto_review",
]

This means that when a new PR is opened, PR-Agent will run the describe and auto_review tools. For the describe tool, the add_original_user_description and keep_original_user_title parameters will be set to true.

However, you can override the default tool parameters by uploading a local configuration file called .pr_agent.toml to the root of your repo. For example, if your local .pr_agent.toml file contains:

[pr_description]
add_original_user_description = false
keep_original_user_title = false

When a new PR is opened, PR-Agent will run the describe tool with the above parameters.

Note that a local .pr_agent.toml file enables you to edit and customize the default parameters of any tool, not just the ones that are run automatically.

Editing the prompts

The prompts for the various PR-Agent tools are defined in the pr_agent/settings folder.

In practice, the prompts are loaded and stored as a standard setting object. Hence, editing them is similar to editing any other configuration value - just place the relevant key in .pr_agent.tomlfile, and override the default value.

For example, if you want to edit the prompts of the describe tool, you can add the following to your .pr_agent.toml file:

[pr_description_prompt]
system="""
...
"""
user="""
...
"""

Note that the new prompt will need to generate an output compatible with the relevant post-process function.

Working with GitHub Action

TBD

Appendix - additional configurations walkthrough

Changing a model

See here for the list of available models.

Azure

To use Azure, set:

api_key = "" # your azure api key
api_type = "azure"
api_version = '2023-05-15'  # Check Azure documentation for the current API version
api_base = ""  # The base URL for your Azure OpenAI resource. e.g. "https://<your resource name>.openai.azure.com"
deployment_id = ""  # The deployment name you chose when you deployed the engine

in your .secrets.toml

and

[config]
model="" # the OpenAI model you've deployed on Azure (e.g. gpt-3.5-turbo)

in the configuration.toml

Huggingface

Local
You can run Huggingface models locally through either VLLM or Ollama

E.g. to use a new Huggingface model locally via Ollama, set:

[__init__.py]
MAX_TOKENS = {
    "model-name-on-ollama": <max_tokens>
}
e.g.
MAX_TOKENS={
    ...,
    "llama2": 4096
}


[config] # in configuration.toml
model = "ollama/llama2"

[ollama] # in .secrets.toml
api_base = ... # the base url for your huggingface inference endpoint 

Inference Endpoints

To use a new model with Huggingface Inference Endpoints, for example, set:

[__init__.py]
MAX_TOKENS = {
    "model-name-on-huggingface": <max_tokens>
}
e.g.
MAX_TOKENS={
    ...,
    "meta-llama/Llama-2-7b-chat-hf": 4096
}
[config] # in configuration.toml
model = "huggingface/meta-llama/Llama-2-7b-chat-hf"

[huggingface] # in .secrets.toml
key = ... # your huggingface api key
api_base = ... # the base url for your huggingface inference endpoint 

(you can obtain a Llama2 key from here)

Replicate

To use Llama2 model with Replicate, for example, set:

[config] # in configuration.toml
model = "replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1"
[replicate] # in .secrets.toml
key = ...

(you can obtain a Llama2 key from here)

Also review the AiHandler file for instruction how to set keys for other models.

Extra instructions

All PR-Agent tools have a parameter called extra_instructions, that enables to add free-text extra instructions. Example usage:

/update_changelog --pr_update_changelog.extra_instructions="Make sure to update also the version ..."

Azure DevOps provider

To use Azure DevOps provider use the following settings in configuration.toml:

[config]
git_provider="azure"
use_repo_settings_file=false

And use the following settings (you have to replace the values) in .secrets.toml:

[azure_devops]
org = "https://dev.azure.com/YOUR_ORGANIZATION/"
pat = "YOUR_PAT_TOKEN"