xiaxinhang/pr-agent
1
0
Fork 0
mirror of https://github.com/qodo-ai/pr-agent.git synced 2025-07-10 15:50:37 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: add prompt example duplication option for improved model output

Browse Source
This commit is contained in:
mrT23
2025-01-02 12:25:42 +02:00
parent 5971a06d73
commit 5318047202
5 changed files with 204 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
docs/docs/usage-guide/changing_a_model.md
View File

@ -32,20 +32,26 @@ fallback_models=["..."]
### Ollama
**Local**
You can run Hugging Face models locally through either [VLLM](https://docs.litellm.ai/docs/providers/vllm) or [Ollama](https://docs.litellm.ai/docs/providers/ollama)
You can run models locally through either [VLLM](https://docs.litellm.ai/docs/providers/vllm) or [Ollama](https://docs.litellm.ai/docs/providers/ollama)
E.g. to use a new Hugging Face model locally via Ollama, set:
E.g. to use a new model locally via Ollama, set in `.secrets.toml` or in a configuration file:
```
[config] # in configuration.toml
model = "ollama/llama2"
fallback_models=["ollama/llama2"]
custom_model_max_tokens=... # set the maximal input tokens for the model
[config]
model = "ollama/qwen2.5-coder:32b"
fallback_models=["ollama/qwen2.5-coder:32b"]
custom_model_max_tokens=128000 # set the maximal input tokens for the model
duplicate_examples=true # will duplicate the examples in the prompt, to help the model to output structured output
[ollama] # in .secrets.toml
[ollama]
api_base = "http://localhost:11434" # or whatever port you're running Ollama on
```
!!! note "Local models vs commercial models"
Qodo Merge is compatible with almost any AI model, but analyzing complex code repositories and pull requests requires a model specifically optimized for code analysis.
Commercial models such as GPT-4, Claude Sonnet, and Gemini have demonstrated robust capabilities in generating structured output for code analysis. In contrast, most open-source models currently available (as of January 2025) face challenges with these complex tasks.
Based on our testing, local open-source models are suitable for experimentation and learning purposes, but they may not be suitable for production-level code analysis tasks.
Hence, for production workflows and real-world code analysis, we recommend using commercial models.
### Hugging Face Inference Endpoints
To use a new model with Hugging Face Inference Endpoints, for example, set: