mirror of
https://github.com/qodo-ai/pr-agent.git
synced 2025-07-10 07:40:39 +08:00
Compare commits
3 Commits
mrT23-patc
...
tr/static_
| Author | SHA1 | Date | |
|---|---|---|---|
| 685f001298 | |||
| 88c2b90860 | |||
| c84d84ace2 |
@ -1,3 +1,6 @@
|
||||
[pr_reviewer]
|
||||
enable_review_labels_effort = true
|
||||
enable_auto_approval = true
|
||||
|
||||
[config]
|
||||
model="claude-3-5-sonnet"
|
||||
|
||||
@ -1,11 +1,10 @@
|
||||
FROM python:3.12 as base
|
||||
FROM python:3.10 as base
|
||||
|
||||
WORKDIR /app
|
||||
ADD pyproject.toml .
|
||||
ADD requirements.txt .
|
||||
RUN pip install . && rm pyproject.toml requirements.txt
|
||||
ENV PYTHONPATH=/app
|
||||
ADD docs docs
|
||||
ADD pr_agent pr_agent
|
||||
ADD github_action/entrypoint.sh /
|
||||
RUN chmod +x /entrypoint.sh
|
||||
|
||||
55
README.md
55
README.md
@ -10,7 +10,7 @@
|
||||
|
||||
</picture>
|
||||
<br/>
|
||||
Qode Merge PR-Agent aims to help efficiently review and handle pull requests, by providing AI feedback and suggestions
|
||||
CodiumAI PR-Agent aims to help efficiently review and handle pull requests, by providing AI feedback and suggestions
|
||||
</div>
|
||||
|
||||
[](https://github.com/Codium-ai/pr-agent/blob/main/LICENSE)
|
||||
@ -25,9 +25,9 @@ Qode Merge PR-Agent aims to help efficiently review and handle pull requests, by
|
||||
</div>
|
||||
|
||||
### [Documentation](https://pr-agent-docs.codium.ai/)
|
||||
- See the [Installation Guide](https://qodo-merge-docs.qodo.ai/installation/) for instructions on installing Qode Merge PR-Agent on different platforms.
|
||||
- See the [Installation Guide](https://qodo-merge-docs.qodo.ai/installation/) for instructions on installing PR-Agent on different platforms.
|
||||
|
||||
- See the [Usage Guide](https://qodo-merge-docs.qodo.ai/usage-guide/) for instructions on running Qode Merge PR-Agent tools via different interfaces, such as CLI, PR Comments, or by automatically triggering them when a new PR is opened.
|
||||
- See the [Usage Guide](https://qodo-merge-docs.qodo.ai/usage-guide/) for instructions on running PR-Agent tools via different interfaces, such as CLI, PR Comments, or by automatically triggering them when a new PR is opened.
|
||||
|
||||
- See the [Tools Guide](https://qodo-merge-docs.qodo.ai/tools/) for a detailed description of the different tools, and the available configurations for each tool.
|
||||
|
||||
@ -43,43 +43,45 @@ Qode Merge PR-Agent aims to help efficiently review and handle pull requests, by
|
||||
|
||||
## News and Updates
|
||||
|
||||
### October 27, 2024
|
||||
### September 21, 2024
|
||||
Need help with PR-Agent? New feature - simply comment `/help "your question"` in a pull request, and PR-Agent will provide you with the [relevant documentation](https://github.com/Codium-ai/pr-agent/pull/1241#issuecomment-2365259334).
|
||||
|
||||
Qodo Merge PR Agent will now automatically document accepted code suggestions in a dedicated wiki page (`.pr_agent_accepted_suggestions`), enabling users to track historical changes, assess the tool's effectiveness, and learn from previously implemented recommendations in the repository.
|
||||
|
||||
This dedicated wiki page will also serve as a foundation for future AI model improvements, allowing it to learn from historically implemented suggestions and generate more targeted, contextually relevant recommendations.
|
||||
Read more about this novel feature [here](https://qodo-merge-docs.qodo.ai/tools/improve/#suggestion-tracking).
|
||||
|
||||
<kbd><img href="https://qodo.ai/images/pr_agent/pr_agent_accepted_suggestions1.png" src="https://qodo.ai/images/pr_agent/pr_agent_accepted_suggestions1.png" width="768"></kbd>
|
||||
<kbd><img src="https://www.codium.ai/images/pr_agent/pr_help_chat.png" width="768"></kbd>
|
||||
|
||||
|
||||
### September 12, 2024
|
||||
[Dynamic context](https://pr-agent-docs.codium.ai/core-abilities/dynamic_context/) is now the default option for context extension.
|
||||
This feature enables PR-Agent to dynamically adjusting the relevant context for each code hunk, while avoiding overflowing the model with too much information.
|
||||
|
||||
### October 21, 2024
|
||||
**Disable publishing labels by default:**
|
||||
### September 3, 2024
|
||||
|
||||
The default setting for `pr_description.publish_labels` has been updated to `false`. This means that labels generated by the `/describe` tool will no longer be published, unless this configuration is explicitly set to `true`.
|
||||
New version of PR-Agent, v0.24 was released. See the [release notes](https://github.com/Codium-ai/pr-agent/releases/tag/v0.24) for more information.
|
||||
|
||||
We constantly strive to balance informative AI analysis with reducing unnecessary noise. User feedback indicated that in many cases, the original PR title alone provides sufficient information, making the generated labels (`enhancement`, `documentation`, `bug fix`, ...) redundant.
|
||||
The [`review_effort`](https://qodo-merge-docs.qodo.ai/tools/review/#configuration-options) label, generated by the `review` tool, will still be published by default, as it provides valuable information enabling reviewers to prioritize small PRs first.
|
||||
### August 26, 2024
|
||||
|
||||
However, every user has different preferences. To still publish the `describe` labels, set `pr_description.publish_labels=true` in the [configuration file](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/).
|
||||
For more tailored and relevant labeling, we recommend using the [`custom_labels 💎`](https://qodo-merge-docs.qodo.ai/tools/custom_labels/) tool, that allows generating labels specific to your project's needs.
|
||||
New version of [PR Agent Chrome Extension](https://chromewebstore.google.com/detail/pr-agent-chrome-extension/ephlnjeghhogofkifjloamocljapahnl) was released, with full support of context-aware **PR Chat**. This novel feature is free to use for any open-source repository. See more details in [here](https://pr-agent-docs.codium.ai/chrome-extension/#pr-chat).
|
||||
|
||||
<kbd></kbd>
|
||||
<kbd><img src="https://www.codium.ai/images/pr_agent/pr_chat_1.png" width="768"></kbd>
|
||||
|
||||
→
|
||||
|
||||
<kbd></kbd>
|
||||
<kbd><img src="https://www.codium.ai/images/pr_agent/pr_chat_2.png" width="768"></kbd>
|
||||
|
||||
|
||||
### August 11, 2024
|
||||
Increased PR context size for improved results, and enabled [asymmetric context](https://github.com/Codium-ai/pr-agent/pull/1114/files#diff-9290a3ad9a86690b31f0450b77acd37ef1914b41fabc8a08682d4da433a77f90R69-R70)
|
||||
|
||||
### October 14, 2024
|
||||
Improved support for GitHub enterprise server with [GitHub Actions](https://qodo-merge-docs.qodo.ai/installation/github/#action-for-github-enterprise-server)
|
||||
### August 10, 2024
|
||||
Added support for [Azure devops pipeline](https://pr-agent-docs.codium.ai/installation/azure/) - you can now easily run PR-Agent as an Azure devops pipeline, without needing to set up your own server.
|
||||
|
||||
### October 10, 2024
|
||||
New ability for the `review` tool - **ticket compliance feedback**. If the PR contains a ticket number, PR-Agent will check if the PR code actually [complies](https://github.com/Codium-ai/pr-agent/pull/1279#issuecomment-2404042130) with the ticket requirements.
|
||||
|
||||
<kbd><img src="https://github.com/user-attachments/assets/4a2a728b-5f47-40fa-80cc-16efd296938c" width="768"></kbd>
|
||||
### August 5, 2024
|
||||
Added support for [GitLab pipeline](https://pr-agent-docs.codium.ai/installation/gitlab/#run-as-a-gitlab-pipeline) - you can now run easily PR-Agent as a GitLab pipeline, without needing to set up your own server.
|
||||
|
||||
### July 28, 2024
|
||||
|
||||
(1) improved support for bitbucket server - [auto commands](https://github.com/Codium-ai/pr-agent/pull/1059) and [direct links](https://github.com/Codium-ai/pr-agent/pull/1061)
|
||||
|
||||
(2) custom models are now [supported](https://pr-agent-docs.codium.ai/usage-guide/changing_a_model/#custom-models)
|
||||
|
||||
|
||||
|
||||
## Overview
|
||||
@ -91,6 +93,7 @@ Supported commands per platform:
|
||||
|-------|---------------------------------------------------------------------------------------------------------|:--------------------:|:--------------------:|:--------------------:|:------------:|
|
||||
| TOOLS | Review | ✅ | ✅ | ✅ | ✅ |
|
||||
| | ⮑ Incremental | ✅ | | | |
|
||||
| | ⮑ [SOC2 Compliance](https://pr-agent-docs.codium.ai/tools/review/#soc2-ticket-compliance) 💎 | ✅ | ✅ | ✅ | |
|
||||
| | Describe | ✅ | ✅ | ✅ | ✅ |
|
||||
| | ⮑ [Inline File Summary](https://pr-agent-docs.codium.ai/tools/describe#inline-file-summary) 💎 | ✅ | | | |
|
||||
| | Improve | ✅ | ✅ | ✅ | ✅ |
|
||||
|
||||
@ -1,9 +1,9 @@
|
||||
FROM python:3.12.3 AS base
|
||||
|
||||
WORKDIR /app
|
||||
ADD docs/chroma_db.zip /app/docs/chroma_db.zip
|
||||
ADD pyproject.toml .
|
||||
ADD requirements.txt .
|
||||
ADD docs docs
|
||||
RUN pip install . && rm pyproject.toml requirements.txt
|
||||
ENV PYTHONPATH=/app
|
||||
|
||||
|
||||
BIN
docs/chroma_db.zip
Normal file
BIN
docs/chroma_db.zip
Normal file
Binary file not shown.
Binary file not shown.
|
Before Width: | Height: | Size: 4.2 KiB After Width: | Height: | Size: 15 KiB |
@ -1 +1,140 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?><svg id="Layer_1" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 109.77 81.94"><defs><style>.cls-1{fill:#7968fa;}.cls-1,.cls-2{stroke-width:0px;}.cls-2{fill:#5ae3ae;}</style></defs><path class="cls-2" d="m109.77,40.98c0,22.62-7.11,40.96-15.89,40.96-3.6,0-6.89-3.09-9.58-8.31,6.82-7.46,11.22-19.3,11.22-32.64s-4.4-25.21-11.22-32.67C86.99,3.09,90.29,0,93.89,0c8.78,0,15.89,18.33,15.89,40.97"/><path class="cls-1" d="m95.53,40.99c0,13.35-4.4,25.19-11.23,32.64-3.81-7.46-6.28-19.3-6.28-32.64s2.47-25.21,6.28-32.67c6.83,7.46,11.23,19.32,11.23,32.67"/><path class="cls-2" d="m55.38,78.15c-4.99,2.42-10.52,3.79-16.38,3.79C17.46,81.93,0,63.6,0,40.98S17.46,0,39,0C44.86,0,50.39,1.37,55.38,3.79c-9.69,6.47-16.43,20.69-16.43,37.19s6.73,30.7,16.43,37.17"/><path class="cls-1" d="m78.02,40.99c0,16.48-9.27,30.7-22.65,37.17-9.69-6.47-16.43-20.69-16.43-37.17S45.68,10.28,55.38,3.81c13.37,6.49,22.65,20.69,22.65,37.19"/><path class="cls-2" d="m84.31,73.63c-4.73,5.22-10.64,8.31-17.06,8.31-4.24,0-8.27-1.35-11.87-3.79,13.37-6.48,22.65-20.7,22.65-37.17,0,13.35,2.47,25.19,6.28,32.64"/><path class="cls-2" d="m84.31,8.31c-3.81,7.46-6.28,19.32-6.28,32.67,0-16.5-9.27-30.7-22.65-37.19,3.6-2.45,7.63-3.8,11.87-3.8,6.43,0,12.33,3.09,17.06,8.31"/></svg>
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- Generator: Adobe Illustrator 28.1.0, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
|
||||
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
|
||||
width="64px" height="64px" viewBox="0 0 64 64" enable-background="new 0 0 64 64" xml:space="preserve">
|
||||
<g>
|
||||
<defs>
|
||||
<rect id="SVGID_1_" x="0.4" y="0.1" width="63.4" height="63.4"/>
|
||||
</defs>
|
||||
<clipPath id="SVGID_00000008836131916906499950000015813697852011234749_">
|
||||
<use xlink:href="#SVGID_1_" overflow="visible"/>
|
||||
</clipPath>
|
||||
<g clip-path="url(#SVGID_00000008836131916906499950000015813697852011234749_)">
|
||||
<path fill="#05E5AD" d="M21.4,9.8c3,0,5.9,0.7,8.5,1.9c-5.7,3.4-9.8,11.1-9.8,20.1c0,9,4,16.7,9.8,20.1c-2.6,1.2-5.5,1.9-8.5,1.9
|
||||
c-11.6,0-21-9.8-21-22S9.8,9.8,21.4,9.8z"/>
|
||||
|
||||
<radialGradient id="SVGID_00000150822754378345238340000008985053211526864828_" cx="-140.0905" cy="350.1757" r="4.8781" gradientTransform="matrix(-4.7708 -6.961580e-02 -0.1061 7.2704 -601.3099 -2523.8489)" gradientUnits="userSpaceOnUse">
|
||||
<stop offset="0" style="stop-color:#6447FF"/>
|
||||
<stop offset="6.666670e-02" style="stop-color:#6348FE"/>
|
||||
<stop offset="0.1333" style="stop-color:#614DFC"/>
|
||||
<stop offset="0.2" style="stop-color:#5C54F8"/>
|
||||
<stop offset="0.2667" style="stop-color:#565EF3"/>
|
||||
<stop offset="0.3333" style="stop-color:#4E6CEC"/>
|
||||
<stop offset="0.4" style="stop-color:#447BE4"/>
|
||||
<stop offset="0.4667" style="stop-color:#3A8DDB"/>
|
||||
<stop offset="0.5333" style="stop-color:#2F9FD1"/>
|
||||
<stop offset="0.6" style="stop-color:#25B1C8"/>
|
||||
<stop offset="0.6667" style="stop-color:#1BC0C0"/>
|
||||
<stop offset="0.7333" style="stop-color:#13CEB9"/>
|
||||
<stop offset="0.8" style="stop-color:#0DD8B4"/>
|
||||
<stop offset="0.8667" style="stop-color:#08DFB0"/>
|
||||
<stop offset="0.9333" style="stop-color:#06E4AE"/>
|
||||
<stop offset="1" style="stop-color:#05E5AD"/>
|
||||
</radialGradient>
|
||||
<path fill="url(#SVGID_00000150822754378345238340000008985053211526864828_)" d="M21.4,9.8c3,0,5.9,0.7,8.5,1.9
|
||||
c-5.7,3.4-9.8,11.1-9.8,20.1c0,9,4,16.7,9.8,20.1c-2.6,1.2-5.5,1.9-8.5,1.9c-11.6,0-21-9.8-21-22S9.8,9.8,21.4,9.8z"/>
|
||||
|
||||
<radialGradient id="SVGID_00000022560571240417802950000012439139323268113305_" cx="-191.7649" cy="385.7387" r="4.8781" gradientTransform="matrix(-2.5514 -0.7616 -0.8125 2.7217 -130.733 -1180.2209)" gradientUnits="userSpaceOnUse">
|
||||
<stop offset="0" style="stop-color:#6447FF"/>
|
||||
<stop offset="6.666670e-02" style="stop-color:#6348FE"/>
|
||||
<stop offset="0.1333" style="stop-color:#614DFC"/>
|
||||
<stop offset="0.2" style="stop-color:#5C54F8"/>
|
||||
<stop offset="0.2667" style="stop-color:#565EF3"/>
|
||||
<stop offset="0.3333" style="stop-color:#4E6CEC"/>
|
||||
<stop offset="0.4" style="stop-color:#447BE4"/>
|
||||
<stop offset="0.4667" style="stop-color:#3A8DDB"/>
|
||||
<stop offset="0.5333" style="stop-color:#2F9FD1"/>
|
||||
<stop offset="0.6" style="stop-color:#25B1C8"/>
|
||||
<stop offset="0.6667" style="stop-color:#1BC0C0"/>
|
||||
<stop offset="0.7333" style="stop-color:#13CEB9"/>
|
||||
<stop offset="0.8" style="stop-color:#0DD8B4"/>
|
||||
<stop offset="0.8667" style="stop-color:#08DFB0"/>
|
||||
<stop offset="0.9333" style="stop-color:#06E4AE"/>
|
||||
<stop offset="1" style="stop-color:#05E5AD"/>
|
||||
</radialGradient>
|
||||
<path fill="url(#SVGID_00000022560571240417802950000012439139323268113305_)" d="M38,18.3c-2.1-2.8-4.9-5.1-8.1-6.6
|
||||
c2-1.2,4.2-1.9,6.6-1.9c2.2,0,4.3,0.6,6.2,1.7C40.8,12.9,39.2,15.3,38,18.3L38,18.3z"/>
|
||||
|
||||
<radialGradient id="SVGID_00000143611122169386473660000017673587931016751800_" cx="-194.7918" cy="395.2442" r="4.8781" gradientTransform="matrix(-2.5514 -0.7616 -0.8125 2.7217 -130.733 -1172.9556)" gradientUnits="userSpaceOnUse">
|
||||
<stop offset="0" style="stop-color:#6447FF"/>
|
||||
<stop offset="6.666670e-02" style="stop-color:#6348FE"/>
|
||||
<stop offset="0.1333" style="stop-color:#614DFC"/>
|
||||
<stop offset="0.2" style="stop-color:#5C54F8"/>
|
||||
<stop offset="0.2667" style="stop-color:#565EF3"/>
|
||||
<stop offset="0.3333" style="stop-color:#4E6CEC"/>
|
||||
<stop offset="0.4" style="stop-color:#447BE4"/>
|
||||
<stop offset="0.4667" style="stop-color:#3A8DDB"/>
|
||||
<stop offset="0.5333" style="stop-color:#2F9FD1"/>
|
||||
<stop offset="0.6" style="stop-color:#25B1C8"/>
|
||||
<stop offset="0.6667" style="stop-color:#1BC0C0"/>
|
||||
<stop offset="0.7333" style="stop-color:#13CEB9"/>
|
||||
<stop offset="0.8" style="stop-color:#0DD8B4"/>
|
||||
<stop offset="0.8667" style="stop-color:#08DFB0"/>
|
||||
<stop offset="0.9333" style="stop-color:#06E4AE"/>
|
||||
<stop offset="1" style="stop-color:#05E5AD"/>
|
||||
</radialGradient>
|
||||
<path fill="url(#SVGID_00000143611122169386473660000017673587931016751800_)" d="M38,45.2c1.2,3,2.9,5.3,4.7,6.8
|
||||
c-1.9,1.1-4,1.7-6.2,1.7c-2.3,0-4.6-0.7-6.6-1.9C33.1,50.4,35.8,48.1,38,45.2L38,45.2z"/>
|
||||
<path fill="#684BFE" d="M20.1,31.8c0-9,4-16.7,9.8-20.1c3.2,1.5,6,3.8,8.1,6.6c-1.5,3.7-2.5,8.4-2.5,13.5s0.9,9.8,2.5,13.5
|
||||
c-2.1,2.8-4.9,5.1-8.1,6.6C24.1,48.4,20.1,40.7,20.1,31.8z"/>
|
||||
|
||||
<radialGradient id="SVGID_00000147942998054305738810000004710078864578628519_" cx="-212.7358" cy="363.2475" r="4.8781" gradientTransform="matrix(-2.3342 -1.063 -1.623 3.5638 149.3813 -1470.1027)" gradientUnits="userSpaceOnUse">
|
||||
<stop offset="0" style="stop-color:#6447FF"/>
|
||||
<stop offset="6.666670e-02" style="stop-color:#6348FE"/>
|
||||
<stop offset="0.1333" style="stop-color:#614DFC"/>
|
||||
<stop offset="0.2" style="stop-color:#5C54F8"/>
|
||||
<stop offset="0.2667" style="stop-color:#565EF3"/>
|
||||
<stop offset="0.3333" style="stop-color:#4E6CEC"/>
|
||||
<stop offset="0.4" style="stop-color:#447BE4"/>
|
||||
<stop offset="0.4667" style="stop-color:#3A8DDB"/>
|
||||
<stop offset="0.5333" style="stop-color:#2F9FD1"/>
|
||||
<stop offset="0.6" style="stop-color:#25B1C8"/>
|
||||
<stop offset="0.6667" style="stop-color:#1BC0C0"/>
|
||||
<stop offset="0.7333" style="stop-color:#13CEB9"/>
|
||||
<stop offset="0.8" style="stop-color:#0DD8B4"/>
|
||||
<stop offset="0.8667" style="stop-color:#08DFB0"/>
|
||||
<stop offset="0.9333" style="stop-color:#06E4AE"/>
|
||||
<stop offset="1" style="stop-color:#05E5AD"/>
|
||||
</radialGradient>
|
||||
<path fill="url(#SVGID_00000147942998054305738810000004710078864578628519_)" d="M50.7,42.5c0.6,3.3,1.5,6.1,2.5,8
|
||||
c-1.8,2-3.8,3.1-6,3.1c-1.6,0-3.1-0.6-4.5-1.7C46.1,50.2,48.9,46.8,50.7,42.5L50.7,42.5z"/>
|
||||
|
||||
<radialGradient id="SVGID_00000083770737908230256670000016126156495859285174_" cx="-208.5327" cy="357.2025" r="4.8781" gradientTransform="matrix(-2.3342 -1.063 -1.623 3.5638 149.3813 -1476.8097)" gradientUnits="userSpaceOnUse">
|
||||
<stop offset="0" style="stop-color:#6447FF"/>
|
||||
<stop offset="6.666670e-02" style="stop-color:#6348FE"/>
|
||||
<stop offset="0.1333" style="stop-color:#614DFC"/>
|
||||
<stop offset="0.2" style="stop-color:#5C54F8"/>
|
||||
<stop offset="0.2667" style="stop-color:#565EF3"/>
|
||||
<stop offset="0.3333" style="stop-color:#4E6CEC"/>
|
||||
<stop offset="0.4" style="stop-color:#447BE4"/>
|
||||
<stop offset="0.4667" style="stop-color:#3A8DDB"/>
|
||||
<stop offset="0.5333" style="stop-color:#2F9FD1"/>
|
||||
<stop offset="0.6" style="stop-color:#25B1C8"/>
|
||||
<stop offset="0.6667" style="stop-color:#1BC0C0"/>
|
||||
<stop offset="0.7333" style="stop-color:#13CEB9"/>
|
||||
<stop offset="0.8" style="stop-color:#0DD8B4"/>
|
||||
<stop offset="0.8667" style="stop-color:#08DFB0"/>
|
||||
<stop offset="0.9333" style="stop-color:#06E4AE"/>
|
||||
<stop offset="1" style="stop-color:#05E5AD"/>
|
||||
</radialGradient>
|
||||
<path fill="url(#SVGID_00000083770737908230256670000016126156495859285174_)" d="M42.7,11.5c1.4-1.1,2.9-1.7,4.5-1.7
|
||||
c2.2,0,4.3,1.1,6,3.1c-1,2-1.9,4.7-2.5,8C48.9,16.7,46.1,13.4,42.7,11.5L42.7,11.5z"/>
|
||||
<path fill="#684BFE" d="M38,45.2c2.8-3.7,4.4-8.4,4.4-13.5c0-5.1-1.7-9.8-4.4-13.5c1.2-3,2.9-5.3,4.7-6.8c3.4,1.9,6.2,5.3,8,9.5
|
||||
c-0.6,3.2-0.9,6.9-0.9,10.8s0.3,7.6,0.9,10.8c-1.8,4.3-4.6,7.6-8,9.5C40.8,50.6,39.2,48.2,38,45.2L38,45.2z"/>
|
||||
<path fill="#321BB2" d="M38,45.2c-1.5-3.7-2.5-8.4-2.5-13.5S36.4,22,38,18.3c2.8,3.7,4.4,8.4,4.4,13.5S40.8,41.5,38,45.2z"/>
|
||||
<path fill="#05E6AD" d="M53.2,12.9c1.1-2,2.3-3.1,3.6-3.1c3.9,0,7,9.8,7,22s-3.1,22-7,22c-1.3,0-2.6-1.1-3.6-3.1
|
||||
c3.4-3.8,5.7-10.8,5.7-18.8C58.8,23.8,56.6,16.8,53.2,12.9z"/>
|
||||
|
||||
<radialGradient id="SVGID_00000009565123575973598080000009335550354766300606_" cx="-7.8671" cy="278.2442" r="4.8781" gradientTransform="matrix(1.5187 0 0 -7.8271 69.237 2209.3281)" gradientUnits="userSpaceOnUse">
|
||||
<stop offset="0" style="stop-color:#05E5AD"/>
|
||||
<stop offset="0.32" style="stop-color:#05E5AD;stop-opacity:0"/>
|
||||
<stop offset="0.9028" style="stop-color:#6447FF"/>
|
||||
</radialGradient>
|
||||
<path fill="url(#SVGID_00000009565123575973598080000009335550354766300606_)" d="M53.2,12.9c1.1-2,2.3-3.1,3.6-3.1
|
||||
c3.9,0,7,9.8,7,22s-3.1,22-7,22c-1.3,0-2.6-1.1-3.6-3.1c3.4-3.8,5.7-10.8,5.7-18.8C58.8,23.8,56.6,16.8,53.2,12.9z"/>
|
||||
<path fill="#684BFE" d="M52.8,31.8c0-3.9-0.8-7.6-2.1-10.8c0.6-3.3,1.5-6.1,2.5-8c3.4,3.8,5.7,10.8,5.7,18.8c0,8-2.3,15-5.7,18.8
|
||||
c-1-2-1.9-4.7-2.5-8C52,39.3,52.8,35.7,52.8,31.8z"/>
|
||||
<path fill="#321BB2" d="M50.7,42.5c-0.6-3.2-0.9-6.9-0.9-10.8s0.3-7.6,0.9-10.8c1.3,3.2,2.1,6.9,2.1,10.8S52,39.3,50.7,42.5z"/>
|
||||
</g>
|
||||
</g>
|
||||
</svg>
|
||||
|
||||
|
Before Width: | Height: | Size: 1.2 KiB After Width: | Height: | Size: 9.0 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 8.7 KiB |
@ -9,20 +9,4 @@ Qodo Merge utilizes a variety of core abilities to provide a comprehensive and e
|
||||
- [Compression strategy](https://qodo-merge-docs.qodo.ai/core-abilities/compression_strategy/)
|
||||
- [Code-oriented YAML](https://qodo-merge-docs.qodo.ai/core-abilities/code_oriented_yaml/)
|
||||
- [Static code analysis](https://qodo-merge-docs.qodo.ai/core-abilities/static_code_analysis/)
|
||||
- [Code fine-tuning benchmark](https://qodo-merge-docs.qodo.ai/finetuning_benchmark/)
|
||||
|
||||
## Blogs
|
||||
|
||||
Here are some additional technical blogs from Qodo, that delve deeper into the core capabilities and features of Large Language Models (LLMs) when applied to coding tasks.
|
||||
These resources provide more comprehensive insights into leveraging LLMs for software development.
|
||||
|
||||
### Code Generation and LLMs
|
||||
- [State-of-the-art Code Generation with AlphaCodium – From Prompt Engineering to Flow Engineering](https://www.qodo.ai/blog/qodoflow-state-of-the-art-code-generation-for-code-contests/)
|
||||
- [RAG for a Codebase with 10k Repos](https://www.qodo.ai/blog/rag-for-large-scale-code-repos/)
|
||||
|
||||
### Development Processes
|
||||
- [Understanding the Challenges and Pain Points of the Pull Request Cycle](https://www.qodo.ai/blog/understanding-the-challenges-and-pain-points-of-the-pull-request-cycle/)
|
||||
- [Introduction to Code Coverage Testing](https://www.qodo.ai/blog/introduction-to-code-coverage-testing/)
|
||||
|
||||
### Cost Optimization
|
||||
- [Reduce Your Costs by 30% When Using GPT for Python Code](https://www.qodo.ai/blog/reduce-your-costs-by-30-when-using-gpt-3-for-python-code/)
|
||||
- [Code fine-tuning benchmark](https://qodo-merge-docs.qodo.ai/finetuning_benchmark/)
|
||||
@ -49,7 +49,7 @@ __old hunk__
|
||||
...
|
||||
```
|
||||
|
||||
(3) The entire PR files that were retrieved are also used to expand and enhance the PR context (see [Dynamic Context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic_context/)).
|
||||
(3) The entire PR files that were retrieved are also used to expand and enhance the PR context (see [Dynamic Context](https://qodo-merge-docs.qodo.ai/core-abilities/dynamic-context/)).
|
||||
|
||||
|
||||
(4) All the metadata described above represents several level of cumulative analysis - ranging from hunk level, to file level, to PR level, to organization level.
|
||||
|
||||
@ -29,6 +29,7 @@ Qodo Merge offers extensive pull request functionalities across various git prov
|
||||
|-------|-----------------------------------------------------------------------------------------------------------------------|:------:|:------:|:---------:|:------------:|
|
||||
| TOOLS | Review | ✅ | ✅ | ✅ | ✅ |
|
||||
| | ⮑ Incremental | ✅ | | | |
|
||||
| | ⮑ [SOC2 Compliance](https://qodo-merge-docs.qodo.ai/tools/review/#soc2-ticket-compliance){:target="_blank"} 💎 | ✅ | ✅ | ✅ | |
|
||||
| | Ask | ✅ | ✅ | ✅ | ✅ |
|
||||
| | Describe | ✅ | ✅ | ✅ | ✅ |
|
||||
| | ⮑ [Inline file summary](https://qodo-merge-docs.qodo.ai/tools/describe/#inline-file-summary){:target="_blank"} 💎 | ✅ | ✅ | | |
|
||||
|
||||
@ -27,6 +27,27 @@ jobs:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
```
|
||||
|
||||
|
||||
if you want to pin your action to a specific release (v0.23 for example) for stability reasons, use:
|
||||
```yaml
|
||||
...
|
||||
steps:
|
||||
- name: PR Agent action step
|
||||
id: pragent
|
||||
uses: docker://codiumai/pr-agent:0.23-github_action
|
||||
...
|
||||
```
|
||||
|
||||
For enhanced security, you can also specify the Docker image by its [digest](https://hub.docker.com/repository/docker/codiumai/pr-agent/tags):
|
||||
```yaml
|
||||
...
|
||||
steps:
|
||||
- name: PR Agent action step
|
||||
id: pragent
|
||||
uses: docker://codiumai/pr-agent@sha256:14165e525678ace7d9b51cda8652c2d74abb4e1d76b57c4a6ccaeba84663cc64
|
||||
...
|
||||
```
|
||||
|
||||
2) Add the following secret to your repository under `Settings > Secrets and variables > Actions > New repository secret > Add secret`:
|
||||
|
||||
```
|
||||
@ -49,40 +70,6 @@ When you open your next PR, you should see a comment from `github-actions` bot w
|
||||
```
|
||||
See detailed usage instructions in the [USAGE GUIDE](https://qodo-merge-docs.qodo.ai/usage-guide/automations_and_usage/#github-action)
|
||||
|
||||
### Using a specific release
|
||||
!!! tip ""
|
||||
if you want to pin your action to a specific release (v0.23 for example) for stability reasons, use:
|
||||
```yaml
|
||||
...
|
||||
steps:
|
||||
- name: PR Agent action step
|
||||
id: pragent
|
||||
uses: docker://codiumai/pr-agent:0.23-github_action
|
||||
...
|
||||
```
|
||||
|
||||
For enhanced security, you can also specify the Docker image by its [digest](https://hub.docker.com/repository/docker/codiumai/pr-agent/tags):
|
||||
```yaml
|
||||
...
|
||||
steps:
|
||||
- name: PR Agent action step
|
||||
id: pragent
|
||||
uses: docker://codiumai/pr-agent@sha256:14165e525678ace7d9b51cda8652c2d74abb4e1d76b57c4a6ccaeba84663cc64
|
||||
...
|
||||
```
|
||||
|
||||
### Action for GitHub enterprise server
|
||||
!!! tip ""
|
||||
To use the action with a GitHub enterprise server, add an environment variable `GITHUB.BASE_URL` with the API URL of your GitHub server.
|
||||
|
||||
For example, if your GitHub server is at `https://github.mycompany.com`, add the following to your workflow file:
|
||||
```yaml
|
||||
env:
|
||||
# ... previous environment values
|
||||
GITHUB.BASE_URL: "https://github.mycompany.com/api/v3"
|
||||
```
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Run as a GitHub App
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
# Installation
|
||||
|
||||
## Self-hosted Qodo Merge
|
||||
If you choose to host your own Qodo Merge, you first need to acquire two tokens:
|
||||
If you choose to host you own Qodo Merge, you first need to acquire two tokens:
|
||||
|
||||
1. An OpenAI key from [here](https://platform.openai.com/api-keys), with access to GPT-4 (or a key for other [language models](https://qodo-merge-docs.qodo.ai/usage-guide/changing_a_model/), if you prefer).
|
||||
2. A GitHub\GitLab\BitBucket personal access token (classic), with the repo scope. [GitHub from [here](https://github.com/settings/tokens)]
|
||||
@ -18,4 +18,4 @@ There are several ways to use self-hosted Qodo Merge:
|
||||
Qodo Merge Pro, an app hosted by CodiumAI for GitHub\GitLab\BitBucket, is also available.
|
||||
<br>
|
||||
With Qodo Merge Pro, installation is as simple as signing up and adding the Qodo Merge app to your relevant repo.
|
||||
See [here](https://qodo-merge-docs.qodo.ai/installation/pr_agent_pro/) for more details.
|
||||
See [here](https://qodo-merge-docs.qodo.ai/installation/pr_agent_pro/) for more details.
|
||||
@ -16,8 +16,8 @@ from pr_agent.config_loader import get_settings
|
||||
|
||||
def main():
|
||||
# Fill in the following values
|
||||
provider = "github" # github/gitlab/bitbucket/azure_devops
|
||||
user_token = "..." # user token
|
||||
provider = "github" # GitHub provider
|
||||
user_token = "..." # GitHub user token
|
||||
openai_key = "..." # OpenAI key
|
||||
pr_url = "..." # PR URL, for example 'https://github.com/Codium-ai/pr-agent/pull/809'
|
||||
command = "/review" # Command to run (e.g. '/review', '/describe', '/ask="What is the purpose of this PR?"', ...)
|
||||
@ -42,34 +42,42 @@ A list of the relevant tools can be found in the [tools guide](../tools/ask.md).
|
||||
To invoke a tool (for example `review`), you can run directly from the Docker image. Here's how:
|
||||
|
||||
- For GitHub:
|
||||
```
|
||||
docker run --rm -it -e OPENAI.KEY=<your key> -e GITHUB.USER_TOKEN=<your token> codiumai/pr-agent:latest --pr_url <pr_url> review
|
||||
```
|
||||
If you are using GitHub enterprise server, you need to specify the custom url as variable.
|
||||
For example, if your GitHub server is at `https://github.mycompany.com`, add the following to the command:
|
||||
```
|
||||
-e GITHUB.BASE_URL=https://github.mycompany.com/api/v3
|
||||
```
|
||||
```
|
||||
docker run --rm -it -e OPENAI.KEY=<your key> -e GITHUB.USER_TOKEN=<your token> codiumai/pr-agent:latest --pr_url <pr_url> review
|
||||
```
|
||||
|
||||
- For GitLab:
|
||||
```
|
||||
docker run --rm -it -e OPENAI.KEY=<your key> -e CONFIG.GIT_PROVIDER=gitlab -e GITLAB.PERSONAL_ACCESS_TOKEN=<your token> codiumai/pr-agent:latest --pr_url <pr_url> review
|
||||
```
|
||||
```
|
||||
docker run --rm -it -e OPENAI.KEY=<your key> -e CONFIG.GIT_PROVIDER=gitlab -e GITLAB.PERSONAL_ACCESS_TOKEN=<your token> codiumai/pr-agent:latest --pr_url <pr_url> review
|
||||
```
|
||||
|
||||
If you have a dedicated GitLab instance, you need to specify the custom url as variable:
|
||||
```
|
||||
-e GITLAB.URL=<your gitlab instance url>
|
||||
```
|
||||
Note: If you have a dedicated GitLab instance, you need to specify the custom url as variable:
|
||||
```
|
||||
docker run --rm -it -e OPENAI.KEY=<your key> -e CONFIG.GIT_PROVIDER=gitlab -e GITLAB.PERSONAL_ACCESS_TOKEN=<your token> -e GITLAB.URL=<your gitlab instance url> codiumai/pr-agent:latest --pr_url <pr_url> review
|
||||
```
|
||||
|
||||
- For BitBucket:
|
||||
```
|
||||
docker run --rm -it -e CONFIG.GIT_PROVIDER=bitbucket -e OPENAI.KEY=$OPENAI_API_KEY -e BITBUCKET.BEARER_TOKEN=$BITBUCKET_BEARER_TOKEN codiumai/pr-agent:latest --pr_url=<pr_url> review
|
||||
```
|
||||
```
|
||||
docker run --rm -it -e CONFIG.GIT_PROVIDER=bitbucket -e OPENAI.KEY=$OPENAI_API_KEY -e BITBUCKET.BEARER_TOKEN=$BITBUCKET_BEARER_TOKEN codiumai/pr-agent:latest --pr_url=<pr_url> review
|
||||
```
|
||||
|
||||
For other git providers, update CONFIG.GIT_PROVIDER accordingly, and check the `pr_agent/settings/.secrets_template.toml` file for the environment variables expected names and values.
|
||||
|
||||
---
|
||||
|
||||
|
||||
If you want to ensure you're running a specific version of the Docker image, consider using the image's digest:
|
||||
```bash
|
||||
docker run --rm -it -e OPENAI.KEY=<your key> -e GITHUB.USER_TOKEN=<your token> codiumai/pr-agent@sha256:71b5ee15df59c745d352d84752d01561ba64b6d51327f97d46152f0c58a5f678 --pr_url <pr_url> review
|
||||
```
|
||||
|
||||
Or you can run a [specific released versions](https://github.com/Codium-ai/pr-agent/blob/main/RELEASE_NOTES.md) of pr-agent, for example:
|
||||
```
|
||||
codiumai/pr-agent@v0.9
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Run from source
|
||||
|
||||
1. Clone this repository:
|
||||
@ -107,7 +115,7 @@ python3 -m pr_agent.cli --issue_url <issue_url> similar_issue
|
||||
...
|
||||
```
|
||||
|
||||
[Optional] Add the pr_agent folder to your PYTHONPATH
|
||||
[Optional] Add the pr_agent folder to your PYTHONPATH
|
||||
```
|
||||
export PYTHONPATH=$PYTHONPATH:<PATH to pr_agent folder>
|
||||
```
|
||||
@ -17,8 +17,8 @@ Users without a purchased seat who interact with a repository featuring Qodo Mer
|
||||
Beyond this limit, Qodo Merge Pro will cease to respond to their inquiries unless a seat is purchased.
|
||||
|
||||
## Install Qodo Merge Pro for GitHub Enterprise Server
|
||||
|
||||
To use Qodo Merge Pro application on your private GitHub Enterprise Server, you will need to contact us for starting an [Enterprise](https://www.codium.ai/pricing/) trial.
|
||||
You can install Qodo Merge Pro application on your GitHub Enterprise Server, and enjoy two weeks of free trial.
|
||||
After the trial period, to continue using Qodo Merge Pro, you will need to contact us for an [Enterprise license](https://www.codium.ai/pricing/).
|
||||
|
||||
|
||||
## Install Qodo Merge Pro for GitLab (Teams & Enterprise)
|
||||
|
||||
@ -29,6 +29,7 @@ Qodo Merge offers extensive pull request functionalities across various git prov
|
||||
|-------|-----------------------------------------------------------------------------------------------------------------------|:------:|:------:|:---------:|:------------:|
|
||||
| TOOLS | Review | ✅ | ✅ | ✅ | ✅ |
|
||||
| | ⮑ Incremental | ✅ | | | |
|
||||
| | ⮑ [SOC2 Compliance](https://qodo-merge-docs.qodo.ai/tools/review/#soc2-ticket-compliance){:target="_blank"} 💎 | ✅ | ✅ | ✅ | ✅ |
|
||||
| | Ask | ✅ | ✅ | ✅ | ✅ |
|
||||
| | Describe | ✅ | ✅ | ✅ | ✅ |
|
||||
| | ⮑ [Inline file summary](https://qodo-merge-docs.qodo.ai/tools/describe/#inline-file-summary){:target="_blank"} 💎 | ✅ | ✅ | | ✅ |
|
||||
|
||||
@ -20,13 +20,14 @@ Here are some of the additional features and capabilities that Qodo Merge Pro of
|
||||
| Feature | Description |
|
||||
|----------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| [**Model selection**](https://qodo-merge-docs.qodo.ai/usage-guide/PR_agent_pro_models/) | Choose the model that best fits your needs, among top models like `GPT4` and `Claude-Sonnet-3.5`
|
||||
| [**Global and wiki configuration**](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/) | Control configurations for many repositories from a single location; <br>Edit configuration of a single repo without committing code |
|
||||
| [**Apply suggestions**](https://qodo-merge-docs.qodo.ai/tools/improve/#overview) | Generate committable code from the relevant suggestions interactively by clicking on a checkbox |
|
||||
| [**Global and wiki configuration**](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/) | Control configurations for many repositories from a single location; <br>Edit configuration of a single repo without commiting code |
|
||||
| [**Apply suggestions**](https://qodo-merge-docs.qodo.ai/tools/improve/#overview) | Generate commitable code from the relevant suggestions interactively by clicking on a checkbox |
|
||||
| [**Suggestions impact**](https://qodo-merge-docs.qodo.ai/tools/improve/#assessing-impact) | Automatically mark suggestions that were implemented by the user (either directly in GitHub, or indirectly in the IDE) to enable tracking of the impact of the suggestions |
|
||||
| [**CI feedback**](https://qodo-merge-docs.qodo.ai/tools/ci_feedback/) | Automatically analyze failed CI checks on GitHub and provide actionable feedback in the PR conversation, helping to resolve issues quickly |
|
||||
| [**Advanced usage statistics**](https://www.codium.ai/contact/#/) | Qodo Merge Pro offers detailed statistics at user, repository, and company levels, including metrics about Qodo Merge usage, and also general statistics and insights |
|
||||
| [**Incorporating companies' best practices**](https://qodo-merge-docs.qodo.ai/tools/improve/#best-practices) | Use the companies' best practices as reference to increase the effectiveness and the relevance of the code suggestions |
|
||||
| [**Interactive triggering**](https://qodo-merge-docs.qodo.ai/tools/analyze/#example-usage) | Interactively apply different tools via the `analyze` command |
|
||||
| [**SOC2 compliance check**](https://qodo-merge-docs.qodo.ai/tools/review/#configuration-options) | Ensures the PR contains a ticket to a project management system (e.g., Jira, Asana, Trello, etc.)
|
||||
| [**Custom labels**](https://qodo-merge-docs.qodo.ai/tools/describe/#handle-custom-labels-from-the-repos-labels-page) | Define custom labels for Qodo Merge to assign to the PR |
|
||||
|
||||
### Additional tools
|
||||
@ -48,4 +49,4 @@ Here are additional tools that are available only for Qodo Merge Pro users:
|
||||
Qodo Merge Pro leverages the world's leading code models - Claude 3.5 Sonnet and GPT-4.
|
||||
As a result, its primary tools such as `describe`, `review`, and `improve`, as well as the PR-chat feature, support virtually all programming languages.
|
||||
|
||||
For specialized commands that require static code analysis, Qodo Merge Pro offers support for specific languages. For more details about features that require static code analysis, please refer to the [documentation](https://qodo-merge-docs.qodo.ai/tools/analyze/#overview).
|
||||
For specialized commands that require static code analysis, Qodo Merge Pro offers support for specific languages. For more details about features that require static code analysis, please refer to the [documentation](https://qodo-merge-docs.qodo.ai/tools/analyze/#overview).
|
||||
@ -25,7 +25,7 @@ There are 3 ways to enable custom labels:
|
||||
When working from CLI, you need to apply the [configuration changes](#configuration-options) to the [custom_labels file](https://github.com/Codium-ai/pr-agent/blob/main/pr_agent/settings/custom_labels.toml):
|
||||
|
||||
#### 2. Repo configuration file
|
||||
To enable custom labels, you need to apply the [configuration changes](#configuration-options) to the local `.pr_agent.toml` file in your repository.
|
||||
To enable custom labels, you need to apply the [configuration changes](#configuration-options) to the local `.pr_agent.toml` file in you repository.
|
||||
|
||||
#### 3. Handle custom labels from the Repo's labels page 💎
|
||||
> This feature is available only in Qodo Merge Pro
|
||||
@ -57,4 +57,4 @@ description = "Description of when AI should suggest this label"
|
||||
|
||||
[custom_labels."Custom Label 2"]
|
||||
description = "Description of when AI should suggest this label 2"
|
||||
```
|
||||
```
|
||||
@ -34,7 +34,7 @@ pr_commands = [
|
||||
]
|
||||
|
||||
[pr_description]
|
||||
publish_labels = true
|
||||
publish_labels = ...
|
||||
...
|
||||
```
|
||||
|
||||
@ -49,7 +49,7 @@ publish_labels = true
|
||||
<table>
|
||||
<tr>
|
||||
<td><b>publish_labels</b></td>
|
||||
<td>If set to true, the tool will publish labels to the PR. Default is false.</td>
|
||||
<td>If set to true, the tool will publish the labels to the PR. Default is true.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>publish_description_as_comment</b></td>
|
||||
|
||||
@ -67,33 +67,6 @@ In post-process, Qodo Merge counts the number of suggestions that were implement
|
||||
|
||||
{width=512}
|
||||
|
||||
## Suggestion tracking 💎
|
||||
`Platforms supported: GitHub, GitLab`
|
||||
|
||||
Qodo Merge employs an novel detection system to automatically [identify](https://qodo-merge-docs.qodo.ai/core-abilities/impact_evaluation/) AI code suggestions that PR authors have accepted and implemented.
|
||||
|
||||
Accepted suggestions are also automatically documented in a dedicated wiki page called `.pr_agent_accepted_suggestions`, allowing users to track historical changes, assess the tool's effectiveness, and learn from previously implemented recommendations in the repository.
|
||||
An example [result](https://github.com/Codium-ai/pr-agent/wiki/.pr_agent_accepted_suggestions):
|
||||
|
||||
[{width=768}](https://github.com/Codium-ai/pr-agent/wiki/.pr_agent_accepted_suggestions)
|
||||
|
||||
This dedicated wiki page will also serve as a foundation for future AI model improvements, allowing it to learn from historically implemented suggestions and generate more targeted, contextually relevant recommendations.
|
||||
|
||||
This feature is controlled by a boolean configuration parameter: `pr_code_suggestions.wiki_page_accepted_suggestions` (default is true).
|
||||
|
||||
!!! note "Wiki must be enabled"
|
||||
While the aggregation process is automatic, GitHub repositories require a one-time manual wiki setup.
|
||||
|
||||
To initialize the wiki: navigate to `Wiki`, select `Create the first page`, then click `Save page`.
|
||||
|
||||
{width=768}
|
||||
|
||||
Once a wiki repo is created, the tool will automatically use this wiki for tracking suggestions.
|
||||
|
||||
!!! note "Why a wiki page?"
|
||||
Your code belongs to you, and we respect your privacy. Hence, we won't store any code suggestions in an external database.
|
||||
|
||||
Instead, we leverage a dedicated private page, within your repository wiki, to track suggestions. This approach offers convenient secure suggestion tracking while avoiding pull requests or any noise to the main repository.
|
||||
|
||||
## Usage Tips
|
||||
|
||||
@ -141,16 +114,9 @@ code_suggestions_self_review_text = "... (your text here) ..."
|
||||
{width=512}
|
||||
|
||||
|
||||
!!! tip "Tip - Reducing visual footprint after self-review 💎"
|
||||
|
||||
The configuration parameter `pr_code_suggestions.fold_suggestions_on_self_review` (default is True)
|
||||
can be used to automatically fold the suggestions after the user clicks the self-review checkbox.
|
||||
|
||||
This reduces the visual footprint of the suggestions, and also indicates to the PR reviewer that the suggestions have been reviewed by the PR author, and don't require further attention.
|
||||
|
||||
|
||||
|
||||
!!! tip "Tip - Demanding self-review from the PR author 💎"
|
||||
!!! tip "Tip - demanding self-review from the PR author 💎"
|
||||
|
||||
By setting:
|
||||
```toml
|
||||
@ -299,10 +265,6 @@ Using a combination of both can help the AI model to provide relevant and tailor
|
||||
<td><b>enable_chat_text</b></td>
|
||||
<td>If set to true, the tool will display a reference to the PR chat in the comment. Default is true.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>wiki_page_accepted_suggestions</b></td>
|
||||
<td>If set to true, the tool will automatically track accepted suggestions in a dedicated wiki page called `.pr_agent_accepted_suggestions`. Default is true.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
??? example "Params for number of suggestions and AI calls"
|
||||
|
||||
@ -138,9 +138,20 @@ num_code_suggestions = ...
|
||||
<td><b>require_security_review</b></td>
|
||||
<td>If set to true, the tool will add a section that checks if the PR contains a possible security or vulnerability issue. Default is true.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
!!! example "SOC2 ticket compliance 💎"
|
||||
|
||||
This sub-tool checks if the PR description properly contains a ticket to a project management system (e.g., Jira, Asana, Trello, etc.), as required by SOC2 compliance. If not, it will add a label to the PR: "Missing SOC2 ticket".
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td><b>require_ticket_analysis_review</b></td>
|
||||
<td>If set to true, and the PR contains a GitHub ticket number, the tool will add a section that checks if the PR in fact fulfilled the ticket requirements. Default is true.</td>
|
||||
<td><b>require_soc2_ticket</b></td>
|
||||
<td>If set to true, the SOC2 ticket checker sub-tool will be enabled. Default is false.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>soc2_ticket_prompt</b></td>
|
||||
<td>The prompt for the SOC2 ticket review. Default is: `Does the PR description include a link to ticket in a project management system (e.g., Jira, Asana, Trello, etc.) ?`. Edit this field if your compliance requirements are different.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
@ -182,7 +193,7 @@ If enabled, the `review` tool can approve a PR when a specific comment, `/review
|
||||
It is recommended to review the [Configuration options](#configuration-options) section, and choose the relevant options for your use case.
|
||||
|
||||
Some of the features that are disabled by default are quite useful, and should be considered for enabling. For example:
|
||||
`require_score_review`, and more.
|
||||
`require_score_review`, `require_soc2_ticket`, and more.
|
||||
|
||||
On the other hand, if you find one of the enabled features to be irrelevant for your use case, disable it. No default configuration can fit all use cases.
|
||||
|
||||
|
||||
@ -18,7 +18,7 @@ In terms of precedence, wiki configurations will override local configurations,
|
||||
|
||||
## Wiki configuration file 💎
|
||||
|
||||
`Platforms supported: GitHub, GitLab, Bitbucket`
|
||||
`Platforms supported: GitHub, GitLab`
|
||||
|
||||
With Qodo Merge Pro, you can set configurations by creating a page called `.pr_agent.toml` in the [wiki](https://github.com/Codium-ai/pr-agent/wiki/pr_agent.toml) of the repo.
|
||||
The advantage of this method is that it allows to set configurations without needing to commit new content to the repo - just edit the wiki page and **save**.
|
||||
|
||||
@ -82,11 +82,11 @@
|
||||
|
||||
<footer class="wrapper">
|
||||
<div class="container">
|
||||
<p class="footer-text">© 2024 <a href="https://www.qodo.ai/" target="_blank" rel="noopener">Qodo</a></p>
|
||||
<p class="footer-text">© 2024 <a href="https://www.codium.ai/" target="_blank" rel="noopener">CodiumAI</a></p>
|
||||
<div class="footer-links">
|
||||
<a href="https://qodo-gen-docs.qodo.ai/">Qodo Gen</a>
|
||||
<a href="https://codiumate-docs.codium.ai/">Codiumate</a>
|
||||
<p>|</p>
|
||||
<a href="https://qodo-flow-docs.qodo.ai/">AlphaCodium</a>
|
||||
<a href="https://alpha-codium-docs.codium.ai/">AlphaCodium</a>
|
||||
</div>
|
||||
<div class="social-icons">
|
||||
<a href="https://github.com/Codium-ai" target="_blank" rel="noopener" title="github.com" class="social-link">
|
||||
@ -95,16 +95,16 @@
|
||||
<a href="https://discord.com/invite/SgSxuQ65GF" target="_blank" rel="noopener" title="discord.com" class="social-link">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 6.5.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2023 Fonticons, Inc.--><path d="M524.531 69.836a1.5 1.5 0 0 0-.764-.7A485.065 485.065 0 0 0 404.081 32.03a1.816 1.816 0 0 0-1.923.91 337.461 337.461 0 0 0-14.9 30.6 447.848 447.848 0 0 0-134.426 0 309.541 309.541 0 0 0-15.135-30.6 1.89 1.89 0 0 0-1.924-.91 483.689 483.689 0 0 0-119.688 37.107 1.712 1.712 0 0 0-.788.676C39.068 183.651 18.186 294.69 28.43 404.354a2.016 2.016 0 0 0 .765 1.375 487.666 487.666 0 0 0 146.825 74.189 1.9 1.9 0 0 0 2.063-.676A348.2 348.2 0 0 0 208.12 430.4a1.86 1.86 0 0 0-1.019-2.588 321.173 321.173 0 0 1-45.868-21.853 1.885 1.885 0 0 1-.185-3.126 251.047 251.047 0 0 0 9.109-7.137 1.819 1.819 0 0 1 1.9-.256c96.229 43.917 200.41 43.917 295.5 0a1.812 1.812 0 0 1 1.924.233 234.533 234.533 0 0 0 9.132 7.16 1.884 1.884 0 0 1-.162 3.126 301.407 301.407 0 0 1-45.89 21.83 1.875 1.875 0 0 0-1 2.611 391.055 391.055 0 0 0 30.014 48.815 1.864 1.864 0 0 0 2.063.7A486.048 486.048 0 0 0 610.7 405.729a1.882 1.882 0 0 0 .765-1.352c12.264-126.783-20.532-236.912-86.934-334.541ZM222.491 337.58c-28.972 0-52.844-26.587-52.844-59.239s23.409-59.241 52.844-59.241c29.665 0 53.306 26.82 52.843 59.239 0 32.654-23.41 59.241-52.843 59.241Zm195.38 0c-28.971 0-52.843-26.587-52.843-59.239s23.409-59.241 52.843-59.241c29.667 0 53.307 26.82 52.844 59.239 0 32.654-23.177 59.241-52.844 59.241Z"></path></svg>
|
||||
</a>
|
||||
<a href="https://www.youtube.com/@QodoAI" target="_blank" rel="noopener" title="www.youtube.com" class="social-link">
|
||||
<a href="https://www.youtube.com/@Codium-AI" target="_blank" rel="noopener" title="www.youtube.com" class="social-link">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 576 512"><!--! Font Awesome Free 6.5.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2023 Fonticons, Inc.--><path d="M549.655 124.083c-6.281-23.65-24.787-42.276-48.284-48.597C458.781 64 288 64 288 64S117.22 64 74.629 75.486c-23.497 6.322-42.003 24.947-48.284 48.597-11.412 42.867-11.412 132.305-11.412 132.305s0 89.438 11.412 132.305c6.281 23.65 24.787 41.5 48.284 47.821C117.22 448 288 448 288 448s170.78 0 213.371-11.486c23.497-6.321 42.003-24.171 48.284-47.821 11.412-42.867 11.412-132.305 11.412-132.305s0-89.438-11.412-132.305zm-317.51 213.508V175.185l142.739 81.205-142.739 81.201z"></path></svg>
|
||||
</a>
|
||||
<a href="https://www.linkedin.com/company/qodoai" target="_blank" rel="noopener" title="www.linkedin.com" class="social-link">
|
||||
<a href="https://www.linkedin.com/company/codiumai" target="_blank" rel="noopener" title="www.linkedin.com" class="social-link">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.5.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2023 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3zM135.4 416H69V202.2h66.5V416zm-33.2-243c-21.3 0-38.5-17.3-38.5-38.5S80.9 96 102.2 96c21.2 0 38.5 17.3 38.5 38.5 0 21.3-17.2 38.5-38.5 38.5zm282.1 243h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9V416z"></path></svg>
|
||||
</a>
|
||||
<a href="https://twitter.com/QodoAI" target="_blank" rel="noopener" title="twitter.com" class="social-link">
|
||||
<a href="https://twitter.com/CodiumAI" target="_blank" rel="noopener" title="twitter.com" class="social-link">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!--! Font Awesome Free 6.5.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2023 Fonticons, Inc.--><path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"></path></svg>
|
||||
</a>
|
||||
<a href="https://www.instagram.com/qodo_ai" target="_blank" rel="noopener" title="www.instagram.com" class="social-link">
|
||||
<a href="https://www.instagram.com/codiumai/" target="_blank" rel="noopener" title="www.instagram.com" class="social-link">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.5.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2023 Fonticons, Inc.--><path d="M224.1 141c-63.6 0-114.9 51.3-114.9 114.9s51.3 114.9 114.9 114.9S339 319.5 339 255.9 287.7 141 224.1 141zm0 189.6c-41.1 0-74.7-33.5-74.7-74.7s33.5-74.7 74.7-74.7 74.7 33.5 74.7 74.7-33.6 74.7-74.7 74.7zm146.4-194.3c0 14.9-12 26.8-26.8 26.8-14.9 0-26.8-12-26.8-26.8s12-26.8 26.8-26.8 26.8 12 26.8 26.8zm76.1 27.2c-1.7-35.9-9.9-67.7-36.2-93.9-26.2-26.2-58-34.4-93.9-36.2-37-2.1-147.9-2.1-184.9 0-35.8 1.7-67.6 9.9-93.9 36.1s-34.4 58-36.2 93.9c-2.1 37-2.1 147.9 0 184.9 1.7 35.9 9.9 67.7 36.2 93.9s58 34.4 93.9 36.2c37 2.1 147.9 2.1 184.9 0 35.9-1.7 67.7-9.9 93.9-36.2 26.2-26.2 34.4-58 36.2-93.9 2.1-37 2.1-147.8 0-184.8zM398.8 388c-7.8 19.6-22.9 34.7-42.6 42.6-29.5 11.7-99.5 9-132.1 9s-102.7 2.6-132.1-9c-19.6-7.8-34.7-22.9-42.6-42.6-11.7-29.5-9-99.5-9-132.1s-2.6-102.7 9-132.1c7.8-19.6 22.9-34.7 42.6-42.6 29.5-11.7 99.5-9 132.1-9s102.7-2.6 132.1 9c19.6 7.8 34.7 22.9 42.6 42.6 11.7 29.5 9 99.5 9 132.1s2.7 102.7-9 132.1z"></path></svg>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
@ -34,7 +34,6 @@ MAX_TOKENS = {
|
||||
'vertex_ai/claude-3-sonnet@20240229': 100000,
|
||||
'vertex_ai/claude-3-opus@20240229': 100000,
|
||||
'vertex_ai/claude-3-5-sonnet@20240620': 100000,
|
||||
'vertex_ai/claude-3-5-sonnet-v2@20241022': 100000,
|
||||
'vertex_ai/gemini-1.5-pro': 1048576,
|
||||
'vertex_ai/gemini-1.5-flash': 1048576,
|
||||
'vertex_ai/gemma2': 8200,
|
||||
@ -45,14 +44,12 @@ MAX_TOKENS = {
|
||||
'anthropic.claude-v2': 100000,
|
||||
'anthropic/claude-3-opus-20240229': 100000,
|
||||
'anthropic/claude-3-5-sonnet-20240620': 100000,
|
||||
'anthropic/claude-3-5-sonnet-20241022': 100000,
|
||||
'bedrock/anthropic.claude-instant-v1': 100000,
|
||||
'bedrock/anthropic.claude-v2': 100000,
|
||||
'bedrock/anthropic.claude-v2:1': 100000,
|
||||
'bedrock/anthropic.claude-3-sonnet-20240229-v1:0': 100000,
|
||||
'bedrock/anthropic.claude-3-haiku-20240307-v1:0': 100000,
|
||||
'bedrock/anthropic.claude-3-5-sonnet-20240620-v1:0': 100000,
|
||||
'bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0': 100000,
|
||||
'claude-3-5-sonnet': 100000,
|
||||
'groq/llama3-8b-8192': 8192,
|
||||
'groq/llama3-70b-8192': 8192,
|
||||
|
||||
@ -171,7 +171,6 @@ class LiteLLMAIHandler(BaseAiHandler):
|
||||
get_logger().warning(
|
||||
"Empty system prompt for claude model. Adding a newline character to prevent OpenAI API error.")
|
||||
messages = [{"role": "system", "content": system}, {"role": "user", "content": user}]
|
||||
|
||||
if img_path:
|
||||
try:
|
||||
# check if the image link is alive
|
||||
@ -186,30 +185,14 @@ class LiteLLMAIHandler(BaseAiHandler):
|
||||
messages[1]["content"] = [{"type": "text", "text": messages[1]["content"]},
|
||||
{"type": "image_url", "image_url": {"url": img_path}}]
|
||||
|
||||
# Currently O1 does not support separate system and user prompts
|
||||
O1_MODEL_PREFIX = 'o1-'
|
||||
model_type = model.split('/')[-1] if '/' in model else model
|
||||
if model_type.startswith(O1_MODEL_PREFIX):
|
||||
user = f"{system}\n\n\n{user}"
|
||||
system = ""
|
||||
get_logger().info(f"Using O1 model, combining system and user prompts")
|
||||
messages = [{"role": "user", "content": user}]
|
||||
kwargs = {
|
||||
"model": model,
|
||||
"deployment_id": deployment_id,
|
||||
"messages": messages,
|
||||
"timeout": get_settings().config.ai_timeout,
|
||||
"api_base": self.api_base,
|
||||
}
|
||||
else:
|
||||
kwargs = {
|
||||
"model": model,
|
||||
"deployment_id": deployment_id,
|
||||
"messages": messages,
|
||||
"temperature": temperature,
|
||||
"timeout": get_settings().config.ai_timeout,
|
||||
"api_base": self.api_base,
|
||||
}
|
||||
kwargs = {
|
||||
"model": model,
|
||||
"deployment_id": deployment_id,
|
||||
"messages": messages,
|
||||
"temperature": temperature,
|
||||
"timeout": get_settings().config.ai_timeout,
|
||||
"api_base": self.api_base,
|
||||
}
|
||||
|
||||
if get_settings().litellm.get("enable_callbacks", False):
|
||||
kwargs = self.add_litellm_callbacks(kwargs)
|
||||
|
||||
@ -281,7 +281,7 @@ __old hunk__
|
||||
prev_header_line = []
|
||||
header_line = []
|
||||
for line_i, line in enumerate(patch_lines):
|
||||
if 'no newline at end of file' in line.lower():
|
||||
if 'no newline at end of file' in line.lower().strip().strip('//'):
|
||||
continue
|
||||
|
||||
if line.startswith('@@'):
|
||||
@ -290,19 +290,18 @@ __old hunk__
|
||||
if match and (new_content_lines or old_content_lines): # found a new hunk, split the previous lines
|
||||
if prev_header_line:
|
||||
patch_with_lines_str += f'\n{prev_header_line}\n'
|
||||
is_plus_lines = is_minus_lines = False
|
||||
if new_content_lines:
|
||||
is_plus_lines = any([line.startswith('+') for line in new_content_lines])
|
||||
if is_plus_lines:
|
||||
patch_with_lines_str = patch_with_lines_str.rstrip() + '\n__new hunk__\n'
|
||||
for i, line_new in enumerate(new_content_lines):
|
||||
patch_with_lines_str += f"{start2 + i} {line_new}\n"
|
||||
if old_content_lines:
|
||||
is_minus_lines = any([line.startswith('-') for line in old_content_lines])
|
||||
if is_plus_lines or is_minus_lines: # notice 'True' here - we always present __new hunk__ for section, otherwise LLM gets confused
|
||||
patch_with_lines_str = patch_with_lines_str.rstrip() + '\n__new hunk__\n'
|
||||
for i, line_new in enumerate(new_content_lines):
|
||||
patch_with_lines_str += f"{start2 + i} {line_new}\n"
|
||||
if is_minus_lines:
|
||||
patch_with_lines_str = patch_with_lines_str.rstrip() + '\n__old hunk__\n'
|
||||
for line_old in old_content_lines:
|
||||
patch_with_lines_str += f"{line_old}\n"
|
||||
if is_minus_lines:
|
||||
patch_with_lines_str = patch_with_lines_str.rstrip() + '\n__old hunk__\n'
|
||||
for line_old in old_content_lines:
|
||||
patch_with_lines_str += f"{line_old}\n"
|
||||
new_content_lines = []
|
||||
old_content_lines = []
|
||||
if match:
|
||||
@ -326,19 +325,18 @@ __old hunk__
|
||||
# finishing last hunk
|
||||
if match and new_content_lines:
|
||||
patch_with_lines_str += f'\n{header_line}\n'
|
||||
is_plus_lines = is_minus_lines = False
|
||||
if new_content_lines:
|
||||
is_plus_lines = any([line.startswith('+') for line in new_content_lines])
|
||||
if is_plus_lines:
|
||||
patch_with_lines_str = patch_with_lines_str.rstrip() + '\n__new hunk__\n'
|
||||
for i, line_new in enumerate(new_content_lines):
|
||||
patch_with_lines_str += f"{start2 + i} {line_new}\n"
|
||||
if old_content_lines:
|
||||
is_minus_lines = any([line.startswith('-') for line in old_content_lines])
|
||||
if is_plus_lines or is_minus_lines: # notice 'True' here - we always present __new hunk__ for section, otherwise LLM gets confused
|
||||
patch_with_lines_str = patch_with_lines_str.rstrip() + '\n__new hunk__\n'
|
||||
for i, line_new in enumerate(new_content_lines):
|
||||
patch_with_lines_str += f"{start2 + i} {line_new}\n"
|
||||
if is_minus_lines:
|
||||
patch_with_lines_str = patch_with_lines_str.rstrip() + '\n__old hunk__\n'
|
||||
for line_old in old_content_lines:
|
||||
patch_with_lines_str += f"{line_old}\n"
|
||||
if is_minus_lines:
|
||||
patch_with_lines_str = patch_with_lines_str.rstrip() + '\n__old hunk__\n'
|
||||
for line_old in old_content_lines:
|
||||
patch_with_lines_str += f"{line_old}\n"
|
||||
|
||||
return patch_with_lines_str.rstrip()
|
||||
|
||||
|
||||
@ -1,22 +1,18 @@
|
||||
from __future__ import annotations
|
||||
import html2text
|
||||
|
||||
import html
|
||||
import copy
|
||||
import difflib
|
||||
import hashlib
|
||||
import html
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import textwrap
|
||||
import time
|
||||
import traceback
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
from typing import Any, List, Tuple
|
||||
|
||||
|
||||
import html2text
|
||||
import requests
|
||||
import yaml
|
||||
from pydantic import BaseModel
|
||||
from starlette_context import context
|
||||
@ -114,7 +110,6 @@ def convert_to_markdown_v2(output_data: dict,
|
||||
"Insights from user's answers": "📝",
|
||||
"Code feedback": "🤖",
|
||||
"Estimated effort to review [1-5]": "⏱️",
|
||||
"Ticket compliance check": "🎫",
|
||||
}
|
||||
markdown_text = ""
|
||||
if not incremental_review:
|
||||
@ -170,8 +165,6 @@ def convert_to_markdown_v2(output_data: dict,
|
||||
markdown_text += f'### {emoji} No relevant tests\n\n'
|
||||
else:
|
||||
markdown_text += f"### PR contains tests\n\n"
|
||||
elif 'ticket compliance check' in key_nice.lower():
|
||||
markdown_text = ticket_markdown_logic(emoji, markdown_text, value, gfm_supported)
|
||||
elif 'security concerns' in key_nice.lower():
|
||||
if gfm_supported:
|
||||
markdown_text += f"<tr><td>"
|
||||
@ -261,52 +254,6 @@ def convert_to_markdown_v2(output_data: dict,
|
||||
return markdown_text
|
||||
|
||||
|
||||
def ticket_markdown_logic(emoji, markdown_text, value, gfm_supported) -> str:
|
||||
ticket_compliance_str = ""
|
||||
final_compliance_level = -1
|
||||
if isinstance(value, list):
|
||||
for v in value:
|
||||
ticket_url = v.get('ticket_url', '').strip()
|
||||
compliance_level = v.get('overall_compliance_level', '').strip()
|
||||
# add emojis, if 'Fully compliant' ✅, 'Partially compliant' 🔶, or 'Not compliant' ❌
|
||||
if compliance_level.lower() == 'fully compliant':
|
||||
# compliance_level = '✅ Fully compliant'
|
||||
final_compliance_level = 2 if final_compliance_level == -1 else 1
|
||||
elif compliance_level.lower() == 'partially compliant':
|
||||
# compliance_level = '🔶 Partially compliant'
|
||||
final_compliance_level = 1
|
||||
elif compliance_level.lower() == 'not compliant':
|
||||
# compliance_level = '❌ Not compliant'
|
||||
final_compliance_level = 0 if final_compliance_level < 1 else 1
|
||||
|
||||
# explanation = v.get('compliance_analysis', '').strip()
|
||||
explanation = ''
|
||||
fully_compliant_str = v.get('fully_compliant_requirements', '').strip()
|
||||
not_compliant_str = v.get('not_compliant_requirements', '').strip()
|
||||
if fully_compliant_str:
|
||||
explanation += f"Fully compliant requirements:\n{fully_compliant_str}\n\n"
|
||||
if not_compliant_str:
|
||||
explanation += f"Not compliant requirements:\n{not_compliant_str}\n\n"
|
||||
|
||||
ticket_compliance_str += f"\n\n**[{ticket_url.split('/')[-1]}]({ticket_url}) - {compliance_level}**\n\n{explanation}\n\n"
|
||||
if final_compliance_level == 2:
|
||||
compliance_level = '✅'
|
||||
elif final_compliance_level == 1:
|
||||
compliance_level = '🔶'
|
||||
else:
|
||||
compliance_level = '❌'
|
||||
|
||||
if gfm_supported:
|
||||
markdown_text += f"<tr><td>\n\n"
|
||||
markdown_text += f"**{emoji} Ticket compliance analysis {compliance_level}**\n\n"
|
||||
markdown_text += ticket_compliance_str
|
||||
markdown_text += f"</td></tr>\n"
|
||||
else:
|
||||
markdown_text += f"### {emoji} Ticket compliance analysis {compliance_level}\n\n"
|
||||
markdown_text += ticket_compliance_str+"\n\n"
|
||||
return markdown_text
|
||||
|
||||
|
||||
def process_can_be_split(emoji, value):
|
||||
try:
|
||||
# key_nice = "Can this PR be split?"
|
||||
@ -607,8 +554,7 @@ def load_yaml(response_text: str, keys_fix_yaml: List[str] = [], first_key="", l
|
||||
get_logger().warning(f"Initial failure to parse AI prediction: {e}")
|
||||
data = try_fix_yaml(response_text, keys_fix_yaml=keys_fix_yaml, first_key=first_key, last_key=last_key)
|
||||
if not data:
|
||||
get_logger().error(f"Failed to parse AI prediction after fallbacks",
|
||||
artifact={'response_text': response_text})
|
||||
get_logger().error(f"Failed to parse AI prediction after fallbacks", artifact={'response_text': response_text})
|
||||
else:
|
||||
get_logger().info(f"Successfully parsed AI prediction after fallbacks",
|
||||
artifact={'response_text': response_text})
|
||||
@ -895,64 +841,56 @@ def find_line_number_of_relevant_line_in_file(diff_files: List[FilePatchInfo],
|
||||
break
|
||||
return position, absolute_position
|
||||
|
||||
def get_rate_limit_status(github_token) -> dict:
|
||||
GITHUB_API_URL = get_settings(use_context=False).get("GITHUB.BASE_URL", "https://api.github.com").rstrip("/") # "https://api.github.com"
|
||||
# GITHUB_API_URL = "https://api.github.com"
|
||||
RATE_LIMIT_URL = f"{GITHUB_API_URL}/rate_limit"
|
||||
HEADERS = {
|
||||
"Accept": "application/vnd.github.v3+json",
|
||||
"Authorization": f"token {github_token}"
|
||||
}
|
||||
def validate_and_await_rate_limit(rate_limit_status=None, git_provider=None, get_rate_limit_status_func=None):
|
||||
if git_provider and not rate_limit_status:
|
||||
rate_limit_status = {'resources': git_provider.github_client.get_rate_limit().raw_data}
|
||||
|
||||
response = requests.get(RATE_LIMIT_URL, headers=HEADERS)
|
||||
try:
|
||||
rate_limit_info = response.json()
|
||||
if rate_limit_info.get('message') == 'Rate limiting is not enabled.': # for github enterprise
|
||||
return {'resources': {}}
|
||||
response.raise_for_status() # Check for HTTP errors
|
||||
except: # retry
|
||||
time.sleep(0.1)
|
||||
response = requests.get(RATE_LIMIT_URL, headers=HEADERS)
|
||||
return response.json()
|
||||
return rate_limit_info
|
||||
|
||||
|
||||
def validate_rate_limit_github(github_token, installation_id=None, threshold=0.1) -> bool:
|
||||
try:
|
||||
rate_limit_status = get_rate_limit_status(github_token)
|
||||
if installation_id:
|
||||
get_logger().debug(f"installation_id: {installation_id}, Rate limit status: {rate_limit_status['rate']}")
|
||||
if not rate_limit_status:
|
||||
rate_limit_status = get_rate_limit_status_func()
|
||||
# validate that the rate limit is not exceeded
|
||||
# validate that the rate limit is not exceeded
|
||||
for key, value in rate_limit_status['resources'].items():
|
||||
if value['remaining'] < value['limit'] * threshold:
|
||||
get_logger().error(f"key: {key}, value: {value}")
|
||||
return False
|
||||
return True
|
||||
except Exception as e:
|
||||
get_logger().error(f"Error in rate limit {e}",
|
||||
artifact={"traceback": traceback.format_exc()})
|
||||
return True
|
||||
is_rate_limit = False
|
||||
for key, value in rate_limit_status['resources'].items():
|
||||
if value['remaining'] == 0:
|
||||
print(f"key: {key}, value: {value}")
|
||||
is_rate_limit = True
|
||||
sleep_time_sec = value['reset'] - datetime.now().timestamp()
|
||||
sleep_time_hour = sleep_time_sec / 3600.0
|
||||
print(f"Rate limit exceeded. Sleeping for {sleep_time_hour} hours")
|
||||
if sleep_time_sec > 0:
|
||||
time.sleep(sleep_time_sec+1)
|
||||
|
||||
if git_provider:
|
||||
rate_limit_status = {'resources': git_provider.github_client.get_rate_limit().raw_data}
|
||||
else:
|
||||
rate_limit_status = get_rate_limit_status_func()
|
||||
|
||||
return is_rate_limit
|
||||
|
||||
|
||||
def validate_and_await_rate_limit(github_token):
|
||||
try:
|
||||
rate_limit_status = get_rate_limit_status(github_token)
|
||||
# validate that the rate limit is not exceeded
|
||||
for key, value in rate_limit_status['resources'].items():
|
||||
if value['remaining'] < value['limit'] // 80:
|
||||
get_logger().error(f"key: {key}, value: {value}")
|
||||
sleep_time_sec = value['reset'] - datetime.now().timestamp()
|
||||
sleep_time_hour = sleep_time_sec / 3600.0
|
||||
get_logger().error(f"Rate limit exceeded. Sleeping for {sleep_time_hour} hours")
|
||||
if sleep_time_sec > 0:
|
||||
time.sleep(sleep_time_sec + 1)
|
||||
rate_limit_status = get_rate_limit_status(github_token)
|
||||
return rate_limit_status
|
||||
except:
|
||||
get_logger().error("Error in rate limit")
|
||||
return None
|
||||
|
||||
def get_largest_component(pr_url):
|
||||
from pr_agent.tools.pr_analyzer import PRAnalyzer
|
||||
publish_output = get_settings().config.publish_output
|
||||
get_settings().config.publish_output = False # disable publish output
|
||||
analyzer = PRAnalyzer(pr_url)
|
||||
methods_dict_files = analyzer.run_sync()
|
||||
get_settings().config.publish_output = publish_output
|
||||
max_lines_changed = 0
|
||||
file_b = ""
|
||||
component_name_b = ""
|
||||
for file in methods_dict_files:
|
||||
for method in methods_dict_files[file]:
|
||||
try:
|
||||
if methods_dict_files[file][method]['num_plus_lines'] > max_lines_changed:
|
||||
max_lines_changed = methods_dict_files[file][method]['num_plus_lines']
|
||||
file_b = file
|
||||
component_name_b = method
|
||||
except:
|
||||
pass
|
||||
if component_name_b:
|
||||
get_logger().info(f"Using the largest changed component: '{component_name_b}'")
|
||||
return component_name_b, file_b
|
||||
else:
|
||||
return None, None
|
||||
|
||||
def github_action_output(output_data: dict, key_name: str):
|
||||
try:
|
||||
@ -968,7 +906,7 @@ def github_action_output(output_data: dict, key_name: str):
|
||||
|
||||
|
||||
def show_relevant_configurations(relevant_section: str) -> str:
|
||||
skip_keys = ['ai_disclaimer', 'ai_disclaimer_title', 'ANALYTICS_FOLDER', 'secret_provider', "skip_keys", "app_id", "redirect",
|
||||
skip_keys = ['ai_disclaimer', 'ai_disclaimer_title', 'ANALYTICS_FOLDER', 'secret_provider', "skip_keys",
|
||||
'trial_prefix_message', 'no_eligible_message', 'identity_provider', 'ALLOWED_REPOS','APP_NAME']
|
||||
extra_skip_keys = get_settings().config.get('config.skip_keys', [])
|
||||
if extra_skip_keys:
|
||||
@ -1001,25 +939,6 @@ def is_value_no(value):
|
||||
return False
|
||||
|
||||
|
||||
def set_pr_string(repo_name, pr_number):
|
||||
return f"{repo_name}#{pr_number}"
|
||||
|
||||
|
||||
def string_to_uniform_number(s: str) -> float:
|
||||
"""
|
||||
Convert a string to a uniform number in the range [0, 1].
|
||||
The uniform distribution is achieved by the nature of the SHA-256 hash function, which produces a uniformly distributed hash value over its output space.
|
||||
"""
|
||||
# Generate a hash of the string
|
||||
hash_object = hashlib.sha256(s.encode())
|
||||
# Convert the hash to an integer
|
||||
hash_int = int(hash_object.hexdigest(), 16)
|
||||
# Normalize the integer to the range [0, 1]
|
||||
max_hash_int = 2 ** 256 - 1
|
||||
uniform_number = float(hash_int) / max_hash_int
|
||||
return uniform_number
|
||||
|
||||
|
||||
def process_description(description_full: str) -> Tuple[str, List]:
|
||||
if not description_full:
|
||||
return "", []
|
||||
@ -1079,10 +998,7 @@ def process_description(description_full: str) -> Tuple[str, List]:
|
||||
'long_summary': long_summary
|
||||
})
|
||||
else:
|
||||
if '<code>...</code>' in file_data:
|
||||
pass # PR with many files. some did not get analyzed
|
||||
else:
|
||||
get_logger().error(f"Failed to parse description", artifact={'description': file_data})
|
||||
get_logger().error(f"Failed to parse description", artifact={'description': file_data})
|
||||
except Exception as e:
|
||||
get_logger().exception(f"Failed to process description: {e}", artifact={'description': file_data})
|
||||
|
||||
|
||||
@ -336,22 +336,19 @@ class AzureDevopsProvider(GitProvider):
|
||||
version = GitVersionDescriptor(
|
||||
version=base_sha.commit_id, version_type="commit"
|
||||
)
|
||||
if edit_type == EDIT_TYPE.ADDED:
|
||||
try:
|
||||
original_file_content_str = self.azure_devops_client.get_item(
|
||||
repository_id=self.repo_slug,
|
||||
path=file,
|
||||
project=self.workspace_slug,
|
||||
version_descriptor=version,
|
||||
download=False,
|
||||
include_content=True,
|
||||
)
|
||||
original_file_content_str = original_file_content_str.content
|
||||
except Exception as error:
|
||||
get_logger().error(f"Failed to retrieve original file content of {file} at version {version}", error=error)
|
||||
original_file_content_str = ""
|
||||
else:
|
||||
try:
|
||||
original_file_content_str = self.azure_devops_client.get_item(
|
||||
repository_id=self.repo_slug,
|
||||
path=file,
|
||||
project=self.workspace_slug,
|
||||
version_descriptor=version,
|
||||
download=False,
|
||||
include_content=True,
|
||||
)
|
||||
original_file_content_str = original_file_content_str.content
|
||||
except Exception as error:
|
||||
get_logger().error(f"Failed to retrieve original file content of {file} at version {version}", error=error)
|
||||
original_file_content_str = ""
|
||||
|
||||
patch = load_large_diff(
|
||||
file, new_file_content_str, original_file_content_str, show_warning=False
|
||||
|
||||
@ -1,22 +1,22 @@
|
||||
import itertools
|
||||
import time
|
||||
import hashlib
|
||||
import traceback
|
||||
from datetime import datetime
|
||||
from typing import Optional, Tuple
|
||||
from urllib.parse import urlparse
|
||||
from github import AppAuthentication, Auth, Github
|
||||
|
||||
from github import AppAuthentication, Auth, Github, GithubException
|
||||
from retry import retry
|
||||
from starlette_context import context
|
||||
|
||||
from ..algo.file_filter import filter_ignored
|
||||
from ..algo.language_handler import is_valid_file
|
||||
from ..algo.types import EDIT_TYPE
|
||||
from ..algo.utils import PRReviewHeader, load_large_diff, clip_tokens, find_line_number_of_relevant_line_in_file, Range
|
||||
from ..config_loader import get_settings
|
||||
from ..log import get_logger
|
||||
from ..servers.utils import RateLimitExceeded
|
||||
from .git_provider import FilePatchInfo, GitProvider, IncrementalPR, MAX_FILES_ALLOWED_FULL
|
||||
from .git_provider import GitProvider, IncrementalPR, MAX_FILES_ALLOWED_FULL
|
||||
from pr_agent.algo.types import EDIT_TYPE, FilePatchInfo
|
||||
|
||||
|
||||
class GithubProvider(GitProvider):
|
||||
@ -27,8 +27,10 @@ class GithubProvider(GitProvider):
|
||||
except Exception:
|
||||
self.installation_id = None
|
||||
self.max_comment_chars = 65000
|
||||
self.base_url = get_settings().get("GITHUB.BASE_URL", "https://api.github.com").rstrip("/") # "https://api.github.com"
|
||||
self.base_url = get_settings().get("GITHUB.BASE_URL", "https://api.github.com").rstrip("/")
|
||||
self.base_url_html = self.base_url.split("api/")[0].rstrip("/") if "api/" in self.base_url else "https://github.com"
|
||||
self.base_domain = self.base_url.replace("https://", "").replace("http://", "")
|
||||
self.base_domain_html = self.base_url_html.replace("https://", "").replace("http://", "")
|
||||
self.github_client = self._get_github_client()
|
||||
self.repo = None
|
||||
self.pr_num = None
|
||||
@ -231,9 +233,8 @@ class GithubProvider(GitProvider):
|
||||
|
||||
return diff_files
|
||||
|
||||
except Exception as e:
|
||||
get_logger().error(f"Failing to get diff files: {e}",
|
||||
artifact={"traceback": traceback.format_exc()})
|
||||
except GithubException.RateLimitExceededException as e:
|
||||
get_logger().error(f"Rate limit exceeded for GitHub API. Original message: {e}")
|
||||
raise RateLimitExceeded("Rate limit exceeded for GitHub API.") from e
|
||||
|
||||
def publish_description(self, pr_title: str, pr_body: str):
|
||||
@ -255,7 +256,7 @@ class GithubProvider(GitProvider):
|
||||
def publish_comment(self, pr_comment: str, is_temporary: bool = False):
|
||||
if is_temporary and not get_settings().config.publish_output_progress:
|
||||
get_logger().debug(f"Skipping publish_comment for temporary comment: {pr_comment}")
|
||||
return None
|
||||
return
|
||||
pr_comment = self.limit_output_characters(pr_comment, self.max_comment_chars)
|
||||
response = self.pr.create_issue_comment(pr_comment)
|
||||
if hasattr(response, "user") and hasattr(response.user, "login"):
|
||||
@ -610,11 +611,8 @@ class GithubProvider(GitProvider):
|
||||
def _parse_pr_url(self, pr_url: str) -> Tuple[str, int]:
|
||||
parsed_url = urlparse(pr_url)
|
||||
|
||||
if parsed_url.path.startswith('/api/v3'):
|
||||
parsed_url = urlparse(pr_url.replace("/api/v3", ""))
|
||||
|
||||
path_parts = parsed_url.path.strip('/').split('/')
|
||||
if 'api.github.com' in parsed_url.netloc or '/api/v3' in pr_url:
|
||||
if self.base_domain in parsed_url.netloc:
|
||||
if len(path_parts) < 5 or path_parts[3] != 'pulls':
|
||||
raise ValueError("The provided URL does not appear to be a GitHub PR URL")
|
||||
repo_name = '/'.join(path_parts[1:3])
|
||||
@ -637,12 +635,8 @@ class GithubProvider(GitProvider):
|
||||
|
||||
def _parse_issue_url(self, issue_url: str) -> Tuple[str, int]:
|
||||
parsed_url = urlparse(issue_url)
|
||||
|
||||
if 'github.com' not in parsed_url.netloc:
|
||||
raise ValueError("The provided URL is not a valid GitHub URL")
|
||||
|
||||
path_parts = parsed_url.path.strip('/').split('/')
|
||||
if 'api.github.com' in parsed_url.netloc:
|
||||
if self.base_domain in parsed_url.netloc:
|
||||
if len(path_parts) < 5 or path_parts[3] != 'issues':
|
||||
raise ValueError("The provided URL does not appear to be a GitHub ISSUE URL")
|
||||
repo_name = '/'.join(path_parts[1:3])
|
||||
|
||||
@ -27,27 +27,18 @@ def apply_repo_settings(pr_url):
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
error_local = None
|
||||
if repo_settings:
|
||||
repo_settings_file = None
|
||||
category = 'local'
|
||||
try:
|
||||
fd, repo_settings_file = tempfile.mkstemp(suffix='.toml')
|
||||
os.write(fd, repo_settings)
|
||||
new_settings = Dynaconf(settings_files=[repo_settings_file])
|
||||
for section, contents in new_settings.as_dict().items():
|
||||
section_dict = copy.deepcopy(get_settings().as_dict().get(section, {}))
|
||||
for key, value in contents.items():
|
||||
section_dict[key] = value
|
||||
get_settings().unset(section)
|
||||
get_settings().set(section, section_dict, merge=False)
|
||||
get_logger().info(f"Applying repo settings:\n{new_settings.as_dict()}")
|
||||
except Exception as e:
|
||||
get_logger().warning(f"Failed to apply repo {category} settings, error: {str(e)}")
|
||||
error_local = {'error': str(e), 'settings': repo_settings, 'category': category}
|
||||
|
||||
if error_local:
|
||||
handle_configurations_errors([error_local], git_provider)
|
||||
fd, repo_settings_file = tempfile.mkstemp(suffix='.toml')
|
||||
os.write(fd, repo_settings)
|
||||
new_settings = Dynaconf(settings_files=[repo_settings_file])
|
||||
for section, contents in new_settings.as_dict().items():
|
||||
section_dict = copy.deepcopy(get_settings().as_dict().get(section, {}))
|
||||
for key, value in contents.items():
|
||||
section_dict[key] = value
|
||||
get_settings().unset(section)
|
||||
get_settings().set(section, section_dict, merge=False)
|
||||
get_logger().info(f"Applying repo settings:\n{new_settings.as_dict()}")
|
||||
except Exception as e:
|
||||
get_logger().exception("Failed to apply repo settings", e)
|
||||
finally:
|
||||
@ -58,40 +49,10 @@ def apply_repo_settings(pr_url):
|
||||
get_logger().error(f"Failed to remove temporary settings file {repo_settings_file}", e)
|
||||
|
||||
# enable switching models with a short definition
|
||||
if get_settings().config.model.lower() == 'claude-3-5-sonnet':
|
||||
if get_settings().config.model.lower()=='claude-3-5-sonnet':
|
||||
set_claude_model()
|
||||
|
||||
|
||||
def handle_configurations_errors(config_errors, git_provider):
|
||||
try:
|
||||
if not any(config_errors):
|
||||
return
|
||||
|
||||
for err in config_errors:
|
||||
if err:
|
||||
configuration_file_content = err['settings'].decode()
|
||||
err_message = err['error']
|
||||
config_type = err['category']
|
||||
header = f"❌ **PR-Agent failed to apply '{config_type}' repo settings**"
|
||||
body = f"{header}\n\nThe configuration file needs to be a valid [TOML](https://qodo-merge-docs.qodo.ai/usage-guide/configuration_options/), please fix it.\n\n"
|
||||
body += f"___\n\n**Error message:**\n`{err_message}`\n\n"
|
||||
if git_provider.is_supported("gfm_markdown"):
|
||||
body += f"\n\n<details><summary>Configuration content:</summary>\n\n```toml\n{configuration_file_content}\n```\n\n</details>"
|
||||
else:
|
||||
body += f"\n\n**Configuration content:**\n\n```toml\n{configuration_file_content}\n```\n\n"
|
||||
get_logger().warning(f"Sending a 'configuration error' comment to the PR", artifact={'body': body})
|
||||
# git_provider.publish_comment(body)
|
||||
if hasattr(git_provider, 'publish_persistent_comment'):
|
||||
git_provider.publish_persistent_comment(body,
|
||||
initial_header=header,
|
||||
update_header=False,
|
||||
final_update_message=False)
|
||||
else:
|
||||
git_provider.publish_comment(body)
|
||||
except Exception as e:
|
||||
get_logger().exception(f"Failed to handle configurations errors", e)
|
||||
|
||||
|
||||
def set_claude_model():
|
||||
"""
|
||||
set the claude-sonnet-3.5 model easily (even by users), just by stating: --config.model='claude-3-5-sonnet'
|
||||
|
||||
@ -16,9 +16,8 @@
|
||||
},
|
||||
"scopes": [
|
||||
"account",
|
||||
"repository:write",
|
||||
"pullrequest:write",
|
||||
"wiki"
|
||||
"repository",
|
||||
"pullrequest"
|
||||
],
|
||||
"contexts": [
|
||||
"account"
|
||||
@ -31,4 +30,4 @@
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -75,11 +75,8 @@ async def handle_manifest(request: Request, response: Response):
|
||||
return JSONResponse(manifest_obj)
|
||||
|
||||
|
||||
async def _perform_commands_bitbucket(commands_conf: str, agent: PRAgent, api_url: str, log_context: dict, data: dict):
|
||||
async def _perform_commands_bitbucket(commands_conf: str, agent: PRAgent, api_url: str, log_context: dict):
|
||||
apply_repo_settings(api_url)
|
||||
if data.get("event", "") == "pullrequest:created":
|
||||
if not should_process_pr_logic(data):
|
||||
return
|
||||
commands = get_settings().get(f"bitbucket_app.{commands_conf}", {})
|
||||
get_settings().set("config.is_auto_command", True)
|
||||
for command in commands:
|
||||
@ -196,7 +193,7 @@ async def handle_github_webhooks(background_tasks: BackgroundTasks, request: Req
|
||||
if get_identity_provider().verify_eligibility("bitbucket",
|
||||
sender_id, pr_url) is not Eligibility.NOT_ELIGIBLE:
|
||||
if get_settings().get("bitbucket_app.pr_commands"):
|
||||
await _perform_commands_bitbucket("pr_commands", PRAgent(), pr_url, log_context, data)
|
||||
await _perform_commands_bitbucket("pr_commands", PRAgent(), pr_url, log_context)
|
||||
elif event == "pullrequest:comment_created":
|
||||
pr_url = data["data"]["pullrequest"]["links"]["html"]["href"]
|
||||
log_context["api_url"] = pr_url
|
||||
|
||||
@ -249,7 +249,7 @@ def is_bot_user(sender, sender_type):
|
||||
return False
|
||||
|
||||
|
||||
def should_process_pr_logic(body) -> bool:
|
||||
def should_process_pr_logic(sender_type, sender, body) -> bool:
|
||||
try:
|
||||
pull_request = body.get("pull_request", {})
|
||||
title = pull_request.get("title", "")
|
||||
@ -306,10 +306,10 @@ async def handle_request(body: Dict[str, Any], event: str):
|
||||
log_context, sender, sender_id, sender_type = get_log_context(body, event, action, build_number)
|
||||
|
||||
# logic to ignore PRs opened by bot, PRs with specific titles, labels, source branches, or target branches
|
||||
if is_bot_user(sender, sender_type) and 'check_run' not in body:
|
||||
if is_bot_user(sender, sender_type):
|
||||
return {}
|
||||
if action != 'created' and 'check_run' not in body:
|
||||
if not should_process_pr_logic(body):
|
||||
if not should_process_pr_logic(sender_type, sender, body):
|
||||
return {}
|
||||
|
||||
if 'check_run' in body: # handle failed checks
|
||||
@ -373,8 +373,6 @@ def _check_pull_request_event(action: str, body: dict, log_context: dict) -> Tup
|
||||
async def _perform_auto_commands_github(commands_conf: str, agent: PRAgent, body: dict, api_url: str,
|
||||
log_context: dict):
|
||||
apply_repo_settings(api_url)
|
||||
if not should_process_pr_logic(body): # Here we already updated the configuration with the repo settings
|
||||
return {}
|
||||
commands = get_settings().get(f"github_app.{commands_conf}")
|
||||
if not commands:
|
||||
get_logger().info(f"New PR, but no auto commands configured")
|
||||
|
||||
@ -59,10 +59,8 @@ async def handle_request(api_url: str, body: str, log_context: dict, sender_id:
|
||||
|
||||
|
||||
async def _perform_commands_gitlab(commands_conf: str, agent: PRAgent, api_url: str,
|
||||
log_context: dict, data: dict):
|
||||
log_context: dict):
|
||||
apply_repo_settings(api_url)
|
||||
if not should_process_pr_logic(data): # Here we already updated the configurations
|
||||
return
|
||||
commands = get_settings().get(f"gitlab.{commands_conf}", {})
|
||||
get_settings().set("config.is_auto_command", True)
|
||||
for command in commands:
|
||||
@ -92,12 +90,8 @@ def is_bot_user(data) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def should_process_pr_logic(data) -> bool:
|
||||
def should_process_pr_logic(data, title) -> bool:
|
||||
try:
|
||||
if not data.get('object_attributes', {}):
|
||||
return False
|
||||
title = data['object_attributes'].get('title')
|
||||
|
||||
# logic to ignore MRs for titles, labels and source, target branches.
|
||||
ignore_mr_title = get_settings().get("CONFIG.IGNORE_PR_TITLE", [])
|
||||
ignore_mr_labels = get_settings().get("CONFIG.IGNORE_PR_LABELS", [])
|
||||
@ -139,7 +133,6 @@ def should_process_pr_logic(data) -> bool:
|
||||
async def gitlab_webhook(background_tasks: BackgroundTasks, request: Request):
|
||||
start_time = datetime.now()
|
||||
request_json = await request.json()
|
||||
context["settings"] = copy.deepcopy(global_settings)
|
||||
|
||||
async def inner(data: dict):
|
||||
log_context = {"server_type": "gitlab_app"}
|
||||
@ -155,6 +148,7 @@ async def gitlab_webhook(background_tasks: BackgroundTasks, request: Request):
|
||||
secret_dict = json.loads(secret)
|
||||
gitlab_token = secret_dict["gitlab_token"]
|
||||
log_context["token_id"] = secret_dict.get("token_name", secret_dict.get("id", "unknown"))
|
||||
context["settings"] = copy.deepcopy(global_settings)
|
||||
context["settings"].gitlab.personal_access_token = gitlab_token
|
||||
except Exception as e:
|
||||
get_logger().error(f"Failed to validate secret {request_token}: {e}")
|
||||
@ -179,9 +173,9 @@ async def gitlab_webhook(background_tasks: BackgroundTasks, request: Request):
|
||||
# ignore bot users
|
||||
if is_bot_user(data):
|
||||
return JSONResponse(status_code=status.HTTP_200_OK, content=jsonable_encoder({"message": "success"}))
|
||||
if data.get('event_type') != 'note': # not a comment
|
||||
if data.get('event_type') != 'note' and data.get('object_attributes', {}): # not a comment
|
||||
# ignore MRs based on title, labels, source and target branches
|
||||
if not should_process_pr_logic(data):
|
||||
if not should_process_pr_logic(data, data['object_attributes'].get('title')):
|
||||
return JSONResponse(status_code=status.HTTP_200_OK, content=jsonable_encoder({"message": "success"}))
|
||||
|
||||
log_context["sender"] = sender
|
||||
@ -194,7 +188,7 @@ async def gitlab_webhook(background_tasks: BackgroundTasks, request: Request):
|
||||
get_logger().info(f"Skipping draft MR: {url}")
|
||||
return JSONResponse(status_code=status.HTTP_200_OK, content=jsonable_encoder({"message": "success"}))
|
||||
|
||||
await _perform_commands_gitlab("pr_commands", PRAgent(), url, log_context, data)
|
||||
await _perform_commands_gitlab("pr_commands", PRAgent(), url, log_context)
|
||||
elif data.get('object_kind') == 'note' and data.get('event_type') == 'note': # comment on MR
|
||||
if 'merge_request' in data:
|
||||
mr = data['merge_request']
|
||||
@ -226,7 +220,7 @@ async def gitlab_webhook(background_tasks: BackgroundTasks, request: Request):
|
||||
content=jsonable_encoder({"message": "success"}))
|
||||
|
||||
get_logger().debug(f'A push event has been received: {url}')
|
||||
await _perform_commands_gitlab("push_commands", PRAgent(), url, log_context, data)
|
||||
await _perform_commands_gitlab("push_commands", PRAgent(), url, log_context)
|
||||
except Exception as e:
|
||||
get_logger().error(f"Failed to handle push event: {e}")
|
||||
|
||||
|
||||
@ -51,7 +51,9 @@ require_tests_review=true
|
||||
require_estimate_effort_to_review=true
|
||||
require_can_be_split_review=false
|
||||
require_security_review=true
|
||||
require_ticket_analysis_review=true
|
||||
# soc2
|
||||
require_soc2_ticket=false
|
||||
soc2_ticket_prompt="Does the PR description include a link to ticket in a project management system (e.g., Jira, Asana, Trello, etc.) ?"
|
||||
# general options
|
||||
num_code_suggestions=0
|
||||
inline_code_comments = false
|
||||
@ -75,7 +77,7 @@ maximal_review_effort=5
|
||||
|
||||
|
||||
[pr_description] # /describe #
|
||||
publish_labels=false
|
||||
publish_labels=true
|
||||
add_original_user_description=true
|
||||
generate_ai_title=false
|
||||
use_bullet_points=true
|
||||
@ -134,10 +136,8 @@ final_clip_factor = 0.8
|
||||
demand_code_suggestions_self_review=false # add a checkbox for the author to self-review the code suggestions
|
||||
code_suggestions_self_review_text= "**Author self-review**: I have reviewed the PR code suggestions, and addressed the relevant ones."
|
||||
approve_pr_on_self_review=false # Pro feature. if true, the PR will be auto-approved after the author clicks on the self-review checkbox
|
||||
fold_suggestions_on_self_review=true # Pro feature. if true, the code suggestions will be folded after the author clicks on the self-review checkbox
|
||||
# Suggestion impact 💎
|
||||
# Suggestion impact
|
||||
publish_post_process_suggestion_impact=true
|
||||
wiki_page_accepted_suggestions=true
|
||||
|
||||
[pr_custom_prompt] # /custom_prompt #
|
||||
prompt = """\
|
||||
|
||||
@ -26,15 +26,16 @@ __old hunk__
|
||||
|
||||
@@ ... @@ def func2():
|
||||
__new hunk__
|
||||
unchanged code line4
|
||||
+new code line5 removed in the PR
|
||||
unchanged code line6
|
||||
...
|
||||
__old hunk__
|
||||
...
|
||||
|
||||
|
||||
## File: 'src/file2.py'
|
||||
...
|
||||
======
|
||||
|
||||
- In the format above, the diff is organized into separate '__new hunk__' and '__old hunk__' sections for each code chunk. '__new hunk__' contains the updated code, while '__old hunk__' shows the removed code. If no code was removed in a specific chunk, the __old hunk__ section will be omitted.
|
||||
- In the format above, the diff is organized into separate '__new hunk__' and '__old hunk__' sections for each code chunk. '__new hunk__' contains the updated code, while '__old hunk__' shows the removed code. If no code was added or removed in a specific chunk, the corresponding section will be omitted.
|
||||
- Line numbers were added for the '__new hunk__' sections to help referencing specific lines in the code suggestions. These numbers are for reference only and are not part of the actual code.
|
||||
- Code lines are prefixed with symbols: '+' for new code added in the PR, '-' for code removed, and ' ' for unchanged code.
|
||||
{%- if is_ai_metadata %}
|
||||
@ -46,9 +47,9 @@ Specific guidelines for generating code suggestions:
|
||||
- Provide up to {{ num_code_suggestions }} distinct and insightful code suggestions.
|
||||
- Focus solely on enhancing new code introduced in the PR, identified by '+' prefixes in '__new hunk__' sections (after the line numbers).
|
||||
- Prioritize suggestions that address potential issues, critical problems, and bugs in the PR code. Avoid repeating changes already implemented in the PR. If no pertinent suggestions are applicable, return an empty list.
|
||||
- Don't suggest to add docstring, type hints, or comments, to remove unused imports, or to use more specific exception types.
|
||||
- Avoid proposing additions of docstrings, type hints, or comments, or the removal of unused imports.
|
||||
- When referencing variables or names from the code, enclose them in backticks (`). Example: "ensure that `variable_name` is..."
|
||||
- Be mindful you are viewing a partial PR code diff, not the full codebase. Avoid suggestions that might conflict with unseen code or alerting variables not declared in the visible scope, as the context is incomplete.
|
||||
- Be mindful you are viewing a partial PR code diff, not the full codebase. Avoid suggestions that might conflict with unseen code or alerting on variables not declared in the visible scope, as the context is incomplete.
|
||||
|
||||
|
||||
{%- if extra_instructions %}
|
||||
|
||||
@ -29,14 +29,6 @@ Key guidelines for evaluation:
|
||||
- Avoid inflating scores for suggestions that, while correct, offer only marginal improvements or optimizations.
|
||||
- Maintain the original order of suggestions in your feedback, corresponding to their input sequence.
|
||||
|
||||
Additional scoring considerations:
|
||||
- If the suggestion is not actionable, and only asks the user to verify or ensure a change, reduce its score by 1-2 points.
|
||||
- Assign a score of 0 to suggestions aiming at:
|
||||
- Adding docstring, type hints, or comments
|
||||
- Remove unused imports or variables
|
||||
- Using more specific exception types.
|
||||
|
||||
|
||||
|
||||
The PR code diff will be presented in the following structured format:
|
||||
======
|
||||
|
||||
@ -78,9 +78,9 @@ pr_files:
|
||||
...
|
||||
...
|
||||
{%- endif %}
|
||||
description: |
|
||||
description: |-
|
||||
...
|
||||
title: |
|
||||
title: |-
|
||||
...
|
||||
{%- if enable_custom_labels %}
|
||||
labels:
|
||||
@ -94,26 +94,7 @@ labels:
|
||||
Answer should be a valid YAML, and nothing else. Each YAML output MUST be after a newline, with proper indent, and block scalar indicator ('|')
|
||||
"""
|
||||
|
||||
user="""
|
||||
{%- if related_tickets %}
|
||||
Related Ticket Info:
|
||||
{% for ticket in related_tickets %}
|
||||
=====
|
||||
Ticket Title: '{{ ticket.title }}'
|
||||
{%- if ticket.labels %}
|
||||
Ticket Labels: {{ ticket.labels }}
|
||||
{%- endif %}
|
||||
{%- if ticket.body %}
|
||||
Ticket Description:
|
||||
#####
|
||||
{{ ticket.body }}
|
||||
#####
|
||||
{%- endif %}
|
||||
=====
|
||||
{% endfor %}
|
||||
{%- endif %}
|
||||
|
||||
PR Info:
|
||||
user="""PR Info:
|
||||
|
||||
Previous title: '{{title}}'
|
||||
|
||||
|
||||
@ -1,24 +1,20 @@
|
||||
[pr_help_prompts]
|
||||
system="""You are Doc-helper, a language models designed to answer questions about a documentation website for an open-soure project called "PR-Agent" (recently renamed to "Qodo Merge").
|
||||
You will recieve a question, and the full documentation website content.
|
||||
Your goal is to provide the best answer to the question using the documentation provided.
|
||||
system="""You are Doc-helper, a language models designed to answer questions about a documentation website for an open-soure project called "PR-Agent".
|
||||
You will recieve a question, and a list of snippets that were collected for a documentation site using RAG as the retrieval method.
|
||||
Your goal is to provide the best answer to the question using the snippets provided.
|
||||
|
||||
Additional instructions:
|
||||
- Try to be short and concise in your answers. Try to give examples if needed.
|
||||
- The main tools of PR-Agent are 'describe', 'review', 'improve'. If there is ambiguity to which tool the user is referring to, prioritize snippets of these tools over others.
|
||||
- If the question has ambiguity and can relate to different tools or platfroms, provide the best answer possible based on what is available, but also state in your answer what additional information would be needed to give a more accurate answer.
|
||||
- Try to be short and concise in your answers. Give examples if needed.
|
||||
- It is possible some of the snippets may not be relevant to the question. In that case, you should ignore them and focus on the ones that are relevant.
|
||||
- The main tools of pr-agent are 'describe', 'review', 'improve'. If there is ambiguity to which tool the user is referring to, prioritize snippets of these tools over others.
|
||||
|
||||
|
||||
The output must be a YAML object equivalent to type $DocHelper, according to the following Pydantic definitions:
|
||||
=====
|
||||
class relevant_section(BaseModel):
|
||||
file_name: str = Field(description="The name of the relevant file")
|
||||
relevant_section_header_string: str = Field(description="From the relevant file, exact text of the relevant section heading. If no markdown heading is relevant, return empty string")
|
||||
|
||||
class DocHelper(BaseModel):
|
||||
user_question: str = Field(description="The user's question")
|
||||
response: str = Field(description="The response to the user's question")
|
||||
relevant_sections: List[relevant_section] = Field(description="A list of the relevant markdown sections in the documentation that answer the user's question, ordered by importance (most relevant first)")
|
||||
relevant_snippets: List[int] = Field(description="One-based index of the relevant snippets in the list of snippets provided. Order the by relevance, with the most relevant first. If a snippet was not relevant, do not include it in the list.")
|
||||
=====
|
||||
|
||||
|
||||
@ -27,12 +23,11 @@ Example output:
|
||||
user_question: |
|
||||
...
|
||||
response: |
|
||||
...
|
||||
relevant_sections:
|
||||
- file_name: "src/file1.py"
|
||||
relevant_section_header_string: |
|
||||
...
|
||||
- ...
|
||||
relevant_snippets:
|
||||
- 1
|
||||
- 2
|
||||
- 4
|
||||
"""
|
||||
|
||||
user="""\
|
||||
@ -42,7 +37,7 @@ User's Question:
|
||||
=====
|
||||
|
||||
|
||||
Documentation website content:
|
||||
Relevant doc snippets retrieved:
|
||||
=====
|
||||
{{ snippets|trim }}
|
||||
=====
|
||||
|
||||
@ -32,15 +32,16 @@ __old hunk__
|
||||
|
||||
@@ ... @@ def func2():
|
||||
__new hunk__
|
||||
unchanged code line4
|
||||
+new code line5 removed in the PR
|
||||
unchanged code line6
|
||||
...
|
||||
__old hunk__
|
||||
...
|
||||
|
||||
|
||||
## File: 'src/file2.py'
|
||||
...
|
||||
======
|
||||
|
||||
- In the format above, the diff is organized into separate '__new hunk__' and '__old hunk__' sections for each code chunk. '__new hunk__' contains the updated code, while '__old hunk__' shows the removed code. If no code was removed in a specific chunk, the __old hunk__ section will be omitted.
|
||||
- In this format, we separated each hunk of diff code to '__new hunk__' and '__old hunk__' sections. The '__new hunk__' section contains the new code of the chunk, and the '__old hunk__' section contains the old code, that was removed. If no new code was added in a specific hunk, '__new hunk__' section will not be presented. If no code was removed, '__old hunk__' section will not be presented.
|
||||
- We also added line numbers for the '__new hunk__' code, to help you refer to the code lines in your suggestions. These line numbers are not part of the actual code, and should only used for reference.
|
||||
- Code lines are prefixed with symbols ('+', '-', ' '). The '+' symbol indicates new code added in the PR, the '-' symbol indicates code removed in the PR, and the ' ' symbol indicates unchanged code. \
|
||||
The review should address new code added in the PR code diff (lines starting with '+')
|
||||
@ -85,20 +86,7 @@ class KeyIssuesComponentLink(BaseModel):
|
||||
start_line: int = Field(description="The start line that corresponds to this issue in the relevant file")
|
||||
end_line: int = Field(description="The end line that corresponds to this issue in the relevant file")
|
||||
|
||||
{%- if related_tickets %}
|
||||
|
||||
class TicketCompliance(BaseModel):
|
||||
ticket_url: str = Field(description="Ticket URL or ID")
|
||||
ticket_requirements: str = Field(description="Repeat, in your own words, all ticket requirements, in bullet points")
|
||||
fully_compliant_requirements: str = Field(description="A list, in bullet points, of which requirements are met by the PR code. Don't explain how the requirements are met, just list them shortly. Can be empty")
|
||||
not_compliant_requirements: str = Field(description="A list, in bullet points, of which requirements are not met by the PR code. Don't explain how the requirements are not met, just list them shortly. Can be empty")
|
||||
overall_compliance_level: str = Field(description="Overall give this PR one of these three values in relation to the ticket: 'Fully compliant', 'Partially compliant', or 'Not compliant'")
|
||||
{%- endif %}
|
||||
|
||||
class Review(BaseModel):
|
||||
{%- if related_tickets %}
|
||||
ticket_compliance_check: List[TicketCompliance] = Field(description="A list of compliance checks for the related tickets")
|
||||
{%- endif %}
|
||||
{%- if require_estimate_effort_to_review %}
|
||||
estimated_effort_to_review_[1-5]: int = Field(description="Estimate, on a scale of 1-5 (inclusive), the time and effort required to review this PR by an experienced and knowledgeable developer. 1 means short and easy review , 5 means long and hard review. Take into account the size, complexity, quality, and the needed changes of the PR code diff.")
|
||||
{%- endif %}
|
||||
@ -143,19 +131,6 @@ class PRReview(BaseModel):
|
||||
Example output:
|
||||
```yaml
|
||||
review:
|
||||
{%- if related_tickets %}
|
||||
ticket_compliance_check:
|
||||
- ticket_url: |
|
||||
...
|
||||
ticket_requirements: |
|
||||
...
|
||||
fully_compliant_requirements: |
|
||||
...
|
||||
not_compliant_requirements: |
|
||||
...
|
||||
overall_compliance_level: |
|
||||
...
|
||||
{%- endif %}
|
||||
{%- if require_estimate_effort_to_review %}
|
||||
estimated_effort_to_review_[1-5]: |
|
||||
3
|
||||
@ -202,33 +177,7 @@ code_feedback:
|
||||
Answer should be a valid YAML, and nothing else. Each YAML output MUST be after a newline, with proper indent, and block scalar indicator ('|')
|
||||
"""
|
||||
|
||||
user="""
|
||||
{%- if related_tickets %}
|
||||
--PR Ticket Info--
|
||||
{%- for ticket in related_tickets %}
|
||||
=====
|
||||
Ticket URL: '{{ ticket.ticket_url }}'
|
||||
|
||||
Ticket Title: '{{ ticket.title }}'
|
||||
|
||||
{%- if ticket.labels %}
|
||||
|
||||
Ticket Labels: {{ ticket.labels }}
|
||||
|
||||
{%- endif %}
|
||||
{%- if ticket.body %}
|
||||
|
||||
Ticket Description:
|
||||
#####
|
||||
{{ ticket.body }}
|
||||
#####
|
||||
{%- endif %}
|
||||
=====
|
||||
{% endfor %}
|
||||
{%- endif %}
|
||||
|
||||
|
||||
--PR Info--
|
||||
user="""--PR Info--
|
||||
|
||||
Title: '{{title}}'
|
||||
|
||||
|
||||
5
pr_agent/static_analysis/queries/Credits.md
Normal file
5
pr_agent/static_analysis/queries/Credits.md
Normal file
@ -0,0 +1,5 @@
|
||||
|
||||
# Credits
|
||||
|
||||
These queries, and some of the logic, were adopted from the excellent [Aider](https://github.com/paul-gauthier/aider/blob/main/aider/queries/README.md) project.
|
||||
|
||||
9
pr_agent/static_analysis/queries/tree-sitter-c-tags.scm
Normal file
9
pr_agent/static_analysis/queries/tree-sitter-c-tags.scm
Normal file
@ -0,0 +1,9 @@
|
||||
(struct_specifier name: (type_identifier) @name.definition.class body:(_)) @definition.class
|
||||
|
||||
(declaration type: (union_specifier name: (type_identifier) @name.definition.class)) @definition.class
|
||||
|
||||
(function_declarator declarator: (identifier) @name.definition.function) @definition.function
|
||||
|
||||
(type_definition declarator: (type_identifier) @name.definition.type) @definition.type
|
||||
|
||||
(enum_specifier name: (type_identifier) @name.definition.type) @definition.type
|
||||
@ -0,0 +1,46 @@
|
||||
(class_declaration
|
||||
name: (identifier) @name.definition.class
|
||||
) @definition.class
|
||||
|
||||
(class_declaration
|
||||
bases: (base_list (_) @name.reference.class)
|
||||
) @reference.class
|
||||
|
||||
(interface_declaration
|
||||
name: (identifier) @name.definition.interface
|
||||
) @definition.interface
|
||||
|
||||
(interface_declaration
|
||||
bases: (base_list (_) @name.reference.interface)
|
||||
) @reference.interface
|
||||
|
||||
(method_declaration
|
||||
name: (identifier) @name.definition.method
|
||||
) @definition.method
|
||||
|
||||
(object_creation_expression
|
||||
type: (identifier) @name.reference.class
|
||||
) @reference.class
|
||||
|
||||
(type_parameter_constraints_clause
|
||||
target: (identifier) @name.reference.class
|
||||
) @reference.class
|
||||
|
||||
(type_constraint
|
||||
type: (identifier) @name.reference.class
|
||||
) @reference.class
|
||||
|
||||
(variable_declaration
|
||||
type: (identifier) @name.reference.class
|
||||
) @reference.class
|
||||
|
||||
(invocation_expression
|
||||
function:
|
||||
(member_access_expression
|
||||
name: (identifier) @name.reference.send
|
||||
)
|
||||
) @reference.send
|
||||
|
||||
(namespace_declaration
|
||||
name: (identifier) @name.definition.module
|
||||
) @definition.module
|
||||
15
pr_agent/static_analysis/queries/tree-sitter-cpp-tags.scm
Normal file
15
pr_agent/static_analysis/queries/tree-sitter-cpp-tags.scm
Normal file
@ -0,0 +1,15 @@
|
||||
(struct_specifier name: (type_identifier) @name.definition.class body:(_)) @definition.class
|
||||
|
||||
(declaration type: (union_specifier name: (type_identifier) @name.definition.class)) @definition.class
|
||||
|
||||
(function_declarator declarator: (identifier) @name.definition.function) @definition.function
|
||||
|
||||
(function_declarator declarator: (field_identifier) @name.definition.function) @definition.function
|
||||
|
||||
(function_declarator declarator: (qualified_identifier scope: (namespace_identifier) @scope name: (identifier) @name.definition.method)) @definition.method
|
||||
|
||||
(type_definition declarator: (type_identifier) @name.definition.type) @definition.type
|
||||
|
||||
(enum_specifier name: (type_identifier) @name.definition.type) @definition.type
|
||||
|
||||
(class_specifier name: (type_identifier) @name.definition.class) @definition.class
|
||||
@ -0,0 +1,8 @@
|
||||
;; defun/defsubst
|
||||
(function_definition name: (symbol) @name.definition.function) @definition.function
|
||||
|
||||
;; Treat macros as function definitions for the sake of TAGS.
|
||||
(macro_definition name: (symbol) @name.definition.function) @definition.function
|
||||
|
||||
;; Match function calls
|
||||
(list (symbol) @name.reference.function) @reference.function
|
||||
54
pr_agent/static_analysis/queries/tree-sitter-elixir-tags.scm
Normal file
54
pr_agent/static_analysis/queries/tree-sitter-elixir-tags.scm
Normal file
@ -0,0 +1,54 @@
|
||||
; Definitions
|
||||
|
||||
; * modules and protocols
|
||||
(call
|
||||
target: (identifier) @ignore
|
||||
(arguments (alias) @name.definition.module)
|
||||
(#match? @ignore "^(defmodule|defprotocol)$")) @definition.module
|
||||
|
||||
; * functions/macros
|
||||
(call
|
||||
target: (identifier) @ignore
|
||||
(arguments
|
||||
[
|
||||
; zero-arity functions with no parentheses
|
||||
(identifier) @name.definition.function
|
||||
; regular function clause
|
||||
(call target: (identifier) @name.definition.function)
|
||||
; function clause with a guard clause
|
||||
(binary_operator
|
||||
left: (call target: (identifier) @name.definition.function)
|
||||
operator: "when")
|
||||
])
|
||||
(#match? @ignore "^(def|defp|defdelegate|defguard|defguardp|defmacro|defmacrop|defn|defnp)$")) @definition.function
|
||||
|
||||
; References
|
||||
|
||||
; ignore calls to kernel/special-forms keywords
|
||||
(call
|
||||
target: (identifier) @ignore
|
||||
(#match? @ignore "^(def|defp|defdelegate|defguard|defguardp|defmacro|defmacrop|defn|defnp|defmodule|defprotocol|defimpl|defstruct|defexception|defoverridable|alias|case|cond|else|for|if|import|quote|raise|receive|require|reraise|super|throw|try|unless|unquote|unquote_splicing|use|with)$"))
|
||||
|
||||
; ignore module attributes
|
||||
(unary_operator
|
||||
operator: "@"
|
||||
operand: (call
|
||||
target: (identifier) @ignore))
|
||||
|
||||
; * function call
|
||||
(call
|
||||
target: [
|
||||
; local
|
||||
(identifier) @name.reference.call
|
||||
; remote
|
||||
(dot
|
||||
right: (identifier) @name.reference.call)
|
||||
]) @reference.call
|
||||
|
||||
; * pipe into function call
|
||||
(binary_operator
|
||||
operator: "|>"
|
||||
right: (identifier) @name.reference.call) @reference.call
|
||||
|
||||
; * modules
|
||||
(alias) @name.reference.module @reference.module
|
||||
19
pr_agent/static_analysis/queries/tree-sitter-elm-tags.scm
Normal file
19
pr_agent/static_analysis/queries/tree-sitter-elm-tags.scm
Normal file
@ -0,0 +1,19 @@
|
||||
(value_declaration (function_declaration_left (lower_case_identifier) @name.definition.function)) @definition.function
|
||||
|
||||
(function_call_expr (value_expr (value_qid) @name.reference.function)) @reference.function
|
||||
(exposed_value (lower_case_identifier) @name.reference.function) @reference.function
|
||||
(type_annotation ((lower_case_identifier) @name.reference.function) (colon)) @reference.function
|
||||
|
||||
(type_declaration ((upper_case_identifier) @name.definition.type)) @definition.type
|
||||
|
||||
(type_ref (upper_case_qid (upper_case_identifier) @name.reference.type)) @reference.type
|
||||
(exposed_type (upper_case_identifier) @name.reference.type) @reference.type
|
||||
|
||||
(type_declaration (union_variant (upper_case_identifier) @name.definition.union)) @definition.union
|
||||
|
||||
(value_expr (upper_case_qid (upper_case_identifier) @name.reference.union)) @reference.union
|
||||
|
||||
|
||||
(module_declaration
|
||||
(upper_case_qid (upper_case_identifier)) @name.definition.module
|
||||
) @definition.module
|
||||
30
pr_agent/static_analysis/queries/tree-sitter-go-tags.scm
Normal file
30
pr_agent/static_analysis/queries/tree-sitter-go-tags.scm
Normal file
@ -0,0 +1,30 @@
|
||||
(
|
||||
(comment)* @doc
|
||||
.
|
||||
(function_declaration
|
||||
name: (identifier) @name.definition.function) @definition.function
|
||||
(#strip! @doc "^//\\s*")
|
||||
(#set-adjacent! @doc @definition.function)
|
||||
)
|
||||
|
||||
(
|
||||
(comment)* @doc
|
||||
.
|
||||
(method_declaration
|
||||
name: (field_identifier) @name.definition.method) @definition.method
|
||||
(#strip! @doc "^//\\s*")
|
||||
(#set-adjacent! @doc @definition.method)
|
||||
)
|
||||
|
||||
(call_expression
|
||||
function: [
|
||||
(identifier) @name.reference.call
|
||||
(parenthesized_expression (identifier) @name.reference.call)
|
||||
(selector_expression field: (field_identifier) @name.reference.call)
|
||||
(parenthesized_expression (selector_expression field: (field_identifier) @name.reference.call))
|
||||
]) @reference.call
|
||||
|
||||
(type_spec
|
||||
name: (type_identifier) @name.definition.type) @definition.type
|
||||
|
||||
(type_identifier) @name.reference.type @reference.type
|
||||
20
pr_agent/static_analysis/queries/tree-sitter-java-tags.scm
Normal file
20
pr_agent/static_analysis/queries/tree-sitter-java-tags.scm
Normal file
@ -0,0 +1,20 @@
|
||||
(class_declaration
|
||||
name: (identifier) @name.definition.class) @definition.class
|
||||
|
||||
(method_declaration
|
||||
name: (identifier) @name.definition.method) @definition.method
|
||||
|
||||
(method_invocation
|
||||
name: (identifier) @name.reference.call
|
||||
arguments: (argument_list) @reference.call)
|
||||
|
||||
(interface_declaration
|
||||
name: (identifier) @name.definition.interface) @definition.interface
|
||||
|
||||
(type_list
|
||||
(type_identifier) @name.reference.implementation) @reference.implementation
|
||||
|
||||
(object_creation_expression
|
||||
type: (type_identifier) @name.reference.class) @reference.class
|
||||
|
||||
(superclass (type_identifier) @name.reference.class) @reference.class
|
||||
@ -0,0 +1,88 @@
|
||||
(
|
||||
(comment)* @doc
|
||||
.
|
||||
(method_definition
|
||||
name: (property_identifier) @name.definition.method) @definition.method
|
||||
(#not-eq? @name.definition.method "constructor")
|
||||
(#strip! @doc "^[\\s\\*/]+|^[\\s\\*/]$")
|
||||
(#select-adjacent! @doc @definition.method)
|
||||
)
|
||||
|
||||
(
|
||||
(comment)* @doc
|
||||
.
|
||||
[
|
||||
(class
|
||||
name: (_) @name.definition.class)
|
||||
(class_declaration
|
||||
name: (_) @name.definition.class)
|
||||
] @definition.class
|
||||
(#strip! @doc "^[\\s\\*/]+|^[\\s\\*/]$")
|
||||
(#select-adjacent! @doc @definition.class)
|
||||
)
|
||||
|
||||
(
|
||||
(comment)* @doc
|
||||
.
|
||||
[
|
||||
(function
|
||||
name: (identifier) @name.definition.function)
|
||||
(function_declaration
|
||||
name: (identifier) @name.definition.function)
|
||||
(generator_function
|
||||
name: (identifier) @name.definition.function)
|
||||
(generator_function_declaration
|
||||
name: (identifier) @name.definition.function)
|
||||
] @definition.function
|
||||
(#strip! @doc "^[\\s\\*/]+|^[\\s\\*/]$")
|
||||
(#select-adjacent! @doc @definition.function)
|
||||
)
|
||||
|
||||
(
|
||||
(comment)* @doc
|
||||
.
|
||||
(lexical_declaration
|
||||
(variable_declarator
|
||||
name: (identifier) @name.definition.function
|
||||
value: [(arrow_function) (function)]) @definition.function)
|
||||
(#strip! @doc "^[\\s\\*/]+|^[\\s\\*/]$")
|
||||
(#select-adjacent! @doc @definition.function)
|
||||
)
|
||||
|
||||
(
|
||||
(comment)* @doc
|
||||
.
|
||||
(variable_declaration
|
||||
(variable_declarator
|
||||
name: (identifier) @name.definition.function
|
||||
value: [(arrow_function) (function)]) @definition.function)
|
||||
(#strip! @doc "^[\\s\\*/]+|^[\\s\\*/]$")
|
||||
(#select-adjacent! @doc @definition.function)
|
||||
)
|
||||
|
||||
(assignment_expression
|
||||
left: [
|
||||
(identifier) @name.definition.function
|
||||
(member_expression
|
||||
property: (property_identifier) @name.definition.function)
|
||||
]
|
||||
right: [(arrow_function) (function)]
|
||||
) @definition.function
|
||||
|
||||
(pair
|
||||
key: (property_identifier) @name.definition.function
|
||||
value: [(arrow_function) (function)]) @definition.function
|
||||
|
||||
(
|
||||
(call_expression
|
||||
function: (identifier) @name.reference.call) @reference.call
|
||||
(#not-match? @name.reference.call "^(require)$")
|
||||
)
|
||||
|
||||
(call_expression
|
||||
function: (member_expression
|
||||
property: (property_identifier) @name.reference.call)
|
||||
arguments: (_) @reference.call)
|
||||
|
||||
(new_expression
|
||||
constructor: (_) @name.reference.class) @reference.class
|
||||
115
pr_agent/static_analysis/queries/tree-sitter-ocaml-tags.scm
Normal file
115
pr_agent/static_analysis/queries/tree-sitter-ocaml-tags.scm
Normal file
@ -0,0 +1,115 @@
|
||||
; Modules
|
||||
;--------
|
||||
|
||||
(
|
||||
(comment)? @doc .
|
||||
(module_definition (module_binding (module_name) @name.definition.module) @definition.module)
|
||||
(#strip! @doc "^\\(\\*\\*?\\s*|\\s\\*\\)$")
|
||||
)
|
||||
|
||||
(module_path (module_name) @name.reference.module) @reference.module
|
||||
|
||||
; Module types
|
||||
;--------------
|
||||
|
||||
(
|
||||
(comment)? @doc .
|
||||
(module_type_definition (module_type_name) @name.definition.interface) @definition.interface
|
||||
(#strip! @doc "^\\(\\*\\*?\\s*|\\s\\*\\)$")
|
||||
)
|
||||
|
||||
(module_type_path (module_type_name) @name.reference.implementation) @reference.implementation
|
||||
|
||||
; Functions
|
||||
;----------
|
||||
|
||||
(
|
||||
(comment)? @doc .
|
||||
(value_definition
|
||||
[
|
||||
(let_binding
|
||||
pattern: (value_name) @name.definition.function
|
||||
(parameter))
|
||||
(let_binding
|
||||
pattern: (value_name) @name.definition.function
|
||||
body: [(fun_expression) (function_expression)])
|
||||
] @definition.function
|
||||
)
|
||||
(#strip! @doc "^\\(\\*\\*?\\s*|\\s\\*\\)$")
|
||||
)
|
||||
|
||||
(
|
||||
(comment)? @doc .
|
||||
(external (value_name) @name.definition.function) @definition.function
|
||||
(#strip! @doc "^\\(\\*\\*?\\s*|\\s\\*\\)$")
|
||||
)
|
||||
|
||||
(application_expression
|
||||
function: (value_path (value_name) @name.reference.call)) @reference.call
|
||||
|
||||
(infix_expression
|
||||
left: (value_path (value_name) @name.reference.call)
|
||||
operator: (concat_operator) @reference.call
|
||||
(#eq? @reference.call "@@"))
|
||||
|
||||
(infix_expression
|
||||
operator: (rel_operator) @reference.call
|
||||
right: (value_path (value_name) @name.reference.call)
|
||||
(#eq? @reference.call "|>"))
|
||||
|
||||
; Operator
|
||||
;---------
|
||||
|
||||
(
|
||||
(comment)? @doc .
|
||||
(value_definition
|
||||
(let_binding
|
||||
pattern: (parenthesized_operator (_) @name.definition.function)) @definition.function)
|
||||
(#strip! @doc "^\\(\\*\\*?\\s*|\\s\\*\\)$")
|
||||
)
|
||||
|
||||
[
|
||||
(prefix_operator)
|
||||
(sign_operator)
|
||||
(pow_operator)
|
||||
(mult_operator)
|
||||
(add_operator)
|
||||
(concat_operator)
|
||||
(rel_operator)
|
||||
(and_operator)
|
||||
(or_operator)
|
||||
(assign_operator)
|
||||
(hash_operator)
|
||||
(indexing_operator)
|
||||
(let_operator)
|
||||
(let_and_operator)
|
||||
(match_operator)
|
||||
] @name.reference.call @reference.call
|
||||
|
||||
; Classes
|
||||
;--------
|
||||
|
||||
(
|
||||
(comment)? @doc .
|
||||
[
|
||||
(class_definition (class_binding (class_name) @name.definition.class) @definition.class)
|
||||
(class_type_definition (class_type_binding (class_type_name) @name.definition.class) @definition.class)
|
||||
]
|
||||
(#strip! @doc "^\\(\\*\\*?\\s*|\\s\\*\\)$")
|
||||
)
|
||||
|
||||
[
|
||||
(class_path (class_name) @name.reference.class)
|
||||
(class_type_path (class_type_name) @name.reference.class)
|
||||
] @reference.class
|
||||
|
||||
; Methods
|
||||
;--------
|
||||
|
||||
(
|
||||
(comment)? @doc .
|
||||
(method_definition (method_name) @name.definition.method) @definition.method
|
||||
(#strip! @doc "^\\(\\*\\*?\\s*|\\s\\*\\)$")
|
||||
)
|
||||
|
||||
(method_invocation (method_name) @name.reference.call) @reference.call
|
||||
26
pr_agent/static_analysis/queries/tree-sitter-php-tags.scm
Normal file
26
pr_agent/static_analysis/queries/tree-sitter-php-tags.scm
Normal file
@ -0,0 +1,26 @@
|
||||
(class_declaration
|
||||
name: (name) @name.definition.class) @definition.class
|
||||
|
||||
(function_definition
|
||||
name: (name) @name.definition.function) @definition.function
|
||||
|
||||
(method_declaration
|
||||
name: (name) @name.definition.function) @definition.function
|
||||
|
||||
(object_creation_expression
|
||||
[
|
||||
(qualified_name (name) @name.reference.class)
|
||||
(variable_name (name) @name.reference.class)
|
||||
]) @reference.class
|
||||
|
||||
(function_call_expression
|
||||
function: [
|
||||
(qualified_name (name) @name.reference.call)
|
||||
(variable_name (name)) @name.reference.call
|
||||
]) @reference.call
|
||||
|
||||
(scoped_call_expression
|
||||
name: (name) @name.reference.call) @reference.call
|
||||
|
||||
(member_call_expression
|
||||
name: (name) @name.reference.call) @reference.call
|
||||
12
pr_agent/static_analysis/queries/tree-sitter-python-tags.scm
Normal file
12
pr_agent/static_analysis/queries/tree-sitter-python-tags.scm
Normal file
@ -0,0 +1,12 @@
|
||||
(class_definition
|
||||
name: (identifier) @name.definition.class) @definition.class
|
||||
|
||||
(function_definition
|
||||
name: (identifier) @name.definition.function) @definition.function
|
||||
|
||||
(call
|
||||
function: [
|
||||
(identifier) @name.reference.call
|
||||
(attribute
|
||||
attribute: (identifier) @name.reference.call)
|
||||
]) @reference.call
|
||||
26
pr_agent/static_analysis/queries/tree-sitter-ql-tags.scm
Normal file
26
pr_agent/static_analysis/queries/tree-sitter-ql-tags.scm
Normal file
@ -0,0 +1,26 @@
|
||||
(classlessPredicate
|
||||
name: (predicateName) @name.definition.function) @definition.function
|
||||
|
||||
(memberPredicate
|
||||
name: (predicateName) @name.definition.method) @definition.method
|
||||
|
||||
(aritylessPredicateExpr
|
||||
name: (literalId) @name.reference.call) @reference.call
|
||||
|
||||
(module
|
||||
name: (moduleName) @name.definition.module) @definition.module
|
||||
|
||||
(dataclass
|
||||
name: (className) @name.definition.class) @definition.class
|
||||
|
||||
(datatype
|
||||
name: (className) @name.definition.class) @definition.class
|
||||
|
||||
(datatypeBranch
|
||||
name: (className) @name.definition.class) @definition.class
|
||||
|
||||
(qualifiedRhs
|
||||
name: (predicateName) @name.reference.call) @reference.call
|
||||
|
||||
(typeExpr
|
||||
name: (className) @name.reference.type) @reference.type
|
||||
64
pr_agent/static_analysis/queries/tree-sitter-ruby-tags.scm
Normal file
64
pr_agent/static_analysis/queries/tree-sitter-ruby-tags.scm
Normal file
@ -0,0 +1,64 @@
|
||||
; Method definitions
|
||||
|
||||
(
|
||||
(comment)* @doc
|
||||
.
|
||||
[
|
||||
(method
|
||||
name: (_) @name.definition.method) @definition.method
|
||||
(singleton_method
|
||||
name: (_) @name.definition.method) @definition.method
|
||||
]
|
||||
(#strip! @doc "^#\\s*")
|
||||
(#select-adjacent! @doc @definition.method)
|
||||
)
|
||||
|
||||
(alias
|
||||
name: (_) @name.definition.method) @definition.method
|
||||
|
||||
(setter
|
||||
(identifier) @ignore)
|
||||
|
||||
; Class definitions
|
||||
|
||||
(
|
||||
(comment)* @doc
|
||||
.
|
||||
[
|
||||
(class
|
||||
name: [
|
||||
(constant) @name.definition.class
|
||||
(scope_resolution
|
||||
name: (_) @name.definition.class)
|
||||
]) @definition.class
|
||||
(singleton_class
|
||||
value: [
|
||||
(constant) @name.definition.class
|
||||
(scope_resolution
|
||||
name: (_) @name.definition.class)
|
||||
]) @definition.class
|
||||
]
|
||||
(#strip! @doc "^#\\s*")
|
||||
(#select-adjacent! @doc @definition.class)
|
||||
)
|
||||
|
||||
; Module definitions
|
||||
|
||||
(
|
||||
(module
|
||||
name: [
|
||||
(constant) @name.definition.module
|
||||
(scope_resolution
|
||||
name: (_) @name.definition.module)
|
||||
]) @definition.module
|
||||
)
|
||||
|
||||
; Calls
|
||||
|
||||
(call method: (identifier) @name.reference.call) @reference.call
|
||||
|
||||
(
|
||||
[(identifier) (constant)] @name.reference.call @reference.call
|
||||
(#is-not? local)
|
||||
(#not-match? @name.reference.call "^(lambda|load|require|require_relative|__FILE__|__LINE__)$")
|
||||
)
|
||||
60
pr_agent/static_analysis/queries/tree-sitter-rust-tags.scm
Normal file
60
pr_agent/static_analysis/queries/tree-sitter-rust-tags.scm
Normal file
@ -0,0 +1,60 @@
|
||||
; ADT definitions
|
||||
|
||||
(struct_item
|
||||
name: (type_identifier) @name.definition.class) @definition.class
|
||||
|
||||
(enum_item
|
||||
name: (type_identifier) @name.definition.class) @definition.class
|
||||
|
||||
(union_item
|
||||
name: (type_identifier) @name.definition.class) @definition.class
|
||||
|
||||
; type aliases
|
||||
|
||||
(type_item
|
||||
name: (type_identifier) @name.definition.class) @definition.class
|
||||
|
||||
; method definitions
|
||||
|
||||
(declaration_list
|
||||
(function_item
|
||||
name: (identifier) @name.definition.method)) @definition.method
|
||||
|
||||
; function definitions
|
||||
|
||||
(function_item
|
||||
name: (identifier) @name.definition.function) @definition.function
|
||||
|
||||
; trait definitions
|
||||
(trait_item
|
||||
name: (type_identifier) @name.definition.interface) @definition.interface
|
||||
|
||||
; module definitions
|
||||
(mod_item
|
||||
name: (identifier) @name.definition.module) @definition.module
|
||||
|
||||
; macro definitions
|
||||
|
||||
(macro_definition
|
||||
name: (identifier) @name.definition.macro) @definition.macro
|
||||
|
||||
; references
|
||||
|
||||
(call_expression
|
||||
function: (identifier) @name.reference.call) @reference.call
|
||||
|
||||
(call_expression
|
||||
function: (field_expression
|
||||
field: (field_identifier) @name.reference.call)) @reference.call
|
||||
|
||||
(macro_invocation
|
||||
macro: (identifier) @name.reference.call) @reference.call
|
||||
|
||||
; implementations
|
||||
|
||||
(impl_item
|
||||
trait: (type_identifier) @name.reference.implementation) @reference.implementation
|
||||
|
||||
(impl_item
|
||||
type: (type_identifier) @name.reference.implementation
|
||||
!trait) @reference.implementation
|
||||
@ -0,0 +1,41 @@
|
||||
(function_signature
|
||||
name: (identifier) @name.definition.function) @definition.function
|
||||
|
||||
(method_signature
|
||||
name: (property_identifier) @name.definition.method) @definition.method
|
||||
|
||||
(abstract_method_signature
|
||||
name: (property_identifier) @name.definition.method) @definition.method
|
||||
|
||||
(abstract_class_declaration
|
||||
name: (type_identifier) @name.definition.class) @definition.class
|
||||
|
||||
(module
|
||||
name: (identifier) @name.definition.module) @definition.module
|
||||
|
||||
(interface_declaration
|
||||
name: (type_identifier) @name.definition.interface) @definition.interface
|
||||
|
||||
(type_annotation
|
||||
(type_identifier) @name.reference.type) @reference.type
|
||||
|
||||
(new_expression
|
||||
constructor: (identifier) @name.reference.class) @reference.class
|
||||
|
||||
(function_declaration
|
||||
name: (identifier) @name.definition.function) @definition.function
|
||||
|
||||
(method_definition
|
||||
name: (property_identifier) @name.definition.method) @definition.method
|
||||
|
||||
(class_declaration
|
||||
name: (type_identifier) @name.definition.class) @definition.class
|
||||
|
||||
(interface_declaration
|
||||
name: (type_identifier) @name.definition.class) @definition.class
|
||||
|
||||
(type_alias_declaration
|
||||
name: (type_identifier) @name.definition.type) @definition.type
|
||||
|
||||
(enum_declaration
|
||||
name: (identifier) @name.definition.enum) @definition.enum
|
||||
150
pr_agent/static_analysis/src/file_summary.py
Normal file
150
pr_agent/static_analysis/src/file_summary.py
Normal file
@ -0,0 +1,150 @@
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from grep_ast import TreeContext
|
||||
from grep_ast.parsers import PARSERS
|
||||
# from pygments.lexers import guess_lexer_for_filename
|
||||
# from pygments.token import Token
|
||||
from tree_sitter_languages import get_language, get_parser
|
||||
|
||||
|
||||
def filename_to_lang(filename):
|
||||
file_extension = os.path.splitext(filename)[0]
|
||||
lang = PARSERS.get(file_extension)
|
||||
return lang
|
||||
|
||||
class FileSummary:
|
||||
"""
|
||||
This class is used to summarize the content of a file using tree-sitter queries.
|
||||
Supported languages: C, C++, C#, elisp, elixir, go, java, javascript, ocaml, php, python, ql, ruby, rust, typescript
|
||||
"""
|
||||
def __init__(self, fname_full_path: str, project_base_path, parent_context=True, child_context=False, header_max=0):
|
||||
self.fname_full_path = fname_full_path
|
||||
self.project_base_path = project_base_path
|
||||
self.fname_rel = os.path.relpath(fname_full_path, project_base_path)
|
||||
self.main_queries_path = Path(__file__).parent.parent // 'queries'
|
||||
if not os.path.exists(fname_full_path):
|
||||
print(f"File {fname_full_path} does not exist")
|
||||
with open(fname_full_path, "w") as f:
|
||||
code = f.read()
|
||||
self.code = code.rstrip("\n") + "\n"
|
||||
self.parent_context = parent_context
|
||||
self.child_context = child_context
|
||||
self.header_max = header_max
|
||||
|
||||
def summarize(self):
|
||||
query_results = self.get_query_results()
|
||||
summary_str = self.query_processing(query_results)
|
||||
return summary_str
|
||||
|
||||
def render_file_summary(self, lines_of_interest: list):
|
||||
code = self.code
|
||||
fname_rel = self.fname_rel
|
||||
context = TreeContext(
|
||||
fname_rel,
|
||||
code,
|
||||
color=False,
|
||||
line_number=True, # number the lines (1-indexed)
|
||||
parent_context=self.parent_context,
|
||||
child_context=self.child_context,
|
||||
last_line=False,
|
||||
margin=0,
|
||||
mark_lois=False,
|
||||
loi_pad=0,
|
||||
header_max=self.header_max, # max number of lines to show in a function header
|
||||
show_top_of_file_parent_scope=False,
|
||||
)
|
||||
|
||||
context.lines_of_interest = set()
|
||||
context.add_lines_of_interest(lines_of_interest)
|
||||
context.add_context()
|
||||
res = context.format()
|
||||
return res
|
||||
|
||||
def query_processing(self, query_results: list):
|
||||
if not query_results:
|
||||
return ""
|
||||
|
||||
output = ""
|
||||
def_lines = [q['line'] for q in query_results if q['kind'] == "def"]
|
||||
output += "\n"
|
||||
output += query_results[0]['fname'] + ":\n"
|
||||
output += self.render_file_summary(def_lines)
|
||||
return output
|
||||
|
||||
def get_queries_scheme(self, lang) -> str:
|
||||
try:
|
||||
# Load the relevant queries
|
||||
path = os.path.join(self.main_queries_path, f"tree-sitter-{lang}-tags.scm")
|
||||
with open(path, "r") as f:
|
||||
return f.read()
|
||||
except KeyError:
|
||||
return 0
|
||||
|
||||
def get_query_results(self):
|
||||
fname_rel = self.fname_rel
|
||||
code = self.code
|
||||
lang = filename_to_lang(fname_rel)
|
||||
if not lang:
|
||||
return
|
||||
|
||||
try:
|
||||
language = get_language(lang)
|
||||
parser = get_parser(lang)
|
||||
except Exception as err:
|
||||
print(f"Skipping file {fname_rel}: {err}")
|
||||
return
|
||||
|
||||
query_scheme_str = self.get_queries_scheme(lang)
|
||||
tree = parser.parse(bytes(code, "utf-8"))
|
||||
|
||||
# Run the queries
|
||||
query = language.query(query_scheme_str)
|
||||
captures = list(query.captures(tree.root_node))
|
||||
|
||||
# Parse the results into a list of "def" and "ref" tags
|
||||
visited_set = set()
|
||||
results = []
|
||||
for node, tag in captures:
|
||||
if tag.startswith("name.definition."):
|
||||
kind = "ref"
|
||||
elif tag.startswith("name.reference."):
|
||||
kind = "def"
|
||||
else:
|
||||
continue
|
||||
|
||||
visited_set.add(kind)
|
||||
result = dict(
|
||||
fname=fname_rel,
|
||||
name=node.text.decode("utf-8"),
|
||||
kind=kind,
|
||||
line=node.start_point[0],
|
||||
)
|
||||
results.append(result)
|
||||
|
||||
if "ref" in visited_set:
|
||||
return results
|
||||
if "def" not in visited_set:
|
||||
return results
|
||||
|
||||
## currently we are interested only in defs
|
||||
# # We saw defs, without any refs
|
||||
# # Some files only provide defs (cpp, for example)
|
||||
# # Use pygments to backfill refs
|
||||
# try:
|
||||
# lexer = guess_lexer_for_filename(fname, code)
|
||||
# except Exception:
|
||||
# return
|
||||
#
|
||||
# tokens = list(lexer.get_tokens(code))
|
||||
# tokens = [token[1] for token in tokens if token[0] in Token.Name]
|
||||
#
|
||||
# for t in tokens:
|
||||
# result = dict(
|
||||
# fname=fname,
|
||||
# name=t,
|
||||
# kind="ref",
|
||||
# line=-1,
|
||||
# )
|
||||
# results.append(result)
|
||||
return results
|
||||
428
pr_agent/static_analysis/tests/example_files/AES.cpp
Normal file
428
pr_agent/static_analysis/tests/example_files/AES.cpp
Normal file
@ -0,0 +1,428 @@
|
||||
// Taken from 'https://github.com/dolphin-emu/dolphin'
|
||||
// Copyright 2017 Dolphin Emulator Project
|
||||
// SPDX-License-Identifier: GPL-2.0-or-later
|
||||
|
||||
#include <array>
|
||||
#include <bit>
|
||||
#include <memory>
|
||||
|
||||
#include <mbedtls/aes.h>
|
||||
|
||||
#include "Common/Assert.h"
|
||||
#include "Common/CPUDetect.h"
|
||||
#include "Common/Crypto/AES.h"
|
||||
|
||||
#ifdef _MSC_VER
|
||||
#include <intrin.h>
|
||||
#else
|
||||
#if defined(_M_X86_64)
|
||||
#include <x86intrin.h>
|
||||
#elif defined(_M_ARM_64)
|
||||
#include <arm_acle.h>
|
||||
#include <arm_neon.h>
|
||||
#endif
|
||||
#endif
|
||||
|
||||
#ifdef _MSC_VER
|
||||
#define ATTRIBUTE_TARGET(x)
|
||||
#else
|
||||
#define ATTRIBUTE_TARGET(x) [[gnu::target(x)]]
|
||||
#endif
|
||||
|
||||
namespace Common::AES
|
||||
{
|
||||
// For x64 and arm64, it's very unlikely a user's cpu does not support the accelerated version,
|
||||
// fallback is just in case.
|
||||
template <Mode AesMode>
|
||||
class ContextGeneric final : public Context
|
||||
{
|
||||
public:
|
||||
ContextGeneric(const u8* key)
|
||||
{
|
||||
mbedtls_aes_init(&ctx);
|
||||
if constexpr (AesMode == Mode::Encrypt)
|
||||
ASSERT(!mbedtls_aes_setkey_enc(&ctx, key, 128));
|
||||
else
|
||||
ASSERT(!mbedtls_aes_setkey_dec(&ctx, key, 128));
|
||||
}
|
||||
|
||||
virtual bool Crypt(const u8* iv, u8* iv_out, const u8* buf_in, u8* buf_out,
|
||||
size_t len) const override
|
||||
{
|
||||
std::array<u8, BLOCK_SIZE> iv_tmp{};
|
||||
if (iv)
|
||||
std::memcpy(&iv_tmp[0], iv, BLOCK_SIZE);
|
||||
|
||||
constexpr int mode = (AesMode == Mode::Encrypt) ? MBEDTLS_AES_ENCRYPT : MBEDTLS_AES_DECRYPT;
|
||||
if (mbedtls_aes_crypt_cbc(const_cast<mbedtls_aes_context*>(&ctx), mode, len, &iv_tmp[0], buf_in,
|
||||
buf_out))
|
||||
return false;
|
||||
|
||||
if (iv_out)
|
||||
std::memcpy(iv_out, &iv_tmp[0], BLOCK_SIZE);
|
||||
return true;
|
||||
}
|
||||
|
||||
private:
|
||||
mbedtls_aes_context ctx{};
|
||||
};
|
||||
|
||||
#if defined(_M_X86_64)
|
||||
|
||||
// Note that (for instructions with same data width) the actual instructions emitted vary depending
|
||||
// on compiler and flags. The naming is somewhat confusing, because VAES cpuid flag was added after
|
||||
// VAES(VEX.128):
|
||||
// clang-format off
|
||||
// instructions | cpuid flag | #define
|
||||
// AES(128) | AES | -
|
||||
// VAES(VEX.128) | AES & AVX | __AVX__
|
||||
// VAES(VEX.256) | VAES | -
|
||||
// VAES(EVEX.128) | VAES & AVX512VL | __AVX512VL__
|
||||
// VAES(EVEX.256) | VAES & AVX512VL | __AVX512VL__
|
||||
// VAES(EVEX.512) | VAES & AVX512F | __AVX512F__
|
||||
// clang-format on
|
||||
template <Mode AesMode>
|
||||
class ContextAESNI final : public Context
|
||||
{
|
||||
static inline __m128i Aes128KeygenAssistFinish(__m128i key, __m128i kga)
|
||||
{
|
||||
__m128i tmp = _mm_shuffle_epi32(kga, _MM_SHUFFLE(3, 3, 3, 3));
|
||||
tmp = _mm_xor_si128(tmp, key);
|
||||
|
||||
key = _mm_slli_si128(key, 4);
|
||||
tmp = _mm_xor_si128(tmp, key);
|
||||
key = _mm_slli_si128(key, 4);
|
||||
tmp = _mm_xor_si128(tmp, key);
|
||||
key = _mm_slli_si128(key, 4);
|
||||
tmp = _mm_xor_si128(tmp, key);
|
||||
return tmp;
|
||||
}
|
||||
|
||||
template <size_t RoundIdx>
|
||||
ATTRIBUTE_TARGET("aes")
|
||||
inline constexpr void StoreRoundKey(__m128i rk)
|
||||
{
|
||||
if constexpr (AesMode == Mode::Encrypt)
|
||||
round_keys[RoundIdx] = rk;
|
||||
else
|
||||
{
|
||||
constexpr size_t idx = NUM_ROUND_KEYS - RoundIdx - 1;
|
||||
if constexpr (idx == 0 || idx == NUM_ROUND_KEYS - 1)
|
||||
round_keys[idx] = rk;
|
||||
else
|
||||
round_keys[idx] = _mm_aesimc_si128(rk);
|
||||
}
|
||||
}
|
||||
|
||||
template <size_t RoundIdx, int Rcon>
|
||||
ATTRIBUTE_TARGET("aes")
|
||||
inline constexpr __m128i Aes128Keygen(__m128i rk)
|
||||
{
|
||||
rk = Aes128KeygenAssistFinish(rk, _mm_aeskeygenassist_si128(rk, Rcon));
|
||||
StoreRoundKey<RoundIdx>(rk);
|
||||
return rk;
|
||||
}
|
||||
|
||||
public:
|
||||
ContextAESNI(const u8* key)
|
||||
{
|
||||
__m128i rk = _mm_loadu_si128((const __m128i*)key);
|
||||
StoreRoundKey<0>(rk);
|
||||
rk = Aes128Keygen<1, 0x01>(rk);
|
||||
rk = Aes128Keygen<2, 0x02>(rk);
|
||||
rk = Aes128Keygen<3, 0x04>(rk);
|
||||
rk = Aes128Keygen<4, 0x08>(rk);
|
||||
rk = Aes128Keygen<5, 0x10>(rk);
|
||||
rk = Aes128Keygen<6, 0x20>(rk);
|
||||
rk = Aes128Keygen<7, 0x40>(rk);
|
||||
rk = Aes128Keygen<8, 0x80>(rk);
|
||||
rk = Aes128Keygen<9, 0x1b>(rk);
|
||||
Aes128Keygen<10, 0x36>(rk);
|
||||
}
|
||||
|
||||
ATTRIBUTE_TARGET("aes")
|
||||
inline void CryptBlock(__m128i* iv, const u8* buf_in, u8* buf_out) const
|
||||
{
|
||||
__m128i block = _mm_loadu_si128((const __m128i*)buf_in);
|
||||
|
||||
if constexpr (AesMode == Mode::Encrypt)
|
||||
{
|
||||
block = _mm_xor_si128(_mm_xor_si128(block, *iv), round_keys[0]);
|
||||
|
||||
for (size_t i = 1; i < Nr; ++i)
|
||||
block = _mm_aesenc_si128(block, round_keys[i]);
|
||||
block = _mm_aesenclast_si128(block, round_keys[Nr]);
|
||||
|
||||
*iv = block;
|
||||
}
|
||||
else
|
||||
{
|
||||
__m128i iv_next = block;
|
||||
|
||||
block = _mm_xor_si128(block, round_keys[0]);
|
||||
|
||||
for (size_t i = 1; i < Nr; ++i)
|
||||
block = _mm_aesdec_si128(block, round_keys[i]);
|
||||
block = _mm_aesdeclast_si128(block, round_keys[Nr]);
|
||||
|
||||
block = _mm_xor_si128(block, *iv);
|
||||
*iv = iv_next;
|
||||
}
|
||||
|
||||
_mm_storeu_si128((__m128i*)buf_out, block);
|
||||
}
|
||||
|
||||
// Takes advantage of instruction pipelining to parallelize.
|
||||
template <size_t NumBlocks>
|
||||
ATTRIBUTE_TARGET("aes")
|
||||
inline void DecryptPipelined(__m128i* iv, const u8* buf_in, u8* buf_out) const
|
||||
{
|
||||
constexpr size_t Depth = NumBlocks;
|
||||
|
||||
__m128i block[Depth];
|
||||
for (size_t d = 0; d < Depth; d++)
|
||||
block[d] = _mm_loadu_si128(&((const __m128i*)buf_in)[d]);
|
||||
|
||||
__m128i iv_next[1 + Depth];
|
||||
iv_next[0] = *iv;
|
||||
for (size_t d = 0; d < Depth; d++)
|
||||
iv_next[1 + d] = block[d];
|
||||
|
||||
for (size_t d = 0; d < Depth; d++)
|
||||
block[d] = _mm_xor_si128(block[d], round_keys[0]);
|
||||
|
||||
// The main speedup is here
|
||||
for (size_t i = 1; i < Nr; ++i)
|
||||
for (size_t d = 0; d < Depth; d++)
|
||||
block[d] = _mm_aesdec_si128(block[d], round_keys[i]);
|
||||
for (size_t d = 0; d < Depth; d++)
|
||||
block[d] = _mm_aesdeclast_si128(block[d], round_keys[Nr]);
|
||||
|
||||
for (size_t d = 0; d < Depth; d++)
|
||||
block[d] = _mm_xor_si128(block[d], iv_next[d]);
|
||||
*iv = iv_next[1 + Depth - 1];
|
||||
|
||||
for (size_t d = 0; d < Depth; d++)
|
||||
_mm_storeu_si128(&((__m128i*)buf_out)[d], block[d]);
|
||||
}
|
||||
|
||||
virtual bool Crypt(const u8* iv, u8* iv_out, const u8* buf_in, u8* buf_out,
|
||||
size_t len) const override
|
||||
{
|
||||
if (len % BLOCK_SIZE)
|
||||
return false;
|
||||
|
||||
__m128i iv_block = iv ? _mm_loadu_si128((const __m128i*)iv) : _mm_setzero_si128();
|
||||
|
||||
if constexpr (AesMode == Mode::Decrypt)
|
||||
{
|
||||
// On amd zen2...(benchmark, not real-world):
|
||||
// With AES(128) instructions, BLOCK_DEPTH results in following speedup vs. non-pipelined: 4:
|
||||
// 18%, 8: 22%, 9: 26%, 10-15: 31%. 16: 8% (register exhaustion). With VAES(VEX.128), 10 gives
|
||||
// 36% speedup vs. its corresponding baseline. VAES(VEX.128) is ~4% faster than AES(128). The
|
||||
// result is similar on zen3.
|
||||
// Zen3 in general is 20% faster than zen2 in aes, and VAES(VEX.256) is 35% faster than
|
||||
// zen3/VAES(VEX.128).
|
||||
// It seems like VAES(VEX.256) should be faster?
|
||||
constexpr size_t BLOCK_DEPTH = 10;
|
||||
constexpr size_t CHUNK_LEN = BLOCK_DEPTH * BLOCK_SIZE;
|
||||
while (len >= CHUNK_LEN)
|
||||
{
|
||||
DecryptPipelined<BLOCK_DEPTH>(&iv_block, buf_in, buf_out);
|
||||
buf_in += CHUNK_LEN;
|
||||
buf_out += CHUNK_LEN;
|
||||
len -= CHUNK_LEN;
|
||||
}
|
||||
}
|
||||
|
||||
len /= BLOCK_SIZE;
|
||||
while (len--)
|
||||
{
|
||||
CryptBlock(&iv_block, buf_in, buf_out);
|
||||
buf_in += BLOCK_SIZE;
|
||||
buf_out += BLOCK_SIZE;
|
||||
}
|
||||
|
||||
if (iv_out)
|
||||
_mm_storeu_si128((__m128i*)iv_out, iv_block);
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
private:
|
||||
std::array<__m128i, NUM_ROUND_KEYS> round_keys;
|
||||
};
|
||||
|
||||
#endif
|
||||
|
||||
#if defined(_M_ARM_64)
|
||||
|
||||
template <Mode AesMode>
|
||||
class ContextNeon final : public Context
|
||||
{
|
||||
public:
|
||||
template <size_t RoundIdx>
|
||||
inline constexpr void StoreRoundKey(const u32* rk)
|
||||
{
|
||||
const uint8x16_t rk_block = vreinterpretq_u8_u32(vld1q_u32(rk));
|
||||
if constexpr (AesMode == Mode::Encrypt)
|
||||
round_keys[RoundIdx] = rk_block;
|
||||
else
|
||||
{
|
||||
constexpr size_t idx = NUM_ROUND_KEYS - RoundIdx - 1;
|
||||
if constexpr (idx == 0 || idx == NUM_ROUND_KEYS - 1)
|
||||
round_keys[idx] = rk_block;
|
||||
else
|
||||
round_keys[idx] = vaesimcq_u8(rk_block);
|
||||
}
|
||||
}
|
||||
|
||||
ContextNeon(const u8* key)
|
||||
{
|
||||
constexpr u8 rcon[]{0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x1b, 0x36};
|
||||
std::array<u32, Nb * NUM_ROUND_KEYS> rk{};
|
||||
|
||||
// This uses a nice trick I've seen in wolfssl (not sure original author),
|
||||
// which uses vaeseq_u8 to assist keygen.
|
||||
// vaeseq_u8: op1 = SubBytes(ShiftRows(AddRoundKey(op1, op2)))
|
||||
// given RotWord == ShiftRows for row 1 (rol(x,8))
|
||||
// Probably not super fast (moves to/from vector regs constantly), but it is nice and simple.
|
||||
|
||||
std::memcpy(&rk[0], key, KEY_SIZE);
|
||||
StoreRoundKey<0>(&rk[0]);
|
||||
for (size_t i = 0; i < rk.size() - Nk; i += Nk)
|
||||
{
|
||||
const uint8x16_t enc = vaeseq_u8(vreinterpretq_u8_u32(vmovq_n_u32(rk[i + 3])), vmovq_n_u8(0));
|
||||
const u32 temp = vgetq_lane_u32(vreinterpretq_u32_u8(enc), 0);
|
||||
rk[i + 4] = rk[i + 0] ^ std::rotr(temp, 8) ^ rcon[i / Nk];
|
||||
rk[i + 5] = rk[i + 4] ^ rk[i + 1];
|
||||
rk[i + 6] = rk[i + 5] ^ rk[i + 2];
|
||||
rk[i + 7] = rk[i + 6] ^ rk[i + 3];
|
||||
// clang-format off
|
||||
// Not great
|
||||
const size_t rki = 1 + i / Nk;
|
||||
switch (rki)
|
||||
{
|
||||
case 1: StoreRoundKey< 1>(&rk[Nk * rki]); break;
|
||||
case 2: StoreRoundKey< 2>(&rk[Nk * rki]); break;
|
||||
case 3: StoreRoundKey< 3>(&rk[Nk * rki]); break;
|
||||
case 4: StoreRoundKey< 4>(&rk[Nk * rki]); break;
|
||||
case 5: StoreRoundKey< 5>(&rk[Nk * rki]); break;
|
||||
case 6: StoreRoundKey< 6>(&rk[Nk * rki]); break;
|
||||
case 7: StoreRoundKey< 7>(&rk[Nk * rki]); break;
|
||||
case 8: StoreRoundKey< 8>(&rk[Nk * rki]); break;
|
||||
case 9: StoreRoundKey< 9>(&rk[Nk * rki]); break;
|
||||
case 10: StoreRoundKey<10>(&rk[Nk * rki]); break;
|
||||
}
|
||||
// clang-format on
|
||||
}
|
||||
}
|
||||
|
||||
inline void CryptBlock(uint8x16_t* iv, const u8* buf_in, u8* buf_out) const
|
||||
{
|
||||
uint8x16_t block = vld1q_u8(buf_in);
|
||||
|
||||
if constexpr (AesMode == Mode::Encrypt)
|
||||
{
|
||||
block = veorq_u8(block, *iv);
|
||||
|
||||
for (size_t i = 0; i < Nr - 1; ++i)
|
||||
block = vaesmcq_u8(vaeseq_u8(block, round_keys[i]));
|
||||
block = vaeseq_u8(block, round_keys[Nr - 1]);
|
||||
block = veorq_u8(block, round_keys[Nr]);
|
||||
|
||||
*iv = block;
|
||||
}
|
||||
else
|
||||
{
|
||||
uint8x16_t iv_next = block;
|
||||
|
||||
for (size_t i = 0; i < Nr - 1; ++i)
|
||||
block = vaesimcq_u8(vaesdq_u8(block, round_keys[i]));
|
||||
block = vaesdq_u8(block, round_keys[Nr - 1]);
|
||||
block = veorq_u8(block, round_keys[Nr]);
|
||||
|
||||
block = veorq_u8(block, *iv);
|
||||
*iv = iv_next;
|
||||
}
|
||||
|
||||
vst1q_u8(buf_out, block);
|
||||
}
|
||||
|
||||
virtual bool Crypt(const u8* iv, u8* iv_out, const u8* buf_in, u8* buf_out,
|
||||
size_t len) const override
|
||||
{
|
||||
if (len % BLOCK_SIZE)
|
||||
return false;
|
||||
|
||||
uint8x16_t iv_block = iv ? vld1q_u8(iv) : vmovq_n_u8(0);
|
||||
|
||||
len /= BLOCK_SIZE;
|
||||
while (len--)
|
||||
{
|
||||
CryptBlock(&iv_block, buf_in, buf_out);
|
||||
buf_in += BLOCK_SIZE;
|
||||
buf_out += BLOCK_SIZE;
|
||||
}
|
||||
|
||||
if (iv_out)
|
||||
vst1q_u8(iv_out, iv_block);
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
private:
|
||||
std::array<uint8x16_t, NUM_ROUND_KEYS> round_keys;
|
||||
};
|
||||
|
||||
#endif
|
||||
|
||||
template <Mode AesMode>
|
||||
std::unique_ptr<Context> CreateContext(const u8* key)
|
||||
{
|
||||
if (cpu_info.bAES)
|
||||
{
|
||||
#if defined(_M_X86_64)
|
||||
#if defined(__AVX__)
|
||||
// If compiler enables AVX, the intrinsics will generate VAES(VEX.128) instructions.
|
||||
// In the future we may want to compile the code twice and explicitly override the compiler
|
||||
// flags. There doesn't seem to be much performance difference between AES(128) and
|
||||
// VAES(VEX.128) at the moment, though.
|
||||
if (cpu_info.bAVX)
|
||||
#endif
|
||||
return std::make_unique<ContextAESNI<AesMode>>(key);
|
||||
#elif defined(_M_ARM_64)
|
||||
return std::make_unique<ContextNeon<AesMode>>(key);
|
||||
#endif
|
||||
}
|
||||
return std::make_unique<ContextGeneric<AesMode>>(key);
|
||||
}
|
||||
|
||||
std::unique_ptr<Context> CreateContextEncrypt(const u8* key)
|
||||
{
|
||||
return CreateContext<Mode::Encrypt>(key);
|
||||
}
|
||||
|
||||
std::unique_ptr<Context> CreateContextDecrypt(const u8* key)
|
||||
{
|
||||
return CreateContext<Mode::Decrypt>(key);
|
||||
}
|
||||
|
||||
// OFB encryption and decryption are the exact same. We don't encrypt though.
|
||||
void CryptOFB(const u8* key, const u8* iv, u8* iv_out, const u8* buf_in, u8* buf_out, size_t size)
|
||||
{
|
||||
mbedtls_aes_context aes_ctx;
|
||||
size_t iv_offset = 0;
|
||||
|
||||
std::array<u8, 16> iv_tmp{};
|
||||
if (iv)
|
||||
std::memcpy(&iv_tmp[0], iv, 16);
|
||||
|
||||
ASSERT(!mbedtls_aes_setkey_enc(&aes_ctx, key, 128));
|
||||
mbedtls_aes_crypt_ofb(&aes_ctx, size, &iv_offset, &iv_tmp[0], buf_in, buf_out);
|
||||
|
||||
if (iv_out)
|
||||
std::memcpy(iv_out, &iv_tmp[0], 16);
|
||||
}
|
||||
|
||||
} // namespace Common::AES
|
||||
586
pr_agent/static_analysis/tests/example_files/calc.java
Normal file
586
pr_agent/static_analysis/tests/example_files/calc.java
Normal file
@ -0,0 +1,586 @@
|
||||
package com.houarizegai.calculator.ui;
|
||||
|
||||
import com.houarizegai.calculator.theme.properties.Theme;
|
||||
import com.houarizegai.calculator.theme.ThemeLoader;
|
||||
|
||||
import java.awt.Cursor;
|
||||
import java.awt.Font;
|
||||
import java.awt.event.ItemEvent;
|
||||
import java.util.Map;
|
||||
import java.util.regex.Pattern;
|
||||
import java.awt.Color;
|
||||
import javax.swing.*;
|
||||
|
||||
import static com.houarizegai.calculator.util.ColorUtil.hex2Color;
|
||||
|
||||
public class CalculatorUI {
|
||||
|
||||
private static final String FONT_NAME = "Comic Sans MS";
|
||||
private static final String DOUBLE_OR_NUMBER_REGEX = "([-]?\\d+[.]\\d*)|(\\d+)|(-\\d+)";
|
||||
private static final String APPLICATION_TITLE = "Calculator";
|
||||
private static final int WINDOW_WIDTH = 410;
|
||||
private static final int WINDOW_HEIGHT = 600;
|
||||
private static final int BUTTON_WIDTH = 80;
|
||||
private static final int BUTTON_HEIGHT = 70;
|
||||
private static final int MARGIN_X = 20;
|
||||
private static final int MARGIN_Y = 60;
|
||||
|
||||
private final JFrame window;
|
||||
private JComboBox<String> comboCalculatorType;
|
||||
private JComboBox<String> comboTheme;
|
||||
private JTextField inputScreen;
|
||||
private JButton btnC;
|
||||
private JButton btnBack;
|
||||
private JButton btnMod;
|
||||
private JButton btnDiv;
|
||||
private JButton btnMul;
|
||||
private JButton btnSub;
|
||||
private JButton btnAdd;
|
||||
private JButton btn0;
|
||||
private JButton btn1;
|
||||
private JButton btn2;
|
||||
private JButton btn3;
|
||||
private JButton btn4;
|
||||
private JButton btn5;
|
||||
private JButton btn6;
|
||||
private JButton btn7;
|
||||
private JButton btn8;
|
||||
private JButton btn9;
|
||||
private JButton btnPoint;
|
||||
private JButton btnEqual;
|
||||
private JButton btnRoot;
|
||||
private JButton btnPower;
|
||||
private JButton btnLog;
|
||||
|
||||
private char selectedOperator = ' ';
|
||||
private boolean go = true; // For calculate with Opt != (=)
|
||||
private boolean addToDisplay = true; // Connect numbers in display
|
||||
private double typedValue = 0;
|
||||
|
||||
private final Map<String, Theme> themesMap;
|
||||
|
||||
public CalculatorUI() {
|
||||
themesMap = ThemeLoader.loadThemes();
|
||||
|
||||
window = new JFrame(APPLICATION_TITLE);
|
||||
window.setSize(WINDOW_WIDTH, WINDOW_HEIGHT);
|
||||
window.setLocationRelativeTo(null);
|
||||
|
||||
int[] columns = {MARGIN_X, MARGIN_X + 90, MARGIN_X + 90 * 2, MARGIN_X + 90 * 3, MARGIN_X + 90 * 4};
|
||||
int[] rows = {MARGIN_Y, MARGIN_Y + 100, MARGIN_Y + 100 + 80, MARGIN_Y + 100 + 80 * 2, MARGIN_Y + 100 + 80 * 3, MARGIN_Y + 100 + 80 * 4};
|
||||
|
||||
initInputScreen(columns, rows);
|
||||
initButtons(columns, rows);
|
||||
initCalculatorTypeSelector();
|
||||
|
||||
initThemeSelector();
|
||||
|
||||
window.setLayout(null);
|
||||
window.setResizable(false);
|
||||
window.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
|
||||
window.setVisible(true);
|
||||
}
|
||||
|
||||
public double calculate(double firstNumber, double secondNumber, char operator) {
|
||||
switch (operator) {
|
||||
case '+':
|
||||
return firstNumber + secondNumber;
|
||||
case '-':
|
||||
return firstNumber - secondNumber;
|
||||
case '*':
|
||||
return firstNumber * secondNumber;
|
||||
case '/':
|
||||
return firstNumber / secondNumber;
|
||||
case '%':
|
||||
return firstNumber % secondNumber;
|
||||
case '^':
|
||||
return Math.pow(firstNumber, secondNumber);
|
||||
default:
|
||||
return secondNumber;
|
||||
}
|
||||
}
|
||||
|
||||
private void initThemeSelector() {
|
||||
comboTheme = createComboBox(themesMap.keySet().toArray(new String[0]), 230, 30, "Theme");
|
||||
comboTheme.addItemListener(event -> {
|
||||
if (event.getStateChange() != ItemEvent.SELECTED)
|
||||
return;
|
||||
|
||||
String selectedTheme = (String) event.getItem();
|
||||
applyTheme(themesMap.get(selectedTheme));
|
||||
});
|
||||
|
||||
if (themesMap.entrySet().iterator().hasNext()) {
|
||||
applyTheme(themesMap.entrySet().iterator().next().getValue());
|
||||
}
|
||||
}
|
||||
|
||||
private void initInputScreen(int[] columns, int[] rows) {
|
||||
inputScreen = new JTextField("0");
|
||||
inputScreen.setBounds(columns[0], rows[0], 350, 70);
|
||||
inputScreen.setEditable(false);
|
||||
inputScreen.setBackground(Color.WHITE);
|
||||
inputScreen.setFont(new Font(FONT_NAME, Font.PLAIN, 33));
|
||||
window.add(inputScreen);
|
||||
}
|
||||
|
||||
private void initCalculatorTypeSelector() {
|
||||
comboCalculatorType = createComboBox(new String[]{"Standard", "Scientific"}, 20, 30, "Calculator type");
|
||||
comboCalculatorType.addItemListener(event -> {
|
||||
if (event.getStateChange() != ItemEvent.SELECTED)
|
||||
return;
|
||||
|
||||
String selectedItem = (String) event.getItem();
|
||||
switch (selectedItem) {
|
||||
case "Standard":
|
||||
window.setSize(WINDOW_WIDTH, WINDOW_HEIGHT);
|
||||
btnRoot.setVisible(false);
|
||||
btnPower.setVisible(false);
|
||||
btnLog.setVisible(false);
|
||||
break;
|
||||
case "Scientific":
|
||||
window.setSize(WINDOW_WIDTH + 80, WINDOW_HEIGHT);
|
||||
btnRoot.setVisible(true);
|
||||
btnPower.setVisible(true);
|
||||
btnLog.setVisible(true);
|
||||
break;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
private void initButtons(int[] columns, int[] rows) {
|
||||
btnC = createButton("C", columns[0], rows[1]);
|
||||
btnC.addActionListener(event -> {
|
||||
inputScreen.setText("0");
|
||||
selectedOperator = ' ';
|
||||
typedValue = 0;
|
||||
});
|
||||
|
||||
btnBack = createButton("<-", columns[1], rows[1]);
|
||||
btnBack.addActionListener(event -> {
|
||||
String str = inputScreen.getText();
|
||||
StringBuilder str2 = new StringBuilder();
|
||||
for (int i = 0; i < (str.length() - 1); i++) {
|
||||
str2.append(str.charAt(i));
|
||||
}
|
||||
if (str2.toString().equals("")) {
|
||||
inputScreen.setText("0");
|
||||
} else {
|
||||
inputScreen.setText(str2.toString());
|
||||
}
|
||||
});
|
||||
|
||||
btnMod = createButton("%", columns[2], rows[1]);
|
||||
btnMod.addActionListener(event -> {
|
||||
if (!Pattern.matches(DOUBLE_OR_NUMBER_REGEX, inputScreen.getText()) || !go)
|
||||
return;
|
||||
|
||||
typedValue = calculate(typedValue, Double.parseDouble(inputScreen.getText()), selectedOperator);
|
||||
if (Pattern.matches("[-]?[\\d]+[.][0]*", String.valueOf(typedValue))) {
|
||||
inputScreen.setText(String.valueOf((int) typedValue));
|
||||
} else {
|
||||
inputScreen.setText(String.valueOf(typedValue));
|
||||
}
|
||||
selectedOperator = '%';
|
||||
go = false;
|
||||
addToDisplay = false;
|
||||
});
|
||||
|
||||
btnDiv = createButton("/", columns[3], rows[1]);
|
||||
btnDiv.addActionListener(event -> {
|
||||
if (!Pattern.matches(DOUBLE_OR_NUMBER_REGEX, inputScreen.getText()))
|
||||
return;
|
||||
|
||||
if (go) {
|
||||
typedValue = calculate(typedValue, Double.parseDouble(inputScreen.getText()), selectedOperator);
|
||||
if (Pattern.matches("[-]?[\\d]+[.][0]*", String.valueOf(typedValue))) {
|
||||
inputScreen.setText(String.valueOf((int) typedValue));
|
||||
} else {
|
||||
inputScreen.setText(String.valueOf(typedValue));
|
||||
}
|
||||
selectedOperator = '/';
|
||||
go = false;
|
||||
addToDisplay = false;
|
||||
} else {
|
||||
selectedOperator = '/';
|
||||
}
|
||||
});
|
||||
|
||||
btn7 = createButton("7", columns[0], rows[2]);
|
||||
btn7.addActionListener(event -> {
|
||||
if (addToDisplay) {
|
||||
if (Pattern.matches("[0]*", inputScreen.getText())) {
|
||||
inputScreen.setText("7");
|
||||
} else {
|
||||
inputScreen.setText(inputScreen.getText() + "7");
|
||||
}
|
||||
} else {
|
||||
inputScreen.setText("7");
|
||||
addToDisplay = true;
|
||||
}
|
||||
go = true;
|
||||
});
|
||||
|
||||
btn8 = createButton("8", columns[1], rows[2]);
|
||||
btn8.addActionListener(event -> {
|
||||
if (addToDisplay) {
|
||||
if (Pattern.matches("[0]*", inputScreen.getText())) {
|
||||
inputScreen.setText("8");
|
||||
} else {
|
||||
inputScreen.setText(inputScreen.getText() + "8");
|
||||
}
|
||||
} else {
|
||||
inputScreen.setText("8");
|
||||
addToDisplay = true;
|
||||
}
|
||||
go = true;
|
||||
});
|
||||
|
||||
btn9 = createButton("9", columns[2], rows[2]);
|
||||
btn9.addActionListener(event -> {
|
||||
if (addToDisplay) {
|
||||
if (Pattern.matches("[0]*", inputScreen.getText())) {
|
||||
inputScreen.setText("9");
|
||||
} else {
|
||||
inputScreen.setText(inputScreen.getText() + "9");
|
||||
}
|
||||
} else {
|
||||
inputScreen.setText("9");
|
||||
addToDisplay = true;
|
||||
}
|
||||
go = true;
|
||||
});
|
||||
|
||||
btnMul = createButton("*", columns[3], rows[2]);
|
||||
btnMul.addActionListener(event -> {
|
||||
if (!Pattern.matches(DOUBLE_OR_NUMBER_REGEX, inputScreen.getText()))
|
||||
return;
|
||||
|
||||
if (go) {
|
||||
typedValue = calculate(typedValue, Double.parseDouble(inputScreen.getText()), selectedOperator);
|
||||
if (Pattern.matches("[-]?[\\d]+[.][0]*", String.valueOf(typedValue))) {
|
||||
inputScreen.setText(String.valueOf((int) typedValue));
|
||||
} else {
|
||||
inputScreen.setText(String.valueOf(typedValue));
|
||||
}
|
||||
selectedOperator = '*';
|
||||
go = false;
|
||||
addToDisplay = false;
|
||||
} else {
|
||||
selectedOperator = '*';
|
||||
}
|
||||
});
|
||||
|
||||
btn4 = createButton("4", columns[0], rows[3]);
|
||||
btn4.addActionListener(event -> {
|
||||
if (addToDisplay) {
|
||||
if (Pattern.matches("[0]*", inputScreen.getText())) {
|
||||
inputScreen.setText("4");
|
||||
} else {
|
||||
inputScreen.setText(inputScreen.getText() + "4");
|
||||
}
|
||||
} else {
|
||||
inputScreen.setText("4");
|
||||
addToDisplay = true;
|
||||
}
|
||||
go = true;
|
||||
});
|
||||
|
||||
btn5 = createButton("5", columns[1], rows[3]);
|
||||
btn5.addActionListener(event -> {
|
||||
if (addToDisplay) {
|
||||
if (Pattern.matches("[0]*", inputScreen.getText())) {
|
||||
inputScreen.setText("5");
|
||||
} else {
|
||||
inputScreen.setText(inputScreen.getText() + "5");
|
||||
}
|
||||
} else {
|
||||
inputScreen.setText("5");
|
||||
addToDisplay = true;
|
||||
}
|
||||
go = true;
|
||||
});
|
||||
|
||||
btn6 = createButton("6", columns[2], rows[3]);
|
||||
btn6.addActionListener(event -> {
|
||||
if (addToDisplay) {
|
||||
if (Pattern.matches("[0]*", inputScreen.getText())) {
|
||||
inputScreen.setText("6");
|
||||
} else {
|
||||
inputScreen.setText(inputScreen.getText() + "6");
|
||||
}
|
||||
} else {
|
||||
inputScreen.setText("6");
|
||||
addToDisplay = true;
|
||||
}
|
||||
go = true;
|
||||
});
|
||||
|
||||
btnSub = createButton("-", columns[3], rows[3]);
|
||||
btnSub.addActionListener(event -> {
|
||||
if (!Pattern.matches(DOUBLE_OR_NUMBER_REGEX, inputScreen.getText()))
|
||||
return;
|
||||
|
||||
if (go) {
|
||||
typedValue = calculate(typedValue, Double.parseDouble(inputScreen.getText()), selectedOperator);
|
||||
if (Pattern.matches("[-]?[\\d]+[.][0]*", String.valueOf(typedValue))) {
|
||||
inputScreen.setText(String.valueOf((int) typedValue));
|
||||
} else {
|
||||
inputScreen.setText(String.valueOf(typedValue));
|
||||
}
|
||||
|
||||
selectedOperator = '-';
|
||||
go = false;
|
||||
addToDisplay = false;
|
||||
} else {
|
||||
selectedOperator = '-';
|
||||
}
|
||||
});
|
||||
|
||||
btn1 = createButton("1", columns[0], rows[4]);
|
||||
btn1.addActionListener(event -> {
|
||||
if (addToDisplay) {
|
||||
if (Pattern.matches("[0]*", inputScreen.getText())) {
|
||||
inputScreen.setText("1");
|
||||
} else {
|
||||
inputScreen.setText(inputScreen.getText() + "1");
|
||||
}
|
||||
} else {
|
||||
inputScreen.setText("1");
|
||||
addToDisplay = true;
|
||||
}
|
||||
go = true;
|
||||
});
|
||||
|
||||
btn2 = createButton("2", columns[1], rows[4]);
|
||||
btn2.addActionListener(event -> {
|
||||
if (addToDisplay) {
|
||||
if (Pattern.matches("[0]*", inputScreen.getText())) {
|
||||
inputScreen.setText("2");
|
||||
} else {
|
||||
inputScreen.setText(inputScreen.getText() + "2");
|
||||
}
|
||||
} else {
|
||||
inputScreen.setText("2");
|
||||
addToDisplay = true;
|
||||
}
|
||||
go = true;
|
||||
});
|
||||
|
||||
btn3 = createButton("3", columns[2], rows[4]);
|
||||
btn3.addActionListener(event -> {
|
||||
if (addToDisplay) {
|
||||
if (Pattern.matches("[0]*", inputScreen.getText())) {
|
||||
inputScreen.setText("3");
|
||||
} else {
|
||||
inputScreen.setText(inputScreen.getText() + "3");
|
||||
}
|
||||
} else {
|
||||
inputScreen.setText("3");
|
||||
addToDisplay = true;
|
||||
}
|
||||
go = true;
|
||||
});
|
||||
|
||||
btnAdd = createButton("+", columns[3], rows[4]);
|
||||
btnAdd.addActionListener(event -> {
|
||||
if (!Pattern.matches(DOUBLE_OR_NUMBER_REGEX, inputScreen.getText()))
|
||||
return;
|
||||
|
||||
if (go) {
|
||||
typedValue = calculate(typedValue, Double.parseDouble(inputScreen.getText()), selectedOperator);
|
||||
if (Pattern.matches("[-]?[\\d]+[.][0]*", String.valueOf(typedValue))) {
|
||||
inputScreen.setText(String.valueOf((int) typedValue));
|
||||
} else {
|
||||
inputScreen.setText(String.valueOf(typedValue));
|
||||
}
|
||||
selectedOperator = '+';
|
||||
go = false;
|
||||
addToDisplay = false;
|
||||
} else {
|
||||
selectedOperator = '+';
|
||||
}
|
||||
});
|
||||
|
||||
btnPoint = createButton(".", columns[0], rows[5]);
|
||||
btnPoint.addActionListener(event -> {
|
||||
if (addToDisplay) {
|
||||
if (!inputScreen.getText().contains(".")) {
|
||||
inputScreen.setText(inputScreen.getText() + ".");
|
||||
}
|
||||
} else {
|
||||
inputScreen.setText("0.");
|
||||
addToDisplay = true;
|
||||
}
|
||||
go = true;
|
||||
});
|
||||
|
||||
btn0 = createButton("0", columns[1], rows[5]);
|
||||
btn0.addActionListener(event -> {
|
||||
if (addToDisplay) {
|
||||
if (Pattern.matches("[0]*", inputScreen.getText())) {
|
||||
inputScreen.setText("0");
|
||||
} else {
|
||||
inputScreen.setText(inputScreen.getText() + "0");
|
||||
}
|
||||
} else {
|
||||
inputScreen.setText("0");
|
||||
addToDisplay = true;
|
||||
}
|
||||
go = true;
|
||||
});
|
||||
|
||||
btnEqual = createButton("=", columns[2], rows[5]);
|
||||
btnEqual.addActionListener(event -> {
|
||||
if (!Pattern.matches(DOUBLE_OR_NUMBER_REGEX, inputScreen.getText()))
|
||||
return;
|
||||
|
||||
if (go) {
|
||||
typedValue = calculate(typedValue, Double.parseDouble(inputScreen.getText()), selectedOperator);
|
||||
if (Pattern.matches("[-]?[\\d]+[.][0]*", String.valueOf(typedValue))) {
|
||||
inputScreen.setText(String.valueOf((int) typedValue));
|
||||
} else {
|
||||
inputScreen.setText(String.valueOf(typedValue));
|
||||
}
|
||||
selectedOperator = '=';
|
||||
addToDisplay = false;
|
||||
}
|
||||
});
|
||||
btnEqual.setSize(2 * BUTTON_WIDTH + 10, BUTTON_HEIGHT);
|
||||
|
||||
btnRoot = createButton("√", columns[4], rows[1]);
|
||||
btnRoot.addActionListener(event -> {
|
||||
if (!Pattern.matches(DOUBLE_OR_NUMBER_REGEX, inputScreen.getText()))
|
||||
return;
|
||||
|
||||
if (go) {
|
||||
typedValue = Math.sqrt(Double.parseDouble(inputScreen.getText()));
|
||||
if (Pattern.matches("[-]?[\\d]+[.][0]*", String.valueOf(typedValue))) {
|
||||
inputScreen.setText(String.valueOf((int) typedValue));
|
||||
} else {
|
||||
inputScreen.setText(String.valueOf(typedValue));
|
||||
}
|
||||
selectedOperator = '√';
|
||||
addToDisplay = false;
|
||||
}
|
||||
});
|
||||
btnRoot.setVisible(false);
|
||||
|
||||
btnPower = createButton("pow", columns[4], rows[2]);
|
||||
btnPower.addActionListener(event -> {
|
||||
if (!Pattern.matches(DOUBLE_OR_NUMBER_REGEX, inputScreen.getText()))
|
||||
return;
|
||||
|
||||
if (go) {
|
||||
typedValue = calculate(typedValue, Double.parseDouble(inputScreen.getText()), selectedOperator);
|
||||
if (Pattern.matches("[-]?[\\d]+[.][0]*", String.valueOf(typedValue))) {
|
||||
inputScreen.setText(String.valueOf((int) typedValue));
|
||||
} else {
|
||||
inputScreen.setText(String.valueOf(typedValue));
|
||||
}
|
||||
selectedOperator = '^';
|
||||
go = false;
|
||||
addToDisplay = false;
|
||||
} else {
|
||||
selectedOperator = '^';
|
||||
}
|
||||
});
|
||||
btnPower.setFont(new Font("Comic Sans MS", Font.PLAIN, 24));
|
||||
btnPower.setVisible(false);
|
||||
|
||||
btnLog = createButton("ln", columns[4], rows[3]);
|
||||
btnLog.addActionListener(event -> {
|
||||
if (!Pattern.matches(DOUBLE_OR_NUMBER_REGEX, inputScreen.getText()))
|
||||
return;
|
||||
|
||||
if (go) {
|
||||
typedValue = Math.log(Double.parseDouble(inputScreen.getText()));
|
||||
if (Pattern.matches("[-]?[\\d]+[.][0]*", String.valueOf(typedValue))) {
|
||||
inputScreen.setText(String.valueOf((int) typedValue));
|
||||
} else {
|
||||
inputScreen.setText(String.valueOf(typedValue));
|
||||
}
|
||||
selectedOperator = 'l';
|
||||
addToDisplay = false;
|
||||
}
|
||||
});
|
||||
btnLog.setVisible(false);
|
||||
}
|
||||
|
||||
private JComboBox<String> createComboBox(String[] items, int x, int y, String toolTip) {
|
||||
JComboBox<String> combo = new JComboBox<>(items);
|
||||
combo.setBounds(x, y, 140, 25);
|
||||
combo.setToolTipText(toolTip);
|
||||
combo.setCursor(new Cursor(Cursor.HAND_CURSOR));
|
||||
window.add(combo);
|
||||
|
||||
return combo;
|
||||
}
|
||||
|
||||
private JButton createButton(String label, int x, int y) {
|
||||
JButton btn = new JButton(label);
|
||||
btn.setBounds(x, y, BUTTON_WIDTH, BUTTON_HEIGHT);
|
||||
btn.setFont(new Font("Comic Sans MS", Font.PLAIN, 28));
|
||||
btn.setCursor(new Cursor(Cursor.HAND_CURSOR));
|
||||
btn.setFocusable(false);
|
||||
window.add(btn);
|
||||
|
||||
return btn;
|
||||
}
|
||||
|
||||
private void applyTheme(Theme theme) {
|
||||
window.getContentPane().setBackground(hex2Color(theme.getApplicationBackground()));
|
||||
|
||||
comboCalculatorType.setForeground(hex2Color(theme.getTextColor()));
|
||||
comboTheme.setForeground(hex2Color(theme.getTextColor()));
|
||||
inputScreen.setForeground(hex2Color(theme.getTextColor()));
|
||||
btn0.setForeground(hex2Color(theme.getTextColor()));
|
||||
btn1.setForeground(hex2Color(theme.getTextColor()));
|
||||
btn2.setForeground(hex2Color(theme.getTextColor()));
|
||||
btn3.setForeground(hex2Color(theme.getTextColor()));
|
||||
btn4.setForeground(hex2Color(theme.getTextColor()));
|
||||
btn5.setForeground(hex2Color(theme.getTextColor()));
|
||||
btn6.setForeground(hex2Color(theme.getTextColor()));
|
||||
btn7.setForeground(hex2Color(theme.getTextColor()));
|
||||
btn8.setForeground(hex2Color(theme.getTextColor()));
|
||||
btn9.setForeground(hex2Color(theme.getTextColor()));
|
||||
btnPoint.setForeground(hex2Color(theme.getTextColor()));
|
||||
btnC.setForeground(hex2Color(theme.getTextColor()));
|
||||
btnBack.setForeground(hex2Color(theme.getTextColor()));
|
||||
btnMod.setForeground(hex2Color(theme.getTextColor()));
|
||||
btnDiv.setForeground(hex2Color(theme.getTextColor()));
|
||||
btnMul.setForeground(hex2Color(theme.getTextColor()));
|
||||
btnSub.setForeground(hex2Color(theme.getTextColor()));
|
||||
btnAdd.setForeground(hex2Color(theme.getTextColor()));
|
||||
btnRoot.setForeground(hex2Color(theme.getTextColor()));
|
||||
btnLog.setForeground(hex2Color(theme.getTextColor()));
|
||||
btnPower.setForeground(hex2Color(theme.getTextColor()));
|
||||
btnEqual.setForeground(hex2Color(theme.getBtnEqualTextColor()));
|
||||
|
||||
comboCalculatorType.setBackground(hex2Color(theme.getApplicationBackground()));
|
||||
comboTheme.setBackground(hex2Color(theme.getApplicationBackground()));
|
||||
inputScreen.setBackground(hex2Color(theme.getApplicationBackground()));
|
||||
btn0.setBackground(hex2Color(theme.getNumbersBackground()));
|
||||
btn1.setBackground(hex2Color(theme.getNumbersBackground()));
|
||||
btn2.setBackground(hex2Color(theme.getNumbersBackground()));
|
||||
btn3.setBackground(hex2Color(theme.getNumbersBackground()));
|
||||
btn4.setBackground(hex2Color(theme.getNumbersBackground()));
|
||||
btn5.setBackground(hex2Color(theme.getNumbersBackground()));
|
||||
btn6.setBackground(hex2Color(theme.getNumbersBackground()));
|
||||
btn7.setBackground(hex2Color(theme.getNumbersBackground()));
|
||||
btn8.setBackground(hex2Color(theme.getNumbersBackground()));
|
||||
btn9.setBackground(hex2Color(theme.getNumbersBackground()));
|
||||
btnPoint.setBackground(hex2Color(theme.getNumbersBackground()));
|
||||
btnC.setBackground(hex2Color(theme.getOperatorBackground()));
|
||||
btnBack.setBackground(hex2Color(theme.getOperatorBackground()));
|
||||
btnMod.setBackground(hex2Color(theme.getOperatorBackground()));
|
||||
btnDiv.setBackground(hex2Color(theme.getOperatorBackground()));
|
||||
btnMul.setBackground(hex2Color(theme.getOperatorBackground()));
|
||||
btnSub.setBackground(hex2Color(theme.getOperatorBackground()));
|
||||
btnAdd.setBackground(hex2Color(theme.getOperatorBackground()));
|
||||
btnRoot.setBackground(hex2Color(theme.getOperatorBackground()));
|
||||
btnLog.setBackground(hex2Color(theme.getOperatorBackground()));
|
||||
btnPower.setBackground(hex2Color(theme.getOperatorBackground()));
|
||||
btnEqual.setBackground(hex2Color(theme.getBtnEqualBackground()));
|
||||
}
|
||||
}
|
||||
138
pr_agent/static_analysis/tests/example_files/match.ts
Normal file
138
pr_agent/static_analysis/tests/example_files/match.ts
Normal file
@ -0,0 +1,138 @@
|
||||
// @ts-ignore
|
||||
import { Pattern } from './types/Pattern'; // @ts-ignore
|
||||
import { Match } from './types/Match'; // @ts-ignore
|
||||
import * as symbols from './internals/symbols'; // @ts-ignore
|
||||
import { matchPattern } from './internals/helpers'; // @ts-ignore
|
||||
|
||||
type MatchState<output> =
|
||||
| { matched: true; value: output }
|
||||
| { matched: false; value: undefined };
|
||||
|
||||
const unmatched: MatchState<never> = {
|
||||
matched: false,
|
||||
value: undefined,
|
||||
};
|
||||
|
||||
/**
|
||||
* `match` creates a **pattern matching expression**.
|
||||
* * Use `.with(pattern, handler)` to pattern match on the input.
|
||||
* * Use `.exhaustive()` or `.otherwise(() => defaultValue)` to end the expression and get the result.
|
||||
*
|
||||
* [Read the documentation for `match` on GitHub](https://github.com/gvergnaud/ts-pattern#match)
|
||||
*
|
||||
* @example
|
||||
* declare let input: "A" | "B";
|
||||
*
|
||||
* return match(input)
|
||||
* .with("A", () => "It's an A!")
|
||||
* .with("B", () => "It's a B!")
|
||||
* .exhaustive();
|
||||
*
|
||||
*/
|
||||
export function match<const input, output = symbols.unset>(
|
||||
value: input
|
||||
): Match<input, output> {
|
||||
return new MatchExpression(value, unmatched) as any;
|
||||
}
|
||||
|
||||
/**
|
||||
* This class represents a match expression. It follows the
|
||||
* builder pattern, we chain methods to add features to the expression
|
||||
* until we call `.exhaustive`, `.otherwise` or the unsafe `.run`
|
||||
* method to execute it.
|
||||
*
|
||||
* The types of this class aren't public, the public type definition
|
||||
* can be found in src/types/Match.ts.
|
||||
*/
|
||||
class MatchExpression<input, output> {
|
||||
constructor(private input: input, private state: MatchState<output>) {}
|
||||
|
||||
with(...args: any[]): MatchExpression<input, output> {
|
||||
if (this.state.matched) return this;
|
||||
|
||||
const handler: (selection: unknown, value: input) => output =
|
||||
args[args.length - 1];
|
||||
|
||||
const patterns: Pattern<input>[] = [args[0]];
|
||||
let predicate: ((value: input) => unknown) | undefined = undefined;
|
||||
|
||||
if (args.length === 3 && typeof args[1] === 'function') {
|
||||
// case with guard as second argument
|
||||
patterns.push(args[0]);
|
||||
predicate = args[1];
|
||||
} else if (args.length > 2) {
|
||||
// case with several patterns
|
||||
patterns.push(...args.slice(1, args.length - 1));
|
||||
}
|
||||
|
||||
let hasSelections = false;
|
||||
let selected: Record<string, unknown> = {};
|
||||
const select = (key: string, value: unknown) => {
|
||||
hasSelections = true;
|
||||
selected[key] = value;
|
||||
};
|
||||
|
||||
const matched =
|
||||
patterns.some((pattern) => matchPattern(pattern, this.input, select)) &&
|
||||
(predicate ? Boolean(predicate(this.input)) : true);
|
||||
|
||||
const selections = hasSelections
|
||||
? symbols.anonymousSelectKey in selected
|
||||
? selected[symbols.anonymousSelectKey]
|
||||
: selected
|
||||
: this.input;
|
||||
|
||||
const state = matched
|
||||
? {
|
||||
matched: true as const,
|
||||
value: handler(selections, this.input),
|
||||
}
|
||||
: unmatched;
|
||||
|
||||
return new MatchExpression(this.input, state);
|
||||
}
|
||||
|
||||
when(
|
||||
predicate: (value: input) => unknown,
|
||||
handler: (selection: input, value: input) => output
|
||||
): MatchExpression<input, output> {
|
||||
if (this.state.matched) return this;
|
||||
|
||||
const matched = Boolean(predicate(this.input));
|
||||
|
||||
return new MatchExpression<input, output>(
|
||||
this.input,
|
||||
matched
|
||||
? { matched: true, value: handler(this.input, this.input) }
|
||||
: unmatched
|
||||
);
|
||||
}
|
||||
|
||||
otherwise(handler: (value: input) => output): output {
|
||||
if (this.state.matched) return this.state.value;
|
||||
return handler(this.input);
|
||||
}
|
||||
|
||||
exhaustive(): output {
|
||||
return this.run();
|
||||
}
|
||||
|
||||
run(): output {
|
||||
if (this.state.matched) return this.state.value;
|
||||
|
||||
let displayedValue;
|
||||
try {
|
||||
displayedValue = JSON.stringify(this.input);
|
||||
} catch (e) {
|
||||
displayedValue = this.input;
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
`Pattern matching error: no pattern matches value ${displayedValue}`
|
||||
);
|
||||
}
|
||||
|
||||
returnType() {
|
||||
return this;
|
||||
}
|
||||
}
|
||||
43
pr_agent/static_analysis/tests/test_file_summary.py
Normal file
43
pr_agent/static_analysis/tests/test_file_summary.py
Normal file
@ -0,0 +1,43 @@
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from pr_agent.static_analysis.src.file_summary import FileSummary
|
||||
|
||||
|
||||
class TestFileSummary:
|
||||
def setup_method(self):
|
||||
self.base_path = Path(__file__).parent
|
||||
self.project_base_path = Path(__file__).parent.parent.parent.parent
|
||||
|
||||
def test_file_summary_cpp(self):
|
||||
fname = os.path.join(self.base_path, 'example_files/AES.cpp')
|
||||
if not os.path.exists(fname):
|
||||
print(f"File {fname} does not exist")
|
||||
return False
|
||||
fname_summary = FileSummary(fname, self.project_base_path, parent_context=False, child_context=False,
|
||||
header_max=0)
|
||||
output = fname_summary.summarize()
|
||||
expected_output = '\npr_agent/static_analysis/tests/example_files/AES.cpp:\n...⋮...\n 32│namespace Common::AES\n 33│{\n...⋮...\n 36│template <Mode AesMode>\n 37│class ContextGeneric final : public Context\n 38│{\n 39│public:\n 40│ ContextGeneric(const u8* key)\n...⋮...\n 49│ virtual bool Crypt(const u8* iv, u8* iv_out, const u8* buf_in, u8* buf_out,\n...⋮...\n 70│#if defined(_M_X86_64)\n 71│\n...⋮...\n 84│template <Mode AesMode>\n 85│class ContextAESNI final : public Context\n 86│{\n 87│ static inline __m128i Aes128KeygenAssistFinish(__m128i key, __m128i kga)\n...⋮...\n103│ inline constexpr void StoreRoundKey(__m128i rk)\n...⋮...\n119│ inline constexpr __m128i Aes128Keygen(__m128i rk)\n...⋮...\n127│ ContextAESNI(const u8* key)\n...⋮...\n144│ inline void CryptBlock(__m128i* iv, const u8* buf_in, u8* buf_out) const\n...⋮...\n178│ inline void DecryptPipelined(__m128i* iv, const u8* buf_in, u8* buf_out) const\n...⋮...\n209│ virtual bool Crypt(const u8* iv, u8* iv_out, const u8* buf_in, u8* buf_out,\n...⋮...\n259│#if defined(_M_ARM_64)\n260│\n261│template <Mode AesMode>\n262│class ContextNeon final : public Context\n263│{\n264│public:\n265│ template <size_t RoundIdx>\n266│ inline constexpr void StoreRoundKey(const u32* rk)\n...⋮...\n281│ ContextNeon(const u8* key)\n...⋮...\n322│ inline void CryptBlock(uint8x16_t* iv, const u8* buf_in, u8* buf_out) const\n...⋮...\n353│ virtual bool Crypt(const u8* iv, u8* iv_out, const u8* buf_in, u8* buf_out,\n...⋮...\n381│template <Mode AesMode>\n382│std::unique_ptr<Context> CreateContext(const u8* key)\n...⋮...\n402│std::unique_ptr<Context> CreateContextEncrypt(const u8* key)\n...⋮...\n407│std::unique_ptr<Context> CreateContextDecrypt(const u8* key)\n...⋮...\n413│void CryptOFB(const u8* key, const u8* iv, u8* iv_out, const u8* buf_in, u8* buf_out, size_t size)\n...⋮...\n'
|
||||
assert output == expected_output
|
||||
|
||||
def test_file_typescript(self):
|
||||
fname = os.path.join(self.base_path, 'example_files/match.ts')
|
||||
if not os.path.exists(fname):
|
||||
print(f"File {fname} does not exist")
|
||||
return False
|
||||
fname_summary = FileSummary(fname, self.project_base_path, parent_context=False, child_context=False,
|
||||
header_max=0)
|
||||
output = fname_summary.summarize()
|
||||
expected_output = '\npr_agent/static_analysis/tests/example_files/match.ts:\n...⋮...\n 6│type MatchState<output> =\n...⋮...\n 31│export function match<const input, output = symbols.unset>(\n...⋮...\n 46│class MatchExpression<input, output> {\n 47│ constructor(private input: input, private state: MatchState<output>) {}\n 48│\n 49│ with(...args: any[]): MatchExpression<input, output> {\n...⋮...\n 94│ when(\n...⋮...\n110│ otherwise(handler: (value: input) => output): output {\n...⋮...\n115│ exhaustive(): output {\n...⋮...\n119│ run(): output {\n...⋮...\n134│ returnType() {\n...⋮...\n'
|
||||
assert output == expected_output
|
||||
|
||||
def test_file_java(self):
|
||||
fname = os.path.join(self.base_path, 'example_files/calc.java')
|
||||
if not os.path.exists(fname):
|
||||
print(f"File {fname} does not exist")
|
||||
return False
|
||||
fname_summary = FileSummary(fname, self.project_base_path, parent_context=False, child_context=False,
|
||||
header_max=0)
|
||||
output = fname_summary.summarize()
|
||||
expected_output = '\npr_agent/static_analysis/tests/example_files/calc.java:\n...⋮...\n 16│public class CalculatorUI {\n 17│\n...⋮...\n 84│ public double calculate(double firstNumber, double secondNumber, char operator) {\n...⋮...\n103│ private void initThemeSelector() {\n...⋮...\n118│ private void initInputScreen(int[] columns, int[] rows) {\n...⋮...\n127│ private void initCalculatorTypeSelector() {\n...⋮...\n151│ private void initButtons(int[] columns, int[] rows) {\n...⋮...\n510│ private JComboBox<String> createComboBox(String[] items, int x, int y, String toolTip) {\n...⋮...\n520│ private JButton createButton(String label, int x, int y) {\n...⋮...\n531│ private void applyTheme(Theme theme) {\n...⋮...\n'
|
||||
assert output == expected_output
|
||||
@ -20,8 +20,6 @@ from pr_agent.git_providers import get_git_provider, GithubProvider, get_git_pro
|
||||
from pr_agent.git_providers.git_provider import get_main_pr_language
|
||||
from pr_agent.log import get_logger
|
||||
from pr_agent.servers.help import HelpMessage
|
||||
from pr_agent.tools.ticket_pr_compliance_check import extract_ticket_links_from_pr_description, extract_tickets, \
|
||||
extract_and_cache_pr_tickets
|
||||
|
||||
|
||||
class PRDescription:
|
||||
@ -40,7 +38,6 @@ class PRDescription:
|
||||
self.git_provider.get_languages(), self.git_provider.get_files()
|
||||
)
|
||||
self.pr_id = self.git_provider.get_pr_id()
|
||||
self.keys_fix = ["filename:", "language:", "changes_summary:", "changes_title:", "description:", "title:"]
|
||||
|
||||
if get_settings().pr_description.enable_semantic_files_types and not self.git_provider.is_supported(
|
||||
"gfm_markdown"):
|
||||
@ -63,7 +60,6 @@ class PRDescription:
|
||||
"enable_custom_labels": get_settings().config.enable_custom_labels,
|
||||
"custom_labels_class": "", # will be filled if necessary in 'set_custom_labels' function
|
||||
"enable_semantic_files_types": get_settings().pr_description.enable_semantic_files_types,
|
||||
"related_tickets": "",
|
||||
}
|
||||
|
||||
self.user_description = self.git_provider.get_user_description()
|
||||
@ -91,9 +87,6 @@ class PRDescription:
|
||||
if get_settings().config.publish_output and not get_settings().config.get('is_auto_command', False):
|
||||
self.git_provider.publish_comment("Preparing PR description...", is_temporary=True)
|
||||
|
||||
# ticket extraction if exists
|
||||
await extract_and_cache_pr_tickets(self.git_provider, self.vars)
|
||||
|
||||
await retry_with_fallback_models(self._prepare_prediction, ModelType.TURBO)
|
||||
|
||||
if self.prediction:
|
||||
@ -133,7 +126,7 @@ class PRDescription:
|
||||
|
||||
if get_settings().config.publish_output:
|
||||
# publish labels
|
||||
if get_settings().pr_description.publish_labels and pr_labels and self.git_provider.is_supported("get_labels"):
|
||||
if get_settings().pr_description.publish_labels and self.git_provider.is_supported("get_labels"):
|
||||
original_labels = self.git_provider.get_pr_labels(update=True)
|
||||
get_logger().debug(f"original labels", artifact=original_labels)
|
||||
user_labels = get_user_labels(original_labels)
|
||||
@ -233,7 +226,7 @@ class PRDescription:
|
||||
file_description_str_list = []
|
||||
for i, result in enumerate(results):
|
||||
prediction_files = result.strip().removeprefix('```yaml').strip('`').strip()
|
||||
if load_yaml(prediction_files, keys_fix_yaml=self.keys_fix) and prediction_files.startswith('pr_files'):
|
||||
if load_yaml(prediction_files) and prediction_files.startswith('pr_files'):
|
||||
prediction_files = prediction_files.removeprefix('pr_files:').strip()
|
||||
file_description_str_list.append(prediction_files)
|
||||
else:
|
||||
@ -311,16 +304,16 @@ extra_file_yaml =
|
||||
|
||||
# final processing
|
||||
self.prediction = prediction_headers + "\n" + "pr_files:\n" + files_walkthrough
|
||||
if not load_yaml(self.prediction, keys_fix_yaml=self.keys_fix):
|
||||
if not load_yaml(self.prediction):
|
||||
get_logger().error(f"Error getting valid YAML in large PR handling for describe {self.pr_id}")
|
||||
if load_yaml(prediction_headers, keys_fix_yaml=self.keys_fix):
|
||||
if load_yaml(prediction_headers):
|
||||
get_logger().debug(f"Using only headers for describe {self.pr_id}")
|
||||
self.prediction = prediction_headers
|
||||
|
||||
async def extend_additional_files(self, remaining_files_list) -> str:
|
||||
prediction = self.prediction
|
||||
try:
|
||||
original_prediction_dict = load_yaml(self.prediction, keys_fix_yaml=self.keys_fix)
|
||||
original_prediction_dict = load_yaml(self.prediction)
|
||||
prediction_extra = "pr_files:"
|
||||
for file in remaining_files_list:
|
||||
extra_file_yaml = f"""\
|
||||
@ -334,12 +327,12 @@ extra_file_yaml =
|
||||
additional files (token-limit)
|
||||
"""
|
||||
prediction_extra = prediction_extra + "\n" + extra_file_yaml.strip()
|
||||
prediction_extra_dict = load_yaml(prediction_extra, keys_fix_yaml=self.keys_fix)
|
||||
prediction_extra_dict = load_yaml(prediction_extra)
|
||||
# merge the two dictionaries
|
||||
if isinstance(original_prediction_dict, dict) and isinstance(prediction_extra_dict, dict):
|
||||
original_prediction_dict["pr_files"].extend(prediction_extra_dict["pr_files"])
|
||||
new_yaml = yaml.dump(original_prediction_dict)
|
||||
if load_yaml(new_yaml, keys_fix_yaml=self.keys_fix):
|
||||
if load_yaml(new_yaml):
|
||||
prediction = new_yaml
|
||||
return prediction
|
||||
except Exception as e:
|
||||
@ -368,7 +361,7 @@ extra_file_yaml =
|
||||
|
||||
def _prepare_data(self):
|
||||
# Load the AI prediction data into a dictionary
|
||||
self.data = load_yaml(self.prediction.strip(), keys_fix_yaml=self.keys_fix)
|
||||
self.data = load_yaml(self.prediction.strip())
|
||||
|
||||
if get_settings().pr_description.add_original_user_description and self.user_description:
|
||||
self.data["User Description"] = self.user_description
|
||||
|
||||
@ -1,17 +1,19 @@
|
||||
import os
|
||||
import traceback
|
||||
import zipfile
|
||||
import tempfile
|
||||
import copy
|
||||
from functools import partial
|
||||
from pathlib import Path
|
||||
|
||||
from jinja2 import Environment, StrictUndefined
|
||||
|
||||
from pr_agent.algo import MAX_TOKENS
|
||||
from pr_agent.algo.ai_handlers.base_ai_handler import BaseAiHandler
|
||||
from pr_agent.algo.ai_handlers.litellm_ai_handler import LiteLLMAIHandler
|
||||
from pr_agent.algo.pr_processing import retry_with_fallback_models
|
||||
from pr_agent.algo.token_handler import TokenHandler
|
||||
from pr_agent.algo.utils import ModelType, load_yaml, clip_tokens
|
||||
from pr_agent.algo.utils import ModelType, load_yaml
|
||||
from pr_agent.config_loader import get_settings
|
||||
from pr_agent.git_providers import GithubProvider, BitbucketServerProvider, \
|
||||
from pr_agent.git_providers import get_git_provider, GithubProvider, BitbucketServerProvider, \
|
||||
get_git_provider_with_context
|
||||
from pr_agent.log import get_logger
|
||||
|
||||
@ -65,6 +67,83 @@ class PRHelpMessage:
|
||||
question_str = ""
|
||||
return question_str
|
||||
|
||||
def get_sim_results_from_s3_db(self, embeddings):
|
||||
get_logger().info("Loading the S3 index...")
|
||||
sim_results = []
|
||||
try:
|
||||
from langchain_chroma import Chroma
|
||||
from urllib import request
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# Define the local file path within the temporary directory
|
||||
local_file_path = os.path.join(temp_dir, 'chroma_db.zip')
|
||||
|
||||
bucket = 'pr-agent'
|
||||
file_name = 'chroma_db.zip'
|
||||
s3_url = f'https://{bucket}.s3.amazonaws.com/{file_name}'
|
||||
request.urlretrieve(s3_url, local_file_path)
|
||||
|
||||
# # Download the file from S3 to the temporary directory
|
||||
# s3 = boto3.client('s3')
|
||||
# s3.download_file(bucket, file_name, local_file_path)
|
||||
|
||||
# Extract the contents of the zip file
|
||||
with zipfile.ZipFile(local_file_path, 'r') as zip_ref:
|
||||
zip_ref.extractall(temp_dir)
|
||||
|
||||
vectorstore = Chroma(persist_directory=temp_dir + "/chroma_db",
|
||||
embedding_function=embeddings)
|
||||
sim_results = vectorstore.similarity_search_with_score(self.question_str, k=self.num_retrieved_snippets)
|
||||
except Exception as e:
|
||||
get_logger().error(f"Error while getting sim from S3: {e}",
|
||||
artifact={"traceback": traceback.format_exc()})
|
||||
return sim_results
|
||||
|
||||
def get_sim_results_from_local_db(self, embeddings):
|
||||
get_logger().info("Loading the local index...")
|
||||
sim_results = []
|
||||
try:
|
||||
from langchain_chroma import Chroma
|
||||
get_logger().info("Loading the Chroma index...")
|
||||
db_path = "./docs/chroma_db.zip"
|
||||
if not os.path.exists(db_path):
|
||||
db_path= "/app/docs/chroma_db.zip"
|
||||
if not os.path.exists(db_path):
|
||||
get_logger().error("Local db not found")
|
||||
return sim_results
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
|
||||
# Extract the ZIP file
|
||||
with zipfile.ZipFile(db_path, 'r') as zip_ref:
|
||||
zip_ref.extractall(temp_dir)
|
||||
|
||||
vectorstore = Chroma(persist_directory=temp_dir + "/chroma_db",
|
||||
embedding_function=embeddings)
|
||||
|
||||
# Do similarity search
|
||||
sim_results = vectorstore.similarity_search_with_score(self.question_str, k=self.num_retrieved_snippets)
|
||||
except Exception as e:
|
||||
get_logger().error(f"Error while getting sim from local db: {e}",
|
||||
artifact={"traceback": traceback.format_exc()})
|
||||
return sim_results
|
||||
|
||||
def get_sim_results_from_pinecone_db(self, embeddings):
|
||||
get_logger().info("Loading the Pinecone index...")
|
||||
sim_results = []
|
||||
try:
|
||||
from langchain_pinecone import PineconeVectorStore
|
||||
INDEX_NAME = "pr-agent-docs"
|
||||
vectorstore = PineconeVectorStore(
|
||||
index_name=INDEX_NAME, embedding=embeddings,
|
||||
pinecone_api_key=get_settings().pinecone.api_key
|
||||
)
|
||||
|
||||
# Do similarity search
|
||||
sim_results = vectorstore.similarity_search_with_score(self.question_str, k=self.num_retrieved_snippets)
|
||||
except Exception as e:
|
||||
get_logger().error(f"Error while getting sim from Pinecone db: {e}",
|
||||
artifact={"traceback": traceback.format_exc()})
|
||||
return sim_results
|
||||
|
||||
async def run(self):
|
||||
try:
|
||||
if self.question_str:
|
||||
@ -78,49 +157,38 @@ class PRHelpMessage:
|
||||
get_logger().error("The `Help` tool chat feature requires an OpenAI API key for calculating embeddings")
|
||||
return
|
||||
|
||||
# current path
|
||||
docs_path= Path(__file__).parent.parent.parent / 'docs' / 'docs'
|
||||
# get all the 'md' files inside docs_path and its subdirectories
|
||||
md_files = list(docs_path.glob('**/*.md'))
|
||||
folders_to_exclude = ['/finetuning_benchmark/']
|
||||
files_to_exclude = {'EXAMPLE_BEST_PRACTICE.md', 'compression_strategy.md', '/docs/overview/index.md'}
|
||||
md_files = [file for file in md_files if not any(folder in str(file) for folder in folders_to_exclude) and not any(file.name == file_to_exclude for file_to_exclude in files_to_exclude)]
|
||||
# Initialize embeddings
|
||||
from langchain_openai import OpenAIEmbeddings
|
||||
embeddings = OpenAIEmbeddings(model="text-embedding-3-small",
|
||||
api_key=get_settings().openai.key)
|
||||
|
||||
# sort the 'md_files' so that 'priority_files' will be at the top
|
||||
priority_files_strings = ['/docs/index.md', '/usage-guide', 'tools/describe.md', 'tools/review.md',
|
||||
'tools/improve.md', '/faq']
|
||||
md_files_priority = [file for file in md_files if
|
||||
any(priority_string in str(file) for priority_string in priority_files_strings)]
|
||||
md_files_not_priority = [file for file in md_files if file not in md_files_priority]
|
||||
md_files = md_files_priority + md_files_not_priority
|
||||
# Get similar snippets via similarity search
|
||||
if get_settings().pr_help.force_local_db:
|
||||
sim_results = self.get_sim_results_from_local_db(embeddings)
|
||||
elif get_settings().get('pinecone.api_key'):
|
||||
sim_results = self.get_sim_results_from_pinecone_db(embeddings)
|
||||
else:
|
||||
sim_results = self.get_sim_results_from_s3_db(embeddings)
|
||||
if not sim_results:
|
||||
get_logger().info("Failed to load the S3 index. Loading the local index...")
|
||||
sim_results = self.get_sim_results_from_local_db(embeddings)
|
||||
if not sim_results:
|
||||
get_logger().error("Failed to retrieve similar snippets. Exiting...")
|
||||
return
|
||||
|
||||
docs_prompt = ""
|
||||
for file in md_files:
|
||||
try:
|
||||
with open(file, 'r') as f:
|
||||
file_path = str(file).replace(str(docs_path), '')
|
||||
docs_prompt += f"\n==file name==\n\n{file_path}\n\n==file content==\n\n{f.read().strip()}\n=========\n\n"
|
||||
except Exception as e:
|
||||
get_logger().error(f"Error while reading the file {file}: {e}")
|
||||
token_count = self.token_handler.count_tokens(docs_prompt)
|
||||
get_logger().debug(f"Token count of full documentation website: {token_count}")
|
||||
|
||||
model = get_settings().config.model
|
||||
max_tokens_full = MAX_TOKENS[model] # note - here we take the actual max tokens, without any reductions. we do aim to get the full documentation website in the prompt
|
||||
delta_output = 2000
|
||||
if token_count > max_tokens_full - delta_output:
|
||||
get_logger().info(f"Token count {token_count} exceeds the limit {max_tokens_full - delta_output}. Skipping the PR Help message.")
|
||||
docs_prompt = clip_tokens(docs_prompt, max_tokens_full - delta_output)
|
||||
self.vars['snippets'] = docs_prompt.strip()
|
||||
# Prepare relevant snippets
|
||||
relevant_pages_full, relevant_snippets_full_header, relevant_snippets_str =\
|
||||
await self.prepare_relevant_snippets(sim_results)
|
||||
self.vars['snippets'] = relevant_snippets_str.strip()
|
||||
|
||||
# run the AI model
|
||||
response = await retry_with_fallback_models(self._prepare_prediction, model_type=ModelType.REGULAR)
|
||||
response_yaml = load_yaml(response)
|
||||
response_str = response_yaml.get('response')
|
||||
relevant_sections = response_yaml.get('relevant_sections')
|
||||
relevant_snippets_numbers = response_yaml.get('relevant_snippets')
|
||||
|
||||
if not relevant_sections:
|
||||
get_logger().info(f"Could not find relevant answer for the question: {self.question_str}")
|
||||
if not relevant_snippets_numbers:
|
||||
get_logger().info(f"Could not find relevant snippets for the question: {self.question_str}")
|
||||
if get_settings().config.publish_output:
|
||||
answer_str = f"### Question: \n{self.question_str}\n\n"
|
||||
answer_str += f"### Answer:\n\n"
|
||||
@ -134,15 +202,16 @@ class PRHelpMessage:
|
||||
answer_str += f"### Question: \n{self.question_str}\n\n"
|
||||
answer_str += f"### Answer:\n{response_str.strip()}\n\n"
|
||||
answer_str += f"#### Relevant Sources:\n\n"
|
||||
base_path = "https://qodo-merge-docs.qodo.ai/"
|
||||
for section in relevant_sections:
|
||||
file = section.get('file_name').strip().removesuffix('.md')
|
||||
if str(section['relevant_section_header_string']).strip():
|
||||
markdown_header = section['relevant_section_header_string'].strip().strip('#').strip().lower().replace(' ', '-').replace("'", '').replace('(', '').replace(')', '').replace(',', '').replace('.', '').replace('?', '').replace('!', '')
|
||||
answer_str += f"> - {base_path}{file}#{markdown_header}\n"
|
||||
else:
|
||||
answer_str += f"> - {base_path}{file}\n"
|
||||
|
||||
paged_published = []
|
||||
for page in relevant_snippets_numbers:
|
||||
page = int(page - 1)
|
||||
if page < len(relevant_pages_full) and page >= 0:
|
||||
if relevant_pages_full[page] in paged_published:
|
||||
continue
|
||||
link = f"{relevant_pages_full[page]}{relevant_snippets_full_header[page]}"
|
||||
# answer_str += f"> - [{relevant_pages_full[page]}]({link})\n"
|
||||
answer_str += f"> - {link}\n"
|
||||
paged_published.append(relevant_pages_full[page])
|
||||
|
||||
# publish the answer
|
||||
if get_settings().config.publish_output:
|
||||
|
||||
@ -1,6 +1,5 @@
|
||||
import copy
|
||||
import datetime
|
||||
import traceback
|
||||
from collections import OrderedDict
|
||||
from functools import partial
|
||||
from typing import List, Tuple
|
||||
@ -16,7 +15,6 @@ from pr_agent.git_providers import get_git_provider, get_git_provider_with_conte
|
||||
from pr_agent.git_providers.git_provider import IncrementalPR, get_main_pr_language
|
||||
from pr_agent.log import get_logger
|
||||
from pr_agent.servers.help import HelpMessage
|
||||
from pr_agent.tools.ticket_pr_compliance_check import extract_tickets, extract_and_cache_pr_tickets
|
||||
|
||||
|
||||
class PRReviewer:
|
||||
@ -86,7 +84,6 @@ class PRReviewer:
|
||||
"custom_labels": "",
|
||||
"enable_custom_labels": get_settings().config.enable_custom_labels,
|
||||
"is_ai_metadata": get_settings().get("config.enable_ai_metadata", False),
|
||||
"related_tickets": get_settings().get('related_tickets', []),
|
||||
}
|
||||
|
||||
self.token_handler = TokenHandler(
|
||||
@ -124,9 +121,6 @@ class PRReviewer:
|
||||
'config': dict(get_settings().config)}
|
||||
get_logger().debug("Relevant configs", artifacts=relevant_configs)
|
||||
|
||||
# ticket extraction if exists
|
||||
await extract_and_cache_pr_tickets(self.git_provider, self.vars)
|
||||
|
||||
if self.incremental.is_incremental and hasattr(self.git_provider, "unreviewed_files_set") and not self.git_provider.unreviewed_files_set:
|
||||
get_logger().info(f"Incremental review is enabled for {self.pr_url} but there are no new files")
|
||||
previous_review_url = ""
|
||||
@ -213,7 +207,7 @@ class PRReviewer:
|
||||
first_key = 'review'
|
||||
last_key = 'security_concerns'
|
||||
data = load_yaml(self.prediction.strip(),
|
||||
keys_fix_yaml=["ticket_compliance_check", "estimated_effort_to_review_[1-5]:", "security_concerns:", "key_issues_to_review:",
|
||||
keys_fix_yaml=["estimated_effort_to_review_[1-5]:", "security_concerns:", "key_issues_to_review:",
|
||||
"relevant_file:", "relevant_line:", "suggestion:"],
|
||||
first_key=first_key, last_key=last_key)
|
||||
github_action_output(data, 'review')
|
||||
@ -288,7 +282,7 @@ class PRReviewer:
|
||||
first_key = 'review'
|
||||
last_key = 'security_concerns'
|
||||
data = load_yaml(self.prediction.strip(),
|
||||
keys_fix_yaml=["ticket_compliance_check", "estimated_effort_to_review_[1-5]:", "security_concerns:", "key_issues_to_review:",
|
||||
keys_fix_yaml=["estimated_effort_to_review_[1-5]:", "security_concerns:", "key_issues_to_review:",
|
||||
"relevant_file:", "relevant_line:", "suggestion:"],
|
||||
first_key=first_key, last_key=last_key)
|
||||
comments: List[str] = []
|
||||
@ -407,16 +401,7 @@ class PRReviewer:
|
||||
review_labels = []
|
||||
if get_settings().pr_reviewer.enable_review_labels_effort:
|
||||
estimated_effort = data['review']['estimated_effort_to_review_[1-5]']
|
||||
estimated_effort_number = 0
|
||||
if isinstance(estimated_effort, str):
|
||||
try:
|
||||
estimated_effort_number = int(estimated_effort.split(',')[0])
|
||||
except ValueError:
|
||||
get_logger().warning(f"Invalid estimated_effort value: {estimated_effort}")
|
||||
elif isinstance(estimated_effort, int):
|
||||
estimated_effort_number = estimated_effort
|
||||
else:
|
||||
get_logger().warning(f"Unexpected type for estimated_effort: {type(estimated_effort)}")
|
||||
estimated_effort_number = int(estimated_effort.split(',')[0])
|
||||
if 1 <= estimated_effort_number <= 5: # 1, because ...
|
||||
review_labels.append(f'Review effort [1-5]: {estimated_effort_number}')
|
||||
if get_settings().pr_reviewer.enable_review_labels_security and get_settings().pr_reviewer.require_security_review:
|
||||
|
||||
@ -1,126 +0,0 @@
|
||||
import re
|
||||
import traceback
|
||||
|
||||
from pr_agent.config_loader import get_settings
|
||||
from pr_agent.git_providers import GithubProvider
|
||||
from pr_agent.log import get_logger
|
||||
|
||||
# Compile the regex pattern once, outside the function
|
||||
GITHUB_TICKET_PATTERN = re.compile(
|
||||
r'(https://github[^/]+/[^/]+/[^/]+/issues/\d+)|(\b(\w+)/(\w+)#(\d+)\b)|(#\d+)'
|
||||
)
|
||||
|
||||
def find_jira_tickets(text):
|
||||
# Regular expression patterns for JIRA tickets
|
||||
patterns = [
|
||||
r'\b[A-Z]{2,10}-\d{1,7}\b', # Standard JIRA ticket format (e.g., PROJ-123)
|
||||
r'(?:https?://[^\s/]+/browse/)?([A-Z]{2,10}-\d{1,7})\b' # JIRA URL or just the ticket
|
||||
]
|
||||
|
||||
tickets = set()
|
||||
for pattern in patterns:
|
||||
matches = re.findall(pattern, text)
|
||||
for match in matches:
|
||||
if isinstance(match, tuple):
|
||||
# If it's a tuple (from the URL pattern), take the last non-empty group
|
||||
ticket = next((m for m in reversed(match) if m), None)
|
||||
else:
|
||||
ticket = match
|
||||
if ticket:
|
||||
tickets.add(ticket)
|
||||
|
||||
return list(tickets)
|
||||
|
||||
|
||||
def extract_ticket_links_from_pr_description(pr_description, repo_path, base_url_html='https://github.com'):
|
||||
"""
|
||||
Extract all ticket links from PR description
|
||||
"""
|
||||
github_tickets = set()
|
||||
try:
|
||||
# Use the updated pattern to find matches
|
||||
matches = GITHUB_TICKET_PATTERN.findall(pr_description)
|
||||
|
||||
for match in matches:
|
||||
if match[0]: # Full URL match
|
||||
github_tickets.add(match[0])
|
||||
elif match[1]: # Shorthand notation match: owner/repo#issue_number
|
||||
owner, repo, issue_number = match[2], match[3], match[4]
|
||||
github_tickets.add(f'{base_url_html.strip("/")}/{owner}/{repo}/issues/{issue_number}')
|
||||
else: # #123 format
|
||||
issue_number = match[5][1:] # remove #
|
||||
if issue_number.isdigit() and len(issue_number) < 5 and repo_path:
|
||||
github_tickets.add(f'{base_url_html.strip("/")}/{repo_path}/issues/{issue_number}')
|
||||
except Exception as e:
|
||||
get_logger().error(f"Error extracting tickets error= {e}",
|
||||
artifact={"traceback": traceback.format_exc()})
|
||||
|
||||
return list(github_tickets)
|
||||
|
||||
|
||||
async def extract_tickets(git_provider):
|
||||
MAX_TICKET_CHARACTERS = 10000
|
||||
try:
|
||||
if isinstance(git_provider, GithubProvider):
|
||||
user_description = git_provider.get_user_description()
|
||||
tickets = extract_ticket_links_from_pr_description(user_description, git_provider.repo, git_provider.base_url_html)
|
||||
tickets_content = []
|
||||
if tickets:
|
||||
for ticket in tickets:
|
||||
# extract ticket number and repo name
|
||||
repo_name, original_issue_number = git_provider._parse_issue_url(ticket)
|
||||
|
||||
# get the ticket object
|
||||
try:
|
||||
issue_main = git_provider.repo_obj.get_issue(original_issue_number)
|
||||
except Exception as e:
|
||||
get_logger().error(f"Error getting issue_main error= {e}",
|
||||
artifact={"traceback": traceback.format_exc()})
|
||||
continue
|
||||
|
||||
# clip issue_main.body max length
|
||||
issue_body_str = issue_main.body
|
||||
if not issue_body_str:
|
||||
issue_body_str = ""
|
||||
if len(issue_body_str) > MAX_TICKET_CHARACTERS:
|
||||
issue_body_str = issue_body_str[:MAX_TICKET_CHARACTERS] + "..."
|
||||
|
||||
# extract labels
|
||||
labels = []
|
||||
try:
|
||||
for label in issue_main.labels:
|
||||
if isinstance(label, str):
|
||||
labels.append(label)
|
||||
else:
|
||||
labels.append(label.name)
|
||||
except Exception as e:
|
||||
get_logger().error(f"Error extracting labels error= {e}",
|
||||
artifact={"traceback": traceback.format_exc()})
|
||||
tickets_content.append(
|
||||
{'ticket_id': issue_main.number,
|
||||
'ticket_url': ticket, 'title': issue_main.title, 'body': issue_body_str,
|
||||
'labels': ", ".join(labels)})
|
||||
return tickets_content
|
||||
|
||||
except Exception as e:
|
||||
get_logger().error(f"Error extracting tickets error= {e}",
|
||||
artifact={"traceback": traceback.format_exc()})
|
||||
|
||||
|
||||
async def extract_and_cache_pr_tickets(git_provider, vars):
|
||||
if get_settings().get('config.require_ticket_analysis_review', False):
|
||||
return
|
||||
related_tickets = get_settings().get('related_tickets', [])
|
||||
if not related_tickets:
|
||||
tickets_content = await extract_tickets(git_provider)
|
||||
if tickets_content:
|
||||
get_logger().info("Extracted tickets from PR description", artifact={"tickets": tickets_content})
|
||||
vars['related_tickets'] = tickets_content
|
||||
get_settings().set('related_tickets', tickets_content)
|
||||
else: # if tickets are already cached
|
||||
get_logger().info("Using cached tickets", artifact={"tickets": related_tickets})
|
||||
vars['related_tickets'] = related_tickets
|
||||
|
||||
|
||||
def check_tickets_relevancy():
|
||||
return True
|
||||
@ -1,5 +1,5 @@
|
||||
aiohttp==3.9.5
|
||||
anthropic[vertex]==0.37.1
|
||||
anthropic[vertex]==0.21.3
|
||||
atlassian-python-api==3.41.4
|
||||
azure-devops==7.1.0b3
|
||||
azure-identity==1.15.0
|
||||
@ -10,10 +10,10 @@ GitPython==3.1.41
|
||||
google-cloud-aiplatform==1.38.0
|
||||
google-cloud-storage==2.10.0
|
||||
Jinja2==3.1.2
|
||||
litellm==1.50.2
|
||||
litellm==1.43.13
|
||||
loguru==0.7.2
|
||||
msrest==0.7.1
|
||||
openai==1.52.1
|
||||
openai==1.46.0
|
||||
pytest==7.4.0
|
||||
PyGithub==1.59.*
|
||||
PyYAML==6.0.1
|
||||
@ -28,6 +28,18 @@ gunicorn==22.0.0
|
||||
pytest-cov==5.0.0
|
||||
pydantic==2.8.2
|
||||
html2text==2024.2.26
|
||||
# help bot
|
||||
langchain==0.3.0
|
||||
langchain-openai==0.2.0
|
||||
langchain-pinecone==0.2.0
|
||||
langchain-chroma==0.1.4
|
||||
chromadb==0.5.7
|
||||
#
|
||||
tree_sitter==0.21.3
|
||||
tree_sitter_languages==1.10.2
|
||||
grep_ast==0.3.3
|
||||
#Pygments==2.18.0
|
||||
|
||||
# Uncomment the following lines to enable the 'similar issue' tool
|
||||
# pinecone-client
|
||||
# pinecone-datasets @ git+https://github.com/mrT23/pinecone-datasets.git@main
|
||||
|
||||
Reference in New Issue
Block a user