Merge pull request #781 from koid/feature/support-bedrock-claude3

added support for bedrock/claude3
This commit is contained in:
Tal
2024-03-18 08:03:29 +02:00
committed by GitHub
5 changed files with 20 additions and 4 deletions

View File

@ -162,8 +162,9 @@ To use Amazon Bedrock and its foundational models, add the below configuration:
```
[config] # in configuration.toml
model = "anthropic.claude-v2"
fallback_models="anthropic.claude-instant-v1"
model="bedrock/anthropic.claude-3-sonnet-20240229-v1:0"
model_turbo="bedrock/anthropic.claude-3-sonnet-20240229-v1:0"
fallback_models=["bedrock/anthropic.claude-v2:1"]
[aws] # in .secrets.toml
bedrock_region = "us-east-1"
@ -171,6 +172,12 @@ bedrock_region = "us-east-1"
Note that you have to add access to foundational models before using them. Please refer to [this document](https://docs.aws.amazon.com/bedrock/latest/userguide/setting-up.html) for more details.
If you are using the claude-3 model, please configure the following settings as there are parameters incompatible with claude-3.
```
[litellm]
drop_params = true
```
AWS session is automatically authenticated from your environment, but you can also explicitly set `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables.