|
@@ -1,55 +1,43 @@
|
|
|
# Azure OpenAI LLM
|
|
# Azure OpenAI LLM
|
|
|
|
|
|
|
|
-## Completion
|
|
|
|
|
-
|
|
|
|
|
OpenHands uses LiteLLM for completion calls. You can find their documentation on Azure [here](https://docs.litellm.ai/docs/providers/azure).
|
|
OpenHands uses LiteLLM for completion calls. You can find their documentation on Azure [here](https://docs.litellm.ai/docs/providers/azure).
|
|
|
|
|
|
|
|
-### Azure openai configs
|
|
|
|
|
|
|
+## Azure OpenAI Configuration
|
|
|
|
|
|
|
|
-When running the OpenHands Docker image, you'll need to set the following environment variables using `-e`:
|
|
|
|
|
|
|
+When running OpenHands, you'll need to set the following environment variable using `-e` in the
|
|
|
|
|
+[docker run command](/modules/usage/getting-started#installation):
|
|
|
|
|
|
|
|
```
|
|
```
|
|
|
-LLM_BASE_URL="<azure-api-base-url>" # e.g. "https://openai-gpt-4-test-v-1.openai.azure.com/"
|
|
|
|
|
-LLM_API_KEY="<azure-api-key>"
|
|
|
|
|
-LLM_MODEL="azure/<your-gpt-deployment-name>"
|
|
|
|
|
-LLM_API_VERSION="<api-version>" # e.g. "2024-02-15-preview"
|
|
|
|
|
|
|
+LLM_API_VERSION="<api-version>" # e.g. "2023-05-15"
|
|
|
```
|
|
```
|
|
|
|
|
|
|
|
Example:
|
|
Example:
|
|
|
```bash
|
|
```bash
|
|
|
-docker run -it \
|
|
|
|
|
---pull=always \
|
|
|
|
|
--e SANDBOX_USER_ID=$(id -u) \
|
|
|
|
|
--e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
|
|
|
|
|
--e LLM_BASE_URL="x.openai.azure.com" \
|
|
|
|
|
--e LLM_API_VERSION="2024-02-15-preview" \
|
|
|
|
|
--v $WORKSPACE_BASE:/opt/workspace_base \
|
|
|
|
|
--v /var/run/docker.sock:/var/run/docker.sock \
|
|
|
|
|
--p 3000:3000 \
|
|
|
|
|
---add-host host.docker.internal:host-gateway \
|
|
|
|
|
---name openhands-app-$(date +%Y%m%d%H%M%S) \
|
|
|
|
|
-ghcr.io/all-hands-ai/openhands:main
|
|
|
|
|
|
|
+docker run -it --pull=always \
|
|
|
|
|
+ -e LLM_API_VERSION="2023-05-15"
|
|
|
|
|
+ ...
|
|
|
```
|
|
```
|
|
|
|
|
|
|
|
-You can also set the model and API key in the OpenHands UI through the Settings.
|
|
|
|
|
|
|
+Then set the following in the OpenHands UI through the Settings:
|
|
|
|
|
|
|
|
:::note
|
|
:::note
|
|
|
-You can find your ChatGPT deployment name on the deployments page in Azure. It could be the same with the chat model
|
|
|
|
|
-name (e.g. 'GPT4-1106-preview'), by default or initially set, but it doesn't have to be the same. Run OpenHands,
|
|
|
|
|
-and when you load it in the browser, go to Settings and set model as above: "azure/<your-actual-gpt-deployment-name>".
|
|
|
|
|
-If it's not in the list, you can open the Settings modal, switch to "Custom Model", and enter your model name.
|
|
|
|
|
|
|
+You will need your ChatGPT deployment name which can be found on the deployments page in Azure. This is referenced as
|
|
|
|
|
+<deployment-name> below.
|
|
|
:::
|
|
:::
|
|
|
|
|
|
|
|
|
|
+* Enable `Advanced Options`
|
|
|
|
|
+* `Custom Model` to azure/<deployment-name>
|
|
|
|
|
+* `Base URL` to your Azure API Base URL (Example: https://example-endpoint.openai.azure.com)
|
|
|
|
|
+* `API Key`
|
|
|
|
|
+
|
|
|
## Embeddings
|
|
## Embeddings
|
|
|
|
|
|
|
|
OpenHands uses llama-index for embeddings. You can find their documentation on Azure [here](https://docs.llamaindex.ai/en/stable/api_reference/embeddings/azure_openai/).
|
|
OpenHands uses llama-index for embeddings. You can find their documentation on Azure [here](https://docs.llamaindex.ai/en/stable/api_reference/embeddings/azure_openai/).
|
|
|
|
|
|
|
|
-### Azure openai configs
|
|
|
|
|
-
|
|
|
|
|
-The model used for Azure OpenAI embeddings is "text-embedding-ada-002".
|
|
|
|
|
-You need the correct deployment name for this model in your Azure account.
|
|
|
|
|
|
|
+### Azure OpenAI Configuration
|
|
|
|
|
|
|
|
-When running OpenHands in Docker, set the following environment variables using `-e`:
|
|
|
|
|
|
|
+When running OpenHands, set the following environment variables using `-e` in the
|
|
|
|
|
+[docker run command](/modules/usage/getting-started#installation):
|
|
|
|
|
|
|
|
```
|
|
```
|
|
|
LLM_EMBEDDING_MODEL="azureopenai"
|
|
LLM_EMBEDDING_MODEL="azureopenai"
|