> For the complete documentation index, see [llms.txt](https://docs.digitalapi.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.digitalapi.ai/api-gateway/plugin-reference/plugins-ai.md).
# AI plugins
Plugins in this category sit between consumers and large-language-model providers. They handle content moderation, prompt and response logging, token-budget enforcement, model routing across providers, and AI-specific rate limiting. Attach them at route scope on the routes that proxy to your LLM provider. The catalogue ships 10 AI plugins.
{% hint style="info" %}
**In this category:** [`ai-aliyun-content-moderation`](#ai-aliyun-content-moderation), [`ai-aws-content-moderation`](#ai-aws-content-moderation), [`ai-prompt-decorator`](#ai-prompt-decorator), [`ai-prompt-guard`](#ai-prompt-guard), [`ai-prompt-template`](#ai-prompt-template), [`ai-proxy`](#ai-proxy), [`ai-proxy-multi`](#ai-proxy-multi), [`ai-rag`](#ai-rag), [`ai-rate-limiting`](#ai-rate-limiting), [`ai-request-rewrite`](#ai-request-rewrite)
{% endhint %}
### `ai-aliyun-content-moderation`
The `ai-aliyun-content-moderation` plugin integrates with Aliyun's content moderation service to check both request and response content for inappropriate material when working with LLMs. It supports both real-time streaming checks and final packet moderation.
{% hint style="info" %}
**Priority** `1029` · **Category** AI · **Required** `endpoint`, `region_id`, `access_key_id`, `access_key_secret`
{% endhint %}
Configuration and details
This plugin must be used in routes that utilize the ai-proxy or ai-proxy-multi plugins.
| **Field** | **Required** | **Type** | **Description** |
| ------------------------------ | ------------ | -------- | ------------------------------------------------------------------------------------------------------------ |
| endpoint | Yes | String | Aliyun service endpoint URL |
| region\_id | Yes | String | Aliyun region identifier |
| access\_key\_id | Yes | String | Aliyun access key ID |
| access\_key\_secret | Yes | String | Aliyun access key secret |
| check\_request | No | Boolean | Enable request content moderation. Default: `true` |
| check\_response | No | Boolean | Enable response content moderation. Default: `false` |
| stream\_check\_mode | No | String | Streaming moderation mode. Default: `"final_packet"`. Valid values: `["realtime", "final_packet"]` |
| stream\_check\_cache\_size | No | Integer | Max characters per moderation batch in realtime mode. Default: `128`. Must be `>= 1`. |
| stream\_check\_interval | No | Number | Seconds between batch checks in realtime mode. Default: `3`. Must be `>= 0.1`. |
| request\_check\_service | No | String | Aliyun service for request moderation. Default: `"llm_query_moderation"` |
| request\_check\_length\_limit | No | Number | Max characters per request moderation chunk. Default: `2000`. |
| response\_check\_service | No | String | Aliyun service for response moderation. Default: `"llm_response_moderation"` |
| response\_check\_length\_limit | No | Number | Max characters per response moderation chunk. Default: `5000`. |
| risk\_level\_bar | No | String | Threshold for content rejection. Default: `"high"`. Valid values: `["none", "low", "medium", "high", "max"]` |
| deny\_code | No | Number | HTTP status code for rejected content. Default: `200`. |
| deny\_message | No | String | Custom message for rejected content. Default: `-`. |
| timeout | No | Integer | Request timeout in milliseconds. Default: `10000`. Must be `>= 1`. |
| ssl\_verify | No | Boolean | Enable SSL certificate verification. Default: `true`. |
### `ai-aws-content-moderation`
The `ai-aws-content-moderation` plugin processes the request body to check for toxicity and rejects the request if it exceeds the configured threshold.
{% hint style="info" %}
**Priority** `1050` · **Category** AI · **Required** `comprehend`
{% endhint %}
Configuration and details
***This plugin must be used in routes that proxy requests to LLMs only.***
***As of now, the plugin only supports the integration with*** [***AWS Comprehend***](https://aws.amazon.com/comprehend/) ***for content moderation. PRs for introducing support for other service providers are welcomed.***
| **Field** | **Required** | **Type** | **Description** |
| ------------------------------ | ------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| comprehend.access\_key\_id | Yes | String | AWS access key ID |
| comprehend.secret\_access\_key | Yes | String | AWS secret access key |
| comprehend.region | Yes | String | AWS region |
| comprehend.endpoint | No | String | AWS Comprehend service endpoint. Must match the pattern `^https?://` |
| comprehend.ssl\_verify | No | String | Enables SSL certificate verification. |
| moderation\_categories | No | Object | Key-value pairs of moderation category and their score. In each pair, the key should be one of the `PROFANITY`, `HATE_SPEECH`, `INSULT`, `HARASSMENT_OR_ABUSE`, `SEXUAL`, or `VIOLENCE_OR_THREAT`; and the value should be between 0 and 1 (inclusive). |
| moderation\_threshold | No | Number | The degree to which content is harmful, offensive, or inappropriate. A higher value indicates more toxic content allowed. Range: 0 - 1. Default: 0.5 |
### `ai-prompt-decorator`
The `ai-prompt-decorator` plugin simplifies access to LLM providers, such as OpenAI and Anthropic, and their models by appending or prepending prompts into the request.
{% hint style="info" %}
**Priority** `1070` · **Category** AI
{% endhint %}
Configuration and details
| **Field** | **Required** | **Type** | **Description** |
| ----------------- | --------------- | -------- | --------------------------------------------------- |
| `prepend` | Conditionally\* | Array | An array of prompt objects to be prepended |
| `prepend.role` | Yes | String | Role of the message (`system`, `user`, `assistant`) |
| `prepend.content` | Yes | String | Content of the message. Minimum length: 1 |
| `append` | Conditionally\* | Array | An array of prompt objects to be appended |
| `append.role` | Yes | String | Role of the message (`system`, `user`, `assistant`) |
| `append.content` | Yes | String | Content of the message. Minimum length: 1 |
### `ai-prompt-guard`
The `ai-prompt-guard` plugin safeguards your AI endpoints by inspecting and validating incoming prompt messages. It checks the content of requests against user-defined allowed and denied patterns to ensure that only approved inputs are processed. Based on its configuration, the plugin can either examine just the latest message or the entire conversation history, and it can be set to check prompts from all roles or only from end users.
{% hint style="info" %}
**Priority** `1072` · **Category** AI
{% endhint %}
Configuration and details
When both **allow** and **deny** patterns are configured, the plugin first ensures that at least one allowed pattern is matched. If none match, the request is rejected with a *"Request doesn't match allow patterns"* error. If an allowed pattern is found, it then checks for any occurrences of denied patterns, rejecting the request with a *"Request contains prohibited content"* error if any are detected.
| **Field** | **Required** | **Type** | **Description** |
| --------------------------------- | ------------ | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| match\_all\_roles | No | boolean | If set to `true`, the plugin will check prompt messages from all roles. Otherwise, it only validates when its role is `"user"`. Default is `false`. |
| match\_all\_conversation\_history | No | boolean | When enabled, all messages in the conversation history are concatenated and checked. If `false`, only the content of the last message is examined. Default is `false`. |
| allow\_patterns | No | array | A list of regex patterns. When provided, the prompt must match **at least one** pattern to be considered valid. |
| deny\_patterns | No | array | A list of regex patterns. If any of these patterns match the prompt content, the request is rejected. |
### `ai-prompt-template`
The `ai-prompt-template` Plugin simplifies access to LLM providers, such as OpenAI and Anthropic, and their models. It pre-configures prompt templates that only accept user inputs in designated template variables, in a "fill in the blank" fashion.
{% hint style="info" %}
**Priority** `1071` · **Category** AI · **Required** `templates`
{% endhint %}
Configuration and details
| **Field** | **Required** | **Type** | **Description** |
| ------------------------------------- | ------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| `templates` | Yes | Array | An array of template objects. |
| `templates.name` | Yes | String | Name of the template. When requesting the route, the request should include the template name that corresponds to the configured template. |
| `templates.template` | Yes | Object | Template specification. |
| `templates.template.model` | Yes | String | Name of the AI Model, such as `gpt-4` or `gpt-3.5`. See your LLM provider API documentation for more available models. |
| `templates.template.messages` | Yes | Array | Template message specification. |
| `templates.template.messages.role` | Yes | String | Role of the message, such as `system`, `user`, or `assistant`. |
| `templates.template.messages.content` | Yes | String | Content of the message (prompt). |
### `ai-proxy`
The `ai-proxy` Plugin simplifies access to LLM and embedding models by transforming Plugin configurations into the designated request format. It supports the integration with OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, OpenRouter, Gemini, Vertex AI, and other OpenAI-compatible APIs.
{% hint style="info" %}
**Priority** `1040` · **Category** AI · **Required** `provider`, `auth`
{% endhint %}
Configuration and details
In addition, the Plugin also supports logging LLM request information in the access log, such as token usage, model, time to the first response, and more.
| Name | Type | Required | Default | Valid values | Description |
| ------------------------------- | ------- | -------- | ------- | ------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| provider | string | True | | \[openai, deepseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, openai-compatible] | LLM service provider. When set to `openai`, the Plugin will proxy the request to `https://api.openai.com/chat/completions`. When set to `deepseek`, the Plugin will proxy the request to `https://api.deepseek.com/chat/completions`. When set to `aimlapi`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://api.aimlapi.com/v1/chat/completions` by default. When set to `anthropic`, the Plugin will proxy the request to `https://api.anthropic.com/v1/chat/completions` by default. When set to `openrouter`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://openrouter.ai/api/v1/chat/completions` by default. When set to `gemini`, the Plugin uses the OpenAI-compatible driver and proxies the request to `https://generativelanguage.googleapis.com/v1beta/openai/chat/completions` by default. When set to `vertex-ai`, the Plugin will proxy the request to `https://aiplatform.googleapis.com` by default and requires `provider_conf` or `override`. When set to `openai-compatible`, the Plugin will proxy the request to the custom endpoint configured in `override`. |
| provider\_conf | object | False | | | Configuration for the specific provider. Required when `provider` is set to `vertex-ai` and `override` is not configured. |
| provider\_conf.project\_id | string | True | | | Google Cloud Project ID. |
| provider\_conf.region | string | True | | | Google Cloud Region. |
| auth | object | True | | | Authentication configurations. |
| auth.header | object | False | | | Authentication headers. At least one of `header` or `query` must be configured. |
| auth.query | object | False | | | Authentication query parameters. At least one of `header` or `query` must be configured. |
| auth.gcp | object | False | | | Configuration for Google Cloud Platform (GCP) authentication. |
| auth.gcp.service\_account\_json | string | False | | | Content of the GCP service account JSON file. This can also be configured by setting the `GCP_SERVICE_ACCOUNT` environment variable. |
| auth.gcp.max\_ttl | integer | False | | minimum = 1 | Maximum TTL (in seconds) for caching the GCP access token. |
| auth.gcp.expire\_early\_secs | integer | False | 60 | minimum = 0 | Seconds to expire the access token before its actual expiration time to avoid edge cases. |
| options | object | False | | | Model configurations. In addition to `model`, you can configure additional parameters and they will be forwarded to the upstream LLM service in the request body. For instance, if you are working with OpenAI, you can configure additional parameters such as `temperature`, `top_p`, and `stream`. See your LLM provider's API documentation for more available options. |
| options.model | string | False | | | Name of the LLM model, such as `gpt-4` or `gpt-3.5`. Refer to the LLM provider's API documentation for available models. |
| override | object | False | | | Override setting. |
| override.endpoint | string | False | | | Custom LLM provider endpoint, required when `provider` is `openai-compatible`. |
| logging | object | False | | | Logging configurations. |
| logging.summaries | boolean | False | false | | If true, logs request LLM model, duration, request, and response tokens. |
| logging.payloads | boolean | False | false | | If true, logs request and response payload. |
| timeout | integer | False | 30000 | ≥ 1 | Request timeout in milliseconds when requesting the LLM service. |
| keepalive | boolean | False | true | | If true, keeps the connection alive when requesting the LLM service. |
| keepalive\_timeout | integer | False | 60000 | ≥ 1000 | Keepalive timeout in milliseconds when connecting to the LLM service. |
| keepalive\_pool | integer | False | 30 | | Keepalive pool size for the LLM service connection. |
| ssl\_verify | boolean | False | true | | If true, verifies the LLM service's certificate. |
### `ai-proxy-multi`
The `ai-proxy-multi` Plugin simplifies access to LLM and embedding models by transforming Plugin configurations into the designated request format for OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, OpenRouter, Gemini, Vertex AI, and other OpenAI-compatible APIs. It extends the capabilities of `ai-proxy` with load balancing, retries, fallbacks, and health checks.
{% hint style="info" %}
**Priority** `1041` · **Category** AI · **Required** `instances`
{% endhint %}
Configuration and details
In addition, the Plugin also supports logging LLM request information in the access log, such as token usage, model, time to the first response, and more.
| Name | Type | Required | Default | Valid Values | Description |
| ----------------------------------------- | --------------- | -------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| fallback\_strategy | string or array | False | | string: "instance\_health\_and\_rate\_limiting", "http\_429", "http\_5xx"
array: \["rate\_limiting", "http\_429", "http\_5xx"]
| Fallback strategy. When set, the Plugin will check whether the specified instance’s token has been exhausted when a request is forwarded. If so, forward the request to the next instance regardless of the instance priority. When not set, the Plugin will not forward the request to low priority instances when token of the high priority instance is exhausted. |
| balancer | object | False | | | Load balancing configurations. |
| balancer.algorithm | string | False | roundrobin | \[roundrobin, chash] | Load balancing algorithm. When set to `roundrobin`, weighted round robin algorithm is used. When set to `chash`, consistent hashing algorithm is used. |
| balancer.hash\_on | string | False | | \[vars, headers, cookie, consumer, vars\_combinations] | Used when `type` is `chash`. Support hashing on [NGINX variables](https://nginx.org/en/docs/varindex.html), headers, cookie, consumer, or a combination of [NGINX variables](https://nginx.org/en/docs/varindex.html). |
| balancer.key | string | False | | | Used when `type` is `chash`. When `hash_on` is set to `header` or `cookie`, `key` is required. When `hash_on` is set to `consumer`, `key` is not required as the consumer name will be used as the key automatically. |
| instances | array\[object] | True | | | LLM instance configurations. |
| instances.name | string | True | | | Name of the LLM service instance. |
| instances.provider | string | True | | \[openai, deepseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, openai-compatible] | LLM service provider. When set to `openai`, the Plugin will proxy the request to `api.openai.com`. When set to `deepseek`, the Plugin will proxy the request to `api.deepseek.com`. When set to `aimlapi`, the Plugin uses the OpenAI-compatible driver and proxies the request to `api.aimlapi.com` by default. When set to `anthropic`, the Plugin will proxy the request to `api.anthropic.com` by default. When set to `openrouter`, the Plugin uses the OpenAI-compatible driver and proxies the request to `openrouter.ai` by default. When set to `gemini`, the Plugin uses the OpenAI-compatible driver and proxies the request to `generativelanguage.googleapis.com` by default. When set to `vertex-ai`, the Plugin will proxy the request to `aiplatform.googleapis.com` by default and requires `provider_conf` or `override`. When set to `openai-compatible`, the Plugin will proxy the request to the custom endpoint configured in `override`. |
| instances.provider\_conf | object | False | | | Configuration for the specific provider. Required when `provider` is set to `vertex-ai` and `override` is not configured. |
| instances.provider\_conf.project\_id | string | True | | | Google Cloud Project ID. |
| instances.provider\_conf.region | string | True | | | Google Cloud Region. |
| instances.priority | integer | False | 0 | | Priority of the LLM instance in load balancing. `priority` takes precedence over `weight`. |
| instances.weight | string | True | 0 | greater or equal to 0 | Weight of the LLM instance in load balancing. |
| instances.auth | object | True | | | Authentication configurations. |
| instances.auth.header | object | False | | | Authentication headers. At least one of the `header` and `query` should be configured. |
| instances.auth.query | object | False | | | Authentication query parameters. At least one of the `header` and `query` should be configured. |
| instances.auth.gcp | object | False | | | Configuration for Google Cloud Platform (GCP) authentication. |
| instances.auth.gcp.service\_account\_json | string | False | | | Content of the GCP service account JSON file. This can also be configured by setting the `GCP_SERVICE_ACCOUNT` environment variable. |
| instances.auth.gcp.max\_ttl | integer | False | | minimum = 1 | Maximum TTL (in seconds) for caching the GCP access token. |
| instances.auth.gcp.expire\_early\_secs | integer | False | 60 | minimum = 0 | Seconds to expire the access token before its actual expiration time to avoid edge cases. |
| instances.options | object | False | | | Model configurations. In addition to `model`, you can configure additional parameters and they will be forwarded to the upstream LLM service in the request body. For instance, if you are working with OpenAI, DeepSeek, or AIMLAPI, you can configure additional parameters such as `max_tokens`, `temperature`, `top_p`, and `stream`. See your LLM provider's API documentation for more available options. |
| instances.options.model | string | False | | | Name of the LLM model, such as `gpt-4` or `gpt-3.5`. See your LLM provider's API documentation for more available models. |
| logging | object | False | | | Logging configurations. |
| logging.summaries | boolean | False | false | | If true, log request LLM model, duration, request, and response tokens. |
| logging.payloads | boolean | False | false | | If true, log request and response payload. |
| logging.override | object | False | | | Override setting. |
| logging.override.endpoint | string | False | | | LLM provider endpoint to replace the default endpoint with. If not configured, the Plugin uses the default OpenAI endpoint `https://api.openai.com/v1/chat/completions`. |
| checks | object | False | | | Health check configurations. Note that at the moment, OpenAI, DeepSeek, and AIMLAPI do not provide an official health check endpoint. Other LLM services that you can configure under `openai-compatible` provider may have available health check endpoints. |
| checks.active | object | True | | | Active health check configurations. |
| checks.active.type | string | False | http | \[http, https, tcp] | Type of health check connection. |
| checks.active.timeout | number | False | 1 | | Health check timeout in seconds. |
| checks.active.concurrency | integer | False | 10 | | Number of upstream nodes to be checked at the same time. |
| checks.active.host | string | False | | | HTTP host. |
| checks.active.port | integer | False | | between 1 and 65535 inclusive | HTTP port. |
| checks.active.http\_path | string | False | / | | Path for HTTP probing requests. |
| checks.active.https\_verify\_certificate | boolean | False | true | | If true, verify the node's TLS certificate. |
| timeout | integer | False | 30000 | greater than or equal to 1 | Request timeout in milliseconds when requesting the LLM service. |
| keepalive | boolean | False | true | | If true, keep the connection alive when requesting the LLM service. |
| keepalive\_timeout | integer | False | 60000 | greater than or equal to 1000 | Request timeout in milliseconds when requesting the LLM service. |
| keepalive\_pool | integer | False | 30 | | Keepalive pool size for when connecting with the LLM service. |
| ssl\_verify | boolean | False | true | | If true, verify the LLM service's certificate. |
### `ai-rag`
The `ai-rag` Plugin provides Retrieval-Augmented Generation (RAG) capabilities with LLMs. It facilitates the efficient retrieval of relevant documents or information from external data sources, which are used to enhance the LLM responses, thereby improving the accuracy and contextual relevance of the generated outputs.
{% hint style="info" %}
**Priority** `1060` · **Category** AI · **Required** `embeddings_provider`, `vector_search_provider`
{% endhint %}
Configuration and details
The Plugin supports using [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service) and [Azure AI Search](https://azure.microsoft.com/en-us/products/ai-services/ai-search) services for generating embeddings and performing vector search.
***As of now only*** [***Azure OpenAI***](https://azure.microsoft.com/en-us/products/ai-services/openai-service) ***and*** [***Azure AI Search***](https://azure.microsoft.com/en-us/products/ai-services/ai-search) ***services are supported for generating embeddings and performing vector search respectively. PRs for introducing support for other service providers are welcomed.***
| Name | Required | Type | Description |
| --------------------------------------------------- | -------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
| embeddings\_provider | True | object | Configurations of the embedding models provider. |
| embeddings\_provider.azure\_openai | True | object | Configurations of [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service) as the embedding models provider. |
| embeddings\_provider.azure\_openai.endpoint | True | string | Azure OpenAI embedding model endpoint. |
| embeddings\_provider.azure\_openai.api\_key | True | string | Azure OpenAI API key. |
| vector\_search\_provider | True | object | Configuration for the vector search provider. |
| vector\_search\_provider.azure\_ai\_search | True | object | Configuration for Azure AI Search. |
| vector\_search\_provider.azure\_ai\_search.endpoint | True | string | Azure AI Search endpoint. |
| vector\_search\_provider.azure\_ai\_search.api\_key | True | string | Azure AI Search API key. |
### `ai-rate-limiting`
The `ai-rate-limiting` Plugin enforces token-based rate limiting for requests sent to LLM services. It helps manage API usage by controlling the number of tokens consumed within a specified time frame, ensuring fair resource allocation and preventing excessive load on the service. It is often used with `ai-proxy` or `ai-proxy-multi` plugin.
{% hint style="info" %}
**Priority** `1030` · **Category** AI
{% endhint %}
Configuration and details
| Name | Type | Required | Default | Valid values | Description |
| -------------------------- | -------------- | -------- | ------------- | ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| limit | integer | False | | >0 | The maximum number of tokens allowed within a given time interval. At least one of `limit` and `instances.limit` should be configured. |
| time\_window | integer | False | | >0 | The time interval corresponding to the rate limiting `limit` in seconds. At least one of `time_window` and `instances.time_window` should be configured. |
| show\_limit\_quota\_header | boolean | False | true | | If true, includes `X-AI-RateLimit-Limit-*`, `X-AI-RateLimit-Remaining-*`, and `X-AI-RateLimit-Reset-*` headers in the response, where `*` is the instance name. |
| limit\_strategy | string | False | total\_tokens | \[total\_tokens, prompt\_tokens, completion\_tokens] | Type of token to apply rate limiting. `total_tokens` is the sum of `prompt_tokens` and `completion_tokens`. |
| instances | array\[object] | False | | | LLM instance rate limiting configurations. |
| instances.name | string | True | | | Name of the LLM service instance. |
| instances.limit | integer | True | | >0 | The maximum number of tokens allowed within a given time interval for an instance. |
| instances.time\_window | integer | True | | >0 | The time interval corresponding to the rate limiting `limit` in seconds for an instance. |
| rejected\_code | integer | False | 503 | \[200, 599] | The HTTP status code returned when a request exceeding the quota is rejected. |
| rejected\_msg | string | False | | | The response body returned when a request exceeding the quota is rejected. |
### `ai-request-rewrite`
The `ai-request-rewrite` plugin intercepts client requests before they are forwarded to the upstream service. It sends a predefined prompt, along with the original request body, to a specified LLM service. The LLM processes the input and returns a modified request body, which is then used for the upstream request. This allows dynamic transformation of API requests based on AI-generated content.
{% hint style="info" %}
**Priority** `1073` · **Category** AI · **Required** `prompt`, `provider`, `auth`
{% endhint %}
Configuration and details
| **Field** | **Required** | **Type** | **Description** |
| ------------------------------- | ------------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| prompt | Yes | String | The prompt send to LLM service. |
| provider | Yes | String | Name of the LLM service. Available options: openai, deekseek, azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, and openai-compatible. When `aimlapi` is selected, the plugin uses the OpenAI-compatible driver with a default endpoint of `https://api.aimlapi.com/v1/chat/completions`. |
| provider\_conf | No | Object | Configuration for the specific provider. Required when `provider` is set to `vertex-ai` and `override` is not configured. |
| provider\_conf.project\_id | Yes | String | Google Cloud Project ID. |
| provider\_conf.region | Yes | String | Google Cloud Region. |
| auth | Yes | Object | Authentication configuration |
| auth.header | No | Object | Authentication headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. |
| auth.query | No | Object | Authentication query parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. |
| auth.gcp | No | Object | Configuration for Google Cloud Platform (GCP) authentication. |
| auth.gcp.service\_account\_json | No | String | Content of the GCP service account JSON file. This can also be configured by setting the `GCP_SERVICE_ACCOUNT` environment variable. |
| auth.gcp.max\_ttl | No | Integer | Maximum TTL (in seconds) for caching the GCP access token. Minimum: 1. |
| auth.gcp.expire\_early\_secs | No | Integer | Seconds to expire the access token before its actual expiration time to avoid edge cases. Minimum: 0. Default: 60. |
| options | No | Object | Key/value settings for the model |
| options.model | No | String | Model to execute. Examples: "gpt-3.5-turbo" for openai, "deepseek-chat" for deekseek, or "qwen-turbo" for openai-compatible or aimlapi services |
| override.endpoint | No | String | Override the default endpoint when using OpenAI-compatible services (e.g., self-hosted models or third-party LLM services). When the provider is 'openai-compatible', the endpoint field is required. |
| timeout | No | Integer | Total timeout in milliseconds for requests to LLM service, including connect, send, and read timeouts. Range: 1 - 60000. Default: 30000 |
| keepalive | No | Boolean | Enable keepalive for requests to LLM service. Default: true |
| keepalive\_timeout | No | Integer | Keepalive timeout in milliseconds for requests to LLM service. Minimum: 1000. Default: 60000 |
| keepalive\_pool | No | Integer | Keepalive pool size for requests to LLM service. Minimum: 1. Default: 30 |
| ssl\_verify | No | Boolean | SSL verification for requests to LLM service. Default: true |
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.digitalapi.ai/api-gateway/plugin-reference/plugins-ai.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.