Bring Your own LLM to DiliTrust

This guide explains what is required if you want to connect your own Large Language Model (LLM) to our platform. It is written for business users and decision-makers, not engineers, and focuses on what you need to prepare to ensure a smooth and reliable integration.

TABLE OF CONTENTS

• Bring Your LLM – General Requirements ✨
  • 1️⃣ Access via a Secured Public API
  • 2️⃣ Compatibility with Our Integration
  • 3️⃣ Supported Model Types
  • 4️⃣ Model Quality & Performance Requirements
• Bring Your LLM – Azure OpenAI ☁️
  • 1️⃣ Azure OpenAI Deployment
  • 2️⃣ Quota Requirements
  • 3️⃣ Secrets Management
  • 4️⃣ Model & Deployment Information

Bring Your LLM – General Requirements ✨

1️⃣ Access via a Secured Public API

To connect your LLM, it must be accessible through a public API that is properly secured. This ensures safe and stable communication between your model and our platform.

• Your model must be exposed behind a public API endpoint.
• The API must be secured using an API key.
• You must securely share this API key with us so we can establish the connection.

Why this matters: A secured API protects your data and ensures only authorized systems can access your model

2️⃣ Compatibility with Our Integration

For the fastest and simplest setup, your API should ideally be compatible with the OpenAI API. This means it can be called using the official OpenAI libraries and follows the same request/response structure.

If your API is not OpenAI-compatible (which is possible but subject to review), you will need to provide:

• Complete technical documentation of your API
• Clear examples of API requests and responses
• Detailed specifications, including:
  • Input and output formats
  • Available endpoints
  • Authentication method

Based on this information, our team will assess whether integration is feasible

Tip: OpenAI-compatible APIs significantly reduce setup time and integration effort

3️⃣ Supported Model Types

We can integrate with a wide range of generative AI models, as long as they meet these basic criteria:

• The model must accept text as input
• The model must return text as output

This includes models used for summarization, chatbots, assistants, and other text-based use cases.

4️⃣ Model Quality & Performance Requirements

To ensure a good user experience, your LLM must meet minimum quality and performance standards. These requirements are essential for handling real-world workloads reliably.

Context Window Capacity

• Document summarization & meeting minutes: at least 32k tokens
• Chatbot and conversational use cases: at least 64k tokens with strong long-context performance

Language Capabilities

• Multilingual support is required
• Strong performance in English plus the languages your teams work with

Reasoning & Tool Usage

• Strong reasoning and instruction-following abilities
• Ability to support tool calling for advanced, agent-based use cases (currently under development)

Throughput & Latency

• Minimum generation speed: 150 tokens per second for long-context prompts
• API capacity must support:
  • At least 500 requests per minute
  • At least 1,000,000 tokens per minute

Why this matters: These limits ensure fast responses and stable performance, even during peak usage

Bring Your LLM – Azure OpenAI ☁️

If you are using Azure OpenAI, the following specific requirements apply.

1️⃣ Azure OpenAI Deployment

• Create a dedicated deployment in your Azure OpenAI service
• Use a recent model to ensure good quality and instruction-following (for example: gpt-4.1 or newer)

Dedicated deployments help isolate workloads and ensure predictable performance

2️⃣ Quota Requirements

Your Azure subscription and OpenAI resource must support the following minimum quotas:

• Requests per minute (RPM): at least 500
• Tokens per minute (TPM): at least 1,000,000

Please ensure your Azure subscription tier allows these limits before starting the integration.

3️⃣ Secrets Management

To securely connect to your Azure OpenAI deployment, you must provide:

• AZURE_OPENAI_API_KEY – API key used for authentication
• AZURE_OPENAI_ENDPOINT – Endpoint URL of your Azure OpenAI resource
• AZURE_OPENAI_API_VERSION – API version (for example: 2024-05-01-preview)

All credentials must be shared securely to protect your data and access rights

4️⃣ Model & Deployment Information

Finally, please share the following details about your Azure setup:

• Deployment Name – The Azure-specific name used in API calls
• Model Name – The original OpenAI model identifier (for example: gpt-4.1)

This information allows us to correctly route requests and validate the integration.

If you are unsure whether your model meets these requirements, our team is happy to review your setup and guide you through the next steps.