mirror of
https://github.com/farcasclaudiu/openclaw.git
synced 2026-06-28 19:01:47 +03:00
feat: add NVIDIA API provider integration
Add support for NVIDIA's API (https://integrate.api.nvidia.com/v1) with three models: - nvidia/llama-3.1-nemotron-70b-instruct (default) - nvidia/llama-3.3-70b-instruct - nvidia/mistral-nemo-minitron-8b-8k-instruct Users can configure via NVIDIA_API_KEY environment variable or auth profiles. Co-authored-by: thesomewhatyou <162917831+thesomewhatyou@users.noreply.github.com>
This commit is contained in:
committed by
Peter Steinberger
parent
84ed9ab554
commit
c640b5f86c
@@ -289,6 +289,7 @@ Docs: https://docs.openclaw.ai
|
|||||||
- Models: support Anthropic Opus 4.6 and OpenAI Codex gpt-5.3-codex (forward-compat fallbacks). (#9853, #10720, #9995) Thanks @TinyTb, @calvin-hpnet, @tyler6204.
|
- Models: support Anthropic Opus 4.6 and OpenAI Codex gpt-5.3-codex (forward-compat fallbacks). (#9853, #10720, #9995) Thanks @TinyTb, @calvin-hpnet, @tyler6204.
|
||||||
- Providers: add xAI (Grok) support. (#9885) Thanks @grp06.
|
- Providers: add xAI (Grok) support. (#9885) Thanks @grp06.
|
||||||
- Providers: add Baidu Qianfan support. (#8868) Thanks @ide-rea.
|
- Providers: add Baidu Qianfan support. (#8868) Thanks @ide-rea.
|
||||||
|
- Providers: add NVIDIA API support with models including Llama 3.1 Nemotron 70B, Llama 3.3 70B, and Mistral NeMo Minitron 8B.
|
||||||
- Web UI: add token usage dashboard. (#10072) Thanks @Takhoffman.
|
- Web UI: add token usage dashboard. (#10072) Thanks @Takhoffman.
|
||||||
- Web UI: add RTL auto-direction support for Hebrew/Arabic text in chat composer and rendered messages. (#11498) Thanks @dirbalak.
|
- Web UI: add RTL auto-direction support for Hebrew/Arabic text in chat composer and rendered messages. (#11498) Thanks @dirbalak.
|
||||||
- Memory: native Voyage AI support. (#7078) Thanks @mcinteerj.
|
- Memory: native Voyage AI support. (#7078) Thanks @mcinteerj.
|
||||||
|
|||||||
+107
@@ -0,0 +1,107 @@
|
|||||||
|
---
|
||||||
|
summary: "NVIDIA API setup for AI model access"
|
||||||
|
read_when:
|
||||||
|
- You want to use NVIDIA's AI models
|
||||||
|
- You need NVIDIA_API_KEY setup
|
||||||
|
title: "NVIDIA API"
|
||||||
|
---
|
||||||
|
|
||||||
|
# NVIDIA API
|
||||||
|
|
||||||
|
OpenClaw can use NVIDIA's API (https://integrate.api.nvidia.com/v1) for accessing various AI models. NVIDIA provides access to state-of-the-art language models through their integration endpoint.
|
||||||
|
|
||||||
|
## API Setup
|
||||||
|
|
||||||
|
### NVIDIA (direct)
|
||||||
|
|
||||||
|
- Base URL: [https://integrate.api.nvidia.com/v1](https://integrate.api.nvidia.com/v1)
|
||||||
|
- Environment variable: `NVIDIA_API_KEY`
|
||||||
|
- Get your API key from: [NVIDIA NGC](https://catalog.ngc.nvidia.com/)
|
||||||
|
|
||||||
|
## Config example
|
||||||
|
|
||||||
|
```json5
|
||||||
|
{
|
||||||
|
models: {
|
||||||
|
providers: {
|
||||||
|
nvidia: {
|
||||||
|
apiKey: "nvapi-...",
|
||||||
|
baseUrl: "https://integrate.api.nvidia.com/v1",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
agents: {
|
||||||
|
default: {
|
||||||
|
provider: "nvidia",
|
||||||
|
model: "nvidia/llama-3.1-nemotron-70b-instruct",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Available Models
|
||||||
|
|
||||||
|
OpenClaw includes support for several NVIDIA models:
|
||||||
|
|
||||||
|
- `nvidia/llama-3.1-nemotron-70b-instruct` (default) — High-performance instruction-following model
|
||||||
|
- `nvidia/llama-3.3-70b-instruct` — Latest Llama 3.3 variant
|
||||||
|
- `nvidia/mistral-nemo-minitron-8b-8k-instruct` — Smaller, efficient model
|
||||||
|
|
||||||
|
## Environment Variable Setup
|
||||||
|
|
||||||
|
Set your NVIDIA API key as an environment variable:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export NVIDIA_API_KEY="nvapi-your-key-here"
|
||||||
|
```
|
||||||
|
|
||||||
|
Or add it to your `.env` file:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
NVIDIA_API_KEY=nvapi-your-key-here
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage in Config
|
||||||
|
|
||||||
|
Minimal configuration (uses environment variable):
|
||||||
|
|
||||||
|
```json5
|
||||||
|
{
|
||||||
|
agents: {
|
||||||
|
default: {
|
||||||
|
provider: "nvidia",
|
||||||
|
model: "nvidia/llama-3.1-nemotron-70b-instruct",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Explicit API key configuration:
|
||||||
|
|
||||||
|
```json5
|
||||||
|
{
|
||||||
|
models: {
|
||||||
|
providers: {
|
||||||
|
nvidia: {
|
||||||
|
apiKey: "NVIDIA_API_KEY",
|
||||||
|
baseUrl: "https://integrate.api.nvidia.com/v1",
|
||||||
|
api: "openai-completions",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Professional and Personal Use
|
||||||
|
|
||||||
|
NVIDIA's API is suitable for both professional and personal applications:
|
||||||
|
|
||||||
|
- **Professional**: Enterprise-grade models for business applications, research, and development
|
||||||
|
- **Personal**: Access to powerful AI models for learning, experimentation, and personal projects
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- NVIDIA API uses OpenAI-compatible endpoints
|
||||||
|
- Models are automatically discovered if `NVIDIA_API_KEY` is set
|
||||||
|
- Default context window: 131,072 tokens
|
||||||
|
- Default max tokens: 4,096 tokens
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
import { mkdtempSync } from "node:fs";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import { resolveImplicitProviders, buildNvidiaProvider } from "./models-config.providers.js";
|
||||||
|
|
||||||
|
describe("NVIDIA provider", () => {
|
||||||
|
it("should include nvidia when NVIDIA_API_KEY is configured", async () => {
|
||||||
|
const agentDir = mkdtempSync(join(tmpdir(), "openclaw-test-"));
|
||||||
|
const previous = process.env.NVIDIA_API_KEY;
|
||||||
|
process.env.NVIDIA_API_KEY = "test-key";
|
||||||
|
|
||||||
|
try {
|
||||||
|
const providers = await resolveImplicitProviders({ agentDir });
|
||||||
|
expect(providers?.nvidia).toBeDefined();
|
||||||
|
expect(providers?.nvidia?.apiKey).toBe("NVIDIA_API_KEY");
|
||||||
|
} finally {
|
||||||
|
if (previous === undefined) {
|
||||||
|
delete process.env.NVIDIA_API_KEY;
|
||||||
|
} else {
|
||||||
|
process.env.NVIDIA_API_KEY = previous;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should build nvidia provider with correct configuration", () => {
|
||||||
|
const provider = buildNvidiaProvider();
|
||||||
|
expect(provider.baseUrl).toBe("https://integrate.api.nvidia.com/v1");
|
||||||
|
expect(provider.api).toBe("openai-completions");
|
||||||
|
expect(provider.models).toBeDefined();
|
||||||
|
expect(provider.models.length).toBeGreaterThan(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should include default nvidia models", () => {
|
||||||
|
const provider = buildNvidiaProvider();
|
||||||
|
const modelIds = provider.models.map((m) => m.id);
|
||||||
|
expect(modelIds).toContain("nvidia/llama-3.1-nemotron-70b-instruct");
|
||||||
|
expect(modelIds).toContain("nvidia/llama-3.3-70b-instruct");
|
||||||
|
expect(modelIds).toContain("nvidia/mistral-nemo-minitron-8b-8k-instruct");
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -112,6 +112,17 @@ const QIANFAN_DEFAULT_COST = {
|
|||||||
cacheWrite: 0,
|
cacheWrite: 0,
|
||||||
};
|
};
|
||||||
|
|
||||||
|
const NVIDIA_BASE_URL = "https://integrate.api.nvidia.com/v1";
|
||||||
|
const NVIDIA_DEFAULT_MODEL_ID = "nvidia/llama-3.1-nemotron-70b-instruct";
|
||||||
|
const NVIDIA_DEFAULT_CONTEXT_WINDOW = 131072;
|
||||||
|
const NVIDIA_DEFAULT_MAX_TOKENS = 4096;
|
||||||
|
const NVIDIA_DEFAULT_COST = {
|
||||||
|
input: 0,
|
||||||
|
output: 0,
|
||||||
|
cacheRead: 0,
|
||||||
|
cacheWrite: 0,
|
||||||
|
};
|
||||||
|
|
||||||
interface OllamaModel {
|
interface OllamaModel {
|
||||||
name: string;
|
name: string;
|
||||||
modified_at: string;
|
modified_at: string;
|
||||||
@@ -609,6 +620,42 @@ export function buildQianfanProvider(): ProviderConfig {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export function buildNvidiaProvider(): ProviderConfig {
|
||||||
|
return {
|
||||||
|
baseUrl: NVIDIA_BASE_URL,
|
||||||
|
api: "openai-completions",
|
||||||
|
models: [
|
||||||
|
{
|
||||||
|
id: NVIDIA_DEFAULT_MODEL_ID,
|
||||||
|
name: "NVIDIA Llama 3.1 Nemotron 70B Instruct",
|
||||||
|
reasoning: false,
|
||||||
|
input: ["text"],
|
||||||
|
cost: NVIDIA_DEFAULT_COST,
|
||||||
|
contextWindow: NVIDIA_DEFAULT_CONTEXT_WINDOW,
|
||||||
|
maxTokens: NVIDIA_DEFAULT_MAX_TOKENS,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "nvidia/llama-3.3-70b-instruct",
|
||||||
|
name: "NVIDIA Llama 3.3 70B Instruct",
|
||||||
|
reasoning: false,
|
||||||
|
input: ["text"],
|
||||||
|
cost: NVIDIA_DEFAULT_COST,
|
||||||
|
contextWindow: 131072,
|
||||||
|
maxTokens: 4096,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "nvidia/mistral-nemo-minitron-8b-8k-instruct",
|
||||||
|
name: "NVIDIA Mistral NeMo Minitron 8B Instruct",
|
||||||
|
reasoning: false,
|
||||||
|
input: ["text"],
|
||||||
|
cost: NVIDIA_DEFAULT_COST,
|
||||||
|
contextWindow: 8192,
|
||||||
|
maxTokens: 2048,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
export async function resolveImplicitProviders(params: {
|
export async function resolveImplicitProviders(params: {
|
||||||
agentDir: string;
|
agentDir: string;
|
||||||
explicitProviders?: Record<string, ProviderConfig> | null;
|
explicitProviders?: Record<string, ProviderConfig> | null;
|
||||||
@@ -753,6 +800,13 @@ export async function resolveImplicitProviders(params: {
|
|||||||
providers.qianfan = { ...buildQianfanProvider(), apiKey: qianfanKey };
|
providers.qianfan = { ...buildQianfanProvider(), apiKey: qianfanKey };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const nvidiaKey =
|
||||||
|
resolveEnvApiKeyVarName("nvidia") ??
|
||||||
|
resolveApiKeyFromProfiles({ provider: "nvidia", store: authStore });
|
||||||
|
if (nvidiaKey) {
|
||||||
|
providers.nvidia = { ...buildNvidiaProvider(), apiKey: nvidiaKey };
|
||||||
|
}
|
||||||
|
|
||||||
return providers;
|
return providers;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user