> ## Documentation Index
> Fetch the complete documentation index at: https://launchdarkly-preview.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Node.js (server-side) AI SDK reference

> This topic documents how to get started with the Node.js (server-side) AI SDK, and links to reference information on all of the supported features. You can use either JavaScript or TypeScript when working with the Node.js (server-side) AI SDK.

> For the complete documentation index, see [llms.txt](/llms.txt). For the full documentation snapshot, see [llms-full.txt](/llms-full.txt).

## Overview

This topic documents how to get started with the Node.js (server-side) AI SDK, and links to reference information on all of the supported features. You can use either JavaScript or TypeScript when working with the Node.js (server-side) AI SDK.

The Node.js (server-side) AI SDK is designed for use with LaunchDarkly's AI Configs. It is currently in a pre-1.0 release and under active development. You can follow development or contribute on [GitHub](https://github.com/launchdarkly/js-core/tree/main/packages/sdk/server-ai/).

<Note title="SDK quick links">
  LaunchDarkly's SDKs are open source. In addition to this reference guide, we provide source, API reference documentation, sample applications, and provider-specific packages:
</Note>

| Resource | Location |
| - | - |
| SDK API documentation | [SDK API docs](https://launchdarkly.github.io/js-core/packages/sdk/server-ai/docs/) |
| GitHub repository | [node-server-sdk-ai](https://github.com/launchdarkly/js-core/tree/main/packages/sdk/server-ai/) |
| Sample application | [Using Bedrock](https://github.com/launchdarkly/js-core/tree/main/packages/sdk/server-ai/examples/bedrock) , [Using OpenAI](https://github.com/launchdarkly/js-core/tree/main/packages/sdk/server-ai/examples/openai) |
| Published module | [npm](https://www.npmjs.com/package/@launchdarkly/server-sdk-ai) |
| Provider-specific packages | [OpenAI](https://github.com/launchdarkly/js-core/tree/main/packages/ai-providers/server-ai-openai) , [LangChain](https://github.com/launchdarkly/js-core/tree/main/packages/ai-providers/server-ai-langchain) , [Vercel AI](https://github.com/launchdarkly/js-core/tree/main/packages/ai-providers/server-ai-vercel) |

<Note title="For use in server-side applications only">
  This SDK is intended for use in multi-user Node.js server applications. To learn more about LaunchDarkly's different SDK types, read [Choosing an SDK type](/sdk/concepts/client-side-server-side).
</Note>

## Get started

LaunchDarkly AI SDKs interact with [AI Configs](/home/getting-started/vocabulary#ai-config). AI Configs are the LaunchDarkly resources that manage model configurations and messages for your generative AI applications.

<Tip title="Try the Quickstart">
  This reference guide describes working specifically with the Node.js (server-side) AI SDK. For a complete introduction to LaunchDarkly AI SDKs and how they interact with AI Configs, read [Quickstart for AI Configs](/home/ai-configs/quickstart).
</Tip>

You can use the Node.js (server-side) AI SDK to customize your AI Config based on the [context](/home/getting-started/vocabulary#context) that you provide. This means both the messages and the model evaluation in your generative AI application are specific to each end user, at runtime. You can also use the AI SDKs to record metrics from your AI model generation, including duration and tokens.

Follow these instructions to start using the Node.js (server-side) AI SDK in your application.

### Install the SDK

First, install the AI SDK as a dependency in your application using your application's dependency manager. If you want to depend on a specific version, refer to the [SDK releases page](https://github.com/launchdarkly/js-core/releases?q=ai) to identify the latest version.

The Node.js (server-side) AI SDK is built on the [Node.js (server-side) SDK](/sdk/server-side/node-js), so you'll need to install that as well. Alternatively, you can install a server-side [edge SDK](/sdk/edge) instead.

In addition, you can also use provider-specific AI SDK packages for better integration and improved version management with your preferred AI framework.

The following provider-specific packages are available:

* [OpenAI](https://github.com/launchdarkly/js-core/tree/main/packages/ai-providers/server-ai-openai)
* [LangChain](https://github.com/launchdarkly/js-core/tree/main/packages/ai-providers/server-ai-langchain)
* [Vercel AI](https://github.com/launchdarkly/js-core/tree/main/packages/ai-providers/server-ai-vercel)

These packages require Node.js version 16 or higher.

<Warning title="Provider-specific packages are in early development">
  These provider-specific packages are currently in early development and are not recommended for production use. They may change without notice, including becoming backwards incompatible.
</Warning>

Here's how:

<CodeGroup>
  ```bash title="Shell" theme={null}
      npm install @launchdarkly/node-server-sdk
      npm install @launchdarkly/server-sdk-ai
      # If you want to install a provider-specific package
      npm install @launchdarkly/server-sdk-ai-openai
      # or
      npm install @launchdarkly/server-sdk-ai-langchain
      # or
      npm install @launchdarkly/server-sdk-ai-vercel
  ```
</CodeGroup>

Next, import `init`, `LDContext`, and `initAi` in your application code. If you are using TypeScript, you can optionally import the LaunchDarkly `LDAIClient` and `LDAIConfig`. These are implied, so are not strictly required.

Here's how:

<CodeGroup>
  ```ts title="Node.js (server-side) AI SDK (TypeScript)" theme={null}
      import { init, LDContext } from '@launchdarkly/node-server-sdk';
      import { initAi, LDAIClient, LDAIConfig } from '@launchdarkly/server-sdk-ai';
  ```

  ```js title="Node.js (server-side) AI SDK (JavaScript)" theme={null}
      import { init, LDContext } from '@launchdarkly/node-server-sdk';
      import { initAi } from '@launchdarkly/server-sdk-ai';
  ```
</CodeGroup>

Here's how to import a provider-specific package:

<CodeGroup>
  ```js title="OpenAI" theme={null}
      import { OpenAIProvider } from '@launchdarkly/server-sdk-ai-openai';
      import { OpenAI } from 'openai';
  ```

  ```js title="LangChain" theme={null}
      import { LangChainProvider } from '@launchdarkly/server-sdk-ai-langchain';
      import { HumanMessage } from '@langchain/core/messages';
  ```

  ```js title="Vercel AI" theme={null}
      import { VercelProvider } from '@launchdarkly/server-sdk-ai-vercel';
      import { generateText } from 'ai';
  ```
</CodeGroup>

### Initialize the client

After you install and import the SDK, create a single, shared instance of `LDClient`. When the `LDClient` is initialized, use it to initialize the `LDAIClient`. The `LDAIClient` is how you interact with AI Configs. Specify the SDK key to authorize your application to connect to a particular environment within LaunchDarkly.

<Note title="The Node.js SDKs use an SDK key">
  Both the Node.js (server-side) AI SDK and the Node.js (server-side) SDK use an SDK key. Keys are specific to each project and environment. They are available from **Project settings**, on the **Environments** list. To learn more about key types, read [Keys](/sdk/concepts/client-side-server-side#keys).
</Note>

Here's how:

<CodeGroup>
  ```ts title="Node.js (server-side) AI SDK (TypeScript)" theme={null}
      const ldClient: LDClient = init('YOUR_SDK_KEY');

      try {
        await ldClient.waitForInitialization({ timeout: 10 });
        // initialization complete
      } catch (error) {
        // timeout or SDK failed to initialize
      }

      const aiClient: LDAIClient = initAi(ldClient);
  ```

  ```js title="Node.js (server-side) AI SDK (JavaScript)" theme={null}
      const ldClient = init('YOUR_SDK_KEY');

      try {
        await ldClient.waitForInitialization({ timeout: 10 });
        // initialization complete
      } catch (error) {
        // timeout or SDK failed to initialize
      }

      const aiClient = initAi(ldClient);
  ```
</CodeGroup>

### Configure the context

Configure the context that will encounter generated AI content in your application. The context attributes determine which variation of the AI Config LaunchDarkly serves to the end user, based on the targeting rules in your AI Config. If you are using template variables in the messages in your AI Config's variations, the context attributes also fill in values for the template variables.

Here's how:

<CodeGroup>
  ```ts title="Node.js (server-side) AI SDK (TypeScript)" theme={null}
      const context: LDContext = {
        kind: 'user',
        key: 'example-user-key',
        firstName: 'Sandy',
        lastName: 'Smith',
        email: 'sandy@example.com',
        groups: ['Acme', 'Global Health Services'],
      };
  ```

  ```js title="Node.js (server-side) AI SDK (JavaScript)" theme={null}
      const context = {
        kind: 'user',
        key: 'example-user-key',
        firstName: 'Sandy',
        lastName: 'Smith',
        email: 'sandy@example.com',
        groups: ['Acme', 'Global Health Services'],
      };
  ```
</CodeGroup>

### Customize an AI Config

Customize your AI Config to set values for variables used in messages or instructions based on the context attributes and variables you provide.

The details of customizing an AI Config depend on whether you are using AI Configs in completion mode or agent mode. You set the mode for a particular AI Config when you [create it](/home/ai-configs/create) in the LaunchDarkly UI.

#### Customize AI Configs in completion mode

In completion mode, each variation in your AI Config includes a single set of roles and messages used to prompt your generative AI model.

In `completion` mode, use `completionConfig()` to customize the AI Config. The `completionConfig()` function takes an AI Config key, a context, a fallback value, and optional variables to use in the customization. It performs the evaluation, then returns the customized messages and model along with a tracker instance for recording metrics. If it cannot perform the evaluation or LaunchDarkly is unreachable, it returns the fallback value. For example, you might use an empty, disabled configuration object as a fallback value, or a fully configured default. Either way, you should make sure to check for this case and handle it appropriately in your application.

Here's how:

<CodeGroup>
  ```ts title="Customize an AI Config in completion mode (TypeScript)" theme={null}
      const fallbackConfig = { enabled: false };

      const aiConfig: LDAIConfig = await aiClient.completionConfig(
        'example-ai-config-key',
        context,
        fallbackConfig,
        { 'exampleCustomVariable': 'exampleCustomValue' },
      );
  ```

  ```js title="Customize an AI Config in completion mode (JavaScript)" theme={null}
      const fallbackConfig = {
        enabled: false,
      };

      const aiConfig = await aiClient.completionConfig(
        'example-ai-config-key',
        context,
        fallbackConfig,
        { 'exampleCustomVariable': 'exampleCustomValue' },
      );
  ```
</CodeGroup>

#### Customize AI Configs in agent mode

In agent mode, each variation in your AI Config includes a set of instructions, which enable multi-step workflows.

In `agent` mode, use `agentConfig()` or `agentConfigs()` to customize the AI Config. The `agentConfig()` function customizes a single AI Config agent, while the `agentConfigs()` function customizes an array of agent configurations.

The instructions returned by the SDK come directly from the instructions you define for the AI Config variation in the LaunchDarkly UI. The goal or task shown in the UI is delivered unchanged as the `instructions` field in the SDK.

Customization requires an AI Config key, fallback value, and optional variables for each agent-mode evaluation. Additionally, customization requires a context. Both functions perform the evaluation and then return the customized instructions for each AI Config, along with a tracker instance for recording metrics. If the function cannot perform the evaluation or LaunchDarkly is unreachable, it returns the fallback value.

For example, you might use an empty, disabled configuration object as a fallback value, or a fully configured default. Either way, you should make sure to check for this case and handle it appropriately in your application.

Here's how:

<CodeGroup>
  ```ts title="Customize an AI Config in agent mode (TypeScript)" theme={null}
      const fallbackConfig = { enabled: false };

      const agent: LDAIAgentConfig = await aiClient.agentConfig(
        'example-ai-config-key',
        context,
        fallbackConfig,
        { 'exampleCustomVariable': 'exampleCustomValue' },
      );
  ```

  ```js title="Customize an AI Config in agent mode (JavaScript)" theme={null}
      const fallbackConfig = {
        enabled: false,
      };

      const agent = await aiClient.agentConfig(
        'example-ai-config-key',
        context,
        fallbackConfig,
        { 'exampleCustomVariable': 'exampleCustomValue' },
      );
  ```
</CodeGroup>

To learn more, read [Customizing AI Configs](/sdk/features/ai-config#nodejs-server-side-ai).

### Create a client or model instance

If you're using a provider-specific package, you can create a client (OpenAI) or a model instance (LangChain and Vercel AI). Here's how:

<CodeGroup>
  ```js title="OpenAI" theme={null}
      const client = new OpenAI({
        apiKey: process.env.OPENAI_API_KEY,
      });
  ```

  ```js title="LangChain" theme={null}
      const llm = await LangChainProvider.createLangChainModel(aiConfig);
  ```

  ```js title="Vercel AI" theme={null}
      const model = await VercelProvider.createVercelModel(aiConfig);
  ```
</CodeGroup>

### Combine AI Config messages with user messages

With your model ready, you can now combine the LaunchDarkly-provided messages with user input. If you're using a provider-specific package, you can combine an AI Config message with a user message. Here's how:

<CodeGroup>
  ```js title="OpenAI" theme={null}
      const configMessages = aiConfig.messages || [];
      const userMessage = { role: 'user', content: 'What is the capital of France?' };
      const allMessages = [...configMessages, userMessage];
  ```

  ```js title="LangChain" theme={null}
      const configMessages = aiConfig.messages || [];
      const userMessage = new HumanMessage('What is the capital of France?');
      const allMessages = [...LangChainProvider.convertMessagesToLangChain(configMessages), userMessage];
  ```

  ```js title="Vercel AI" theme={null}
      const configMessages = aiConfig.messages || [];
      const userMessage = { role: 'user', content: 'What is the capital of France?' };
      const allMessages = [...configMessages, userMessage];
  ```
</CodeGroup>

### Call the provider and record AI metrics

Finally, using a provider-specific package, make a request to your generative AI provider and record metrics from your AI model generation.

Tracking AI metrics does not automatically associate traces with the AI Config.

<Note title="Run model calls inside an active span to associate traces">
  LaunchDarkly can associate traces with an evaluated AI Config only when the model request runs inside an active OpenTelemetry span. For new application development, we recommend using the LaunchDarkly Observability SDK plugin to create and manage spans.

  If the model call does not run inside an active span, LaunchDarkly cannot associate the trace with the AI Config.

  To learn more, read [LLM observability](/home/observability/llm-observability) and [Monitor AI Configs](/home/ai-configs/monitor).
</Note>

The following example shows how to use the LaunchDarkly Observability SDK plugin to run a model request inside an active span so LaunchDarkly can associate the resulting trace with the evaluated AI Config.

This example assumes you have configured the LaunchDarkly Observability SDK plugin and have access to request headers.

<CodeGroup>
  ```js title="Using the LaunchDarkly Observability SDK plugin" theme={null}
      await LDObserve.runWithHeaders('llm.chat', req.headers, async (span) => {
        return await aiConfig.tracker.trackMetricsOf(
          (result) => OpenAIProvider.createAIMetrics(result),
          () => client.chat.completions.create({
            model: aiConfig.model?.name || 'gpt-4',
            messages: aiConfig.messages || [],
          })
        );
      });
  ```
</CodeGroup>

After you ensure your model request runs inside an active span, you can call your provider and record AI metrics as shown in the examples below.

<CodeGroup>
  ```js title="OpenAI" theme={null}
      const response = await aiConfig.tracker.trackMetricsOf(
        (result) => OpenAIProvider.createAIMetrics(result),
        () => client.chat.completions.create({
          model: aiConfig.model?.name || 'gpt-4',
          messages: aiConfig.messages || [],
          temperature: (aiConfig.model?.parameters?.temperature as number) ?? 0.5,
        })
      );

      console.log('AI Response:', response.choices[0].message.content);

  ```

  ```js title="LangChain" theme={null}
      const response = await aiConfig.tracker.trackMetricsOf(
        (result) => LangChainProvider.createAIMetrics(result),
        () => llm.invoke(allMessages)
      );

      console.log('AI Response:', response.content);
  ```

  ```js title="Vercel AI" theme={null}
      const response = await aiConfig.tracker.trackMetricsOf(
        (result) => VercelProvider.createAIMetrics(result),
        () => generateText({ model, messages: allMessages })
      );

      console.log('AI Response:', response.text);
  ```

  ```js title="Amazon Bedrock" theme={null}
      import { ConverseCommand } from '@aws-sdk/client-bedrock-runtime';

      // Use the LaunchDarkly Node.js AI SDK to call an Amazon Bedrock model
      // and automatically record metrics such as token usage and latency.

      const response = await aiConfig.tracker.trackBedrockConverseMetrics(async () => {
        return awsClient.send(
          new ConverseCommand({
            modelId: aiConfig.model?.name ?? 'anthropic.claude-3',
            messages: mapPromptToConversation(aiConfig.messages ?? []),
            inferenceConfig: {
              temperature: (aiConfig.model?.parameters?.temperature as number) ?? 0.5,
              maxTokens: (aiConfig.model?.parameters?.maxTokens as number) ?? 4096,
            },
          })
        );
      });

      console.log('AI Response:', response.output?.message?.content);
  ```
</CodeGroup>

To learn more, read [Tracking AI metrics](/sdk/features/ai-metrics#nodejs-server-side-ai).

## Supported features

This SDK supports the following features:

* [Anonymous contexts](/sdk/features/anonymous#nodejs-server-side-ai)
* [Context configuration](/sdk/features/context-config/ai#nodejs-server-side-ai)
* [Customizing AI Configs](/sdk/features/ai-config#nodejs-server-side-ai)
* [Private attributes](/sdk/features/private-attributes/ai#nodejs-server-side-ai)
* [Tracking AI metrics](/sdk/features/ai-metrics#nodejs-server-side-ai)