> ## Documentation Index
> Fetch the complete documentation index at: https://launchdarkly-preview.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Tracking AI metrics
> This topic explains how to record metrics from your AI model generation, including duration, generation, satisfaction, and several token-related metrics.
> For the complete documentation index, see [llms.txt](/llms.txt). For the full documentation snapshot, see [llms-full.txt](/llms-full.txt).
## Overview
This topic explains how to record metrics from your AI model generation, including duration, generation, satisfaction, and several token-related metrics.
Use LaunchDarkly AI SDKs to record these metrics by wrapping your AI model provider call with a tracking method so metrics are captured as part of generation. LaunchDarkly displays recorded metrics on the [AI Config **Monitoring** tab](/home/ai-configs/monitor).
This feature is available for AI SDKs only. LaunchDarkly AI SDKs are designed for use with LaunchDarkly AI Configs and are currently in a pre-1.0 release under active development.
All AI SDKs include `track*` methods to record:
* duration
* token usage
* generation success
* generation error
* time to first token
* output satisfaction
Each AI SDK also includes a method to retrieve a summary of the metrics that have been automatically collected.
Some AI SDKs also include provider-specific `track_[model]_metrics` methods for AI Configs in completion mode. These methods take the result of a provider call and record:
* duration
* token usage
* generation success
* generation error
You can use provider-specific methods as a shorthand, or call `track*` methods directly to record additional metrics.
Both `track*` and `track_[model]_metrics` methods are called from a `tracker`. The `tracker` is returned by an AI Config [customization call](/sdk/features/ai-config) and is available for AI Configs in both completion mode and agent mode.
## Record delayed feedback events
The SDK expects you to use the `tracker` within the same request lifecycle that generates AI content. Do not reuse a `tracker` across separate requests.
If user feedback arrives later, you must persist the tracking metadata to ensure the feedback is attributed to the same AI Config variation that generated the content. The `tracker` exposes `getTrackData()` so you can capture this metadata at generation time and reuse it when feedback arrives.
Use the following pattern to record delayed feedback:
1. Use `tracker.getTrackData()` to capture the tracker metadata at generation time.
2. Store the metadata alongside the context where generation results are stored.
3. Retrieve the stored metadata when feedback arrives.
4. Send the tracking event with the original metadata.
Here is an example using `ldClient.track`:
```ts theme={null}
// Generation time
const aiConfig = await aiClient.completionConfig(
aiConfigKey,
context,
defaultValue,
variables
);
const trackData = aiConfig.tracker.getTrackData();
// Persist trackData together with the context
// Feedback time
ldClient.track(
feedback.kind === "positive"
? "$ld:ai:feedback:user:positive"
: "$ld:ai:feedback:user:negative",
context, // context from generation time
persistedTrackData,
1
);
```
## AI SDKs
This feature is available for the following AI SDKs:
* [.NET AI](#net-ai)
* [Go AI](#go-ai)
* [Node.js (server-side) AI](#nodejs-server-side-ai)
* [Python AI](#python-ai)
* [Ruby AI](#ruby-ai)
### .NET AI
Use the `TrackRequest` function to wrap your AI model provider call and record metrics when the model generates content.
The `tracker` is returned from your call to [customize the AI Config](/sdk/features/ai-config#net-ai) and is specific to that AI Config. Make sure to call `Config` again each time you generate content from your AI model so that metrics are correctly associated with the customized AI Config variation.
Here's how to call an AI model provider and record metrics from generation:
```csharp title=".NET AI SDK, any model" theme={null}
if (tracker.Config.Enabled == true) {
// Use Task.Run here only as an example. You can wrap any async provider call.
var response = tracker.TrackRequest(Task.Run(() =>
{
// Make request to a provider, which automatically tracks metrics in LaunchDarkly.
// When sending the request to a provider, use details from tracker.Config.
// For example, you can pass tracker.Config.Model and tracker.Config.Messages.
// Optionally, return response metadata for additional logging.
//
// CAUTION: If the call inside Task.Run throws an exception,
// the SDK will re-throw that exception.
return new Response
{
Usage = new Usage { Total = 1, Input = 1, Output = 1 }, // Token usage data
Metrics = new Metrics { LatencyMs = 100 } // Metrics data
};
}
));
} else {
// Application path to take when tracker.Config is disabled
}
```
If you want to perform additional tracking beyond what LaunchDarkly provides automatically, populate the `Response` object with the metrics you want to record.
You can use the SDK's other `Track*` functions to record metrics manually. The `TrackRequest` function expects a response, so manual tracking may be required for streaming use cases.
Each of the `Track*` functions sends data back to LaunchDarkly. The [Monitoring tab](/home/ai-configs/monitor) of the AI Config in the LaunchDarkly UI aggregates metrics from all variations of the AI Config.
Here's how to record metrics manually:
```csharp title="Track duration" theme={null}
/// Track your own start and stop time.
/// Set duration to the time (in ms) that your AI model generation takes.
/// The duration may include network latency, depending on how you calculate it.
tracker.TrackDuration(response.Metrics.LatencyMs);
```
```csharp title="Track token usage" theme={null}
/// Track your own token usage.
tracker.TrackTokens(response.Usage);
```
```csharp title="Track output satisfaction rate" theme={null}
/// Track your own output satisfaction rate.
/// Pass in Feedback.Positive or Feedback.Negative.
tracker.TrackFeedback(Feedback.Positive);
```
```csharp title="Track generation (success)" theme={null}
/// Track a successful generation event using cfg
tracker.TrackSuccess();
```
```csharp title="Track generation (error)" theme={null}
/// Track an unsuccessful generation event using cfg
tracker.TrackError();
```
```csharp title="Track time to first token" theme={null}
/// Track the time it takes to generate the first token
/// Pass in the time (in ms) until your first token is generated
/// This may include network latency, depending on how you calculate it
tracker.TrackTimeToFirstToken(1000);
```
The SDK automatically flushes pending analytics events to LaunchDarkly at regular intervals. If you have a short-lived application, you may need to explicitly request that the underlying LaunchDarkly client deliver any pending analytics events to LaunchDarkly, using [`flush()`](/sdk/features/flush) or [`close()`](/sdk/features/shutdown).
Here's how:
```csharp title="Flush tracking events" theme={null}
baseClient.Flush();
```
To learn more, read [`LDAIConfigTracker`](https://launchdarkly.github.io/dotnet-core/pkgs/sdk/server-ai/api/LaunchDarkly.Sdk.Server.Ai.LdAiConfigTracker.html).
### Go AI
Use the `TrackRequest()` function to wrap your AI model provider call and record metrics from generation.
The `tracker` is returned from your call to [customize the AI Config](/sdk/features/ai-config#go-ai) and is specific to that AI Config. Make sure to call `Config()` again each time you use the tracker and generate content from your AI model so that your metrics are correctly associated with the customized AI Config variation.
Here's how to call an AI model provider and record metrics from generation:
```go title="Go AI SDK, any model" theme={null}
if cfg.Enabled() {
response, err := tracker.TrackRequest(func(config *Config) (ProviderResponse, error) {
// Make request to a provider, which automatically tracks metrics in LaunchDarkly.
// When sending the request to a provider, use details from config.
// For example, you can pass a model parameter (config.ModelParam) or messages (config.Messages).
// Optionally, return response metadata for additional logging.
return ProviderResponse{
Usage: TokenUsage{
Total: 1, // Token usage data
},
Metrics: Metrics{
Latency: 10 * time.Millisecond, // Metrics data
},
}, nil
})
} else {
// Application path to take when the configuration is disabled
}
```
You can use the SDK's other `Track*` functions to record metrics manually. The `TrackRequest` function expects a response, so manual tracking may be required for streaming use cases.
Each `Track*` function sends data back to LaunchDarkly. To review the metrics that have been recorded, use `GetSummary`:
```go title="Get automatically recorded metrics" theme={null}
summary := tracker.GetSummary();
// recorded metrics available in summary.Duration, summary.Feedback,
// summary.Tokens, summary.Success, summary.TimeToFirstToken
```
To learn more, read [`GetSummary`](https://pkg.go.dev/github.com/launchdarkly/go-server-sdk/ldai#Tracker.GetSummary) and [`MetricSummary`](https://pkg.go.dev/github.com/launchdarkly/go-server-sdk/ldai#MetricSummary).
The [Monitoring tab](/home/ai-configs/monitor) of the AI Config in the LaunchDarkly UI aggregates data from the `Track*` functions from across all variations of the AI Config.
Here's how to record metrics manually:
```go title="Track duration" theme={null}
// Track your own start and stop time.
// Set duration to the time that your AI model generation takes.
// The duration may include network latency, depending on how you calculate it.
tracker.TrackDuration(10 * time.Millisecond);
```
```go title="Track token usage" theme={null}
// Track your own token usage.
tracker.TrackUsage(response.Usage);
```
```go title="Track output satisfaction rate" theme={null}
// Track your own output satisfaction rate.
// Pass in feedbackPositive or feedbackNegative.
tracker.TrackFeedback(ldai.FeedbackPositive);
```
```go title="Track generation (success)" theme={null}
// Track a successful generation event using cfg
tracker.TrackSuccess();
```
```go title="Track generation (error)" theme={null}
// Track an unsuccessful generation event using cfg
tracker.TrackError();
```
```go title="Track time to first token" theme={null}
// Track the time it takes to generate the first token
// Pass in the time (in ms) until your first token is generated
// This may include network latency, depending on how you calculate it
tracker.TrackTimeToFirstToken(10 * time.Millisecond);
```
The SDK automatically flushes these pending analytics events to LaunchDarkly at regular intervals. If you have a short-lived application, you may need to explicitly request that the underlying LaunchDarkly client deliver any pending analytics events to LaunchDarkly, using [`flush()`](/sdk/features/flush) or [`close()`](/sdk/features/shutdown).
Here's how:
```go title="Flush tracking events" theme={null}
client.Flush();
```
To learn more, read [`Tracker`](https://pkg.go.dev/github.com/launchdarkly/go-server-sdk/ldai#Tracker).
### Node.js (server-side) AI
If your AI Config uses completion mode, the Node.js (server-side) AI SDK provides several options for making a request to your generative AI provider and recording metrics from your AI model generation. You can use any of the following options:
* If you are working with OpenAI or Bedrock Converse, use the `trackOpenAIMetrics` or `trackBedrockConverseMetrics` functions, respectively, to record metrics. These functions take the result of your generative operation as a parameter.
* If you are working with Vercel, use either of the `trackVercelAISDKGenerateTextMetrics` or `trackVercelAISDKStreamTextMetrics` functions to record metrics. These functions take the result of a generative operation from any provider supported by the Vercel AI SDK as a parameter.
* If you are using a generative AI provider or framework for which the SDK does not provide a convenience function, use the SDK's other `track*` functions to record metrics manually.
If your AI Config uses `agent` mode, you can access the `instructions` returned from the `agentConfig()` call to send to your AI model. Use the `tracker` returned in this call to record metrics.
In the following examples, the `tracker` is from your call to [customize the AI Config](/sdk/features/ai-config#nodejs-server-side-ai), and is specific to that AI Config. Make sure to call `completionConfig` again each time you use the tracker and generate content from your AI model, so that your metrics are correctly associated with the customized AI Config variation.
Here's how:
```ts title="Using OpenAI model, completion mode" theme={null}
const { tracker } = aiConfig;
if (aiConfig.enabled) {
// Pass in the result of the OpenAI operation.
// When you call the OpenAI operation, use details from aiConfig.
// For instance, you can pass aiConfig.messages
// and aiConfig.model to your specific OpenAI operation.
//
// CAUTION: If the call inside of trackOpenAIMetrics throws an exception,
// the SDK will re-throw that exception
const completion = await tracker.trackOpenAIMetrics(async () =>
client.chat.completions.create({
messages: aiConfig.messages || [],
model: aiConfig.model?.name || 'gpt-4',
temperature: (aiConfig.model?.parameters?.temperature as number) ?? 0.5,
maxTokens: (aiConfig.model?.parameters?.maxTokens as number) ?? 4096,
}),
);
} else {
// Application path to take when the aiConfig is disabled
}
```
```ts title="Using Bedrock model, completion mode" theme={null}
const { tracker } = aiConfig;
if (aiConfig.enabled) {
// Pass in the result of the Bedrock Converse command.
// When you call the Bedrock Converse command, use details from aiConfig.
// For instance, you can pass aiConfig.messages
// and aiConfig.model to your specific Bedrock Converse command.
const completion = tracker.trackBedrockConverseMetrics(
await awsClient.send(
new ConverseCommand({
modelId: aiConfig.model?.name ?? 'no-model',
messages: mapPromptToConversation(aiConfig.messages ?? []),
inferenceConfig: {
temperature: (aiConfig.model?.parameters?.temperature as number) ?? 0.5,
maxTokens: (aiConfig.model?.parameters?.maxTokens as number) ?? 4096,
},
}),
),
);
} else {
// Application path to take when the aiConfig is disabled
}
```
```js title="Accessing instructions and recording metrics, agent mode" theme={null}
if (agent.enabled) {
// Retrieve instructions from the AI Config and pass to your AI model
const result = example_model_api(agent.instructions)
// Track metrics from the result
agent.tracker.track_success()
} else {
// Application path to take when the AI Config agent is disabled
}
```
Here's how to make a request using the Vercel AI SDK's [`generateText`](https://ai-sdk.dev/docs/ai-sdk-core/generating-text#generatetext) or [`streamText`](https://ai-sdk.dev/docs/ai-sdk-core/generating-text#streamtext), and record the metrics:
```ts title="Using Vercel and generateText" theme={null}
const { tracker } = aiConfig;
// Pass in the result of the Vercel AI SDK's generateText function.
// When you call generateText, use details from the aiConfig,
// mapped to the input format required for the Vercel AI SDK.
//
// CAUTION: The toVercelAISDK function may throw an exception
// if a Vercel AI SDK model cannot be determined.
const completion = await tracker.trackVercelAISDKGenerateTextMetrics(
generateText(
aiConfig.toVercelAISDK(vercelProvider, vercelProviderOptions)
)
)
```
```ts title="Using Vercel and streamText" theme={null}
const { tracker } = aiConfig;
// Pass in the result of the Vercel AI SDK's streamText function.
// When you call streamText, use details from the aiConfig,
// mapped to the input format required for the Vercel AI SDK.
//
// CAUTION: The toVercelAISDK function may throw an exception
// if a Vercel AI SDK model cannot be determined.
const completion = tracker.trackVercelAISDKStreamTextMetrics(
streamText(
aiConfig.toVercelAISDK(vercelProvider, vercelProviderOptions)
)
)
```
To learn more, read [`trackVercelAISDKGenerateTextMetrics`](https://launchdarkly.github.io/js-core/packages/sdk/server-ai/docs/interfaces/LDAIConfigTracker.html#trackVercelAISDKGenerateTextMetrics), [`trackVercelAISDKStreamTextMetrics`](https://launchdarkly.github.io/js-core/packages/sdk/server-ai/docs/interfaces/LDAIConfigTracker.html#trackVercelAISDKStreamTextMetrics), and [`toVercelAISDK`](https://launchdarkly.github.io/js-core/packages/sdk/server-ai/docs/interfaces/LDAIConfig.html#toVercelAISDK).
You can use the SDK's other `track*` functions to record these metrics manually. You may need to do this if you are using a model for which the SDK does not provide a convenience `track[Model]Metrics` function, and you are not using the Vercel AI SDK. The `track[Model]Metrics` functions are expecting a response, so you may also need to do this if your application requires streaming.
Each of the `track*` functions sends data back to LaunchDarkly. To review the metrics that have been recorded, use `getSummary`:
```ts title="Get automatically recorded metrics, completion mode" theme={null}
const summary = aiConfig.tracker.getSummary();
// recorded metrics available in summary.durationMs, summary.feedback,
// summary.tokens, summary.success, summary.timeToFirstTokenMs
```
```ts title="Get automatically recorded metrics, agent mode" theme={null}
const summary = agent.tracker.getSummary();
// recorded metrics available in summary.durationMs, summary.feedback,
// summary.tokens, summary.success, summary.timeToFirstTokenMs
```
To learn more, read [`getSummary`](https://launchdarkly.github.io/js-core/packages/sdk/server-ai/docs/interfaces/LDAIConfigTracker.html#getSummary).
The [Monitoring tab](/home/ai-configs/monitor) of the AI Config in the LaunchDarkly UI aggregates data from the `track*` functions from across all variations of the AI Config.
Here's how to record metrics manually:
```ts title="Track duration" theme={null}
// Track your own start and stop time.
// Set duration to the time (in ms) that your AI model generation takes.
// The duration may include network latency, depending on how you calculate it.
aiConfig.tracker.trackDuration(duration);
```
```ts title="Track token usage" theme={null}
import { LDTokenUsage } from '@launchdarkly/server-sdk-ai';
// Track your own token usage.
// First, set up an LDTokenUsage object.
// Update the input, output, and total fields
// with return values from your AI model generation.
const tokens: LDTokenUsage = {
input: 0,
output: 0,
total: 0,
}
aiConfig.tracker.trackTokens(tokens);
```
```ts title="Track output satisfaction rate" theme={null}
import { LDFeedbackKind } from '@launchdarkly/server-sdk-ai';
// Track your own output satisfaction rate.
// Pass in LDFeedbackKind.Positive or LDFeedbackKind.Negative.
aiConfig.tracker.trackFeedback({ kind: LDFeedbackKind.Positive });
```
```ts title="Track generation (success)" theme={null}
// Track a successful generation event using cfg
aiConfig.tracker.trackSuccess()
```
```ts title="Track generation (error)" theme={null}
// Track an unsuccessful generation event using cfg
aiConfig.tracker.trackError()
```
```ts title="Track time to first token" theme={null}
// Track the time it takes to generate the first token
// Pass in the time (in ms) until your first token is generated
// This may include network latency, depending on how you calculate it
aiConfig.tracker.trackTimeToFirstToken(1000);
```
In `completion` mode, the `tracker` is returned from the `completionConfig()` call, so you can access it directly, as in the examples above. Make sure to call `completionConfig()` again each time you use the tracker and generate content from your AI model.
In `agent` mode, the `tracker` is part of the `agent` returned from the `agentConfig()` or `agentConfigs()` call. If you are working in `agent` mode, replace `tracker` with `agent.tracker` in the examples above.
The SDK automatically flushes these pending analytics events to LaunchDarkly at regular intervals. If you have a short-lived application, you may need to explicitly request that the underlying LaunchDarkly client deliver any pending analytics events to LaunchDarkly, using [`flush()`](/sdk/features/flush) or [`close()`](/sdk/features/shutdown).
Here's how:
```js title="Flush tracking events" theme={null}
ldClient.flush();
```
To learn more, read [`LDAIConfigTracker`](https://launchdarkly.github.io/js-core/packages/sdk/server-ai/docs/interfaces/LDAIConfigTracker.html).
### Python AI
If your AI Config uses completion mode, use one of the `track_[model]_metrics` functions to record metrics from your AI model generation. The SDK provides separate `track_[model]_metrics` functions for several of the models that you can select when you set up your [AI Config variations](/home/ai-configs/create-variation) in the LaunchDarkly user interface.
If your AI Config uses `agent` mode, you can access the `instructions` returned from the customized AI Config to send to your AI model. Use the `tracker` returned as part of the `agent_config()` or `agent_configs()` functions to record metrics.
The `tracker` is returned from your call to [customize the AI Config](/sdk/features/ai-config#python-ai), and is specific to that AI Config. Make sure to call `completion_config` again each time you use the tracker and generate content from your AI model, so that your metrics are correctly associated with the customized AI Config variation.
Here's how:
```python title="Using OpenAI model, completion mode" theme={null}
if config.enabled:
# Pass in the result of the OpenAI operation.
# When calling the OpenAI operation, use details from config.
# For instance, you can pass config.model.name
# and config.messages[0].content to your specific OpenAI operation.
#
# CAUTION: If the call inside of track_openai_metrics throws an exception,
# the SDK will re-throw that exception
messages = [] if config.messages is None else config.messages
completion = tracker.track_openai_metrics(
lambda:
openai_client.chat.completions.create(
model=config.model.name,
messages=[message.to_dict() for message in messages],
)
)
else:
# Application path to take when the config is disabled
```
```python title="Using Bedrock model, completion mode" theme={null}
if config.enabled:
# Pass in the result of the Bedrock Converse command.
# When calling the Bedrock Converse command, use details from config.
# For instance, you can pass config.model.name
# and config.messages[0].content to your specific Bedrock Converse command.
completion = tracker.track_bedrock_converse_metrics(
client.converse(
modelId=config.model.name,
messages=map_messages_to_conversation(config.messages)
)
)
else:
# Application path to take when the config is disabled
```
```python title="Accessing instructions and recording metrics, agent mode" theme={null}
if agent.enabled:
# Retrieve instructions from the AI Config and pass to your AI model
var result = example_model_api(agent.instructions)
# Track metrics from the result
agent.tracker.track_success()
else:
# Application path to take when the agent is disabled
```
You can use the SDK's other `track*` functions to record metrics manually. This is useful when the SDK does not provide a convenience `track_[model]_metrics` function for your model, or when your application requires streaming, because `track_[model]_metrics` functions expect a response.
Each `track*` function sends data back to LaunchDarkly. To review the metrics that have been recorded, use `get_summary`:
```python title="Get automatically recorded metrics, completion mode" theme={null}
tracker.get_summary()
# recorded metrics available in tracker.get_summary().duration, .feedback,
# .success, .usage, and .time_to_first_token
```
```python title="Get automatically recorded metrics, agent mode" theme={null}
agent.tracker.get_summary()
# recorded metrics available in agent.tracker.get_summary().duration, .feedback,
# .success, .usage, and .time_to_first_token
```
To learn more, read [`get_summary`](https://launchdarkly-python-sdk-ai.readthedocs.io/en/latest/api-main.html#ldai.tracker.LDAIConfigTracker.get_summary) and [`LDAIMetricSummary`](https://launchdarkly-python-sdk-ai.readthedocs.io/en/latest/api-main.html#ldai.tracker.LDAIMetricSummary).
The [Monitoring tab](/home/ai-configs/monitor) of the AI Config in the LaunchDarkly UI aggregates data from the `track*` functions across all variations of the AI Config.
Here's how to record metrics manually:
```python title="Track duration" theme={null}
# Track your own start and stop time.
# Set duration to the time (in ms) that your AI model generation takes.
# The duration may include network latency, depending on how you calculate it.
tracker.track_duration(duration)
```
```python title="Track token usage" theme={null}
# Track your own token usage.
# TokenUsage is provided by the LaunchDarkly AI SDK.
# Update the input, output, and total values
# with return values from your AI model generation.
tokens = TokenUsage(0, 0, 0)
tracker.track_tokens(tokens)
```
```python title="Track output satisfaction rate" theme={null}
# Track your own output satisfaction rate.
# Pass in FeedbackKind.Positive or FeedbackKind.Negative.
tracker.track_feedback(FeedbackKind.Positive)
```
```python title="Track generation (success)" theme={null}
# Track a successful generation event using your AI Config.
tracker.track_success()
```
```python title="Track generation (error)" theme={null}
# Track an unsuccessful generation event using your AI Config.
tracker.track_error()
```
```python title="Track time to first token" theme={null}
# Track the time it takes to generate the first token.
# Pass in the time (in ms) until your first token is generated.
# This may include network latency, depending on how you calculate it.
tracker.track_time_to_first_token(1000)
```
In `completion` mode, the `tracker` is returned from the `completionConfig()` call, so you can access it directly, as in the examples above. Make sure to call `completionConfig()` again each time you use the tracker and generate content from your AI model.
In `agent` mode, the `tracker` is part of the `agent` returned from the `agentConfig()` or `agentConfigs()` call. If you are working in `agent` mode, replace `tracker` with `agent.tracker` in the examples above.
The SDK automatically flushes pending analytics events to LaunchDarkly at regular intervals. If you have a short-lived application, you may need to explicitly request that the underlying LaunchDarkly client deliver any pending analytics events to LaunchDarkly, using [`flush()`](/sdk/features/flush) or [`close()`](/sdk/features/shutdown).
Here's how:
```python title="Flush tracking events" theme={null}
ldclient.get().flush()
```
To learn more, read [`LDAIConfigTracker`](https://launchdarkly-python-sdk-ai.readthedocs.io/en/latest/api-main.html#ldai.tracker.LDAIConfigTracker).
### Ruby AI
Use one of the `track_[model]_metrics` functions to wrap your AI model provider call and record metrics from generation. The SDK provides separate `track_[model]_metrics` functions for several models that you can select when you set up your [AI Config variations](/home/ai-configs/create-variation) in the LaunchDarkly UI.
The `tracker` is returned as part of your call to [customize the AI Config](/sdk/features/ai-config#ruby-ai) and is specific to that AI Config. Make sure to call `config` again each time you use the tracker and generate content from your AI model so your metrics are correctly associated with the customized AI Config variation.
Here's how to use a provider-specific function to call OpenAI or Bedrock providers and record metrics from your AI model generation:
```ruby title="Using OpenAI model" theme={null}
if ai_config.enabled
# Wrap the OpenAI operation to record metrics.
# When calling the OpenAI operation, use details from ai_config.
# For instance, you can pass ai_config.model.name
# and ai_config.messages to your specific OpenAI operation.
#
# CAUTION: If the call inside track_openai_metrics throws an exception,
# the SDK will re-throw that exception.
completion = ai_config.tracker.track_openai_metrics(
openai_client.chat.completions.create(
model: ai_config.model.name,
messages: ai_config.messages.map(&:to_h)
)
)
else
# Application path to take when the ai_config is disabled
end
```
```ruby title="Using Bedrock model" theme={null}
if ai_config.enabled
# Wrap the Bedrock Converse command to record metrics.
# When calling the Bedrock Converse command, use details from ai_config.
# For instance, you can pass ai_config.model.name
# and ai_config.messages to your specific Bedrock Converse command.
completion = ai_config.tracker.track_bedrock_converse_metrics(
bedrock_client.converse(
map_converse_arguments(
ai_config.model.name,
ai_config.messages
)
)
)
else
# Application path to take when the ai_config is disabled
end
```
```ruby title="Bedrock helper function" theme={null}
# The map_converse_arguments helper function transforms the messages
# to fit the Bedrock SDK.
def map_converse_arguments(model_id, messages)
args = {
model_id: model_id,
}
mapped_messages = []
user_messages = messages.select { |msg| msg.role == 'user' }
mapped_messages << { role: 'user', content: user_messages.map { |msg| { text: msg.content } } } unless user_messages.empty?
assistant_messages = messages.select { |msg| msg.role == 'assistant' }
mapped_messages << { role: 'assistant', content: assistant_messages.map { |msg| { text: msg.content } } } unless assistant_messages.empty?
args[:messages] = mapped_messages unless mapped_messages.empty?
system_messages = messages.select { |msg| msg.role == 'system' }
args[:system] = system_messages.map { |msg| { text: msg.content } } unless system_messages.empty?
args
end
```
You can use the SDK's other `track*` functions to record these metrics manually. You may need to do this if you are using a model for which the SDK does not provide a convenience `track_[model]_metrics` function. The `track_[model]_metrics` functions are expecting a response, so you may also need to do this if your application requires streaming.
Each of the `track*` functions sends data back to LaunchDarkly. To review the metrics that have been recorded, use the `summary` property in the `tracker`:
```ruby title="Get automatically recorded metrics" theme={null}
ai_config.tracker.summary
# recorded metrics available in tracker.summary include: duration, feedback,
# success, usage, and time_to_first_token
```
To learn more, read [`summary`](https://launchdarkly.github.io/ruby-server-sdk-ai/LaunchDarkly/Server/AI/AIConfigTracker.html#summary-instance_method) and [`MetricSummary`](https://launchdarkly.github.io/ruby-server-sdk-ai/LaunchDarkly/Server/AI/MetricSummary.html).
The [Monitoring tab](/home/ai-configs/monitor) of the AI Config in the LaunchDarkly UI aggregates data from the `track*` functions from across all variations of the AI Config.
Here's how to record metrics manually:
```ruby title="Track duration" theme={null}
# Track your own start and stop time.
# Set duration to the time (in ms) that your AI model generation takes.
# The duration may include network latency, depending on how you calculate it.
ai_config.tracker.track_duration(duration)
```
```ruby title="Track token usage" theme={null}
# Track your own token usage.
# First, create an instance of TokenUsage.
# Update the input, output, and total values
# with return values from your AI model generation.
tokens = LaunchDarkly::AI::TokenUsage.new(total: 300, input: 200, output: 100)
ai_config.tracker.track_tokens(tokens)
```
```ruby title="Track output satisfaction rate" theme={null}
# Track your own output satisfaction rate
# Pass in kind: :positive or kind: :negative
ai_config.tracker.track_feedback(kind: :positive)
```
```ruby title="Track generation (success)" theme={null}
# Track each time there was a successful generation event using your config
ai_config.tracker.track_success
```
```ruby title="Track generation (error)" theme={null}
# Track each time there was an unsuccessful generation event using your config
ai_config.tracker.track_error
```
```ruby title="Track time to first token" theme={null}
# Track the time it takes to generate the first token
# Pass in the time (in ms) until your first token is generated
# This may include network latency, depending on how you calculate it
ai_config.tracker.track_time_to_first_token(1000)
```
Make sure to call `config` again each time you use the tracker and generate content from your AI model.
The SDK automatically flushes these pending analytics events to LaunchDarkly at regular intervals. If you have a short-lived application, you may need to explicitly request that the underlying LaunchDarkly client deliver any pending analytics events to LaunchDarkly, using [`flush`](/sdk/features/flush) or [`close`](/sdk/features/shutdown).
Here's how:
```ruby title="Flush tracking events" theme={null}
ld_client.flush()
```
To learn more, read [`ConfigTracker`](https://launchdarkly.github.io/ruby-server-sdk-ai/LaunchDarkly/Server/AI/AIConfigTracker.html).