Skip to main content

Documentation Index

Fetch the complete documentation index at: https://launchdarkly-preview.mintlify.app/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

This guide shows you how to build a simple AI-powered chatbot using LaunchDarkly AI Configs with multiple AI providers, including Anthropic, OpenAI, and Google. Using AI Configs, you can manage models and prompts outside of code, switch providers without redeploying, and monitor performance in real time. You’ll learn how to:
  • Create a basic chatbot application
  • Configure AI models dynamically without code changes
  • Create and manage multiple AI Config variations
  • Apply user contexts for personalizing AI behavior
  • Switch between different AI providers seamlessly
  • Monitor and track AI performance metrics
By the end of this tutorial, you’ll have a working chatbot that demonstrates LaunchDarkly’s AI Config capabilities across multiple providers. The complete code for this tutorial is available in the simple-chatbot repository. For additional code examples and implementations, check out the LaunchDarkly Python AI Examples repository, which includes practical examples of AI Configs with various providers and use cases.

​
Prerequisites

Before you begin, you need the following:

​
Required accounts

Access to the following accounts:

​
Development environment requirements

A development environment with:
  • Python 3.8 or later
  • pip package manager
  • Basic Python knowledge
  • A code editor, such as VS Code or PyCharm

​
API and SDK keys

The following keys:
  • Your LaunchDarkly SDK key
  • An API key from at least one AI provider

​
Before you start

This guide builds a chatbot using completion-based AI Configs in a messages array format. If you use LangGraph or CrewAI, you may want to use agent mode instead. The following sections include best practices to help you avoid common issues and reduce debugging time.

​
Do not cache configs across users

Reusing configs across users breaks targeting. Instead, fetch a fresh config for each request: 
    # This breaks targeting - all users get the first user's config
    config = ai_client.config("my-key", first_user_context, fallback)
    for user in users:
        response = generate(config, user.message)  # Wrong!

    # Fresh config per request
    for user in users:
        config = ai_client.config("my-key", user.context, fallback)
        response = generate(config, user.message)

​
Provide a fallback config

Provide a fallback so your application does not crash when unexpected issues occur, such as LaunchDarkly being unavailable or API keys being incorrect: 
    fallback = AIConfig(
        enabled=True,
        model=ModelConfig(name="claude-3-haiku-20240307"),
        messages=[LDMessage(role="system", content="You are helpful")]
    )

    config = ai_client.config("my-key", context, fallback)

​
Check if the config is enabled

Check if the config is enabled before using it: 
    if config.enabled:
        response = call_ai_provider(config)
    else:
        response = "AI is temporarily unavailable"

​
Do not include personally identifiable information (PII) in contexts

Never send PII to LaunchDarkly. Here’s a bad and a good example: 
    # Bad - do not include PII
    context = Context.builder(user.email)

    # Good - opaque ID
    context = Context.builder(user.id)  # "usr_abc123"
        .set("tier", "premium")  # Non-PII attributes are fine

​
Limit conversation history

Your chat history grows with every turn. After 50 exchanges, each request may include thousands of tokens. Here’s how to limit it: 
    MAX_TURNS = 20

    def add_to_history(history, role, content):
        history.append({"role": role, "content": content})
        if len(history) > MAX_TURNS:
            history = [history[0]] + history[-(MAX_TURNS-1):]  # Keep system prompt
        return history

​
Track token usage

Without tracking, it is difficult to understand how token usage affects cost. Here’s how to track token usage: 
    import time
    from ldai.tracker import TokenUsage

    # Track duration
    start = time.time()
    response = client.generate(messages)  # Get full response object
    tracker.track_duration(time.time() - start)

    # Track tokens
    if hasattr(response, 'usage'):
        tracker.track_tokens(TokenUsage(
            input=response.usage.input_tokens,
            output=response.usage.output_tokens,
            total=response.usage.input_tokens + response.usage.output_tokens
        ))

    # Track success/error
    tracker.track_success()  # or tracker.track_error("error message")

    # Extract text for display
    text = response.content[0].text  # Anthropic
    # or response.choices[0].message.content  # OpenAI
    # or response.text  # Google
Your provider methods should return the full response object, not just text, so you can access usage metadata. The code examples here return full responses where tracking is needed.

​
Tutorial steps

Continue with the task-focused sections below: