Create experiment

For the complete documentation index, see llms.txt. For the full documentation snapshot, see llms-full.txt.

Create an experiment.

To run this experiment, you’ll need to create an iteration and then update the experiment with the startIteration instruction.

Experiment types

Use the type field to specify the experiment type: experiment (default), mab (multi-armed bandit), or holdout.

For multi-armed bandit experiments, set reallocationFrequencyMillis on the iteration to control how often traffic is reallocated across variations.

Results analysis

Use the methodology field to specify the results analysis approach: bayesian (default) or frequentist.

Use the dataSource field to specify the source of metric data: launchdarkly (default), snowflake, or databricks.

Use the analysisConfig field to customize analysis settings such as the Bayesian threshold, significance threshold, or multiple comparison correction method.

To learn more, read Creating experiments.

POST /api/v2/projects/{projectKey}/environments/{environmentKey}/experiments

Create experiment

cURL

curl --request POST \
  --url https://app.launchdarkly.com/api/v2/projects/{projectKey}/environments/{environmentKey}/experiments \
  --header 'Authorization: <api-key>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "Example experiment",
  "key": "experiment-key-123abc",
  "iteration": {
    "hypothesis": "Example hypothesis, the new button placement will increase conversion",
    "metrics": [
      {
        "key": "metric-key-123abc",
        "isGroup": true,
        "primary": true
      }
    ],
    "treatments": [
      {
        "name": "Treatment 1",
        "baseline": true,
        "allocationPercent": "10",
        "parameters": [
          {
            "flagKey": "example-flag-for-experiment",
            "variationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05"
          }
        ]
      }
    ],
    "flags": "{\"example-flag-key\": { \"ruleId\": \"e432f62b-55f6-49dd-a02f-eb24acf39d05\", \"flagConfigVersion\": 12, \"notInExperimentVariationId\": \"e432f62b-55f6-49dd-a02f-eb24acf39d05\" }}",
    "canReshuffleTraffic": true,
    "primarySingleMetricKey": "metric-key-123abc",
    "primaryFunnelKey": "metric-group-key-123abc",
    "randomizationUnit": "user",
    "reallocationFrequencyMillis": 3600000,
    "covariateId": "<string>",
    "attributes": [
      "country",
      "device",
      "os"
    ]
  },
  "description": "An example experiment, used in testing",
  "maintainerId": "12ab3c45de678910fgh12345",
  "holdoutId": "f3b74309-d581-44e1-8a2b-bb2933b4fe40",
  "tags": [
    "<string>"
  ],
  "methodology": "bayesian",
  "analysisConfig": {
    "bayesianThreshold": "10",
    "significanceThreshold": "5",
    "testDirection": "<string>",
    "multipleComparisonCorrectionMethod": "bonferroni",
    "multipleComparisonCorrectionScope": "variations",
    "sequentialTestingEnabled": true
  },
  "dataSource": "snowflake",
  "type": "experiment"
}
'

201

{
  "key": "experiment-key-123abc",
  "name": "Example experiment",
  "_maintainerId": "12ab3c45de678910fgh12345",
  "_creationDate": "1654104600000",
  "environmentKey": "<string>",
  "_links": {
    "parent": {
      "href": "/api/v2/projects/my-project/environments/my-environment",
      "type": "application/json"
    },
    "self": {
      "href": "/api/v2/projects/my-project/environments/my-environment/experiments/my-experiment",
      "type": "application/json"
    }
  },
  "_id": "12ab3c45de678910fgh12345",
  "description": "An example experiment, used in testing",
  "methodology": "bayesian",
  "dataSource": "snowflake",
  "archivedDate": "1654104600000",
  "tags": [
    "experiment",
    "feature"
  ],
  "holdoutId": "f3b74309-d581-44e1-8a2b-bb2933b4fe40",
  "currentIteration": {
    "hypothesis": "The new button placement will increase conversion",
    "status": "running",
    "createdAt": "1654104600000",
    "_id": "12ab3c45de678910fgh12345",
    "startedAt": "1655314200000",
    "endedAt": "1656610200000",
    "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58",
    "winningReason": "We ran this iteration for two weeks and the winning variation was clear",
    "canReshuffleTraffic": true,
    "flags": {},
    "reallocationFrequencyMillis": 3600000,
    "version": 0,
    "primaryMetric": {
      "key": "metric-key-123abc",
      "_versionId": "<string>",
      "name": "My metric",
      "kind": "custom",
      "_links": {
        "self": {
          "href": "/api/v2/metrics/my-project/my-metric",
          "type": "application/json"
        }
      },
      "isGroup": true,
      "isNumeric": true,
      "eventKey": "event-key-123abc",
      "metrics": [
        {
          "key": "metric-key-123abc",
          "name": "Example metric",
          "kind": "custom",
          "_links": {
            "self": {
              "href": "/api/v2/metrics/my-project/my-metric",
              "type": "application/json"
            }
          },
          "_versionId": "version-id-123abc",
          "isNumeric": true,
          "unitAggregationType": "sum",
          "eventKey": "event-key-123abc",
          "nameInGroup": "Step 1",
          "randomizationUnits": [
            "user"
          ]
        }
      ]
    },
    "primarySingleMetric": {
      "key": "metric-key-123abc",
      "name": "Example metric",
      "kind": "custom",
      "_links": {
        "self": {
          "href": "/api/v2/metrics/my-project/my-metric",
          "type": "application/json"
        }
      },
      "_versionId": "version-id-123abc",
      "isNumeric": true,
      "unitAggregationType": "sum",
      "eventKey": "event-key-123abc"
    },
    "primaryFunnel": {
      "key": "metric-group-key-123abc",
      "name": "My metric group",
      "kind": "funnel",
      "_links": {
        "parent": {
          "href": "/api/v2/projects/my-project",
          "type": "application/json"
        },
        "self": {
          "href": "/api/v2/projects/my-project/metric-groups/my-metric-group",
          "type": "application/json"
        }
      },
      "metrics": [
        {
          "key": "metric-key-123abc",
          "name": "Example metric",
          "kind": "custom",
          "_links": {
            "self": {
              "href": "/api/v2/metrics/my-project/my-metric",
              "type": "application/json"
            }
          },
          "_versionId": "version-id-123abc",
          "isNumeric": true,
          "unitAggregationType": "sum",
          "eventKey": "event-key-123abc",
          "nameInGroup": "Step 1",
          "randomizationUnits": [
            "user"
          ]
        }
      ]
    },
    "randomizationUnit": "user",
    "attributes": [
      "<string>"
    ],
    "treatments": [
      {
        "name": "Treatment 1",
        "allocationPercent": "10",
        "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58",
        "baseline": true,
        "parameters": [
          {
            "variationId": "<string>",
            "flagKey": "<string>"
          }
        ]
      }
    ],
    "secondaryMetrics": [
      {
        "key": "metric-key-123abc",
        "name": "Example metric",
        "kind": "custom",
        "_links": {
          "self": {
            "href": "/api/v2/metrics/my-project/my-metric",
            "type": "application/json"
          }
        },
        "_versionId": "version-id-123abc",
        "isNumeric": true,
        "unitAggregationType": "sum",
        "eventKey": "event-key-123abc"
      }
    ],
    "metrics": [
      {
        "key": "metric-key-123abc",
        "_versionId": "<string>",
        "name": "My metric",
        "kind": "custom",
        "_links": {
          "self": {
            "href": "/api/v2/metrics/my-project/my-metric",
            "type": "application/json"
          }
        },
        "isGroup": true,
        "isNumeric": true,
        "eventKey": "event-key-123abc",
        "metrics": [
          {
            "key": "metric-key-123abc",
            "name": "Example metric",
            "kind": "custom",
            "_links": {
              "self": {
                "href": "/api/v2/metrics/my-project/my-metric",
                "type": "application/json"
              }
            },
            "_versionId": "version-id-123abc",
            "isNumeric": true,
            "unitAggregationType": "sum",
            "eventKey": "event-key-123abc",
            "nameInGroup": "Step 1",
            "randomizationUnits": [
              "user"
            ]
          }
        ]
      }
    ],
    "layerSnapshot": {
      "key": "checkout-flow",
      "name": "Checkout Flow",
      "reservationPercent": 10,
      "otherReservationPercent": 70
    },
    "covariateInfo": {
      "id": "74a49a2b-4834-4246-917e-5d85231d8c2a",
      "fileName": "covariate.csv",
      "createdAt": "1654104600000"
    }
  },
  "type": "experiment",
  "draftIteration": {
    "hypothesis": "The new button placement will increase conversion",
    "status": "running",
    "createdAt": "1654104600000",
    "_id": "12ab3c45de678910fgh12345",
    "startedAt": "1655314200000",
    "endedAt": "1656610200000",
    "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58",
    "winningReason": "We ran this iteration for two weeks and the winning variation was clear",
    "canReshuffleTraffic": true,
    "flags": {},
    "reallocationFrequencyMillis": 3600000,
    "version": 0,
    "primaryMetric": {
      "key": "metric-key-123abc",
      "_versionId": "<string>",
      "name": "My metric",
      "kind": "custom",
      "_links": {
        "self": {
          "href": "/api/v2/metrics/my-project/my-metric",
          "type": "application/json"
        }
      },
      "isGroup": true,
      "isNumeric": true,
      "eventKey": "event-key-123abc",
      "metrics": [
        {
          "key": "metric-key-123abc",
          "name": "Example metric",
          "kind": "custom",
          "_links": {
            "self": {
              "href": "/api/v2/metrics/my-project/my-metric",
              "type": "application/json"
            }
          },
          "_versionId": "version-id-123abc",
          "isNumeric": true,
          "unitAggregationType": "sum",
          "eventKey": "event-key-123abc",
          "nameInGroup": "Step 1",
          "randomizationUnits": [
            "user"
          ]
        }
      ]
    },
    "primarySingleMetric": {
      "key": "metric-key-123abc",
      "name": "Example metric",
      "kind": "custom",
      "_links": {
        "self": {
          "href": "/api/v2/metrics/my-project/my-metric",
          "type": "application/json"
        }
      },
      "_versionId": "version-id-123abc",
      "isNumeric": true,
      "unitAggregationType": "sum",
      "eventKey": "event-key-123abc"
    },
    "primaryFunnel": {
      "key": "metric-group-key-123abc",
      "name": "My metric group",
      "kind": "funnel",
      "_links": {
        "parent": {
          "href": "/api/v2/projects/my-project",
          "type": "application/json"
        },
        "self": {
          "href": "/api/v2/projects/my-project/metric-groups/my-metric-group",
          "type": "application/json"
        }
      },
      "metrics": [
        {
          "key": "metric-key-123abc",
          "name": "Example metric",
          "kind": "custom",
          "_links": {
            "self": {
              "href": "/api/v2/metrics/my-project/my-metric",
              "type": "application/json"
            }
          },
          "_versionId": "version-id-123abc",
          "isNumeric": true,
          "unitAggregationType": "sum",
          "eventKey": "event-key-123abc",
          "nameInGroup": "Step 1",
          "randomizationUnits": [
            "user"
          ]
        }
      ]
    },
    "randomizationUnit": "user",
    "attributes": [
      "<string>"
    ],
    "treatments": [
      {
        "name": "Treatment 1",
        "allocationPercent": "10",
        "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58",
        "baseline": true,
        "parameters": [
          {
            "variationId": "<string>",
            "flagKey": "<string>"
          }
        ]
      }
    ],
    "secondaryMetrics": [
      {
        "key": "metric-key-123abc",
        "name": "Example metric",
        "kind": "custom",
        "_links": {
          "self": {
            "href": "/api/v2/metrics/my-project/my-metric",
            "type": "application/json"
          }
        },
        "_versionId": "version-id-123abc",
        "isNumeric": true,
        "unitAggregationType": "sum",
        "eventKey": "event-key-123abc"
      }
    ],
    "metrics": [
      {
        "key": "metric-key-123abc",
        "_versionId": "<string>",
        "name": "My metric",
        "kind": "custom",
        "_links": {
          "self": {
            "href": "/api/v2/metrics/my-project/my-metric",
            "type": "application/json"
          }
        },
        "isGroup": true,
        "isNumeric": true,
        "eventKey": "event-key-123abc",
        "metrics": [
          {
            "key": "metric-key-123abc",
            "name": "Example metric",
            "kind": "custom",
            "_links": {
              "self": {
                "href": "/api/v2/metrics/my-project/my-metric",
                "type": "application/json"
              }
            },
            "_versionId": "version-id-123abc",
            "isNumeric": true,
            "unitAggregationType": "sum",
            "eventKey": "event-key-123abc",
            "nameInGroup": "Step 1",
            "randomizationUnits": [
              "user"
            ]
          }
        ]
      }
    ],
    "layerSnapshot": {
      "key": "checkout-flow",
      "name": "Checkout Flow",
      "reservationPercent": 10,
      "otherReservationPercent": 70
    },
    "covariateInfo": {
      "id": "74a49a2b-4834-4246-917e-5d85231d8c2a",
      "fileName": "covariate.csv",
      "createdAt": "1654104600000"
    }
  },
  "previousIterations": [
    {
      "hypothesis": "The new button placement will increase conversion",
      "status": "running",
      "createdAt": "1654104600000",
      "_id": "12ab3c45de678910fgh12345",
      "startedAt": "1655314200000",
      "endedAt": "1656610200000",
      "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58",
      "winningReason": "We ran this iteration for two weeks and the winning variation was clear",
      "canReshuffleTraffic": true,
      "flags": {},
      "reallocationFrequencyMillis": 3600000,
      "version": 0,
      "primaryMetric": {
        "key": "metric-key-123abc",
        "_versionId": "<string>",
        "name": "My metric",
        "kind": "custom",
        "_links": {
          "self": {
            "href": "/api/v2/metrics/my-project/my-metric",
            "type": "application/json"
          }
        },
        "isGroup": true,
        "isNumeric": true,
        "eventKey": "event-key-123abc",
        "metrics": [
          {
            "key": "metric-key-123abc",
            "name": "Example metric",
            "kind": "custom",
            "_links": {
              "self": {
                "href": "/api/v2/metrics/my-project/my-metric",
                "type": "application/json"
              }
            },
            "_versionId": "version-id-123abc",
            "isNumeric": true,
            "unitAggregationType": "sum",
            "eventKey": "event-key-123abc",
            "nameInGroup": "Step 1",
            "randomizationUnits": [
              "user"
            ]
          }
        ]
      },
      "primarySingleMetric": {
        "key": "metric-key-123abc",
        "name": "Example metric",
        "kind": "custom",
        "_links": {
          "self": {
            "href": "/api/v2/metrics/my-project/my-metric",
            "type": "application/json"
          }
        },
        "_versionId": "version-id-123abc",
        "isNumeric": true,
        "unitAggregationType": "sum",
        "eventKey": "event-key-123abc"
      },
      "primaryFunnel": {
        "key": "metric-group-key-123abc",
        "name": "My metric group",
        "kind": "funnel",
        "_links": {
          "parent": {
            "href": "/api/v2/projects/my-project",
            "type": "application/json"
          },
          "self": {
            "href": "/api/v2/projects/my-project/metric-groups/my-metric-group",
            "type": "application/json"
          }
        },
        "metrics": [
          {
            "key": "metric-key-123abc",
            "name": "Example metric",
            "kind": "custom",
            "_links": {
              "self": {
                "href": "/api/v2/metrics/my-project/my-metric",
                "type": "application/json"
              }
            },
            "_versionId": "version-id-123abc",
            "isNumeric": true,
            "unitAggregationType": "sum",
            "eventKey": "event-key-123abc",
            "nameInGroup": "Step 1",
            "randomizationUnits": [
              "user"
            ]
          }
        ]
      },
      "randomizationUnit": "user",
      "attributes": [
        "<string>"
      ],
      "treatments": [
        {
          "name": "Treatment 1",
          "allocationPercent": "10",
          "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58",
          "baseline": true,
          "parameters": [
            {
              "variationId": "<string>",
              "flagKey": "<string>"
            }
          ]
        }
      ],
      "secondaryMetrics": [
        {
          "key": "metric-key-123abc",
          "name": "Example metric",
          "kind": "custom",
          "_links": {
            "self": {
              "href": "/api/v2/metrics/my-project/my-metric",
              "type": "application/json"
            }
          },
          "_versionId": "version-id-123abc",
          "isNumeric": true,
          "unitAggregationType": "sum",
          "eventKey": "event-key-123abc"
        }
      ],
      "metrics": [
        {
          "key": "metric-key-123abc",
          "_versionId": "<string>",
          "name": "My metric",
          "kind": "custom",
          "_links": {
            "self": {
              "href": "/api/v2/metrics/my-project/my-metric",
              "type": "application/json"
            }
          },
          "isGroup": true,
          "isNumeric": true,
          "eventKey": "event-key-123abc",
          "metrics": [
            {
              "key": "metric-key-123abc",
              "name": "Example metric",
              "kind": "custom",
              "_links": {
                "self": {
                  "href": "/api/v2/metrics/my-project/my-metric",
                  "type": "application/json"
                }
              },
              "_versionId": "version-id-123abc",
              "isNumeric": true,
              "unitAggregationType": "sum",
              "eventKey": "event-key-123abc",
              "nameInGroup": "Step 1",
              "randomizationUnits": [
                "user"
              ]
            }
          ]
        }
      ],
      "layerSnapshot": {
        "key": "checkout-flow",
        "name": "Checkout Flow",
        "reservationPercent": 10,
        "otherReservationPercent": 70
      },
      "covariateInfo": {
        "id": "74a49a2b-4834-4246-917e-5d85231d8c2a",
        "fileName": "covariate.csv",
        "createdAt": "1654104600000"
      }
    }
  ],
  "analysisConfig": {
    "bayesianThreshold": "<string>",
    "significanceThreshold": "<string>",
    "testDirection": "<string>",
    "multipleComparisonCorrectionMethod": "bonferroni",
    "multipleComparisonCorrectionScope": "variations",
    "sequentialTestingEnabled": true
  }
}

Authorizations

Authorization · string · header · required

Path Parameters

projectKey · string<string> · required

The project key

environmentKey · string<string> · required

The environment key

Body

application/json

name · string · required

The experiment name

Example:

"Example experiment"

key · string · required

The experiment key

Example:

"experiment-key-123abc"

iteration · object · required

Details on the construction of the initial iteration

Show child attributes

description · string

The experiment description

Example:

"An example experiment, used in testing"

maintainerId · string

The ID of the member who maintains this experiment

Example:

"12ab3c45de678910fgh12345"

holdoutId · string

The ID of the holdout

Example:

"f3b74309-d581-44e1-8a2b-bb2933b4fe40"

tags · string[]

Tags for the experiment

methodology · enum<string>

The results analysis approach.

Available options: bayesian, frequentist, export_only

Example:

"bayesian"

analysisConfig · object

Details of the Analysis Configuration for this experiment

Show child attributes

dataSource · enum<string>

The source of metric data in order to analyze results. Defaults to "launchdarkly" when not provided.

Available options: launchdarkly, snowflake

Example:

"snowflake"

type · enum<string>

The type of experiment.

Available options: experiment, mab, holdout

Example:

"experiment"

Response

201

application/json

Experiment response

key · string · required

The experiment key

Example:

"experiment-key-123abc"

name · string · required

The experiment name

Example:

"Example experiment"

_maintainerId · string · required

The ID of the member who maintains this experiment.

Example:

"12ab3c45de678910fgh12345"

_creationDate · integer<int64> · required

Timestamp of when the experiment was created

Example:

"1654104600000"

environmentKey · string · required

_links · object · required

The location and content type of related resources

Show child attributes

Example:

{
  "parent": {
    "href": "/api/v2/projects/my-project/environments/my-environment",
    "type": "application/json"
  },
  "self": {
    "href": "/api/v2/projects/my-project/environments/my-environment/experiments/my-experiment",
    "type": "application/json"
  }
}

_id · string

The experiment ID

Example:

"12ab3c45de678910fgh12345"

description · string

The experiment description

Example:

"An example experiment, used in testing"

methodology · enum<string>

The results analysis approach.

Available options: bayesian, frequentist, export_only

Example:

"bayesian"

dataSource · enum<string>

The source of metric data in order to analyze results. Defaults to "launchdarkly" when not provided.

Available options: launchdarkly, snowflake

Example:

"snowflake"

archivedDate · integer<int64>

Timestamp of when the experiment was archived

Example:

"1654104600000"

tags · string[]

Tags for the experiment

Example:

["experiment", "feature"]

holdoutId · string

The holdout ID

Example:

"f3b74309-d581-44e1-8a2b-bb2933b4fe40"

currentIteration · object

Details on the current iteration

Show child attributes

type · string

The experiment type

Example:

"experiment"

draftIteration · object

Details on the current iteration. This iteration may be already started, or may still be a draft.

Show child attributes

previousIterations · object[]

Details on the previous iterations for this experiment.

Show child attributes

analysisConfig · object

Details of the Analysis Configuration for this experiment.

Show child attributes