Patch holdout

For the complete documentation index, see llms.txt. For the full documentation snapshot, see llms-full.txt.

Updates an existing holdout, and returns the updated holdout. Updating holdouts uses the semantic patch format.

To make a semantic patch request, you must append domain-model=launchdarkly.semanticpatch to your Content-Type header. To learn more, read Updates using semantic patch.

Instructions

Semantic patch requests support the following kind instructions for updating holdouts.

Click to expand instructions for updating holdouts

endHoldout

Ends a holdout.

Parameters

None.

Here’s an example:

{
  "comment": "Optional comment describing why the holdout is ending",
  "instructions": [{
    "kind": "endHoldout"
  }]
}

removeExperiment

Removes an experiment from a holdout.

Parameters

• value: The key of the experiment to remove

Here’s an example:

{
  "comment": "Optional comment describing the change",
  "instructions": [{
    "kind": "removeExperiment",
    "value": "experiment-key"
  }]
}

updateDescription

Updates the description of the holdout.

Parameters

• value: The new description.

Here’s an example:

{
  "comment": "Optional comment describing the update",
  "instructions": [{
    "kind": "updateDescription",
    "value": "Updated holdout description"
  }]
}

updateName

Updates the name of the holdout.

Parameters

• value: The new name.

Here’s an example:

{
  "comment": "Optional comment describing the update",
  "instructions": [{
    "kind": "updateName",
    "value": "Updated holdout name"
  }]
}

PATCH /api/v2/projects/{projectKey}/environments/{environmentKey}/holdouts/{holdoutKey}

Patch holdout

cURL

curl --request PATCH \
  --url https://app.launchdarkly.com/api/v2/projects/{projectKey}/environments/{environmentKey}/holdouts/{holdoutKey} \
  --header 'Authorization: <api-key>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "comment": "Optional comment describing the update",
  "instructions": [
    {
      "kind": "updateName",
      "value": "Updated holdout name"
    }
  ]
}
'

200

{
  "_id": "<string>",
  "status": "created",
  "holdoutAmount": "<string>",
  "createdAt": 123,
  "updatedAt": 123,
  "baseExperiment": {
    "key": "experiment-key-123abc",
    "name": "Example experiment",
    "_maintainerId": "12ab3c45de678910fgh12345",
    "_creationDate": "1654104600000",
    "environmentKey": "<string>",
    "_links": {
      "parent": {
        "href": "/api/v2/projects/my-project/environments/my-environment",
        "type": "application/json"
      },
      "self": {
        "href": "/api/v2/projects/my-project/environments/my-environment/experiments/my-experiment",
        "type": "application/json"
      }
    },
    "_id": "12ab3c45de678910fgh12345",
    "description": "An example experiment, used in testing",
    "methodology": "bayesian",
    "dataSource": "snowflake",
    "archivedDate": "1654104600000",
    "tags": [
      "experiment",
      "feature"
    ],
    "holdoutId": "f3b74309-d581-44e1-8a2b-bb2933b4fe40",
    "currentIteration": {
      "hypothesis": "The new button placement will increase conversion",
      "status": "running",
      "createdAt": "1654104600000",
      "_id": "12ab3c45de678910fgh12345",
      "startedAt": "1655314200000",
      "endedAt": "1656610200000",
      "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58",
      "winningReason": "We ran this iteration for two weeks and the winning variation was clear",
      "canReshuffleTraffic": true,
      "flags": {},
      "reallocationFrequencyMillis": 3600000,
      "version": 0,
      "primaryMetric": {
        "key": "metric-key-123abc",
        "_versionId": "<string>",
        "name": "My metric",
        "kind": "custom",
        "_links": {
          "self": {
            "href": "/api/v2/metrics/my-project/my-metric",
            "type": "application/json"
          }
        },
        "isGroup": true,
        "isNumeric": true,
        "eventKey": "event-key-123abc",
        "metrics": [
          {
            "key": "metric-key-123abc",
            "name": "Example metric",
            "kind": "custom",
            "_links": {
              "self": {
                "href": "/api/v2/metrics/my-project/my-metric",
                "type": "application/json"
              }
            },
            "_versionId": "version-id-123abc",
            "isNumeric": true,
            "unitAggregationType": "sum",
            "eventKey": "event-key-123abc",
            "nameInGroup": "Step 1",
            "randomizationUnits": [
              "user"
            ]
          }
        ]
      },
      "primarySingleMetric": {
        "key": "metric-key-123abc",
        "name": "Example metric",
        "kind": "custom",
        "_links": {
          "self": {
            "href": "/api/v2/metrics/my-project/my-metric",
            "type": "application/json"
          }
        },
        "_versionId": "version-id-123abc",
        "isNumeric": true,
        "unitAggregationType": "sum",
        "eventKey": "event-key-123abc"
      },
      "primaryFunnel": {
        "key": "metric-group-key-123abc",
        "name": "My metric group",
        "kind": "funnel",
        "_links": {
          "parent": {
            "href": "/api/v2/projects/my-project",
            "type": "application/json"
          },
          "self": {
            "href": "/api/v2/projects/my-project/metric-groups/my-metric-group",
            "type": "application/json"
          }
        },
        "metrics": [
          {
            "key": "metric-key-123abc",
            "name": "Example metric",
            "kind": "custom",
            "_links": {
              "self": {
                "href": "/api/v2/metrics/my-project/my-metric",
                "type": "application/json"
              }
            },
            "_versionId": "version-id-123abc",
            "isNumeric": true,
            "unitAggregationType": "sum",
            "eventKey": "event-key-123abc",
            "nameInGroup": "Step 1",
            "randomizationUnits": [
              "user"
            ]
          }
        ]
      },
      "randomizationUnit": "user",
      "attributes": [
        "<string>"
      ],
      "treatments": [
        {
          "name": "Treatment 1",
          "allocationPercent": "10",
          "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58",
          "baseline": true,
          "parameters": [
            {
              "variationId": "<string>",
              "flagKey": "<string>"
            }
          ]
        }
      ],
      "secondaryMetrics": [
        {
          "key": "metric-key-123abc",
          "name": "Example metric",
          "kind": "custom",
          "_links": {
            "self": {
              "href": "/api/v2/metrics/my-project/my-metric",
              "type": "application/json"
            }
          },
          "_versionId": "version-id-123abc",
          "isNumeric": true,
          "unitAggregationType": "sum",
          "eventKey": "event-key-123abc"
        }
      ],
      "metrics": [
        {
          "key": "metric-key-123abc",
          "_versionId": "<string>",
          "name": "My metric",
          "kind": "custom",
          "_links": {
            "self": {
              "href": "/api/v2/metrics/my-project/my-metric",
              "type": "application/json"
            }
          },
          "isGroup": true,
          "isNumeric": true,
          "eventKey": "event-key-123abc",
          "metrics": [
            {
              "key": "metric-key-123abc",
              "name": "Example metric",
              "kind": "custom",
              "_links": {
                "self": {
                  "href": "/api/v2/metrics/my-project/my-metric",
                  "type": "application/json"
                }
              },
              "_versionId": "version-id-123abc",
              "isNumeric": true,
              "unitAggregationType": "sum",
              "eventKey": "event-key-123abc",
              "nameInGroup": "Step 1",
              "randomizationUnits": [
                "user"
              ]
            }
          ]
        }
      ],
      "layerSnapshot": {
        "key": "checkout-flow",
        "name": "Checkout Flow",
        "reservationPercent": 10,
        "otherReservationPercent": 70
      },
      "covariateInfo": {
        "id": "74a49a2b-4834-4246-917e-5d85231d8c2a",
        "fileName": "covariate.csv",
        "createdAt": "1654104600000"
      }
    },
    "type": "experiment",
    "draftIteration": {
      "hypothesis": "The new button placement will increase conversion",
      "status": "running",
      "createdAt": "1654104600000",
      "_id": "12ab3c45de678910fgh12345",
      "startedAt": "1655314200000",
      "endedAt": "1656610200000",
      "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58",
      "winningReason": "We ran this iteration for two weeks and the winning variation was clear",
      "canReshuffleTraffic": true,
      "flags": {},
      "reallocationFrequencyMillis": 3600000,
      "version": 0,
      "primaryMetric": {
        "key": "metric-key-123abc",
        "_versionId": "<string>",
        "name": "My metric",
        "kind": "custom",
        "_links": {
          "self": {
            "href": "/api/v2/metrics/my-project/my-metric",
            "type": "application/json"
          }
        },
        "isGroup": true,
        "isNumeric": true,
        "eventKey": "event-key-123abc",
        "metrics": [
          {
            "key": "metric-key-123abc",
            "name": "Example metric",
            "kind": "custom",
            "_links": {
              "self": {
                "href": "/api/v2/metrics/my-project/my-metric",
                "type": "application/json"
              }
            },
            "_versionId": "version-id-123abc",
            "isNumeric": true,
            "unitAggregationType": "sum",
            "eventKey": "event-key-123abc",
            "nameInGroup": "Step 1",
            "randomizationUnits": [
              "user"
            ]
          }
        ]
      },
      "primarySingleMetric": {
        "key": "metric-key-123abc",
        "name": "Example metric",
        "kind": "custom",
        "_links": {
          "self": {
            "href": "/api/v2/metrics/my-project/my-metric",
            "type": "application/json"
          }
        },
        "_versionId": "version-id-123abc",
        "isNumeric": true,
        "unitAggregationType": "sum",
        "eventKey": "event-key-123abc"
      },
      "primaryFunnel": {
        "key": "metric-group-key-123abc",
        "name": "My metric group",
        "kind": "funnel",
        "_links": {
          "parent": {
            "href": "/api/v2/projects/my-project",
            "type": "application/json"
          },
          "self": {
            "href": "/api/v2/projects/my-project/metric-groups/my-metric-group",
            "type": "application/json"
          }
        },
        "metrics": [
          {
            "key": "metric-key-123abc",
            "name": "Example metric",
            "kind": "custom",
            "_links": {
              "self": {
                "href": "/api/v2/metrics/my-project/my-metric",
                "type": "application/json"
              }
            },
            "_versionId": "version-id-123abc",
            "isNumeric": true,
            "unitAggregationType": "sum",
            "eventKey": "event-key-123abc",
            "nameInGroup": "Step 1",
            "randomizationUnits": [
              "user"
            ]
          }
        ]
      },
      "randomizationUnit": "user",
      "attributes": [
        "<string>"
      ],
      "treatments": [
        {
          "name": "Treatment 1",
          "allocationPercent": "10",
          "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58",
          "baseline": true,
          "parameters": [
            {
              "variationId": "<string>",
              "flagKey": "<string>"
            }
          ]
        }
      ],
      "secondaryMetrics": [
        {
          "key": "metric-key-123abc",
          "name": "Example metric",
          "kind": "custom",
          "_links": {
            "self": {
              "href": "/api/v2/metrics/my-project/my-metric",
              "type": "application/json"
            }
          },
          "_versionId": "version-id-123abc",
          "isNumeric": true,
          "unitAggregationType": "sum",
          "eventKey": "event-key-123abc"
        }
      ],
      "metrics": [
        {
          "key": "metric-key-123abc",
          "_versionId": "<string>",
          "name": "My metric",
          "kind": "custom",
          "_links": {
            "self": {
              "href": "/api/v2/metrics/my-project/my-metric",
              "type": "application/json"
            }
          },
          "isGroup": true,
          "isNumeric": true,
          "eventKey": "event-key-123abc",
          "metrics": [
            {
              "key": "metric-key-123abc",
              "name": "Example metric",
              "kind": "custom",
              "_links": {
                "self": {
                  "href": "/api/v2/metrics/my-project/my-metric",
                  "type": "application/json"
                }
              },
              "_versionId": "version-id-123abc",
              "isNumeric": true,
              "unitAggregationType": "sum",
              "eventKey": "event-key-123abc",
              "nameInGroup": "Step 1",
              "randomizationUnits": [
                "user"
              ]
            }
          ]
        }
      ],
      "layerSnapshot": {
        "key": "checkout-flow",
        "name": "Checkout Flow",
        "reservationPercent": 10,
        "otherReservationPercent": 70
      },
      "covariateInfo": {
        "id": "74a49a2b-4834-4246-917e-5d85231d8c2a",
        "fileName": "covariate.csv",
        "createdAt": "1654104600000"
      }
    },
    "previousIterations": [
      {
        "hypothesis": "The new button placement will increase conversion",
        "status": "running",
        "createdAt": "1654104600000",
        "_id": "12ab3c45de678910fgh12345",
        "startedAt": "1655314200000",
        "endedAt": "1656610200000",
        "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58",
        "winningReason": "We ran this iteration for two weeks and the winning variation was clear",
        "canReshuffleTraffic": true,
        "flags": {},
        "reallocationFrequencyMillis": 3600000,
        "version": 0,
        "primaryMetric": {
          "key": "metric-key-123abc",
          "_versionId": "<string>",
          "name": "My metric",
          "kind": "custom",
          "_links": {
            "self": {
              "href": "/api/v2/metrics/my-project/my-metric",
              "type": "application/json"
            }
          },
          "isGroup": true,
          "isNumeric": true,
          "eventKey": "event-key-123abc",
          "metrics": [
            {
              "key": "metric-key-123abc",
              "name": "Example metric",
              "kind": "custom",
              "_links": {
                "self": {
                  "href": "/api/v2/metrics/my-project/my-metric",
                  "type": "application/json"
                }
              },
              "_versionId": "version-id-123abc",
              "isNumeric": true,
              "unitAggregationType": "sum",
              "eventKey": "event-key-123abc",
              "nameInGroup": "Step 1",
              "randomizationUnits": [
                "user"
              ]
            }
          ]
        },
        "primarySingleMetric": {
          "key": "metric-key-123abc",
          "name": "Example metric",
          "kind": "custom",
          "_links": {
            "self": {
              "href": "/api/v2/metrics/my-project/my-metric",
              "type": "application/json"
            }
          },
          "_versionId": "version-id-123abc",
          "isNumeric": true,
          "unitAggregationType": "sum",
          "eventKey": "event-key-123abc"
        },
        "primaryFunnel": {
          "key": "metric-group-key-123abc",
          "name": "My metric group",
          "kind": "funnel",
          "_links": {
            "parent": {
              "href": "/api/v2/projects/my-project",
              "type": "application/json"
            },
            "self": {
              "href": "/api/v2/projects/my-project/metric-groups/my-metric-group",
              "type": "application/json"
            }
          },
          "metrics": [
            {
              "key": "metric-key-123abc",
              "name": "Example metric",
              "kind": "custom",
              "_links": {
                "self": {
                  "href": "/api/v2/metrics/my-project/my-metric",
                  "type": "application/json"
                }
              },
              "_versionId": "version-id-123abc",
              "isNumeric": true,
              "unitAggregationType": "sum",
              "eventKey": "event-key-123abc",
              "nameInGroup": "Step 1",
              "randomizationUnits": [
                "user"
              ]
            }
          ]
        },
        "randomizationUnit": "user",
        "attributes": [
          "<string>"
        ],
        "treatments": [
          {
            "name": "Treatment 1",
            "allocationPercent": "10",
            "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58",
            "baseline": true,
            "parameters": [
              {
                "variationId": "<string>",
                "flagKey": "<string>"
              }
            ]
          }
        ],
        "secondaryMetrics": [
          {
            "key": "metric-key-123abc",
            "name": "Example metric",
            "kind": "custom",
            "_links": {
              "self": {
                "href": "/api/v2/metrics/my-project/my-metric",
                "type": "application/json"
              }
            },
            "_versionId": "version-id-123abc",
            "isNumeric": true,
            "unitAggregationType": "sum",
            "eventKey": "event-key-123abc"
          }
        ],
        "metrics": [
          {
            "key": "metric-key-123abc",
            "_versionId": "<string>",
            "name": "My metric",
            "kind": "custom",
            "_links": {
              "self": {
                "href": "/api/v2/metrics/my-project/my-metric",
                "type": "application/json"
              }
            },
            "isGroup": true,
            "isNumeric": true,
            "eventKey": "event-key-123abc",
            "metrics": [
              {
                "key": "metric-key-123abc",
                "name": "Example metric",
                "kind": "custom",
                "_links": {
                  "self": {
                    "href": "/api/v2/metrics/my-project/my-metric",
                    "type": "application/json"
                  }
                },
                "_versionId": "version-id-123abc",
                "isNumeric": true,
                "unitAggregationType": "sum",
                "eventKey": "event-key-123abc",
                "nameInGroup": "Step 1",
                "randomizationUnits": [
                  "user"
                ]
              }
            ]
          }
        ],
        "layerSnapshot": {
          "key": "checkout-flow",
          "name": "Checkout Flow",
          "reservationPercent": 10,
          "otherReservationPercent": 70
        },
        "covariateInfo": {
          "id": "74a49a2b-4834-4246-917e-5d85231d8c2a",
          "fileName": "covariate.csv",
          "createdAt": "1654104600000"
        }
      }
    ],
    "analysisConfig": {
      "bayesianThreshold": "<string>",
      "significanceThreshold": "<string>",
      "testDirection": "<string>",
      "multipleComparisonCorrectionMethod": "bonferroni",
      "multipleComparisonCorrectionScope": "variations",
      "sequentialTestingEnabled": true
    }
  },
  "description": "<string>",
  "experiments": [
    {
      "key": "<string>",
      "name": "<string>",
      "environment": "<string>"
    }
  ]
}

Authorizations

Authorization · string · header · required

Path Parameters

projectKey · string<string> · required

The project key

environmentKey · string<string> · required

The environment key

holdoutKey · string<string> · required

The holdout key

Body

application/json

instructions · object[] · required

The instructions to perform when updating. This should be an array with objects that look like {"kind": "update_action"}. Some instructions also require a value field in the array element.

Show child attributes

Example:

"[{\"kind\": \"updateName\", \"name\": \"Updated holdout name\"}]"

comment · string

Optional comment describing the update

Example:

"Optional comment"

Response

200

application/json

Holdout response

_id · string · required

status · enum<string> · required

Available options: created, enabled, running, ended

holdoutAmount · string · required

The percentage of traffic allocated to this holdout.

createdAt · integer<int64> · required

updatedAt · integer<int64> · required

baseExperiment · object · required

Show child attributes

description · string

experiments · object[]

Show child attributes