> This page is for version v1.3 (default). > For other versions, use one of these documentation indexes: > - v1.3 (default): https://beta.docs.twelvelabs.io/v1.3/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://beta.docs.twelvelabs.io/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://beta.docs.twelvelabs.io/_mcp/server. The `AnalyzeAsync.Tasks` interface provides methods to analyze videos or images asynchronously, generate text, and extract structured, timestamped segments. Each request must contain a video or one or more images, but not both. The platform supports two analysis modes: * **General analysis**: Prompt-based text generation with a video or one or more images. * **Video segmentation**: Timestamped metadata with custom segment definitions. **When to use this interface**: * Generate custom text from your video or one or more images using a prompt (general analysis) * Extract timestamped metadata with custom segment definitions from your video * Analyze videos longer than 1 hour, or a portion of a video up to 4 hours long * Process videos or images asynchronously without blocking your application **Do not use this interface for**: * Videos or images for which you need immediate results or real-time streaming. Use the [`analyze`](/v1.3/sdk-reference/node-js/analyze-videos/sync-analysis#sync-analysis) method instead. #### Input requirements **Videos** * The video can be up to 2 hours long, or up to 4 hours when you analyze only a portion of it. You can analyze between 1 second and 2 hours of the video. HLS and base64 videos are limited to 2 hours. * Formats: [FFmpeg supported formats](https://ffmpeg.org/ffmpeg-formats.html) * Resolution: 360x360 to 5184x2160 pixels * Aspect ratio: Between 1:1 and 1:2.4, or between 2.4:1 and 1:1 **Images** * You can provide one to twenty images per request. * Formats: JPEG, PNG, WebP, GIF, and BMP. * Maximum size: 20 MB per image. * Maximum pixel count: 16,777,216 pixels per image (width × height). In legacy billing, on the Free plan, analysis hours count toward a shared limit that also covers indexing - the number of segment definitions does not affect this limit. On paid plans, you pay based on how much video you process and how many segment definitions you include - see the [Frequently asked questions](/v1.3/docs/resources/frequently-asked-questions#how-is-video-segmentation-priced) page for examples. In the new billing, the allowances of your plan apply. See [Plans, allowances, and overage](/v1.3/docs/get-started/plans-and-billing/plans-allowances-and-overage). Analyzing content asynchronously requires three steps: 1. Create an analysis task using the [`create`](#create-an-async-analysis-task) method. The platform returns a task identifier. 2. Poll the status of the task using the [`retrieve`](#retrieve-task-status-and-results) method. Wait until the status is `ready`, `failed`, or `canceled`. 3. When the status is `ready`, retrieve the results using the [`retrieve`](#retrieve-task-status-and-results) method. # Methods ## List analysis tasks **Description**: This method returns a list of the analysis tasks in your account. The platform returns your analysis tasks sorted by creation date, with the newest at the top of the list. **Function signature and example**: **`Function signature`** ```javascript Function signature list( request?: TwelvelabsApi.analyzeAsync.TasksListRequest, requestOptions?: Tasks.RequestOptions ): core.HttpResponsePromise ``` **`Node.js example`** ```javascript Node.js example import { TwelvelabsApiClient } from "twelvelabs-js"; const response = await client.analyzeAsync.tasks.list({ page: 1, pageLimit: 10, }); console.log(`Total tasks: ${response.pageInfo?.totalResults}`); for (const task of response.data) { console.log(` Task ID: ${task.taskId}`); console.log(` Status: ${task.status}`); console.log(` Created: ${task.createdAt}`); } ``` **Parameters**: | Name | Type | Required | Description | | ---------------- | --------------------------------------------- | -------- | -------------------------------------- | | `request` | `TwelvelabsApi.analyzeAsync.TasksListRequest` | No | Parameters for listing analysis tasks. | | `requestOptions` | `Tasks.RequestOptions` | No | Request-specific configuration. | The `TwelvelabsApi.analyzeAsync.TasksListRequest` interface has the following properties: | Name | Type | Required | Description | | -------------- | --------------------------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------ | | `page` | `number` | No | A number that identifies the page to retrieve. Default: `1`. | | `pageLimit` | `number` | No | The number of items to return on each page. Default: `10`. Max: `50`. | | `status` | `TwelvelabsApi.AnalyzeTaskStatus` | No | Filter analysis tasks by status. Values: `queued`, `pending`, `processing`, `ready`, `failed`, `canceled`. | | `videoUrl` | `string` | No | Filter tasks by exact video source URL. | | `assetId` | `string` | No | Filter tasks by asset ID. | | `analysisMode` | `TwelvelabsApi.analyzeAsync.TasksListRequestAnalysisMode` | No | Filter tasks by the analysis mode used when creating the task. Values: `"general"`, `"time_based_metadata"`. | **Return value**: Returns an `HttpResponsePromise` that resolves to a `TwelvelabsApi.analyzeAsync.TasksListResponse` object. The `TwelvelabsApi.analyzeAsync.TasksListResponse` interface contains the following properties: | Name | Type | Description | | ---------- | ------------------------------------- | -------------------------------------------------------- | | `data` | `TwelvelabsApi.AnalyzeTaskResponse[]` | An array that contains up to `pageLimit` analysis tasks. | | `pageInfo` | `TwelvelabsApi.PageInfo` | An object that provides information about pagination. | The `TwelvelabsApi.PageInfo` interface contains the following properties: | Name | Type | Description | | -------------- | -------- | --------------------------------------------------- | | `limitPerPage` | `number` | The number of items returned per page. | | `page` | `number` | The current page number. | | `totalPage` | `number` | The total number of pages. | | `totalResults` | `number` | The total number of analysis tasks in your account. | For details about `AnalyzeTaskResponse`, see [Retrieve task status and results](#retrieve-task-status-and-results). **API Reference**: [List async analysis tasks](/v1.3/api-reference/analyze-videos/list-async-analysis-tasks) ## Create an async analysis task **Description**: This method analyzes a video or one or more images asynchronously. Each request must contain a video or one or more images, but not both. The method supports two analysis modes: * **General analysis**: Prompt-based text generation with a video or one or more images. * **Video segmentation**: Timestamped metadata with custom segment definitions. On the Free plan in legacy billing, you have a total of 600 minutes (10 hours) shared across indexing, analysis, and segmentation. For details, see the [Video hours and video count limits](/v1.3/docs/concepts/indexes#video-hours-and-video-count-limits) section. In the new billing, the allowances of your plan apply in place of this limit. See [Plans, allowances, and overage](/v1.3/docs/get-started/plans-and-billing/plans-allowances-and-overage). > **Note** > > This method is rate-limited. For details, see the [Rate limits](/v1.3/docs/get-started/rate-limits) page. **Function signature and example**: **`Function signature`** ```javascript Function signature create( request: TwelvelabsApi.analyzeAsync.CreateAsyncAnalyzeRequest, requestOptions?: Tasks.RequestOptions ): core.HttpResponsePromise ``` **`General analysis example`** ```javascript General analysis example import { TwelvelabsApiClient } from "twelvelabs-js"; const task = await client.analyzeAsync.tasks.create({ video: { type: "url", url: "", }, promptV2: { inputText: "", }, temperature: 0.2, }); console.log(`Task ID: ${task.taskId}`); console.log(`Status: ${task.status}`); ``` **`Image input example`** ```javascript Image input example import { TwelvelabsApiClient } from "twelvelabs-js"; const task = await client.analyzeAsync.tasks.create({ modelName: "pegasus1.6", image: [ { name: "product_shot", url: "", }, ], prompt: "Describe the product in <@product_shot> and list every visible feature.", }); console.log(`Task ID: ${task.taskId}`); console.log(`Status: ${task.status}`); ``` **`Segmentation example`** ```javascript Segmentation example import { TwelvelabsApiClient } from "twelvelabs-js"; const task = await client.analyzeAsync.tasks.create({ modelName: "pegasus1.5", video: { type: "url", url: "", }, analysisMode: "time_based_metadata", responseFormat: { type: "segment_definitions", segmentDefinitions: [ { id: "scene", description: "A distinct scene or setting change in the video", fields: [ { name: "sentiment", type: "string", description: "The emotional tone of this segment", enum: ["positive", "negative", "neutral"], }, { name: "key_objects", type: "array", description: "Notable objects visible in this segment", items: { type: "string" }, }, ], }, ], }, minSegmentDuration: 5.0, maxSegmentDuration: 30.0, }); console.log(`Task ID: ${task.taskId}`); console.log(`Status: ${task.status}`); ``` **`Time array example`** ```javascript Time array example import { TwelvelabsApiClient } from "twelvelabs-js"; const task = await client.analyzeAsync.tasks.create({ modelName: "pegasus1.6", video: { type: "url", url: "", }, analysisMode: "time_based_metadata", responseFormat: { type: "segment_definitions", segmentTimeFormat: "hh:mm:ss", segmentDefinitions: [ { id: "scenes", description: "A distinct scene or setting change in the video", fields: [ { name: "label", type: "string", description: "A short label for this scene.", }, { name: "dialogue", type: "time_array", description: "Each spoken line in this scene.", items: { type: "object", fields: [ { name: "speaker", type: "string", description: "The name of the person speaking.", }, { name: "line", type: "string", description: "What the speaker says.", }, ], }, }, ], }, ], }, maxTokens: 32768, }); console.log(`Task ID: ${task.taskId}`); console.log(`Status: ${task.status}`); ``` **`Segmentation without a token limit example`** ```javascript Segmentation without a token limit example import { TwelvelabsApiClient } from "twelvelabs-js"; const task = await client.analyzeAsync.tasks.create({ modelName: "pegasus1.6", video: { type: "url", url: "", }, analysisMode: "time_based_metadata", maxTokens: "unlimited", responseFormat: { type: "segment_definitions", segmentDefinitions: [ { id: "scenes", description: "A distinct scene or setting change in the video", fields: [ { name: "label", type: "string", description: "A short label for this scene.", }, ], }, ], }, }); console.log(`Task ID: ${task.taskId}`); console.log(`Status: ${task.status}`); ``` **Parameters**: | Name | Type | Required | Description | | ---------------- | ------------------------------------------------------ | -------- | ----------------------------------------------- | | `request` | `TwelvelabsApi.analyzeAsync.CreateAsyncAnalyzeRequest` | Yes | Parameters for creating an async analysis task. | | `requestOptions` | `Tasks.RequestOptions` | No | Request-specific configuration. | The `TwelvelabsApi.analyzeAsync.CreateAsyncAnalyzeRequest` interface has the following properties: | Name | Type | Required | Description | | -------------------- | ------------------------------------------------------------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `modelName` | `TwelvelabsApi.analyzeAsync.CreateAsyncAnalyzeRequestModelName` | No | The video understanding model to use for analysis. Values: - `"pegasus1.6"`: For details about this version, see the [Pegasus 1.6](/v1.3/docs/concepts/models/pegasus/pegasus-1-6) page. - `"pegasus1.5"`: For details about this version, see the [Pegasus 1.5](/v1.3/docs/concepts/models/pegasus/pegasus-1-5) page. **Default:** `"pegasus1.5"` | | `customId` | `string` | No | An optional identifier that you set when you create the task. Use this field to correlate tasks across responses, for example, to distinguish tasks by type or environment. Must match the pattern `^[a-zA-Z0-9_-]{1,64}$`. | | `video` | `TwelvelabsApi.VideoContext` | No | An object that specifies the source of the video. Mutually exclusive with the `image` parameter. See [VideoContext](#videocontext) for details. | | `image` | `TwelvelabsApi.AnalyzeImageInput[]` | No | A list of up to twenty objects containing the images to analyze. For each image, include exactly one source. Requires Pegasus 1.6. Using any other model returns a `parameter_invalid` error. Mutually exclusive with the `video` and `promptV2` parameters. The `prompt` parameter is required when you provide images. See [AnalyzeImageInput](#analyzeimageinput) for details. | | `prompt` | `string` | No | Natural-language instructions for analyzing the video or one or more images. Required for general analysis (prompt-based text generation). Not supported when `analysisMode` is `"time_based_metadata"`. To include reference images in your prompt, use the `promptV2` parameter instead. Mutually exclusive with the `promptV2` parameter. | | `promptV2` | `TwelvelabsApi.AnalyzePromptV2` | No | A structured prompt that uses the `<@name>` placeholders to reference images. Not supported when the `analysisMode` parameter is `"time_based_metadata"`. Mutually exclusive with the `prompt` and `image` parameters. See [AnalyzePromptV2](#analyzepromptv2). | | `analysisMode` | `TwelvelabsApi.analyzeAsync.CreateAsyncAnalyzeRequestAnalysisMode` | No | The analysis mode for this task. Values: - `"general"`: Analyze the video and generate a response based on your prompt. Supports both free-form text and structured output via `responseFormat`. - `"time_based_metadata"`: Segment the video into time-based intervals and extract custom metadata for each segment. Requires `responseFormat.type` set to `"segment_definitions"`. **Default:** `"general"` | | `temperature` | `number` | No | Controls the randomness of the text output. **Default:** 0.2, **Min:** 0, **Max:** 1 | | `maxTokens` | `TwelvelabsApi.analyzeAsync.CreateAsyncAnalyzeRequestMaxTokens` | No | The maximum response length, in tokens. Provide an integer or the `"unlimited"` value. The allowed integer range and default depend on the analysis mode: `"general"` (Min: `512`, Max: `98,304`, Default: `4,096`); `"time_based_metadata"` (Min: `2,048`, Max: `98,304`, Default: `32,768`). The input and response must fit within the [context window](/v1.3/docs/concepts/models/pegasus/pegasus-1-6#context-window). - **Integer**: With video segmentation, the task fails if the output exceeds the limit. No partial output is returned. - **Unlimited**: Removes the token limit from video segmentation. It requires `analysisMode` set to `"time_based_metadata"` and `modelName` set to `"pegasus1.6"`. Mutually exclusive with `responseFormat.segmentDefinitions[].timeRanges`. The platform extracts as many segments as it can. If it stops before extracting every segment, the task still completes with the `status` field set to `"ready"`. The `error` field contains the warning that the results may be incomplete. | | `responseFormat` | `TwelvelabsApi.AsyncResponseFormat` | No | Controls the response format. When you omit this parameter, you receive unstructured text. | | `minSegmentDuration` | `number` | No | Minimum duration for each extracted segment, in seconds. Prevents the model from creating very short segments. Requires `analysisMode` set to `"time_based_metadata"`. Min: `2`. | | `maxSegmentDuration` | `number` | No | Maximum duration for each extracted segment, in seconds. Breaks long continuous sections into shorter segments. Must be greater than or equal to the `minSegmentDuration` parameter. Requires `analysisMode` set to `"time_based_metadata"`. Min: `2`. | | `startTime` | `number` | No | Start of the analysis window, as an absolute timestamp in seconds, based on the internal metadata of the video. Use with `endTime` to analyze only a portion of the video. If omitted, defaults to the internal start time of the video. Most videos start at 0, but some (for example, from cameras or broadcast recordings) may have a non-zero start time. To find the value, run `ffprobe -v error -show_entries format=start_time,duration -of default=noprint_wrappers=1 your_video.mp4`. Must be less than `endTime` and the video duration. The window (`endTime - startTime`) must be at least 1 second and at most 2 hours. The video may be up to 4 hours as long as the window stays within that limit. Mutually exclusive with `responseFormat.segmentDefinitions[].timeRanges`. Together with `endTime`, this parameter determines the billable video duration. If you omit both, billing uses the full video duration. For details, see the [Frequently asked questions](/v1.3/docs/resources/frequently-asked-questions#how-is-video-segmentation-priced) page. | | `endTime` | `number` | No | End of the analysis window, as an absolute timestamp in seconds, based on the internal metadata of the video. Use with `startTime` to analyze only a portion of the video. If omitted, defaults to the internal start time of the video plus its duration. Most videos start at 0, but some (for example, from cameras or broadcast recordings) may have a non-zero start time. To find the value, run `ffprobe -v error -show_entries format=start_time,duration -of default=noprint_wrappers=1 your_video.mp4`. Must be greater than `startTime` and less than or equal to the video duration. The window (`endTime - startTime`) must be at least 1 second and at most 2 hours. The video may be up to 4 hours as long as the window stays within that limit. Mutually exclusive with `responseFormat.segmentDefinitions[].timeRanges`. Together with `startTime`, this parameter determines the billable video duration. If you omit both, billing uses the full video duration. For details, see the [Frequently asked questions](/v1.3/docs/resources/frequently-asked-questions#how-is-video-segmentation-priced) page. | ### VideoContext The `TwelvelabsApi.VideoContext` type specifies the source of the video. Provide exactly one of the following. Mutually exclusive with the `image` parameter. | Type | Field | Type | Description | | --------------------------- | -------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `VideoContext.Url` | `url` | `string` | The publicly accessible URL of the video file or HLS manifest. Use direct links to raw media files, or the URL of a VOD HLS manifest. Live video streams are rejected with a `400` error. Video hosting platforms and cloud storage sharing links are not supported. For HLS sources, if the duration cannot be determined, the platform calculates it from the manifest and the duration limits apply to that value. | | `VideoContext.AssetId` | `assetId` | `string` | The unique identifier of an asset from a [direct](/v1.3/sdk-reference/node-js/upload-content/direct-uploads) or [multipart](/v1.3/sdk-reference/node-js/upload-content/multipart-uploads) upload. The asset status must be `ready`. Use [`assets.retrieve`](/v1.3/sdk-reference/node-js/manage-assets#retrieve-an-asset) to check the status. | | `VideoContext.Base64String` | `base64String` | `string` | The base64-encoded video data. The maximum size is 30 MB. | Set the `type` discriminator field to `"url"`, `"asset_id"`, or `"base64_string"` to indicate which source you are providing. ### AnalyzeImageInput The `TwelvelabsApi.AnalyzeImageInput` interface defines an image to analyze. Provide exactly one of `url`, `assetId`, or `base64String`. | Name | Type | Required | Description | | -------------- | -------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `name` | `string` | Yes | The name of this image. Must be unique within the `image` array. Can contain only letters, digits, and underscores. To refer to a specific image in your prompt, use its name as the `<@name>` placeholder. Reference every image or none of them: - If the prompt references every image, the platform analyzes each one (Example: `Compare <@before> with <@after> and describe what changed.`). - If the prompt references none of them, the platform analyzes every image you provided (Example: `List every visible feature in these images.`). - If the prompt references only some of the images, the platform returns a `400` error. | | `url` | `string` | No | A publicly accessible HTTPS URL of the image. The platform rejects HTTP URLs. When you send the request, the platform verifies that the URL is reachable and that the image is within the [size, pixel count, and format limits](/v1.3/docs/concepts/models/pegasus/pegasus-1-6#image-file-requirements). If either check fails, the platform returns a `400` error. | | `assetId` | `string` | No | The unique identifier of an uploaded asset. | | `base64String` | `string` | No | Base64-encoded image data. The maximum decoded size is 20 MB. Supported image types: `image/jpeg`, `image/png`, `image/webp`, `image/gif`, and `image/bmp`. Standard and unpadded base64 encodings are both accepted. | The `AsyncResponseFormat` class contains the following properties: | Name | Type | Description | | -------------------- | ---------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | `TwelvelabsApi.AsyncResponseFormatType` | The response format to use. Values: `"json_schema"` (structured JSON conforming to a provided schema), `"segment_definitions"` (timestamped metadata with custom fields, requires `analysisMode` set to `"time_based_metadata"`). | | `jsonSchema` | `Record` | Contains the JSON schema that defines the response structure. The schema must adhere to the [JSON Schema Draft 2020-12](https://json-schema.org/draft/2020-12) specification. For details, see the [`json_schema`](/v1.3/docs/guides/analyze-videos-and-images#request.body.response_format.json_schema) parameter in the API Reference section. | | `segmentDefinitions` | `TwelvelabsApi.SegmentDefinition[]` | Define the types of segments to extract from your video. Required when `type` is `"segment_definitions"`. Minimum 1, maximum 20 definitions. The number of segment definitions affects billing. For details, see the [Frequently asked questions](/v1.3/docs/resources/frequently-asked-questions#how-is-video-segmentation-priced) page. See [SegmentDefinition](/v1.3/sdk-reference/node-js/analyze-videos/async-analysis#segmentdefinition) for details. | | `segmentTimeFormat` | `TwelvelabsApi.AsyncResponseFormatSegmentTimeFormat` | Set the output format for the automatic `start_time` and `end_time` keys returned on each segment. Requires the `type` parameter set to `"segment_definitions"`. Omitting this parameter is equivalent to setting it to `"seconds"`. Values: `"seconds"` (JSON number in seconds, Example: `12.5`), `"hh:mm:ss"` (JSON string rounded to the nearest second, Example: `"00:00:13"`), `"hh:mm:ss.fff"` (JSON string with millisecond precision, Example: `"00:00:12.500"`). This parameter applies only to the automatic segment boundaries (`start_time` and `end_time`). For a `time_array` field, it also formats the `start_time` and `end_time` of each event inside the segment. Custom `timestamp` fields always use their own format, regardless of the value of this field. | ### AnalyzePromptV2 The `AnalyzePromptV2` interface defines a structured prompt that uses `<@name>` placeholders to reference images. Mutually exclusive with the `prompt` and `image` parameters. | Name | Type | Required | Description | | -------------- | -------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `inputText` | `string` | Yes | The text of the prompt. Use `<@name>` placeholders to reference images declared in `mediaSources` (Example: `"Is there a <@tiger-1> in the video?"`). This text counts toward the [context window](/v1.3/docs/concepts/models/pegasus/pegasus-1-6#context-window). | | `mediaSources` | `TwelvelabsApi.SmeMediaSource[]` | No | Reference images for the `<@name>` placeholders in the prompt. Maximum 4 sources. See [SmeMediaSource](#smemediasource). | ### SegmentDefinition The `TwelvelabsApi.SegmentDefinition` interface defines a type of segment to extract from the video. | Name | Type | Required | Description | | -------------- | -------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `id` | `string` | Yes | A unique identifier for this segment definition. | | `description` | `string` | Yes | Describe what this type of segment looks like in the video. The model uses this text to identify matching segments. | | `fields` | `TwelvelabsApi.SegmentField[]` | No | Custom fields to extract for each segment instance. Maximum 20 fields. Set the `type` field of an item to the `time_array` value to extract a list of events inside each segment. See [SegmentField](#segmentfield) for details. | | `mediaSources` | `TwelvelabsApi.SmeMediaSource[]` | No | Reference images that help the model identify segments. Maximum 4 sources. See [SmeMediaSource](#smemediasource) for details. | ### SegmentField The `TwelvelabsApi.SegmentField` interface defines a custom field to extract for each segment. | Name | Type | Required | Description | | ------------- | ---------------------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `name` | `string` | Yes | The name of the field. | | `type` | `TwelvelabsApi.SegmentFieldType` | Yes | The data type of the field. Values: `"string"`, `"boolean"`, `"number"`, `"integer"`, `"array"`, `"timestamp"`, `"time_array"`. When set to `"timestamp"`, the `format` property is required and controls the format of the returned value. When set to the `"time_array"` value, the field extracts a list of events inside each segment. Requires the `items.type` field to be `"object"` and a non-empty `items.fields` list. Requires Pegasus 1.6. Any other model rejects `"time_array"` as an invalid type and lists only the types it accepts. | | `description` | `string` | Yes | Instructions that guide the model on what this field should contain and how to extract it from the video. | | `format` | `TwelvelabsApi.SegmentFieldFormat` | No | The output format for `timestamp` fields. Required when `type` is `"timestamp"`. Must be omitted for any other type. Values: `"seconds"` (JSON number in seconds, Example: `10.5`), `"hh:mm:ss"` (JSON string rounded to the nearest second, Example: `"00:01:23"`), `"hh:mm:ss.fff"` (JSON string with millisecond precision, Example: `"00:01:23.500"`). | | `enum` | `string[]` | No | Allowed values for this field. Maximum 100 values. Not supported when `type` is `"timestamp"`. | | `items` | `TwelvelabsApi.SegmentFieldItems` | No | Required when `type` is `"array"` or `"time_array"`. For `"array"`, specifies the type of array elements. For `"time_array"`, set it to `{"type": "object", "fields": [...]}`. Not supported when `type` is `"timestamp"`. See [SegmentFieldItems](#segmentfielditems) for details. | ### SegmentFieldItems The `TwelvelabsApi.SegmentFieldItems` interface defines the per-item structure of a segment field whose `type` is `array` or `time_array`. For an `array` field, the `type` value specifies the scalar element type. For a `time_array` field, use `object` as the `type` value and declare the fields of each event in the `fields` array. | Name | Type | Required | Description | | -------- | ------------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | `TwelvelabsApi.SegmentFieldItemsType` | Yes | The item type. For an `array` field, use one of the scalar types (`string`, `number`, `boolean`, `integer`). For a `time_array` field, use `object` and declare the per-event schema in the `fields` array. Setting `object` as the `type` value on a plain `array` field returns a `400` error. Setting the `fields` array on a plain `array` field also returns a `400` error. | | `fields` | `TwelvelabsApi.TimeArrayItemField[]` | No | The fields to extract from each event. Required when the `type` of the parent field is `time_array`, with at least one entry. Not permitted on any other field type. | ### TimeArrayItemField The `TwelvelabsApi.TimeArrayItemField` interface defines a field inside a `time_array` item. The platform adds `start_time` and `end_time` keys to every event automatically. You cannot declare fields with these names. A `time_array` field cannot contain another `time_array` field. The `format` property is not accepted on per-event fields. | Name | Type | Required | Description | | ------------- | --------------------------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `name` | `string` | Yes | The name of the field inside each event object. Must be unique within the item. The names `start_time`, `end_time`, and `metadata` are reserved and rejected with a `400` error. | | `type` | `TwelvelabsApi.TimeArrayItemFieldType` | Yes | The data type of this field. It does not accept `timestamp` or `time_array`. When set to the `array` value, the `items` object is required. Values: `"string"`, `"boolean"`, `"number"`, `"integer"`, `"array"`. | | `description` | `string` | Yes | Instructions that guide the model on what this field should contain in each event. | | `enum` | `string[]` | No | Allowed values for this field. Maximum 100 values. Only supported when `type` is `string`; any other type returns a `400` error. | | `items` | `TwelvelabsApi.TimeArrayItemFieldItems` | No | The element type for an `array` field. Required when `type` is `array`; not supported on any other type. The `items` object has a single property `type` with values: `"string"`, `"number"`, `"boolean"`, `"integer"`. | ### SmeMediaSource The `TwelvelabsApi.SmeMediaSource` interface defines a reference image that provides visual context for segment identification. Provide exactly one of `url`, `assetId`, or `base64String`. | Name | Type | Required | Description | | -------------- | --------------------------------------- | -------- | ------------------------------------------------------------- | | `name` | `string` | Yes | A descriptive name for this media source. | | `mediaType` | `TwelvelabsApi.SmeMediaSourceMediaType` | Yes | The media type. Value: `"image"`. | | `url` | `string` | No | A publicly accessible HTTPS URL of the image. | | `assetId` | `string` | No | The unique identifier of an uploaded asset. | | `base64String` | `string` | No | Base64-encoded image data. The maximum decoded size is 32 MB. | **Return value**: Returns an `HttpResponsePromise` that resolves to a `TwelvelabsApi.CreateAnalyzeTaskResponse` object containing the task details. The `TwelvelabsApi.CreateAnalyzeTaskResponse` interface contains the following properties: | Name | Type | Description | | -------- | --------------------------------- | ------------------------------------------------ | | `taskId` | `string` | The unique identifier of the analysis task. | | `status` | `TwelvelabsApi.AnalyzeTaskStatus` | The initial status of the task. Value: `queued`. | **API Reference**: [Create an async analysis task](/v1.3/api-reference/analyze-videos/create-async-analysis-task) ## Retrieve task status and results **Description**: This method retrieves the status and results of an analysis task. **Task statuses**: * `queued`: The task is waiting to be processed. * `pending`: The task is queued and waiting to start. * `processing`: The platform is analyzing the video. * `ready`: Processing is complete. Results are available in the response. * `failed`: The task failed. No result is available. The `error` field describes the failure. * `canceled`: The task was canceled. No result is available. The `error` field describes the cancellation reason, if available. The `ready`, `failed`, and `canceled` statuses are final. Poll this method until `status` is `ready`, `failed`, or `canceled`. When `status` is `ready`, use the results from the response. **Function signature and example**: **`Function signature`** ```javascript Function signature retrieve( taskId: string, requestOptions?: Tasks.RequestOptions ): core.HttpResponsePromise ``` **`Poll for task completion`** ```javascript Poll for task completion import { TwelvelabsApiClient } from "twelvelabs-js"; while (true) { const task = await client.analyzeAsync.tasks.retrieve(""); console.log(`Status: ${task.status}`); if (task.status === "ready") { console.log(`Generated text: ${task.result?.data}`); console.log(`Output tokens: ${task.result?.usage.outputTokens}`); break; } else if (task.status === "failed" || task.status === "canceled") { throw new Error(task.error?.message ?? `Task ended with status: ${task.status}`); } else { await new Promise(resolve => setTimeout(resolve, 5000)); } } ``` **Parameters**: | Name | Type | Required | Description | | ---------------- | ---------------------- | -------- | ------------------------------------------- | | `taskId` | `string` | Yes | The unique identifier of the analysis task. | | `requestOptions` | `Tasks.RequestOptions` | No | Request-specific configuration. | **Return value**: Returns an `HttpResponsePromise` that resolves to a `TwelvelabsApi.AnalyzeTaskResponse` object containing the task status and results. The `TwelvelabsApi.AnalyzeTaskResponse` interface contains the following properties: | Name | Type | Description | | --------------- | ------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `taskId` | `string` | The unique identifier of the analysis task. | | `customId` | `string` | The identifier you provided when you created the task, or `null` if you did not set one. This property is always present. | | `batchId` | `string` | The unique identifier of the batch that the task was created in. The platform returns this property only for tasks created as part of a batch. | | `videoSource` | `TwelvelabsApi.AnalyzeTaskResponseVideoSource` | The video source associated with the task. When the video was uploaded using the [`POST`](/v1.3/api-reference/index-content/create) method of the `/tasks` endpoint, the source type is `video_id`. Otherwise, the source type is `url`, `base64_string`, or `asset_id`. | | `requestParams` | `TwelvelabsApi.AnalyzeTaskResponseRequestParams` | The request parameters for this task. | | `status` | `TwelvelabsApi.AnalyzeTaskStatus` | The current status of the task. Values: `queued`, `pending`, `processing`, `ready`, `failed`, `canceled`. | | `createdAt` | `Date` | The date and time when the task was created, in RFC 3339 format. | | `completedAt` | `Date` | The date and time when the task completed, failed, or was canceled, in RFC 3339 format. The platform returns this field only when `status` is `ready`, `failed`, or `canceled`. | | `result` | `TwelvelabsApi.AnalyzeTaskResult` | An object that contains the generated text and additional information. The platform returns this object only when `status` is `ready`. A failed or canceled task has no `result` object, so `generationId` and `usage` are absent. | | `error` | `TwelvelabsApi.AnalyzeTaskError` | A message attached to the task response. The platform sets this field in the following cases: - `status` is `"failed"`: The `message` field describes the failure reason. With video segmentation, a task can fail because the analysis reached the maximum response length or the [context window](/v1.3/docs/concepts/models/pegasus/pegasus-1-6#context-window) before it could complete. The response contains no `result` object. - `status` is `"canceled"` and a cancellation reason is available: `code` identifies the machine-readable reason and `message` describes it. A task newly canceled through the task cancellation endpoint has `code` set to `"user_canceled"`. The field is omitted when no cancellation reason is available. - `status` is `"ready"` and the output for some segments could not be recovered (video segmentation): The `message` field contains the warning that the results may be incomplete. The segments the platform extracted are in `result.data`. - `status` is `"ready"` and `result.finishReason` is `"length"` (general analysis): The `message` field describes the truncation cause (either the maximum response length was reached or the context window was reached). The partial output is in `result.data`. Not set when `status` is `"ready"` and `result.finishReason` is `"stop"`, except for the incomplete results warning. | | `webhooks` | `TwelvelabsApi.AnalyzeTaskWebhookInfo[]` | The delivery status of each webhook endpoint. The platform omits this field when there are no webhooks configured. | The `TwelvelabsApi.AnalyzeTaskResponseVideoSource` interface contains the following properties: | Name | Type | Description | | ---------------- | ------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | `TwelvelabsApi.AnalyzeTaskResponseVideoSourceType` | The type of video source. Values: `"url"`, `"base64_string"`, `"asset_id"`, `"video_id"`. | | `url` | `string` | The video URL. Present when `type` is `"url"`. | | `assetId` | `string` | The asset ID. Present when `type` is `"asset_id"`. | | `videoId` | `string` | The deprecated video identifier. Present when `type` is `"video_id"`. | | `indexId` | `string` | The deprecated identifier of the index associated with the video. Present on a best-effort basis when `type` is `"video_id"`. | | `systemMetadata` | `TwelvelabsApi.AnalyzeTaskResponseVideoSourceSystemMetadata` | Video metadata that the platform extracted during processing, such as its duration. Present on a best-effort basis once the video has been processed. | The `TwelvelabsApi.AnalyzeTaskResponseVideoSourceSystemMetadata` interface contains the following properties: | Name | Type | Description | | ---------- | -------- | ------------------------------ | | `duration` | `number` | The video duration in seconds. | The `TwelvelabsApi.AnalyzeTaskResponseRequestParams` interface contains the following properties: | Name | Type | Description | | -------------------- | -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `analysisMode` | `TwelvelabsApi.AnalyzeTaskResponseRequestParamsAnalysisMode` | The analysis mode for this task. For a task created with the `image` parameter, the value is `general`. Values: `"general"`, `"time_based_metadata"`. | | `prompt` | `string` | The natural-language prompt for this task. Present only when `analysisMode` is `general` and the task was created with `prompt` (not `promptV2`). On the [List](#list-analysis-tasks) method, truncated to the first 30 characters; on the [Retrieve](#retrieve-task-status-and-results) method, returns the full text. | | `promptV2` | `TwelvelabsApi.AnalyzeTaskResponseRequestParamsPromptV2` | The structured prompt for this task. Present only when `analysisMode` is `general` and the task was created with `promptV2`. When present, the response excludes the flat `prompt` field. On the [List](#list-analysis-tasks) method, `input_text` is truncated to the first 30 characters; on the [Retrieve](#retrieve-task-status-and-results) method, returns the full text. | | `image` | `TwelvelabsApi.AnalyzeTaskMediaSource[]` | The image input you provided. Present only when the task was created with the `image` parameter. When present, the response omits `video_source` instead of returning it as `null`. | | `responseFormat` | `TwelvelabsApi.AnalyzeTaskResponseRequestParamsResponseFormat` | The response format for this task. Present only when the request included a response format. | | `temperature` | `number` | The temperature value for this analysis. | | `maxTokens` | `TwelvelabsApi.AnalyzeTaskResponseRequestParamsMaxTokens` | The maximum response length you set. The value is an integer in tokens or the string `unlimited`. | | `minSegmentDuration` | `number` | The minimum segment duration you set, in seconds. Present when `analysisMode` is `"time_based_metadata"`. | | `maxSegmentDuration` | `number` | The maximum segment duration you set, in seconds. Present when `analysisMode` is `"time_based_metadata"`. | | `startTime` | `number` | The start of the analysis window, in seconds. Present only when the task was created with `startTime`. | | `endTime` | `number` | The end of the analysis window, in seconds. Present only when the task was created with `endTime`. | The `TwelvelabsApi.AnalyzeTaskMediaSource` interface contains the following properties: | Name | Type | Description | | ----------- | -------- | ------------------------------------------------------------ | | `name` | `string` | The reference name you supplied. | | `mediaType` | `string` | The media type. The value is always `image`. | | `url` | `string` | Present when the source was provided as a URL. | | `assetId` | `string` | Present when the source was provided as an asset identifier. | The `TwelvelabsApi.AnalyzeTaskResponseRequestParamsResponseFormat` interface contains the following properties: | Name | Type | Description | | -------------------- | -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | `TwelvelabsApi.AnalyzeTaskResponseRequestParamsResponseFormatType` | Values: `"json_schema"`, `"segment_definitions"`. | | `jsonSchema` | `Record` | The JSON schema for this task. Present only when `type` is `json_schema`. - On the [List](#list-analysis-tasks) method: omitted. - On the [Retrieve](#retrieve-task-status-and-results) method: returns the full schema. | | `segmentTimeFormat` | `TwelvelabsApi.AnalyzeTaskResponseRequestParamsResponseFormatSegmentTimeFormat` | The `segment_time_format` value you set. Omitted when you did not set it. Automatic boundaries are then returned as JSON numbers in seconds. Values: `"seconds"`, `"hh:mm:ss"`, `"hh:mm:ss.fff"`. | | `segmentDefinitions` | `TwelvelabsApi.AnalyzeTaskResponseRequestParamsResponseFormatSegmentDefinitionsItem[]` | The segment definitions for this task. Present only when `type` is `segment_definitions`. - On the [List](#list-analysis-tasks) method: returns only the first element. - On the [Retrieve](#retrieve-task-status-and-results) method: returns the full array. | The `TwelvelabsApi.AnalyzeTaskResult` interface contains the following properties: | Name | Type | Description | | -------------- | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `generationId` | `string` | The unique identifier for the generation session. | | `data` | `string` | The generated text for this analysis task. When `analysisMode` is not set, a plain-text string based on the prompt you provided. When `analysisMode` is `"time_based_metadata"`, a JSON-encoded string keyed by segment definition `id`. Each key maps to an array of segment objects with `start_time` (number), `end_time` (number), and `metadata` (object with your custom fields). The value of a `time_array` field is an array of event objects, one for each event inside the segment. Each event object contains `start_time` and `end_time`, which the platform adds automatically, plus the fields you declared in the `items.fields` array. - Segments are within the duration of the video, and events are within the duration of their segment. - The `min_segment_duration` and `max_segment_duration` values limit segments only, not the events inside them. - Event boundaries use the same format as segment boundaries, set by the `segment_time_format` value. | | `finishReason` | `TwelvelabsApi.FinishReason` | The reason the generation stopped. Values: `null` (generation has not finished), `"stop"` (the generation reached the end of the output text), `"length"` (the generation reached the maximum response length or the [context window](/v1.3/docs/concepts/models/pegasus/pegasus-1-6#context-window); with a JSON response format, the output may be truncated and fail to parse). When the task uses general analysis, the partial output is in `data`, and a warning is in the task's `error` field. With video segmentation and a numeric `maxTokens`, the task fails when the analysis reaches either limit. This field is never `"length"` for a segmentation task. With `maxTokens` set to `"unlimited"`, this field is `"stop"` even when the output is incomplete. | | `usage` | `TwelvelabsApi.AnalyzeTaskResultUsage` | The number of tokens used in the generation. | The `TwelvelabsApi.AnalyzeTaskResultUsage` interface contains the following properties: | Name | Type | Description | | -------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `outputTokens` | `number` | The number of tokens in the generated text. | | `inputTokens` | `number` | The number of tokens the input consumed. Together with `outputTokens`, this value must fit within the [context window](/v1.3/docs/concepts/models/pegasus/pegasus-1-6#context-window). | The `TwelvelabsApi.AnalyzeTaskError` interface contains the following properties: | Name | Type | Description | | --------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `code` | `string` | A machine-readable code that identifies the task's condition. Cancellation values are `"user_canceled"`, `"batch_canceled"`, and `"batch_expired"`; `"batch_processing_timeout"` identifies a batch processing failure. Clients must tolerate unrecognized values. These task condition codes are distinct from API error response codes, even when a string appears in both. The platform omits this field for failures and truncation warnings that have no code. | | `message` | `string` | A human-readable message for task failure, cancellation, a video-segmentation incomplete-results warning, or a general-analysis truncation warning. Do not parse this field as a discriminator; use `status`, `code`, and `finishReason` instead. For documented messages, see [Error codes](/v1.3/api-reference/error-codes#the-analyze-endpoint). | **API Reference**: [Retrieve analysis task status and results](/v1.3/api-reference/analyze-videos/retrieve-analysis-task-status-results) ## Cancel an async analysis task **Description**: Use this method to cancel an asynchronous analysis task in your account. To cancel a task created as part of a batch, use the [`analyzeAsync.batches.cancel`](/v1.3/sdk-reference/node-js/analyze-videos/batch-analysis#cancel-a-batch) method. You can cancel a task with the `queued`, `pending`, or `processing` status. This action cannot be undone. Processing that has already started can continue briefly after cancellation. When you cancel a task, the platform can send an `analyze.task.canceled` webhook. Delivery is best-effort: a `200` response is not a delivery guarantee. When you receive the event, retrieve the task for its current state. **Function signature and example**: **`Function signature`** ```javascript Function signature cancel( taskId: string, requestOptions?: Tasks.RequestOptions ): core.HttpResponsePromise ``` **`Node.js example`** ```javascript Node.js example import { TwelveLabs } from "twelvelabs-js"; const client = new TwelveLabs({ apiKey: "" }); const receipt = await client.analyzeAsync.tasks.cancel(""); console.log(`Status: ${receipt.status}`); ``` **Parameters**: | Name | Type | Required | Description | | ---------------- | ---------------------- | -------- | -------------------------------------------------------------- | | `taskId` | `string` | Yes | The unique identifier of the analysis task you want to cancel. | | `requestOptions` | `Tasks.RequestOptions` | No | Request-specific configuration. | **Return value**: Returns an `HttpResponsePromise` that resolves to a `TwelvelabsApi.CancelAnalyzeTaskResponse` containing `taskId` and the final status `canceled`. This response is also returned when the task was canceled by a previous request. **API Reference**: [Cancel an async analysis task](/v1.3/api-reference/analyze-videos/cancel-an-async-analysis-task) ## Delete an analysis task **Description**: This method deletes an analysis task. You can only delete tasks that are not currently being processed. **Function signature and example**: **`Function signature`** ```javascript Function signature delete( taskId: string, requestOptions?: Tasks.RequestOptions ): core.HttpResponsePromise ``` **`Node.js example`** ```javascript Node.js example import { TwelvelabsApiClient } from "twelvelabs-js"; await client.analyzeAsync.tasks.delete(""); ``` **Parameters**: | Name | Type | Required | Description | | ---------------- | ---------------------- | -------- | ------------------------------------------- | | `taskId` | `string` | Yes | The unique identifier of the analysis task. | | `requestOptions` | `Tasks.RequestOptions` | No | Request-specific configuration. | **Return value**: Returns an `HttpResponsePromise` that resolves to `void`. If successful, the platform returns a `204 No Content` response. **API Reference**: [Delete an analysis task](/v1.3/api-reference/analyze-videos/delete-analysis-task)