Async analysis

The AnalyzeAsyncClient.TasksClient class provides methods to analyze videos asynchronously, generate text, and extract structured, timestamped segments. The platform supports two analysis modes: general analysis (prompt-based text generation) and video segmentation with custom segment definitions. Both modes use Pegasus 1.5.

When to use this class:

  • Generate custom text from your video using a prompt (general analysis)
  • Extract timestamped metadata with custom segment definitions from your video
  • Analyze videos longer than 1 hour, or a portion of a video up to 4 hours long
  • Process videos asynchronously without blocking your application

Do not use this class for:

  • Videos for which you need immediate results or real-time streaming. Use the analyze method instead.
  • The video can be up to 2 hours long, or up to 4 hours when you analyze only a portion of it. You can analyze between 1 second and 2 hours of the video. HLS and base64 videos are limited to 2 hours.
  • Formats: FFmpeg supported formats
  • Resolution: 360x360 to 5184x2160 pixels
  • Aspect ratio: Between 1:1 and 1:2.4, or between 2.4:1 and 1:1

On the Free plan, analysis hours count toward a shared limit that also covers indexing - the number of segment definitions does not affect this limit. On paid plans, you pay based on how much video you process and how many segment definitions you include - see the Frequently asked questions page for examples.

Analyzing videos asynchronously requires three steps:

  1. Create an analysis task using the create method. The platform returns a task identifier.
  2. Poll the status of the task using the retrieve method. Wait until the status is ready.
  3. Retrieve the results from the response when the status is ready using the retrieve method.

Methods

List analysis tasks

Description: This method returns a list of the analysis tasks in your account. The platform returns your analysis tasks sorted by creation date, with the newest at the top of the list.

Function signature and example:

def list(
self,
*,
page: typing.Optional[int] = None,
page_limit: typing.Optional[int] = None,
status: typing.Optional[AnalyzeTaskStatus] = None,
video_url: typing.Optional[str] = None,
asset_id: typing.Optional[str] = None,
analysis_mode: typing.Optional[TasksListRequestAnalysisMode] = None,
request_options: typing.Optional[RequestOptions] = None,
) -> TasksListResponse:

Parameters:

NameTypeRequiredDescription
pageintNoA number that identifies the page to retrieve. Default: 1.
page_limitintNoThe number of items to return on each page. Default: 10. Max: 50.
statusAnalyzeTaskStatusNoFilter analysis tasks by status. Values: queued, pending, processing, ready, failed.
video_urlstrNoFilter tasks by exact video source URL.
asset_idstrNoFilter tasks by asset ID.
analysis_modeTasksListRequestAnalysisModeNoFilter tasks by the analysis mode used when creating the task. Values: "general", "time_based_metadata".
request_optionsRequestOptionsNoRequest-specific configuration.

Return value: Returns a TasksListResponse object.

The TasksListResponse class contains the following properties:

NameTypeDescription
dataList[AnalyzeTaskResponse]An array that contains up to page_limit analysis tasks.
page_infoPageInfoAn object that provides information about pagination.

The PageInfo class contains the following properties:

NameTypeDescription
limit_per_pageOptional[int]The number of items returned per page.
pageOptional[int]The current page number.
total_pageOptional[int]The total number of pages.
total_resultsOptional[int]The total number of analysis tasks in your account.

For details about AnalyzeTaskResponse, see Retrieve task status and results.

API Reference: List async analysis tasks

Create an async analysis task

Description: This method asynchronously analyzes your videos. It supports two analysis modes: general analysis (prompt-based text generation) and video segmentation with custom segment definitions. Both modes use Pegasus 1.5.

On the Free plan, you have a total of 600 minutes (10 hours) shared across indexing, analysis, and segmentation. For details, see the Video hours and video count limits section.

Note

This method is rate-limited. For details, see the Rate limits page.

Function signature and example:

def create(
self,
*,
video: VideoContext,
model_name: typing.Optional[CreateAsyncAnalyzeRequestModelName] = OMIT,
custom_id: typing.Optional[str] = OMIT,
prompt: typing.Optional[AnalyzeTextPrompt] = OMIT,
prompt_v_2: typing.Optional[AnalyzePromptV2] = OMIT,
analysis_mode: typing.Optional[CreateAsyncAnalyzeRequestAnalysisMode] = OMIT,
temperature: typing.Optional[AnalyzeTemperature] = OMIT,
max_tokens: typing.Optional[int] = OMIT,
response_format: typing.Optional[AsyncResponseFormat] = OMIT,
min_segment_duration: typing.Optional[float] = OMIT,
max_segment_duration: typing.Optional[float] = OMIT,
start_time: typing.Optional[float] = OMIT,
end_time: typing.Optional[float] = OMIT,
request_options: typing.Optional[RequestOptions] = None,
) -> CreateAnalyzeTaskResponse:

Parameters:

NameTypeRequiredDescription
model_nameCreateAsyncAnalyzeRequestModelNameNoThe video understanding model to use for analysis. Values:
- "pegasus1.5": General analysis (prompt-based text generation) with video clipping, structured prompts with reference images, and video segmentation. See the Pegasus page for token limits.
Default: "pegasus1.5"
custom_idstrNoAn optional identifier that you set when you create the task. Use this field to correlate tasks across responses, for example, to distinguish tasks by type or environment. Must match the pattern ^[a-zA-Z0-9_-]{1,64}$.
videoVideoContextYesAn object that specifies the source of the video. See VideoContext for details.
promptstrNoNatural-language instructions for analyzing the video. Required for general analysis (prompt-based text generation). Not supported when analysis_mode is "time_based_metadata". To include reference images in your prompt, use the prompt_v_2 parameter instead. Mutually exclusive with the prompt_v_2 parameter.
prompt_v_2AnalyzePromptV2NoA structured prompt with the <@name> placeholders for referencing images. Not supported when the analysis_mode parameter is "time_based_metadata". Mutually exclusive with the prompt parameter. See AnalyzePromptV2.
analysis_modeCreateAsyncAnalyzeRequestAnalysisModeNoThe analysis approach for this task. Values:
- "general": Analyze the video and generate a response based on your prompt. Supports both free-form text and structured output via response_format.
- "time_based_metadata": Segment the video into time-based intervals and extract custom metadata for each segment. Requires response_format.type set to "segment_definitions".
Default: "general"
temperaturefloatNoControls the randomness of the text output. Default: 0.2, Min: 0, Max: 1
max_tokensintNoThe maximum response length, in tokens. For "pegasus1.5" general mode: Min: 512, Max: 98,304, Default: 4,096. For "pegasus1.5" time_based_metadata mode: Min: 2,048, Max: 98,304, Default: 32,768. The input and response must fit within the context window. With video segmentation, if the response needs more tokens than max_tokens allows, the task fails and no partial output is returned.
response_formatAsyncResponseFormatNoControls the response format. When you omit this parameter, you receive unstructured text.
min_segment_durationfloatNoMinimum duration for each extracted segment, in seconds. Prevents the model from creating very short segments. Requires analysis_mode set to "time_based_metadata". Min: 2.
max_segment_durationfloatNoMaximum duration for each extracted segment, in seconds. Breaks long continuous sections into shorter segments. Must be greater than or equal to the min_segment_duration parameter. Requires analysis_mode set to "time_based_metadata". Min: 2.
start_timefloatNoStart of the analysis window, as an absolute timestamp in seconds, based on the internal metadata of the video. Use with end_time to analyze only a portion of the video. If omitted, defaults to the internal start time of the video. Most videos start at 0, but some (for example, from cameras or broadcast recordings) may have a non-zero start time. To find the value, run ffprobe -v error -show_entries format=start_time,duration -of default=noprint_wrappers=1 your_video.mp4. Must be less than end_time and the video duration. The window (end_time - start_time) must be at least 1 second and at most 2 hours. The video may be up to 4 hours as long as the window stays within that limit. Mutually exclusive with response_format.segment_definitions[].time_ranges. Together with end_time, this parameter determines the billable video duration. If you omit both, billing uses the full video duration. For details, see the Frequently asked questions page.
end_timefloatNoEnd of the analysis window, as an absolute timestamp in seconds, based on the internal metadata of the video. Use with start_time to analyze only a portion of the video. If omitted, defaults to the internal start time of the video plus its duration. Most videos start at 0, but some (for example, from cameras or broadcast recordings) may have a non-zero start time. To find the value, run ffprobe -v error -show_entries format=start_time,duration -of default=noprint_wrappers=1 your_video.mp4. Must be greater than start_time and less than or equal to the video duration. The window (end_time - start_time) must be at least 1 second and at most 2 hours. The video may be up to 4 hours as long as the window stays within that limit. Mutually exclusive with response_format.segment_definitions[].time_ranges. Together with start_time, this parameter determines the billable video duration. If you omit both, billing uses the full video duration. For details, see the Frequently asked questions page.
request_optionsRequestOptionsNoRequest-specific configuration.

VideoContext

The VideoContext type specifies the source of the video. Provide exactly one of the following:

ClassFieldTypeDescription
VideoContext_UrlurlstrThe publicly accessible URL of the video file or HLS manifest. Use direct links to raw media files, or the URL of a VOD HLS manifest. Live video streams are rejected with a 400 error. Video hosting platforms and cloud storage sharing links are not supported. For HLS sources, if the duration cannot be determined, the platform calculates it from the manifest and the duration limits apply to that value.
VideoContext_AssetIdasset_idstrThe unique identifier of an asset from a direct or multipart upload. The asset status must be ready. Use assets.retrieve to check the status.
VideoContext_Base64Stringbase_64_stringstrThe base64-encoded video data. The maximum size is 30 MB.

The AsyncResponseFormat class contains the following properties:

NameTypeDescription
typeAsyncResponseFormatTypeThe response format to use. Values: "json_schema" (structured JSON conforming to a provided schema), "segment_definitions" (timestamped metadata with custom fields, requires analysis_mode set to "time_based_metadata").
json_schemaOptional[Dict[str, Optional[Any]]]Contains the JSON schema that defines the response structure. The schema must adhere to the JSON Schema Draft 2020-12 specification. For details, see the json_schema parameter in the API Reference section.
segment_definitionsOptional[List[SegmentDefinition]]Define the types of segments to extract from your video. Required when type is "segment_definitions". Minimum 1, maximum 20 definitions. The number of segment definitions affects billing. For details, see the Frequently asked questions page. See SegmentDefinition for class details.
segment_time_formatOptional[AsyncResponseFormatSegmentTimeFormat]Set the output format for the automatic start_time and end_time keys returned on each segment. Requires the type parameter set to "segment_definitions". Omitting this parameter is equivalent to setting it to "seconds". Values: "seconds" (JSON number in seconds, Example: 12.5), "hh:mm:ss" (JSON string rounded to the nearest second, Example: "00:00:13"), "hh:mm:ss.fff" (JSON string with millisecond precision, Example: "00:00:12.500"). This parameter applies only to the automatic segment boundaries. Custom timestamp fields always use their own declared format.

AnalyzePromptV2

The AnalyzePromptV2 class defines a structured prompt with image references.

NameTypeRequiredDescription
input_textstrYesThe text of the prompt. Use <@name> placeholders to reference images declared in media_sources (Example: "Is there a <@tiger-1> in the video?"). This text counts toward the context window.
media_sourcesOptional[List[SmeMediaSource]]NoReference images for the <@name> placeholders in the prompt. Maximum 4 sources. See SmeMediaSource.

SegmentDefinition

The SegmentDefinition class defines a type of segment to extract from the video.

NameTypeRequiredDescription
idstrYesA unique identifier for this segment definition.
descriptionstrYesDescribe what this type of segment looks like in the video. The model uses this text to identify matching segments.
fieldsOptional[List[SegmentField]]NoCustom fields to extract for each segment instance. Maximum 20 fields. See SegmentField for details.
media_sourcesOptional[List[SmeMediaSource]]NoReference images that help the model identify segments. Maximum 4 sources. See SmeMediaSource for details.

SegmentField

The SegmentField class defines a custom field to extract for each segment.

NameTypeRequiredDescription
namestrYesThe name of the field.
typeSegmentFieldTypeYesThe data type of the field. Values: "string", "boolean", "number", "integer", "array", "timestamp". When set to "timestamp", the format property is required and controls the format of the returned value.
descriptionstrYesInstructions that guide the model on what this field should contain and how to extract it from the video.
formatOptional[SegmentFieldFormat]NoThe output format for timestamp fields. Required when type is "timestamp". Must be omitted for any other type. Values: "seconds" (JSON number in seconds, Example: 10.5), "hh:mm:ss" (JSON string rounded to the nearest second, Example: "00:01:23"), "hh:mm:ss.fff" (JSON string with millisecond precision, Example: "00:01:23.500").
enumOptional[List[str]]NoAllowed values for this field. Maximum 100 values. Not supported when type is "timestamp".
itemsOptional[SegmentFieldItems]NoRequired when type is "array". Specifies the type of array elements. The items object has a single property type with values: "string", "number", "boolean", "integer". Not supported when type is "timestamp".

SmeMediaSource

The SmeMediaSource class defines a reference image that provides visual context for segment identification. Provide exactly one of the url, asset_id, or base_64_string fields.

NameTypeRequiredDescription
namestrYesA descriptive name for this media source.
media_typeSmeMediaSourceMediaTypeYesThe media type. Value: "image".
urlOptional[str]NoA publicly accessible HTTPS URL of the image.
asset_idOptional[str]NoThe unique identifier of an uploaded asset.
base_64_stringOptional[str]NoBase64-encoded image data. The maximum size is 30MB.

Return value: Returns a CreateAnalyzeTaskResponse object containing the task details.

The CreateAnalyzeTaskResponse class contains the following properties:

NameTypeDescription
task_idstrThe unique identifier of the analysis task.
statusAnalyzeTaskStatusThe initial status of the task. Value: queued.

API Reference: Create an async analysis task

Retrieve task status and results

Description: This method retrieves the status and results of an analysis task.

Task statuses:

  • queued: The task is waiting to be processed.
  • pending: The task is queued and waiting to start.
  • processing: The platform is analyzing the video.
  • ready: Processing is complete. Results are available in the response.
  • failed: The task failed. No results are available. The error field describes the failure.

Poll this method until status is ready or failed. When status is ready, use the results from the response.

Function signature and example:

def retrieve(
self,
task_id: str,
*,
request_options: typing.Optional[RequestOptions] = None,
) -> AnalyzeTaskResponse:

Parameters:

NameTypeRequiredDescription
task_idstrYesThe unique identifier of the analysis task.
request_optionsRequestOptionsNoRequest-specific configuration.

Return value: Returns an AnalyzeTaskResponse object containing the task status and results.

The AnalyzeTaskResponse class contains the following properties:

NameTypeDescription
task_idstrThe unique identifier of the analysis task.
video_sourceOptional[AnalyzeTaskResponseVideoSource]The video source you provided.
request_paramsOptional[AnalyzeTaskResponseRequestParams]The parameters you sent when creating this task.
statusAnalyzeTaskStatusThe current status of the task. Values: queued, pending, processing, ready, failed.
created_atdatetimeThe date and time when the task was created, in RFC 3339 format.
completed_atOptional[datetime]The date and time when the task completed or failed, in RFC 3339 format. The platform returns this field only when status is ready or failed.
resultOptional[AnalyzeTaskResult]An object that contains the generated text and additional information. The platform returns this object only when status is ready. When the task fails, the response contains no result object, so generation_id and usage are absent.
errorOptional[AnalyzeTaskError]A message attached to the task response. The platform sets this field in the following cases:
- status is "failed": The message field describes the failure reason. With video segmentation, a task can fail because the analysis reached the maximum response length or the context window before it could complete. The response contains no result object.
- status is "ready" and result.finish_reason is "length" (general analysis): The message field describes the truncation cause (either the maximum response length was reached or the context window was reached). The partial output is in result.data.
Not set when status is "ready" and result.finish_reason is "stop".
webhooksOptional[List[AnalyzeTaskWebhookInfo]]The delivery status of each webhook endpoint. The platform omits this field when there are no webhooks configured.

The AnalyzeTaskResponseVideoSource class contains the following properties:

NameTypeDescription
typeOptional[AnalyzeTaskResponseVideoSourceType]The type of video source. Values: "url", "base64_string", "asset_id".
urlOptional[str]The video URL. Present when type is "url".
asset_idOptional[str]The asset ID. Present when type is "asset_id".
system_metadataOptional[AnalyzeTaskResponseVideoSourceSystemMetadata]System-extracted video metadata. Present on a best-effort basis once the video has been processed.

The AnalyzeTaskResponseVideoSourceSystemMetadata class contains the following properties:

NameTypeDescription
durationOptional[float]The video duration in seconds.

The AnalyzeTaskResponseRequestParams class contains the following properties:

NameTypeDescription
analysis_modeOptional[AnalyzeTaskResponseRequestParamsAnalysisMode]The analysis approach for this task. Values: "general", "time_based_metadata".
promptOptional[str]The natural-language prompt for this task. Present only when analysis_mode is general and the task was created with prompt (not prompt_v_2). On the List method, truncated to the first 30 characters; on the Retrieve method, returns the full text.
prompt_v_2Optional[AnalyzeTaskResponseRequestParamsPromptV2]The structured prompt for this task. Present only when analysis_mode is general and the task was created with prompt_v_2. When present, the response excludes the flat prompt field. On the List method, input_text is truncated to the first 30 characters; on the Retrieve method, returns the full text.
response_formatOptional[AnalyzeTaskResponseRequestParamsResponseFormat]The response format for this task. Present only when the request included a response format.
temperatureOptional[float]The temperature value for this analysis.
max_tokensOptional[int]The maximum response length you set, in tokens.
min_segment_durationOptional[float]The minimum segment duration you set, in seconds. Present when analysis_mode is "time_based_metadata".
max_segment_durationOptional[float]The maximum segment duration you set, in seconds. Present when analysis_mode is "time_based_metadata".
start_timeOptional[float]The start of the analysis window, in seconds. Present only when the task was created with start_time.
end_timeOptional[float]The end of the analysis window, in seconds. Present only when the task was created with end_time.

The AnalyzeTaskResult class contains the following properties:

NameTypeDescription
generation_idstrThe unique identifier for the generation session.
datastrThe generated text for this analysis task. When analysis_mode is not set, a plain-text string based on the prompt you provided. When analysis_mode is "time_based_metadata", a JSON-encoded string keyed by segment definition id. Each key maps to an array of segment objects with start_time (number), end_time (number), and metadata (object with your custom fields).
finish_reasonFinishReasonThe reason the generation stopped. Values: null (generation has not finished), "stop" (the generation reached the end of the output text), "length" (the generation reached the maximum response length or the context window; with a JSON response format, the output may be truncated and fail to parse). When the task uses general analysis, the partial output is in data, and a warning is in the task’s error field. With video segmentation, if the analysis reaches either limit, the task fails and "length" never occurs.
usageAnalyzeTaskResultUsageThe number of tokens used in the generation.

The AnalyzeTaskResultUsage class contains the following properties:

NameTypeDescription
output_tokensintThe number of tokens in the generated text.
input_tokensOptional[int]The number of tokens the input consumed. Together with output_tokens, this value must fit within the context window.

The AnalyzeTaskError class contains the following properties:

NameTypeDescription
messagestrA human-readable message. The platform sets this field in the following cases: task failure (status is "failed") — describes the failure reason; truncation warning (general analysis, finish_reason is "length") — to obtain the full output, increase max_tokens or reduce the input size, and the partial output is in result.data. With general analysis, check finish_reason instead of parsing the message text; with video segmentation, check status. For the message strings, see Error codes.

API Reference: Retrieve analysis task status and results

Delete an analysis task

Description: This method deletes an analysis task. You can only delete tasks that are not currently being processed.

Function signature and example:

def delete(
self,
task_id: str,
*,
request_options: typing.Optional[RequestOptions] = None,
) -> None:

Parameters:

NameTypeRequiredDescription
task_idstrYesThe unique identifier of the analysis task.
request_optionsRequestOptionsNoRequest-specific configuration.

Return value: Returns None. If successful, the platform returns a 204 No Content response.

API Reference: Delete an analysis task