The TwelveLabs Node.js SDK provides methods to analyze videos to generate text from their content.

Titles, topics, and hashtags

Description: This method analyzes a specific video and generates titles, topics, and hashtags based on its content. It uses predefined formats and doesn’t require a custom prompt, and it’s best for generating immediate and straightforward text representations without specific customization.

Function signature and example:

1 async gist(
2   request: TwelvelabsApi.GistRequest,
3   requestOptions?: TwelvelabsApiClient.RequestOptions
4 ): core.HttpResponsePromise<TwelvelabsApi.Gist>

Parameters:

Name	Type	Required	Description
`request`	`TwelvelabsApi.GistRequest`	Yes	The request object containing the video ID and gist types.
`requestOptions`	`TwelvelabsApiClient.RequestOptions`	No	Request-specific configuration.

The GistRequest interface contains the following properties:

Name	Type	Required	Description
`videoId`	`string`	Yes	The unique identifier of the video for which you want to generate text.
`types`	`TwelvelabsApi.GistRequestTypesItem[]`	Yes	The types of text you want to generate. Available values: `"title"`, `"topic"`, `"hashtag"`.

Return value: Returns an HttpResponsePromise that resolves to a TwelvelabsApi.Gist instance.

The Gist interface contains the following properties:

Name	Type	Description
`id`	`string`	Unique identifier of the response.
`title`	`string`	Suggested title for the video.
`topics`	`string[]`	An array of topics that are relevant to the video.
`hashtags`	`string[]`	An array of hashtags that are relevant to the video.
`usage`	`TokenUsage`	The number of tokens used in the generation.

The TokenUsage interface contains the following properties:

Name	Type	Description
`outputTokens`	`number`	The number of tokens in the generated text.

API Reference: Titles, topics, and hashtags page.

Related guide: Titles, topics, and hashtags.

Summaries, chapters, and highlights

Description: This method analyzes a video and generates summaries, chapters, or highlights based on its content. Optionally, you can provide a prompt to customize the output.

Function signature and example:

1 summarize(
2   request: TwelvelabsApi.SummarizeRequest,
3   requestOptions?: TwelvelabsApiClient.RequestOptions
4 ): core.HttpResponsePromise<TwelvelabsApi.SummarizeResponse>;

Parameters:

Name	Type	Required	Description
`request`	`TwelvelabsApi.SummarizeRequest`	Yes	Specifies the type of text you wish to generate.
`request_options`	`TwelvelabsApiClient.RequestOptions`	No	Request-specific configuration.

The SummarizeRequest interface contains the following properties:

Name	Type	Required	Description
`videoId`	`string`	Yes	The unique identifier of the video that you want to summarize.
`type`	`string`	Yes	Specifies the type of text. Use `summary` for a brief report, `chapter` for a chronological list of chapters, or `highlight` for key moments.
`prompt`	`string`	No	Use this field to provide context for the summarization task, such as the target audience, style, tone of voice, and purpose. Maximum length is 2,000 tokens.
`temperature`	`number`	No	Controls the randomness of the text output. A higher value generates more creative text, while a lower value produces more deterministic text. Default: 0.2, Min: 0, Max: 1.
`responseFormat`	`TwelvelabsApi.ResponseFormat`	No	Specifies the format of the response. If you omit this parameter, the platform returns unstructured text. This parameter is only valid when type is set to summary.
`max_tokens`	`number`	No	The maximum number of tokens to generate. Min: 1. Max: 4096.

The ResponseFormat class contains the following properties:

Name	Type	Description
`type`	`string`	Set this parameter to “json_schema” to receive structured JSON responses.
`jsonSchema`	`Record<string, unknown>`	Contains the JSON schema that defines the response structure. The schema must adhere to the JSON Schema Draft 2020-12 specification. For details, see the `json_schema` parameter in the API Reference section.

Return value: Returns a Promise that resolves to a SummarizeResponse object containing the generated content. The response type varies based on the type parameter.

When type is "summary": Returns a SummarizeSummaryResult object with the following properties:

Name	Type	Description
`summarizeType`	`"summary"`	Indicates this is a summary response.
`id`	`string`	Unique identifier of the response.
`summary`	`string`	A brief report of the main points of the video.
`usage`	`TokenUsage`	The number of tokens used in the generation.

When type is "chapter": Returns a SummarizeChapterResult object with the following properties:

Name	Type	Description
`summarizeType`	`"chapter"`	Indicates this is a chapter response.
`id`	`string`	Unique identifier of the response.
`chapters`	`SummarizeChapterResultChaptersItem[]`	An array that contains details about the detected chapters and their content.
`usage`	`TokenUsage`	The number of tokens used in the generation.

When type is "highlight": Returns a SummarizeHighlightResult object with the following properties:

Name	Type	Description
`summarizeType`	`"highlight"`	Indicates this is a highlight response.
`id`	`string`	Unique identifier of the response.
`highlights`	`SummarizeHighlightResultHighlightsItem[]`	An array that contains the highlights.
`usage`	`TokenUsage`	The number of tokens used in the generation.

The SummarizeChapterResultChaptersItem interface contains the following properties:

Name	Type	Description
`chapterNumber`	`number`	The sequence number of the chapter. Note that this field starts at 0.
`startSec`	`number`	The starting time of the chapter, measured in seconds from the beginning of the video.
`endSec`	`number`	The ending time of the chapter, measured in seconds from the beginning of the video.
`chapterTitle`	`string`	The title of the chapter.
`chapterSummary`	`string`	A brief summary describing the content of the chapter.

The SummarizeHighlightResultHighlightsItem interface contains the following properties:

Name	Type	Description
`startSec`	`number`	The starting time of the highlight, measured in seconds from the beginning of the video.
`endSec`	`number`	The ending time of the highlight, measured in seconds from the beginning of the video.
`highlight`	`string`	The title of the highlight.
`highlightSummary`	`string`	A brief description that captures the essence of this part of the video.

The TokenUsage interface contains the following properties:

Name	Type	Description
`outputTokens`	`number`	The number of tokens in the generated text.

API Reference: Summaries, chapters, and highlights.

Related guide: Summaries, chapters, and highlights.

Open-ended analysis

Description: This method analyzes a video and generates text based on its content.

Function signature and example:

1 analyze(request: TwelvelabsApi.AnalyzeRequest, requestOptions?: TwelvelabsApiClient.RequestOptions): core.HttpResponsePromise<TwelvelabsApi.NonStreamAnalyzeResponse>;

Parameters:

Name	Type	Required	Description
`request`	`TwelvelabsApi.AnalyzeRequest`	Yes	The request object containing the parameters for performing an open-ended analysis.
`requestOptions`	`RequestOptions`	No	Request-specific configuration.

The AnalyzeRequest class contains the following properties:

Name	Type	Required	Description
`videoId`	`string`	Yes	The unique identifier of the video you wish to analyze and generate text for.
`prompt`	`string`	Yes	The prompt to customize the output. The maximum length is 2,000 tokens.
`temperature`	`number`	No	Controls the randomness of the text output. Higher values generate more creative text, while lower values produce more deterministic output. Default: 0.2, Min: 0, Max: 1.
`response_format`	`typing.Optional[ResponseFormat]`	No	Specifies the format of the response. If you omit this parameter, the platform returns unstructured text.
`max_tokens`	`typing.Optional[int]`	No	The maximum number of tokens to generate. Min: 1. Max: 4096.

Return value: Returns a Promise that resolves to a NonStreamAnalyzeResponse object containing the generated text.

The NonStreamAnalyzeResponse interface contains the following properties:

Name	Type	Description
`id`	`string`	Unique identifier of the response.
`data`	`string`	The generated text based on the prompt you provided.
`finishReason`	`TwelvelabsApi.FinishReason`	The reason for the generation to finish. Values: - `null`: The generation hasn’t finished yet. - `stop`: The generation stopped because the model reached the maximum number of tokens. For JSON responses, this may result in truncated, invalid JSON that fails to parse.
`usage`	`TokenUsage`	The number of tokens used in the generation.

The TokenUsage interface contains the following properties:

Name	Type	Description
`outputTokens`	`number`	The number of tokens in the generated text.

The maximum length of the response is 4,096 tokens.

API Reference: Open-ended analysis page.

Related guide: Open-ended analysis.

Open-ended analysis with streaming responses

Description: This method analyzes a video and generates text based on its content.

Function signature and example:

1 async analyzeStream(
2   request: TwelvelabsApi.AnalyzeStreamRequest,
3   requestOptions?: TwelvelabsApiClient.RequestOptions
4 ): core.HttpResponsePromise<core.Stream<TwelvelabsApi.StreamAnalyzeResponse>>

Parameters:

Name	Type	Required	Description
`request`	`TwelvelabsApi.AnalyzeStreamRequest`	Yes	The request object containing the parameters for performing an open-ended analysis.
`requestOptions`	[`RequestOptions`]	No	Request-specific configuration.

The AnalyzeStreamRequest class contains the following properties:

Name	Type	Required	Description
`videoId`	`string`	Yes	The unique identifier of the video you wish to analyze and generate text for.
`prompt`	`string`	Yes	The prompt to customize the output. The maximum length is 2,000 tokens.
`temperature`	`number`	No	Controls the randomness of the text output. Higher values generate more creative text, while lower values produce more deterministic output. Default: 0.2, Min: 0, Max: 1.
`responseFormat`	`TwelvelabsApi.ResponseFormat`	No	Specifies the format of the response. If you omit this parameter, the platform returns unstructured text. This parameter is only valid when type is set to summary.
`max_tokens`	`number`	No	The maximum number of tokens to generate. Min: 1. Max: 4096.

The ResponseFormat class contains the following properties:

Name	Type	Description
`type`	`string`	Set this parameter to “json_schema” to receive structured JSON responses.
`jsonSchema`	`Record<string, unknown>`	Contains the JSON schema that defines the response structure. The schema must adhere to the JSON Schema Draft 2020-12 specification. For details, see the `json_schema` parameter in the API Reference section.

Return value: Returns a promise that resolves to a Stream<TwelvelabsApi.StreamAnalyzeResponse> object that can be iterated over to receive streaming text chunks.

The StreamAnalyzeResponse can be either a StreamAnalyzeResponse.StreamStart, a StreamAnalyzeResponse.TextGeneration, or a StreamAnalyzeResponse.StreamEnd.

The StreamAnalyzeResponse.StreamStart interface contains the following properties:

Name	Type	Description
`eventType`	`string`	This field is always set to `stream_start` for this event.
`metadata`	`StreamStartResponseMetadata`	An object containing metadata about the stream.

The StreamAnalyzeResponse.TextGeneration interface contains the following properties:

Name	Type	Description
`eventType`	`string`	This field is always set to `text_generation` for this event.
`text`	`string`	A fragment of the generated text. Note that text fragments may be split at arbitrary points, not necessarily at word or sentence boundaries.

The StreamAnalyzeResponse.StreamEnd interface contains the following properties:

Name	Type	Description
`eventType`	`string`	This field is always set to `stream_end` for this event.
`metadata`	`StreamEndResponseMetadata`	An object containing metadata about the stream.
`finishReason`	`TwelvelabsApi.FinishReason`	The reason for the generation to finish. Values: - `null`: The generation hasn’t finished yet. - `stop`: The generation stopped because the model reached the maximum number of tokens. For JSON responses, this may result in truncated, invalid JSON that fails to parse.

The StreamStartResponseMetadata interface contains the following properties:

Name	Type	Description
`generationId`	`string`	A unique identifier for the generation session.

The StreamEndResponseMetadata interface contains the following properties:

Name	Type	Description
`generationId`	`string`	The same unique identifier provided in the `stream_start` event.
`usage`	`TokenUsage`	The number of tokens used in the generation.

The TokenUsage interface contains the following properties:

Name	Type	Description
`outputTokens`	`number`	The number of tokens in the generated text.

The maximum length of the response is 4,096 tokens.

API Reference: Open-ended analysis page.

Related guide: Open-ended analysis.

Error codes

This section lists the most common error messages you may encounter while analyzing videos.

token_limit_exceeded
- Your request could not be processed due to exceeding maximum token limit. Please try with another request or another video with shorter duration.
index_not_supported_for_generate
- You can only summarize videos uploaded to an index with an engine from the Pegasus family enabled.