To migrate to Pegasus 1.6, change the model name. Everything else in your requests can stay unchanged.

Pegasus 1.5 remains available. Requests without a model name use Pegasus 1.5.

Batch analysis

If you use batch analysis, keep Pegasus 1.5. Batch analysis does not support Pegasus 1.6.

Pegasus 1.6 adds egocentric video understanding, image analysis, improved entity recognition, improved metadata extraction, in-segment events, and segmentation without a token limit. For the full capability list, see the Pegasus 1.6 page.

Migration steps

Upgrade your SDK

Python

Node.js

Install the latest version of the Python SDK.

Python

pip install --upgrade twelvelabs

Update your analysis calls

Set model_name to "pegasus1.6".

Python

Node.js

cURL

Before:

Python

result = client.analyze(
    model_name="pegasus1.5",
    video=VideoContext_AssetId(asset_id="<YOUR_ASSET_ID>"),
    prompt_v_2=AnalyzePromptV2(
        input_text="<YOUR_PROMPT>",
    ),
)

After:

Python

result = client.analyze(
    model_name="pegasus1.6",
    video=VideoContext_AssetId(asset_id="<YOUR_ASSET_ID>"),
    prompt_v_2=AnalyzePromptV2(
        input_text="<YOUR_PROMPT>",
    ),
)

No other request changes are required.

Use new Pegasus 1.6 features

The following examples show three of the new capabilities: image analysis, in-segment events, and segmentation without a token limit.

Analyze images

Pegasus 1.6 analyzes up to 20 images per request.

Python

Node.js

cURL

Python

from twelvelabs.types import AnalyzeImageInput
images = [
    AnalyzeImageInput(name="before", url="<FIRST_IMAGE_URL>"),
    AnalyzeImageInput(name="after", url="<SECOND_IMAGE_URL>"),
]
result = client.analyze(
    model_name="pegasus1.6",
    image=images,
    prompt="Compare <@before> and <@after>. Describe the important differences.",
)

Note

The image parameter is mutually exclusive with the video and prompt_v2 parameters. The prompt parameter is required when you provide images.

For the complete instructions and examples, see the Analyze images guide.

Extract in-segment events

Use time_array to extract a list of timestamped events inside each segment. For example, segment a video into scenes and return the start and end time of each spoken line.

Python

Node.js

cURL

Python

from twelvelabs.types import AsyncResponseFormat, VideoContext_Url
task = client.analyze_async.tasks.create(
    model_name="pegasus1.6",
    video=VideoContext_Url(url="<YOUR_VIDEO_URL>"),
    analysis_mode="time_based_metadata",
    response_format=AsyncResponseFormat(
        type="segment_definitions",
        segment_time_format="hh:mm:ss",
        segment_definitions=[
            {
                "id": "scenes",
                "description": "Segment the video into distinct scenes; within each scene, identify each spoken line of dialogue",
                "fields": [
                    {"name": "label", "type": "string", "description": "A short label for this scene"},
                    {
                        "name": "dialogue",
                        "type": "time_array",
                        "description": "The spoken lines of dialogue in this scene",
                        "items": {
                            "type": "object",
                            "fields": [
                                {"name": "speaker", "type": "string", "description": "The name of the person speaking"},
                                {"name": "line", "type": "string", "description": "What the speaker says"},
                            ],
                        },
                    },
                ],
            },
        ],
    ),
)

For the complete instructions and examples, see the Segment videos guide.

Segment long videos without a token limit

Set max_tokens to unlimited to segment long videos without a token ceiling. The task completes even when the platform cannot extract every segment. The error field then states that the results may be incomplete.

Python

Node.js

cURL

Python

from twelvelabs.types import AsyncResponseFormat, VideoContext_Url
task = client.analyze_async.tasks.create(
    model_name="pegasus1.6",
    video=VideoContext_Url(url="<YOUR_VIDEO_URL>"),
    analysis_mode="time_based_metadata",
    max_tokens="unlimited",
    response_format=AsyncResponseFormat(
        type="segment_definitions",
        segment_time_format="hh:mm:ss",
        segment_definitions=[
            {
                "id": "scenes",
                "description": "Segment the video into distinct scenes; within each scene, identify each spoken line of dialogue",
                "fields": [
                    {"name": "label", "type": "string", "description": "A short label for this scene"},
                    {
                        "name": "dialogue",
                        "type": "time_array",
                        "description": "The spoken lines of dialogue in this scene",
                        "items": {
                            "type": "object",
                            "fields": [
                                {"name": "speaker", "type": "string", "description": "The name of the person speaking"},
                                {"name": "line", "type": "string", "description": "What the speaker says"},
                            ],
                        },
                    },
                ],
            },
        ],
    ),
)

Additional resources

Analyze videos and images

Segment videos

Pegasus 1.6

Python SDK Reference

Node.js SDK Reference

API Reference