This quickstart guide provides a simplified introduction to creating embeddings using the TwelveLabs Video Understanding Platform. It includes the following:

A working example for each method: embed a query and embed content at scale
Minimal implementation details
Core parameters for common use cases

For comprehensive guides, see the Create embeddings section.

Key concepts

This section explains the key concepts and terminology used in this guide:

Asset: Your uploaded content. Once created, you can reference the same asset across multiple operations without uploading the file again.
Embedding: Vector representation of your content.
Embedding task: An asynchronous operation for processing your content and creating embeddings. Contains a status and the resulting embeddings when complete.

Workflow

The platform provides two methods to create embeddings. Choose the method that fits your use case:

Embed a query

Embed the text and media you search with. Returns one embedding in the response.

Embed content at scale

Embed the files you search through, one per request. Returns a task to poll.

Use these embeddings for similarity search, content classification, clustering, recommendations, or Retrieval-Augmented Generation (RAG).

Prerequisites

To use the platform, you need an API key:

1
If you don’t have an account, sign up for a free account.
2
Go to the API Keys page.
3
If you need to create a new key, select the Create API Key button. Enter a name and set the expiration period. The default is 12 months.
4
Select the Copy icon next to your key to copy it to your clipboard.
Depending on the programming language you are using, install the TwelveLabs SDK by entering one of the following commands:
```
pip install --upgrade twelvelabs
```
Your media files must meet the following requirements:
- Embed a query: Up to 10 media sources (images, video, or audio). Each media source can be up to 32 MB, and video and audio can be up to 30 seconds. Your text can be up to 2,000 tokens.
- Embed content at scale: Public video and audio URLs up to 4 GB, local video and audio files up to 200 MB, and images up to 32 MB. For local files up to 4 GB, see the Upload and processing methods page. For documents, local files up to 200 MB or public URLs up to 512 MB.
- Model capabilities: See the complete input requirements for Marengo 3.5.

Embed a query

Create a single embedding from your text. To create a combined embedding, add up to 10 image, video, or audio files to your request. The platform processes your request synchronously and returns the embedding in the response.

Starter code

Copy and paste the code below, replacing the placeholders surrounded by <> with your values.

from twelvelabs import (
    TwelveLabs,
    MultiInputRequest,
    # To combine your text with media, uncomment the next line:
    # MultiInputMediaSource,
)
# 1. Initialize the client
client = TwelveLabs(api_key="<YOUR_API_KEY>")
# 2. Create an embedding for your query
response = client.embed.v_2.create(
    input_type="multi_input",
    model_name="marengo3.5",
    multi_input=MultiInputRequest(
        input_text="<YOUR_TEXT>",
        # To combine your text with media, uncomment the following lines:
        # media_sources=[
        #     MultiInputMediaSource(
        #         media_type="image", # Or "video" or "audio"
        #         asset_id="<YOUR_ASSET_ID>", # Upload your file first to create a reusable asset
        #         # Or use url="<YOUR_MEDIA_URL>" for a direct link to a raw media file. Video hosting platforms and cloud storage sharing links are not supported
        #     ),
        # ],
    ),
)
# 3. Process the results
print(f"Number of embeddings: {len(response.data)}")
for embedding_data in response.data:
    print(f"Embedding dimensions: {len(embedding_data.embedding)}")
    print(f"First 10 values: {embedding_data.embedding[:10]}")

Code explanation

Import the SDK and initialize the client

Create a client instance to interact with the TwelveLabs Video Understanding Platform.

Create an embedding for your query

Create an embedding for your text. To combine your text with an image, video, or audio file, uncomment the media source lines.

Process the results

Process and display the embeddings. This example prints the embedding dimensions and first 10 values to the standard output.

Embed content at scale

Create embeddings for your media files: video, audio, images, and documents. The platform processes your files asynchronously, one file per request. Use this method for long files and large collections. This example embeds one video; repeat the request for each file. The request differs for each type of content.

Retention policy

Embeddings created with the asynchronous method are stored for seven days. After this, you must recreate them to obtain the results again.

Starter code

Copy and paste the code below, replacing the placeholders surrounded by <> with your values.

import time
from twelvelabs import TwelveLabs, AsyncVideoInputRequest, MediaSource
# 1. Initialize the client
client = TwelveLabs(api_key="<YOUR_API_KEY>")
# 2. Upload a video
asset = client.assets.create(
    method="url",
    url="<YOUR_VIDEO_URL>" # Use direct links to raw media files. Video hosting platforms and cloud storage sharing links are not supported
    # Or use method="direct" and file=open("<PATH_TO_VIDEO_FILE>", "rb") to upload a local file up to 200 MB
)
print(f"Created asset: id={asset.id}")
# 3. Check the status of the asset
print("Waiting for asset to be ready...")
while True:
    asset = client.assets.retrieve(asset.id)
    if asset.status == "ready":
        print("Asset is ready")
        break
    if asset.status == "failed":
        raise RuntimeError(f"Asset processing failed: id={asset.id}")
    time.sleep(5)
# 4. Create an embedding task
task = client.embed.v_2.tasks.create(
    input_type="video",
    model_name="marengo3.5",
    video=AsyncVideoInputRequest(
        media_source=MediaSource(
            asset_id=asset.id,
            # url="<YOUR_VIDEO_URL>", # Use direct links to raw media files. Video hosting platforms and cloud storage sharing links are not supported
            # base_64_string="<BASE_64_ENCODED_DATA>",
        ),
        embedding_option=["visual", "audio"],
    ),
)
print(f"Task ID: {task.id}")
# 5. Monitor the status
while True:
    task = client.embed.v_2.tasks.retrieve(task_id=task.id)
    if task.status == "ready":
        print("Task completed")
        break
    elif task.status == "failed":
        print("Task failed")
        break
    else:
        print("Task still processing...")
        time.sleep(5)
# 6. Process the results
print(f"Number of embeddings: {len(task.data)}")
for embedding_data in task.data:
    print(f"[{embedding_data.embedding_option}] {embedding_data.start_sec}s - {embedding_data.end_sec}s")
    print(f"Embedding dimensions: {len(embedding_data.embedding)}")
    print(f"First 10 values: {embedding_data.embedding[:10]}")

Code explanation

Import the SDK and initialize the client

Create a client instance to interact with the TwelveLabs Video Understanding Platform.

Upload a video

Upload a video to create an asset.

Check the status of the asset

Asset processing is asynchronous. Poll the status of the asset until it is ready before you use it.

Create an embedding task

Create an embedding task using the identifier of the asset.

Monitor the status

Poll the task until it reaches the ready state.

Process the results

Process and display the embeddings. This example prints the time range, embedding dimensions, and first 10 values for each segment to the standard output.