Organize a video library

This recipe shows you how to organize an entire video library in a multi-turn session. The final output is a complete library catalog as structured JSON. Every video is assigned to a group, ready for your CMS or DAM system.

Use cases:

  • Library management: Categorize a new or growing video collection
  • Content audit: Discover natural groupings before restructuring a library
  • Pipeline automation: Generate a machine-readable catalog for a CMS or DAM system

Key concepts

  • Knowledge store: A persistent store of your videos and images plus the understanding the platform derives from them — spatiotemporal context, a typed ontology, and embeddings — that together enable corpus-level reasoning.
  • Multi-turn session: A conversation where each request builds on previous turns. The first request creates a session, and follow-up requests reference that session to continue the conversation. For details, see the Multi-turn sessions page.
  • Structured output: A JSON schema you provide with your request. Jockey constrains its output to match your schema, so you retrieve machine-readable results instead of plain text. For a guide on designing schemas and parsing responses, see the Structured output page.

Workflow

This recipe uses a multi-turn session in which each call builds on the previous ones. The first call requests a plain-text overview of the collection. The second call passes a JSON schema and asks Jockey to discover and rank categorization strategies. The third call takes the top-ranked strategy and categorizes every video. Jockey refines its analysis as the session progresses. You can adapt this recipe to organize by different criteria or target a specific domain.

Prerequisites

  • You’ve already created a knowledge store with at least one item in ready status. See the Quickstart page for details.
  • You’re familiar with the request and response format. See the Create a response page for details.
  • For best results, use a knowledge store with 10 or more videos.

Complete example

Copy and paste the code below, replacing the placeholders surrounded by <> with your values.

1import json
2from twelvelabs import TwelveLabs, TextParam
3from twelvelabs.types.text_param_format import TextParamFormat_JsonSchema
4
5client = TwelveLabs(api_key="<YOUR_API_KEY>")
6STORE_ID = "<YOUR_KNOWLEDGE_STORE_ID>"
7
8# Request a collection overview
9print("Understanding your collection...")
10response = client.responses.create(
11 knowledge_store_id=STORE_ID,
12 input=[{"type": "message", "role": "user", "content": "Give me a high-level overview of this video collection. What themes, subjects, and patterns do you see?"}],
13)
14
15# Read the overview and store the session ID
16session_id = response.session_id
17for output in response.output:
18 if output.type == "message":
19 for content in output.content:
20 print(content.text)
21
22# Define the organization axes schema
23axes_schema = {
24 "type": "object",
25 "properties": {
26 "recommended_axes": {
27 "type": "array",
28 "items": {
29 "type": "object",
30 "properties": {
31 "axis": {"type": "string"}, # Name of the categorization dimension
32 "reason": {"type": "string"}, # Why this axis works well
33 "expected_categories": { # Sample category names
34 "type": "array",
35 "items": {"type": "string"},
36 },
37 "score": {"type": "number"}, # 0-1 effectiveness score
38 },
39 },
40 },
41 "best_axis": {"type": "string"}, # Top-ranked axis name
42 "reasoning": {"type": "string"}, # Overall recommendation rationale
43 },
44}
45
46# Discover organization strategies
47print("\nFinding the best organization strategy...")
48response = client.responses.create(
49 knowledge_store_id=STORE_ID,
50 session_id=session_id,
51 input=[{"type": "message", "role": "user", "content": "What are the best ways to organize this collection? Rank the top 3 axes by how well they'd separate the content into useful groups."}],
52 text=TextParam(format=TextParamFormat_JsonSchema(name="organization_axes", schema_=axes_schema)),
53)
54
55# Parse the organization strategies
56for output in response.output:
57 if output.type == "message":
58 for content in output.content:
59 axes = json.loads(content.text)
60 print(f"Best axis: {axes['best_axis']}")
61 print(f"Reasoning: {axes['reasoning']}")
62 for ax in axes["recommended_axes"]:
63 print(f" {ax['axis']} (score: {ax['score']}): {ax['reason']}")
64
65# Define the catalog schema
66catalog_schema = {
67 "type": "object",
68 "properties": {
69 "organization_axis": {"type": "string"}, # The axis used for categorization
70 "categories": {
71 "type": "array",
72 "items": {
73 "type": "object",
74 "properties": {
75 "name": {"type": "string"}, # Category name
76 "description": {"type": "string"}, # What this category contains
77 "videos": {
78 "type": "array",
79 "items": {
80 "type": "object",
81 "properties": {
82 "reference": {"type": "string"}, # Source video identifier
83 "title": {"type": "string"}, # Video title
84 "summary": {"type": "string"}, # Brief description
85 },
86 },
87 },
88 },
89 },
90 },
91 "total_videos": {"type": "integer"}, # Total number of videos categorized
92 },
93}
94
95# Categorize videos
96print(f"\nOrganizing by '{axes['best_axis']}'...")
97response = client.responses.create(
98 knowledge_store_id=STORE_ID,
99 session_id=session_id,
100 input=[{"type": "message", "role": "user", "content": f"Now organize every video by '{axes['best_axis']}'. Include all videos."}],
101 text=TextParam(format=TextParamFormat_JsonSchema(name="video_catalog", schema_=catalog_schema)),
102)
103
104# Parse the catalog
105for output in response.output:
106 if output.type == "message":
107 for content in output.content:
108 catalog = json.loads(content.text)
109 print(f"\nLibrary Catalog ({catalog['total_videos']} videos)")
110 print(f"Organized by: {catalog['organization_axis']}")
111 for cat in catalog["categories"]:
112 print(f"\n [{cat['name']}] - {cat['description']}")
113 for v in cat["videos"]:
114 print(f" {v['title']}: {v['summary']}")

Code explanation

1

Request a collection overview

Ask for a high-level overview. This first request creates the session that the later turns build on.
Function call: You call the responses.create method with a prompt that requests an overview.
Parameters:

  • knowledge_store_id: The unique identifier of the knowledge store to reason over.
  • input: An array of input items. Each item is a message you send to Jockey. This example requests a high-level overview of themes, subjects, and patterns.

Return value: An object of type ResponseObject. Read the overview text from the output items, and keep the session_id for the follow-up turns. Jockey retains this overview as session context, so later turns build on it. The response also includes id, status, and usage.

2

Discover organization strategies

Continue the session with a prompt that ranks categorization strategies. Jockey uses the collection context from the previous turn.
Function call: You call the responses.create method with the session_id and a text parameter that describes your axes schema.
Parameters:

  • knowledge_store_id: The unique identifier of the knowledge store. Must match the store used to create the session.
  • session_id: The session identifier from the first response.
  • input: An array of input items. Each item is a message you send to Jockey. This example asks Jockey to rank the top three categorization strategies by effectiveness.
  • text: An object that specifies the response format. To return structured JSON, set the format field: its schema_ field defines the structure the output must match, and its name field identifies the schema. Design the schema to fit your use case; the inline comments in the example describe each field. For the schema rules, see the Structured output page.

Return value: An object of type ResponseObject. The text field of each content part in a message item is a JSON string that matches your axes schema.

3

Process the strategies

Parse the text field of each content part in a message item with json.loads(), then iterate the recommended_axes array to read each ranked strategy. Keep the best_axis value; the next step uses it to categorize the videos.

4

Categorize videos

Continue the session with a prompt that organizes every video by the top-ranked axis.
Function call: You call the responses.create method with the session_id and a text parameter that describes your catalog schema.
Parameters:

  • knowledge_store_id: The unique identifier of the knowledge store. Must match the store used to create the session.
  • session_id: The session identifier from the first response.
  • input: An array of input items. Each item is a message you send to Jockey. This example inserts the best_axis value from the previous step into the prompt, so Jockey organizes every video by that axis.
  • text: The catalog schema, in the same form as the previous step.

Return value: An object of type ResponseObject. The text field of each content part in a message item is a JSON string that matches your catalog schema.

5

Process the catalog

Parse the text field of each content part in a message item with json.loads(), then iterate the categories array, and the videos within each category, to read the final library catalog.

Example response

The text field inside each content part is a JSON string that matches your schema. After parsing, a typical result from the final categorization step looks like this:

1{
2 "organization_axis": "Content Type",
3 "categories": [
4 {
5 "name": "Tutorials",
6 "description": "Step-by-step instructional videos covering product features",
7 "videos": [
8 {
9 "reference": "069eb4e8-aeb0-7e83-8000-86413fcc296a",
10 "title": "Getting Started with the Dashboard",
11 "summary": "Walks through initial setup, navigation, and key workflows"
12 },
13 {
14 "reference": "069e1e97-27f8-7a8e-8000-bf32ecd7fc8c",
15 "title": "API Integration Tutorial",
16 "summary": "Covers authentication, first API call, and error handling"
17 }
18 ]
19 },
20 {
21 "name": "Product Demos",
22 "description": "Feature demonstrations and product showcases",
23 "videos": [
24 {
25 "reference": "069ea183-ed43-7d69-8000-e2e3ab3ea1d1",
26 "title": "Analytics Dashboard Demo",
27 "summary": "Live walkthrough of the analytics dashboard and reporting tools"
28 }
29 ]
30 }
31 ],
32 "total_videos": 12
33}
Notes
  • The reference field in each video entry contains the asset identifier of the knowledge store item.
  • Jockey generates the title field from the content of the video, not from a filename or metadata field.
  • Jockey evaluates the actual content in your knowledge store. Results change depending on the videos you have indexed.

Variations

Change the prompts or add the instructions parameter to adapt this recipe for different organizational needs.

  • Multi-axis catalog: Run step 7 multiple times with different axes from step 5 to produce catalogs along different dimensions.

  • Hierarchical organization: After the first categorization, send a follow-up prompt: “Now sub-organize the [category] group by difficulty level.”

  • Export: Pipe the JSON output to a file for use in your CMS or DAM system.

  • Known criteria: Skip the discovery steps and send a single request. Set the instructions parameter to a librarian role (for example, “You are a content librarian. Organize videos into clear, mutually exclusive categories.”) and specify the categories directly in the prompt.

    Example:

    CriteriaPrompt
    By topic”Organize by main subject matter”
    By mood”Organize by emotional tone: upbeat, serious, neutral”
    By audience”Organize by target audience: beginners, intermediate, advanced”
    By format”Organize by format: tutorial, interview, demo, vlog”
    By speaker”Group videos by who appears on screen”

Jupyter notebook

Open In Colab

See also