This guide shows you how to generate a response from a knowledge store and customize Jockey’s behavior for a specific domain. You can also use this endpoint for agentic search: have Jockey reason over your content to find and explain matches. For direct, single-call retrieval of ranked clips and images instead, use Search a knowledge store.

Key features:

• Corpus-level reasoning: Answers questions by reasoning across every item in a knowledge store.
• Domain-specific behavior: Adapts responses to your domain or task through per-request instructions.
• Multi-turn sessions: Maintains conversation context across follow-up requests.
• Streaming: Returns tokens in real time instead of waiting for the full response.
• Structured output: Returns typed JSON responses when you provide a JSON Schema.
• Intermediate outputs: Returns Jockey’s reasoning steps alongside the final answer.

Use cases:

• Corpus overview: Generate responses that reason across all your videos and images.
• Entity extraction: Identify people, places, objects, and other entities across a knowledge store.
• Search and discovery: Find moments in videos and matching images by describing what you want in natural language.
• Content organization: Categorize videos and images by themes, topics, formats, or other useful dimensions.
• Cross-video tracking: Track a subject across multiple videos and reconstruct a timeline of appearances.
• Content enrichment: Analyze existing content through a specific domain lens or schema.

Key concepts

• Knowledge store: A persistent store of your videos and images plus the understanding the platform derives from them — spatiotemporal context, a typed ontology, and embeddings — that together enable corpus-level reasoning.
• Instructions: A system-level prompt that shapes Jockey’s behavior for a specific domain or task. Same endpoint, different results depending on the instructions you provide.

Prerequisites

• You’ve already uploaded your content, and the asset has reached the ready status. See the Upload content page for details.
• You’ve already created a knowledge store. See the Create a knowledge store page for details.
• You’ve already added at least one asset to a knowledge store, and the knowledge store item has reached the ready status. See the Add assets to a knowledge store page for details.

Example

Copy and paste the code below, replacing the placeholders surrounded by <> with your values.

1
from twelvelabs import TwelveLabs
2
3
client = TwelveLabs(api_key="<YOUR_API_KEY>")
4
5
# Create a response
6
response = client.responses.create(
7
    knowledge_store_id="<YOUR_KNOWLEDGE_STORE_ID>",
8
    input=[{"type": "message", "role": "user", "content": "What are the main themes across these videos and images?"}],
9
    # session_id=...,  # Optional. Continue a multi-turn conversation. See Multi-turn sessions.
10
    # instructions=...,  # Optional. Add a per-request system prompt. See Customize behavior with instructions.
11
    # include=...,  # Optional. Return intermediate reasoning steps. See Inspect intermediate outputs.
12
    # selections=...,  # Optional. Restrict the request to specific items or item collections.
13
    # text=...,  # Optional. Return typed JSON output. See Structured output.
14
)
15
16
# Read the response
17
print(f"ID: {response.id}")
18
print(f"Session: {response.session_id}")
19
20
for output in response.output:
21
    if output.type == "message":
22
        for content in output.content:
23
            print(content.text)
24
25
print(f"Tokens: {response.usage}")

Code explanation

Example response

The response object contains the generated text along with the session identifier and token usage:

1
{
2
  "id": "resp_abc123",
3
  "session_id": "sess_xyz789",
4
  "status": "completed",
5
  "output": [
6
    {
7
      "type": "message",
8
      "content": [{"type": "output_text", "text": "The main themes across your videos and images include..."}]
9
    }
10
  ],
11
  "usage": {"input_tokens": 1250, "output_tokens": 340}
12
}

Customize behavior with instructions

Add the instructions field to specialize Jockey for your domain. Same endpoint, different results depending on what you provide.

1
response = client.responses.create(
2
    knowledge_store_id="<YOUR_KNOWLEDGE_STORE_ID>",
3
    instructions="You are a sports analyst. Focus on player performance, tactics, and key moments.",
4
    input=[{"type": "message", "role": "user", "content": "Summarize the key plays from this game"}],
5
)

Swap the instructions field to reshape Jockey for different contexts:

Inspect intermediate outputs

Jockey performs multi-step reasoning internally: searching, analyzing, and synthesizing across your videos and images. Add "intermediate_outputs" to the include array to see these reasoning steps alongside the final answer. This is useful for debugging or understanding how Jockey produced the answer.

1
response = client.responses.create(
2
    knowledge_store_id="<YOUR_KNOWLEDGE_STORE_ID>",
3
    input=[{"type": "message", "role": "user", "content": "What are the main themes?"}],
4
    include=["intermediate_outputs"],
5
)

Capabilities

The Responses API handles many tasks through the same endpoint. These examples are not exhaustive.

Common pitfalls

• Index at least one item before you query. A knowledge store with no items in the ready state has no content for the request to draw on.
• Include the knowledge store identifier. Every request must include the knowledge_store_id field.

Next steps

• Streaming - receive tokens in real time instead of waiting for the full response
• Structured output - receive typed JSON by providing a schema
• Multi-turn sessions - maintain conversation context across requests

Jupyter notebook