Skip to main content

Streaming

Streaming shows progress and the answer as a turn runs. The API streams with Server-Sent Events: a long HTTP response of event: and data: lines, where each data line is JSON.

event: message.delta
data: {"superanalyst_id":"default","session_id":"…","message_id":"…","delta":"NVIDIA reported…"}

There are two ways to open a stream:

  • POST /sessions with Accept: text/event-stream creates a session and streams its first turn.
  • GET /sessions/{session_id}/events attaches to an existing session.

A stream stays open after a turn ends. A follow-up's turn streams on the connection you already hold, so keep one stream per session and close it when you are done.

Events​

EventPayloadWhat it means
session.createdThe session objectFirst event on POST /sessions. Read the new session's id from id.
session.resumedstatusFirst event on /events: the session's status when you attached.
progress.updatedlabelWhat SuperAnalyst is working on.
message.deltamessage_id, deltaAnswer text. Append each delta in order.
citation.addedmessage_id, citationA source. See Citations.
artifact.addedmessage_id, artifactA file the turn produced: id, file_id, name, mime_type.
turn.completedstatus, message_idFinal event for the turn. It finished.
turn.failedstatus, errorFinal event for the turn. It failed.
turn.timed_outstatus, errorFinal event for the turn. See Timeouts.
turn.cancelledstatusFinal event for the turn. It was cancelled.
  • Every payload except session.created includes superanalyst_id and session_id.
  • Each turn ends with exactly one turn.* event. The stream then stays open for the next turn.
  • message_id groups one turn's deltas, citations, and artifacts. Each turn gets a new one, and it changes if you reconnect mid-turn. It is not the id of a message in the transcript. turn.completed carries it only if this connection streamed output for the turn.
  • error is only on turn.failed and turn.timed_out.
  • An artifact's id works with the artifact and file endpoints. size_bytes and type are only on those endpoints.
  • The answer text is Markdown. SuperAnalyst currently sends it near the end of the turn, often as a single delta, after the progress.updated events. Its citation.added events come with it.
  • A citation's text_ranges count from the start of the turn's deltas joined in order. A citation's id differs from the same citation's in the transcript.
  • Lines that start with : are heartbeats, about every 10 seconds. Skip them.
  • Ignore event names you don't recognize. New ones may be added.

A short stream looks like this, with payloads trimmed:

event: session.created
data: {"id": "…", "object": "session", "status": "queued"}

event: progress.updated
data: {"superanalyst_id": "default", "session_id": "…", "label": "Searching earnings call transcripts"}

: heartbeat

event: message.delta
data: {"superanalyst_id": "default", "session_id": "…", "message_id": "…", "delta": "NVIDIA's call centered on three themes…"}

event: turn.completed
data: {"superanalyst_id": "default", "session_id": "…", "status": "completed", "message_id": "…"}

: heartbeat

Errors before a stream opens, such as 401 or 503, come back as a normal JSON error. Check the status code before you parse events.

/events opens the stream before it sends session.resumed, so the first event can be slow. If it can't open the stream, it returns 502, 503, or 504 instead. After a 504, the turn may still be running: check the session or attach again, but don't send the question again.

Parse the stream​

Any SSE library works. These parsers yield one event at a time and collect a turn's answer.

import json

FINAL_EVENTS = {"turn.completed", "turn.failed", "turn.timed_out", "turn.cancelled"}


def iter_events(response):
"""Yield (event, data) pairs from a streamed requests response."""
event, data = "message", []
for line in response.iter_lines(chunk_size=None):
line = line.decode("utf-8") # decode per line: answers can contain U+2028
if line == "":
if data:
yield event, json.loads("\n".join(data))
event, data = "message", []
elif line.startswith("event:"):
event = line[6:].strip()
elif line.startswith("data:"):
data.append(line[5:].lstrip())


def collect_turn(events):
"""Return (final_event, answer, citations, artifacts) for the next turn on the stream."""
answer, citations, artifacts = "", [], []
for event, data in events:
if event == "message.delta":
answer += data["delta"]
elif event == "citation.added":
citations.append(data["citation"])
elif event == "artifact.added":
artifacts.append(data["artifact"])
elif event in FINAL_EVENTS:
return event, answer, citations, artifacts
raise ConnectionError("The stream closed before the turn ended.")

Heartbeats start with :, so neither parser treats them as events. Set your HTTP client's read timeout for streams to at least the request limit in Timeouts and limits, or turn it off.

Stream a new session​

question = {"input": {"text": "What were the key themes in NVIDIA's latest earnings call?"}}
with api.post(
f"{API_URL}/sessions", json=question, headers={"Accept": "text/event-stream"}, stream=True
) as response:
events = iter_events(response)
_, session = next(events, (None, None)) # session.created
if session is None:
raise ConnectionError("The stream closed before session.created.")
final_event, answer, citations, artifacts = collect_turn(events)

print(session["id"], final_event)
print(answer)

Leaving the with block closes the stream.

Stream a follow-up​

Attach to the session's stream first, then send the follow-up. Its turn streams on the connection you already hold, so no event is missed. Send it while the session is idle: a follow-up sent while a turn is running returns 409 SESSION_BUSY.

Rarely, a turn that ended just as you attached still sends its final event after session.resumed. So after a final event, check the session: if it is still queued or in_progress, that event was the earlier turn's, and the follow-up is still running.

def stream_followup(session_url, text):
"""Ask a follow-up and collect its turn."""
with api.get(
f"{session_url}/events", headers={"Accept": "text/event-stream"}, stream=True
) as response:
events = iter_events(response)
_, opening = next(events, (None, None)) # session.resumed
if opening is None:
raise ConnectionError("The stream closed before session.resumed.")
api.post(f"{session_url}/messages", json={"input": {"text": text}})
while True:
turn = collect_turn(events)
# The earlier turn's final event can come first. The follow-up has ended once the session is idle.
if api.get(session_url).json()["status"] not in ("queued", "in_progress"):
return turn


final_event, answer, citations, artifacts = stream_followup(
session_url, "Which of those themes did analysts push back on?"
)
print(final_event)
print(answer)

To ask several follow-ups, keep one stream open and call collect_turn / collectTurn again after each one.

Reconnecting​

A stream can close before the turn ends, with no final event:

  • when the service is updated, which can happen at any time,
  • if your client stops reading for about 10 seconds.

Closing a stream does not cancel the turn. It keeps running and still uses credits. Only cancel stops it. To keep watching, attach again with /events and read the status in session.resumed:

  • queued or in_progress: the turn is still running, and the rest of it streams on the new connection. Earlier events are not replayed, so read the full answer from the transcript afterwards.
  • Anything else: the turn has ended. Read the answer from the transcript. Rarely, that turn's last events still arrive after session.resumed. Don't take them as a new turn: check the session for the outcome and read the answer from the transcript.

Never send the question again to recover from a dropped stream. That starts a second turn.

import time

DROPPED = (ConnectionError, requests.exceptions.ConnectionError,
requests.exceptions.ChunkedEncodingError)


def retry_after(response):
"""Seconds to wait before retrying a failed read, or None if retrying can't help."""
is_json = response.headers.get("Content-Type", "").startswith("application/json")
error = response.json()["error"] if is_json else {}
if response.status_code not in (502, 503, 504) or error.get("retryable") is False:
return None
wait = response.headers.get("Retry-After", "")
return int(wait) if wait.isdigit() else 5


def wait_for_turn_to_end(session_url):
"""Reattach until the current turn ends, then return the newest message."""
while True:
try:
with api.get(
f"{session_url}/events", headers={"Accept": "text/event-stream"}, stream=True
) as response:
events = iter_events(response)
_, opening = next(events, (None, None)) # session.resumed
if opening is None:
raise ConnectionError("The stream closed before session.resumed.")
if opening["status"] in ("queued", "in_progress"):
collect_turn(events)
# A stream that hit its time limit ends with turn.timed_out while the turn still runs.
if api.get(session_url).json()["status"] not in ("queued", "in_progress"):
return api.get(f"{session_url}/messages", params={"limit": 1}).json()["data"][0]
except DROPPED:
time.sleep(1)
except requests.HTTPError as error:
wait = retry_after(error.response)
if wait is None:
raise
time.sleep(wait)

Timeouts​

A stream has a time limit (see Timeouts and limits). If a turn is still running then, the stream sends turn.timed_out and closes, but the turn keeps running. Check error.retryable:

  • false: the stream stopped watching. The turn may still finish. Attach again or check the session, then read the transcript. Don't send the question again.
  • true: the turn itself timed out, and the session's own status is timed_out. Ask again as a follow-up if you want an answer.