Streaming
Streaming shows progress and the answer as a turn runs. The API streams with
Server-Sent Events: a long HTTP
response of event: and data: lines, where each data line is JSON.
event: message.delta
data: {"superanalyst_id":"default","session_id":"…","message_id":"…","delta":"NVIDIA reported…"}
There are two ways to open a stream:
POST /sessionswithAccept: text/event-streamcreates a session and streams its first turn.GET /sessions/{session_id}/eventsattaches to an existing session.
A stream stays open after a turn ends. A follow-up's turn streams on the connection you already hold, so keep one stream per session and close it when you are done.
Events
| Event | Payload | What it means |
|---|---|---|
session.created | The session object | First event on POST /sessions. Read the new session's id from id. |
session.resumed | status | First event on /events: the session's status when you attached. |
progress.updated | label | What SuperAnalyst is working on. |
message.delta | message_id, delta | Answer text. Append each delta in order. |
citation.added | message_id, citation | A source. See Citations. |
artifact.added | message_id, artifact | A file the turn produced: id, file_id, name, mime_type. |
turn.completed | status, message_id | Final event for the turn. It finished. |
turn.failed | status, error | Final event for the turn. It failed. |
turn.timed_out | status, error | Final event for the turn. See Timeouts. |
turn.cancelled | status | Final event for the turn. It was cancelled. |
- Every payload except
session.createdincludessuperanalyst_idandsession_id. - Each turn ends with exactly one
turn.*event. The stream then stays open for the next turn. message_idgroups one turn's deltas, citations, and artifacts. Each turn gets a new one, and it changes if you reconnect mid-turn. It is not the id of a message in the transcript.turn.completedcarries it only if this connection streamed output for the turn.erroris only onturn.failedandturn.timed_out.- An artifact's
idworks with the artifact and file endpoints.size_bytesandtypeare only on those endpoints. - The answer text is Markdown. SuperAnalyst currently sends it near the end of the turn, often as a
single delta, after the
progress.updatedevents. Itscitation.addedevents come with it. - A citation's
text_rangescount from the start of the turn's deltas joined in order. A citation'siddiffers from the same citation's in the transcript. - Lines that start with
:are heartbeats, about every 10 seconds. Skip them. - Ignore event names you don't recognize. New ones may be added.
A short stream looks like this, with payloads trimmed:
event: session.created
data: {"id": "…", "object": "session", "status": "queued"}
event: progress.updated
data: {"superanalyst_id": "default", "session_id": "…", "label": "Searching earnings call transcripts"}
: heartbeat
event: message.delta
data: {"superanalyst_id": "default", "session_id": "…", "message_id": "…", "delta": "NVIDIA's call centered on three themes…"}
event: turn.completed
data: {"superanalyst_id": "default", "session_id": "…", "status": "completed", "message_id": "…"}
: heartbeat
Errors before a stream opens, such as 401 or 503, come back as a normal JSON error. Check the
status code before you parse events.
/events opens the stream before it sends session.resumed, so the first event can be slow. If it
can't open the stream, it returns 502, 503, or 504 instead. After a 504, the turn may still
be running: check the session or attach again, but don't send the question again.
Parse the stream
Any SSE library works. These parsers yield one event at a time and collect a turn's answer.
- Python
- JavaScript
import json
FINAL_EVENTS = {"turn.completed", "turn.failed", "turn.timed_out", "turn.cancelled"}
def iter_events(response):
"""Yield (event, data) pairs from a streamed requests response."""
event, data = "message", []
for line in response.iter_lines(chunk_size=None):
line = line.decode("utf-8") # decode per line: answers can contain U+2028
if line == "":
if data:
yield event, json.loads("\n".join(data))
event, data = "message", []
elif line.startswith("event:"):
event = line[6:].strip()
elif line.startswith("data:"):
data.append(line[5:].lstrip())
def collect_turn(events):
"""Return (final_event, answer, citations, artifacts) for the next turn on the stream."""
answer, citations, artifacts = "", [], []
for event, data in events:
if event == "message.delta":
answer += data["delta"]
elif event == "citation.added":
citations.append(data["citation"])
elif event == "artifact.added":
artifacts.append(data["artifact"])
elif event in FINAL_EVENTS:
return event, answer, citations, artifacts
raise ConnectionError("The stream closed before the turn ended.")
const FINAL_EVENTS = ['turn.completed', 'turn.failed', 'turn.timed_out', 'turn.cancelled']
// Yield {event, data} objects from a streamed fetch response.
async function* iterEvents(response) {
const decoder = new TextDecoder()
let buffer = ''
let event = 'message'
let data = []
for await (const chunk of response.body) {
buffer += decoder.decode(chunk, {stream: true})
const lines = buffer.split('\n')
buffer = lines.pop()
for (const rawLine of lines) {
const line = rawLine.replace(/\r$/, '')
if (line === '') {
if (data.length) yield {event, data: JSON.parse(data.join('\n'))}
event = 'message'
data = []
} else if (line.startsWith('event:')) {
event = line.slice(6).trim()
} else if (line.startsWith('data:')) {
data.push(line.slice(5).trimStart())
}
}
}
}
// Return {event, answer, citations, artifacts} for the next turn on the stream.
async function collectTurn(events) {
let answer = ''
const citations = []
const artifacts = []
while (true) {
const {value, done} = await events.next()
if (done) throw new Error('The stream closed before the turn ended.')
const {event, data} = value
if (event === 'message.delta') answer += data.delta
else if (event === 'citation.added') citations.push(data.citation)
else if (event === 'artifact.added') artifacts.push(data.artifact)
else if (FINAL_EVENTS.includes(event)) return {event, answer, citations, artifacts}
}
}
collectTurn reads with events.next() rather than for await, so the stream stays open after the
turn for the next one. Close it with await events.return().
Heartbeats start with :, so neither parser treats them as events. Set your HTTP client's read
timeout for streams to at least the request limit in
Timeouts and limits, or turn it off.
Stream a new session
- Python
- JavaScript
- cURL
question = {"input": {"text": "What were the key themes in NVIDIA's latest earnings call?"}}
with api.post(
f"{API_URL}/sessions", json=question, headers={"Accept": "text/event-stream"}, stream=True
) as response:
events = iter_events(response)
_, session = next(events, (None, None)) # session.created
if session is None:
raise ConnectionError("The stream closed before session.created.")
final_event, answer, citations, artifacts = collect_turn(events)
print(session["id"], final_event)
print(answer)
Leaving the with block closes the stream.
const response = await fetch(`${API_URL}/sessions`, {
method: 'POST',
headers: {...authHeader, 'Content-Type': 'application/json', Accept: 'text/event-stream'},
body: JSON.stringify({
input: {text: "What were the key themes in NVIDIA's latest earnings call?"},
}),
})
if (!response.ok) throw new Error(`${response.status} ${await response.text()}`)
const events = iterEvents(response)
const {value: created} = await events.next() // session.created
if (!created) throw new Error('The stream closed before session.created.')
const {event, answer} = await collectTurn(events)
await events.return() // close the stream
console.log(created.data.id, event)
console.log(answer)
curl -sS -N -X POST "$API_URL/sessions" \
-H "Authorization: Bearer $ACCESS_TOKEN" \
-H "Content-Type: application/json" \
-H "Accept: text/event-stream" \
-d '{"input": {"text": "What were the key themes in NVIDIA'\''s latest earnings call?"}}' \
| awk '{ print } /^event: turn\./ { getline; print; exit }'
-N prints events as they arrive. The stream stays open after the turn, so awk stops at its final
event. curl then reports (23) Failure writing output, which only means awk stopped reading.
Stream a follow-up
Attach to the session's stream first, then send the follow-up. Its turn streams on the connection
you already hold, so no event is missed. Send it while the session is idle: a follow-up sent while a
turn is running returns 409 SESSION_BUSY.
Rarely, a turn that ended just as you attached still sends its final event after session.resumed.
So after a final event, check the session: if it is still queued or in_progress, that event was
the earlier turn's, and the follow-up is still running.
- Python
- JavaScript
def stream_followup(session_url, text):
"""Ask a follow-up and collect its turn."""
with api.get(
f"{session_url}/events", headers={"Accept": "text/event-stream"}, stream=True
) as response:
events = iter_events(response)
_, opening = next(events, (None, None)) # session.resumed
if opening is None:
raise ConnectionError("The stream closed before session.resumed.")
api.post(f"{session_url}/messages", json={"input": {"text": text}})
while True:
turn = collect_turn(events)
# The earlier turn's final event can come first. The follow-up has ended once the session is idle.
if api.get(session_url).json()["status"] not in ("queued", "in_progress"):
return turn
final_event, answer, citations, artifacts = stream_followup(
session_url, "Which of those themes did analysts push back on?"
)
print(final_event)
print(answer)
// Ask a follow-up and collect its turn.
async function streamFollowup(sessionPath, text) {
const response = await fetch(`${API_URL}${sessionPath}/events`, {
headers: {...authHeader, Accept: 'text/event-stream'},
})
if (!response.ok) throw new Error(`${response.status} ${await response.text()}`)
const events = iterEvents(response)
try {
const {done} = await events.next() // session.resumed
if (done) throw new Error('The stream closed before the turn ended.')
await api(`${sessionPath}/messages`, {method: 'POST', body: {input: {text}}})
while (true) {
const turn = await collectTurn(events)
// The earlier turn's final event can come first. The follow-up has ended once the session is idle.
const {status} = await api(sessionPath)
if (!['queued', 'in_progress'].includes(status)) return turn
}
} finally {
await events.return() // close the stream
}
}
const followup = await streamFollowup(
sessionPath,
'Which of those themes did analysts push back on?',
)
console.log(followup.event)
console.log(followup.answer)
To ask several follow-ups, keep one stream open and call collect_turn / collectTurn again after
each one.
Reconnecting
A stream can close before the turn ends, with no final event:
- when the service is updated, which can happen at any time,
- if your client stops reading for about 10 seconds.
Closing a stream does not cancel the turn. It keeps running and still uses credits. Only
cancel stops it. To keep watching, attach again with
/events and read the status in session.resumed:
queuedorin_progress: the turn is still running, and the rest of it streams on the new connection. Earlier events are not replayed, so read the full answer from the transcript afterwards.- Anything else: the turn has ended. Read the answer from the transcript. Rarely, that turn's last
events still arrive after
session.resumed. Don't take them as a new turn: check the session for the outcome and read the answer from the transcript.
Never send the question again to recover from a dropped stream. That starts a second turn.
- Python
- JavaScript
import time
DROPPED = (ConnectionError, requests.exceptions.ConnectionError,
requests.exceptions.ChunkedEncodingError)
def retry_after(response):
"""Seconds to wait before retrying a failed read, or None if retrying can't help."""
is_json = response.headers.get("Content-Type", "").startswith("application/json")
error = response.json()["error"] if is_json else {}
if response.status_code not in (502, 503, 504) or error.get("retryable") is False:
return None
wait = response.headers.get("Retry-After", "")
return int(wait) if wait.isdigit() else 5
def wait_for_turn_to_end(session_url):
"""Reattach until the current turn ends, then return the newest message."""
while True:
try:
with api.get(
f"{session_url}/events", headers={"Accept": "text/event-stream"}, stream=True
) as response:
events = iter_events(response)
_, opening = next(events, (None, None)) # session.resumed
if opening is None:
raise ConnectionError("The stream closed before session.resumed.")
if opening["status"] in ("queued", "in_progress"):
collect_turn(events)
# A stream that hit its time limit ends with turn.timed_out while the turn still runs.
if api.get(session_url).json()["status"] not in ("queued", "in_progress"):
return api.get(f"{session_url}/messages", params={"limit": 1}).json()["data"][0]
except DROPPED:
time.sleep(1)
except requests.HTTPError as error:
wait = retry_after(error.response)
if wait is None:
raise
time.sleep(wait)
// Reattach until the current turn ends, then return the newest message.
async function waitForTurnToEnd(sessionPath) {
while (true) {
let events
try {
const response = await fetch(`${API_URL}${sessionPath}/events`, {
headers: {...authHeader, Accept: 'text/event-stream'},
})
if (!response.ok) {
const body = await response.text()
const isJson = response.headers.get('Content-Type')?.startsWith('application/json')
throw Object.assign(new Error(`${response.status} ${body}`), {
status: response.status,
retryable: isJson ? JSON.parse(body).error.retryable : true,
retryAfter: response.headers.get('Retry-After'),
})
}
events = iterEvents(response)
const {value: opening} = await events.next() // session.resumed
if (!opening) throw new Error('The stream closed before the turn ended.')
if (['queued', 'in_progress'].includes(opening.data.status)) await collectTurn(events)
// A stream that hit its time limit ends with turn.timed_out while the turn still runs.
const session = await api(sessionPath)
if (!['queued', 'in_progress'].includes(session.status)) {
const {data} = await api(`${sessionPath}/messages?limit=1`)
return data[0]
}
} catch (error) {
// A dropped connection is a TypeError in fetch, or a stream that ended early.
const dropped =
error instanceof TypeError || error.message === 'The stream closed before the turn ended.'
const retryable = [502, 503, 504].includes(error.status) && error.retryable !== false
if (!dropped && !retryable) throw error
const seconds = dropped ? 1 : Number(error.retryAfter) || 5
await new Promise(resolve => setTimeout(resolve, seconds * 1000))
} finally {
await events?.return()
}
}
}
Timeouts
A stream has a time limit (see
Timeouts and limits). If a turn is still
running then, the stream sends turn.timed_out and closes, but the turn keeps running. Check
error.retryable:
false: the stream stopped watching. The turn may still finish. Attach again or check the session, then read the transcript. Don't send the question again.true: the turn itself timed out, and the session's ownstatusistimed_out. Ask again as a follow-up if you want an answer.