N
NexusDigitalLabs
← Back to Academy

Phase 2 — Working with LLMs · Lesson 21 · 20 XP

Streaming

A non-streaming call waits for the entire response to finish generating before you see anything. Streaming delivers tokens as they're produced, so a user sees text appearing immediately — the total generation time is the same, but the perceived latency is much lower.

with client.messages.stream(
    model="claude-sonnet-5", max_tokens=300,
    messages=[{"role": "user", "content": "Write a haiku about oceans."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

Under the hood this is usually server-sent events: a long-lived HTTP connection delivering a sequence of small chunks. Your code has to handle partial output correctly — a chunk isn't guaranteed to be a complete word, sentence, or JSON object, so code that assumes it can immediately parse each chunk as something complete will break.

Exercise

Convert your Lesson 20 chatbot loop to stream tokens to the terminal as they arrive instead of printing the full response at once.

Check yourself

1. What problem does streaming solve for the user, given that the total generation time is the same either way?

2. What could go wrong if your code assumes every streamed chunk is one complete word or JSON object?

← Previous lesson

First API calls: messages and system prompts

Answer the check-yourself questions to unlock this