messages currently used by the model are cleaned and updated. Each placeholder records the archive file path and record ID so that your application can retrieve the result.
Implement the tool result cleaning function
The cleanup policy is controlled by two token thresholds:
The protection window accumulates token counts from the newest tool result backward until the total reaches the window size; every counted result keeps its full content, including the one that brings the total up to the window size. So no matter how small the window is, the newest tool result always keeps its full content.
For example, suppose the history contains four tool results whose accumulated token counts from newest to oldest are 10000, 35000, 43000, and 71000, with a protection window of
40000: accumulation reaches the window size at 43000, so the first three results remain in full; the oldest result is outside the window, and the old results total 28000 tokens (71000 - 43000), exceeding the trigger threshold of 20000, so they are replaced with placeholders.
The following code demonstrates archiving and cleanup in a single-process, single-conversation setting. Applications should use a separate archive for each conversation; for large histories, consider a database for complete results.
Before running the example, set your API key in the MOONSHOT_API_KEY environment variable:
Update message history with the cleaning function
This example usesmessages for the message history currently sent to the model and new_tool_messages for the tool results produced in the current round.
To use an existing history that has not yet been archived, first archive each tool result in messages. If the loaded history was already cleaned and archived, skip this step. For results that still contain their original content, the function checks the corresponding archive records and content before cleanup. If the check fails, it raises an error and stops cleanup.
Use a separate archive file for each conversation:
messages and archive them. Then clean old results and update messages. Send the request with the application’s existing tools:
messages by tool_call_id before retrying to avoid duplicate messages.
This example sets the protection window protect_recent_tokens to 40000 tokens and the trigger threshold min_clearable_tokens to 20000 tokens. Adjust these values to fit your task; for example, setting min_clearable_tokens to 0 cleans old results outside the window on every round.
Choose the protection window and trigger threshold
Increase the protection window if later steps still need older results; reduce it when those results are no longer needed. Evaluate different values by comparing answer quality andusage.prompt_tokens on your task.
Replacing an old result with a placeholder for the first time changes the replaced message and subsequent content, which may affect cache hits when using context caching. When evaluating cleanup, also check cache hit rates.
If some tool results contain instructions that must be followed later (such as skill descriptions), you can exempt them by tool name so they are not cleaned.