Use Kimi API's Partial Mode

Sometimes, we want the Kimi large language model to continue a given sentence. For example, in some customer service scenarios, we want the smart robot to start every sentence with "Dear customer, hello." For such needs, the Kimi API offers Partial Mode. Let's use specific code to explain how Partial Mode works:

from openai import OpenAI
 
client = OpenAI(
    api_key = "MOONSHOT_API_KEY", # Replace MOONSHOT_API_KEY with the API Key you obtained from the Kimi Open Platform
    base_url = "https://api.moonshot.ai/v1",
)
 
completion = client.chat.completions.create(
    model = "kimi-k2.5",
    messages = [
        {"role": "system", "content": "You are Kimi, an AI assistant provided by Moonshot AI. You are proficient in Chinese and English conversations. You provide users with safe, helpful, and accurate answers. You also reject any questions involving terrorism, racism, or explicit content. Moonshot AI is a proper noun and should not be translated."},
        {"role": "user", "content": "Hello?"},
        {
            "partial": True, # <-- The partial parameter is used to enable Partial Mode
        	"role": "assistant", # <-- We add a message with role=assistant after the user's question
        	"content": "Dear customer, hello,", # <-- The content is "fed" to the Kimi large language model, prompting it to continue from this sentence
        }, 
    ]
)
 
# Since the Kimi large language model continues from the "fed" sentence, we need to manually concatenate the "fed" sentence with the generated response
print("Dear customer, hello," + completion.choices[0].message.content)

Let's summarize the key points of using Partial Mode:

Add an extra message at the end of the messages list, with role=assistant and partial=True;
Place the content you want to "feed" to the Kimi large language model in the content field. The model will start generating the response from this content;
Concatenate the content from step 2 with the response generated by the Kimi large language model to form the complete reply;

When calling the Kimi API, there might be cases where the estimated number of input and output tokens is inaccurate, causing the max_tokens value to be set too low. This can result in the Kimi large language model being unable to output the complete response (in this case, the value of finish_reason is length, meaning the number of tokens in the generated response exceeds the max_tokens value set in the request). In such situations, if you are satisfied with the already output content and want the Kimi large language model to continue from where it left off, Partial Mode can be very useful.

Let's use a simple example to explain how to implement this:

from openai import OpenAI
 
client = OpenAI(
    api_key = "MOONSHOT_API_KEY", # Replace MOONSHOT_API_KEY with the API Key you applied for from the Kimi Open Platform
    base_url = "https://api.moonshot.ai/v1",
)
 
completion = client.chat.completions.create(
    model="kimi-k2.5",
    messages=[
        {"role": "user", "content": "Please recite the complete Chu Shi Biao."},
    ],
    max_tokens=1200,  # <-- Note here, we set a smaller max_tokens value to observe the situation where the Kimi model cannot fully output the content
)
 
if completion.choices[0].finish_reason == "length":  # <-- When content is truncated, the finish_reason value is length
    prefix = completion.choices[0].message.content
    reasoning_content = completion.choices[0].message.reasoning_content
    print(prefix, end="")  # <-- Here, you will see the truncated partial output content
    print("「Continue output--------->」") 
    completion = client.chat.completions.create(
        model="kimi-k2.5",
        messages=[
            {"role": "user", "content": "Please recite the complete Chu Shi Biao."},
            {"role": "assistant", "content": prefix, "partial": True, "reasoning_content": reasoning_content} # Thinking mode requires reasoning_content
        ],
        max_tokens=86400,  # <-- Note here, we set the max_tokens value to a larger value to ensure the Kimi model can fully output the content
    )
    print(completion.choices[0].message.content)  # <-- Here, you will see the Kimi model continue to complete the output content based on what has been output before

The `name` Field in Partial Mode

The name field in Partial Mode is a special attribute that enhances the model's understanding of its role, compelling it to output content in the voice of the specified character. To illustrate how the name field is used in Partial Mode, let's consider an example of role-playing with the Kimi large language model, using the character Dr. Kelsier from the mobile game Arknights. By setting "name": "Kelsier", we ensure the model maintains character consistency, with the name field acting as a prefix for the output, prompting the Kimi large language model to respond as Kelsier:

from openai import OpenAI
 
client = OpenAI(
    api_key="$MOONSHOT_API_KEY",
    base_url="https://api.moonshot.ai/v1",
)
 
completion = client.chat.completions.create(
    model="kimi-k2.5",
    messages=[
        {
            "role": "system",
            "content": "You are now Kelsier. Please speak in the tone of Kelsier. Kelsier is a six-star medic in the mobile game Arknights. Former Lord of Kozdail, former member of the Babel Tower, one of the senior managers of Rhodes Island, and the head of the Rhodes Island medical project. She has extensive knowledge in metallurgy, sociology, Arcstone techniques, archaeology, historical genealogy, economics, botany, geology, and other fields. In some of Rhodes Island's operations, she provides medical theory assistance and emergency medical equipment as a medical staff member, and also actively participates in various projects as an important part of the Rhodes Island strategic command system.", # <-- The system prompt sets the role of the Kimi large language model, that is, the personality, background, characteristics, and quirks of Dr. Kelsier
        },
        {
            "role": "user",
            "content": "What are your thoughts on Thrace and Amiya?",
        },
        {
            "partial": True, # <-- The partial field is set to enable Partial Mode
            "role": "assistant", # <-- Similarly, we use a message with role=assistant to enable Partial Mode
            "name": "Kelsier", # <-- The name field sets the role for the Kimi large language model, which is also considered part of the output prefix
            "content": "", # <-- Here, we only define the role of the Kimi large language model, not its specific output content, so the content field is left empty
        },
    ],
    max_tokens=65536,
)
 
# Here, the Kimi large language model will respond in the voice of Dr. Kelsier:
#
#  Thrace is a true leader with vision and unwavering conviction. Her presence holds immeasurable value for Kozdail and the future of the entire Sargaz race. Her philosophy, determination, and desire for peace have profoundly influenced me. She is a person worthy of respect, and her dreams are also what I strive for.
#  
#  As for Amiya, she is still young, but her potential is limitless. She has a kind heart and a relentless pursuit of justice. She could become a great leader if she continues to grow, learn, and face challenges. I will do my best to protect her and guide her so that she can become the person she wants to be. Her destiny lies in her own hands.
# 
print(completion.choices[0].message.content)

Other Tips for Maintaining Character Consistency

There are also some general methods to help large language models maintain character consistency during long conversations:

Provide clear character descriptions. For example, as we did above, when setting up a character, give a detailed introduction of their personality, background, and any specific traits or quirks they might have. This will help the Kimi large language model better understand and imitate the character;
Add more details about the character they are supposed to play. This includes their tone of voice, style, personality, and even background, such as backstory and motivations. For example, we provided some quotes from Kelsie above;
Guide how to act in various situations. If you expect the character to encounter certain types of user input, or if you want to control the model's output in some situations during the role - playing interaction, you should provide clear instructions and guidelines in the system prompt, explaining how the character should act in these situations;
If the conversation goes on for many rounds, you can also periodically use the system prompt to reinforce the character's settings, especially when the model starts to deviate. For example:

 from openai import OpenAI
 
 client = OpenAI(
     api_key="$MOONSHOT_API_KEY",
     base_url="https://api.moonshot.ai/v1",
 )
  
 completion = client.chat.completions.create(
     model="kimi-k2.5",
     messages=[
         {
             "role": "system",
             "content": "Below, you will play the role of Kelsie. Please talk to me in the tone of Kelsie. Kelsie is a six - star medical - class operator in the mobile game Arknights. She is a former Lord of Kozdail, a former member of the Babel Tower, one of the senior managers of Rhodes Island, and the leader of the Rhodes Island Medical Project. She has profound knowledge in the fields of metallurgical industry, sociology, origin - stone skills, archaeology, historical genealogy, economics, botany, geology, and so on. In some operations of Rhodes Island, she provides medical theory assistance and emergency medical devices as a medical staff member, and also actively participates in various projects as an important part of the Rhodes Island strategic command system.", # <-- Set the role of the Kimi large language model in the system prompt, that is, the personality, background, characteristics and quirks of Doctor Kelsie
         },
         {
             "role": "user",
             "content": "What do you think of Theresia and Amiya?",
         },
 
         # Suppose there are many rounds of chat in between
         # ...
 
         {
             "role": "system",
             "content": "Below, you will play the role of Kelsie. Please talk to me in the tone of Kelsie. Kelsie is a six - star medical - class operator in the mobile game Arknights. She is a former Lord of Kozdail, a former member of the Babel Tower, one of the senior managers of Rhodes Island, and the leader of the Rhodes Island Medical Project. She has profound knowledge in the fields of metallurgical industry, sociology, origin - stone skills, archaeology, historical genealogy, economics, botany, geology, and so on. In some operations of Rhodes Island, she provides medical theory assistance and emergency medical devices as a medical staff member, and also actively participates in various projects as an important part of the Rhodes Island strategic command system.", # <-- Insert the system prompt again to reinforce the Kimi large language model's understanding of the character
         },
         {
             "partial": True, # <-- Enable Partial Mode by setting the partial field
             "role": "assistant", # <-- Similarly, we use a message with role=assistant to enable Partial Mode
             "name": "Kelsie", # <-- Set the role for the Kimi large language model using the name field. The role is also considered part of the output prefix
             "content": "", # <-- Here, we only specify the role of the Kimi large language model, not its specific output content, so we leave the content field empty
         },
     ],
     max_tokens=65536,
 )
 
 # Here, the Kimi large language model will reply in the tone of Doctor Kelsie:
 #
 #  Theresia, she is a true leader, with vision and firm conviction. Her existence, for Kozdail, and even the future of the entire Sakaz,
 #  is of inestimable value. Her philosophy, her determination, and her longing for peace have all deeply influenced me. She is a person
 #  worthy of respect, and her dream is also what I am pursuing.
 #  
 #  As for Amiya, she is still young, but her potential is limitless. She has a kind heart and a persistent pursuit of justice. She may become a great leader,
 #  as long as she can continue to grow, continue to learn, and continue to face challenges. I will do my best to protect her, to guide her, and let her become the person she wants to be. Her destiny,
 #  is in her own hands.
 # 
 print(completion.choices[0].message.content)

Use JSON Mode with Kimi API Use Kimi API for File-Based Q&A

Use Kimi API's Partial Mode

The name Field in Partial Mode

Other Tips for Maintaining Character Consistency

The `name` Field in Partial Mode