原文：https://www.together.ai/blog/function-calling-json-mode JSON模式和函数调用简介虽然 JSON 模式和函数调用都可以增强与 LLMs的交互，但重要的是要了解它们不可互换 - 它们服务于不同的目的并提供独特的好处。具体说来：JSON 模式允许您指定 JSON 架构，该架构将用于LLM以此格式输出数据。这意味着您可以指定响应的格式和数据类型，从而获得更结构化和可预测的输出，以满足您的特定需求。 通过函数调用LLMs，可以智能地输出一个 JSON 对象，其中包含已定义的外部函数的参数。当需要实时数据访问（例如天气更新、产品信息或股票市场数据）时，或者当您希望了解LLM您定义的某些功能时，这特别有用。它还使LLM它能够智能地确定从用户那里收集哪些信息，如果它确定应该调用函数。我们的终结点确保这些函数调用与规定的函数架构保持一致，并将必要的参数与适当的数据类型相结合。 JSON ModeJSON 模式使用 JSON 模式，您可以为 .LLM虽然 OpenAI API 本身并不允许指定 JSON 模式，但我们 response_format 用 schema .当传入模式时，我们强制模型生成与给定模式一致的输出。 下面是如何将 JSON 模式与 Mixtral 结合使用的示例： 示例:import osimport jsonimport openaifrom pydantic import BaseModel, Field# Create clientclient = openai.OpenAI( base_url = "https://api.together.xyz/v1", api_key = os.environ['TOGETHER_API_KEY'],)# Define the schema for the output.class User(BaseModel): name: str = Field(description="user name") address: str = Field(description="address") # Generatechat_completion = client.chat.completions.create( model="mistralai/Mixtral-8x7B-Instruct-v0.1", response_format={ "type": "json_object", "schema": User.model_json_schema() }, messages=[ {"role": "system", "content": "You are a helpful assistant that answers in JSON."}, {"role": "user", "content": "Create a user named Alice, who lives in 42, Wonderland Avenue."} ],)created_user = json.loads(chat_completion.choices[0].message.content)print(json.dumps(created_user, indent=2))在此示例中，我们为包含其名称和地址的 User 对象定义一个架构。然后生成LLM与此架构匹配的响应，提供一个结构化的 JSON 对象，我们可以以确定性的方式直接在应用程序中使用该对象。 此示例的预期输出为： { "address": "42, Wonderland Avenue", "name": "Alice"} 更多示例：数组和可选参数：我们还支持可选参数。此示例基于上面的示例，但具有一个名为 explanation 的可选参数。 from typing import Optionalclient = openai.OpenAI( base_url = "https://api.together.xyz/v1", api_key = os.environ['TOGETHER_API_KEY'],)class Result(BaseModel): ordered_numbers: List[int] explanation: Optional['str']chat_completion = client.chat.completions.create( model="mistralai/Mixtral-8x7B-Instruct-v0.1", response_format={ "type": "json_object", "schema": Result.model_json_schema() }, messages=[ {"role": "system", "content": "You are a helpful assistant that answers in JSON."}, {"role": "user", "content": "Please output this list in order of DESC [1, 4, 2, 8]."} ])response = json.loads(chat_completion.choices[0].message.content)print(json.dumps(response, indent=2))'''{ "ordered_numbers": [ 8, 4, 2, 1 ], "explanation": "The function 'desc' sorts the input list in descending order."}'''‍ 嵌套数据类型：此示例演示如何处理 JSON 响应中的嵌套数据类型。定义了两个 Pydantic 模型：字段 、 、 和 ，以及 User 字段 name 、 is_active 和 address （使用模型 Address ）。 country zip street city Address 该模型生成一个 JSON 响应，该响应按照定义的架构创建具有给定详细信息的用户。 import osimport jsonimport openaifrom pydantic import BaseModel, Field# Create clientclient = openai.OpenAI( base_url = "https://api.together.xyz/v1", api_key = os.environ['TOGETHER_API_KEY'],)# Define the schema for the output.class Address(BaseModel): street: str city: str country: str zip: strclass User(BaseModel): name: str = Field(description="user name") is_active: bool = Field(default=True) address: Address # Generatechat_completion = client.chat.completions.create( model="mistralai/Mixtral-8x7B-Instruct-v0.1", response_format={ "type": "json_object", "schema": User.model_json_schema() }, messages=[ {"role": "system", "content": "You are a helpful assistant that answers in JSON."}, {"role": "user", "content": "Create a user named Alice, who lives in 42, Wonderland Avenue, Wonderland city, 91234 Dreamland."} ],)created_user = json.loads(chat_completion.choices[0].message.content)print(json.dumps(created_user, indent=2))'''{ "name": "Alice", "address": { "street": "Wonderland Avenue", "city": "Wonderland", "country": "Dreamland", "zip": "91234" }, "is_active": true}''' 函数调用（Function Calling）通过函数调用，它将输出一个 JSON 对象，其中包含定义的外部函数的参数。定义函数后，LLM将智能地确定是否需要调用函数，如果需要调用，它将在 JSON 对象中建议具有正确参数的适当函数。之后，您可以在应用程序中执行 API 调用，并将响应中继回 LLM 继续工作。 让我们用一个简单的例子来说明这个过程：创建一个可以访问天气数据的聊天机器人。该函数定义如下 tools ： 示例：import osimport jsonimport openai# Create clientclient = openai.OpenAI( base_url = "https://api.together.xyz/v1", api_key = os.environ['TOGETHER_API_KEY'],)# Define function(s)tools = [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "enum": [ "celsius", "fahrenheit" ] } } } } }] # Generateresponse = client.chat.completions.create( model="mistralai/Mixtral-8x7B-Instruct-v0.1", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What is the current temperature of New York?"} ], tools=tools, tool_choice="auto",)print(json.dumps(response.choices[0].message.dict()['tool_calls'], indent=2))在此示例中，我们定义了一个外部函数，用于获取给定位置的当前天气。然后，我们在聊天完成请求中使用此函数。AI 模型生成一个响应，其中包括对此函数的调用，为请求的位置提供实时天气数据。预期输出为： [ { "id": "...", "function": { "arguments": "{\"location\":\"New York\",\"unit\":\"fahrenheit\"}", "name": "get_current_weather" }, "type": "function" }]‍ 如果 tool_choice="auto" ，模型可能会选择不调用任何函数调用。要始终使用函数，只需指定 tool_choice= {"type": "function", "function": {"name": ""}} 。此外，您可以通过指定 tool_choice="none" 来阻止模型调用函数。 更多示例并行函数调用：此示例演示如何为多个输入并行调用函数。用户要求获取纽约、旧金山和芝加哥的当前温度。该模型生成一个响应，该响应并行调用每个位置的 get_current_weather 函数，返回函数调用对象数组。 # Define function(s)tools = [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "enum": [ "celsius", "fahrenheit" ] } } } } }] # Generateresponse = client.chat.completions.create( model="mistralai/Mixtral-8x7B-Instruct-v0.1", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What is the current temperature of New York, San Francisco and Chicago?"} ], tools=tools, tool_choice="auto",)print(json.dumps(response.choices[0].message.dict()['tool_calls'], indent=2))'''[ { "id": "...", "function": { "arguments": "{\"location\":\"New York, NY\",\"unit\":\"fahrenheit\"}", "name": "get_current_weather" }, "type": "function" }, { "id": "...", "function": { "arguments": "{\"location\":\"San Francisco, CA\",\"unit\":\"fahrenheit\"}", "name": "get_current_weather" }, "type": "function" }, { "id": "...", "function": { "arguments": "{\"location\":\"Chicago, IL\",\"unit\":\"fahrenheit\"}", "name": "get_current_weather" }, "type": "function" }]'''‍无函数调用：此示例显示当用户请求不需要函数调用时模型的行为方式。相同的功能 get_current_weather 被定义为工具，但用户请求是查找苏黎世的位置，这不需要该功能。该模型会生成一个响应，通知用户它无法提供地理信息，并且只能检索给定位置的当前天气。 # Define function(s)tools = [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "enum": [ "celsius", "fahrenheit" ] } } } } }] # Generateresponse = client.chat.completions.create( model="mistralai/Mixtral-8x7B-Instruct-v0.1", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Where is Zurich?"} ], tools=tools, tool_choice="auto",)print(response.choices[0].message.dict()['content'])'''I'm sorry, but I don't have the capability to provide geographical information. My current function allows me to retrieve the current weather in a given location. If you need help with that, feel free to ask!'''‍多轮交互示例： 此示例演示如何在多轮对话中使用函数调用来丰富具有函数响应的对话，并根据更新的对话历史记录生成响应。 # Example function to make available to modeldef get_current_weather(location, unit="fahrenheit"): """Get the weather for some location""" if "chicago" in location.lower(): return json.dumps({"location": "Chicago", "temperature": "13", "unit": unit}) elif "san francisco" in location.lower(): return json.dumps({"location": "San Francisco", "temperature": "55", "unit": unit}) elif "new york" in location.lower(): return json.dumps({"location": "New York", "temperature": "11", "unit": unit}) else: return json.dumps({"location": location, "temperature": "unknown"})tools = [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "enum": [ "celsius", "fahrenheit" ] } } } } }]messages = [ {"role": "system", "content": "You are a helpful assistant that can access external functions. The responses from these function calls will be appended to this dialogue. Please provide responses based on the information from these function calls."}, {"role": "user", "content": "What is the current temperature of New York, San Francisco and Chicago?"}] response = client.chat.completions.create( model="mistralai/Mixtral-8x7B-Instruct-v0.1", messages=messages, tools=tools, tool_choice="auto",)tool_calls = response.choices[0].message.tool_callsif tool_calls: for tool_call in tool_calls: function_name = tool_call.function.name function_args = json.loads(tool_call.function.arguments) if function_name == "get_current_weather": function_response = get_current_weather( location=function_args.get("location"), unit=function_args.get("unit"), ) messages.append( { "tool_call_id": tool_call.id, "role": "tool", "name": function_name, "content": function_response, } ) function_enriched_response = client.chat.completions.create( model="mistralai/Mixtral-8x7B-Instruct-v0.1", messages=messages, ) print(function_enriched_response.choices[0].message.model_dump()['content'])'''The current temperature in New York is 11 degrees Fahrenheit, in San Francisco it is 55 degrees Fahrenheit, and in Chicago it is 13 degrees Fahrenheit.'''‍ 结论我们相信，JSON模式和函数调用是向前迈出的重要一步，为AI应用程序带来了新的多功能性和功能。通过实现与模型的更结构化的交互，并允许特定类型的输出和行为，我们相信它将成为开发人员的宝贵工具。