TypeSafe 基础如何使用 TypeSafe 构建

System One 是 TypeSafe 用于构建 AI 驱动软件(而非智能体)的模型。它不生成代码,也不自行选择下一步动作。它提供可嵌入软件的 AI 原语,让代码始终掌控全局,而由模型处理对非结构化数据的常识性判断。

摘要: 构建普通的软件工作流,只在需要 AI 的地方插入 System One。

  • 将控制流、确定性规则和副作用保留在代码中。
  • 把宽泛的判断拆解为狭窄的、带类型的问题,并附上明确的 instructions 和 criteria。
  • 只给每个问题它所需的上下文。
  • 利用概率和置信度来执行、请求审查或升级处理。
  • 将独立的问题放在同一次请求中提出,然后在代码中组合它们的答案。

三种软件架构

TypeSafe 专为构建 AI 驱动的软件 而设计:代码掌管工作流,AI 负责狭窄、结构化的决策。

传统代码是由简单的软件原语构成的复杂决策树。由于每个原语都可靠,开发者可以将它们组合成更高层的抽象。

智能体处理指令并选择下一步。在有人监控流程时这运作良好,但每一次循环都多了一次偏离正轨的机会。

代码处理确定性工作并掌管控制流。模型只在系统需要可编程常识或需要解读非结构化数据的地方出现。每个 AI 任务都保持原子化并受到约束。

Traditional software, agents, and AI-powered software shown as three different system architectures.

System One 的可组合性来自哪里

结构化

System One 在构造上就是类型安全的。决策和概率都符合你代码所期望的结构化软件类型和 JSON schema,因此它从不需要从生成的散文文本中恢复数值。

并行

各问题独立且并行地求值。一个原语的结果不会变成改变另一个原语结果的隐藏上下文。

可比较

输出可排序,可用于驱动智能的 if 语句、阈值和比较。

快速

大多数查询在大约 100 毫秒内完成。System One 的速度足以支撑实时请求路径和用户界面。

校准的置信度

RLCD 通过校准的概率传达不确定性,而不是趋向于过度自信。

自洽

System One 被设计为在重复求值时返回稳定的答案。参见自洽性指南。

由于每个输出都被约束在所提供的选项之内,模型返回的是这些选项上的完整概率分布,而不是凭空编造一个 schema 之外的值。TypeSafe 的目标是让智能与速度和成本之比超过 100 倍;这背后的押注是:更便宜的智能将催生大得多的需求。

设计 System One 工作流

  1. 能用代码时就用代码

    把确定性工作保留在代码中,它既可靠又便宜。当软件工作流能够表达相同行为时,避免使用智能体的 while 循环。

    示例:将确定性规则保留在代码中
    python
    days_overdue = (today - invoice.due_date).days
    
    if days_overdue > 30:
        route_to_collections(invoice)
    

    浏览 System One 模式,了解以受约束的方式将模型决策与代码组合的做法。

  2. 拆解输入状态

    只包含与当前问题相关的上下文。这有助于模型避免分心和上下文退化。当最新信息可以来自你自己的知识库时,不要依赖模型权重中存储的知识。

    示例:只发送相关上下文
    request
    json
    {
      "state": {
        "ticket_message": "My flight was cancelled. Can I get a refund?",
        "refund_policy": "Cancelled flights are eligible for a full refund."
      },
      "questions": {
        "policy_supports_refund": {
          "type": "noul",
          "instructions": "Does the refund policy support the refund requested in the ticket?"
        }
      }
    }
  3. 在输入状态中使用结构

    在 state 和 questions 字段中使用嵌套 JSON。当指向特定值可以消除歧义时,让问题指向这些值,并在问题内部为每个路径保留反引号字符。

    示例:引用嵌套值

    使用带反引号的点与索引路径让问题指向某个特定的嵌套值,例如 support.tickets[0].message。

    request
    json
    {
      "state": {
        "support": {
          "tickets": [
            {
              "message": "I was charged twice for order A-104."
            },
            {
              "message": "How do I reset my password?"
            }
          ]
        },
        "commerce": {
          "orders": [
            {
              "id": "A-104",
              "charges": [
                {
                  "amount_usd": 49,
                  "status": "captured"
                },
                {
                  "amount_usd": 49,
                  "status": "captured"
                }
              ]
            }
          ]
        },
        "account": {
          "security": {
            "password_reset": "Email a reset link to the address on file."
          }
        }
      },
      "questions": {
        "duplicate_charge": {
          "type": "noul",
          "instructions": "Do `support.tickets[0].message` and `commerce.orders[0].charges` indicate a duplicate charge?"
        },
        "password_reset_supported": {
          "type": "noul",
          "instructions": "Can `account.security.password_reset` resolve the request in `support.tickets[1].message`?"
        }
      }
    }
  4. 拆解问题

    尽可能提出最明确、最狭窄、最具体、最原子化的问题。把复杂的或定义不清的问题拆成多个问题,让每个问题只评估一个属性。

    这可能是本指南中最重要的概念。宽泛的问题会把多个判断藏在同一个答案背后。原子化的问题把这些判断暴露出来,便于你在代码中检查、调优和组合。

    示例:拆解垃圾信息检测
    一个宽泛的问题(差)
    json
    {
      "is_spam": {
        "type": "noul",
        "instructions": "Is `message` spam?"
      }
    }
    拆解后的多个问题(好)
    json
    {
      "requests_credentials": {
        "type": "noul",
        "instructions": "Does `message.body` ask the recipient to provide a password or other login credential?"
      },
      "offers_unexpected_reward": {
        "type": "noul",
        "instructions": "Does `message.body` claim the recipient received an unexpected prize, payment, or reward?"
      },
      "creates_time_pressure": {
        "type": "noul",
        "instructions": "Does `message.subject` or `message.body` pressure the recipient to act quickly?"
      },
      "sender_identity_mismatch": {
        "type": "noul",
        "instructions": "Does the organization named in `message.sender.display_name` conflict with the domain in `message.sender.email`?"
      },
      "link_domain_mismatch": {
        "type": "noul",
        "instructions": "Does the domain in `message.links[0].url` conflict with the organization named in `message.sender.display_name`?"
      },
      "disguises_link_destination": {
        "type": "noul",
        "instructions": "Does `message.links[0].text` conceal or misrepresent the destination in `message.links[0].url`?"
      }
    }
    示例:验证工具调用轨迹
    一个宽泛的问题(差)
    json
    {
      "tool_calls_are_correct": {
        "type": "noul",
        "instructions": "Is `trace.tool_calls` correct for `request` and `available_tools`?"
      }
    }
    拆解后的多个问题(好)
    json
    {
      "geocode_tool_is_relevant": {
        "type": "noul",
        "instructions": "Is `trace.tool_calls[0].name` an appropriate tool for resolving `request.location`?"
      },
      "geocode_location_matches": {
        "type": "noul",
        "instructions": "Does `trace.tool_calls[0].arguments.city` match `request.location`?"
      },
      "geocode_arguments_match_schema": {
        "type": "noul",
        "instructions": "Does `trace.tool_calls[0].arguments` conform to `available_tools.geocode_city.parameters`?"
      },
      "geocode_result_matches_call": {
        "type": "noul",
        "instructions": "Does `trace.tool_results[0].tool_call_id` match `trace.tool_calls[0].id`?"
      },
      "weather_tool_is_relevant": {
        "type": "noul",
        "instructions": "Is `trace.tool_calls[1].name` an appropriate tool for answering `request.text`?"
      },
      "weather_arguments_match_schema": {
        "type": "noul",
        "instructions": "Does `trace.tool_calls[1].arguments` conform to `available_tools.get_weather.parameters`?"
      },
      "weather_uses_geocoded_coordinates": {
        "type": "noul",
        "instructions": "Do the coordinates in `trace.tool_calls[1].arguments` match those in `trace.tool_results[0].output`?"
      },
      "weather_date_matches": {
        "type": "noul",
        "instructions": "Does `trace.tool_calls[1].arguments.date` match `request.date`?"
      },
      "weather_unit_matches": {
        "type": "noul",
        "instructions": "Does `trace.tool_calls[1].arguments.unit` match `request.unit`?"
      }
    }
  5. 在问题中使用结构

    保持问题简短。instructions 和 criteria 通常是字符串;对于简短、无歧义的问题,一个字符串就足够了。它们也可以是对象或数组。把问题本身放在一个字段中,把引导该问题的数据放在其他字段中。

    在以下情况下,结构会派上用场:

    • 问题需要上下文或示例。较长的背景信息或示例输入列表应放进问题旁边的具名字段中,这样你的代码无需改写问题就能增删或替换它们。
    • 问题的一部分来自你的代码。当某个值来自数据库时,把它放进独立的字段,而不是拼进字符串模板。
    • 多个问题的指令相似。一次请求接收一个 state,并且可以包含多个问题。添加补充数据有助于让问题彼此区分。
    示例:引用来自你代码的记录

    这个 Noul 将 state 中的一份简历与候选人数据库中的一条记录进行比较。记录原样放进 potential_duplicate,问题则通过名称引用它。

    questions
    json
    {
      "same_as_record_18": {
        "type": "noul",
        "instructions": {
          "potential_duplicate": {
            "name": "John Smith",
            "location": "Oakland, California",
            "last_employer": "Google"
          },
          "question": "Is the resume for the same person as `potential_duplicate`?"
        }
      }
    }

    来自代码的 "potential_duplicate" 数据会随时间变化。"question" 使用反引号引用它。

    criteria 内的描述也可以是对象。对于 Choice,每个选项的描述可以是一个对象,说明该选项涵盖什么、哪些内容属于其他选项,以及若干示例。在各选项之间使用相同的字段名,以便模型直接比较。

    示例:定义对比式 Choice criteria
    questions
    json
    {
      "card_help_topic": {
        "type": "choice",
        "instructions": {
          "question": "Which disposable virtual card topic is the user asking about?",
          "focus": "Classify the information the user wants."
        },
        "criteria": {
          "get_disposable_virtual_card": {
            "what": "Purpose, eligibility, or setup",
            "not_for": "Quantity, transaction, or merchant restrictions",
            "examples": [
              "How can I get a disposable virtual card?",
              "What are disposable cards for?"
            ]
          },
          "disposable_card_limits": {
            "what": "Quantity, transaction, or merchant restrictions",
            "not_for": "Purpose, eligibility, or setup",
            "examples": [
              "How many disposable cards can I make per day?",
              "Where can I use a disposable card?"
            ]
          }
        }
      }
    }

    每种问题类型的页面都有一个完整的示例:

    • Noul 将一份简历与多条候选人记录进行比较,每条记录一个问题,问题由代码构建。
    • Choice 描述两个容易混淆的选项,说明各自涵盖什么、不适用于什么,并给出示例。
    • Score 为每个级别提供描述和示例场景。

    结构化数据提取级联指南展示了共用措辞的场景:对提取记录的每个字段提问同一组问题。

    简短、无歧义的问题或标准可以仍然是一个字符串。当结构能把原本混在一起的引导信息区分开时,就加上结构。关于所有接受结构的位置,请参阅进阶:结构。

  6. 提出大量问题

    在同一次请求中针对同一个 state 提出许多狭窄、独立的问题。这是通过 API 最大化单位美元效能与智能的方式:问题并行运行,代码可以组合它们的信号,而无需增加串行的模型往返。

    参见投机性扇出模式和并行问题指南。

  7. 在代码中组合问题输出(或喂给经典机器学习模型)

    用确定性规则或加权和组合独立的答案。若要学习出组合方式,可以把概率作为下游经典机器学习模型的特征。

    示例:用加权分数组合信号
    python
    answers = response.answers
    
    # Combine independent signals into one application-specific score.
    quality = (
        0.4 * answers["answers_request"].noul
        + 0.4 * answers["citations_are_supported"].noul
        + 0.2 * (1 - answers["contradicts_context"].noul)
    )
    

    组合评分展示了如何在组合各判断的同时保留它们。如果你没有用于下游模型的标签,可以用一组昂贵的推理模型来生成标签;AutoResearch 指南展示了如何在 System One 输出上训练经典模型。

  8. 根据不确定性进行路由

    让代码对高置信度和低置信度的答案采取不同动作。把不确定的案例升级给人工或更昂贵的推理模型。通过在你自己的数据上绘制置信度与准确率的关系来测试阈值。

    示例:按置信度路由
    python
    answer = response.answers["card_help_topic"]
    
    if answer.confidence < 0.8:
        route_to_human_review(ticket)
    else:
        route_to_handler(answer.choice, ticket)
    

    关于如何选择阈值并使之匹配每个动作的风险,参见置信度和置信度门控路由。

拆解并不意味着更多的往返请求。针对同一个 state 的问题会并行运行。

综合运用

这个支持工单工作流把确定性工作保留在代码中,只发送相关的结构化上下文,在同一次请求中评估许多原子化问题,并用显式的置信度门槛组合答案。

python
from typesafe_sdk import Choice, Noul, NoulCriteria, Score, TypeSafeClient


def triage_ticket(ticket, customer):
    # Handle deterministic states without calling a model.
    if ticket["status"] == "closed":
        return "no_action"

    open_orders = [
        order for order in customer["orders"] if order["status"] != "delivered"
    ]

    # Include only the structured context needed by the questions below.
    state = {
        "ticket": {
            "message": ticket["message"],
            "sender": ticket["sender"],
            "links": ticket["links"],
        },
        "customer": {
            "plan": customer["plan"],
            "open_orders": open_orders,
        },
        "policy": {
            "sensitive_credentials": ["password", "security code", "API key"],
        },
    }

    # Ask structured, atomic questions together so they run in parallel.
    questions = {
        "topic": Choice(
            instructions={
                "question": "Which team should handle `ticket.message`?",
                "focus": "Classify the customer's primary request.",
            },
            criteria={
                "billing": {
                    "what": "Charges, invoices, refunds, or subscriptions",
                    "not_for": "Order tracking or account access",
                    "examples": ["I was charged twice", "Where is my refund?"],
                },
                "orders": {
                    "what": "Order status, delivery, cancellation, or returns",
                    "not_for": "Charges or account access",
                    "examples": ["Where is my order?", "Cancel my shipment"],
                },
                "account": {
                    "what": "Login, profile, permissions, or security",
                    "not_for": "Charges or order tracking",
                    "examples": ["Reset my password", "I cannot sign in"],
                },
            },
        ),
        "requests_credentials": Noul(
            instructions={
                "question": "Does the message request a sensitive credential?",
                "compare": [
                    "`ticket.message`",
                    "`policy.sensitive_credentials`",
                ],
                "focus": "Look for a request to disclose the credential itself.",
            },
            criteria=NoulCriteria(
                true={
                    "what": "Asks the recipient to disclose a listed credential",
                    "examples": [
                        "Reply with your password",
                        "Send us your API key",
                    ],
                },
                false={
                    "what": "Does not ask the recipient to disclose a credential",
                    "not_for": "A legitimate instruction to reset a credential",
                    "examples": ["Use this link to reset your password"],
                },
            ),
        ),
        "sender_identity_mismatch": Noul(
            instructions={
                "question": "Does the claimed sender identity conflict with its domain?",
                "compare": [
                    "`ticket.sender.display_name`",
                    "`ticket.sender.email`",
                ],
                "focus": "Compare the named organization with the email domain.",
            },
            criteria=NoulCriteria(
                true={
                    "what": "Claims an organization unrelated to the email domain",
                    "examples": ["Acme Payroll sent from claim-bonus.example"],
                },
                false={
                    "what": "The identity and domain agree or make no conflicting claim",
                    "examples": ["Acme Payroll sent from acme.example"],
                },
            ),
        ),
        "unexpected_reward": Noul(
            instructions={
                "question": "Does the message announce an unexpected reward?",
                "inspect": "`ticket.message`",
                "focus": "Look for an unsolicited prize, payment, or reward claim.",
            },
            criteria=NoulCriteria(
                true={
                    "what": "Announces an unrequested prize, payment, or reward",
                    "examples": ["You were selected for a $1,000 bonus"],
                },
                false={
                    "what": "Contains no reward claim or discusses an expected payment",
                    "not_for": "A customer asking about a known refund or payroll deposit",
                    "examples": ["When will my approved refund arrive?"],
                },
            ),
        ),
        "refund_requested": Noul(
            instructions={
                "question": "Does the customer explicitly request a refund or credit?",
                "inspect": "`ticket.message`",
                "focus": "Require a requested remedy, not a billing complaint alone.",
            },
            criteria=NoulCriteria(
                true={
                    "what": "Directly asks for money back or an account credit",
                    "examples": ["Please refund the duplicate charge"],
                },
                false={
                    "what": "Does not ask for a refund or credit",
                    "not_for": "A complaint or billing question without a requested remedy",
                    "examples": ["Why was I charged twice?"],
                },
            ),
        ),
        "mentions_open_order": Noul(
            instructions={
                "question": "Does the message refer to a supplied open order?",
                "compare": [
                    "`ticket.message`",
                    "`customer.open_orders`",
                ],
                "focus": "Match an order id or other identifying details.",
            },
            criteria=NoulCriteria(
                true={
                    "what": "Refers to an open order by id or identifying details",
                    "examples": ["Where is order A-104?"],
                },
                false={
                    "what": "Does not identify any supplied open order",
                    "not_for": "A generic order question with no matching details",
                    "examples": ["How long does shipping usually take?"],
                },
            ),
        ),
        "frustration": Score(
            instructions={
                "question": "How frustrated does the customer appear?",
                "inspect": "`ticket.message`",
                "focus": "Judge expressed frustration, not issue severity.",
            },
            criteria=[
                {
                    "what": "Calm and matter-of-fact",
                    "signals": ["Neutral wording", "No complaint about the experience"],
                },
                {
                    "what": "Frustrated but civil",
                    "signals": ["Expresses annoyance", "Remains constructive"],
                },
                {
                    "what": "Very angry or threatening to leave",
                    "signals": ["Hostile language", "Threatens cancellation or churn"],
                },
            ],
        ),
    }

    with TypeSafeClient() as client:
        response = client.system_one(
            state=state,
            questions=questions,
        )

    # Compose independent spam signals with weights controlled by code.
    answers = response.answers
    spam_risk = (
        0.45 * answers["requests_credentials"].noul
        + 0.30 * answers["sender_identity_mismatch"].noul
        + 0.25 * answers["unexpected_reward"].noul
    )

    # Escalate uncertain judgments instead of guessing.
    spam_is_uncertain = 0.4 < spam_risk < 0.6
    if spam_is_uncertain or answers["topic"].confidence < 0.75:
        return route_to_human_review(ticket)
    if spam_risk >= 0.6:
        return quarantine_as_spam(ticket)

    # Let code decide which speculative answers matter on this path.
    if answers["topic"].choice == "billing":
        return route_to_billing(
            ticket,
            refund_requested=answers["refund_requested"].noul >= 0.7,
        )
    if answers["topic"].choice == "orders":
        return route_to_orders(
            ticket,
            mentions_open_order=answers["mentions_open_order"].noul >= 0.7,
        )

    priority = (
        "high"
        if answers["frustration"].confidence >= 0.7
        and answers["frustration"].score >= 1.5
        else "normal"
    )
    return route_to_account_support(ticket, priority=priority)