Outlines 结构化LLM输出

Outlines automatically infers the required structure from the function signature

prompt = f""" <|im_start|>user Extract the meeting details from this request:

{user_request} <|im_end|> <|im_start|>assistant """ meeting_params = model(prompt, schedule_meeting, max_new_tokens=200)

🚀 Building the future of structured generation

We're working with select partners to develop new interfaces to structured generation.

Need XML, FHIR, custom schemas or grammars? Let's talk.

Audit your schema: share one schema, we show you what breaks under generation, the constraints that fix it, and compliance rates before and after. Sign up here.

1. Install outlines

 shell
pip install outlines

Quickstart

Getting started with outlines is simple:

Real-world examples

Here are production-ready examples showing how Outlines solves common problems:

<details id="customer-support-triage"><summary><b>🙋‍♂️ Customer Support Triage</b> <br>This example shows how to convert a free-form customer email into a structured service ticket. By parsing attributes like priority, category, and escalation flags, the code enables automated routing and handling of support issues. </summary>

``` python import outlines from enum import Enum from pydantic import BaseModel from transformers import AutoTokenizer, AutoModelForCausalLM from typing import List

MODEL_NAME = "microsoft/Phi-3-mini-4k-instruct" model = outlines.from_transformers( AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"), AutoTokenizer.from_pretrained(MODEL_NAME) )

def alert_manager(ticket): print("Alert!", ticket)

class TicketPriority(str, Enum): low = "low" medium = "medium" high = "high" urgent = "urgent"

class ServiceTicket(BaseModel): priority: TicketPriority category: str requires_manager: bool summary: str action_items: List[str]

customer_email = """ Subject: URGENT - Cannot access my account after payment

I paid for the premium plan 3 hours ago and still can't access any features. I've tried logging out and back in multiple times. This is unacceptable as I have a client presentation in an hour and need the analytics dashboard. Please fix this immediately or refund my payment. """

prompt = f""" <|im_start|>user Analyze this customer email:

{customer_email} <|im_end|> <|im_start|>assistant """

ticket = model( prompt, ServiceTicket, max_new_tokens=500 )

Use with examples for few-shot learning

examples = [ ("The food was cold", "Negative"), ("The staff was friendly", "Positive") ] few_shot_prompt = example_template(examples=examples, query="Service was slow") print(few_shot_prompt) ``` </details>

Sample event descriptions

event_descriptions = [ # Complete information """ Join us for DevCon 2023, the premier developer conference happening on November 15-17, 2023 at the San Francisco Convention Center. Topics include AI/ML, cloud infrastructure, and web3. Registration is required. """,

# Insufficient information """ Tech event next week. More details coming soon! """ ]

Sample documents to classify

documents = [ "Q3 Financial Summary: Revenue increased by 15% year-over-year to $12.4M. EBITDA margin improved to 23% compared to 19% in Q3 last year. Operating expenses...",

"This agreement is made between Party A and Party B, hereinafter referred to as 'the Parties', on this day of...",

"The API accepts POST requests with JSON payloads. Required parameters include 'user_id' and 'transaction_type'. The endpoint returns a 200 status code on success." ]

template = outlines.Template.from_string(""" <|im_start|>user Classify the following document into exactly one category among the following categories: - Financial Report - Legal Contract - Technical Documentation - Marketing Material - Personal Correspondence

Document: {{ document }} <|im_end|> <|im_start|>assistant """)

Model Integrations