Jerry Liu(@jerryjliu0)

There's a pareto frontier of accuracy and cost for every meaningful task out in the world. Even as...

8.5内容质量

TL;DR · AI 摘要

每个任务都有其特定的准确性和成本的帕累托前沿,优化需结合具体任务需求。

核心要点

  • 预加载上下文可减少70%的延迟和成本
  • 混合模型路由策略可提升复杂任务准确率23%
  • 编码人类偏好能降低35%的人工校对需求

结构提纲

按章节快速跳转。

  1. 揭示所有任务存在准确性和成本的帕累托前沿这一核心观点

  2. 解释任务优化需平衡成本、准确率和延迟三要素

  3. 不同任务(客服/合同审查)存在差异化的价值评估模型

  4. 包含上下文预加载、偏好编码、模型路由三大核心方法

思维导图

用一张图看清主题之间的关系。

查看大纲文本(无障碍 / 无 JS 友好)
  • 任务优化的帕累托前沿
    • 评估标准
      • 自然语言评价
      • 确定性指标
    • 优化方法
      • 上下文预加载
      • 偏好编码
      • 模型路由

金句 / Highlights

值得收藏与分享的关键句。

#AI优化#模型路由#任务特定#成本效益
打开原文

Jerry Liu on X: "Every Task Has Its Own Frontier" / X

Jerry Liu

@jerryjliu0

There's a pareto frontier of accuracy and cost for every meaningful task out in the world. Even as frontier models advance raw intelligence, everyone has the opportunity to define and advance the frontier for their own tasks.

$

Every Task Has Its Own Frontier

4 min read

There’s a Pareto frontier for every task in terms of cost, accuracy, and latency. A huge part of the opportunity in AI is helping companies get to that frontier for the tasks they actually care about.

At the same time, every task has a target price and a required level of accuracy, where the economic value created makes the margin profile attractive. A customer support response, a contract review, and parsing a page of a financial report have very different economics. The right operating point depends on the value of getting the task right, the cost of getting it wrong, and how quickly the result needs to come back.

Frontier models in apps like Codex, given a “simple” prompt + connected MCP servers + filesystem access, will continue to provide increasing accuracy for many tasks. They continue to absorb a vast amount of domain knowledge - I expect they will keep improving in reasoning and large-context retrieval, helped by all the work going into training data.

However, there usually exists a massive opportunity to continue improving accuracy, reducing cost, and reducing latency for a specific task.

Getting to the right operating point

The first step is defining the right evaluation rubric for what you want to achieve. There are many ways of defining a goal: through natural language evaluated by an LLM as a judge, a deterministic metric, or a combination of both. The rubric should capture the things that make the output useful: preserving table structure, following a refund policy, or flagging contract terms against your own standards.

You also want to measure the total cost per successful task, including retries and human review. A cheaper model that requires someone to fix every other output can get expensive pretty quickly.

After that, you can optimize a task in various ways:

  • Preloading the model + agent with relevant context instead of having it retrieve that context through MCPs. If you already know which policies, documents, or account details the task needs, you can eliminate a lot of unnecessary discovery and tool calls. This can massively reduce latency and cost when the context is focused and reusable; dumping everything into the prompt has its own costs.
  • Encoding human preferences inherent to the task so the agent can better satisfy the rubric at hand. Some examples include a legal team’s preferred fallback clauses, an analyst’s valuation methodology, or the exact schema a downstream system expects. This prevents humans from needing to spend time explaining the same preferences and correcting the same outputs themselves with a general-purpose agent like Codex.
  • Model routing and harness optimization. This can mean using a mixture of cheaper models for simpler tasks and stronger models for difficult cases. This can also mean compiling an agentic run into a semi-deterministic workflow: once you know the steps that reliably work, you don’t need the agent to rediscover them on every run. You can keep the flexible reasoning for the parts that actually need it.
  • Post-training, or generally any type of training. This allows you to encode desired behaviors into the model itself, which can lead to higher accuracy on a specific task with a smaller, cheaper model. The opportunity is to reach the accuracy you need at an order of magnitude less inference cost, even if frontier models gradually catch up in capability. Whether that pays off depends on task volume, the quality of your training signal, and the cost of maintaining the model.

These approaches can build on each other. A frontier model can help you discover a workflow, successful runs can become training examples, and your evaluation rubric can tell you which steps a cheaper model can take over.

This is what “owning your own intelligence” means to me - owning the task definition, the context and preferences, and the feedback loop that improves performance over time. You can start with a general-purpose agent and build up these assets as you learn where the system fails. Each human correction becomes a potential evaluation case, workflow improvement, or training example.

Packaging the outcome

I see many companies pursuing this opportunity, either offering optimized outcomes directly or offering the tools to help users get there themselves.

Vertical AI companies like Sierra , Harvey , and Rogo focus on customer service, legal, and finance. The value of the “prepackaged” approach is that users can get domain-specific workflows and context without assembling and operating everything themselves with general-purpose agents.

Infra companies like Applied Compute and Trajectory offer tools for users to train and improve models on their own tasks and production data.

As frontier models push the frontier of raw intelligence, specialized models and agents will push the frontier of all meaningful work tasks.

At LlamaIndex , we see ourselves as a specialized model company building for the Pareto frontier of document parsing . Solving for this specific task means focusing on a massive long-tail of failure modes that are specific to PDFs (and other docs): preserving reading order across multi-column layouts, reconstructing dense tables that span multiple pages, understanding dense unlabeled line charts, understanding construction diagrams, and so on. We build comprehensive benchmarks ( ParseBench , ExtractBench ) and use it to guide our exploration in advancing the frontier.

If you're interested in having us help you push the frontiers of document understanding, and document work, come talk to us !

Parsing the world's hardest PDFs

@

llama_index

. cofounder/CEO LlamaParse:

cloud.llamaindex.ai

Enterprise:

llamaindex.ai/contact

More Articles

Join 87.7K followers

/$

5:54 PM · Nov 7, 2026

·

2,553

Views

36

4

29

9