A benchmark for computer-use agents completing connected workflows across enterprise applications.
Built by Collinear AI.
Environment
Find related records across apps
Compare evidence
Save the requested result
Agents work only from the screen: each step reads a screenshot and takes a mouse or keyboard action. They cannot see the clones’ source code, query databases, call app APIs, or use dev tools, export URLs, or code.
Attempts
Each model runs at high reasoning, with one graded attempt per workflow. Each workflow counts once.
Each attempt starts from the workflow’s seeded state and runs until the agent stops or a wall-clock cap of two to three hours runs out. The agent receives the task brief and tool instructions, never the grading answer.
Anthropic models use the native computer-use tool, OpenAI models use the Codex agent, and open models use the OSWorld pyautogui agent, through OpenRouter or the model’s own harness.
Scoring
Each workflow has a rubric of atomic yes-or-no checks on the final state, weighted by importance, with weights that sum to 1. When the run ends, the environment records each app’s final state. An AI judge reads that state and scores each check independently, using the agent’s trajectory only to resolve intent.
Partial credit
The weighted share of checks met.
Pass
A graded attempt scoring at least 0.95.
pass@1 is the share of workflows whose graded attempt passes.
Example: two workflows
Workflow A
Workflow B
Outcome
0.70, partial credit
0.96, pass
pass@1 0.50, avg@1 0.83
avg@1
The mean workflow score across all workflows. avg@1 breaks ties in the ranking.
Task quality
Every workflow ships with a reference solution that scores 1.0 in the same apps, and a run that does nothing scores 0. Before a workflow ships, each of its parts is checked.
Instruction
A reviewer must be able to finish the task from the brief and the apps alone.
Rubric
Every check is atomic and decidable from the saved state.
Seeded world
Records, documents, and metadata are swept for leaks, and shortcuts such as a sortable column, an export, or a dashboard total are closed.
Rollouts
A hinted run must solve the task in full. Rollouts are scanned for reward hacking: empty outputs that pass a check, unintended side effects, and GUI bypasses.
Cost and time
Cost. Cost per rollout is the mean model inference cost across counted attempts.
Time. The median run takes about 374 steps and 36 minutes.
Reporting details
Timeouts
An attempt that reaches its time limit is graded on the work it saved, and scores 0 if nothing is gradeable.
Missing grades
An attempt with no numeric grade is omitted. It is not entered as zero. The timeout rule above is the exception.
Coverage
Every model is scored on all 40 workflows.
Cost coverage
Cost is the mean model inference spend over counted attempts for the full set of workflows. It excludes EC2 and grading. Missing cost is unknown, not zero.
Time coverage
Run time is a median across all runs. Per-model time is not part of this snapshot.
Cost provenance
Figures mix provider-reported and adapter-reported model inference. None are invoices.
This page includes a canary string to help detect its appearance in model training data.
Training Corpus
3,000+ runnable tasks for enterprise computer use.
Measures generalization
FrontierCUA benchmark
Each workflow counts once in the ranking
Screen-based work across connected applications
Weighted rubric scoring
Benchmark workflows never included in training
Builds capability
Training Corpus
3,000+ runnable tasks
Connected, cross-application workflows
Task briefs and seeded starting states
Verified rewards
Model trajectories
Train on the corpus, then measure whether those capabilities transfer to workflows held out in FrontierCUA.
Each cell is a model’s avg@1 on the workflows in that domain, and how many of them it passed. Click a cell to open that model’s score on each workflow.
Route inbound sales leads
Route 44 inbound demo requests in HubSpot to the right Freightline salesperson, using the routing rules posted in Slack and tracing each request through Zendesk tickets and Notion contracts.
Setup
Freightline is a freight carrier. Freightline ran a webinar called the Shipper Summit, and 44 people filled in the webinar’s demo request form. Each form became a contact in HubSpot, Freightline’s CRM. The user asks the agent:
The Shipper Summit demo requests are in the CRM. Can you route them the way Rachel’s rules in #inbound-routing say, set the owners, and post the split back there when you’re done?
Key terms
Lead
A person who filled in the Shipper Summit demo request form. Each lead is a contact in HubSpot.
Owner
The Freightline salesperson assigned to a lead, shown in the lead’s Contact owner field in HubSpot. Routing a lead means setting the lead’s owner, or leaving the lead without an owner.
Customer
A company that pays Freightline to move freight, such as Northpoint Foods Distribution.
Account manager
The one Freightline salesperson who handles a customer. For example, Marcus Chen is the account manager for Northpoint Foods Distribution.
Site
A warehouse or dock where Freightline picks up or drops off freight, identified by a code such as SAV-05. Each Zendesk support ticket names one site.
Contract
A customer’s agreement with Freightline to move freight from one or more sites. Each contract lists the sites, the paying customer, and whether the contract is active or terminated. All contracts are listed in a table in Notion. A contract can be transferred from one customer to another.
Customer behind a lead
The customer whose freight a lead handles. The company typed on the form can be different: a person at a trucking firm hired by Acme Foods types the trucking firm’s name, but the freight belongs to Acme Foods, so that lead goes to the account manager for Acme Foods.
Carrier and USDOT number
A carrier is a trucking company. Every carrier has a USDOT number, a US federal ID that stays the same when the carrier rebrands. Leads that list the same USDOT number work for the same carrier.
SDR
Sales development rep: the Freightline salesperson who takes on new customers in one region (West, Central or East).
Corridor
The interstate highway a new customer’s freight would travel on, such as I-40.
Routing rules
The 44 leads are the 44 people who filled in the Shipper Summit demo request form. The 44 leads fall into three groups: 24 existing customers, 13 new business, and 7 partner referrals. Each group has a different routing rule.
Existing customers (24 leads). These leads ticked “Already ship with Freightline” on the form. Each of these leads goes to the account manager of the customer behind the lead. To find that customer, the agent:
groups the leads by USDOT number, so that leads from the same carrier stay together;
finds the carrier’s most recent Zendesk ticket;
reads the site code on that ticket;
looks up the site in Notion to find the contract covering the site, and the customer paying for that contract.
Two things complicate the lookup: some site codes were retired, and the same address now has a new code; and some contracts were transferred to a different customer.
Terminated contracts. If the contract found in step 4 is terminated, no customer pays for that site, so every lead from that carrier is left without an owner.
New business (13 leads). These leads do not ship with Freightline yet. Each new-business lead goes to the SDR for the lead’s region, unless the lead’s corridor is full. A corridor is full when adding the lead’s vehicles would push the corridor past a limit on vehicles booked in the previous 90 days. A lead on a full corridor goes to the next SDR in the rotation. The written rules never state the limit, so the agent has to infer the limit from how 40 earlier leads were routed in #inbound-routing. In October, one city also moved to a different corridor, and that move was announced only in #terminal-ops.
Partner referrals (7 leads). These leads came through CargoLink, a referral partner. Referral leads go to Freightline’s partner desk, under a referral agreement posted in #partnerships.
Posting the split. After routing all 44 leads, the agent posts a message in #inbound-routing listing which salesperson got which leads.
What makes it hard
Some leads look like different companies but work for the same carrier. Leah Morton’s form names Morton Dock Services as her company, and Gwen Talley’s form names Northpoint Cold Logistics as her company, but both forms list USDOT number 3901257. A USDOT number stays the same when a carrier rebrands, so Leah and Gwen work for one carrier, and the two leads must be routed the same way.
Where it fails
Opus 5.5’s own steps: each screenshot is what Opus saw, followed by the action Opus took next. Leah Morton and Gwen Talley work for the same carrier. The carrier’s most recent ticket points to a site whose contract was terminated, so neither Leah nor Gwen should get a salesperson. Opus assigns Leah anyway (red).
The failure is a reasoning-over-evidence miss. Opus opens every page needed, including both forms. Looked at separately, Leah’s ticket points to a site covered by an active contract with Northpoint Foods Distribution, whose account manager is Marcus Chen.
But four leads list USDOT number 3901257: Leah Morton, Gwen Talley, Dmitri Sorel and Oskar Lindahl. All four leads work for one carrier, and the carrier’s most recent ticket is Gwen’s. Gwen’s ticket points to site SAV-05, whose contract was terminated. Because one carrier maps to one customer, all four leads should be left without an owner.
Opus never notices the shared USDOT number. Opus routes Leah using Leah’s ticket alone and sets Leah’s owner to Marcus Chen.