If the Pilot Doesn't Change These Metrics, It Isn't Working

Key Takeaways

An AI logistics agent is working only if specific metrics change against the baseline, on the same lanes, compared with how those lanes ran before. "The team likes it" isn't a result.
8 metrics, in 4 groups, show whether it is working: workload (manual touches per load, coordinator hours), speed (time to book, time from exception to notification, POD collection time), reliability (pickups confirmed without anyone asking, missed pickups) and trust (approvals accepted without changes).
A baseline comes first. Without "before" numbers on the first lanes, results turn into opinions.
A flat metric is a diagnosis, not just a failure: it usually points to a scope, data or approval setting that needs adjusting.
On the first lanes, there are 3 decisions: expand, extend or stop. With Wilson, by Cartage, expanding means changing approval rules and adding lanes, not handing over control all at once.

How do you know if an AI logistics agent is working?

A distributor has run its first few lanes with an AI logistics agent for a month. Everyone agrees it “went well.” Then the CFO asks how many hours it saved, and nobody knows. The work didn’t fail. The team just never defined what success looked like.

You know an AI logistics agent is working when specific, agreed metrics change against a baseline on the same lanes, compared with how those lanes ran before.

An AI logistics agent is supposed to execute coordination work: quoting, booking, confirming, following up, responding to exceptions and collecting PODs. If it does that work, the effects are measurable. Fewer emails and calls per load. Faster bookings. Fewer missed pickups. Faster PODs. If none of those numbers move, the AI agent isn’t taking work off the team, whatever the demo showed.

The stakes for getting this right are high. ATRI found truck drivers were detained at 39.3% of stops in 2023, and many of those delays trace back to coordination gaps that a working AI agent should close. For how to choose the first lanes and get started, see how to start an AI logistics agent on a few lanes and measure it.

Which 8 metrics should change when an AI logistics agent starts on your lanes?

The 8 metrics that should change when an AI logistics agent starts on your lanes measure workload, speed, reliability and trust, each compared to the baseline on the same lanes.

Group	Metric	Expected direction	How to measure it
Workload	Manual touches per load	Down	Emails, calls and messages the team sends per load
Workload	Coordinator hours on those lanes	Down	Time spent on coordination for the lanes the AI agent runs
Speed	Time to book	Down	Time from order received to load booked with a carrier
Speed	Time from exception to notification	Down	Time from a delay or cancellation to the team and customer being told
Speed	POD collection time	Down	Time from delivery to the POD being filed
Reliability	Pickups confirmed and reconfirmed without anyone asking	Up	Share of loads confirmed by the AI agent, not the team
Reliability	Missed pickups	Down	Count per month on those lanes
Trust	Approvals accepted without changes	Up	Share of AI agent actions the team approves as proposed

The first 2 metrics answer “is it taking work off the team?” The middle 5 answer “is the work getting done faster and more reliably?” The last one answers “does the team trust it enough to expand?”

For the work behind each metric, see carrier follow-up, missed pickups and POD collection automation.

How do you set a baseline before the AI agent starts?

A baseline is the "before" picture of the same 8 metrics on the same lanes, recorded before the AI agent starts working.

Pick the first lanes, so the baseline matches what will be measured.
Pull recent history for those lanes: loads shipped, missed pickups, POD dates and booking times, from email, the ERP or spreadsheets.
Estimate manual effort: have coordinators log touches and time per load for a short period, or estimate from email volume.
Agree on targets up front: which metrics matter most and what change would justify expanding.
Write it down. A shared baseline keeps each review about numbers, not impressions.

With Wilson, by Cartage, the baseline is set while the scope is defined up front, before Wilson starts on the lanes. Once it's running, Wilson keeps a full change history of every message, confirmation and action, which makes the "after" numbers easy to pull.

What does it mean when a metric doesn't change?

When a metric doesn’t change, it usually points to a specific setting, scope or data issue, not to a failed start. Each flat metric has a likely cause and a fix.

Metric that didn’t move	Likely cause	What the AI agent does after the fix	What the team adjusts
Manual touches per load	The team is still doing steps the AI agent could take	Takes on more follow-ups and updates	Hands off the remaining steps; stops double-handling
Time to book	Every booking waits for approval	Books routine loads within set limits	Raises the approval limit for routine bookings
Time from exception to notification	Notifications aren’t configured for every party	Notifies the team, shipper and consignee automatically	Configures customer notifications
Missed pickups	Orders arrive too late or with missing details	Confirms and reconfirms as soon as the order lands	Fixes order intake timing or data
Approvals accepted without changes	Preferences weren’t fully captured in setup	Follows the updated rules and carrier preferences	Updates carrier preferences and rules

One flat metric is a signal to adjust. Several flat metrics after adjustments is a signal that the fit isn’t there. For how approval settings shape results, see Autonomy without rules is just another operational risk.

When should a team expand, extend or stop?

After the first lanes, the metrics point to 1 of 3 decisions: expand, extend or stop.

Decision	When it fits	What happens next
Expand	Most metrics moved in the right direction and the team trusts the work	Add lanes and modes; loosen approvals where results are strong
Extend	Some metrics moved, others are flat for a fixable reason	Adjust settings and keep measuring on the same lanes
Stop	Metrics stayed flat after adjustments	Stop with a clear record of why

Expanding doesn’t mean giving up control. Traditional logistics software gives teams information. Wilson executes the work required to move shipments forward. Wilson books only carriers the company has approved, follows configurable approvals, escalates to the team and records every action. Teams expand autonomy by changing approval rules one step at a time. Wilson supports LTL, FTL, ocean, drayage, parcel and cross-border freight, so expansion can mean new modes as well as new lanes. For scaling without new hires, see how to automate logistics without adding headcount.

FAQs

Does Cartage run pilots? No. Most customers start Wilson on a few lanes and expand from there. Impact is measured by comparing the lanes Wilson runs with how those lanes ran before.
How do you measure an AI logistics agent? By comparing 8 metrics to a baseline on the same lanes: manual touches per load, coordinator hours, time to book, time from exception to notification, POD collection time, pickups confirmed without anyone asking, missed pickups and approvals accepted without changes.
What is the most important metric? Manual touches per load, because it shows directly whether the AI agent is taking coordination work off the team.
How long does it take to deploy Wilson? It depends on scope. Standard freight with existing carriers is the fastest to set up; ERP integrations and custom workflows are scoped up front.
What if the metrics don't improve? Check the likely cause first: approvals that are too strict, steps the team is still doing manually, missing notifications or late order data. Adjust and extend if the cause is fixable.
How do you calculate ROI for an AI logistics agent? Start from coordinator hours returned and manual touches removed, then add the cost of avoided missed pickups, detention and slow POD collection on the lanes the AI agent runs.
Do I need a baseline before starting? Yes. Without "before" numbers on the same lanes, there's no reliable way to show what changed.
What happens when the first lanes work? The team adds lanes and modes and loosens approvals where results are strong, while keeping guardrails such as approved carriers and escalation.
How is Wilson priced? There's no subscription fee. Pricing is a percentage of the carrier payments that run through Wilson, and it covers the full lifecycle: create, quote, tender, track, audit and pay.

Conclusion

An AI logistics agent doesn't prove itself because a demo was impressive or because the team liked it. It proves itself when manual touches per load go down, bookings get faster, missed pickups drop and PODs arrive sooner, against a baseline everyone agreed on. If those metrics don't change, it isn't working yet, and the flat metric usually shows what to fix. Wilson, by Cartage, is built to be measured this way, with a full record of every action. The first step is to start on a few lanes, set the baseline and the targets up front, and compare the lanes Wilson runs with how those lanes ran before.

Sources

American Transportation Research Institute (ATRI), September 2024: https://truckingresearch.org/2024/09/new-research-documents-substantial-financial-and-safety-impacts-from-truck-driver-detention/
Cartage, Wilson product FAQ, October 2026: https://cartage.ai