Designed & piloted

Cohesity · AI Sales Enablement Intern

Designing Trust Into an Autonomous Reporting Agent

An AI agent reports on sales-training completion on its own, and only messages a real person when it's confident enough to be right.

RoleProduct Designer
Duration12 weeks
TeamEnablement lead, platform engineer, program owners
ToolsAutomation platform, data warehouse, Slack

The agent decides, on its own, whether it's confident enough to message a real person, and refuses to guess when it isn't.

The problem

The obvious fix would have broken on contact with reality.

Enablement specialists manually pulled training-completion data every week, emailing reps about their own progress on courses they'd been assigned, and emailing managers about their team's reps' progress on those same courses. My assignment was simple on paper: replace that manual process with automation.

Instead of building the obvious version right away, I first interviewed the specialists doing this work today. Two things they said changed how I approached the whole design:

  1. Trust was uneven. People worried less about the system missing something than about it being confidently wrong
  2. There was no single schedule. Different programs check in weekly, some monthly, some quarterly, so one shared send schedule wouldn't work for any of them

Those two things shaped almost every decision that followed.

The research

Four findings, from interviews and a data audit, that shaped every decision after

Interviews with the specialists doing this work today, plus a column-by-column audit of the source data rather than trusting what its field names claimed.

Insight 1 · trust, not capability
"My worry isn't that it breaks. It's that it's confidently wrong and nobody notices until a manager complains."

The strongest pushback wasn't about what the system could do. It was about trust in something unattended.

Insight 2 · no single schedule
"Some of my programs check in weekly. Others are a 30-day thing, some are quarterly. There's no one schedule."

A single company-wide send schedule would spam the weekly programs and leave the longer ones silent for weeks.

Insight 3 · tribal knowledge beats the record
"Honestly, I just know which of these are actually mine at this point. The system hasn't been right in a while."

Specialists' own knowledge of who really owned what was more accurate than the system of record.

Insight 4 · a wrong send, not a missed one
"The thing I'd actually be embarrassed by is a rep getting told they're behind on something they finished last month."

People feared one wrong message far more than the system occasionally saying nothing at all.

Key decisions

Six decisions, each traced back to something someone told me

Not features that seemed nice to have. Every gate and threshold below answers a specific worry from the research above.

DecisionWhy
One lookup table, not five workaroundsA single, specialist-verified source of truth beats patching each data gap on its own. One thing to explain, audit, and hand off.
Confidence score: High or Low, no MediumA gray zone quietly puts a human back in the loop every time it fires. Two states force a real decision up front.
Low confidence escalates, it doesn't fail silentlyNothing sends to any rep or manager. The course owner gets the exact reason instead of a vague failure.
Per-course cadence, not one shared scheduleThe weekly check only decides when the agent looks. A separate cadence per course decides whether it actually sends.
A single-switch test mode for rolloutEvery message routes only to the designer until the content earns trust. Going live for real is one flag, not a rebuild.
Deterministic message text, not AI-generated wordingPredictable, auditable text comes before a warmer tone. A wrong sentence in a manager's inbox can't be unsent.

How it works, end to end

The agent re-evaluates itself every run. It never assumes last week still holds.

Every run wakes up on a fixed schedule, walks every course that's due through the same pipeline below, and logs what happened before going back to sleep. Nothing carries over from the run before it.

System design: the decision flow
Data inputsTraining completion recordsManager hierarchy andassigned specialistWeekly schedule · joins both sourcesCourse discovery rulesEnd date passed → close course, stop100% complete → close course, stopRenewed or new course → configureDecision pipeline1 · Query the data warehouse2 · Lifecycle gate3 · Confidence scoringHigh or Low. No in-between4 · Governance gateHigh → proceed · Low → escalate5 · Build message content6 · Send and logoutputOutputCC: enablement lead on every messageREP UPDATEHi [rep name],You're currently [status] on [course].Please finish by [date].MANAGER UPDATEHi [manager name],Your team is [percent]% complete on [course].Follow up with: [rep names].

Shipping without an incident

The rollout was its own design problem, not an afterthought

Because the agent's entire value is acting without a human checking every message, the path to live had to make a bad send structurally impossible, not just unlikely.

  1. Build and prove the full decision path against realistic stand-in data before requesting access to any real, sensitive system
  2. Run repeatedly in test mode, reviewing real message output with the enablement lead, before a single real send
  3. Treat every open question, an unverified owner, an unset link, a mismatched date, as a blocker to flipping the switch, not a footnote to fix later
  4. Design the data source to be swappable, so turning on the real feed changes nothing else in the decision logic

The problem

Specialists manually tracked training completion and messaged people about it, by hand, every week, for every course.

What I learned

The pushback was never about capability. It was about trust in a system that acts without anyone checking each message first.

What I built

Decision logic that scores its own confidence, only speaks when it's sure, and tells a human exactly why when it isn't.

What this taught me

"The interesting question isn't whether this can be automated. It's what the system needs to refuse to do, and how loudly it needs to say so when it refuses."

Next, if this continued: letting the system draft its own new-course setup requests instead of asking a human to type them in, and testing whether a warmer tone changes how people respond, once the plain version has earned enough trust to risk it.

4Research insights
6Design decisions
2Confidence levels, by design
12Weeks