HITL Service Blueprint

v0.4 · Updated May 14, 2026
Part 1 · Bets

Four bets built the human-in-the-loop.
Each one is being tested.
The roadmap is already responding.

  1. One Expert

    A DART specialist owns each approval. Expertise is what makes 37 seconds enough.

  2. One Decision

    Approve or reject. No third option. Binary fits in 37 seconds; nuance doesn't.

  3. 37 Seconds

    The review window HITL's AHT math depends on. Anything longer and the gain evaporates.

  4. Customer never knows

    The customer sees a bot resolution. CSA review stays backstage. Trust rides on seamlessness.

Part 2 · Scale

What HITL did at launch. And at scale.

OORW (out of return window) A/B test in blue. Five months later, the operational story got more honest.

0.0% less time to resolve 14.75 min control → 11.24 min HITL
0× bot resolution rate 4.62% → 32.84% of contacts
0% of decidable cases approved 48% rejected · 1,504 of 2,960 contacts
0 contacts tested, 4 weeks 6,887 customers · Nov 9 to Dec 6, 2025
See the launch-experiment bars
Handle time per contact
Control 14.75 min
HITL 11.24 min
3.51 minutes saved per contact
Of decidable cases
52% 48%

Five months later, the numbers held. The story got more honest.

0K HITL reviews at 2.0% coverage

34% 53% 13%
4.8 min from 17.5 min baseline · 12.7 minutes saved · 96.5% bot RAP
+45 sec vs direct handoff (5% additional) · 90.9% CSA RAP for rejected, above 84.0% non-HITL baseline

CSA engaged time drifted: 37 seconds at launch, 47 at scale. Four CX gaps are in flight as Phase 1 of the strategy.

OORW A/B test: 7,219 contacts over 4 weeks, Nov 9 – Dec 6, 2025, 99.9% confidence (p<0.001). YTD: weeks 1–18 of 2026, through April. Sourced from HITL Product Strategy v6, May 2026. · Sources & methodology

The architecture delivers both. Here's how each side plays out, stage by stage.

Part 3 · Journey

Seven stages across the HITL review

What the customer experiences — and everything they never see that makes it work.

Shown: OORTW (late return). HITL pattern applies across other CSA-decision use cases.

Stages Time flows left to right
1 Customer initiates Customer asks
2 CS Chatbot gathers context CS Chatbot gathers
3 Confidence boundary CS Chatbot pauses
4 Routes to DART System routes, customer waits
5 CSA reviews in 37s CSA accepts, reads, decides
6 Decision CSA decides
7a Approve path CS Chatbot resolves
7b Reject path Transfers to standard CSA
Customer Actions What the customer does and feels
Describes issue Types problem in chat
Waiting, comfortable Normal pacing
Still within tolerance Typing dots carry them
Wait begins

"Is it not loading? Is it my Wi-Fi?"

Wensu N, DNR · HITL Customer Usability, Apr 2026
OORW +28%
DNR +16%
Start Return +43%

Gap widens where customers anchor to a faster flow.

Resolution delivered
Caveat / completeness

Approved doesn't equal satisfied when the options don't match expectations.

"The only button is 'Get a refund'."

Wensu W, Start Return · HITL Customer Usability, Apr 2026
Critical CX risk Rejection hits customer

"Frustrated and disregarded. I'd at least like to plead my case to somebody live rather than an agent."

John T, OORW · HITL Customer Usability, Apr 2026
  1. To be heard
  2. To provide context
  3. To understand the reasoning
  4. To access human empathy
Frontstage What the customer sees in the CS Chatbot
Chat input Free-text
Stage 1: Customer landing screen, 1 of 3 1 of 3
Typing dots "CS Chatbot is thinking"
Stage 2: Chatbot greet, 1 of 7 1 of 7
"Let me look into this" Same message regardless of path
Stage 3: Chatbot asks for reason, 1 of 3 1 of 3
"Looking for a solution..." Same screen across stages 4-6 · CSA involvement not surfaced
Stages 4-6: Customer wait state
Approved action Refund / replacement / credit
Stage 7a: Approval resolution offered
Transfer or dead-end Structured chat limits
Stage 7b: Rejection transfer wait, 1 of 2 1 of 2
Line of Visibility
04 Customer never knows Holds while the CSA review stays invisible · cracks on rejection
Backstage What the CSA sees on AC3; bot internals
CS Chatbot gathers context Order API · profile query
Confidence boundary Cannot resolve autonomously
Enters DART queue Specialist queue. CSA picks up within seconds.
Stage 4 backstage: AC3 lobby, CSA sees incoming contact AC3 lobby · stage 4
Specialist reads the case One expert, 37 seconds, under time pressure.

CSA sees issue summary, customer utterance, solve card, and chat transcript.

01 One Expert 03 37 Seconds
Approve or reject click
52% approve
48% reject
Handle time after the click
Approve
3.73 min
Reject
12.32 min
Reject path is 3.3x longer
02 One Decision
Stages 5-6 backstage: AC3 HITL review screen AC3 review screen · stages 5-6
CS Chatbot executes 0.0% of approvals need zero further CSA time
Transfers to standard CSA Today: DART writes transfer note · case re-queues to chat CSA Re-queue, re-auth, new context-load. Future: DART takes over the chat directly.
Line of Internal Interaction
Where CSA action meets routing infrastructure
Support Processes Routing, plumbing
Connect / CCMS 14% forced transfer · plumbing, not HITL logic
Live-agent transfer Re-queue, re-auth, context-load

What the map implies

Part 4 · Evidence

Where the bets hold. Where they crack.

Each bet works today within a specific envelope. Customer research shows the edges of that envelope and the teams already working on them.

  1. One Expert

    DART specialists review OORW approvals. Specialization is what makes the 37s window defensible.

    Pilot surfaced three context gaps in 2025. The critical one is live; the others haven't resurfaced.

    • Concession history live
    • Warranty info not resurfaced
    • Return label history not resurfaced
    Evidence loop closed

  2. One Decision

    The CSA's approve-or-reject call lands cleanly on most OORW contacts. Binary fits the case shape.

    The customer needed a third path and the CSA had two buttons.

    The only button is 'Get a refund.'

    Wensu W, Start Return

  3. 37 Seconds

    Customers wait comfortably through simple-seeming issues as long as the decision arrives before anxiety does.

    Loop anxiety starts at roughly 45s. Perceived time is worst for simple-seeming issues like Start Return.

    Is it not loading? Is it my Wi-Fi?

    Wensu N, DNR

  4. Customer never knows

    The review is invisible and the outcome matches expectations. Async chat masks the review latency. Approvals feel like a fast bot.

    The invisibility bet has no graceful failure mode on rejection. 10 of 14 customers saw value in knowing a human had reviewed their case.

    Frustrated and disregarded. I'd at least like to plead my case to somebody live.

    John T, OORW
Part 5 · Takeaway

Architectural choices are experience design choices.

Four bets tested. The roadmap responds. Phase 1 fixes the cracks this blueprint surfaces. The journey map's job from here is to stay in the architecture conversation as each phase reshapes the bets.

Now · Phase 1

Fix the CX gaps

Closes the four experience gaps this blueprint surfaces — approve / reject / takeover, pre-check routing, clear option-surfacing for bot vs human paths.

In parallel

Phase 2

Learning flywheel

Decisions feed the science stack.

Phase 3

Platform decoupling

Extend beyond ULisa to Ally and future platforms.

Phase 4

Operational model at scale

37 DART specialists → 50–100 L2 CSAs.

Phase 5

Expand use cases

Once Phase 1 is delivered, hit 6.5% coverage and expand into erroneous charges, cancel pre-ship, and subscription management.