Before
Manual sampling
- Selected fragments
- Manual listening
- Fixed checklists
- Limited sampling
- Human training for each review
- Linear staffing cost
Case 01 · Echo
Replacing manual call quality control with a production AI platform.
Product image placeholder · sanitized public case study
30,000+reviews per month
1,000+on a normal working day
50+review types
~1 minuteend to end
100–200×lower estimated operating costCompared with equivalent manual review capacity.
The process that stopped scaling
Quality control could inspect selected fragments of selected calls. It could not continuously understand the company’s full conversation volume.
Reviewers opened calls on request, listened to a selected fragment, and filled in a form. The work was valuable, but coverage depended directly on available time and headcount.
Every new check introduced another script or checklist to learn. Increasing review volume meant increasing manual capacity at almost the same rate.
Manager request→Open call→Listen to fragment→Fill form→Next call
What Echo changed
The review logic moved from individual listening sessions into a repeatable product workflow.
After
Prompt-defined coverage
- Complete conversations
- Automated transcription
- Prompt-defined reviews
- Structured database results
- New checks created as prompts
- Near-complete call coverage
Echo replaced the routine manual review workflow. The former quality-control lead transitioned into prompt engineering and automated-review evaluation, managing review logic instead of a team of manual reviewers.
Used across real-estate sales, call-center, and HR workflows, with results available to reports and operational dashboards.
Product result
A twenty-minute conversation becomes data an operation can use.
Sanitized input
{
"audio_url": "https://example.com/call.mp3",
"review_type": "client_information"
}Unstructured conversation → structured operational data
Structured output
{
"facts": ["Budget was discussed"],
"missing_information": ["Purchase timeline"],
"next_action": "Confirm the timeline",
"status": "ready"
}From script to platform
The production architecture was earned one observed constraint at a time.
Local experiments
Python transcription scripts, ASR comparisons, and manual prompt tests.
Usable prototype
A web workflow connected prompts, calls, and structured model responses.
Controlled pilot
Three review types of different complexity tested the workflow on real calls.
Operational transition
Routine manual control moved into Echo as coverage and confidence increased.
Production platform
Prompt library, LLM gateway, datasets, reports, remote GPU work, monitoring, and alerts.
Core internal system
50+ review types and 30,000+ automated review jobs every month.
What I owned
I was the sole engineer responsible for the core Echo platform.
Product
Workflow design, initial review logic, prompt library, user experience, and product iteration.
Engineering
Architecture, backend, frontend, asynchronous jobs, Redis Streams, persistence, and service integration.
AI
ASR benchmarking, LLM comparison, prompt evaluation, real-call review, and model migration.
Operations
Deployment, GPU lifecycle, monitoring, Telegram alerts, production support, and failure investigation.
Contribution boundariesData engineers created the MSSQL tables and automated the originating database procedure based on my requirements. Reporting specialists later built additional dashboards using Echo data.
Implementation practiceLLM coding agents accelerated implementation, refactoring, benchmark tooling, and large-log analysis. I remained responsible for requirements, architecture, decisions, code review, human evaluation, user testing, deployment, and production operation.
Engineering decisions
Four decisions shaped reliability, cost, and the speed of iteration.
Decision A
Queue expensive work
Transcription and LLM analysis outlive a normal integration request. Echo accepts work through the API and processes durable jobs through Redis Streams.
- Trade-off
- Consumer groups, retries, recovery, and observability became necessary.
- Outcome
- Request acceptance is decoupled from processing, and jobs survive worker restarts.
Integration→Accepted job→Durable queue→Worker→Stored result
Decision B
Transcribe once, review many times
Transcripts are stored independently from reviews and checked before new ASR work is scheduled.
- Trade-off
- Caching, deterministic identity, stale-data handling, and controlled regeneration.
- Outcome
- One expensive transcript can support many business checks.
One recording→One transcript→
Review AReview BReview C
Decision C
Pay for GPU work, not GPU idle time
ASR moved from a continuously rented server to an API-managed remote GPU worker scheduled around actual working hours.
- Trade-off
- Startup, remote connectivity, model environment, and queue reliability must be managed.
- Outcome
- GPU infrastructure cost fell by approximately seven times.
Echo→Queued ASR job→Managed GPU→Transcript
Decision D
Benchmark models on real calls
ASR and LLM candidates were compared through human evaluation on anonymized real-call samples, not popularity or synthetic leaderboards alone.
- Evaluated
- Meaning preservation, missed facts, hallucinations, schema compliance, consistency, latency, and cost.
- Outcome
- Production models changed when observed call performance justified the move.
Real call samples
AccuracyConsistencyLatencyCost
Human-reviewed decisionOptional technical deep dive
Follow one Auto Review job through the complete processing pipeline.
Production reality
The first thirty seconds were missing. The obvious fixes did not generalize.
A repeatable gap
Calls with a recorded promotional introduction lost approximately the first 30 seconds of the real conversation.
Several plausible causes
Volume, normalization, VAD, preprocessing, and Whisper configuration were tested. Some changes helped individual calls but not the affected set.
Test against evidence
An ASR model trained for Russian telephone speech was benchmarked on the problematic anonymized calls.
Introduce an alternative
T-One consistently preserved the conversation start and became an alternative production ASR backend.
BeforeRecorded introMissing conversation startRemaining call
AfterRecorded introPreserved conversation startRemaining call
Its streaming architecture may contribute, but the production decision was based on observed performance on the affected calls.
Impact
From selective manual review to operational coverage.
100–200×lower estimated operating costCompared with equivalent manual review capacity.
30,000+automated reviews per month
1,000+reviews on a normal working day
50+prompt-defined review types
~1 minuteend-to-end review time
Fewer than 10reported issues in a typical monthReported user issues, not a measured technical failure rate.
Echo made complete-call analysis practical across sales, call-center, and HR workflows. New reviews became prompt definitions rather than new manual training programs.
The result is structured database data available to custom reports and dashboards, while human expertise remains central through prompt engineering and evaluation.