System architecture
Executive Summary
Generic outbound is noisy, risky and expensive: bought lists, templated emails, follow-ups that keep firing after someone has replied, and no record of why a lead was chosen. We built the Wyncrest AI Lead Engine to run our own business development the way we build client systems — company-first research, explainable decisions and safety controls that cannot be switched off.
Every fetched company and person is stored before analysis, every selection or rejection is recorded with its reason, and a daily overview shows exactly what the pipeline did and what it cost.
Business Challenge
- Personalisation that is actually specific to the company, without hours of manual research per account.
- Paid data APIs (credits per search or reveal) that must never be wasted on poor-fit companies or duplicate calls.
- Strict compliance: US CAN-SPAM, UK PECR and GDPR — opt-outs, suppression, postal address and country rules.
- No double sends and no follow-ups after a reply, even if a worker crashes mid-send.
- A system one founder can run in minutes a day, with human approval where it matters.
Solution Overview
A modular-monolith Python platform with a PostgreSQL job queue drives a 12-stage pipeline: discover → group → research → qualify → enrich → generate → validate → approve → send → detect reply → classify → cancel follow-ups. LLM agents do the research, qualification and writing through a provider-neutral gateway with schema-validated structured output; deterministic code enforces every rule.
Core Platform Capabilities
Company-first discovery
Companies are screened and researched before any person is revealed, so paid contact credits are spent only on companies that qualify. A credit ledger reserves budget before each paid call and releases it automatically when a provider plan blocks an endpoint.
Agentic research & qualification
An LLM agent reads the company's public website, extracts evidence (services, hiring, technology, AI maturity), scores fit into HIGH / MEDIUM / LOW bands and picks an outreach strategy with a confidence score. Low-confidence results go to manual review instead of being guessed.
Explainable lead database
Every company and person is appended with fetch date, Selected / Rejected / Pending status and a plain-English reason. A Daily Overview dashboard shows fetched, selected and rejected counts, selection rate and the top reasons, with daily snapshots for history.
Writing with guardrails
Emails are generated from verified evidence only, checked by a second model for quality and unsupported claims, then rendered as a branded HTML + plain-text email with a compliant footer and one-click unsubscribe.
Reply-aware sequencing
An IMAP poller matches replies to threads, classifies them (interested, not now, unsubscribe, out-of-office, bounce) and cancels all pending follow-ups in a single transaction.
Technical Architecture
- PostgreSQL job queue with SKIP LOCKED leases, idempotent enqueue (dedupe keys) and crash-safe retries.
- At-most-once sending: compare-and-set claim from SCHEDULED to SENDING; an unknown provider outcome is never retried automatically.
- Env-only safety switches: kill switch, dry-run, simulation mode and a non-production recipient allowlist; suppression checks cannot be disabled.
- Layered configuration (defaults → environment → audited database overrides → campaign) validated with Pydantic.
- Versioned prompts, JSON-schema structured output, validation with a single retry, then deterministic fallback.
- Per-country compliance policies (US, UK, IE, AU, NZ allowed; stricter markets blocked by default) and business-hours sending in the recipient's time zone.
- Three Cloud Run roles (web dashboard, worker, public unsubscribe endpoint) with a fail-closed guard on GCP project and service account.
Business Impact
- Research that took 20–30 minutes per account now runs in seconds for about $0.014 in model cost.
- Every decision is explainable: why a company or person was selected or rejected is one click away.
- No double sends and no follow-ups after a reply, by design rather than by hope.
- The same architecture powers the research and outreach agents we build for clients.