Data Engineering · Applied AI · 2026
Multifamily real-estate deal-sourcing engine
Turns a fragmented, anti-bot web of brokerage portals into one deduplicated, scored, reviewable deal pipeline.
Representative UI, abstract, with no client data.
How it works
Playwright-first; Scrapfly / Exa / Jina fallbacks for WAF-heavy sites
Haiku parses messy listing pages, with a regex fallback and no API key
address fingerprint + 4-signal buy-box radar score
map + table cockpit; star and promote deals
doc extraction, conflict resolution, IRR and DSCR engine
Architecture
By the numbers
Scraping cost tiering, free-first, pay only where needed
Context
Sourcing small multifamily deals across US markets is a data problem before it's an investment one: inventory is spread across portals that render differently, gate behind anti-bot walls, and describe the same building inconsistently.
What I did
- Built ~20 connectors behind one base class, Playwright-first (free) with paid fallbacks reserved for the hard, WAF-heavy sites and a key pool that fails over on quota.
- Extracted messy listing pages with an LLM (cached prompts) that falls back to regex with no API key, so the pipeline runs deterministically and cheaply.
- Deduplicated by address fingerprint with enrichment-updates that never reset human decisions, and scored each property against buy-box profiles stored as data.
- Promoted shortlisted deals into an LLM-assisted underwriting flow of document extraction, conflict resolution and a side-effect-free financial engine (IRR, DSCR, scenarios).
Outcome
- An analyst works one live map/table cockpit instead of a dozen browser tabs.
- A real over-expiry bug taught a guarded status-reconciliation design (exhaustive-only, ratio-capped), the kind of distributed-consistency care that shows up at scale.
Under the hood
Connectors and anti-bot tiering
- Every platform connector extends one base class; scraping is Playwright-first and free, with paid tools (Scrapfly, Exa, Jina) reserved for the sites that need them.
- A comma-separated key pool auto-fails-over on quota, treating scraping reliability as a budget-allocated system rather than one brittle scraper.
Dedup and guarded reconciliation
- Candidates are keyed by an address fingerprint; re-discovery enriches null fields and never resets a human decision (a dismissed deal stays dismissed).
- A status-reconciliation pass expires unseen listings, but only for exhaustive platforms, with a min-count floor and a 50% ratio cap after a real over-expiry bug taught the lesson.
Underwriting engine
- Promoted deals run offering memoranda through Sonnet extraction with per-field confidence, then a conflict-resolution layer when documents disagree.
- A side-effect-free financial engine computes IRR (Newton-Raphson), DSCR and cash-on-cash across base / conservative / optimistic scenarios.