Applied AI · Founding Engineer · 2025 to 2026
Design-to-estimate platform for a construction company
Drawings and site photos in, a priced line-item estimate out, with a takeoff that refuses its own numbers when they do not close.
Estimate #1042
Bathroom renovation · 3 categories
ItemQtyRateTotal
Demolition & disposal1$1,200$1,200
Waterproofing60 sf$14$840
Tile supply & install60 sf$22$1,320
Plumbing fixtures1$2,400$2,400
Subtotal$5,760
GST 5%$288
Total$6,048
Representative UI, abstract, with no client data.
±0.3%
measured floor area against the drawing's own dimension labels; closure is a gate, not a report
Vision
extracts construction scopes from scanned drawings via a text-vs-image fallback
Eval-gated
offline eval harness + pgvector RAG, regression-tested in CI
How it works
1Upload
drawing, permit or contract dropped into a project
↓→
2Vision extract
async queue, per-kind model routing; text-vs-image fallback reads scanned drawings
↓→
3Takeoff + closure
areas measured off the drawing, checked against its own dimension labels
↓→
4Signed review
no estimate runs until a person approves the takeoff
↓→
5Itemized estimate
cost-library-backed estimate + milestone schedule
↓→
6Contract + portal
e-sign contract and a live client portal
Architecture
Frontend
Office UI (Next.js)
Client portal
↓→
Backend
FastAPI
Takeoff engine
Closure gate
Async AI job queue
Vision extractor
Estimate engine
↓→
Data
PostgreSQL 16
pgvector
↓→
External
Anthropic (Opus/Sonnet/Haiku)
Cloudflare R2
By the numbers
Measured against the drawing's own labels; closure decides what ships
Floor area, measured 670.1 vs labelled 668 sq ft+0.3% · passes
Area after perimeter correction-0.2% · passes
Perimeter, engine returned 156.9 vs actual 120.1 lfrejected by the gate
Context
A licensed builder's back office needed measurement and AI on the critical path of money-adjacent artifacts: measure a building off its drawings, turn that into a scope, generate an itemized estimate. A quantity that is wrong by a third is a bid that loses money for a year, so the hard part was never generating a number. It was knowing when not to trust one.
What I did
- Built a region-extraction engine that measures floor and roof areas directly off CAD drawings, and validates every measurement against the dimension labels printed on the sheet.
- Made area closure a gate rather than a report, so no quantity moves downstream until the geometry agrees with the drawing.
- Built an async AI-job worker (queue + retries) so a 60-second vision call never blocks a request and failures are observable, not lost.
- Routed each job kind to the right model tier with an exact Decimal cost model and truncation detection (surfacing stop_reason instead of choking on partial JSON).
- Added an offline eval harness of golden cases, deterministic scorers and cost and latency reporting, running with no API key and gating regressions in CI.
- Layered pgvector RAG for grounded, cited answers over project documents, with the embedder/store injected behind interfaces for offline testing.
Outcome
- The platform is in daily production use across the office and client portal.
- Takeoff quantities carry a stated confidence and a human signature before they can price anything.
- Changing a prompt or swapping a model is now a measured change, with accuracy, cost and latency drift visible before they ship.
- A green CI badge and a public eval harness are the sanitized proof of an otherwise-private system.
Under the hood
Measuring a building from a PDF
- A region-extraction engine measures floor and roof areas directly off CAD drawings, scoped to the project boundary rather than the whole model, a distinction that quietly doubles a quantity if you get it wrong.
- On the reference project the measured floor area came out at 670.1 sq ft against the drawing's own dimension label of 668, a 0.3% difference, and that tolerance is what the rest of the estimate is built on.
- Domain rules the engine has to encode: roof area is the largest storey's projection excluding overhang and slope, and the area labels printed across a sheet are not summable, because adding them double-counts.
Area closure as a gate
- Every measurement is validated by closure: the geometry has to agree with the drawing's own labels before any quantity is allowed downstream.
- The check earned its place immediately. The engine produced a perimeter of 156.9 linear feet and closure rejected it; re-measured off the sheet, the correct figure was 120.1 lf, and the area then closed to within 0.2%.
- That result is now encoded as a rule: area validity does not transfer to length validity. Two quantities derived from the same geometry can carry very different confidence, and the gate treats them separately rather than passing both because one checked out.
A human signature between takeoff and price
- CAD projects cannot produce an estimate until the takeoff has been reviewed and signed. The gate is structural, not advisory.
- Sheets sum, exclusions bind, and each drawing is read exactly once, so a re-run cannot silently change a quantity a person has already approved.
AI as a real system, not an inline call
- AI tasks are rows in a job table processed by a dedicated worker with retries, so a slow vision call never blocks a request and failures are observable.
- Each job kind routes to the right model tier (Opus for vision, Sonnet for generation, Haiku for OCR) and records an exact USD cost per job.
Vision scope extraction
- Uploaded PDFs get text extraction first; when the text is sparse, which is the signature of a scanned drawing, the pipeline renders the page to an image and uses the vision path.
- Render DPI and image downscaling are tuned deliberately to read a title block without blowing the token budget.
Evaluation + retrieval
- An offline eval harness regression-tests model outputs (pass-rate, cost, latency) and gates CI, so a prompt or model change is a measured change, not a surprise.
- A pgvector retrieval layer answers project-document questions with grounded, cited responses.
Stack
FastAPIPostgreSQLSQLAlchemyNext.jsAnthropic (Opus/Sonnet/Haiku)pgvector