Status: 🟢 Built & wired end-to-end (owner-marked first-class) · Audience: GoBuild staff (eng, sales, ops) — internal.
A contractor uploads a jobsite photo or a floor plan and Gemini vision drafts a scope of line items (photo) or a room-by-room takeoff (plan). The drafts land as realEstimate/Proposal/TakeoffItemrows and drop the user straight into review. Everything AI touches lands as a draft that a human must actively send — but note the gate is soft, not enforced (see §7 Human-review gate).
Status badges: 🟢 Live · 🟡 WIP / config-gated · ⚪ Dormant · ❌ Dead-end / vaporware
House terms: a Client is a contractor org on GoBuild; a Job is a project. On this page the parties are the contractor (the GoBuild-tenant staff running the estimate) and the homeowner/prospect (the person whose space is being estimated).
Cross-links: Jobs Command Center · AI features · Sales Command Center
takeoff_ai.pyEvery estimator flow on the hub — express lanes, the desktop estimator, the field app — funnels into two functions in app/services/takeoff_ai.py. The module docstring frames the design intent bluntly: "raw vision models are weak at exact measurement, so AI items land unconfirmed and the human verifies" (takeoff_ai.py:1-13).
| Function | Input | Output | Used by |
|---|---|---|---|
detect_from_plan() |
floor-plan image/PDF page | [{kind, label, quantity, unit}] — room areas (sf), element counts (ea), wall lengths (lf) |
plan express lane, plan viewer auto-takeoff |
estimate_from_photo() |
jobsite photo | {title, markup_pct, lines:[…], annotations:[]} — a rough scope of work |
photo express lanes, desktop + field estimator |
Both fail soft: if Gemini isn't configured or errors, they return an empty result and never raise into the request handler (takeoff_ai.py:11-12). Trust is bounded — max 25 takeoff items, max 30 scope lines (takeoff_ai.py:24-26).
configured() (takeoff_ai.py:225-228) gates whether the AI buttons even render — it just checks gemini_api_key.
estimate_from_photo doesn't just list work — for every sf/lf line it asks the model to return the real-world size in feet (dims), a confidence ("good" only if it could scale to a known-size object — a door ≈ 6.7 ft, a 4×8 sheet, an outlet ≈ 4.5 in, a person ≈ 5.7 ft), and an anchor phrase naming what it scaled to (takeoff_ai.py:96-119). Those become the rough on-photo measurements and drive a redline sketch drawn over the photo. The caller can pass an intent (what the client wants done) and a reference scale cue ("that wall is 14 ft") to tighten the estimate (takeoff_ai.py:166-186).
Output is sanitized hard: quantities coerced to positive floats, dims bounded to 1–1000 ft, units clamped to the allowed set, descriptions truncated (takeoff_ai.py:122-137, 150-214).
The estimator runs entirely on Google Gemini via app/integrations/ai/gemini.py. DeepSeek (the platform's text LLM) can't read images, so anything visual goes to Gemini (gemini.py:1-7).
gemini-2.5-flash (default, app/config.py:114), image-gen uses gemini-2.5-flash-image (config.py:115).generateContent endpoint; responseMimeType: application/json in JSON mode (gemini.py:195-240).maxOutputTokens cap on extract() — 2.5-flash is a thinking model whose reasoning tokens count against the output budget and run before the JSON; a small cap gets eaten by thinking and truncates the JSON (gemini.py:223-227).BLOCK_NONE — a normal jobsite shot (a bedroom, a person on site, a pet) can otherwise trip default filters and come back with zero candidates (gemini.py:228-238)._extract_json() retries transient empty/unparseable candidates up to 3× but stops early on 429 / RESOURCE_EXHAUSTED / quota / not-configured so it doesn't burn limited quota (takeoff_ai.py:52-76). _loads() strips markdown fences and falls back to the first {…} block (takeoff_ai.py:29-49).These are the "snap a photo, get a job" fast paths in app/routers/jobextras.py.
/portal/jobs/from-photo (jobextras.py:908-965)Create a Job straight from a site photo. It resolves-or-creates the Client, spins up a Job (a draft if no client was picked, so an abandoned upload doesn't litter the list — jobextras.py:930-932), saves the photo as a Document, then calls _photo_to_estimate() and drops the user on the redline sketch for review. Optionally seeds schedule + budget from a JobTemplate.
/portal/jobs/from-plan (jobextras.py:968-1006)Create a Job from a plan set. Saves the plan, optionally seeds a template schedule, then opens the unified plan viewer at /portal/documents/{doc_id}/view?takeoff=auto — the browser rasterizes the page (so PDFs work with no server-side PDF lib) and AI takeoff auto-runs.
/portal/jobs/{job_id}/takeoff/from-photo (jobextras.py:876-905)Same photo→estimate, but onto an existing Job.
/takeoff/ai-detect (jobextras.py:785-825)Gemini reads the plan and inserts each suggestion as a TakeoffItem with source="ai", confirmed=False (jobextras.py:809-814). These are explicitly unconfirmed drafts. It also feeds the learning loop (ai_feedback.record) so a contractor's confirmed labels/units personalize future runs via personalization.context() (jobextras.py:805-808, 816-824).
_photo_to_estimate() — the shared photo path (jobextras.py:828-873)estimate_from_photo() drafts the lines.Estimate with status = EstimateStatus.draft and notes="Drafted by AI from a jobsite photo — review before sending." (jobextras.py:845-849).EstimateLine: the AI unit_price is treated as the cost basis (unit_cost), and client price = unit_cost × (1 + markup) using the job's per-category markup (photo lines default to the material category — jobextras.py:852-863).sketch TakeoffItem carrying the on-photo annotations + a link back to the estimate + photo (jobextras.py:864-871).A separate, richer path in app/services/sales_estimator.py — photo-first, but the deliverable is a draft Proposal, not a job. Reached via the desktop /portal/estimator (app/routers/sales.py:66-124) and three field-app entry points (app/routers/field.py:1215, 1308, 1515).
build_quote_from_photos() (sales_estimator.py:188-300):
match_or_create_lead_contact, sales_estimator.py:143-185).sales_estimator.py:203-228).sales_estimator.py:276-287) and a payment schedule (deposit + draws — draft_payments, sales_estimator.py:99-118).Proposal(status=ProposalStatus.draft); the lead advances to estimating (sales_estimator.py:289-299).Crucially — "Lead-stage only — no job is created until the client accepts" (sales_estimator.py:10-11); acceptance runs through conversion.convert_proposal. append_area_to_proposal() lets a walkthrough grow room-by-room (sales_estimator.py:303-334).
Important architectural finding. The "homeowner uploads a photo on the marketing/showcase site" estimator does not run in this (hub) codebase. The GoBuild Showcase / control-plane side hosts that experience; the hub only receives the resulting Lead.
POST /admin/orgs/{org_id}/leads (app/routers/admin_api.py:659-701) — a signed, CP-only Admin API endpoint whose docstring says "Create a CRM Lead in this org from an off-hub source (e.g. the showcase estimator)" (admin_api.py:662). It accepts only name / email / phone / notes / source and fires the same lead_created follow-up trigger as an on-site form. No estimate figure and no photo cross the boundary — whatever rough number the prospect sees is computed off-hub, and only the contact lands in the CRM (a summary may ride along in notes)./f/{token} embed (app/routers/landing_portal.py:822-887) is a plain iframe contact form — name/email/phone/message/project_type → a Lead. There is no photo upload and no estimate on this route. If you were told /f/{token} is the prospect estimator, that's not what the code does today.Net: on the hub, the prospect estimator surfaces as CRM leads only. See Open questions on where the rough number is generated and whether it's ever shown.
Two distinct mappings depending on the flow:
A. Photo lines (immediate). The AI line's unit_price becomes the cost basis; cost_code is carried through if the model returned one (usually None for photo drafts), and lines default to the material category markup (takeoff_ai.py:203, jobextras.py:852-863).
B. Confirmed takeoff → priced estimate (app/services/takeoff_pricing.py). This is the deliberate, reviewable mapping:
takeoff_pricing.py:160-165)._ai_match, takeoff_pricing.py:104-118), which either returns {"match": <catalog index>} or proposes a new catalog entry {name, unit, unit_cost (contractor's cost, not retail), category, cost_code} — picking the closest code from the org's cost-code list or coining one like "06-100" (takeoff_pricing.py:82-101).price() returns lines tagged source = catalog | ai-match | ai-new | unpriced; the human reviews/edits, then commits via POST /documents/{doc_id}/takeoff/estimate-priced (documents.py:1225-1285), which builds EstimateLines and optionally adds accepted AI-proposed items to the catalog (documents.py:1271-1275).unit_cost 0) with a keyword-guessed cost code via costcodes.guess_code (takeoff_pricing.py:73-79), never an error.The manual takeoff→estimate builder (/jobs/{job_id}/takeoff/estimate, jobextras.py:743-782) only pulls confirmed.is_(True) items — the confirm flag is the real gate between "AI guessed" and "goes on an estimate."
Is there a human-review gate before an AI estimate reaches a Client? Yes — but it is a soft gate, not an enforced one. The safeguards that exist:
EstimateStatus.draft; sales proposals are ProposalStatus.draft. Nothing is auto-sent to a homeowner — the contractor must actively send/share.jobextras.py:848).confirmed=False (jobextras.py:813) and only confirmed=True items reach an estimate (jobextras.py:758-760).takeoff_pricing.price() writes nothing; the commit is a separate reviewed POST (documents.py:1225-1285).rough/good confidence and an anchor so a contractor can see what a number was (or wasn't) scaled against (takeoff_ai.py:110-115).What is not enforced (the liability surface):
unit_price figures are "typical North American residential ballparks" by the model's own prompt (takeoff_ai.py:116), and vision measurements are rough unless a scale reference was given. A wrong number that reaches a homeowner is the contractor's number the moment they send it — there is no GoBuild-side sign-off, disclaimer injection, or margin-floor check between the AI draft and the sent document.takeoff_ai.py:2-5); the mitigation is the draft-and-confirm workflow, which relies on the contractor actually doing the review.Bottom line: the architecture makes review natural and expected but does not make it mandatory. See known risks and the open questions below.
STAFF (on-hub, authenticated)
photo → estimate_from_photo() → Estimate(DRAFT) + redline sketch → review → send
routes: /jobs/from-photo, /jobs/{id}/takeoff/from-photo, /portal/estimator, field app
plan → detect_from_plan() → TakeoffItem(confirmed=FALSE) → confirm → /takeoff/estimate → Estimate(DRAFT)
routes: /jobs/from-plan, /jobs/{id}/takeoff/ai-detect
confirmed takeoff → takeoff_pricing.price() (catalog → Gemini) → review → estimate-priced (commit)
PROSPECT (off-hub: showcase / control plane)
homeowner photo → [rough estimate computed OFF-HUB] → Admin API POST /orgs/{id}/leads → CRM Lead only
(no estimate figure, no photo reaches the hub)
rough-confidence, model-ballpark number can be sent to a homeowner unedited. Should the send action require a distinct "reviewed" acknowledgement, or inject an "estimate, subject to site verification" disclaimer?admin_api.py:659). Does the showcase/CP side run estimate_from_photo (or its own model), and does the prospect see a dollar figure — or just a "we'll be in touch"? This determines the real off-hub liability exposure.confirmed=False step that plan takeoff enforces — is that intentional, or an inconsistency worth closing?/jobs/from-photo, /jobs/from-plan, or the desktop/field /estimator? No usage telemetry was found in the estimator code; drives where to invest accuracy work.estimating proposals and then won jobs? Is the rough number a conversion driver or a source of anchoring disputes when the real quote differs?ai-new proposals — how often does Gemini's proposed cost_code (takeoff_pricing.py:88-95) match the contractor's real coding scheme vs. inventing an orphan like 06-100 that pollutes the catalog on accept?sales_estimator.py:203-228), plus a pricing call. Is there per-org rate limiting or cost attribution, given the pooled-vs-BYO-key credential model?/jobs/from-plan relies on the browser to rasterize PDF pages before takeoff (jobextras.py:974-975). What's the failure mode on unusual PDFs or headless/embedded viewers, and is there a server-side fallback?