3D & CAD visualization

Facts verified 2026-07-30 by claude-code/2026.07. Generated from the authored source at docs/llm-wiki/visualization-3d.md — the wiki owns these facts (ADR: D1 of docs/plans/20260811_documentation_system_execution.md). Edit that file, then regenerate: python docs/portfolio/_build/generate_wiki_pages.py.

The three kinds of 3D

  • The world (geographic context): Google Photorealistic 3D Tiles + Cesium Ion imagery/terrain, streamed directly to the browser (not through the Next.js server).
  • The site (reality capture): 3D Gaussian Splat reconstructions computed offline from campaign drone photos, served as Cesium 3D Tiles from Cesium Ion.
  • The design (CAD): engineering models rendered by the Autodesk APS Viewer — a separate module, no Cesium involved.

CesiumJS

  • npm dep cesium 1.134.1; static runtime assets copied by postinstall web/scripts/copy-cesium.mjs from node_modules/cesium/Build/Cesium to git-ignored web/public/cesium/; runtime sets window.CESIUM_BASE_URL = '/cesium/'.
  • Three modules embed Cesium: photo-map-3d (full inspection scene), gallery (owns the shared Viewer wrapper in core/utils/viewer.ts), gas-heatmap (own plain Cesium.Viewer, Ion world imagery + heatmap ImageryLayer).
  • Ion token: NEXT_PUBLIC_CESIUM_ACCESS_TOKEN, wired by ensureCesiumIonToken() in photo-map-3d/core/hooks/use-cesium-viewer.ts (build-time embedding caveats; window.checkCesiumIonToken() diagnostic). gas-heatmap reads the env var independently.
  • The Viewer wrapper: all stock widgets off, requestRenderMode, FXAA, fog/shadows off; base maps = Google 3D Tiles (default) or Bing via IonImageryProvider.fromAssetId(3); toggle3DGS() = every tileset except the Google one.

Google Photorealistic 3D Tiles

  • Loaded in Viewer.addGoogle3DTiles(): Cesium3DTileset.fromUrl('https://tile.googleapis.com/v1/3dtiles/root.json?key=…') with NEXT_PUBLIC_GOOGLE_MAPS_API_KEY; missing key or load failure is non-fatal (scene continues without base world).
  • Heavy tuning: maximumScreenSpaceError: 24, LOD skipping, 256 MB cache, foveated rendering, showCreditsOnScreen: true.

3D Gaussian Splats

  • Geometry lives on Cesium Ion as 3D Tiles; raw .splat files are explicitly not loaded (skipped in use-supabase-3d-gaussian-splats.ts, kept for compatibility).
  • Config, not geometry, is in Postgres: datasets.gaussian_splats (plus 3d_tilesets, default_camera, and the shared offsets blob — photo/camera offsets + marker alignment, merged server-side on write) JSON columns.
  • API (both authed + RLS): GET/PUT /api/datasets/{slug}/3d-config (via Dataset3DService); GET /api/gallery/assets/{assetId}/tileset resolves tiles — all-numeric assetId ⇒ Cesium Ion (https://assets.cesium.com/{id}/tileset.json), else search datasets.gaussian_splats (fetch-tileset-manifest.ts).
  • Loading hooks in photo-map-3d/core/hooks/: use-3d-asset-loader.ts (central tileset registry, dedup + StrictMode guard, load-state reporting), use-3dgs-state.ts (visibility toggles), use-supabase-3d-gaussian-splats.ts (config fetch).

CAD viewing (Autodesk APS)

  • Module web/modules/cad-viewer/; browser API wrapper services/aps-client.ts (retries via fetchWithRetry; uploads never auto-retry).
  • Pipeline: GET /api/aps/auth (token) → POST /api/aps/upload (.nwd accepted, to APS OSS, returns URN) → POST /api/aps/translate (Model Derivative job to SVF2, 2D+3D views) → GET /api/aps/status/{urn} (polled by translation-status.tsx) → view.
  • Viewer runtime injected at runtime from developer.api.autodesk.com (viewer3D.min.js v7.*, not an npm dep) in core/hooks/use-aps-viewer.ts; Autodesk.Viewing.GuiViewer3D.
  • Demo flow: GET /api/aps/demo-file, POST /api/aps/upload-demo.
  • ⚠ The /api/aps/* routes are currently unauthenticated (see Backend API security worklist).
  • CAD data standards (IFC, DEXPI, tagging) are a different domain: docs/handbooks/content/cad-pid/.
  • Focusing one asset — the map takes a one-shot focusAssetId prop (equipment id; modules/photo-map-3d/core/hooks/use-asset-markers.ts): fly to the marker at 70 m / −35°, highlight, force the label; the parent clears it on onAssetFocused. The 3D ↔︎ Assets bridge is the ?asset=<equipmentId> query param in both directions — /w/integrity/assets?asset= selects the asset in the register (kept in the URL), /w/integrity/evidence-explorer?asset= flies the map to it and is then stripped (one-shot). Sender: “View in 3D” in components/workspaces/assets-register-client.tsx; receiver: evidence-explorer-client.tsx. Tests: tests/components/evidence-explorer-deeplink.test.tsx, assets-register-view-in-3d.test.tsx, assets-register-deeplink.test.tsx. Handbook: visualization-3d “Focusing one asset”. The marker is the asset’s registered position (equipment_positions ← pds_assets), so it is only as right as that entry.
  • Rendering the plant from a camera (scripts/cad_twin/, indexed in its README) — the plant model at a photograph’s own pose (the CAD twin, blended with the photo to judge pose and placement), the splat and CAD model flown along the drone’s real path (the tour), and photograph + splat + Navisworks side by side from one camera (the triptych). Two renderers that cannot be composited in 3D: Cesium (the Gaussian splat and the CAD tileset, georeferenced by Ion, 0.4 s/frame on a GPU) and the Autodesk viewer (the whole Navisworks model, ~8 s/frame, and the only one that can turn a pixel into a cad_objects.id). Four traps, each of which cost a day: the Ion tile’s local frame is a rotated ENU (x is 28.9° from east), so camera directions built from yaw must be rotated into it; the lens is 65.05°, not the 73.74° the photo metadata implies; a splat needs Cesium’s wasm_splats_bg.wasm fetched same-origin or the tileset hangs with no error; and rendering must use the GPU, not SwiftShader (48.6 s/frame against 0.4). See docs/data/cad-twin.md and, for the rebuild of the poses and the splat from one COLMAP run on the TrueNAS GPU box, docs/data/reconstruction-rebuild.md. The renders ARE reachable from the web backend since 2026-09-15 — GET /datasets/{slug}/render/scene (splat + CAD mesh + Google basemap) and GET /datasets/{slug}/render/cad (the Navisworks model, with pick), both from an explicit camera or an image_id’s pose of record, and to the assistant as kap_render_from_image and kap_cad_view(render: true) — drawn by kap-render, a scale-to-zero L4 service on Cloud Run us-east4, invokable only by kap-dev’s identity, that kap-dev hands render/* to (web/lib/render/proxy.ts; KAP_RENDER_SERVICE_URL, KAP_RENDER_SERVICE_AUTH=iam) because kap-dev has no GPU — live on kap-dev since 2026-09-27 (revision 00216): gate met the same day (200/200 settled, render median 367 ms, 2–3 s round trip through kap-dev, ~26 s for the first call after idle), 60 renders a minute per caller, then 429. The image viewer shows the twin for every photograph with a pose of record (PhotoTwinPanel, lens=calibrated: the dataset’s measured lens from its render calibration, never a guess). Until that day the Navisworks render never settled because the page waited on an event the v7 viewer does not define (fixed: ~0.1 s a twin on a warm page). Measured and planned in docs/plans/20260915_scene_render_endpoints_execution.md; the proposal is docs/proposals/20260915_scene_render_endpoints.md.
  • Polycarbon’s reconstruction is TWO blocks, not one. At 19:38:20 the drone took no photographs for 29 s while it moved 19.8 m and swung 82°. Matches spanning that gap: 110 pairs / 2,014 inliers, against 707 pairs / 572,422 inliers within the block before it — 284× less evidence, barely above COLMAP’s validity threshold. Each block is internally consistent; their relative orientation is a guess, and the offset against the photographs reverses sign across the seam (−3.8° before, +3.3° after). Fit an alignment per block, never one across the flight, and do not expect more matching or the splat to fix it — the splat is trained on these poses. See docs/data/photo-cad-gps-alignment.md.
  • Measuring whether the model sits where the photograph says — four measures, ranked by trust: landmark reprojection (an identified CAD point against its pixel — the gate), best-shift residual, zero-shift gradient correlation, and identity checks. Do not optimise the correlation: repeating steel gives it false optima and maximising it made alignment worse. One degree is 18.4 px at 1200 px across a 65.05° lens. Two landmark clusters at different ranges separate an orientation error (shift ratio 1.00) from a position error (ratio = range ratio) — but only on fixed features: an offset read off a curved silhouette measures nothing, because a point on a smooth shell slides with the viewpoint, and a 2026-09-17 finding built on two of them was withdrawn the same day. Prefer an object’s centre (landmarks/capture_by_tag.py, by equipment tag) over a point on its skin; note that a tag names an assembly, so the object must be chosen rather than derived. A fit is evidence about the photographs it was measured on and nothing else — the landmark fit is recorded and off ("apply": true required), because every landmark sits in one ninety-second window and a hold-out only tests the axis you split on. The single source of truth for alignment numbers is the ‘Current state’ table in docs/data/measuring-alignment.md — reports narrate a day’s work and keep their retractions, so a number quoted out of one may be withdrawn. A cheap independent pose check that needs no landmarks: landmarks/audit_pose_vs_plates.py compares each camera’s heading with the bearing to the assets whose plates the OCR pass read in that frame; 47 of 50 agree, and the three that do not are frames not to spend control points on. Six recorded ways these numbers have gone wrong, including re-deriving the coordinate chain, treating a 200 as evidence, and twice believing a measure that cannot referee a camera change — an edge-map correlation, then an eye on a silhouette: docs/data/measuring-alignment.md, docs/reports/20260917_the_fit_was_the_wrong_shape.md.
  • Two different “what is here?” questions, and they are not the same. Right-click on the 3D map (use-map-context-menu.ts, map-context-menu.tsx) resolves the clicked point to lon/lat/alt and lists the five nearest registered asset markers with distances, via nearestMarkers() (equirectangular, deliberately not geodesic). Those markers are equipment_positions — so the answer is only as good as the register, which is wrong for most assets (see the anchor-point proposal). Picking in the Navisworks render (render/cad?pick=u:v) returns the actual CAD object under the pixel, exactly. Use the first for “what is registered around here”, the second for “what is this geometry”. Neither substitutes for the other.
  • Where an asset is defined, and picking — an asset is a tag; tag → cad_objects is an exact properties->>'Equip no' filter; the APS viewer’s objectid is a different id space BUT its Item/GUID property is cad_objects.id, so a picked pixel identifies a database row exactly (window.aps.pick in aps_twin.html, GET /datasets/{slug}/render/cad?pick=u:v). viewer.search needs the BARE attribute name ('Equip no', not 'PDS Component Data/Equip no' — the latter matches nothing, silently). The Ion CAD tileset has no batch table, so a Cesium pixel cannot say what it hit. Facts: Data & storage; full chain: docs/data/where-an-asset-is-defined.md.
  • Spatial alignment — why a marker sits where it does, not how it renders — is also a different domain: does a photo show an asset, is the CAD frame placed correctly in WGS84, and which CAD object represents a given asset tag (open as of 2026-09-14). See docs/data/photo-cad-gps-alignment.md.

Environment variables

  • NEXT_PUBLIC_CESIUM_ACCESS_TOKEN (Cesium Ion), NEXT_PUBLIC_GOOGLE_MAPS_API_KEY (Google 3D Tiles), server-side Autodesk APS credentials (read by /api/aps/* handlers).