Backend Blueprint

How the backend captures, stores, searches, and transforms build images.

Image capture pipeline

1. Turn the request into a stream

src/routes/api/builds/stream/+server.ts validates Year, Make, Model, and an allowlist of filters. “All pages” becomes 0, which Python treats as up to 250 pages with early stopping.

• Node starts python3 src/web_stream.py year make model max_pages filters_json.
• Python prints one compact JSON event per line.
• Node forwards stdout as application/x-ndjson with proxy buffering disabled.
• Cancelling the HTTP request sends SIGTERM to the Python process.

2. Scan gallery pages

Direct requests commonly hit AWS WAF, so fetch_gallery_text() URL-encodes the search and asks Jina Reader for a text version of the page.

https://r.jina.ai/http://www.customwheeloffset.com/wheel-offset-gallery?year=2026&make=GMC&page=2

parse_builds() processes that text as follows:

1. Find every Custom Offsets thumbnail URL with THUMB_ANY_RE.
2. Split the filename into build ID, photo index, year, and remaining slug.
3. Find a known make token in the slug. Text before it becomes Model; text after it becomes setup data.
4. Reject mismatched Year or Make values and deduplicate records by build ID.
5. Replace /thumb/ with /web-compressed/ to create the initial full-image URL.

Each new build is emitted immediately with its first image. The scan stops after an empty page, two pages with no new IDs, or pages that no longer contain the requested year. This logic is in src/gallery.py and src/archive.py.

3. Expand one build into its complete photo group

The listing only exposes the first photo. discover_details() requests the build detail page through Jina:

https://www.customwheeloffset.com/wheel-offset-gallery/{build-id}/build

DETAIL_IMAGE_RE captures the build ID, numeric photo index, and filename remainder from every web-compressed CDN URL. Matches belonging to another build are discarded. The remaining URLs are stored by photo index and returned in numeric order. If none are found, the first image from stage 2 is used as a fallback.

The same page contains fitment data. parse_specs() uses label-specific patterns to extract wheels, offsets, tires, suspension, rubbing, trimming, stance, and spacers. Markdown links and formatting characters are removed before storage. This logic is in src/archive.py.

4. Download and normalize the files

The stream archives three builds concurrently. Within each archive job, thread pools allow up to four detail requests and six image downloads. Existing non-empty PNGs are skipped, so repeat searches only fetch missing files.

For every missing image, download_image():

1. Requests the CDN URL with a browser User-Agent and Custom Offsets Referer.
2. Writes the response to a temporary binary file.
3. Uses Pillow to detect the real format because a .jpg URL may contain WebP bytes.
4. Decodes the image, converts it to RGB when necessary, and writes a numbered PNG.
5. Deletes the temporary download.

data/archive/images/{build-id}/001.png
data/archive/images/{build-id}/002.png
data/archive/images/{build-id}/003.png

5. Index and emit the completed group

SQLite uses WAL mode and three main tables:

• builds stores Year, Make, Model, and the complete serialized build.
• images maps each unique source URL to its build, photo index, and local filename.
• build_specs stores raw, normalized, and numeric versions of each specification.

Text is case-folded and reduced to alphanumeric tokens. Wheel dimensions, offsets, and tire measurements are also parsed into numbers with units. Indexed SQL EXISTS clauses can therefore combine exact text, partial text, and numeric filters without reparsing build JSON.

After the transaction commits, Python emits the build ID again with its complete local image list and specs. The stream consumer replaces the earlier thumbnail-only version. On a later search, matching SQLite records are emitted before live scanning begins.

Storage layout

Image transformations

Routes accept only numeric build IDs and three-digit PNG names. They read the edited copy if one exists; otherwise they start from the original. Originals are never overwritten.

Mirror

Goal: flip the complete photo group without touching the originals. I use Pillow because this is a simple pixel operation. It writes temporary files first, then replaces all edited copies together. Other options are ImageMagick or Sharp for Node.

Background removal

Goal: make a clean transparent PNG and preserve wheels, windows, and small vehicle parts. I use BiRefNet with the official weights. It runs locally on GPU or CPU and the model stays loaded between images. Production sends this operation to a RunPod RTX 4090 Flex worker; local development can use the same model on CPU or GPU. Other options are rembg for easier and lighter setup, or the smaller BiRefNet Lite for more speed.

SHARP 3D

Goal: move around one photo and save a different view, not create a production 3D model. I use Apple SHARP because it creates a Gaussian scene from one image and works on GPU or CPU. Production uses the same RunPod worker, then the worker uploads the large PLY directly back to a signed temporary route. Balanced mode keeps half the Gaussians for faster viewing. Other options are TripoSR for a mesh with lighter requirements, or TRELLIS.2 for higher-quality 3D with much more GPU memory.

Sun direction

Goal: change where the sunlight comes from while keeping the same vehicle. I use the hosted Sun Direction FLUX.2 Klein Space. The backend sends the newest edited PNG, or the archive original when there is no edit, with the light-ball settings, waits for the Gradio job, and saves the returned PNG. Other options are IC-Light for local relighting, running FLUX.2 Klein 9B locally with the Sun Direction LoRA, or a normal/depth-map shader when speed matters more than realism.

Main backend files

• src/gallery.py — HTTP requests, listing parsing, and image decoding.
• src/archive.py — grouped discovery, capture, SQLite persistence, and archive search.
• src/web_stream.py — converts the Python generator into NDJSON on stdout.
• src/routes/api/builds/stream/+server.ts — validates requests and proxies the worker stream.
• src/background_worker.py — persistent BiRefNet inference process.
• src/sharp_worker.py — persistent Apple SHARP inference process.
• src/lib/server/runpod-worker.ts — submits and monitors remote GPU jobs.
• runpod/handler.py — runs both models inside the RTX 4090 worker.