Windows App & CLI · Python / CUDA Open Source · MIT Released

LUMEN-PS

Turns an ordinary flatbed scanner into a photometric stereo material scanner. Four scans of the same subject — rotated 90° each time — are enough to recover a normal map, a lighting-free albedo, a height field, and an alpha mask. No camera rig, no synchronized light array, no special optics.

Documentation View on GitHub

A scanned kiwi leaf reconstructed by LUMEN-PS and relit through a full 360-degree orbit — A real 1200 dpi kiwi leaf reconstruction, relit through 360°. Nothing here is sculpted or painted — the relighting uses the normal map recovered from four flatbed scans.

The idea in one minute

A flatbed scanner already contains a stable moving light and a calibrated line sensor. They sit extremely close together — but crucially, they are not perfectly coaxial. That small baseline gives the incident light a slight sideways component, so a microscopic slope facing the lamp returns a little more light than the same slope facing away.

Cross-section diagram of a scanner lamp and sensor showing the offset between them — The lamp–sensor offset is tiny, but it is mechanically fixed and perfectly repeatable on every pass — which is exactly what a hand-held camera cannot offer.

The lamp stays fixed in scanner coordinates. Rotate the subject by 90° between scans and the light appears to orbit it in the subject's own frame. After the four images are registered onto one pixel grid, every pixel has four measured intensities under four known light directions — enough to separate surface orientation from base colour.

Diagram showing how four subject rotations convert a scanner's fixed lamp into four subject-relative lighting directions — Rotating the subject changes the light's azimuth but not its elevation — the four lights land on a single cone.

Four real kiwi leaf scanner captures at 0, 90, 180 and 270 degrees — The four raw captures. Everything below is derived from exactly these images.

What it does

Automated four-rotation capture

Drives the scanner directly through WIA 2.0 with locked exposure and colour settings, so all four scans are photometrically comparable. A local bench app walks through session creation, capture, processing, and interactive relighting in the browser.

Rigid and non-rigid registration

A leaf or a sheet of fabric settles slightly differently every time it is handled. LUMEN-PS de-rotates each scan, then corrects the elastic difference with optical flow computed on a lighting-invariant proxy — so the alignment step can never quietly bend the subject to match the shading it is supposed to measure.

Robust photometric solve with self-repair

The solver rejects highlight and shadow outliers per pixel. Where a normal still looks wrong, it re-solves that pixel from the three consistent scans, and only inpaints the pixels that have no trustworthy solution left — so genuine vein relief survives instead of being blurred away.

CUDA-accelerated, with a CPU fallback

A 1200 dpi run covers tens of millions of pixels. Albedo recovery, QA re-rendering, and the Frankot–Chellappa height integration run in VRAM-aware tiles on CUDA, while the tiny grouped linear systems stay on the faster CPU path. Without an NVIDIA GPU, everything falls back automatically.

What comes out

Every output below was reconstructed from the four scans above — a lighting-free albedo, a 16-bit OpenGL normal map, and a height field integrated from the recovered slopes. LUMEN-PS also exports a DirectX normal map, linear albedo, an alpha mask, and ready-to-use RGBA albedo/normal pairs.

Recovered kiwi leaf albedo with all lighting removed — Albedo — base colour with the scanner's lighting divided out

Recovered kiwi leaf OpenGL normal map — Normal map — the direct photometric result, exported at 16-bit

Integrated kiwi leaf height field — Height — the normal field integrated into surface relief

The measurement it actually sees

Because the lamp and sensor sit so close together, the useful difference between the four scans is far smaller than the albedo they share. Subtracting each pixel's four-view mean and amplifying what remains 7× makes the photometric signal visible: orange is brighter than that pixel's mean, blue is darker, and the opposing directional patterns across the four views are precisely what the solver reads.

Lighting-only deviations from the four-view mean, amplified seven times — Not an edge filter and not invented depth — this is the raw directional signal left over once shared colour and brightness are removed.

The reconstruction then checks its own work. The recovered albedo and normals are re-rendered under each fitted scanner light and compared against the original scans. A dark, noise-like residual means the model explains the measurement; structured bright areas would reveal misregistration, gloss, or a wrong light elevation. The filtered kiwi run lands at a mean residual of 0.0092–0.0099 on normalized linear intensity.

The observed scanner capture used for quality assurance — Observed — the original scan

The same view predicted from the recovered albedo and normal maps — Predicted — re-rendered from the recovered maps

Absolute residual between the observed and predicted scan — Residual — what the model failed to explain

LUMEN-PS started as a foliage scanner, but the method is not leaf-specific — it recovers shallow relief from paper, fabric, bark, pressed flowers, prints, and other mostly diffuse subjects. Very shiny or mirror-like materials break the lighting model. It is open source under the MIT licence; the full derivation, the scanner compatibility checklist, and the pipeline breakdown live in the documentation.