Light Is an Equation: on the physics of 3D visualizationLight Is an Equation: on the physics of 3D visualization

In the summer of 1998, at the SIGGRAPH conference, Paul Debevec showed an ordinary photograph taken in a garden in Berkeley: a few objects on a wooden table. Then he said that some of the objects had never existed; they had been generated on a computer and placed into the photograph. The trick was this: Debevec had set a mirrored steel sphere in the middle of the scene, photographed the sphere at different exposures, and converted all the light the sphere reflected, with its direction and intensity, into a "light map." The virtual objects were lit by the real light of the real garden. Today every product visualization begins with that sphere; its name is HDRI, and it explains why 3D visualization is a measurement, not a picture.

Johann Heinrich Lambert wrote down in 1760 how bright a surface will appear: the brightness of light falling on a matte surface is proportional to the cosine of its angle of incidence. If the light comes straight on, the surface looks its brightest; if it comes in at a slant, its palest. In 1823, Augustin-Jean Fresnel completed the second half: the proportion of light reflected depends on the viewing angle, and at a shallow angle every surface turns into a mirror. Look straight down into a lake and you see the bottom; look at it from a shallow angle and you see the sky; even the rim of a matte ceramic mug catches the light. In renders, the glow along an object's edges is not decoration; it is the Fresnel equation, and when that glow is missing the eye reads the object as plastic.

In 1986, James Kajiya combined these two laws in a single expression: the rendering equation. The light that reaches the eye from a point is the light that point emits itself, plus all the light arriving from the surroundings, multiplied by the surface's reflectance behavior and summed. Every modern render engine solves this equation approximately; the difference lies in how accurately and how quickly it does so.

Most surfaces are neither perfect mirrors nor perfectly matte. In 1981, Robert Cook and Kenneth Torrance modeled a surface as a landscape of microscopic mirrors: if the mirrors are aligned, the surface looks glossy; if they are scattered, matte. This "microfacet" model preserves one rule of physics: a surface cannot reflect more light than it receives. In 2012, Brent Burley of Disney presented at SIGGRAPH the "principled" reflectance model, which reduced all of the studio's materials to a few intuitive parameters: base color, metalness, roughness. The game and film industries made it the standard under the name "physically based rendering."

For design, the standard means a great deal: the look of a material is no longer defined by the artist's eye but by measurable values. The roughness of brushed aluminum, the reflection of glass ceramic, the non-metallic base color of painted metal; all of them are numbers, and the same numbers behave consistently under every light. What photography has to set up anew for every shoot, a render measures once.

Debevec's sphere rested on two papers. In 1997, together with Jitendra Malik, he published a method for recovering a scene's true range of brightness from photographs taken at different exposures; the thousands-fold difference between sun and shadow that no single photograph can hold could be stored in a "high dynamic range" image. In 1998 he used that image as a light source. Today the credibility of a product visualization depends largely on which HDRI it is lit with: light measured in a real studio always looks more "right" than an invented light, because the eye recognizes the statistics of real light.

Color demands the same discipline. Render engines compute light in linear space; the screen displays it in gamma-curved sRGB. When the two are confused, shadows go black, colors oversaturate, and the object detaches from reality. In 2014 the film industry put an end to this confusion with the Academy's ACES color encoding system: camera, render, and screen speak on the same scale. For product visualization, using the same system means this: the color that comes from a Pantone reference stays within the ΔE threshold in the render as well.

In 1988, Vilayanur Ramachandran published a simple experiment in Nature: circles that are light on top and shaded underneath look convex, and the reverse look concave; turn the page upside down and the bumps become dents. When the brain derives shape from shading, it assumes the light comes from above; it is the shortcut of a system that evolved under the sun. In 1998, Jennifer Sun and Pietro Perona measured that this assumption also leans slightly to the left: people prefer to see an object lit from the upper left, and they recognize an object lit that way faster.

For product visualization this yields a rule: when the key light does not come from above and slightly to the left, the product's form is misread; an embossed logo looks recessed, a rounded edge looks sharp. The century-old habit of photography studios comes from the same place; the render turns that habit into a number.

How do we know how glossy a surface is? Roland Fleming, Ron Dror, and Edward Adelson tested this question in 2003 and found a surprising answer: people judge surface gloss correctly only in environments that have the light statistics of the real world. When an object is lit with a flat, uniformly colored artificial light, the judgment of gloss collapses. This explains why renders produced with synthetic HDRIs look "plastic": the complexity of light that the eye needs in order to read gloss is not there.

The second illusion lies beneath the surface. In 2001, at SIGGRAPH, Henrik Wann Jensen modeled how light enters materials such as marble, milk, skin, and ceramic, scatters inside them, and then exits: subsurface scattering. Before this model, computer-generated faces looked like wax and ceramics looked like plastic. For a region that produces ceramics, this is not a technical detail: without that translucent depth beneath a tile's glazed surface, the tile does not read as the right material in a render.

In 1970, the Japanese roboticist Masahiro Mori drew a graph: the more something resembles a human, the more likable it becomes, but when it comes very close to full likeness and misses, likability suddenly plunges into revulsion; Mori called this dip the "uncanny valley." The paper was fully translated into English forty years later, in 2012, and became the fundamental warning of computer graphics. The valley is not only for faces: a product render that comes very close to photorealism and misses a small detail (micro-scratches, dust, edge wear, the way roughness breaks up light) looks more repellent than a bad drawing.

A decision follows from this: every render must be either fully photorealistic or openly stylized. The space between the two is Mori's dip. For full realism, imperfection is designed in; for the stylized, imperfection is deliberately removed.

In 2014, Martin Enthed, the manager in charge of IKEA's image production, revealed in an interview that roughly three quarters of the product images in the company's catalog were computer generated; two years earlier the share had been one quarter. Readers had not noticed the difference for years. The reason was that IKEA produced its renders not in place of photography but by photography's rules: the same lenses, the same lighting setups, the same material measurements. When render and photograph stand in the same world, the viewer sees not which is which but the whole.

Our protocol begins with measurement: a physical sample of the material is photographed, a Pantone and ΔE reference is taken, and the roughness and metalness values are set against that sample. Light comes from a measured HDRI, not an invented one; we photographed our own studio with Debevec's method and mapped it, so that the photo shoot and the render are made under the same light. The camera is matched to a real camera's focal length and sensor size. The key light comes from the upper left, by Ramachandran and Perona's rule. The color pipeline is ACES, and the final image is accepted by placing it next to a photograph of the physical sample: if the difference is above ΔE 2, the render is redone.

Debevec's sphere is still on our desk. 3D visualization is not a drawing skill but the discipline of measuring light and rebuilding it; the eye always recognizes measured light.

Lambert, J. H. (1760). Photometria. Augsburg.

Fresnel, A.-J. (1823). Mémoire sur la loi des modifications que la réflexion imprime à la lumière polarisée. Académie des Sciences.

Kajiya, J. T. (1986). The rendering equation. SIGGRAPH '86, 143–150.

Cook, R. L. & Torrance, K. E. (1981). A reflectance model for computer graphics. SIGGRAPH '81, 307–316.

Burley, B. (2012). Physically-based shading at Disney. SIGGRAPH 2012 Course: Practical Physically Based Shading in Film and Game Production.

Debevec, P. E. & Malik, J. (1997). Recovering high dynamic range radiance maps from photographs. SIGGRAPH '97, 369–378.

Debevec, P. (1998). Rendering synthetic objects into real scenes. SIGGRAPH '98, 189–198.

Academy of Motion Picture Arts and Sciences (2014). ACES 1.0, Academy Color Encoding System.

Ramachandran, V. S. (1988). Perception of shape from shading. Nature, 331, 163–166.

Sun, J. & Perona, P. (1998). Where is the sun? Nature Neuroscience, 1(3), 183–184.

Fleming, R. W., Dror, R. O. & Adelson, E. H. (2003). Real-world illumination and the perception of surface reflectance properties. Journal of Vision, 3(5), 347–368.

Jensen, H. W., Marschner, S. R., Levoy, M. & Hanrahan, P. (2001). A practical model for subsurface light transport. SIGGRAPH 2001, 511–518.

Mori, M. (1970). Bukimi no tani [The uncanny valley]. Energy, 7(4), 33–35; English translation: MacDorman & Kageki, IEEE Robotics & Automation Magazine, 2012.

Enthed, M. (2014). Interview on IKEA image production, CGSociety.