Cg Programming/Unity/Many Light Sources

“Venus de Milo”, a famous ancient Greek sculpture. Note the complex lighting environment.

This tutorial introduces image-based lighting, in particular diffuse (irradiance) environment mapping and its implementation with cube maps. (Unity's Light Probes presumably work in a similar way but with dynamically rendered cube maps.)

This tutorial is based on {{Template:BOOKTEMPLATE/Unity SectionRef|Reflecting Surfaces}}. If you haven't read that tutorial, this would be a very good time to read it.

Diffuse Lighting by Many Lights

Consider the lighting of the sculpture in the image Template:Hide in printTemplate:Only in print. There is natural light coming through the windows. Some of this light bounces off the floor, walls and visitors before reaching the sculpture. Additionally, there are artificial light sources, and their light is also shining directly and indirectly onto the sculpture. How many directional lights and point lights would be needed to simulate this kind of complex lighting environment convincingly? At least more than a handful (probably more than a dozen) and therefore the performance of the lighting computations is challenging.

This problem is addressed by image-based lighting. For static lighting environments that are described by an environment map, e.g. a cube map, image-based lighting allows us to compute the lighting by an arbitrary number of light sources with a single texture lookup in a cube map (see {{Template:BOOKTEMPLATE/Unity SectionRef|Reflecting Surfaces}} for a description of cube maps). How does it work?

In this section we focus on diffuse lighting. Assume that every texel (i.e. pixel) of a cube map acts as a directional light source. (Remember that cube maps are usually assumed to be infinitely large such that only directions matter, but positions don't.) The resulting lighting for a given surface normal direction can be computed as described in {{Template:BOOKTEMPLATE/Unity SectionRef|Diffuse Reflection}}. It's basically the cosine between the surface normal vector N and the vector to the light source L:

$I_{\text{diffuse}}=I_{\text{incoming}}{k}_{\text{diffuse}}\max (0,𝐍\cdot 𝐋)$

Since the texels are the light sources, L is just the direction from the center of the cube to the center of the texel in the cube map. A small cube map with 32×32 texels per face has already 32×32×6 = 6144 texels. Adding the illumination by thousands of light sources is not going to work in real time. However, for a static cube map we can compute the diffuse illumination for all possible surface normal vectors N in advance and store them in a lookup table. When lighting a point on a surface with a specific surface normal vector, we can then just look up the diffuse illumination for the specific surface normal vector N in that precomputed lookup table.

Thus, for a specific surface normal vector N we add (i.e. integrate) the diffuse illumination by all texels of the cube map. We store the resulting diffuse illumination for this surface normal vector in a second cube map (the “diffuse irradiance environment map” or “diffuse environment map” for short). This second cube map will act as a lookup table, where each direction (i.e. surface normal vector) is mapped to a color (i.e. diffuse illumination by potentially thousands of light sources). The fragment shader is therefore really simple (this one could use the vertex shader from {{Template:BOOKTEMPLATE/Unity SectionRef|Reflecting Surfaces}}):

         float4 frag(vertexOutput input) : COLOR
         {
            return texCUBE(_Cube, input.normalDir);
         }

It is just a lookup of the precomputed diffuse illumination using the surface normal vector of the rasterized surface point. However, the precomputation of the diffuse environment map is somewhat more complicated as described in the next section.

Computation of Diffuse Environment Maps

This section presents some C# code to illustrate the computation of cube maps for diffuse (irradiance) environment maps. In order to use it in Unity, choose Create > C# script in the Project Window and call is "ComputeDiffuseEnvironmentMap". Then open the script in Unity's text editor, copy the C# code into it, and attach the script to the game object that has a material with the shader presented below. When a new readable cube map of sufficiently small dimensions is specified for the shader property _OriginalCube (which is labeled Environment Map in the shader user interface), the script will update the shader property _Cube (i.e. Diffuse Environment Map in the user interface) with a corresponding diffuse environment map. Note that the cube map has to be "readable," i.e. you have to check Readable in the Inspector when creating the cube map. Also note that you should use small cube maps of face dimensions 32×32 or smaller because the computation time tends to be very long for larger cube maps. Thus, when creating a cube map in Unity, make sure to choose a sufficiently small size.

The script includes only a handful of functions: Awake() initializes the variables; Update() takes care of communicating with the user and the material (i.e. reading and writing shader properties); computeFilteredCubemap() does the actual work of computing the diffuse environment map; and getDirection() is a small utility function for computeFilteredCubemap() to compute the direction associated with each texel of a cube map. Note that computeFilteredCubemap() not only integrates the diffuse illumination but also avoids discontinuous seams between faces of the cube map by setting neighboring texels along the seams to the same averaged color.

Make sure to call the C# script file "ComputeDiffuseEnvironmentMap".

Complete Shader Code

As promised, the actual shader code is very short; the vertex shader is a reduced version of the vertex shader of {{Template:BOOKTEMPLATE/Unity SectionRef|Reflecting Surfaces}}:

Shader "Cg shader with image-based diffuse lighting" {
   Properties {
      _OriginalCube ("Environment Map", Cube) = "" {}
      _Cube ("Diffuse Environment Map", Cube) = "" {}
   }
   SubShader {
      Pass {   
         CGPROGRAM
 
         #pragma vertex vert  
         #pragma fragment frag 
 
         #include "UnityCG.cginc"

         // User-specified uniforms
         uniform samplerCUBE _Cube;   
 
         struct vertexInput {
            float4 vertex : POSITION;
            float3 normal : NORMAL;
         };
         struct vertexOutput {
            float4 pos : SV_POSITION;
            float3 normalDir : TEXCOORD0;
         };
 
         vertexOutput vert(vertexInput input) 
         {
            vertexOutput output;
 
            float4x4 modelMatrixInverse = unity_WorldToObject; 
               // multiplication with unity_Scale.w is unnecessary 
               // because we normalize transformed vectors
 
            output.normalDir = normalize(
               mul(float4(input.normal, 0.0), modelMatrixInverse).xyz);
            output.pos = UnityObjectToClipPos(input.vertex);
            return output;
         }
 
         float4 frag(vertexOutput input) : COLOR
         {
            return texCUBE(_Cube, input.normalDir);
         }
 
         ENDCG
      }
   }
}

Changes for Specular (i.e. Glossy) Reflection

The shader and script above are sufficient to compute diffuse illumination by a large number of static, directional light sources. But what about the specular illumination discussed in {{Template:BOOKTEMPLATE/Unity SectionRef|Specular Highlights}}, i.e.:

$I_{\text{specular}}=I_{\text{incoming}}{k}_{\text{specular}}\max (0,𝐑\cdot 𝐕{)}^{n_{\text{shininess}}}$

First, we have to rewrite this equation such that it depends only on the direction to the light source L and the reflected view vector R${}_{\text{view}}$:

$I_{\text{specular}}=I_{\text{incoming}}{k}_{\text{specular}}\max (0,{𝐑}_{\text{view}}\cdot 𝐋{)}^{n_{\text{shininess}}}$

With this equation, we can compute a lookup table (i.e. a cube map) that contains the specular illumination by many light sources for any reflected view vector R${}_{\text{view}}$. In order to look up the specular illumination with such a table, we just need to compute the reflected view vector and perform a texture lookup in a cube map. In fact, this is exactly what the shader code of {{Template:BOOKTEMPLATE/Unity SectionRef|Reflecting Surfaces}} does. Thus, we actually only need to compute the lookup table.

It turns out that the JavaScript code presented above can be easily adapted to compute such a lookup table. All we have to do is to change the line

   weight = weight * Mathf.Max(0.0, 
      Vector3.Dot(filteredDirection, originalDirection)); 
      // directional filter for diffuse illumination

to

   weight = weight * Mathf.Pow(Mathf.Max(0.0, 
      Vector3.Dot(filteredDirection, originalDirection)), 50.0); 
      // directional filter for specular illumination

where 50.0 should be replaced by a variable for $n_{\text{shininess}}$. This allows us to compute lookup tables for any specific shininess. (The same cube map could be used for varying values of the shininess if the mipmap-level was specified explicitly using the textureCubeLod instruction in the shader; however, this technique is beyond the scope of this tutorial.)

Summary

Congratulations, you have reached the end of a rather advanced tutorial! We have seen:

• What image-based rendering is about.
• How to compute and use a cube map to implement a diffuse environment map.
• How to adapt the code for specular reflection.

Further reading

If you still want to know more

• about cube maps, you should read {{Template:BOOKTEMPLATE/Unity SectionRef|Reflecting Surfaces}}.
• about (dynamic) diffuse environment maps, you could read Chapter 10, “Real-Time Computation of Dynamic Irradiance Environment Maps” by Gary King of the book “GPU Gems 2” by Matt Pharr (editor) published 2005 by Addison-Wesley, which is available online.
• about Unity's built-in method for dynamic image-based lighting, you should read Unity's documentation of Light Probes.

{{Template:BOOKTEMPLATE/BottomNav}}