By Jon Hasselgren, Jacob Munkberg, and Tomas Akenine-Möller
Graphics processing units supporting tessellation of curved surfaces with displacement mapping exist today. Still, to our knowledge, culling only occurs after tessellation, i.e., after the base primitives have been tessellated into triangles. We introduce an algorithm for automatically computing tight positional and normal bounds on the fly for a base primitive. These bounds are derived from an arbitrary vertex shader program, which may include a curved surface evaluation and different types of displacements, for example. The obtained bounds are used for backface, view frustum, and occlusion culling before tessellation. For highly tessellated scenes, we show that up to 80% of the vertex shader instructions can be avoided, which implies an "instruction speedup" of 5x. Our technique can also be used for offline software rendering.
Read the white paper: Automatic Pre-Tessellation Culling [PDF 8.29MB]
Published previously "ACM Transactions on Graphics" Vol. 28(2):19, 2009 and the Lund University Department of Computer Science web site.