WO2007002592A2 - Image synthesis by rank-1 lattices - Google Patents
Image synthesis by rank-1 lattices Download PDFInfo
- Publication number
- WO2007002592A2 WO2007002592A2 PCT/US2006/024820 US2006024820W WO2007002592A2 WO 2007002592 A2 WO2007002592 A2 WO 2007002592A2 US 2006024820 W US2006024820 W US 2006024820W WO 2007002592 A2 WO2007002592 A2 WO 2007002592A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- lattice
- rank
- improvement
- display
- points
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/40—Filling a planar surface by adding surface attributes, e.g. colour or texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/06—Ray-tracing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/12—Indexing scheme for image data processing or generation, in general involving antialiasing
Definitions
- the present invention relates generally to methods and systems for image synthesis in and by digital computing systems, such as for motion pictures and other applications, and in particular, relates to methods, systems, devices, computer software, and display technology for image synthesis by rank-1 lattices.
- a synthetic image typically represents a two-dimensional array of digital values, called picture elements or pixels, and thus is commonly regarded as a two-dimensional function.
- Image synthesis is the process of creating synthetic images from scenes.
- digital images are generated by rasterization or, in the case of photorealistic images of three-dimensional scenes, by ray tracing. Both approaches aim at determining the appropriate color for each pixel by projecting the original function into the pixel basis. Due to the discrete representation of the original function, the problem of aliasing arises, as described below.
- lattice theory Another computational approach used in computer graphics is lattice theory.
- the accumulation buffer typically used in computer graphics exploits the structure of regular grids.
- efficient rasterization techniques have been designed by taking advantage of such grids.
- Certain known sampling techniques employ Fibonacci lattices, which are a type of rank- 1 lattice.
- a first aspect of the invention is directed to techniques used to select rank-1 lattices for use in image synthesis.
- a number of lattice selection techniques are described, including techniques for searching Korobov lattices in two dimensions by maximized minimum distance, thereby achieving maximum uniformity in the sense of ( ⁇ 4, ⁇ )-unifo ⁇ m point sets, which is used in quasi-Monte-Carlo theory. Further described are techniques for searching the lattice basis, and analyzing the relation of shifted rank-1 lattices to (t, m, s)-nets.
- Another aspect of the invention provides a technique for structuring an image pixel layout using rank-1 lattices, and using the pixel layout as the basis of a rasterization scheme.
- the described rasterization scheme may be implemented using traditional raster displays.
- a further aspect of the invention provides a technique for anti-aliasing by rank-1 lattices, in which maximum minimum distance lattices are compared with respect to their convergence to other Monte Carlo and quasi-Monte Carlo point sets.
- the problem of aliasing varies from pixel to pixel.
- the described aspect of the invention includes a technique for adaptive refinement by rank-1 lattices. Further described is a scheme for depth adaptive filtering in a ray tracing environment.
- FIG. 1 is a schematic diagram of a conventional digital processing system in which the present invention can be deployed.
- FIG. 2 is a schematic diagram of a conventional personal computer, or like computing apparatus, in which the present invention can be deployed.
- FIGS. 3 A and 3B are a pair of graphs respectively illustrating a Fibonacci lattice and its dual lattice.
- FIG. 4 is a flowchart illustrating an overall method in accordance with a first aspect of the present invention.
- FIG. 5 is a diagram of a Fibonacci lattice X(34, 21).
- FIGS. 6 A and 6B show, respectively, Korobov lattices Z(8, 1) and Z(8, 5).
- FIG. 7 is a graph illustrating the computation of the distance of the point (2, 4) to the origin for the lattice 1(5, 2).
- FIG. 8 A shows a fundamental parallelepiped of a hexagonal lattice
- FIG. 8B shows an equilateral triangle representing half of the fundamental parallelepiped.
- FIGS. 9A-C illustrate a lattice selected, respectively, in the unit square, in the unit square scaled to an actual region, and in the actual region.
- FIGS. 1 OA and 1 OB illustrate a scaled sampling lattice before and after shifting to obtain exactly one lattice point per pixel.
- FIG. 11 shows a table setting forth max-min-dist lattices with valid shifts.
- FIG. 12 shows a table setting forth lattices produced according to a search technique under the condition of a valid shift.
- FIG. 13 shows a graph of a periodically extended lattice covered with a family of equidistant parallel lines.
- FIG. 14 shows a graph illustrating a lattice with its Minkowsky reduced basis.
- FIG. 15 shows a graph illustrating a (0, 2, 2)-net in basis 5.
- FIG. 16 shows a table setting forth lattice parameters for some (0, 2, 2)-nets in base h.
- FIG. 17 shows a flowchart of a rasterization method using rank-1 lattices in accordance with a further aspect of the invention.
- FIGS. 18A and 18B are a pair of graphs respectively illustrating intensification of a pixel and a line on a raster grid.
- FIGS. 19A and 19B are a pair of graphs respectively illustrating intensification of a pixel and a line on a lattice grid.
- FIG. 20 is a table setting forth single steps performed in the rasterization of a line on a lattice grid.
- FIG. 21 is a table setting forth variables used in Crow's rasterization algorithm for convex polygons.
- FIG. 22 is a graph illustrating a polygon rasterized on lattice L(576, 155), in a resolution of 24 x 24.
- FIGS. 23 A and 23B are diagrams illustrating rasterization on lattice grids at different resolutions.
- FIG. 24 is a diagram illustrating the smoothing of a polygon by splitting cells of the lattice into two triangles.
- FIG. 25 shows a raster display built using a maximum minimum distance lattice.
- FIG. 26 A shows a layout of area light sources for monochrome displays
- FIG. 26B shows a layout for color displays, in a system for anti-aliasing by rank-1 lattices
- FIG. 27 shows a Voroni diagram of the maximum minimum rank-1 lattice
- FIG. 28 A shows a layout of area light sources for hexagonal pixels in a monochrome display.
- FIG. 28B shows a layout for a color display.
- FIGS. 29 A and 29B show diagrams of a display comprising four modules, each of which contains 256 cells.
- FIG. 3 IA shows a quadratic layout of the individual modules
- FIG. 3 IB shows a rectangular layout.
- FIG. 30 shows a diagram of a system in which display modules are addressed by a demultiplexer.
- FIG. 31 shows a table setting forth parameters for 2" displays and images.
- FIG. 32 shows a flowchart of a method for anti-aliasing by rank-1 lattices according to a further aspect of the invention.
- FIG. 33 shows an image of an infinite checkerboard used to illustrate anti-aliasing by rank-1 lattices.
- FIG. 34 shows a diagram of a pixel samples with a 4 x 4 grid.
- FIG. 35 A shows a uniform 4 x 4 pattern and FIG. 35B shows a jittered grid with one random sample in each of the 4 x 4 strata.
- FIG. 36A is a graph illustrating the reconstruction of an original frequency with additional noise
- FIG. 36B is a graph illustrating the recovery of only noise.
- FIG. 38B shows a randomized Larcher-Pillichshammer point set.
- FIG. 38B shows a randomized lattice with the same parameters.
- FIG. 39A is a graph showing the first 5 x 5 pixels of a screen with a resolution of 512 x 512 covered by the max-min-dist lattice L(252144, 549).
- FIG. 39B shows the shifted lattice with shift (259, 0).
- FIG. 40 shows a method for adaptive refinement by rank-1 lattices according to a further aspect of the invention.
- FIG. 41 shows a max-min-dist lattice £(8,5) determined in the sheared basis of the lattice 1(1048576, 33919).
- FIG. 42 shows a refinement geometry for a gradient criterion.
- FIG. 43 shows a placement diagram of a refinement max-min-dist lattice for a gradient criterion.
- FIG. 44 shows a graph of a decision function for the B-spline degree.
- FIG. 45 shows a diagram illustrating the influence of a sampling point for both a support of 1 pixel radius and 1.5 pixel radius.
- FIG. 47A shows a diagram of a rendering of a sloped diagonal line without knowledge of the sub-pixel structure.
- FIG. 47B shows the rendering after tripling the horizontal resolution.
- FIG. 48 shows a diagram of an RGB decimation using one-pixel- wide box filters.
- FIGS. 49-51 are a series of drawings illustrating lattices used to render an infinite checkerboard image.
- the present invention relates to the generation and synthesis of images, such as for display in a motion picture or other dynamic display, using rank-1 lattices.
- the techniques described herein are practiced as part of a computer graphics system, in which a pixel value is generated for each pixel in an image.
- the pixel value is representative of a point in a scene as recorded on an image plane of a simulated camera.
- the computer graphics system is configured to generate the pixel value for an image using a selected methodology.
- Methods, devices or software products in accordance with the invention can operate on any of a wide range of conventional computing devices and systems, such as those depicted by way of example in FIG. 1 (e.g., network system 100), whether standalone, networked, portable or fixed, including conventional PCs 102, laptops 104, handheld or mobile computers 106, or across the Internet or other networks 108, which may in turn include servers 110 and storage 112.
- a software application configured in accordance with the invention can operate within, e.g., a PC 102 like that shown in FIG. 2, in which program instructions can be read from CD ROM 116, magnetic disk or other storage 120 and loaded into RAM 114 for execution by CPU 118.
- Data can be input into the system via any known device or means, including a conventional keyboard, scanner, mouse or other elements 103.
- a synthetic image represents a two-dimensional array of digital values, called picture elements or "pixels" and thus can be regarded as a two-dimensional function.
- image synthesis refers to the process of creating synthetic images from scenes.
- digital images are generated by a process known as "rasterization,” or in the case of photorealistic images of three dimensional scenes, by a process known as “ray tracing.”
- rasterization or in the case of photorealistic images of three dimensional scenes, by a process known as “ray tracing.”
- ray tracing Both approaches aim at determining the appropriate color for each pixel by projecting the original function into the pixel basis. Due to the discrete representation of the original function the problem of aliasing arises, discussed below.
- (M, ⁇ )-uniform point sets include radical inversion based points, points from the Cartesian product midpoint rule and rank-1 lattices. Using the probability space
- SB corresponds to the Borel-sets and ⁇ $ to the ⁇ -dimensional Lebesgue-measure
- Theorem 1 has been generalized to parametric integration:
- a lattice L is a discrete subset in K*
- L : P,, + Z" C R S which is closed under addition and subtraction.
- regular axis-aligned grids represent one type of simple lattices. Every lattice in R s can be generated from a set of
- the vectors Jb 1 , . . . , b s ⁇ are one basis of the lattice. As they generate the lattice, they are also said to be generator vectors. The number s of basis vectors yields the dimension ofZ.
- the determinant of Z 1 arises from the reciprocal of the determinant of Z.
- Typical problems in lattice theory are the closest-point and the shortest vector problem, discussed below.
- the closest-point problem arises, for example, in communication theory where lattices are applied both for modulation and quantization.
- the shortest vector problem has been used, for example, in order to measure the quality of random number generators. Another application of this problem is found in cryptography.
- a further problem, discussed below, is that of lattice basis reduction and the related search for a lattice basis.
- Other lattice problems include determining the kissing number or finding the Voronoi-relevant vectors.
- FIG. 4 is a flowchart illustrating a method 220 according to the present invention.
- a rank-1 lattice is selected in accordance with a maximized minimum distance function corresponding to pixels in a synthetic image to be generated.
- pixel values are calculated in accordance with the selected rank-1 lattice.
- a display-controlling electronic output signal is generated, based on the calculated pixel values. Lattices are now discussed in greater detail, particularly techniques for selecting rank-1 lattices, including techniques for searching Korobov lattices in two dimensions by maximized minimum distance, thereby achieving maximum uniformity in the sense of
- rank-1 lattices in accordance with other aspects of the invention described herein require a "lattice basis," i.e., a set of linearly independent vectors capable of generating the subspace defined by a selected rank-1 lattice.
- the following discussion therefore includes a technique for searching the lattice basis.
- the relation of shifted rank-1 lattices to (t, m, ,y)-nets is analyzed.
- the "rank" of a lattice L in M s is defined to be the minimum number of vectors
- Korobov lattices provide an example of s-dimensional rank- 1 -lattices.
- the mod 1 operation restricts the lattice to the unit square resulting in a one-periodic pattern. This means that rank-1 lattices are point symmetric to the origin.
- lattices can be tiled seamlessly in order to fill the s-dimensional space and that they can be generated very quickly.
- the quality of rank-1 lattices is significantly influenced by the integer generator vector (gj, . . . , g s ) e N s . Lattices in Korobov form, which do
- rank- 1 -lattices are applied in the contexts of anti-aliasing and rasterization. The present discussion addresses the question as to what kind of rank- 1 -lattices to use.
- rank-1 lattices are (M, ⁇ )-uniform point sets. This property of rank-1 lattices can be seen by considering the partitions ⁇ x+ ⁇ of / s which are induced by the fundamental parallelepiped. Maximum uniformity in this sense can be achieved by maximizing the mutual minimum distance
- rank- 1 -lattices are used herein in the Korobov form: (2.3) featuring a so-called "max-min-dist-property" and identify them by search. Since a lattice in Korobov form is uniquely determined by the tuple (n, a), this means that for a fixed number of points n the search is conducted for such an ⁇ e ⁇ 1, . . . , « — 1 ⁇ that the minimum distance between two lattice points achieves the maximum on the unit torus. The more the minimum distance between two lattice points grows, the more the resulting lattice approximates a hexagonal lattice, thereby increasing the uniform distribution of the point set.
- the points lie on the diagonal of the unit square, whereas the points of the max-min-dist lattice shown in the FIG. 6B lattice 240 are more uniformly distributed.
- the task of calculating the minimum distance between two lattice points is related to a well known problem in lattice theory.
- the "closest-point problem" consists in finding a vector c for a given lattice L and a given input point x e l s such that
- Calculating the minimum distance in a lattice L corresponds to determining the shortest vector in L which can be accomplished by a simple modification of the closest-point problem. For that purpose, only vectors v ⁇ 0 are considered. Then the closest point to the origin yields the shortest vector, that is, the minimum distance in L. Closest-point search methods have been described in the literature for lattices without a regular structure. But the methods presented there are not appropriate to rank-1 lattices, as they provide a special structure and are only considered for low dimensions. In the following discussion, several approaches for lattice searching are described that exploit both the periodic and regular structure and the application of these lattices.
- Section 2.1.1 of the Appendix sets forth a code listing for a basic search on the unit square which, for a given ⁇ ⁇ N ; considers all possible lattices (n, a) , 2 ⁇ a ⁇ n — 1.
- the algorithm determines the length of the shortest vector by looping over all lattice points Xi, ⁇ ⁇ i ⁇ n.
- the lattice is defined on the unit torus, all computations in the ⁇ 2.1.1 code listing are performed in integer arithmetic in order to avoid rounding errors due to floating point imprecision. This means that the present discussion focuses on integer lattices on [0, n) instead of [0, 1) 2 :
- FIG. 7 shows a graph 260 illustrating the above technique for the lattice L(5, 2).
- the data type for the variables is cither chosen to be tmsigtied int. (UINT), especially for the tuple (»., ⁇ ), or unsigned long long hit (ULLINT) for the entities involved in the Euclidean norm calculation.
- MAXJJLLINT + 1 0.
- MAX.ULLINT z - 1 an overflow is evident if ,r + y ⁇ x or .r + y ⁇ y.
- the shortest vector Once the shortest vector is found, its length is compared to the length of the vector having maximum minimum distance maxmindist so far and possibly remember the related parameter a. But this is not done until the Boolean flag noOverflow is set true the first time in order to assure the new maxmindist not to become MAXJJLLINT which is the initial value for mindist in the ⁇ 2.2.1 code listing. Depending on whether the new maxmindist is greater or equal to the old one the vector a opt is cleared before storing the new value.
- Each lattice cell i.e., the fundamental parallelepiped, is divided into two equilateral triangles.
- the lattice covers an area of 77 2 .
- each of the n lattice cells (one for each point) has an area of
- FIGS. 8 A and 8B are a pair of diagrams summarizing the above-described process.
- FIG. 8 A shows a fundamental parallelepiped 270 of a hexagonal lattice.
- one half of the fundamental parallelepiped represents an equilateral triangle 280.
- h indicates the height of the triangle.
- / represents the length of one triangle side.
- the basic search algorithm yields quite a huge number of max-min-dist lattices.
- the ⁇ 2.2.1 code listing is extended such that also the distance of the second shortest vector is maximized. In the present case, considering the third shortest vector brings no further improvement.
- FIG. 9B shows a graph 300 illustrating the lattice in the unit square, scaled to the actual region.
- FIG. 9C shows a graph 310 illustrating the lattice in the actual region.
- the Euclidean norm is replaced by a weighted norm. Therefore, the described technique is conducted as if on the unit square, but with the squared Euclidean norm modified by two scaling factors in order to make up for the region size. These scaling factors are derived by dividing both the x- and y-Resolution through their greatest common divisor (gcd)
- Section 2.1.2 of the Appendix shows a code listing including an extended search considering the actual region size.
- the ⁇ 2.1.2 code listing incorporates the search on an arbitrary region size as well as the improvements discussed above.
- the dots in the ⁇ 2.1.2 code listing represent the unchanged part of the ⁇ 2.1.1 code listing.
- FIG. 1OA shows a graph 320 illustrating this situation.
- the lattice L(25, 7) c [0,I) 2 is scaled to [0,5) 2 and samples the screen with a resolution of 5x5. Pixel (0, 1) is missed, while pixel (0, 0) contains two samples.
- the search domain i.e., one pixel, is discretized by looking at the low dimensional projections of the lattice points. If these are perfect in the sense of equally stratifying each coordinate axis, the distance between two lattice points is — on the x-axis and ⁇ - on the y-axis, respectively. This means that there are
- FIG. 11 shows a table 340 listing some maximum minimum distance lattices, for which valid shifts have been found, ad well as the underlying screen resolution.
- the process can switch over to the next lattice shift at once because in this case one pixel will surely contain less than nl(xRes • yRes) samples.
- the dots in the ⁇ 2.1.4 code listing represent unchanged parts of the previous code listings. Due to the three nestedybr-loops, the ⁇ 2.1.4 code listing has a complexity of O(n 3 ) and is very slow for large n. But the search for shifts can be shortened considerably by exchanging loop (1) and (2). That means that the search domain is passed through in a different order. This is based on the observation that the y-component of at least one shift is zero in all examined cases. FIG.
- FIG. 12 sets forth a table 350 listing lattices that result from the ⁇ 2.1.4 code listing.
- shifts which have a non-zero y-component were obtained by not stopping the shift search after the first shift was found. It took approximately five days to compute the lattice L(4194304, 182183) on a Intel Pentium 4, 2.8 GHz, despite switching loops (1) and (2).
- L 1 has lattice structure.
- the lattice points can be covered by a small number of parallel lines which can be chosen from many different families. In higher dimensions the families of lines generalize to families of hyperplanes.
- v t is defined as the reciprocal of the maximum distance between hyperplanes, taken over all families of parallel (t-l)-dimensional hyperplanes covering all points.
- v t corresponds to the length of the shortest vector in the lattice which is dual to the lattice of equation 2.17. This means that the spectral test can be applied for the search for max-min-dist lattices.
- the mod 1 -operation is disposed of by extending the set Z 2 periodically, i.e., by covering the whole M with copies of the two dimensional hypercube:
- the set 2.21 can further be simplified by changing (x, k ⁇ , hi) to (x + him, 0, h ⁇ - ak ⁇ ) since and i + ' k 9 ⁇ thereby setting the variable k ⁇ to zero. Doing so allows the above point set to be described by
- set 2.22 contains the m points of the original set 2.19 which satisfy
- v 2 corresponds to the length of the shortest vector u which defines a family of hyperplanes ⁇ x ⁇ u
- Such a vector u (W 1 , W 2 ) must satisfy the following conditions:
- W 11 m
- Section 2.1.5 of the Appendix sets forth a code listing of a technique for determining the value of the two-dimensional accuracy v 2 . It incorporates a "Euclidean step,” which represents a simplification of the two-dimensional spectral test with respect to higher dimensions and is based on Gauss's notion of a reduced binary quadratic form.
- the shortest vector u e I/ defines the family of lines with maximum distance. Every other choice of u brings the lines closer together. There are now selected two neighboring lines of a family and one lattice point on each of them such that for one fixed point pi on the first line the other point p 2 on the second line is closest to pi. Then the distance between pi and p 2 decreases as the distance between two adjacent lines increases. Therefore the shortest vector in L 1 also indicates the shortest
- v 2 i.e., the length of the shortest vector in L x
- L x the length of the shortest vector in L up to some scaling factor.
- G (gi, . . . , g s ) ⁇ with gi, . . . , g s being row vectors is said to be "Hermite reduced” if
- a less strict reduction criterion is used for the presently described basis search.
- a basis G (g l5 . . . , g s ) ⁇ is reduced according to this criterion if
- gi is tlic shortest lion-zoro vector in L and
- a search is first conducted for the shortest non-zero lattice vector gi. Then, the subsequent basis vector g 2 is determined by searching the second shortest lattice vector in L which is linearly independent of gi. This can be done by modifying the basic search technique. For a given lattice L(n, a) all lattice points Xj are looped over in order to calculate mindist ⁇ x ⁇ ) like in equation 2.5, discussed above. The length of both the shortest and the second shortest vector are checked for, updating the basis vectors if necessary.
- Section 2.2.1 of the Appendix sets forth a code listing that checks for the linear independence of two basis vectors. If the signs of the vector components were not involved, the ⁇ 2.2.1 code listing would take the vectors (1, 1) and (1,-1) for linearly dependent, for example.
- Section 2.2.2 of the Appendix shows a code listing in which a search is conducted for a Minkowsky reduced basis. It should be remembered that computations are always performed on [0, n) 2 and therefore also get the basis vectors on [0, n) 2 . Concerning shifted lattices, the basis search algorithm can be applied in the same way because a shift just represents a translation of the whole lattice and does not influence the basis.
- FIG. 15 shows a graph 380 illustrating a refinement of the grid structure of FIG. 1OB and shows the lattice 1(25, 7) c [0, I) 2 shifted by ( i
- the lattice points perfectly fill the fine grid structure in a low dimensional projection. This is indicated by the hollow lattice point which is projected onto the x- and y-axis with no other lattice point lying on these lines. Furthermore, each interval
- t is said to be the quality parameter.
- FIG. 16 shows a table 390 listing some further (0, 2, 2)-nets in base b.
- a raster display is typically formed from a matrix of picture elements (pixels) representing the whole screen.
- a rasterizer approximates the mathematical ideal primitives, such as lines, triangles or circles described in terms of vertices, on a Cartesian grid by assigning the appropriate colors to sets of pixels.
- the rasterizer converts the two-dimensional vertices in screen space into pixels on the screen.
- a pixel is typically represented as a square on the integral raster [0, . . . , ResX) * [0, . . . , Res Y).
- FIG. 17 is a flowchart of a method 400 according to this aspect of the invention.
- a rank- 1 lattice is selected in accordance with a maximized minimum distance function corresponding to points in a synthetic image to be generated.
- a point/? in Cartesian coordinates is transformed into the basis of the rank-1 lattice.
- step 403 it is determined the lattice cell into which the transformed point falls.
- step 404 the Cartesian coordinates are converted into rank-1 lattice coordinates.
- step 405 the rank-1 lattice coordinates are used to rasterize the image.
- the rasterization technique is described in conjunction with a Digital Differential Analyzer (DDA), which is a scan conversion technique for lines, and in conjunction with Crow's algorithm, which is a rasterization technique for convex polygons.
- DDA Digital Differential Analyzer
- Crow's algorithm which is a rasterization technique for convex polygons.
- the described technique can be performed using traditional raster displays.
- the technique may also form the basis of a new concept of display technology.
- a scan-conversion technique for lines is based on computing the coordinates of those pixels that best approximate an ideal, straight line drawn on the two-dimensional raster.
- a line is characterized by a start point, an end point, a width, and by and additional style attributes.
- the present discussion focuses on lines having a width of one pixel, without additional attributes and without anti-aliasing.
- DDA Digital Differential Analyzer
- a point (t + T, Pit + T )) on the line can incrementally be calculated from its preceding point (t, P(t)):
- the rasterization process has to be discretized in n steps, where n - max ⁇ ⁇ ⁇ x o
- FIG. 18A shows a graph 410, in which the pixel (2, 2) is intensified on the raster grid.
- Appendix sets forth a source code listing for the DDA.
- the DDA has been described with respect to traditional raster displays.
- the rasterization technique is converted to rank-1 lattices by changing the basis in which the rasterization is performed.
- each lattice cell is interpreted as a sample in the center of the fundamental parallelepiped spanned by the Minkowsky reduced lattice basis.
- Each cell (x, y) is addressed in lattice coordinates, i.e., in the coordinate system formed by the lattice basis.
- the use of lattice coordinates is illustrated in FIGS. 19A and 19B.
- FIG. 19A shows a graph 430 illustrating the intensification of lattice cell (0, 2) on the lattice grid.
- FIG. 19B shows a graph 440 illustrating rasterization of the line extending from (1, 3) to (7, 14) on the max-min-dist lattice Z(576, 155).
- the intensified (shaded) lattice cells in FIG. 19B represent the scan-converted line.
- the scan-conversion technique assumes that its endpoints are given in Cartesian coordinates on the unit square.
- the lattice coordinates are obtained by transforming these points into the lattice basis and by determining the lattice cells into which the transformed points fall. The coordinates of these two lattice cells then define the start and endpoint of the line.
- Section 3.1.2 of the Appendix sets forth a source code listing for DDA rasterization on rank-1 lattices.
- FIG. 20 shows a table 450 setting forth the single steps performed by the ⁇ 3.1.2 code listing for the rasterization of the line in FIG. 19B from (1, 3) to (7, 14).
- the last line of the FIG. 20 table 450 contains the lattice cells approximating the ideal line.
- P x (7.4999, 14.5) .
- P 0 lies within the lattice cell (1, 3) and P 1 within (7, 14)
- the line is identified by the start point (1, 3) and the end point (7, 14).
- This technique is as a method called determineLatt iceCell that is implemented in the code listing set forth in section 3.1.4 of the Appendix.
- a polygon is rasterized by finding the pixels which fill the polygon given by a set of vertices and edges.
- Crow's algorithm assumes that the vertices are ordered in a consistent way and that the polygons to be converted are convex. However, because every polygon can be decomposed into convex polygons, this requirement does not present a limitation.
- Crow's algorithm takes advantage of the coherence in both the horizontal and the vertical scanline. Looping over a horizontal scanline from bottom to top, the algorithm determines in each scanline which pixels are inside the polygon by means of an incremental Jordan test for horizontal coherence.
- the Jordan test is based on the Jordan curve theorem, which states that any ray from inside a polygon will intersect the edges of a polygon an odd number of times. As only convex polygons are considered, there is just one span per scanline which is bordered by points VscanLeft and VscanRight on the current left and right edges. These points are found by exploiting the vertical coherence by means of a DDA approach.
- the same approach is used as above.
- the algorithm is performed in the coordinate system spanned by the basis of the particular rank-1 lattice L(n, a) instead of the Cartesian coordinate system.
- the first basis vector b ⁇ of the Minkowsky reduced basis ot L(n, a) corresponds to the horizontal and the second basis vector hi to the vertical coordinate axis.
- the rasterization algorithm takes as arguments an array of vertices and the size of this array. According to the code listing set forth in ⁇ 3.1.2, the vertices which have to be ordered counterclockwise are assumed to be given in Cartesian coordinates on the unit square.
- the rasterization can be implemented the same way as in the code listing of ⁇ 3.2.1 up to one exception, i.e., the basic rasterization algorithm remains the same, as shown in the code listing in ⁇ 3.2.4.
- FIGS. 23A and 23B show graphs 480 and 490 displaying the same polygon rendered in higher resolutions.
- the principle for the rasterization on rank-1 lattices which has been described in this chapter considering as example the DDA and the algorithm of Crow, can also be applied for other rasterization algorithms, like for example for the Bresenham algorithm for lines and the y-x rasterization algorithm for polygons.
- One disadvantage of this rasterization approach is that edges being almost vertical or rather horizontal appear very jagged due to the shape of the lattice cells.
- the described techniques provide the basis for a new display technology.
- the first computer monitors created images by moving an electron beam directly along the paths between specified points on the display.
- These vector displays were able to generate sharp and smooth lines.
- they only drew wire frames and could only display scenes of limited complexity.
- Even commonly used for scientific instruments like oscilloscopes they have been replaced by raster displays.
- Such a display usually is constituted by a rectangular grid of pixels which are represented as squares on this raster.
- images are created by scan line algorithms.
- the pixels can be addressed individually and they can be colored by means of individual red, green and blue phosphors, in the case of a CRT display.
- the rasterization can be performed on lattice points, as discussed above. Whereas this method can be simulated on traditional raster displays by means of a software solution, it can even be performed on current graphics hardware in the following way. Since the rasterizer is only capable of operating on rectangular grids, in a first step the scene has to be transformed into the lattice basis, which in fact corresponds to a shear of the rectangular grid. After this change of frame the rasterization can be performed on the graphics hardware as usual. In order to display the rasterized scene, the resulting image has to be transformed back into the pixel basis. i Jtiis procedure yields the same results as if the rasterization would have directly been performed on a rank-1 lattice.
- the raster display is conceived as a matrix of lattice cells, i.e., the single picture elements are represented as the cells which are induced by the Delaunay triangulation of the rank-1 lattice points, i.e., the Voronoi diagram.
- the raster display is conceptualized as a rank-1 lattice which is selected by maximized minimum distance.
- the picture elements approximate the hexagonal grid for an increasing number of pixels.
- FIG. 25 illustrates a layout 510 of a display module consisting of 256 lattice cells.
- a first implementation comprises a display of point light sources, for example RGB LEDs, which are arranged in the center of each lattice cell, indicated by the small black dots in FIG. 25. Since the rank-1 lattice is chosen by maximum minimum distance, the RGB-LEDs are distributed over the screen in an optimal way.
- a second implementation composes the display of area light sources covering the single picture elements. This yields a technique for thin-film transistor (TFT) displays and liquid crystal displays (LCDs), respectively, which may be realized by means of organic light-emitting diodes (OLEDs) for instance.
- TFT thin-film transistor
- LCDs liquid crystal displays
- the overall resolution of a rank-1 lattice display can be doubled by splitting the lattice cells along their shorter diagonal, thus reducing aliasing artifacts, as shown in the FIG. 24 graph 500.
- This principle can directly be realized in hardware by adapting the layout of the area light sources. This is shown in FIGS. 26 A and 26B for a monochrome display 520 and a color display 530. So far the pixel layout has been determined by the cells induced by the Delaunay triangulation of the rank-1 lattice points. By means of the Voronoi diagram, which is dual to the Delaunay triangulation in M , the M. can be partitioned into almost hexagonal
- FIG. 27 shows a Voronoi diagram 540 of the maximum minimum distance rank-1 lattice £(64, 19). Structuring the pixel layout in this sense leads to substantially hexagonal pixel lattices.
- ommatidia in compound eyes or the visual sensors in the retina are arranged in this manner, for example.
- the concept of hexagonal pixels has already been studied in the context of hexagonal image processing and is used, for example, in SmartSlab LED panels.
- FIG. 28A shows a layout 550 of area light sources for hexagonal pixels for a monochrome display.
- FIG. 28B shows a layout 560 of area light sources for hexagonal pixels for a color display.
- the layout of the sensors, i.e., the photosites, of a charge-coupled device (CCD) camera take the form of a rank-1 lattice cell which is either induced by the Delaunay triangulation or the Voronoi diagram.
- CCD charge-coupled device
- rank-1 lattice patterns are used in projector technology.
- the hinge-mounted microscopic mirrors of digital light processing (DLP) integrated circuits could be arranged in this manner.
- the described patterns can be applied in the context of 3D displays, such as the Sharp LL-151 -3D monitor, and other monitors disclosed at www.sharpsystems.com, in order to realize both the display layout and the parallax barrier, by which the 3D mode is enabled.
- FIGS. 29A and 29B show displays 570 and 580 according to this aspect of the invention.
- the number of cells for a display module is selected to be a power of 2.
- This type of display module has the advantage that the single cells easily can be addressed by means of a demultiplexer. If the number k of modules is set to a power of 2 as well, the single modules can be controlled the same way.
- FIG. 30 is a diagram of a system 590 according to this aspect of the invention. Such displays can be produced economically by fabricating small modules of the same layout and the same number of lattice cells which can easily be assembled to a display of desired resolution.
- FIG. 31 shows a table 600 that lists the parameters for some 2" display modules and images, respectively. The listed sizes represent the proposed resolution for the single modules according to which the maximum minimum distance rank-1 lattices have been selected.
- FIG. 32 shows a flowchart of a method 610 according to a further aspect of the invention for anti-aliasing by rank-1 lattices.
- rank-1 lattices are selected in accordance with a maximized minimum distance function corresponding to points in a synthetic image to be generated.
- the selected rank-1 lattices are compared with respect to their convergence to either Monte Carlo or quasi-Monte Carlo point sets.
- an anti-aliasing function is applied, based on the result of this comparison.
- FIG. 33 shows a synthesized image 620 of a checkerboard of infinite size that has been used to develop and illustrate techniques for anti-aliasing by rank-1 lattices.
- a synthesized image generated by a computer is a digital signal. Since a computer is only capable of storing discrete values of finite precision, a computer-generated image must perforce also be discrete. Thus, in generating an image, a signal that is typically aperiodic and discontinuous must be converted into a digital signal.
- the resulting signal c is resatnpled with a set of samples reflecting the frame buffer memory locations.
- the convolution of r with the display reconstruction filter gives the actually displayed signal.
- the sampled signal b is directly reconstructed for each single pixel, thereby omitting the resampling step.
- sampling and reconstruction of a one-dimensional continuous signal is now discussed.
- the two-dimensional case is very similar to the one- dimensional case, from which it differs only in its complexity.
- ideal sampling and reconstruction it is meant that the continuous, band-limited signal is uniformly sampled and reconstructed in such a way that the original signal is re-obtained in the end.
- the frequency ⁇ o is called the cutoff frequency for/ If ⁇ o — oo, the function/is not band-limited.
- the theorem states that a band-limited function/may be perfectly reconstructed from its uniform samples if the sampling frequency is at least twice as high as the maximum frequency ⁇ o in the spectrum of UQ. ⁇ o is also knows as the "Nyquist frequency.”
- undersampling refers to sampling below the Nyquist frequency. If a signal is sampled above the Nyquist frequency, it is said to be “oversampled.”
- the reconstruction filter depends on whether we calculate in time or in frequency space. In time space it is necessary to convolve the correctly sampled signal with the .sine-function, whereas in frequency space, it is necessary to multiply F s ( ⁇ ) with a box spectrum.
- the checkerboard image 620 displayed in FIG. 33 was uniformly sampled with one sample placed in the center of every pixel and a resolution of 512 x 512. So the sample pattern represents a regular grid, such as the grid 670 shown in FIG. 35 A.
- ⁇ lp should be twice as high as the cutoff frequency in the signal.
- the checkerboard provides many sharp edges, the problem arises of having a signal with compact support in signal space which cannot simultaneously have a compact support in frequency space. This means that the signal cannot be band-limited, thereby featuring increasingly high frequencies.
- there are frequencies > l/p which cannot be represented directly and which turn into aliases, as shown in FIG. 33.
- sampling Since the samples are taken at a higher frequency than once per pixel, the described sampling is referred to herein as "supersampling.”
- supersampling One common supersampling scheme is to use an « ⁇ « grid as sampling pattern within each pixel and average the samples to one value by weighting them appropriately. This can by described by the "pixel integral" where in this case the sampling function S represents the fine uniform grid and the reconstruction filter r weights each sample equally with VN.
- sampling function which takes one sample in the center of each pixel with p being the distance between pixel centers. According to the sampling theorem, any energy above the frequency — cannot be represented properly in the reconstructed signal and will therefore result in aliasing. Taking N samples per pixel means to apply the sampling function
- the new Nyquist frequency is ⁇ .
- a low-pass filter is applied to prevent new aliasing.
- a low-pass filter G( ⁇ ) is chosen with cutoff frequency -L . It is
- R ( ⁇ ) usually is not applied in practice. Instead, only a low-pass filter is deployed. It is
- n x n sampling points it is very likely for the n x n sampling points to behave as just n.
- the sampling grid 630 shown in FIG. 34 demonstrates that it would have been sufficient to sample with only 4 points, one in each horizontal scanline, instead of 16 points.
- Another disadvantage of the supersampling method described above is that the rendering time increases quadratically with n. Alternatively the sampling points could be placed
- the basic test pattern used herein is given by the regular grid with one or n x m samples within each pixel.
- This sampling pattern has already been discussed in detail above, since the sampling theory is based on the uniform grid. Taking one sample in the center of each pixel generally introduces aliasing. Also, using an n x n grid per pixel cannot avoid aliasing in the case of a signal being not band-limited. But this sampling scheme is easy to implement, as shown in the code listing set forth in Section 4.2.1 of the Appendix. Furthermore it can efficiently be realized in graphics hardware by means of the accumulation buffer which integrates every pixel at the same time.
- An example of this technique is illustrated by the grids 640 and 650 shown in FIGS. 35A and 35B.
- the number n of strata has to be factorized which means that not any choice of n yields a good division of/. Due to the stratification, the samples cannot clump together in one place, as might happen for purely random samples. Therefore, the uniform distribution of the samples is improved over the whole unit cube.
- Section 4.2.2 of the Appendix sets forth the source code to generate a pattern with n x m samples within the unit cube.
- the max-min-dist lattices described above are applied. In contrast to stratification, where a factorization of n is required, there is no restriction to the sampling rate, since rank-1 lattices can be generated for any number of points. Since there is interest both in pixel sampling and in sampling the whole screen, use is also made of the shifted lattices described above. As lattice parameters, the results are used from the above lattice search techniques. The code listing set forth in section 4.2.5 of the Appendix creates both max-min-dist and shifted lattices. In the case of max-min-dist lattices the parameters for the x and ⁇ -resolution have to be set to 1. Moreover, shifiX and shifiY must be zero.
- Section 4.2.3 of the Appendix sets forth a code listing of an efficient implementation of the described technique. If the parameter r is given by a random integer, the point set is randomized by the XOR-operation. The randomized Larcher-Pillichshammer point set is generated by the code listing set forth in the code listing in section 4.2.6 of the Appendix. The lattice patterns of code listing ⁇ 4.2.5 are randomized by simply adding a random shift modulo 1 to each lattice point, i.e.,
- Section 4.2.7 of the Appendix sets forth a code listing for performing the described operation. Further below, these sampling patterns are compared with respect to their convergence.
- the H t -Sobolev norm additionally takes into account noise and the edges of an image.
- the evaluation is done by computing the error norm of a converged reference image to the corresponding test image.
- the resulting values are then plotted in an error graph with the number of samples on the x-axis and the error norm on the y-axis.
- the color is determined by adding the x- and the z-coordinate of the hit point. In the case of the sum modulo 2 being zero, 0.75 returned as color value, 0.25 otherwise.
- the box filter is as a reconstruction filter throughout the present examples. Since this filter is only a coarse approximation of the sine-function, being the ideal reconstruction filter in time space as discussed above, it should be borne in mind that reconstruction errors may be introduced.
- test patterns are compared by means of an error graph.
- an error norm of a reference image to a test image is calculated.
- the resulting value is diagrammed in the error graph, thus displaying the number of samples on the x- and the error norm on the y-axis.
- both axes are scaled logarithmically.
- a reference image was computed by applying a jittered grid sampling pattern with 256 x 256 samples at each pixel.
- the image still contained aliasing artifacts. This can partly be explained by the choice of the reconstruction filter. But on the other hand, the situation would not change much by still increasing the number of samples per pixel.
- the reference image thus converged to an incorrect image. This is due to the fact that the wrong integral has been computed, even if the integration was performed analytically.
- test patterns are applied for each pixel, including: uniform, jittered grid, max-min-dist lattice, randomized lattice, Larcher-Pillichsharnrner, and randomized Larcher-Pillichshammer.
- n rays are shot from the camera into the screen and the resulting color values are averaged by means of the box filter. Then, the randomized sampling patterns have to be recreated for each pixel. Taking a look at the curve of the H ⁇ error-norm, it was found that the uniform grid delivers the worst result. Since the edges of the checkerboard cells introduce very high frequencies into the image signal, the checkerboard is not band-limited. Additionally, structure in the signal combines with structure in the sampling points, when a regular pattern is used for sampling. This explains the aliasing artifacts and thus the high error norm. Increasing the number of samples only attenuates aliasing, as discussed above.
- Jittered grid sampling produces much better results. This can be explained by examining its Fourier transform. There is a spike at the origin with a small "clear zone" around. Thus, the high frequencies are attenuated. Moreover, the disturbing aliasing artifacts are mostly turned into noise. Although the initial error norm for the max-min-dist lattices almost equals the value of uniform and jittered grid, the error declines very fast for an increasing number of samples. It can also be seen that the error curve of the Larcher-Pillichshammer sampling pattern and the max-min-dist lattice nearly coincide and converge to the same value. These two point sets share the important property of having a low discrepancy. In fact their discrepancy is much smaller than the discrepancy of the regular and j ittered grid with the sampling points being much better uniformly distributed.
- both the randomized lattices and the randomized Larcher-Pillichsharnmer sampling pattern have a smaller initial error than their deterministic counterparts. But it has been found that the error curves quickly nearly coincide. Also, both of the randomized point sets behave very similarly with respect to their error norm. However, it has been found that the randomized Larcher-Pillichshammer points achieve slightly better results than the randomized max-min-dist lattices. Though the deterministic and randomized point sets hardly differ with respect to their convergence, the images generated by these patterns distinguish from each other with respect to their visual appearance, particularly for smaller sampling rates ( ⁇ 32).
- the deterministic and randomized quasi-Monte- Carlo sampling patterns deliver the same results, i.e., they converge to the same image.
- the number of sampling points has to be chosen as xRes • yRes • n.
- the test images are created similar to the above scheme. But instead of calling the sampling method for each pixel and looping over the sampling points within that method, all sampling points are looped over once, and it is decided for each point which pixel it belongs to.
- one buffer accumulates the color values and another buffer stores the number of hits for each pixel. After processing all of the sampling points these two buffers are used to average the color values for each pixel in the sense of a box reconstruction filter. Comparing an error graph with one sampling pattern per pixel with an error graph for the whole screen, it will be seen that the overall error of the latter highly exceeds the first and also converges on a higher level.
- the deterministic sampling patterns are applied on pixel basis, neighboring pixels are sampled the same way. The randomization of the max-min-dist lattices brings about a random shift of the lattice pattern within each pixel.
- the pattern itself stays the same, i.e., every pixel is sampled with the same, but randomly shifted lattice. By using one pattern over the whole screen, this correlation disappears. Thus, each pixel is sampled by different points, which do not necessarily cover the pixel in a uniform way. This makes the image quite noisy, especially for low sampling rates.
- the chosen randomization keeps the net structure of the points. Hence, neighboring pixels are still uniformly sampled. If the randomized Larcher-Pillichshammer points are scaled to the whole screen, this does not apply anymore. This is mostly notable at the edges which get very noisy.
- xRes • xRes • n sampling points for a resolution of xRes ⁇ yRes means only that there are n points per pixel on the average. This provides a motivation to search for shifted lattices, which guarantee the same number of lattice points per pixel, as discussed above. Comparing the resulting checkerboard for the lattice Z(262144, 37303) with valid shift (259, 0) c [0, n) 2 , generated by the code listing in section 2.1.4 in the Appendix, the shifted lattice seems to be superior to the pure max-min-dist lattice.
- the checkerboard generated with the shifted lattice 1,(1048576, 10235) contains more aliasing artifacts than the one generated by sampling the whole screen with the max-min-dist lattice 1(10488576, 33919). This might be due to the fact that the shifted lattices are not ideal in the sense of maximum minimum distance, since the first lattice is picked out that provides a valid shift whose min-dist very probably does not equal the maximal possible minimum distance, as discussed above. Therefore only the pure max-min-dist lattices were used for comparison.
- the max-min-dist lattices produce much more aliasing than the Larcher-Pillichshammer points.
- rank-1 lattices are in fact uniform, i.e., regular or rather periodic sampling patterns in contrast to the Larcher-Pillichshammer points, structure in the pattern may regularly sample the high frequencies near the horizon and thus create new structure, visible as aliasing. Therefore the Larcher-Pillichshammer points perform slightly better anti-aliasing than the max-min-dist lattices in the limit.
- the Z 2 -norm could have been chosen instead.
- the Z 2 -norm delivers very similar results, but on a lower error level, because it does not reflect the energy of the edges. Altogether the test patterns converge to the reference image, which in fact represents the numerical approximation of the wrong integral. However, there has to be taken into account not using an ideal reconstruction filter.
- the Larcher-Pillichshammer and max-min-dist sampling patterns show the fastest convergence rate, with the first outperforming the latter.
- FIG. 40 shows a flowchart of a method 740 according to this aspect of the invention.
- a rank-1 lattice is selected in accordance with a maximized minimum distance function corresponding to points in a synthetic image to be generated.
- the lattice is scaled to the entire display screen.
- the image is sampled at low resolution using the lattice as the initial sampling pattern.
- an initial approximation of the image to be rendered is computed using the initial sampling pattern.
- refinement criteria are evaluated for each lattice point, such that the refinement criteria are evaluated in the lattice basis.
- the results of the evaluation are used to evaluate the refinement test for each pixel of the image for which sampling points may need to be refined.
- the sampling pattern is adaptively refined according to a predetermined criterion.
- rank-1 lattices providing maximum minimum distance were used for image synthesis by sampling the whole screen. Considering the resulting images, it can be observed that near the camera there are large areas where the single pixels are completely covered by a light or dark gray checkerboard cell, whereas near the horizon many checkerboard cells fall within one pixel.
- refinement test geometry To determine the complex regions requiring a higher sampling rate, a nearby set of samples, the so-called “refinement test geometry,” is evaluated with respect to some refinement test which typically tries to estimate the local bandwidth of the signal.
- refinement tests can be divided into image space and object space criteria. Image based refinement tests are based on local intensity values of the image. In the simplest form the intensity of the samples in the refinement test geometry are compared. In another test, the maximum and minimum intensity value of four samples forming a small square are compared, suggesting refinement if the difference exceeds a user-deimed threshold. Further tests involve, for example, the local contrast or intensity statistics.
- An aspect of the present invention provides a method for adaptive refinement by rank-1 lattices is developed. As an initial sampling pattern, a maximum minimum distance rank-1 lattice is scaled to the whole screen.
- an initial approximation of the image to be rendered is computed by means of the initial sampling pattern in a first pass. Then the refinement test is evaluated for each pixel, for which the sampling points possibly need to be refined. Since we sample the whole screen, we cannot directly tell for an arbitrary pixel (x, y), which lattice points belong to it. Therefore the refinement criteria are not applied for each pixel, but for each lattice point, i.e., the corresponding criteria are evaluated in the lattice basis.
- the first rendering pass consists in calculating the image in a lattice point buffer, which means that for each sampling point the result of shooting a ray from the camera into the scene is stored at the corresponding location in the buffer.
- n the number of sampling points. Choosing the point buffer as a float array of size n, the resulting value for a sampling point
- the refinement pattern is chosen as a further maximum minimum distance rank-1 lattice by exploiting the lattice cell structure of the initial sampling pattern. This means that the refinement pattern is identified by determining a maximum minimum rank-1 lattice
- the refinement pattern depends on the basis of the initial sampling pattern, i.e., the initial rank-1 lattice, the lattice LQn, a) has to be searched in a sheared basis.
- the refinement pattern is anchored at the point x - ⁇ bi - - x so that the lattice points of the refinement pattern are centered around x.
- the placement of the refinement max-min-dist lattice for the gradient criterion is illustrated by the lattice 770 shown in FIG. 43.
- One special case of this principle is represented by the so-called sampling scheme" which makes use of a hierarchy of uniform grids scaled by a factor and rotated by an angle of arctan i . Whereas this sampling technique recursively operates in object space, it has been adapted to an iterative computation of the point samples in image space.
- Section 4.3.2 of the Appendix shows a code listing that implements the described method for adaptive refinement by rank-1 lattices for one level of refinement.
- it takes the initial sampling pattern, the initial max-min-dist lattice (lattice, not scaled to screen resolution) and the refinement pattern (re ⁇ attice), whose parameters are pre-computed by an initialization method.
- an intensity value is stored in the point buffer by shooting a ray into the scene.
- the selected refinement criterion is evaluated, generating new samples if the resulting value exceeds a predefined threshold. These samples are added to a copy of the initial sampling pattern.
- the refined sampling pattern is used to compute the final image in image space.
- the values of the points belonging to the refinement geometry are accessed from the point buffer via their x-coordinate.
- Section 4.3.2 of the Appendix sets forth a code listing illustrating the adaptive refinement technique described herein. Since the checkerboard scene features only three different intensity values
- the ⁇ 4.3.2 code listing was applied to the image, employing the lattice £(1048576, 33919) as initial sampling pattern and the lattice 1(11, 4) shifted randomly within the lattice cell of 1(1048576, 33919).
- the variation criterion was applied. After refining, the initial pattern was extended by 679452 to a total number of 1728028 sampling points. Each pixel was sampled by 6.6 points on the average with maximal 48 samples per pixel. Since the sampling rate was increased only in regions featuring sharp changes in value, the quality of the refined image was improved by using a smaller number of sampling points at the same time. However, it should be noted that new aliasing artifacts would have appeared in the resulting image if the refinement pattern had not been shifted randomly.
- aliasing is the result of sampling a signal below the Nyquist rate. After sampling, the signal is reconstructed and projected on the medium (TFT, CRT, etc.). Reconstruction errors occur, when the signal is incorrectly recovered from its samples.
- TFT time-function
- this function has an infinite support. Therefore approximations like the box, Gauss or triangle filter are used in practice instead. So far, the discussion has focused on uniform sampling. Since anti-aliasing by non-uniform samples are also examined, it is necessary to consider the non-uniform reconstruction as well.
- approaches have been used, including functional, warping and iteration techniques.
- the simplest approach is to place a copy of an appropriate reconstruction filter over the resample point, just as in the case of a uniformly sampled signal. This is called local filtering.
- the filters are designed with regard to some criteria. In the first place they should not attenuate the central copy of the signal spectrum too much. On the other hand they should dimmish the influence of additional energy resulting from the spectral replicas as good as possible.
- the reconstruction filters have to be normalized, i.e., , where p(x) is the filter function. In other words, every sample is weighted by the filter, with the total filter response summing up to 1. This means that a weighted integration is performed.
- several filters have been proposed, like non-uniform cubic B-splines, difference of Gaussians and the Mitchell and Netravali filters.
- a color value has to be determined for each pixel. This is done by calculating the pixel integral. As seen above, a parametric integration is performed
- g(y) identifies the image function and y parameterizes the pixels in the image.
- test function Z the test function Z :
- the B-spline functions can be implemented in a straightforward way, as illustrated in the code listing set forth in section 4.4.1 of the Appendix.
- the parameter i in the above process is set to — , thereby shifting the support from [i, i + k) to [ ⁇ k/2, k/2).
- the B-spline basis function is called for where 0 ⁇ m ⁇ ResX and 0 ⁇ n ⁇ ResY parameterize the pixels on an integer grid, i.e.
- the parameter ⁇ measures the distance from the ray origin along its direction to the plane, i.e., the checkerboard.
- this parameter t which results from the ray-plane intersection, can be used as a measure for the degree or rather order of the B-spline to choose.
- Section 4.4.3 of the Appendix provides source code for a filtering scheme in accordance with the present discussion that assumes sampling the whole screen, as discussed above. The source code loops over all sampling points, shooting one ray for each point through the camera into the screen.
- the weight of the sampling point is calculated according to equation 4.18 and the weighted color of the hit point is accumulated in an accumulation buffer (accBuffer).
- a second buffer (countBuffer) only accumulates the weights.
- the sum of the weighted color values is divided by the sum of the filter weights in the end.
- variable offset in the above algorithm is used to determine the pixels in whose support the corresponding sampling point (x, y) possibly lies.
- Nr(x, y) ⁇ ( x, y ) I max ⁇ 13c - x ⁇ ,
- y ⁇ y ⁇ ⁇ ⁇ r) This is illustrated by the graph 790 shown in FIG. 45, where the central square defines the set of pixels possibly influenced by a sample for both 1 -pixel and 1.5-pixel radius of the filter kernel.
- the degree of the filter kernel determines the size of its support, which highly affects the run time of the rendering algorithm. Therefore we bound the degree by a certain constant (which is set to 7 in the code listing in section 4.4.3 of the Appendix). However, both this constant and the computation of the B-spline degree depend on the scene to be rendered.
- ClearType technology In contrast to a CRT display, where a pixel is assumed to represent a single square, a single pixel of a TFT (thin film transistor) or LCD (liquid crystal display) display consists of three separate "sub-pixels": a red, a green and a blue one. The human eye perceives such a triplet as one pixel.
- the ClearType technology makes use of those directly addressable subpixels in order to triple the horizontal screen resolution. This means that the horizontal luminance resolution is given by the subpixel resolution, i.e., three times the horizontal screen resolution.
- each subpixel does not only represent a luminance source, but is also characterized by its color. So the luminance resolution is tripled on different color channels.
- ClearType takes into account the different structure of the screen for anti-aliasing.
- ClearType was introduced as a technique for font anti-aliasing in order to improve the readability of text on patterned displays, such as a LCD screen.
- a similar technique was already used in the Apple II personal computer in order to double the effective horizontal resolution of the video display.
- this sub-pixel technology composed one single pixel of two sub-pixels, namely a green and a purple one.
- a pixel is eittier on" or "oil" w ⁇ icn is t ⁇ e reason wny sloped lines appear quite jagged, as it is the line section 810 shown in FIG. 47A. Standard anti-aliasing procedures tend to blur such edges.
- ClearType takes advantage of the subpixel structure by using a part of an adjacent pixel to extend an existing pattern of subpixels, as shown in the line section 820 shown in FIG. 47B. However, ignoring the subpixel color results in color fringing due to a color imbalance. ClearType balances the trade-off between the color error and spatial resolution by means of a perceptual error metric.
- RGB-decimation After prefiltering each color channel of the input image by a low-pass filter, samples are taken at the spatial locations of the same colored sub- pixels, i.e., sampling is performed by considering the displacement of the subpixels (displaced sampling). The principle of RGB decimation is clarified in FIG.
- the number of sampling points must be at least 3 • num • ResX- Res Y, where num represents the desired number of samples per subpixel.
- num represents the desired number of samples per subpixel.
- the clear type rendering algorithm makes use of an accumulation DUirer ⁇ nptsujjer) wnicn accumulates tne weighted color values and an additional buffer (countBuffer) for accumulating the filter weights. Both buffers are initialized with tripled horizontal resolution. By means of these two buffers the checkerboard is rendered in sub-pixel resolution.
- red and green sub-pixel values of each pixel of a TFT or LCD screen are computed in the same place, where they will be displayed afterwards.
- This does not hold true for standard anti-aliasing, which computes the red and green subpixel values as if they were displayed on the same place as the green subpixel. In consequence this leads to blurring, as the red and blue components are shifted a third of a pixel to the right and to the left, respectively.
- the filter weights result from a five-sub-pixel-wide box-filter [11-A-] being anchored on each output subpixel.
- This filter was chosen for the RGB decimation in order to reduce the energy of the side subpixels while still upholding the local color balance.
- After performing the RGB decimation we return to pixel resolution by assigning each pixel the color of its three subpixels. In other words the TFT display is directly addressed on sub-pixel level in this step. This is done in the last nestedy ⁇ r-loop.
- FIGS. 49-51 are a series of drawings illustrating some lattices used to render an infinite checkerboard image.
- FIG. 49 shows a set of lattices 840 with Lin, d) e [0, 1] 2 , withgcd(n, a) ⁇ 1.
- FIG. 50 shows a set of lattices 850, scaled to 512 x 512, with
- the adaptive refinement algorithm described herein can be extended to several levels of refinement. Also, anisotropic filtering can be included in the adaptive filtering method in order to get control of the blur induced by the B-spline filters, especially at the horizon a particular image.
- MAXOP sqrt(MAX_ULLINT) if (x > MAXOP) continue; // Integer overflow for x*x. if (y > MAXOP) continue; // Integer overflow for y*y.
- ⁇ maxmin mindist; a_opt . clear () ; a_opt.push_back (a) ;
Abstract
Description
Claims
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP06774011A EP1908017A2 (en) | 2005-06-23 | 2006-06-23 | Image synthesis by rank-1 lattices |
CA002609286A CA2609286A1 (en) | 2005-06-23 | 2006-06-23 | Image synthesis by rank-1 lattices |
JP2008518498A JP4981798B2 (en) | 2005-06-23 | 2006-06-23 | Image composition by rank-1 lattice |
AU2006261874A AU2006261874B2 (en) | 2005-06-23 | 2006-06-23 | Image synthesis by rank-1 lattices |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US69323205P | 2005-06-23 | 2005-06-23 | |
US60/693,232 | 2005-06-23 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2007002592A2 true WO2007002592A2 (en) | 2007-01-04 |
WO2007002592A3 WO2007002592A3 (en) | 2008-11-20 |
Family
ID=37595946
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2006/024820 WO2007002592A2 (en) | 2005-06-23 | 2006-06-23 | Image synthesis by rank-1 lattices |
Country Status (5)
Country | Link |
---|---|
EP (1) | EP1908017A2 (en) |
JP (2) | JP4981798B2 (en) |
AU (1) | AU2006261874B2 (en) |
CA (1) | CA2609286A1 (en) |
WO (1) | WO2007002592A2 (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2459024A (en) * | 2008-04-03 | 2009-10-14 | Nvidia Corp | Generating a synthetic image using a rank-1 lattice |
US8259106B2 (en) | 2002-05-15 | 2012-09-04 | Mental Images Gmbh | Low-dimensional rank-1 lattices in computer image synthesis |
US11170254B2 (en) | 2017-09-07 | 2021-11-09 | Aurora Innovation, Inc. | Method for image analysis |
US11334762B1 (en) | 2017-09-07 | 2022-05-17 | Aurora Operations, Inc. | Method for image analysis |
CN115115765A (en) * | 2021-03-23 | 2022-09-27 | 想象技术有限公司 | Intersection testing in ray tracing systems |
CN117726774A (en) * | 2024-02-07 | 2024-03-19 | 芯瑞微(上海)电子科技有限公司 | Triangle rasterization method and device based on line generation algorithm and related equipment |
CN117726774B (en) * | 2024-02-07 | 2024-04-30 | 芯瑞微(上海)电子科技有限公司 | Triangle rasterization method and device based on line generation algorithm and related equipment |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150202464A1 (en) * | 2014-01-23 | 2015-07-23 | Mitsubis | Multi-Criteria Optimization in Particle Beam Dose Optimization |
US10388059B2 (en) | 2016-10-03 | 2019-08-20 | Nvidia Corporation | Stable ray tracing |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4973111A (en) * | 1988-09-14 | 1990-11-27 | Case Western Reserve University | Parametric image reconstruction using a high-resolution, high signal-to-noise technique |
US20020126129A1 (en) * | 2001-01-16 | 2002-09-12 | Snyder John M. | Sampling-efficient mapping of images |
US20050278406A1 (en) * | 2002-05-15 | 2005-12-15 | Alexander Keller | System and computer-implemented method for evaluating integrals using stratification by rank-1 lattices |
US7006110B2 (en) * | 2003-04-15 | 2006-02-28 | Nokia Corporation | Determining a coverage mask for a pixel |
US20070046686A1 (en) * | 2002-11-19 | 2007-03-01 | Alexander Keller | Image synthesis methods and systems |
US20070109320A1 (en) * | 2002-05-15 | 2007-05-17 | Sabrina Skibak | Image synthesis by rank-1 lattices |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2594212B2 (en) * | 1993-01-14 | 1997-03-26 | 株式会社エイ・ティ・アール通信システム研究所 | Adaptive and hierarchical grid representation generator for images |
US6678406B1 (en) * | 2000-01-26 | 2004-01-13 | Lucent Technologies Inc. | Method of color quantization in color images |
JP2002328211A (en) * | 2000-07-28 | 2002-11-15 | Matsushita Electric Ind Co Ltd | Reflection sheet, method for manufacturing the same, and display device using the same |
JP2003132353A (en) * | 2001-10-29 | 2003-05-09 | Pasuko:Kk | Center line generator |
JP2004022565A (en) * | 2002-06-12 | 2004-01-22 | Nippon Telegr & Teleph Corp <Ntt> | Light receiving element and collar sensor device using it |
US20050093894A1 (en) * | 2003-10-30 | 2005-05-05 | Tretter Daniel R. | Generating an displaying spatially offset sub-frames on different types of grids |
-
2006
- 2006-06-23 EP EP06774011A patent/EP1908017A2/en not_active Withdrawn
- 2006-06-23 CA CA002609286A patent/CA2609286A1/en not_active Abandoned
- 2006-06-23 WO PCT/US2006/024820 patent/WO2007002592A2/en active Application Filing
- 2006-06-23 AU AU2006261874A patent/AU2006261874B2/en not_active Ceased
- 2006-06-23 JP JP2008518498A patent/JP4981798B2/en active Active
-
2011
- 2011-08-08 JP JP2011173177A patent/JP5199424B2/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4973111A (en) * | 1988-09-14 | 1990-11-27 | Case Western Reserve University | Parametric image reconstruction using a high-resolution, high signal-to-noise technique |
US20020126129A1 (en) * | 2001-01-16 | 2002-09-12 | Snyder John M. | Sampling-efficient mapping of images |
US20050278406A1 (en) * | 2002-05-15 | 2005-12-15 | Alexander Keller | System and computer-implemented method for evaluating integrals using stratification by rank-1 lattices |
US20070109320A1 (en) * | 2002-05-15 | 2007-05-17 | Sabrina Skibak | Image synthesis by rank-1 lattices |
US20070271322A1 (en) * | 2002-05-15 | 2007-11-22 | Alexander Keller | Computer Graphics Systems and Methods Using Stratification by Rank-1 Lattices |
US20070046686A1 (en) * | 2002-11-19 | 2007-03-01 | Alexander Keller | Image synthesis methods and systems |
US7006110B2 (en) * | 2003-04-15 | 2006-02-28 | Nokia Corporation | Determining a coverage mask for a pixel |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8259106B2 (en) | 2002-05-15 | 2012-09-04 | Mental Images Gmbh | Low-dimensional rank-1 lattices in computer image synthesis |
GB2459024A (en) * | 2008-04-03 | 2009-10-14 | Nvidia Corp | Generating a synthetic image using a rank-1 lattice |
GB2459024B (en) * | 2008-04-03 | 2011-01-26 | Nvidia Corp | Low-dimensional rank-1 lattics in computer image synthesis |
US11170254B2 (en) | 2017-09-07 | 2021-11-09 | Aurora Innovation, Inc. | Method for image analysis |
US11334762B1 (en) | 2017-09-07 | 2022-05-17 | Aurora Operations, Inc. | Method for image analysis |
US11748446B2 (en) | 2017-09-07 | 2023-09-05 | Aurora Operations, Inc. | Method for image analysis |
CN115115765A (en) * | 2021-03-23 | 2022-09-27 | 想象技术有限公司 | Intersection testing in ray tracing systems |
CN115115765B (en) * | 2021-03-23 | 2024-01-09 | 想象技术有限公司 | Intersection testing in ray tracing systems |
CN117726774A (en) * | 2024-02-07 | 2024-03-19 | 芯瑞微(上海)电子科技有限公司 | Triangle rasterization method and device based on line generation algorithm and related equipment |
CN117726774B (en) * | 2024-02-07 | 2024-04-30 | 芯瑞微(上海)电子科技有限公司 | Triangle rasterization method and device based on line generation algorithm and related equipment |
Also Published As
Publication number | Publication date |
---|---|
CA2609286A1 (en) | 2007-01-04 |
WO2007002592A3 (en) | 2008-11-20 |
AU2006261874B2 (en) | 2010-09-02 |
JP2009501966A (en) | 2009-01-22 |
EP1908017A2 (en) | 2008-04-09 |
AU2006261874A1 (en) | 2007-01-04 |
JP4981798B2 (en) | 2012-07-25 |
JP5199424B2 (en) | 2013-05-15 |
JP2012009053A (en) | 2012-01-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7589729B2 (en) | Image synthesis by rank-1 lattices | |
US7432935B2 (en) | Image synthesis methods and systems for generating sample points in a graphics scene | |
US7106322B2 (en) | Dynamically adjusting a sample-to-pixel filter to compensate for the effects of negative lobes | |
US6850236B2 (en) | Dynamically adjusting a sample-to-pixel filter in response to user input and/or sensor input | |
WO2007002592A2 (en) | Image synthesis by rank-1 lattices | |
KR20160001675A (en) | Graphics system and Method of graphics processing | |
US11062499B2 (en) | Distance-field texture interpolation optimizations | |
EP3598391B1 (en) | Variable resolution rendering | |
Neumann | Volume reconstruction and parallel rendering algorithms: A comparative analysis | |
WO2002069274A1 (en) | Memory efficient shear-warp voxel projection algorithm | |
AU2006279337B2 (en) | Image synthesis methods and systems | |
JP4448692B2 (en) | Data compression and method and apparatus for decompressing compressed data | |
Greisen et al. | Analysis and VLSI implementation of EWA rendering for real-time HD video applications | |
EP4125042A1 (en) | Anisotropic texture filtering using weights of an anisotropic filter | |
Dammertz et al. | Image synthesis by rank-1 lattices | |
WO2019042272A2 (en) | System and method for multi-view rendering | |
Becher et al. | Accelerating GPU rendering of 2D visualizations using resolution scaling and temporal reconstruction | |
Goss et al. | Study of supersampling methods for computer graphics hardware antialiasing | |
GB2459024A (en) | Generating a synthetic image using a rank-1 lattice | |
Xie et al. | Multi-resolution LOD volume rendering in medicine | |
Mason | Predictive hierarchical level of detail optimization | |
Manson et al. | Bilinear accelerated filter approximation | |
JP2023538828A (en) | Antialiasing for distance field graphics rendering | |
Grünschloß | Motion blur | |
Horé et al. | A simple scaling algorithm based on areas pixels |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
ENP | Entry into the national phase in: |
Ref document number: 2609286 Country of ref document: CA |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2006774011 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2006261874 Country of ref document: AU |
|
ENP | Entry into the national phase in: |
Ref document number: 2008518498 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase in: |
Ref country code: DE |
|
ENP | Entry into the national phase in: |
Ref document number: 2006261874 Country of ref document: AU Date of ref document: 20060623 Kind code of ref document: A |