CN101588437B - Method and apparatus for block-based compression of light-field images - Google Patents
Method and apparatus for block-based compression of light-field images Download PDFInfo
- Publication number
- CN101588437B CN101588437B CN2009101385392A CN200910138539A CN101588437B CN 101588437 B CN101588437 B CN 101588437B CN 2009101385392 A CN2009101385392 A CN 2009101385392A CN 200910138539 A CN200910138539 A CN 200910138539A CN 101588437 B CN101588437 B CN 101588437B
- Authority
- CN
- China
- Prior art keywords
- light field
- image
- field image
- block
- little
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/765—Interface circuits between an apparatus for recording and another apparatus
- H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
- H04N5/772—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera the recording apparatus and the television camera being placed in the same enclosure
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0075—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for altering, e.g. increasing, the depth of field or depth of focus
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/79—Processing of colour television signals in connection with recording
- H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
- H04N9/804—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
- H04N9/8042—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction
Abstract
A method and apparatus for the block-based compression of light-field images. Light-field images may be preprocessed by a preprocessing module into a format that is compatible with the blocking schemeof a block-based compression technique, for example JPEG. The compression technique is then used to compress the preprocessed light-field images. The light-field preprocessing module reshapes the ang ular data in a captured light-field image into shapes compatible with the blocking scheme of the compression technique so that blocking artifacts of block-based compression are not introduced in the final compressed image. Embodiments may produce compressed 2D images for which no specific light-field image viewer is needed to preview the full light-field image. Full light-field information is contained in one compressed 2D image.
Description
Technical field
The present invention relates to the light field treatment of picture, more specifically, relate to the method and apparatus that is used for the light field image is carried out block-based compression.
Background technology
Traditional camera can not be caught a large amount of optical information.Particularly, traditional camera can not be caught the information that goes up the position about aperture (aperture), and wherein aperture is the place that different light rays enters camera.During operation, the conventional digital camera captured representative is beaten two dimension (2D) image of the light summation of each point on the light-sensitive element in camera.Yet this 2D image does not comprise any information that distributes about the direction of light of beating on light-sensitive element.The directional information at pixel place is corresponding to the positional information at aperture place.
On the contrary, the light field camera is sampled to four-dimensional (4D) optical phase space or light field, and in doing so, catches the information about the direction distribution of light.This information that the light field camera is caught can be called light field, full light function or radiance.In calculating photography, light field is the 4D record of all light among the 3D.Radiance is described spatial information and angle information, and radiance is defined as the density of the energy of every three-dimensional viewpoin (is unit with the radian), per unit area.The light field camera is caught radiance; Therefore, the burnt light field image that obtains of initial mistake can focus on again, and noise can be lowered, and the visual angle can change, and can obtain other light-field effects.
Can utilize traditional camera to catch light field.In a conventional method, utilize MxN the image of traditional camera from the diverse location capturing scenes.For example, if catch the 8x8 image, then produce 64 images from 64 diverse locations.(i j) obtains pixel, and is placed in the piece, to generate 64 pieces in each position from each image.
Figure 1A shows the light field camera of exemplary prior art, perhaps camera array, and this array has used the array of two or more object lens 110.Each object lens focusing perhaps alternatively focuses on the independent light-sensitive element 108 in the specific region of light-sensitive element 108.This light field camera 100 can be considered as the combination of two or more traditional cameras, and wherein each camera is recorded in the image of object on the specific region of light-sensitive element 108 simultaneously, perhaps alternatively is recorded on the specific light photosensitive elements 108.The image of catching can be combined subsequently to form an image.
Figure 1B shows exemplary prior art plenoptic camera, and it is the light field camera of another kind of type, and uses single object lens and microlens or lenslet array 106, and this array for example comprises about 100,000 lenslets.Lenslet array 106 usually with light-sensitive element 108 (for example, charge-coupled device (CCD)) apart than short distance (about 0.5mm).The original image that plenoptic camera 102 is caught is made up of the array of the little image of main camera lens 108 (normally circular).These little images can be called little image.Lenslet array 106 supports plenoptic cameras 102 to catch light field, that is, and and document image intensity not only, and write down intensity distributions on each some place different directions.The beam splitting that each lenslet will enter from main lens 104 becomes come from the ray of different " pin hole " positions on the aperture of main lens 108.Each of these rays is registered as the pixel on the light-sensitive element 108, and the pixel under each lenslet forms the n pixel image jointly.This n pixel region under each lenslet can be called grand pixel, and camera 102 generates little image at each Hong Xiangsuchu.Have the full exposure sheet that the camera 102 of 100,000 lenslets for example catches and will comprise 100,000 grand pixels, and thereby 100,000 little images of formation object.For example, referring to Fig. 2.By suitably from each grand pixel selection pixel, the conventional pictures of little image creation object of the object that can catch from Hong Xiangsuchu.
The light field of catching from light field camera (comprising plenoptic camera) saves as the 2D image usually, and this 2D image comprises the array of " fragment (tile) " or " little image ".The compression of light field image is an important problem for calculating photography.Owing to the 4D character of light field, and the 2D sheet of light field is equal to the such fact of conventional pictures, and unpressed file trends towards bigger, and may occupy the space of thousands of Mbytes.Simultaneously, there is redundancy in the data: have approximately uniform radiance (, being identical) for lambert (Lambertian) surface since all rays of a surface point.Like this, there is the motivation that the light field image is compressed.
Traditionally, used existing lossy and loss-free image/video compress technique that the light field image is compressed.Some traditional approach is considered as independent image with the 2D sheet in the light field image, and each is compressed separately.In other modes, 4D light field image packets is contained in the 2D image, and it is an image by the conventional method simple compression.These modes are not utilized distinctive information of light field image and redundancy, but they are considered as general image.
Propose some modes and come special squeezed light field picture.Some conventional method is considered as frame in the video with each the 2D sheet in the 4D light field image.In other words, the 2D angular image in the 4D light field image is used to create video, and this video uses video-frequency compression method to compress.Yet, utilize the light field image of the special-purpose compress technique compression of light field to need specific reader to check the light field image usually.
JPEG is common traditional images compression standard.JPEG represents JPEG (joint photographic experts group), promptly creates the title of the committee of Joint Photographic Experts Group.JPEG is based on the example of the compress technique of piece.JPEG is divided into the block of pixels of 8x 8 with image, and perhaps more generally, block-based compress technique becomes the block of pixels of mxn with image division, and uses certain transforming function transformation function to compress these pieces.Be well known that, because image division is become piece, there are the problem that generates " piecemeal artifact " in JPEG and other block-based compress techniques, wherein the image after the compression looks like and is made up of piece, vertical artifact/horizontal the artifact (for example, vertical line or horizontal line, discontinuous or striped) that perhaps has other introducings.
Joint Photographic Experts Group and other block-based compress techniques can be used for direct squeezed light field picture, and need not consider the details of light field data.Yet owing to the characteristic and the block character of compression paracycle of light field image, the result trends towards relatively poor, comprises tangible piecemeal artifact.This piecemeal artifact can seriously be damaged the angle information in the light field image, and therefore may limit level and the vertical parallax that uses these images to reach.
Fig. 2 shows the light field image 200 that utilizes plenoptic camera (for example, the plenoptic camera shown in Figure 1B 102) to catch.The zone 202 that comes from the expansion of image 200 shows light field image 200 to be made up of a lot of independent grand pixels, and each grand pixel comprises circular little image.Zone 202 shows light field data compression, that do not capture.Zone 204 shows with JPEG compression applications light field data to image 200, that come from the same area of image 200.The piecemeal artifact that the JPEG compression is introduced is for example high-visible in little image of 16 grand pixels shown in the subregion 206 that launches.Image 208 shows unpressed same subregion, to be used for comparison.
Yet, usually, utilize the light field of traditional light field special method compression can not utilize traditional and open and check than the general image reader.Do not have light field and check that the user of application may wish to utilize traditional image viewer (such as the image viewer of the image that can check the JPEG compression) to come the simple 2D of preview light field to represent.
Summary of the invention
The execution mode that is used for the light field image is carried out the method and apparatus of block-based compression has been described.Execution mode the light field image can be anticipated into the compatible form of the piecemeal of block-based compress technique (for example, the JPEG compression standard) mechanism, wherein this block-based compress technique is used to carry out the actual compression to the light field image.The execution mode that is used for the light field image is carried out the method for block-based compression can be implemented as instrument, module, built-in function, plug-in unit, stand-alone application etc., perhaps can realize in these.In order to simplify, being implemented in this and can being called the light field pretreatment module of execution mode.
The execution mode of light field pretreatment module can become suitable form with the light field image transitions of catching, and makes not introduce block-based compression (for example, piecemeal artifact JPEG) in final compressed image.The shape of the piecemeal mechanism compatibility of the block-based compress technique that the light field pretreatment module is shaped as the angle-data in the light field image of catching again and is using (for example, for JPEG, size is the square of 8x8,16x16 etc.), the light field image that obtains is suitable for the piecemeal mechanism of compress technique.For example, if used JPEG, then jpeg block boundary has become the natural boundary of angular image in the light field.
Execution mode can produce the 2D image of compression, for this 2D image, does not need special light field image viewer to come preview overall optical field picture.Whole light field information is comprised in the 2D image, it can be by traditional block-based compress technique (for example, JPEG) compress in the mode of quick and robust, and final compressed image is not comprised in not use under the pretreated situation described herein light field is compressed issuable piecemeal artifact.In other words, execution mode provides the high-performance compression of no piecemeal artifact in final compression light field.
In execution mode, obtain the light field image catch by the light field pretreatment module.The light field image of catching carries out preliminary treatment by the light field pretreatment module, with produce block size and shape all with will be used for the preliminary treatment light field image of block-based compress technique compatibility of squeezed light field picture.In one embodiment, if at the calibration chart of light field camera as available, then the camera light field image of catching can use calibration chart to look like to carry out regularization.Pretreated light field image is compressed by compression engine subsequently, and to generate the light field image through overcompression, this compression engine realizes block-based compress technique.Light field image through overcompression for example can store in the storage medium storage medium such as system storage, disk drive, DVD, CD etc. into.Can use through the light field image of overcompression and to be configured to check or otherwise handle in order to open and to check any application of the image that generates by block-based compress technique.For example,, then can use any application that to open and to show jpeg file, check light field image through overcompression if used the JPEG compression.
Description of drawings
Figure 1A shows the light field camera of exemplary prior art, perhaps camera array, and it uses the array of two or more object lens.
Figure 1B shows the plenoptic camera of exemplary prior art, and it uses single object lens and microlens array.
Fig. 2 shows the light field image that utilizes plenoptic camera (for example, the plenoptic camera shown in Figure 1B) to catch, and shows issuable piecemeal artifact when utilizing block-based compress technique to compress original image.
Fig. 3 A and Fig. 3 B show according to an execution mode, using before block-based compress technique compresses the preliminary treatment that the exemplary light field picture is carried out.
Fig. 4 shows the exemplary light field picture of utilizing plenoptic camera (for example, the plenoptic camera shown in Figure 1B) to catch.
Fig. 5 according to an execution mode, show the high level block diagram of the data flow in light field preliminary treatment and the compression method.
Fig. 6 according to an execution mode, be used for before the light field image is carried out block-based compression, grand pixel or piece from the light field image are carried out pretreated method flow diagram.
Fig. 7 shows the example calibration image at full light light field camera.
Fig. 8 A shows according to an exemplary interpolative method execution mode, that use Laplce (Laplacian) interpolation.
Fig. 8 B shows the exemplary convolution kernels that can use in execution mode.
Fig. 9 A show according to an execution mode, to utilizing the preliminary treatment of the light field image that optical field imaging system (such as, the plenoptic camera of Figure 1B) catches, in this optical field imaging system, use microlens in the back of main lens.
Fig. 9 B show according to an execution mode, to utilizing the preliminary treatment of the light field image that optical field imaging system (such as, the light field camera of Figure 1A) catches.
Figure 10 shows the exemplary computer system that can use in execution mode.
Although described the present invention at plurality of embodiments and exemplary drawings at this, it will be understood to those of skill in the art that execution mode or accompanying drawing that the present invention is not limited to describe by example.Should be understood that the drawings and specific embodiments also are not intended to the present invention is limited to particular forms disclosed, on the contrary, are intended to cover all improvement, equivalent and the substitute that fall into spirit and scope of the invention.Title is only for organizational goal as used herein, and is not intended to the scope that is used to limit this description.As use in the application's full text, vocabulary " can " be that (that is, the meaning is possible) uses on the permission meaning, rather than (that is, the meaning is necessary) use on the compulsory meaning.Similarly, word " comprises ", the meaning of " comprising " and " containing " is to include, but are not limited to.
Embodiment
The various execution modes that the light field image carried out the method and apparatus of block-based compression have been described.Execution mode can be and the compatible form of piecemeal mechanism of block-based compress technique (for example, the JPEG compression standard) that wherein this block-based compress technique is used for the light field image is carried out actual compression with the preliminary treatment of light field image.The execution mode that is used for the light field image is carried out the method for block-based compression can be implemented as instrument, module, built-in function, plug-in unit, stand-alone application etc., perhaps can realize in these.In order to simplify, being implemented in this and can being called the light field pretreatment module of execution mode.
The execution mode of light field pretreatment module can become suitable form with the light field image transitions of catching, and makes not introduce block-based compression (for example, piecemeal artifact JPEG) in final compressed image.The shape of the piecemeal mechanism compatibility of the block-based compress technique that the light field pretreatment module is shaped as the angle-data in the light field again and is using (for example, for JPEG, size is the square of 8x8,16x16 etc.), the light field image that obtains is suitable for the piecemeal mechanism of compress technique.For example, if use JPEG, then jpeg block boundary has become the natural boundary of angular image in the light field.
Execution mode can produce the 2D image of compression, for this 2D image, does not need special light field image viewer to come preview overall optical field picture.Full light-field information is comprised in the 2D image, it by traditional block-based compress technique (for example, JPEG) compress in the mode of quick and robust, and final compressed image is not comprised in not use under the pretreated situation described herein light field is compressed issuable piecemeal artifact.In other words, execution mode provides the high-performance compression of no piecemeal artifact in final compression light field.
In order to simplify, usually execution mode is described as: the block-based compress technique that the pretreated light field image that the JPEG compression standard is generated with the execution mode of doing the light field pretreatment module compresses at this.Use light field image and traditional Joint Photographic Experts Group back compatible through overcompression of the execution mode generation of JPEG compression standard.Use the execution mode of JPEG compression can realize good quality compression to the light field image, making simultaneously will can be the same with simple 2D picture through the light field of overcompression, and utilization can open and show that any application of JPEG (.jpg) file checks.When using the JPEG compression, the light field through overcompression can be output as 2D JPEG (.jpg) file, and thereby can utilize any should being used for that to open and to show the .jpg file to check.Yet, should be understood that execution mode is not limited to the JPEG compression standard as block-based compress technique, but can be suitable for any block-based compress technique.
Fig. 3 A and Fig. 3 B show according to an execution mode, using before block-based compress technique compresses the preliminary treatment that the exemplary light field picture is carried out.Fig. 3 A shows the exemplary 4D light field 300 of being caught by light field camera (such as the camera 100 of Fig. 1).It should be noted that the 4D light field is the 2D array of 2D image (view that comes from different projection centres).These 2D images can be called little image.It should be noted that the information in each little image is rectangle.In this example, array is 6x6, thus in light field 300, there are 36 independent images, or little image.
Make location of pixels in the little image of given 2D (for example, little image 302) be (i, j), wherein i=1...N and j=1...M, wherein N is that the row number of pixel in the little image of 2D and M are the row number of pixel.(i j), and puts it among the piece Bij (referring to piece 304) to obtain pixel from each little image of 36 little images.If have m horizontal image and n vertical image in light field 300, then this piece is initially nxm pixel.In this example, this piece will be a 6x6 pixel.In addition, there be NxM piece 304.This NxM piece 304 is placed in the image, makes all angles of light field and spatial information all be comprised in (referring to the image 310 of Fig. 3 B) in this image.If use the original block 304 of dimension as nxm, the big young pathbreaker of the image that then obtains is the individual pixel of (Nxn) x (Mxm).
Yet, should be noted that block size nxm not necessarily directly with block-based compress technique (such as, piecemeal mechanism JPEG) is compatible.In this example, the block size of 6x6 pixel typical case not direct and JPEG compresses branch block size (8x8 pixel) compatibility.Therefore, the execution mode of light field pretreatment module described here can carry out preliminary treatment (as Fig. 3 B 306 indicated) to piece 304, so as to produce with the compatible dimension of the piecemeal mechanism of block-based compress technique be the piece 308 of n ' xm '.In this example, piece 304 is resized and is 8x8 pixel, so that compatible with the piecemeal mechanism of JPEG.The NxM piece 308 of adjusting size can place image 310 subsequently.The size of this image 310 is the individual pixel of (Nxn ') x (Mxm ').Subsequently, can use block-based compress technique that image 310 is compressed, and can not be created in the image 310 issuable piecemeal artifact under the situation of the piece 304 that uses initial incompatible dimension.
In this example, the block size nxm of piece 304 is 6x 6, and (8x8) is little for this block size than block-based compress technique.In this case, the Pixel Information in original block 304 or the zone can be by interpolation, to fill up the pixel value in the new bigger piece 308.Can use any in the various interpolation techniques.In one embodiment, for example, can use Laplce's interpolation.
Below situation also be possible, that is, and in some light field image, the block size that nxm (default or " original " block size) uses greater than block-based compress technique.In addition, n must equal m.For example, nxm can be 11x14 or 11x11 or 9x9 or 8x10 or 21x21 etc.In some block-based compress technique (such as JPEG), piecemeal mechanism can allow multiple basis to divide block size.For example, it is 8 multiple that JPEG can allow block size, such as 8x8,16x16 or 32x32 etc.Like this, in one embodiment, next bigger block size that block-based compress technique provides can be used for the block size of piece 308.For example, if nxm is 12x14, and the block size of block-based compress technique permission 8x8 or 16x16, then 16x16 can be used for the big or small n ' xm ' of piece 308.
Alternatively, if n ' xm ' less than nxm, then simply cutting piece 306 to produce littler piece 308.For example, if nxm is 10x10, and n ' xm ' is 8x8, then can simply piece 306 be cut to 8x8.It should be noted that, in some light field image, marginal information in little image may be noisy owing to the character of the light field camera that is used to catch light field information under any circumstance, so can carry out cutting to edge pixel under the situation of not losing a lot of available light field information.It is also noted that the pixel in little image is many more, the negative effect that cutting causes (losing of light field information) is more little.Like this, cutting may not be used on the little image that comprises limited number of pixels, and the light field information in each pixel of so little image is more valuable than having the light field information that more manys in each pixel of little image of pixel.
More than described a kind of like this process, by this process, the light field pretreatment module can be adjusted into size and the machine-processed consistent piece 308 of the piecemeal of block-based compress technique with it before piece 304 is put into image 310.Alternatively, can form image from the piece 304 of original dimensions (nxm), the light field pretreatment module can repeat each piece is adjusted size on the piece 304 in image subsequently, to form the new images of being made up of the size piece 308 consistent with the piecemeal mechanism of block-based compress technique 310.
In some light field image, different except being the rectangle shown in Fig. 3 A and Fig. 3 B with each little image, the information that writes down in each piece or grand pixel can be to represent the circle or the disk (being called little image) of camera aperture.Fig. 4 shows the exemplary light field picture 400 of utilizing plenoptic camera (for example, the plenoptic camera shown in Figure 1B 102) to catch.Zone 402 shows the spreading area of light field image 400, and shows exemplary circular little image from the grand pixel 404 of spreading area 402.In one embodiment, be not as under the situation of the image 300 of Fig. 3 A, each grand pixel 404 simply to be adjusted size, at first shaping is (for example again with the little image of circle for the light field pretreatment module, by cutting) be rectangle region or piece, and will be adjusted into as the size of the piece that obtains subsequently and the piecemeal of block-based compression mechanism machine-processed compatible (for example, being adjusted into for the JPEG pressure is the piece of 8x8).If desired, the Pixel Information in the piece can be by interpolation, to produce the pixel value in the new piece.Can use distinct methods to obtain having piece with the compatible size of the piecemeal mechanism of block-based compress technique.For example, suppose to use the JPEG compression.If less than 8x8, then piece is resized and is 8x8, wherein utilizes interpolation to produce pixel value from the district of little image (piece) cutting.But if block is greater than 8x8 less than 16x16, and then can to reduce (for example, by cutting) be 8x8 to the size of piece, perhaps is used to produce the interpolation of pixel value and is adjusted into 16x16.
In case little image has been carried out cutting, then the piece of cutting is adjusted to the piece with the compatible size of the piecemeal of block-based compress technique mechanism, adjusts the image that big or small piece can place the image 310 that is similar to Fig. 3 B.The angle of light field and spatial information are comprised in the consequent image.Can use block-based compress technique to come consequent image is compressed subsequently, and can not produce the piecemeal artifact, this is because the block size of consequent image is identical with the block size that uses in the piecemeal mechanism of block-based compression mechanism.
Fig. 5 according to an execution mode, show the high level block diagram of the data flow in light field preliminary treatment and the compression method.The light field image 500 (for example, the image 300 of Fig. 3 A, the perhaps image 400 of Fig. 4) that 502 acquisitions of light field pretreatment module are caught.Image 500 carries out preliminary treatment by light field pretreatment module 502, to generate pretreated light field image 504, its block size and the block-based compress technique compatibility that will be used for compressed image 504.In one embodiment, if can use as 501 at the calibration chart of light field camera, then the camera light field image 500 of catching can use calibration chart to look like to carry out regularization.Usually with the light field image of catching divided by the calibration chart picture with this image of regularization.Subsequently, compress, to generate light field image 508 through overcompression by 506 pairs of images 504 of compression engine of realizing block-based compress technique.Image 508 through overcompression for example can be stored in the storage medium 510, such as system storage, disk drive, DVD, CD etc.Light field image 508 through overcompression can use any application to check or otherwise handle, and wherein this any application is configured to and opens and check the image that block-based compress technique generates.For example, if used the JPEG compression, then can use to open and to show jpeg file any should be used for checking image 508.
The execution mode of light field pretreatment module 502 and compression engine 506 can be implemented on the computer system.Figure 10 illustrates the example system that execution mode can be realized thereon.Light field pretreatment module 502 can be implemented as instrument, module, built-in function, plug-in unit, stand-alone application etc., perhaps can realize in these.Light field pretreatment module 502 can provide user interface, and this user interface provides one or more texts and/or graphical user interface parts, pattern or technology, and the user can control the pretreated various aspects of light field image via it.For example, user interface can comprise user interface component, these user interface component allow the user to select input file and output file, appointment whether the compress technique of using, block size, the appointment that appointment is used for pretreated image 504 are amplified (interpolation) or dwindled (cutting) piece, or the like.
Fig. 6-Fig. 9 illustrated in greater detail before image is carried out block-based compression, be used for the light field image is carried out the execution mode of pretreated method, particularly, this preliminary treatment is applied to the image the image 400 in Fig. 4, and wherein each grand pixel comprises circular little image in image 400.Yet, it should be noted that some preliminary treatment at least that illustrates and describe at these accompanying drawings can also be applied in Fig. 3 A image 300 in the preliminary treatment of image, in image 300, each grand pixel is the little image of rectangle.As example, these accompanying drawing hypothesis will be carried out block-based compression according to the JPEG compression standard, and the block size of the piecemeal mechanism of using in the JPEG compression is a 8x8 pixel.As example, these accompanying drawings are also supposed: the part of little image will be extracted and convergent-divergent or interpolation so that be suitable for bigger piece, and will use Laplce's interpolation technique.In various execution modes, can use other the block-based compress techniques except that JPEG, the block size except that 8x8 pixel can be used, and other amplification methods that comprise other interpolation techniques can be used.It should be noted that in one embodiment, can be by the simple cutting of importing little image be dwindled, to be fit to less block size.
Fig. 6 according to an execution mode, be used for the flow chart that before the light field image is carried out block-based compression, grand pixel or piece from the light field image carried out pretreated method.Method shown in Fig. 6 can realize in the light field pretreatment module such as the module 502 of Fig. 5.
Light field pretreatment module 502 can obtain the first light field image block 602 or grand pixel from the pretreated light field image 500 of needs.In this example, illustrated blocks 602 is 20x20 pixel approximately.Yet, the execution mode of light field pretreatment module can be configured to work with the light field image block 602 of all size, and piece 602 must not be foursquare, even must not rectangle, this is because the execution mode of preprocess method can be worked together with the input block of other shapes (such as, hexagonal shape).In this example, the light field informational needs that comes from the little image in the piece 602 (it is a rectangle, and is about 20x20 pixel) is suitable for using block-based compress technique (8x 8 block of pixels of for example, JPEG) compressing.
In one embodiment, if can use as 501 at the calibration chart of light field camera, then the camera light field image 500 of catching can use calibration chart to look like to carry out regularization.Usually with the light field image of catching divided by the calibration chart picture with this image of regularization.Like this, in one embodiment, can be by utilizing calibration chart that the light field camera catches as 501 respective alignment piece 600, with light field image block 602 regularizations (shown in 604) divided by coming from light field image block 602.The light field image 500 of catching is being carried out in the preliminary treatment, can by the light field piece 602 of will catch divided by from calibration chart as 501 respective alignment piece 600, and with each piece 602 regularization in the image 502.In one embodiment, not to carry out regularization to whole 602, but only to comprising the regional regularization of circular little image.
Fig. 7 shows the example calibration image that is used for full light light field camera.Calibration chart is images of the complete white scene obtained of light field camera (in this example, be such as camera among Figure 1B 102 plenoptic camera) as 700.With reference to camera 102, calibration chart is as the aperture of circle bigger in 700 corresponding to object lens 104, and each small circle in the great circle or little image are corresponding to the concrete microlens in the microlens array 106 or the aperture of tiny lens.The regularization process can be improved the quality of the boundary of each microlens aperture circle, and strengthens the overall appearance of light field image, and removing may be by the color error ratio of light field equipment (camera) introducing.Color error ratio is positioned at the edge of micrographics, and is because microlens causes in the decay of boundary.Zone 702 shows and comes from calibration chart as 700 exemplary spreading area, and grand pixel 704 show the grand pixel of exemplary expansion that comes from zone 702 or piece with and circular little image.
If carried out regularization, then after regularization, piece 602 by shaping again so that obtain the rectangle region of light field information.In some light field image 500 (those that plenoptic camera 102 is caught shown in Figure 1B), the light field information of record is disk or the circle (representative camera aperture) in each piece 602, is called as little image.This circular light field information need be shaped as the piecemeal mechanism compatible rectangle region of size with block-based compress technique again.In one embodiment, this can finish by the little image in the piece 602 being carried out the less piece of cutting acquisition, shown in 606.In this example, cutting 606 has produced the 5x5 piece 608 of light field information.At 610 places, the size of the piece 608 of cutting is adjusted to compress technique subsequently with the block size that uses, and is 8x 8 in this example, and uses interpolation technique to fill up pixel value in the 8x8 piece, adjusts big or small piece 612 to generate.In one embodiment, used Laplce's interpolation technique, found the solution Laplace's equation therein to determine pixel value.The piece 612 of adjusting size is created in corresponding position at pretreated light field image 504, and the piece 612 that perhaps will adjust size is placed in one.At 614 places, will pretreated more light field image block if exist, then this method obtains next image block 602 from the light field image 500 of catching, and continues.Otherwise the preliminary treatment of light field image 500 is finished.With reference to figure 5, by the light field pretreatment module preliminary treatment that the light field image 500 of catching carries out is finished, and pretreated light field image 504 is output for compression engine 506 and compresses.
Refer again to Fig. 6, the size of the piece 608 of cutting can be different from the 5x5 that only uses as example, and adjust big or small process can be different because of the cutting size of using.For example, suppose to use the JPEG compression.If less than 8x8, then piece is resized and is 8x8, wherein utilizes interpolation to produce pixel value from district's (for example, piece 608) of little image cutting.If block 608 greater than 8x8 but less than 16x16, then the size of piece can be adjusted to 8x8 down, perhaps be used to produce pixel value interpolation and on be adjusted to 16x16.
In one embodiment, the interpolation technique of light field pretreatment module 502 uses can be Laplce's solver.Fig. 8 A and Fig. 8 B show according to an exemplary interpolative method execution mode, that use Laplce's interpolation.The target of interpolation is to use the pixel value in the zone 806 of cutting, filling comes from the black region between the external boundary of the zone 806 (at 802 places by cutting) of grand pixel 800 and piece 804, in this example, zone 806 is 5x5 pixel squares, and piece 804 is 8x8 pixel squares, but it should be noted that in them any one or the two can be difformity or dimension.An execution mode can use Dirichlet boundary conditions 810 at the zone of cutting 806 boundaries, and uses Neumann boundary condition 812 at the boundary of piece 804.Be well known that, can be by iteration on this piece and the suitable convolution of kernel (in the indication of 814 places), and utilize these boundary conditions to realize to the finding the solution of Laplace's equation, thus generate IOB 816 (in this example, being 8x 8 IOB).Fig. 8 B shows the exemplary convolution kernels that can use in execution mode.In this execution mode, convolution is from the border in the zone 806 of cutting, thereby fills up black region by diffusing information from the inside to surface.Usually, about three to four iteration are enough filled up whole 804.
Fig. 9 A and Fig. 9 B show the preliminary treatment to the light field image according to an execution mode.Fig. 9 A show according to an execution mode, to utilize the optical field imaging system (such as, the plenoptic camera 102 of Figure 1B) preliminary treatment of the light field image of catching (such as the example images among Fig. 4 400), wherein in this optical field imaging system, use microlens in the back of main lens.Fig. 9 B show according to an execution mode, utilize the light field image that optical field imaging system (such as, the light field camera 100 of Figure 1A) the catches preliminary treatment of (such as, the example images 300 among Fig. 3).The execution mode of light field pretreatment module can be configured to the light field image of one or both types is handled.
With reference to figure 9A, (it is to use such as the optical field imaging system acquisition of the plenoptic camera 102 of Figure 1B the light field pretreatment module at the light field image to the example images in Fig. 4 400, in this optical field imaging system, use microlens in the main lens back) can carry out key element 902,904 and 906 when carrying out preliminary treatment.These key elements can be performed to detect and to extract independent microlens image from the light field image 900 that utilizes optical field imaging system (such as the plenoptic camera 102 of Figure 1B) to catch.The 2D image that exemplary light field picture 900 is made up of a plurality of microlens images.Each microlens image is the image of the aperture of master's (thing) lens of seeing from different perspectives.In one embodiment, the light field pretreatment module can use little image detecting method to extract independent microlens image (little image) from light field image 900.In little image detecting method, the light field pretreatment module detects first little image (shown in 902), and estimates the cycle (shown in 904) of little image subsequently.In one embodiment, this can be by following realization: the pixel value to vertical direction and horizontal direction is averaged, and obtains the auto-correlation of these mean values subsequently, and finally checks the distance from zero drift to first peak value in auto-correlation.These values have provided vertical range and horizontal range approximate between little image.The light field pretreatment module obtains the coordinate of little image in the upper left corner subsequently, by iteration in image, extracts each little image (shown in 906) subsequently.The size of microlens piece (that is grand pixel) is set to equal half of little cycle of images.Subsequently, can carry out regularization (shown in 908) to little image of each extraction alternatively, and carry out and adjust size and under the situation of needs, carry out interpolation (as the indication of 910 places), so that generate pretreated light field image 912.
In one embodiment, the light field pretreatment module is not that image 900 is carried out iteration and extracts all little images, and subsequently the little image that extracts is carried out as the preliminary treatment shown in 908 and 910, but can be arranged to: extract next little image, the little image that extracts is carried out as the preliminary treatment shown in 908 and 910, and to repeating to extract and to the preliminary treatment of each little image, the little image of all in image 900 be extracted with preliminary treatment till.
Fig. 9 B show according to an execution mode, to the preliminary treatment of (such as, the example images 300 among Fig. 3) of the light field image that utilizes optical field imaging system (such as, the light field camera 100 of Figure 1A) to catch.In these images, in each grand pixel of the light field image of importing 920 of catching, there is not circular little image.Each grand pixel in the light field image 920 of basically, catching is a little image.For the image of these types, can carry out regularization to grand pixel alternatively, shown in 922.The block of pixels of nxm forms from grand pixel, as 924 places indications, for example at as described in Fig. 3 A and Fig. 3 B.Piece is carried out the adjustment size and carry out interpolation (shown in 926) under the situation of needs, so that the block of pixels of the n ' xm ' of the piecemeal of generation and block-based compress technique mechanism compatibility.All processed from all pieces that image 920 extracts, thus pretreated light field image 928 generated.
The pretreated light field image 912 of Fig. 9 A and the pretreated light field image 928 of Fig. 9 B can use block-based compress technique (such as, JPEG compression) to compress subsequently.Because the preliminary treatment that the light field pretreatment module is carried out is adjusted into the compatible size of using with compress technique (and shape) of piecemeal mechanism with the size of the piece in the pretreated light field image, so consequent compressed image can not have the piecemeal artifact, and directly can produce the piecemeal artifact to its situation about compressing is next image not being carried out preliminary treatment described herein.
Example system
Can carry out the various execution modes of light field pretreatment module on one or more computer systems, wherein said computer system can be mutual with various other equipment.Figure 10 shows a this computer system.In the execution mode that illustrates, computer system 1000 comprises one or more processors 1010, and it is connected to system storage 1020 by I/O (I/O) interface 1030.Computer system 1000 also comprises the network interface 1040 that is coupled to I/O interface 1030, and one or more input-output apparatus 1050, such as cursor control device 1060, keyboard 1070, audio frequency apparatus 1090 and display 1080.In some embodiments, can expect: the single instance of the system that can use a computer 1000 is realized execution mode, and in other embodiments, a plurality of nodes of a plurality of this systems or formation computer system 1000 can be arranged to the different piece or the example of management implementation mode.For example, in one embodiment, some key element can realize by the different one or more nodes of those nodes with realizing other key elements of computer system 1000.
In various execution modes, computer system 1000 can be the single processor system that comprises a processor 1010, or comprises the multicomputer system (for example, two, four, number that eight or another are fit to) of some processors 1010.Processor 1010 can be the processor that is fit to arbitrarily that can execute instruction.For example, in each execution mode, processor 1010 can be to realize various instruction set architectures (ISA) such as x86, PowerPC, SPARC or MIPS ISA or the general or flush bonding processor of any one among other ISA that are fit to arbitrarily.In multicomputer system, each in the processor 1010 can realize identical ISA usually, but so uninevitable.
System storage 1020 can be configured to store can be by the program command and/or the data of processor 1010 visits.In each execution mode, can use suitable memory technology, such as the memory of static random-access memory (SRAM), asynchronous dynamical RAM (SDRAM), non-volatile/flash-type memory or any other types, realize system storage 1020.In the execution mode that illustrates, realization is such as top those desirable functional programs instruction and datas at the light field treatment of picture that is used to utilize block-based compress technique to compress, and illustrated respectively as program command 1025 and storage 1035 to be stored in the system storage 1020.In other embodiments, program command and/or data can be received, send or be stored on the dissimilar computer accessible medium or with similar mediums that system storage 1020 or computer system 1000 are separated on.In general, computer accessible can comprise storage medium or storage medium, such as magnetic or light medium, for example is connected to the dish or the CD/DVD-ROM of computer system 1000 by I/O interface 1030.Can transmit with transmission medium or the signal such as electricity, electromagnetism or digital signal by computer accessible program stored instruction and data, can transmit these signals by the communication medium such as network and/or wireless link, such as realizing by network interface 1040.
In one embodiment, I/O interface 1030 can be configured to coprocessor 1010, the I/O flow between any ancillary equipment in system storage 1020 and the equipment (comprising network interface 1040 or other peripheral interfaces, such as input-output apparatus 1050).In some embodiments, I/O interface 1030 can be carried out essential agreement, timing or other transformation of data arbitrarily, so that the data-signal of a kind of assembly (for example, system storage 1020) is converted to the form that is applicable to another kind of assembly (for example, processor 1010).In some embodiments, I/O interface 1030 can comprise to passing through various types of peripheral buses, such as the support of various periphery component interconnections (PCI) bus standard or USB (USB) standard supplementary equipment therefore.In some embodiments, the function of I/O interface 1030 can be divided into two or more independent assemblies, such as for example north bridge and south bridge.In addition, in some embodiments, some or all function of I/O interface 1030 can directly be integrated in the processor 1010 such as the interface to system storage 1020.
In some embodiments, input-output apparatus 1050 can comprise one or more display terminals, keyboard, keypad, touch pad, scanning device, voice or optical identification equipment, or is applicable to any other equipment by one or more computer system 1000 inputs or retrieve data.A plurality of input-output apparatus 1050 can be present in the computer system 1000, maybe can be distributed on each node of computer system 1000.In some embodiments, similarly input-output apparatus can be separated with computer system 1000, and can be by wired or wireless connection, and is such as on network interface 1040, mutual with one or more nodes of computer system 1000.
As shown in figure 10, memory 1020 can comprise the program command 1025 of the execution mode that is configured to realize the light field pretreatment module, and comprise can be by the storage 1035 of the various data of program command 1025 visits.In one embodiment, program command 1025 can comprise the software unit of block-based compression engine shown in the last figure and light field pretreatment module.Storage 1035 can comprise the data that can be used for these execution modes.In other embodiments, can comprise other or different software unit and data.
Those skilled in the art should be understood that computer system 1000 only is illustrative, and is not to be intended to scope with the light field pretreatment module to be restricted to disclosing herein.Particularly, department of computer science's equipment of unifying can comprise the hardware that can carry out the function of pointing out or the combination in any of software, comprises computer, the network equipment, internet, applications, PDA, radio telephone, beep-pager etc.Computer system 1000 can also be connected to unshowned other equipment, or as independently system operation.In addition, in some embodiments, the function that is provided by the assembly that illustrates can be combined in still less the assembly or be distributed on the add-on assemble.Similarly, in some embodiments, certain some function in the execution mode that illustrates can be do not provided, and/or other additional functions can be obtained.
Those skilled in the art should be understood that, though being shown as, various projects are stored in use on memory or the memory device, can between memory and other memory devices, transmit these projects or their part for the purpose of storage management and data integrity.Replacedly, in other embodiments, can be on memory or another equipment certain in the executive software assembly some or all, and communicate by letter with the computer system that illustrates by computer to computer communication.Some or all (for example can also be stored in system component or the data structure certain, as instruction or structural data) on computer accessible medium, or on the portable product that will read with proper driver, their various examples described above.In some embodiments, the instruction that is stored on the computer accessible of separating with computer system 1000 can be transferred to computer system 1000 by transmission medium or signal (such as by communication medium, for example electricity, electromagnetism or the digital signal transmitted of network and/or Radio Link).Various execution modes also are included on the computer accessible and receive, and send or store instruction and/or data according to execution noted earlier.Therefore, can implement the present invention with other computer system configurations.
Conclusion
Various execution modes can also be included on the computer accessible and receive, send or store instruction and/or the data according to execution noted earlier.Generally speaking, computer accessible can comprise: storage medium or storage medium, such as magnetic or optical medium (for example, dish or DVD/CD-ROM); Easily lose or non-volatile media, such as RAM (for example, SDRAM, DDR, RDRAM, SRAM etc.), ROM etc.; And transmission medium or signal, such as electricity, electromagnetism or the digital signal of the communication medium transmission by for example network and/or Radio Link.
Shown in the figure and herein the whole bag of tricks of describing is represented the example of the execution mode of these methods.Can use software, hardware or its combination to realize these methods.The order of method can be changed, and each element can be increased, rearranges, makes up, ignores, modification etc.
Benefit from the disclosure, those skilled in the art clearly can make various modifications and change.The present invention is intended to comprise all such modifications and change, and therefore foregoing description is considered to illustrative rather than limited significance.
Claims (17)
1. method that is used for the light field image is carried out block-based compression comprises:
Obtain the light field image, wherein said light field image comprises a plurality of little images, and the dimension of wherein said little image is different with the employed piecemeal dimension of block-based compress technique;
The little image of from described light field image each generates the piece in the pretreated light field image, and wherein each piece comprises the light field information that comes from corresponding little image, and the dimension of each piece is identical with the piecemeal dimension of described block-based compress technique use;
According to described block-based compress technique described pretreated light field image is compressed, to generate light field image through overcompression; And
Described light field image through overcompression is stored in the storage medium.
2. method according to claim 1, wherein the piece that generates in the pretreated light field image of the little image of each from described light field image comprises, at each circular little image:
The described little image of cutting to be extracting pixel region from described little image, the dimension in wherein said zone is less than described dimension; And
Pixel region to described cutting carries out interpolation, produces the pixel value of the pixel that is used for relevant block with the pixel value of the pixel from the zone of described cutting.
3. method according to claim 2, wherein the pixel region of described cutting is carried out interpolation and comprise:
Find the solution Laplace's equation, generate the pixel value of the pixel in the relevant block with the pixel value of the pixel from the zone of described cutting.
4. method according to claim 1, wherein said dimension are 8 to take advantage of 8 pixels.
5. method according to claim 1 further comprises:
Before the piece in the pretreated light field image of each the little image generation from described light field image, described light field image is carried out regularization according to the calibration chart picture.
6. method according to claim 1, wherein said light field image is caught by plenoptic camera, and each little image approximate is circular.
7. method according to claim 6 further comprises, before the piece in the pretreated light field image of each the little image generation from described light field image:
Locate each the little image in the described light field image; And
From described light field image, extract little image of described location.
8. method according to claim 1, wherein said block-based compress technique are compressed described pretreated light field image according to JPEG (JPEG (joint photographic experts group)) compression standard.
9. method according to claim 1, wherein said light field image through overcompression can use any should being used for that can visit the file that utilizes described block-based compress technique compression to check.
10. equipment that is used for the light field image is carried out block-based compression comprises:
Be used to obtain the device of light field image, wherein said light field image comprises a plurality of little images, and the dimension of wherein said little image is different with the employed piecemeal dimension of block-based compress technique;
Be used for generating the device of the piece the pretreated light field image from each little image of described light field image, wherein each piece comprises the light field information that comes from corresponding little image, and the dimension of each piece is identical with the piecemeal dimension of described block-based compress technique use;
Be used for described pretreated light field image being compressed, to generate device through the light field image of overcompression according to described block-based compress technique; And
Be used for the device that stores storage medium into through the light field image of overcompression with described.
11. equipment according to claim 10, wherein said generation comprises, at each circular little image:
The described little image of cutting to be extracting pixel region from described little image, and the dimension in wherein said zone is less than described dimension; And
Pixel region to described cutting carries out interpolation, produces the pixel value of the pixel that is used for relevant block with the pixel value of the pixel from the zone of described cutting.
12. equipment according to claim 11, wherein said interpolation comprises:
Find the solution Laplace's equation, generate the pixel value of the pixel in the relevant block with the pixel value of the pixel from the zone of described cutting.
13. equipment according to claim 10, wherein said dimension are 8 to take advantage of 8 pixels.
14. equipment according to claim 10 further comprises:
Be used for before little image generates piece the pretreated light field image from each of described light field image the device that described light field image is carried out regularization according to calibration chart picture at described light field image.
15. equipment according to claim 10, wherein said light field image is caught by plenoptic camera, and each little image approximate is circular.
16. equipment according to claim 15 further comprises such device, it was used for before the piece from the pretreated light field image of each little image generation of described light field image:
Locate each the little image in the described light field image; And
Extract little image of described location from described light field image.
17. equipment according to claim 10, wherein said block-based compress technique compress described pretreated light field image according to JPEG (JPEG (joint photographic experts group)) compression standard.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US12/111,735 US8155456B2 (en) | 2008-04-29 | 2008-04-29 | Method and apparatus for block-based compression of light-field images |
| US12/111,735 | 2008-04-29 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101588437A CN101588437A (en) | 2009-11-25 |
| CN101588437B true CN101588437B (en) | 2011-07-13 |
Family
ID=40888250
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009101385392A Active CN101588437B (en) | 2008-04-29 | 2009-04-29 | Method and apparatus for block-based compression of light-field images |
Country Status (3)
| Country | Link |
|---|---|
| US (2) | US8155456B2 (en) |
| EP (1) | EP2114072A1 (en) |
| CN (1) | CN101588437B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8471920B2 (en) | 2009-07-15 | 2013-06-25 | Adobe Systems Incorporated | Focused plenoptic camera employing different apertures or filtering at different microlenses |
| US8559756B2 (en) | 2007-08-06 | 2013-10-15 | Adobe Systems Incorporated | Radiance processing by demultiplexing in the frequency domain |
| US8611693B2 (en) | 2008-05-30 | 2013-12-17 | Adobe Systems Incorporated | Managing artifacts in frequency domain processing of light-field images |
| US8665341B2 (en) | 2010-08-27 | 2014-03-04 | Adobe Systems Incorporated | Methods and apparatus for rendering output images with simulated artistic effects from focused plenoptic camera data |
| US8724000B2 (en) | 2010-08-27 | 2014-05-13 | Adobe Systems Incorporated | Methods and apparatus for super-resolution in integral photography |
| US8749694B2 (en) | 2010-08-27 | 2014-06-10 | Adobe Systems Incorporated | Methods and apparatus for rendering focused plenoptic camera data using super-resolved demosaicing |
| US8803918B2 (en) | 2010-08-27 | 2014-08-12 | Adobe Systems Incorporated | Methods and apparatus for calibrating focused plenoptic camera data |
| US8817015B2 (en) | 2010-03-03 | 2014-08-26 | Adobe Systems Incorporated | Methods, apparatus, and computer-readable storage media for depth-based rendering of focused plenoptic camera data |
| US9030550B2 (en) | 2011-03-25 | 2015-05-12 | Adobe Systems Incorporated | Thin plenoptic cameras using solid immersion lenses |
| US9316840B2 (en) | 2009-01-20 | 2016-04-19 | Adobe Systems Incorporated | Methods and apparatus for reducing plenoptic camera artifacts |
Families Citing this family (139)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8406563B2 (en) * | 2009-07-28 | 2013-03-26 | National Taiwan University | Photometric calibration method and device |
| US20100265385A1 (en) * | 2009-04-18 | 2010-10-21 | Knight Timothy J | Light Field Camera Image, File and Configuration Data, and Methods of Using, Storing and Communicating Same |
| US10298834B2 (en) | 2006-12-01 | 2019-05-21 | Google Llc | Video refocusing |
| US8559705B2 (en) | 2006-12-01 | 2013-10-15 | Lytro, Inc. | Interactive refocusing of electronic images |
| PL2289173T3 (en) * | 2008-05-15 | 2018-03-30 | Koninklijke Philips N.V. | Method, apparatus, and computer program product for compression and decompression of a gene sequencing image |
| US11792538B2 (en) | 2008-05-20 | 2023-10-17 | Adeia Imaging Llc | Capturing and processing of images including occlusions focused on an image sensor by a lens stack array |
| US8866920B2 (en) | 2008-05-20 | 2014-10-21 | Pelican Imaging Corporation | Capturing and processing of images using monolithic camera array with heterogeneous imagers |
| US8902321B2 (en) | 2008-05-20 | 2014-12-02 | Pelican Imaging Corporation | Capturing and processing of images using monolithic camera array with heterogeneous imagers |
| EP2133726B1 (en) * | 2008-06-10 | 2011-06-01 | Thomson Licensing | Multi-image capture system with improved depth image resolution |
| US8279325B2 (en) * | 2008-11-25 | 2012-10-02 | Lytro, Inc. | System and method for acquiring, editing, generating and outputting video data |
| WO2010077625A1 (en) | 2008-12-08 | 2010-07-08 | Refocus Imaging, Inc. | Light field data acquisition devices, and methods of using and manufacturing same |
| US8854379B2 (en) * | 2009-02-25 | 2014-10-07 | Empire Technology Development Llc | Routing across multicore networks using real world or modeled data |
| US8908058B2 (en) * | 2009-04-18 | 2014-12-09 | Lytro, Inc. | Storage and transmission of pictures including multiple frames |
| US8514491B2 (en) | 2009-11-20 | 2013-08-20 | Pelican Imaging Corporation | Capturing and processing of images using monolithic camera array with heterogeneous imagers |
| WO2011066275A2 (en) * | 2009-11-25 | 2011-06-03 | Massachusetts Institute Of Technology | Actively addressable aperture light field camera |
| US8558915B2 (en) * | 2009-12-22 | 2013-10-15 | Samsung Electronics Co., Ltd. | Photographing apparatus and method |
| US8749620B1 (en) * | 2010-02-20 | 2014-06-10 | Lytro, Inc. | 3D light field cameras, images and files, and methods of using, operating, processing and viewing same |
| SG10201503516VA (en) | 2010-05-12 | 2015-06-29 | Pelican Imaging Corp | Architectures for imager arrays and array cameras |
| US8675993B2 (en) | 2010-09-03 | 2014-03-18 | Adobe Systems Incorporated | Methods and apparatus for patch-based shape from shading |
| US8878950B2 (en) | 2010-12-14 | 2014-11-04 | Pelican Imaging Corporation | Systems and methods for synthesizing high resolution images using super-resolution processes |
| US8768102B1 (en) | 2011-02-09 | 2014-07-01 | Lytro, Inc. | Downsampling light field images |
| US8625910B2 (en) * | 2011-02-25 | 2014-01-07 | Adobe Systems Incorporated | Compression of image data |
| WO2012155119A1 (en) | 2011-05-11 | 2012-11-15 | Pelican Imaging Corporation | Systems and methods for transmitting and receiving array camera image data |
| US8605199B2 (en) | 2011-06-28 | 2013-12-10 | Canon Kabushiki Kaisha | Adjustment of imaging properties for an imaging assembly having light-field optics |
| EP2726930A4 (en) | 2011-06-28 | 2015-03-04 | Pelican Imaging Corp | Optical arrangements for use with an array camera |
| US20130265459A1 (en) | 2011-06-28 | 2013-10-10 | Pelican Imaging Corporation | Optical arrangements for use with an array camera |
| US9184199B2 (en) | 2011-08-01 | 2015-11-10 | Lytro, Inc. | Optical assembly including plenoptic microlens array |
| WO2013043761A1 (en) | 2011-09-19 | 2013-03-28 | Pelican Imaging Corporation | Determining depth from multiple views of a scene that include aliasing using hypothesized fusion |
| WO2013049699A1 (en) | 2011-09-28 | 2013-04-04 | Pelican Imaging Corporation | Systems and methods for encoding and decoding light field image files |
| US9165401B1 (en) * | 2011-10-24 | 2015-10-20 | Disney Enterprises, Inc. | Multi-perspective stereoscopy from light fields |
| US9113043B1 (en) | 2011-10-24 | 2015-08-18 | Disney Enterprises, Inc. | Multi-perspective stereoscopy from light fields |
| EP2817955B1 (en) | 2012-02-21 | 2018-04-11 | FotoNation Cayman Limited | Systems and methods for the manipulation of captured light field image data |
| US9420276B2 (en) | 2012-02-28 | 2016-08-16 | Lytro, Inc. | Calibration of light-field camera geometry via robust fitting |
| US8831377B2 (en) | 2012-02-28 | 2014-09-09 | Lytro, Inc. | Compensating for variation in microlens position during light-field image processing |
| US8948545B2 (en) | 2012-02-28 | 2015-02-03 | Lytro, Inc. | Compensating for sensor saturation and microlens modulation during light-field image processing |
| US8811769B1 (en) | 2012-02-28 | 2014-08-19 | Lytro, Inc. | Extended depth of field and variable center of perspective in light-field processing |
| US8995785B2 (en) * | 2012-02-28 | 2015-03-31 | Lytro, Inc. | Light-field processing and analysis, camera control, and user interfaces and interaction on light-field capture devices |
| US9160900B2 (en) | 2012-02-29 | 2015-10-13 | Canon Kabushiki Kaisha | Systems and methods for compressive light sensing using multiple spatial light modulators |
| US9210392B2 (en) | 2012-05-01 | 2015-12-08 | Pelican Imaging Coporation | Camera modules patterned with pi filter groups |
| WO2013169671A1 (en) | 2012-05-09 | 2013-11-14 | Lytro, Inc. | Optimization of optical systems for improved light field capture and manipulation |
| US9858649B2 (en) | 2015-09-30 | 2018-01-02 | Lytro, Inc. | Depth-based image blurring |
| US10129524B2 (en) | 2012-06-26 | 2018-11-13 | Google Llc | Depth-assigned content for depth-enhanced virtual reality images |
| US9607424B2 (en) | 2012-06-26 | 2017-03-28 | Lytro, Inc. | Depth-assigned content for depth-enhanced pictures |
| US9100635B2 (en) | 2012-06-28 | 2015-08-04 | Pelican Imaging Corporation | Systems and methods for detecting defective camera arrays and optic arrays |
| GB2503656B (en) * | 2012-06-28 | 2014-10-15 | Canon Kk | Method and apparatus for compressing or decompressing light field images |
| US20140002674A1 (en) | 2012-06-30 | 2014-01-02 | Pelican Imaging Corporation | Systems and Methods for Manufacturing Camera Modules Using Active Alignment of Lens Stack Arrays and Sensors |
| JP6045250B2 (en) | 2012-08-10 | 2016-12-14 | オリンパス株式会社 | Solid-state imaging device and imaging device |
| CN107346061B (en) | 2012-08-21 | 2020-04-24 | 快图有限公司 | System and method for parallax detection and correction in images captured using an array camera |
| EP2888698A4 (en) | 2012-08-23 | 2016-06-29 | Pelican Imaging Corp | Feature based high resolution motion estimation from low resolution images captured using an array source |
| WO2014043641A1 (en) | 2012-09-14 | 2014-03-20 | Pelican Imaging Corporation | Systems and methods for correcting user identified artifacts in light field images |
| CN104685860A (en) | 2012-09-28 | 2015-06-03 | 派力肯影像公司 | Generating images from light fields utilizing virtual viewpoints |
| US10044985B1 (en) * | 2012-10-19 | 2018-08-07 | Amazon Technologies, Inc. | Video monitoring using plenoptic cameras and mirrors |
| US8997021B2 (en) | 2012-11-06 | 2015-03-31 | Lytro, Inc. | Parallax and/or three-dimensional effects for thumbnail image displays |
| US9143711B2 (en) | 2012-11-13 | 2015-09-22 | Pelican Imaging Corporation | Systems and methods for array camera focal plane control |
| US9001226B1 (en) | 2012-12-04 | 2015-04-07 | Lytro, Inc. | Capturing and relighting images using multiple devices |
| US9462164B2 (en) | 2013-02-21 | 2016-10-04 | Pelican Imaging Corporation | Systems and methods for generating compressed light field representation data using captured light fields, array geometry, and parallax information |
| US9374512B2 (en) | 2013-02-24 | 2016-06-21 | Pelican Imaging Corporation | Thin form factor computational array cameras and modular array cameras |
| US9774789B2 (en) | 2013-03-08 | 2017-09-26 | Fotonation Cayman Limited | Systems and methods for high dynamic range imaging using array cameras |
| US8866912B2 (en) | 2013-03-10 | 2014-10-21 | Pelican Imaging Corporation | System and methods for calibration of an array camera using a single captured image |
| US9521416B1 (en) | 2013-03-11 | 2016-12-13 | Kip Peli P1 Lp | Systems and methods for image data compression |
| US9124831B2 (en) | 2013-03-13 | 2015-09-01 | Pelican Imaging Corporation | System and methods for calibration of an array camera |
| US9106784B2 (en) | 2013-03-13 | 2015-08-11 | Pelican Imaging Corporation | Systems and methods for controlling aliasing in images captured by an array camera for use in super-resolution processing |
| US9519972B2 (en) | 2013-03-13 | 2016-12-13 | Kip Peli P1 Lp | Systems and methods for synthesizing images from image data captured by an array camera using restricted depth of field depth maps in which depth estimation precision varies |
| US9888194B2 (en) | 2013-03-13 | 2018-02-06 | Fotonation Cayman Limited | Array camera architecture implementing quantum film image sensors |
| WO2014159779A1 (en) | 2013-03-14 | 2014-10-02 | Pelican Imaging Corporation | Systems and methods for reducing motion blur in images or video in ultra low light with array cameras |
| US9100586B2 (en) | 2013-03-14 | 2015-08-04 | Pelican Imaging Corporation | Systems and methods for photometric normalization in array cameras |
| US9633442B2 (en) | 2013-03-15 | 2017-04-25 | Fotonation Cayman Limited | Array cameras including an array camera module augmented with a separate camera |
| US10122993B2 (en) | 2013-03-15 | 2018-11-06 | Fotonation Limited | Autofocus system for a conventional camera that uses depth information from an array camera |
| US9497429B2 (en) | 2013-03-15 | 2016-11-15 | Pelican Imaging Corporation | Extended color processing on pelican array cameras |
| WO2014150856A1 (en) | 2013-03-15 | 2014-09-25 | Pelican Imaging Corporation | Array camera implementing quantum dot color filters |
| US9445003B1 (en) | 2013-03-15 | 2016-09-13 | Pelican Imaging Corporation | Systems and methods for synthesizing high resolution images using image deconvolution based on motion and depth information |
| JP2016524125A (en) | 2013-03-15 | 2016-08-12 | ペリカン イメージング コーポレイション | System and method for stereoscopic imaging using a camera array |
| US10334151B2 (en) | 2013-04-22 | 2019-06-25 | Google Llc | Phase detection autofocus using subaperture images |
| US9898856B2 (en) | 2013-09-27 | 2018-02-20 | Fotonation Cayman Limited | Systems and methods for depth-assisted perspective distortion correction |
| US9185276B2 (en) | 2013-11-07 | 2015-11-10 | Pelican Imaging Corporation | Methods of manufacturing array camera modules incorporating independently aligned lens stacks |
| US10119808B2 (en) | 2013-11-18 | 2018-11-06 | Fotonation Limited | Systems and methods for estimating depth from projected texture using camera arrays |
| EP3075140B1 (en) | 2013-11-26 | 2018-06-13 | FotoNation Cayman Limited | Array camera configurations incorporating multiple constituent array cameras |
| US9538075B2 (en) | 2013-12-30 | 2017-01-03 | Indiana University Research And Technology Corporation | Frequency domain processing techniques for plenoptic images |
| US10244223B2 (en) | 2014-01-10 | 2019-03-26 | Ostendo Technologies, Inc. | Methods for full parallax compressed light field 3D imaging systems |
| US10089740B2 (en) | 2014-03-07 | 2018-10-02 | Fotonation Limited | System and methods for depth regularization and semiautomatic interactive matting using RGB-D images |
| US9247117B2 (en) | 2014-04-07 | 2016-01-26 | Pelican Imaging Corporation | Systems and methods for correcting for warpage of a sensor array in an array camera module by introducing warpage into a focal plane of a lens stack array |
| US9414087B2 (en) | 2014-04-24 | 2016-08-09 | Lytro, Inc. | Compression of light field images |
| US9712820B2 (en) | 2014-04-24 | 2017-07-18 | Lytro, Inc. | Predictive light field compression |
| US9521319B2 (en) | 2014-06-18 | 2016-12-13 | Pelican Imaging Corporation | Array cameras and array camera modules including spectral filters disposed outside of a constituent image sensor |
| GB2544946B (en) | 2014-08-31 | 2021-03-10 | Berestka John | Systems and methods for analyzing the eye |
| US10250871B2 (en) | 2014-09-29 | 2019-04-02 | Fotonation Limited | Systems and methods for dynamic calibration of array cameras |
| CN104469372B (en) * | 2014-11-06 | 2018-09-07 | 中国科学院计算技术研究所 | Method and system for the light field image for compressing microlens array acquisition |
| GB2535475B (en) | 2015-02-16 | 2019-09-18 | Canon Kk | Optimized plenoptic image encoding |
| US10412373B2 (en) | 2015-04-15 | 2019-09-10 | Google Llc | Image capture for virtual reality displays |
| US10341632B2 (en) | 2015-04-15 | 2019-07-02 | Google Llc. | Spatial random access enabled video system with a three-dimensional viewing volume |
| US11328446B2 (en) | 2015-04-15 | 2022-05-10 | Google Llc | Combining light-field data with active depth data for depth map generation |
| US10540818B2 (en) | 2015-04-15 | 2020-01-21 | Google Llc | Stereo image generation and interactive playback |
| US10567464B2 (en) | 2015-04-15 | 2020-02-18 | Google Llc | Video compression with adaptive view-dependent lighting removal |
| US10565734B2 (en) | 2015-04-15 | 2020-02-18 | Google Llc | Video capture, processing, calibration, computational fiber artifact removal, and light-field pipeline |
| US10444931B2 (en) | 2017-05-09 | 2019-10-15 | Google Llc | Vantage generation and interactive playback |
| US10469873B2 (en) | 2015-04-15 | 2019-11-05 | Google Llc | Encoding and decoding virtual reality video |
| US10275898B1 (en) | 2015-04-15 | 2019-04-30 | Google Llc | Wedge-based light-field video capture |
| US10546424B2 (en) | 2015-04-15 | 2020-01-28 | Google Llc | Layered content delivery for virtual and augmented reality experiences |
| US10440407B2 (en) | 2017-05-09 | 2019-10-08 | Google Llc | Adaptive control for immersive experience delivery |
| US10419737B2 (en) | 2015-04-15 | 2019-09-17 | Google Llc | Data structures and delivery methods for expediting virtual reality playback |
| US9942474B2 (en) | 2015-04-17 | 2018-04-10 | Fotonation Cayman Limited | Systems and methods for performing high speed video capture and depth estimation using array cameras |
| EP3286737A1 (en) | 2015-04-23 | 2018-02-28 | Ostendo Technologies, Inc. | Methods for full parallax compressed light field synthesis utilizing depth information |
| JP6866299B2 (en) * | 2015-04-23 | 2021-04-28 | オステンド・テクノロジーズ・インコーポレーテッド | Methods and equipment for omnidirectional parallax light field display systems |
| EP3104595A1 (en) | 2015-06-08 | 2016-12-14 | Thomson Licensing | Light field imaging device |
| US9979909B2 (en) | 2015-07-24 | 2018-05-22 | Lytro, Inc. | Automatic lens flare detection and correction for light-field images |
| JP6450288B2 (en) * | 2015-09-18 | 2019-01-09 | 日本電信電話株式会社 | Image encoding device, image decoding device, image encoding method, and image decoding method |
| US10448030B2 (en) | 2015-11-16 | 2019-10-15 | Ostendo Technologies, Inc. | Content adaptive light field compression |
| US10453431B2 (en) | 2016-04-28 | 2019-10-22 | Ostendo Technologies, Inc. | Integrated near-far light field display systems |
| US10089788B2 (en) * | 2016-05-25 | 2018-10-02 | Google Llc | Light-field viewpoint and pixel culling for a head mounted display device |
| US10275892B2 (en) | 2016-06-09 | 2019-04-30 | Google Llc | Multi-view scene segmentation and propagation |
| CN106254719B (en) * | 2016-07-25 | 2018-11-30 | 清华大学深圳研究生院 | A kind of light field image compression method based on linear transformation and image interpolation |
| US10679361B2 (en) | 2016-12-05 | 2020-06-09 | Google Llc | Multi-view rotoscope contour propagation |
| US10373384B2 (en) | 2016-12-12 | 2019-08-06 | Google Llc | Lightfield compression using disparity predicted replacement |
| US10460505B2 (en) * | 2016-12-30 | 2019-10-29 | Google Llc | Systems and methods for lightfield reconstruction utilizing contribution regions |
| US10375398B2 (en) | 2017-03-24 | 2019-08-06 | Google Llc | Lightfield compression for per-pixel, on-demand access by a graphics processing unit |
| CN106961605B (en) * | 2017-03-28 | 2019-06-28 | 清华大学深圳研究生院 | One kind being based on the matched light field image compression method of macro pixel boundary |
| US10594945B2 (en) | 2017-04-03 | 2020-03-17 | Google Llc | Generating dolly zoom effect using light field image data |
| CN107135393B (en) * | 2017-05-03 | 2020-11-06 | 北京航空航天大学 | Compression method of light field image |
| US10474227B2 (en) | 2017-05-09 | 2019-11-12 | Google Llc | Generation of virtual reality with 6 degrees of freedom from limited viewer data |
| US10354399B2 (en) | 2017-05-25 | 2019-07-16 | Google Llc | Multi-view back-projection to a light-field |
| US11051039B2 (en) | 2017-06-02 | 2021-06-29 | Ostendo Technologies, Inc. | Methods for full parallax light field compression |
| US20180350038A1 (en) | 2017-06-02 | 2018-12-06 | Ostendo Technologies, Inc. | Methods and Systems for Light Field Compression With Residuals |
| EP3422722A1 (en) | 2017-06-30 | 2019-01-02 | Thomson Licensing | Method for encoding a matrix of image views obtained from data acquired by a plenoptic camera |
| US10482618B2 (en) | 2017-08-21 | 2019-11-19 | Fotonation Limited | Systems and methods for hybrid depth regularization |
| US10545215B2 (en) | 2017-09-13 | 2020-01-28 | Google Llc | 4D camera tracking and optical stabilization |
| US10776995B2 (en) * | 2017-10-17 | 2020-09-15 | Nvidia Corporation | Light fields as better backgrounds in rendering |
| US10965862B2 (en) | 2018-01-18 | 2021-03-30 | Google Llc | Multi-camera navigation interface |
| US10931956B2 (en) | 2018-04-12 | 2021-02-23 | Ostendo Technologies, Inc. | Methods for MR-DIBR disparity map merging and disparity threshold determination |
| US11172222B2 (en) | 2018-06-26 | 2021-11-09 | Ostendo Technologies, Inc. | Random access in encoded full parallax light field images |
| US11182872B2 (en) * | 2018-11-02 | 2021-11-23 | Electronics And Telecommunications Research Institute | Plenoptic data storage system and operating method thereof |
| KR102646521B1 (en) | 2019-09-17 | 2024-03-21 | 인트린식 이노베이션 엘엘씨 | Surface modeling system and method using polarization cue |
| CA3157194C (en) | 2019-10-07 | 2023-08-29 | Boston Polarimetrics, Inc. | Systems and methods for augmentation of sensor systems and imaging systems with polarization |
| WO2021108002A1 (en) | 2019-11-30 | 2021-06-03 | Boston Polarimetrics, Inc. | Systems and methods for transparent object segmentation using polarization cues |
| KR20220132620A (en) | 2020-01-29 | 2022-09-30 | 인트린식 이노베이션 엘엘씨 | Systems and methods for characterizing object pose detection and measurement systems |
| KR20220133973A (en) | 2020-01-30 | 2022-10-05 | 인트린식 이노베이션 엘엘씨 | Systems and methods for synthesizing data to train statistical models for different imaging modalities, including polarized images |
| US11953700B2 (en) | 2020-05-27 | 2024-04-09 | Intrinsic Innovation Llc | Multi-aperture polarization optical systems using beam splitters |
| US11954886B2 (en) | 2021-04-15 | 2024-04-09 | Intrinsic Innovation Llc | Systems and methods for six-degree of freedom pose estimation of deformable objects |
| US11290658B1 (en) | 2021-04-15 | 2022-03-29 | Boston Polarimetrics, Inc. | Systems and methods for camera exposure control |
| US11689813B2 (en) | 2021-07-01 | 2023-06-27 | Intrinsic Innovation Llc | Systems and methods for high dynamic range imaging using crossed polarizers |
Family Cites Families (64)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US725567A (en) | 1902-09-25 | 1903-04-14 | Frederic E Ives | Parallax stereogram and process of making same. |
| US2039648A (en) | 1933-05-06 | 1936-05-05 | Perser Corp | Camera for making parallax panoramagrams |
| US3985419A (en) | 1970-10-05 | 1976-10-12 | Canon Kabushiki Kaisha | Method of making a synthetic focused image hologram |
| JPS5946376B2 (en) | 1977-03-11 | 1984-11-12 | 富士写真フイルム株式会社 | stereoscopic photography device |
| US4193093A (en) | 1978-08-03 | 1980-03-11 | The United States Of America As Represented By The Secretary Of The Navy | CCD camera interface circuit |
| US4230942A (en) | 1979-03-26 | 1980-10-28 | Honeywell Inc. | Crossed cylindrical lens |
| US4580219A (en) | 1983-05-02 | 1986-04-01 | General Electric Company | Method for reducing image artifacts due to projection measurement inconsistencies |
| US4732453A (en) | 1984-12-10 | 1988-03-22 | Integrated Images, Inc. | Integral photography apparatus and method of forming same |
| US4849782A (en) | 1985-06-28 | 1989-07-18 | Canon Kabushiki Kaisha | Focus detecting device |
| US5076687A (en) | 1990-08-28 | 1991-12-31 | Massachusetts Institute Of Technology | Optical ranging apparatus |
| US5361127A (en) | 1992-08-07 | 1994-11-01 | Hughes Aircraft Company | Multi-image single sensor depth recovery system |
| US5400093A (en) | 1992-12-28 | 1995-03-21 | U.S. Philips Corporation | Image projection system with autofocusing |
| KR950704670A (en) | 1993-09-30 | 1995-11-20 | 가따다 데쯔야 | Confocal Optics |
| WO1997046913A1 (en) | 1996-06-03 | 1997-12-11 | Mims, Herman, D. | Method and apparatus for three-dimensional photography |
| JPH08233658A (en) | 1995-02-24 | 1996-09-13 | Olympus Optical Co Ltd | Spectroscope and spectral image recorder |
| FR2748579B1 (en) | 1996-05-09 | 1998-08-21 | Allio Pierre | AUTOSTEREOSCOPIC IMAGE FORMING DEVICE AND SYSTEM COMPRISING SAME |
| US6137535A (en) | 1996-11-04 | 2000-10-24 | Eastman Kodak Company | Compact digital camera with segmented fields of view |
| GB9702006D0 (en) | 1997-01-31 | 1997-03-19 | Univ Montfort | Lens arrangements |
| US6097394A (en) * | 1997-04-28 | 2000-08-01 | Board Of Trustees, Leland Stanford, Jr. University | Method and system for light field rendering |
| US7002749B2 (en) | 1997-07-08 | 2006-02-21 | Kremen Stanley H | Modular integral magnifier |
| US7532372B2 (en) | 1997-07-08 | 2009-05-12 | Kremen Stanley H | Method for creating a holographic screen that reconstructs uniformly magnified three-dimensional images from projected integral photographs |
| US20010012149A1 (en) | 1997-10-30 | 2001-08-09 | Shawn-Yu Lin | Optical elements comprising photonic crystals and applications thereof |
| US6351269B1 (en) | 1998-04-17 | 2002-02-26 | Adobe Systems Incorporated | Multiple image morphing |
| JP3869116B2 (en) | 1998-04-27 | 2007-01-17 | 日本放送協会 | Stereoscopic image device |
| US6268846B1 (en) | 1998-06-22 | 2001-07-31 | Adobe Systems Incorporated | 3D graphics based on images and morphing |
| JP3790371B2 (en) | 1998-10-09 | 2006-06-28 | 日本放送協会 | Stereoscopic image device |
| US6339506B1 (en) | 1998-11-06 | 2002-01-15 | Oni Systems Corp. | Microlens array with spatially varying optical property |
| US6838650B1 (en) | 1999-11-16 | 2005-01-04 | Agilent Technologies, Inc. | Confocal imaging |
| US6476805B1 (en) | 1999-12-23 | 2002-11-05 | Microsoft Corporation | Techniques for spatial displacement estimation and multi-resolution operations on light fields |
| US6738533B1 (en) | 2000-03-29 | 2004-05-18 | Microsoft Corporation | Minimum sampling rate and minimum sampling curve for image-based rendering |
| US7061532B2 (en) | 2001-03-27 | 2006-06-13 | Hewlett-Packard Development Company, L.P. | Single sensor chip digital stereo camera |
| US7260323B2 (en) | 2002-06-12 | 2007-08-21 | Eastman Kodak Company | Imaging using silver halide films with micro-lens capture, scanning and digital reconstruction |
| CN1692296A (en) | 2002-09-30 | 2005-11-02 | 独立行政法人科学技术振兴机构 | Cofocal microscope, fluorescence measuring method and polarized light measuring metod using cofocal microscope |
| US20040114807A1 (en) | 2002-12-13 | 2004-06-17 | Dan Lelescu | Statistical representation and coding of light field data |
| JP2004200975A (en) | 2002-12-18 | 2004-07-15 | Canon Inc | Imaging apparatus |
| US7057806B2 (en) | 2003-05-09 | 2006-06-06 | 3M Innovative Properties Company | Scanning laser microscope with wavefront sensor |
| US7702878B2 (en) | 2004-03-19 | 2010-04-20 | Broadcom Corporation | Method and system for scalable video data width |
| US7054067B2 (en) | 2003-05-27 | 2006-05-30 | Nippon Hoso Kyokai | Three-dimensional image optical system |
| DE10327733C5 (en) | 2003-06-18 | 2012-04-19 | Limo Patentverwaltung Gmbh & Co. Kg | Device for shaping a light beam |
| US7236304B2 (en) | 2003-11-04 | 2007-06-26 | Matsushita Electric Industrial Co., Ltd. | Imaging Device |
| JP4508619B2 (en) | 2003-12-03 | 2010-07-21 | キヤノン株式会社 | Method for manufacturing solid-state imaging device |
| WO2007092545A2 (en) | 2006-02-07 | 2007-08-16 | The Board Of Trustees Of The Leland Stanford Junior University | Variable imaging arrangements and methods therefor |
| EP2398224B1 (en) | 2004-10-01 | 2016-01-13 | The Board of Trustees of The Leland Stanford Junior University | Imaging arrangements and methods therefor |
| US7653261B2 (en) | 2004-11-12 | 2010-01-26 | Microsoft Corporation | Image tapestry |
| JP5001142B2 (en) | 2005-02-25 | 2012-08-15 | 株式会社日立メディコ | X-ray CT system |
| CN101147392B (en) | 2005-03-24 | 2012-05-09 | 松下电器产业株式会社 | Imaging device and lens array used therein |
| US7905287B2 (en) * | 2005-04-19 | 2011-03-15 | Halliburton Energy Services Inc. | Methods of using a polymeric precipitate to reduce the loss of fluid to a subterranean formation |
| JP4826152B2 (en) | 2005-06-23 | 2011-11-30 | 株式会社ニコン | Image composition method and imaging apparatus |
| US8155418B2 (en) | 2005-09-13 | 2012-04-10 | Koninklijke Philips Electronics N.V. | Automatic generation of optimal views for computed tomography thoracic diagnosis |
| WO2007036055A1 (en) * | 2005-09-30 | 2007-04-05 | Simon Fraser University | Methods and apparatus for detecting defects in imaging arrays by image analysis |
| US7723662B2 (en) | 2005-10-07 | 2010-05-25 | The Board Of Trustees Of The Leland Stanford Junior University | Microscopy arrangements and approaches |
| US7620309B2 (en) | 2006-04-04 | 2009-11-17 | Adobe Systems, Incorporated | Plenoptic camera |
| US7676073B2 (en) | 2006-08-29 | 2010-03-09 | Siemens Medical Solutions Usa, Inc. | System and method for reducing circular artifacts in tomographic imaging |
| US8103111B2 (en) | 2006-12-26 | 2012-01-24 | Olympus Imaging Corp. | Coding method, electronic camera, recording medium storing coded program, and decoding method |
| US7860333B2 (en) | 2007-01-09 | 2010-12-28 | University Of Utah Research Foundation | Systems and methods for deblurring data corrupted by shift variant blurring |
| JP4264569B2 (en) | 2007-01-09 | 2009-05-20 | ソニー株式会社 | Imaging device |
| US7792423B2 (en) | 2007-02-06 | 2010-09-07 | Mitsubishi Electric Research Laboratories, Inc. | 4D light field cameras |
| JP4969474B2 (en) | 2007-02-09 | 2012-07-04 | オリンパスイメージング株式会社 | Decoding method, decoding device, and decoding program |
| WO2009020977A1 (en) | 2007-08-06 | 2009-02-12 | Adobe Systems Incorporated | Method and apparatus for radiance capture by multiplexing in the frequency domain |
| KR20090022596A (en) | 2007-08-31 | 2009-03-04 | 삼성전기주식회사 | Diffraction type optical modulator and display apparatus including this |
| US8189065B2 (en) | 2008-01-23 | 2012-05-29 | Adobe Systems Incorporated | Methods and apparatus for full-resolution light-field capture and rendering |
| US7962033B2 (en) | 2008-01-23 | 2011-06-14 | Adobe Systems Incorporated | Methods and apparatus for full-resolution light-field capture and rendering |
| KR101441586B1 (en) | 2008-10-06 | 2014-09-23 | 삼성전자 주식회사 | Apparatus and method for capturing image |
| WO2010077625A1 (en) | 2008-12-08 | 2010-07-08 | Refocus Imaging, Inc. | Light field data acquisition devices, and methods of using and manufacturing same |
-
2008
- 2008-04-29 US US12/111,735 patent/US8155456B2/en active Active
-
2009
- 2009-04-29 EP EP09159086A patent/EP2114072A1/en not_active Withdrawn
- 2009-04-29 CN CN2009101385392A patent/CN101588437B/en active Active
-
2012
- 2012-03-23 US US13/429,226 patent/US8401316B2/en active Active
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8559756B2 (en) | 2007-08-06 | 2013-10-15 | Adobe Systems Incorporated | Radiance processing by demultiplexing in the frequency domain |
| US8611693B2 (en) | 2008-05-30 | 2013-12-17 | Adobe Systems Incorporated | Managing artifacts in frequency domain processing of light-field images |
| US9316840B2 (en) | 2009-01-20 | 2016-04-19 | Adobe Systems Incorporated | Methods and apparatus for reducing plenoptic camera artifacts |
| US8471920B2 (en) | 2009-07-15 | 2013-06-25 | Adobe Systems Incorporated | Focused plenoptic camera employing different apertures or filtering at different microlenses |
| US8817015B2 (en) | 2010-03-03 | 2014-08-26 | Adobe Systems Incorporated | Methods, apparatus, and computer-readable storage media for depth-based rendering of focused plenoptic camera data |
| US8860833B2 (en) | 2010-03-03 | 2014-10-14 | Adobe Systems Incorporated | Blended rendering of focused plenoptic camera data |
| US8665341B2 (en) | 2010-08-27 | 2014-03-04 | Adobe Systems Incorporated | Methods and apparatus for rendering output images with simulated artistic effects from focused plenoptic camera data |
| US8724000B2 (en) | 2010-08-27 | 2014-05-13 | Adobe Systems Incorporated | Methods and apparatus for super-resolution in integral photography |
| US8749694B2 (en) | 2010-08-27 | 2014-06-10 | Adobe Systems Incorporated | Methods and apparatus for rendering focused plenoptic camera data using super-resolved demosaicing |
| US8803918B2 (en) | 2010-08-27 | 2014-08-12 | Adobe Systems Incorporated | Methods and apparatus for calibrating focused plenoptic camera data |
| US9030550B2 (en) | 2011-03-25 | 2015-05-12 | Adobe Systems Incorporated | Thin plenoptic cameras using solid immersion lenses |
| US9197798B2 (en) | 2011-03-25 | 2015-11-24 | Adobe Systems Incorporated | Thin plenoptic cameras using microspheres |
Also Published As
| Publication number | Publication date |
|---|---|
| US20090268970A1 (en) | 2009-10-29 |
| US8155456B2 (en) | 2012-04-10 |
| US8401316B2 (en) | 2013-03-19 |
| US20120183232A1 (en) | 2012-07-19 |
| CN101588437A (en) | 2009-11-25 |
| EP2114072A1 (en) | 2009-11-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101588437B (en) | Method and apparatus for block-based compression of light-field images | |
| US11526712B2 (en) | Training method and apparatus for image fusion processing model, device, and storage medium | |
| Adel et al. | Image stitching based on feature extraction techniques: a survey | |
| Georgiev et al. | Reducing plenoptic camera artifacts | |
| US8290358B1 (en) | Methods and apparatus for light-field imaging | |
| CN102595171B (en) | Imaging method and imaging system of dynamic optical fields of multichannel space-time coding apertures | |
| CN101689292B (en) | Banana codec | |
| US20180027178A1 (en) | Mapping of spherical image data into rectangular faces for transport and decoding across networks | |
| TWI615027B (en) | High dynamic range image generation method, imaging device, terminal device and imaging method | |
| CN112750085A (en) | Image restoration method and image restoration apparatus | |
| EP3247107B1 (en) | Method and device for obtaining a hdr image by graph signal processing | |
| US20230106939A1 (en) | Light field image processing method, light field image encoder and decoder, and storage medium | |
| CN103888641A (en) | Image Processing Apparatus And Image Refocusing Method | |
| KR102061152B1 (en) | System for providing ar service and method for generating 360 angle rotatable image file thereof | |
| CN108875751A (en) | Image processing method and device, the training method of neural network, storage medium | |
| CN103500471A (en) | Method for realizing high-resolution augmented reality system | |
| Zhou et al. | Unmodnet: Learning to unwrap a modulo image for high dynamic range imaging | |
| Hong et al. | Reflection removal with nir and rgb image feature fusion | |
| EP3657786A1 (en) | Light field reconstruction | |
| CN111489323A (en) | Double-light-field image fusion method, device and equipment and readable storage medium | |
| JP2014147062A (en) | Image pickup device, and control method and program therefor | |
| Lv et al. | Panoramic light field video acquisition | |
| Yu et al. | Dynamic depth of field on live video streams: A stereo solution | |
| JP3981454B2 (en) | Non-uniform resolution image data generation apparatus and method, and image processing apparatus using non-uniform resolution image data | |
| CN111986309A (en) | System and method for generating special film Pre-vis based on three-dimensional scanning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |