WO2006130734A3 - Method and system to increase x-y resolution in a depth (z) camera using red, blue, green (rgb) sensing - Google Patents

Method and system to increase x-y resolution in a depth (z) camera using red, blue, green (rgb) sensing Download PDF

Info

Publication number
WO2006130734A3
WO2006130734A3 PCT/US2006/021193 US2006021193W WO2006130734A3 WO 2006130734 A3 WO2006130734 A3 WO 2006130734A3 US 2006021193 W US2006021193 W US 2006021193W WO 2006130734 A3 WO2006130734 A3 WO 2006130734A3
Authority
WO
WIPO (PCT)
Prior art keywords
sensors
data
resolution
depth
many
Prior art date
Application number
PCT/US2006/021193
Other languages
French (fr)
Other versions
WO2006130734A2 (en
Inventor
Christopher J Rossbach
Abbas Rafii
Peigian Zhao
Original Assignee
Canesta Inc
Christopher J Rossbach
Abbas Rafii
Peigian Zhao
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canesta Inc, Christopher J Rossbach, Abbas Rafii, Peigian Zhao filed Critical Canesta Inc
Publication of WO2006130734A2 publication Critical patent/WO2006130734A2/en
Publication of WO2006130734A3 publication Critical patent/WO2006130734A3/en

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/483—Details of pulse systems
    • G01S7/486—Receivers
    • G01S7/4861—Circuits for detection, sampling, integration or read-out
    • G01S7/4863—Detector arrays, e.g. charge-transfer gates
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/128—Adjusting depth or disparity
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/13—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths with multiple sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/843—Demosaicing, e.g. interpolating colour pixel values
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/135—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on four or more different wavelength filter elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/30—Transforming light or analogous information into electric information
    • H04N5/33—Transforming infrared radiation

Abstract

An imaging system substantially simultaneously acquires z-depth and brightness data from first sensors, and acquires higher resolution RGB data from second sensors, and fuses data from the first and second sensors to model an RGBZ image whose resolution can be as high as resolution of the second sensors. Time correlation of captured data from first and second sensors is associated with captured image data, which permits arbitrary mapping between the two data sources, ranging from 1 :many to many:1. Preferably pixels from each set of sensors that image the same target point are mapped. Many z-depth sensor settings may be used to create a static environmental model. Non-correlative and correlative filtering is carried out, and up-sampling to increase z-resolution occurs, from which a three- dimensional model is constructed using registration and calibration data.
PCT/US2006/021193 2005-06-01 2006-06-01 Method and system to increase x-y resolution in a depth (z) camera using red, blue, green (rgb) sensing WO2006130734A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US68647005P 2005-06-01 2005-06-01
US60/686,470 2005-06-01

Publications (2)

Publication Number Publication Date
WO2006130734A2 WO2006130734A2 (en) 2006-12-07
WO2006130734A3 true WO2006130734A3 (en) 2007-12-13

Family

ID=37482304

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2006/021193 WO2006130734A2 (en) 2005-06-01 2006-06-01 Method and system to increase x-y resolution in a depth (z) camera using red, blue, green (rgb) sensing

Country Status (1)

Country Link
WO (1) WO2006130734A2 (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101527039B (en) * 2008-03-06 2011-12-28 河海大学 Automatic image registration and rapid super-resolution fusion method based on edge feature
US7961301B2 (en) 2008-05-09 2011-06-14 Ball Aerospace & Technologies Corp. Flash LADAR system
US9041915B2 (en) 2008-05-09 2015-05-26 Ball Aerospace & Technologies Corp. Systems and methods of scene and action capture using imaging system incorporating 3D LIDAR
JP5512675B2 (en) 2008-08-03 2014-06-04 マイクロソフト インターナショナル ホールディングス ビイ．ヴイ． Rolling camera system
US8736818B2 (en) 2010-08-16 2014-05-27 Ball Aerospace & Technologies Corp. Electronically steered flash LIDAR
DE102011053219B4 (en) 2011-09-02 2022-03-03 pmdtechnologies ag Combined pixel with phase-sensitive and color-selective sub-pixel
US9851245B2 (en) * 2012-11-06 2017-12-26 Microsoft Technology Licensing, Llc Accumulating charge from multiple imaging exposure periods
GB2511863A (en) * 2013-03-15 2014-09-17 Infrared Integrated Syst Ltd Apparatus and method for multispectral imaging
CN103500471A (en) * 2013-09-27 2014-01-08 深圳市中视典数字科技有限公司 Method for realizing high-resolution augmented reality system
WO2017058901A1 (en) 2015-09-28 2017-04-06 Ball Aerospace & Technologies Corp. Differential absorption lidar
WO2018183206A1 (en) 2017-03-26 2018-10-04 Apple, Inc. Enhancing spatial resolution in a stereo camera imaging system
US10921245B2 (en) 2018-06-08 2021-02-16 Ball Aerospace & Technologies Corp. Method and systems for remote emission detection and rate determination

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002029711A2 (en) * 2000-10-05 2002-04-11 Canesta, Inc. Methods for enhancing performance and data acquired from three-dimensional image systems
WO2003015395A1 (en) * 2001-08-06 2003-02-20 Silverbrook Research Pty. Ltd. Image sensing apparatus including a microcontroller

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6512838B1 (en) * 1999-09-22 2003-01-28 Canesta, Inc. Methods for enhancing performance and data acquired from three-dimensional image systems
WO2002029711A2 (en) * 2000-10-05 2002-04-11 Canesta, Inc. Methods for enhancing performance and data acquired from three-dimensional image systems
WO2003015395A1 (en) * 2001-08-06 2003-02-20 Silverbrook Research Pty. Ltd. Image sensing apparatus including a microcontroller

Also Published As

Publication number Publication date
WO2006130734A2 (en) 2006-12-07

Similar Documents

Publication Publication Date Title
WO2006130734A3 (en) Method and system to increase x-y resolution in a depth (z) camera using red, blue, green (rgb) sensing
WO2008121280A3 (en) Edge mapping using panchromatic pixels
IL291274B1 (en) Thin multi-aperture imaging system with auto-focus and methods for using same
WO2008121272A3 (en) Edge mapping incorporating panchromatic pixels
CN105005970B (en) The implementation method and device of a kind of augmented reality
WO2019236626A3 (en) Hdr image generation from single-shot hdr color image sensors
WO2005101308A3 (en) Method for the rotation compensation of spherical images
US9952020B2 (en) Adaptive visual camouflage
EP1901228A4 (en) Image processor
EP1876809A3 (en) Image processing device
EP1414250A3 (en) Image display apparatus and image display method
WO2008036693A3 (en) Image mask generation
JP2003022440A5 (en)
EP2546781A3 (en) Method and system for identifying illumination fields in an image
CA2609526A1 (en) Vehicle and road sign recognition device
WO2007056768A3 (en) Determining camera motion
JP2023525191A (en) Displays that use light sensors to generate environmentally consistent artificial reality content
JP2010504664A (en) Imaging method and apparatus for photographing color images with a monochrome camera
CN104933706B (en) A kind of imaging system color information scaling method
CN105869115B (en) A kind of depth image super-resolution method based on kinect2.0
CA2477517A1 (en) Color selection scheme for digital video watermarking
CN109615601A (en) A method of fusion colour and gray scale depth image
EP1395059A3 (en) Intruding-object detection apparatus
EP1923840A3 (en) Picking on fused 3D volume rendered images and updating corresponding views according to a picking action
WO2007103388A3 (en) Flexible landscape display system for information display and control

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 06771781

Country of ref document: EP

Kind code of ref document: A2

122 Ep: pct application non-entry in european phase

Ref document number: 06771781

Country of ref document: EP

Kind code of ref document: A2