USRE43722E1 - Three-dimensional ladar module with alignment reference insert circuitry - Google Patents

Three-dimensional ladar module with alignment reference insert circuitry Download PDF

Info

Publication number
USRE43722E1
USRE43722E1 US12/607,253 US60725309A USRE43722E US RE43722 E1 USRE43722 E1 US RE43722E1 US 60725309 A US60725309 A US 60725309A US RE43722 E USRE43722 E US RE43722E
Authority
US
United States
Prior art keywords
circuit
image data
data set
channels
reference point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
US12/607,253
Inventor
John Kennedy
David Ludwig
Christian Krutzik
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Longhorn Automotive Group LLC
Original Assignee
Aprolase Development Co LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US10/806,037 external-priority patent/US7180579B1/en
Application filed by Aprolase Development Co LLC filed Critical Aprolase Development Co LLC
Priority to US12/607,253 priority Critical patent/USRE43722E1/en
Assigned to APROLASE DEVELOPMENT CO., LLC reassignment APROLASE DEVELOPMENT CO., LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: IRVINE SENSORS CORPORATION
Application granted granted Critical
Publication of USRE43722E1 publication Critical patent/USRE43722E1/en
Assigned to Nytell Software LLC reassignment Nytell Software LLC MERGER (SEE DOCUMENT FOR DETAILS). Assignors: APROLASE DEVELOPMENT CO., LLC
Assigned to INTELLECTUAL VENTURES ASSETS 190 LLC reassignment INTELLECTUAL VENTURES ASSETS 190 LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: Nytell Software LLC
Assigned to AI-CORE TECHNOLOGIES, LLC reassignment AI-CORE TECHNOLOGIES, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: INTELLECTUAL VENTURES ASSETS 190 LLC
Anticipated expiration legal-status Critical
Assigned to LONGHORN AUTOMOTIVE GROUP LLC reassignment LONGHORN AUTOMOTIVE GROUP LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AI-CORE TECHNOLOGIES, LLC
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/483Details of pulse systems
    • G01S7/486Receivers
    • G01S7/487Extracting wanted echo signals, e.g. pulse detection
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06Systems determining position data of a target
    • G01S17/42Simultaneous measurement of distance and other co-ordinates
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88Lidar systems specially adapted for specific applications
    • G01S17/89Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/8943D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/483Details of pulse systems
    • G01S7/486Receivers
    • G01S7/4861Circuits for detection, sampling, integration or read-out
    • G01S7/4863Detector arrays, e.g. charge-transfer gates
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/51Display arrangements

Definitions

  • the instant invention relates generally to the field of LADAR (laser-radar) imaging technology. Specifically, the instant invention relates to a device which allows three-dimensional laser imaging of, for instance, partially obscured or camouflaged targets with very high range resolution and sensitivity.
  • LADAR laser-radar
  • existing LADAR imaging systems include a laser source, appropriate optics in conjunction with a detector array, processing circuitry suitable for processing the detector array output into a usable form and post-processing circuitry and software capable of taking the processed detector array output and converting it into a usable format such as an image on an electronic display.
  • Current LADAR imaging methods typically comprise scanning a target with a series of laser pulses and detecting the reflected photons (also referred to as a laser echo) with a photon detector array such as a focal plane array.
  • the time measured between the initiation of each laser pulse and the return of the corresponding laser echo reflected from the target to the photon detectors on the array is used to calculate target range and to define surface features on a three-dimensional object.
  • Such imaging capability is valuable in situations where, for instance, a vehicle is camouflaged or obscured by foliage or in an urban environment when an imaging sensor can acquire only a limited or angular view of a target.
  • a three-dimensional image of a target can be assembled based upon the relative echo delays calculated from the corresponding outputs of the individual detectors in the array.
  • a laser echo delay between two laser echoes (traveling at about the speed of light) of one nano-second (10 ⁇ 9 ) suggests a target surface variation of about 15 centimeters.
  • a similar laser echo delay of 500 picoseconds (10 ⁇ 12 ) between two laser echoes translates into a target surface variation of about eight centimeters.
  • Imaging means include conventional passive visible sensors, such as CCD video sensors, which provide easily interpreted information to an observer. Nonetheless, these types of sensors are undesirable where accurate scene information in a complex video environment (i.e., camouflaged or partially obscured targets) is an important factor in the observer's decision-making.
  • the instant invention addresses the aforementioned problems in prior art devices by providing a reliable, high speed, high circuit density LADAR detector system and device capable of providing range resolution and sensitivity at a centimeter level.
  • a first aspect of the present invention includes a photon source, such as a laser, beam shaping optics, a two-dimensional photo-detector array comprised of multiple photo-detector pixels, an optical lens for collecting reflected photons (laser echo) upon the detector array, and a processing module comprised of stacked layers of readout electronics integrated circuit chips (ROIC), wherein each ROIC-containing layer has one or more channels, each channel containing a digitizing and storage circuitry for processing the detector array output.
  • a photon source such as a laser, beam shaping optics, a two-dimensional photo-detector array comprised of multiple photo-detector pixels, an optical lens for collecting reflected photons (laser echo) upon the detector array, and a processing module comprised of stacked layers of readout electronics integrated circuit chips (ROIC), wherein each ROIC-containing layer has one or more channels, each channel containing a digitizing and storage circuitry for processing the detector array output.
  • ROIC readout electronics integrated circuit chips
  • external support circuitry is provided for the generating of a three-dimensional image on an electronic display from the output of the processing module.
  • one or more alignment reference points are inserted at a predetermined point in the image capture cycle to allow the electronic alignment of selected sets of image data collected by the FIFO shift registers in the one or more channels in one or more ROICs contained in the processing module.
  • alignment reference point insertion and alignment circuitry is provided as means to minimize layer-to-layer or channel-to-channel “jitter” which may be present due to variances in layer operating or data capture start-up characteristics.
  • the invention allows a user to selectively insert an alignment reference point in an individual channel.
  • the enhanced imaging capability provided by the claimed invention is achieved, in part, by the use of stacked layers containing the ROIC circuitry, which increases detector output processing circuit density while minimizing circuit lead length and associated capacitance.
  • the result of the stacked layers of ROIC circuitry is the ability to integrate a large (e.g., 128 ⁇ 128 or larger) detector array with associated dedicated detector readout circuitry (amplifier, threshold detector, sampling circuitry, analog to digital (ADC), digital to analog converters (DAC) and first in, first out, (FIFO) shift register “range” bins all within a very small module.
  • the resultant module permits the circuit speeds and densities required to resolve small, three-dimensional target features based on one or more laser echoes sensed by the detector pixels on the detector array while simultaneously providing dedicated processing channels for each detector on the detector array.
  • the multi-layer ROIC processing module is preferably comprised of a stack of layers containing thinned, integrated circuit chips, each layer including one or more receiver channels.
  • Each channel comprises circuitry which detects the laser echo time from T 0 (the start of a laser pulse or a user-assigned T 0 point) to the time of laser echo return, based on the receipt of photons that are reflected from the imaged target surfaces.
  • the laser echo time of flight information is pre-processed, and then converted to a digital bit or bits which are stored in a FIFO shift register comprising a set of “range bins” on the ROIC.
  • a high bit in a range bin may, for instance, be designated as indicating the time of arrival of a laser echo, based on its position within the set of range bins.
  • the FIFO range bin data is preferably multiplexed out of the channels in the ROIC processing module to external post-processing circuitry which, in turn, aligns and interprets the data and converts it to a usable form, such as an electronic representation of 3-D point cloud for display as an electronic image on a screen.
  • FIG. 1 is a block diagram of the present invention.
  • FIGS. 2 a and 2 b are perspective views of the individual layers and stacked configuration of the present invention
  • FIG. 3 is a cross-section the processing module of the present invention showing an exemplar T-connect and bump bond.
  • FIG. 4 is a circuit block diagram of the integrated circuit of the present invention.
  • FIG. 5 reflects a data set in a single FIFO shift register after the register is filled.
  • FIG. 6 illustrates selected circuit elements in a single layer ROIC.
  • FIG. 7 reflects a data set showing reference line insertion data in a set of range bins in a FIFO shift register over multiple clock cycles.
  • FIG. 8 is a histogram for a null image where no data are captured from the detector output.
  • FIGS. 9 a, 9 b, 9 c and 9 d show representative data sets in the FIFO of the invention in various states of data capture.
  • FIGS. 10 a and 10 b are exemplar output illustrations of pre-histogram processing and post-histogram processing of null image data in an eight layer module.
  • FIG. 1 shows a block diagram of the present invention.
  • a photon source such as a laser 1 , generates a beam pulse 5 , which is directed toward a scene or target 7 , through suitable beam-shaping optics 10 .
  • laser 1 is a 1064 nm, mechanically aligned, 300 micro-Joule, seed YAG laser capable of producing a pulse width of 500 picoseconds.
  • a beam amplifier is provided (not shown), such as a master oscillator power amplifier, wherein a seed beam is fed into the amplifier.
  • the seed beam passes through a Faraday rotator and enters a four-pass, thermally-controlled amplifier which includes two pumped YAG diodes.
  • beam-shaping optics 10 is a beam-shaping holographic lens suitable for projecting a rectangular beam area on a target. It has been determined that a rectangular beam area optimizes the ability of a detector array to receive and process reflected photons in its field of view.
  • Beam 5 is preferably amplified to generate a detectable photon reflection off of a target having 10% reflectivity at 100 meters.
  • Suitable collection optics 20 are provided, preferably a 75-mm, F1.4 lens with an appropriate spectral filter for optimization of the selected laser pulse wavelength. Beam collection optics 20 are oriented so as to receive photons that are reflected from the imaged target 7 , (i.e., the collective obscuration, subject and ground) and focus them upon the detector array as is discussed below.
  • beam 5 may optionally be pre-aligned with the detector array by passing beam 5 through Risley prisms (not shown) to steer and rotate beam 5 for “bore sighting” to the detector array. It has been determined that placing beam-shaping optics 10 in close proximity to and aligned with collection optics 20 minimizes parallax error with respect to the two optical subassemblies.
  • sensor means such as a detector assembly 30 comprising an electronic sensor, such as a detector array 40 having a detector output signal and a multilayer readout electronics processing module 50 comprising one or more integrated circuit layers.
  • Detector array 40 is preferably a focal plane array comprising individual photo-detector pixels suitable for detecting reflected photons from a target and generating an output signal in response thereto.
  • an InGaAs detector array is used.
  • Irvine Sensors Corp. the assignee herein, has demonstrated a 128 ⁇ 128 InGaAs detector array with 40-micron active detectors on 50-micron centers in the disclosed invention, wherein a 128 ⁇ 8 set of detectors on the array were accessed.
  • processing module 50 includes a bonded stack of two or more layers 60 , each layer containing one or more readout electronic integrated circuits or ROICs 70 , preferably in the form of integrated circuit chips, for the processing of signals received from detector array 40 .
  • Each readout electronic circuit 70 comprises one or more channels 75 comprised of sensor-converting electronic circuitry for the converting of the electronic output of detector array 40 to a digitized data value comprised of one or more digital bits, a plurality of which comprise an image data set.
  • Individual layers 60 are formed so that appropriate input, output, power and ground paths of read-out electronic integrated circuits 70 are rerouted using metallized traces 73 to one or more edges of each layer 60 to form one or more access leads 80 .
  • Layers 60 are bonded together using an appropriate adhesive 62 such as an epoxy to form a three-dimensional module and the lateral surfaces processed (e.g., ground or lapped) so as to expose conductive access leads 80 .
  • Layers 60 may optionally be thinned prior to bonding using techniques that are known in the field of semiconductor processing and packaging. Thinned active layers may also be embedded silicon cap chips for ease of handling.
  • Suitable interconnect pads or equivalent structures 90 are formed on one or more lateral surfaces of processing module 50 to form one or more “T-connects” 85 to allow the electrical connection of access leads 80 directly to the outputs of detector array 40 and to the image processing electronics in the stack such as by the use of conductive bumps 92 .
  • FIG. 3 illustrates a cross-section of an exemplar T-connect 85 between a bump bond on a detector array 40 and a metallized access lead 80 .
  • the T-connect structures conveniently allow the interconnection of the high-density stack of layers to detector array 40 and external circuitry contained external of the processing module.
  • the preferred embodiment provides interconnect pads 90 formed on one or more processor module lateral surfaces so as to allow the electrical connection, such as by bump bonding, of detector array 40 .
  • Indium bump bonding is a preferred method of bump bonding in the disclosed invention.
  • the volume occupied by perpendicularly disposed detector array 40 and processor module 50 is minimized yet the detector-to-readout circuitry lead length is optimized to increase speed while reducing parasitic capacitance.
  • Such an interconnection orientation takes full advantage of the high circuit density of the processing module while providing very short circuit paths necessary for high-speed laser range resolution.
  • external electronic circuitry 100 for receiving and processing the output of processor module 50 may be connected to interconnect pads 90 through the use of compressible conductors such as “Fuzz Buttons” available from Technical Wire Products, Inc., as disclosed in U.S. patent application Ser. No. 11/499,403; filed Aug. 4, 2006, entitled High Density Interconnect Scheme, to Kennedy, which application is fully incorporated herein by reference.
  • connection of external electronic circuitry 100 to interconnect pads 90 include solder, conductive epoxies, wirebonding or other suitable means formed on the accessible conductive surfaces of processing module 50 .
  • beam 5 expands outward from beam-shaping optics 10 and is imaged upon the desired target.
  • Photons from the pulsed beam 5 which are not absorbed or scattered by the target, are reflected and returned to collection optics 20 from the target surface. Because the distance between the differing target surfaces and collection optics varies, so will the time that the reflected photons are received by the detector assembly. Note that each individual detector in the detector array may receive multiple signal returns from photon reflections off of scene surfaces at different distances from the detector array.
  • Beam 5 is swept across the target area in a scan to attain, by way of example, 256 vertical samples.
  • the laser is pulsed at 94 Hz to attain 256 horizontal samples while the channel data records any received laser echo in suitable storage circuit means such as a FIFO shift register for the storing of a digitized data value.
  • storage circuit means comprises 1024 FIFO shift register range bin circuits. Each range bin circuit has a variable logic state and a unique position in the storage circuit.
  • each ROIC chip is preferably comprised of multiple channels.
  • each channel 75 provides a dedicated detector signal input circuit and includes circuitry that receives a T 0 trigger signal (initial laser pulse trigger) to provide the FIFO shift register with a reference for the time of the laser echo's arrival at detector array 40 .
  • the preferred laser pulse is short relative to the capture interval (e.g. ⁇ 500 picoseconds vs. 500 nanoseconds) and multiple returns can be detected by a single detector pixel.
  • Each channel 75 is preferably comprised of an analog section that operates with a relatively high bandwidth.
  • Each channel further comprises a digital section operating at, in a preferred embodiment, up to 2 GHz during the time the FIFO shift register is filling with data.
  • a digital to analog converter is included in channel 75 as well as circuitry to allow a user to adjust an included comparator's threshold.
  • Each ROIC is preferably provided with multiple channels (e.g., 128 channels).
  • Received laser echoes are integrated and filtered by the channel during an image capture interval.
  • the integrated signals are then amplified and differentiated using suitable electronic circuitry for converting received echoes into electronic pulses with an amplified signal strength sufficient to allow detection by the comparator circuitry.
  • the comparator circuitry detects whether the pulses are above or below a programmable predetermined threshold. In the preferred embodiment, the comparator's output is sampled at a 2 GHz rate, resulting in a one bit, analog to digital converted signal.
  • each channel 75 includes a 1024 deep, FIFO shift register (i.e., a 1024 bin shift register) which receives and stores the comparator's output from T 0 to T 0 plus approximately 500 nanoseconds.
  • a FIFO shift register bin set of 1024 this results in each bin representing a period of about 500 picoseconds, (i.e., 500 ns/1024 bins).
  • a “1” or high bit in a FIFO register may be regarded as marker representing the arrival time of a laser echo with respect to the T 0 signal, as determined by the bit's location in the register.
  • each bin of the shift register represents about 500 picoseconds of time history, which in turn, equates to about 7.5 cm of range history based on the laser echo traveling at about the speed of light.
  • the initiation point of the T 0 trigger signal is preferably variable by a user to allow system adjustment to accommodate for longer return times for laser echoes returning from more distant targets.
  • the captured FIFO contents are read out to external processing circuitry 100 .
  • a preferred method of reading out the FIFO register data is by multiplexing the digitized FIFO data out to external processing circuitry 100 using conventional frame and line signals at a 20 MHz rate.
  • the multiplexed data may be processed at this point using appropriate external circuitry to create a 3-D point cloud defined by the target and further processed to create a 3-D electronic image or may be further processed to minimize jitter as discussed below.
  • the 256 ⁇ 256 ⁇ 1024 data is recorded as a frame. Multiple frames of FIFO image data are electronically registered and coalesced into a 3-D point cloud in suitable external electronic circuitry for display as a 3-D image for classification of target objects.
  • FIG. 5 illustrates an exemplar set of image data that have been captured on a channel in a set of 0-n FIFO register bins on an ROIC layer X from a returning laser echo. It can be seen that during the capture cycle in which the bins in layer X were filling, a laser echo was received in bin 5 as a logical 1.
  • the logical 1 at bin 5 can be used to calculate the time of flight of the initial laser pulse and its return, which in turn is used to calculate the distance of the surface from which the laser was reflected.
  • the individual layers may capture data asynchronously with respect to the other layers in the stack.
  • layer-to-layer asynchronous data capture or jitter resulting from variances in layer-to-layer operation is minimized by inserting a digital alignment reference point or “brick wall” at about the end of each image data capture cycle at the same point in time in each selected channel during range bin filling.
  • a reference insert circuit means is provided for inserting one or more user-defined reference points at one or more predetermined positions in the image data set.
  • Layer-to-layer jitter can then be minimized by realigning the inserted alignment reference points on each layer to each of the other layers, thus shifting the images of the layers (i.e., image data sets) into alignment.
  • a solid electronic “wall” of logical “1”s is asserted on the reference point insert line which is common to all FIFO inputs in each of the layers, and is seen in the set of selected bins as an electronic alignment point.
  • This method and device rely on the fact that when the bulk of the data have been sampled, the alignment reference point insert line is activated and, from that data capture point forward, all inputs into the FIFO shift registers are registered as logical “1”s.
  • the alignment reference points are inserted using appropriate reference insert circuit means in electrical connection with at least one preselected bin circuit (e.g. a data input port or TESTIN herein) in the FIFO shift registers for selectively asserting a user-defined predetermined bin logic state in at least one preselected bin circuit in the FIFO shift register in one or more ROIC channels.
  • a preselected bin circuit e.g. a data input port or TESTIN herein
  • TESTIN data input port
  • FIG. 6 a block diagram is shown reflecting an exemplar single layer of the ROIC inside a 3 D laser imaging system.
  • Each detector is in electrical connection with an amplifier, which detector output undergoes an analog to digital conversion.
  • the digitized output is received by the FIFO at the data input port of the shift register, which, in this example, can store 1024 consecutive samples of data.
  • the resulting image size is (number ofpixels)(1)(1024).
  • Multiple layers are stacked to form a three-dimensional image, i.e., eight layers of an ROIC would provide an image of 68 ⁇ 8 ⁇ 1024 pixels.
  • FIG. 7 shows a basic illustration of the operation of the reference insert line in a FIFO shift register having 0-n bins in LADAR system in an exemplar 13 clock cycles.
  • the Xs represent “don't care” states that are unfilled bins in a FIFO shift register during an image capture cycle.
  • the digitized output of a detector is received by the input bin in the FIFO shift register.
  • the input is a logical 0.
  • the logical 0 represents no laser echo captured by the detector in clock cycle 0 .
  • a digitized laser echo is received, resulting in a logical 1 being received into the FIFO bin set.
  • the FIFO shift register bins continue to fill with and shift data from the output of the detector in the system.
  • a predetermined insert time has been selected, in this instance, at about bin 3 , wherein at the point in time where detector data (either 0 or 1) has been received by bin 3 , the reference insert line is activated such that alignment reference points are fed into and received by the FIFO shift register bins in each of the layers. Logical “1”s from the reference insert line will continue to be received by the bins in the shift register until the entire set of bins is full (e.g., clock cycle 12 in this example) at which time, the entire data set from the channel is read out for further data aligning and processing as discussed further below.
  • Reference point synchronization circuit means is provided for performing such histogram calculations which means can desirably be implemented in a field programmable gate array.
  • FIG. 8 a histogram for a null image is shown, where no data are captured from the detector output.
  • 950 FIFO bins of valid data have been captured, the reference point insert activated and the remaining bins 951 to 1023 now contain a “brick wall” of logical “1”s.
  • the bin count ramps up to the maximum at the alignment reference point insertion, i.e., 950 .
  • FIG. 9 a shows an exemplar data set prior to reference point correction where the laser echo return indicated by a “1” physically occurs at bin 150 but due to layer jitter, the individual layers acquire the echoes as offset in varying amounts with respect to each other.
  • the corresponding alignment reference point data would then appear as that illustrated in FIG. 9 b where the predetermined reference insert time is designated as occurring at about the time bin 950 receives digitized detector data.
  • the user selectively defines a time period or number of clock cycles at which point the reference line is asserted so as to insert the brick wall into the bins in each of the layers or channels. For example, where each bin represents about 500 picoseconds in a 2 GHz clock system, a user can define the reference insert line to assert at, for instance, 475 nanoseconds, which translates to a brick wall insert at about bin number 950 .
  • the correction offset table is shown in FIG. 9 c, showing the number of bins a layer's data must be moved to align the data set to the other layers' data sets.
  • FIG. 9 d illustrates how the jitter correction operates where the acquired image data is located around bin 150 once the images in each layer have been moved by the number of bins calculated as a correction amount.
  • the brick wall is shown from bin 950 to the remaining number of bins in the register.
  • FIGS. 10 a and 10 b Exemplar outputs of pre-histogram processing and post-histogram processing of null image data in an eight layer module are shown in FIGS. 10 a and 10 b respectively. As is seen, the bin count increase at the alignment reference point of bin 950 is much steeper in the corrected figure indicating the data alignment shift.
  • Appropriate reference point synchronization circuit means such as an FPGA is incorporated with output circuit means, such as multiplexing circuitry for the receiving of and synchronizing of the outputted digitized data values of the FIFO shift registers on one or more of said layers using the inserted reference points for generating a plurality of synchronized digitized data values.
  • the digitized and aligned data values are further processed by suitable post-processing circuitry 100 to create a 3 D point cloud for producing an image.

Abstract

A 3-D LADAR imaging system incorporating stacked microelectronic layers is provided. A light source such as a laser is imaged upon a target through beam shaping optics. Photons reflected from the target are collected and imaged upon a detector array though collection optics. The detector array signals are fed into a multilayer processing module wherein each layer includes detector signal processing circuitry. The detector array signals are amplified, compared to a user-defined threshold, digitized and fed into a high speed FIFO shift register range bin. Dependant on the value of the digit contained in the bins in the register, and the digit's bin location, the time of a photon reflection from a target surface can be determined. A T0 trigger signal defines the reflection time represented at each bin location by resetting appropriate circuitry to begin processing.
A reference insert circuit inserts data into the FIFO registers at a preselected location to provide a reference point at which all FIFO shift register data may be aligned to accommodate for timing differences between layers and channels. The bin data representing the photon reflections from the various target surfaces are read out of the FIFO and processed using appropriate circuitry such as a field programmable gate array to create a synchronized 3-D point cloud for creating a 3-D target image.

Description

CROSS REFERENCE TO RELATED APPLICATIONS
This application is related a Broadening Reissue of U.S. patent application Ser. No. 11/706,724, filed Feb. 15, 2007 (now U.S. Pat. No. 7,436,494), which claims priority to U.S. provisional patent application Provisional Patent Application No. 60/785,135, filed on Mar. 24, 2006, entitled, “Method for Image Jitter Reduction in a Multilayer LADAR Device” and is a continuation-in-part Continuation-in-Part of U.S. patent application Ser. No. 10/806,037, filed on Mar. 22, 2004, entitled, “Three-Dimensional Imaging Processing Module Incorporating Stacked Layers Containing Microelectronic Circuitry,”, (now U.S. Pat. No. 7,180,579), issued Feb. 20, 2007 to Ludwig, et al., which claims the benefit of U.S. provisional application Provisional Patent Application No. 60,462,677, filed Mar. 28, 2003., all of which are U.S. Provisional Patent Application No. 60,785,135 and U.S. patent application Ser. No. 10/806,037 are both incorporated herein by reference and to which priority is claimed pursuant to 35 USC 119.
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH AND DEVELOPMENT
This invention was made with U.S. Government support under Contract No. DAAD17-01-D-0006-0002, awarded by the Army Research Laboratory (ARL). The U.S. Government has certain rights in the invention.
BACKGROUND OF THE INVENTION
1. Field of the Invention
The instant invention relates generally to the field of LADAR (laser-radar) imaging technology. Specifically, the instant invention relates to a device which allows three-dimensional laser imaging of, for instance, partially obscured or camouflaged targets with very high range resolution and sensitivity.
2. Description of the Related Art
In general, existing LADAR imaging systems include a laser source, appropriate optics in conjunction with a detector array, processing circuitry suitable for processing the detector array output into a usable form and post-processing circuitry and software capable of taking the processed detector array output and converting it into a usable format such as an image on an electronic display.
Current LADAR imaging methods typically comprise scanning a target with a series of laser pulses and detecting the reflected photons (also referred to as a laser echo) with a photon detector array such as a focal plane array. The time measured between the initiation of each laser pulse and the return of the corresponding laser echo reflected from the target to the photon detectors on the array is used to calculate target range and to define surface features on a three-dimensional object. Such imaging capability is valuable in situations where, for instance, a vehicle is camouflaged or obscured by foliage or in an urban environment when an imaging sensor can acquire only a limited or angular view of a target.
Because the time of flight of the returning laser echoes will vary based on the distance between the detector array plane and the individual surface features from which the echoes are received, a three-dimensional image of a target can be assembled based upon the relative echo delays calculated from the corresponding outputs of the individual detectors in the array.
As an example, a laser echo delay between two laser echoes (traveling at about the speed of light) of one nano-second (10−9) suggests a target surface variation of about 15 centimeters. A similar laser echo delay of 500 picoseconds (10−12) between two laser echoes translates into a target surface variation of about eight centimeters.
As is evident from these short time periods, very high detector signal processing and timing circuit speeds are desirable in order to resolve target surface feature variations at a centimeter-level depth resolution. Unfortunately, existing LADAR imaging systems lack the necessary circuit speed and capacity to achieve very high (i.e., centimeter) range resolution and sensitivity.
Alternative prior art imaging means include conventional passive visible sensors, such as CCD video sensors, which provide easily interpreted information to an observer. Nonetheless, these types of sensors are undesirable where accurate scene information in a complex video environment (i.e., camouflaged or partially obscured targets) is an important factor in the observer's decision-making.
Accordingly, a need exists for a LADAR imaging system that has the circuit speed and density required to achieve range resolution and sensitivity necessary to define small target surface variations based on picosecond time of flight differences in laser echoes.
The instant invention addresses the aforementioned problems in prior art devices by providing a reliable, high speed, high circuit density LADAR detector system and device capable of providing range resolution and sensitivity at a centimeter level.
BRIEF SUMMARY OF THE INVENTION
A first aspect of the present invention includes a photon source, such as a laser, beam shaping optics, a two-dimensional photo-detector array comprised of multiple photo-detector pixels, an optical lens for collecting reflected photons (laser echo) upon the detector array, and a processing module comprised of stacked layers of readout electronics integrated circuit chips (ROIC), wherein each ROIC-containing layer has one or more channels, each channel containing a digitizing and storage circuitry for processing the detector array output.
In a further aspect of the invention, external support circuitry is provided for the generating of a three-dimensional image on an electronic display from the output of the processing module.
In yet a further aspect of the invention, one or more alignment reference points are inserted at a predetermined point in the image capture cycle to allow the electronic alignment of selected sets of image data collected by the FIFO shift registers in the one or more channels in one or more ROICs contained in the processing module. In this aspect of the invention, alignment reference point insertion and alignment circuitry is provided as means to minimize layer-to-layer or channel-to-channel “jitter” which may be present due to variances in layer operating or data capture start-up characteristics. Alternatively, the invention allows a user to selectively insert an alignment reference point in an individual channel.
The enhanced imaging capability provided by the claimed invention is achieved, in part, by the use of stacked layers containing the ROIC circuitry, which increases detector output processing circuit density while minimizing circuit lead length and associated capacitance. The result of the stacked layers of ROIC circuitry is the ability to integrate a large (e.g., 128×128 or larger) detector array with associated dedicated detector readout circuitry (amplifier, threshold detector, sampling circuitry, analog to digital (ADC), digital to analog converters (DAC) and first in, first out, (FIFO) shift register “range” bins all within a very small module.
The resultant module permits the circuit speeds and densities required to resolve small, three-dimensional target features based on one or more laser echoes sensed by the detector pixels on the detector array while simultaneously providing dedicated processing channels for each detector on the detector array.
The multi-layer ROIC processing module is preferably comprised of a stack of layers containing thinned, integrated circuit chips, each layer including one or more receiver channels. Each channel comprises circuitry which detects the laser echo time from T0 (the start of a laser pulse or a user-assigned T0 point) to the time of laser echo return, based on the receipt of photons that are reflected from the imaged target surfaces. The laser echo time of flight information is pre-processed, and then converted to a digital bit or bits which are stored in a FIFO shift register comprising a set of “range bins” on the ROIC. A high bit in a range bin may, for instance, be designated as indicating the time of arrival of a laser echo, based on its position within the set of range bins.
The FIFO range bin data is preferably multiplexed out of the channels in the ROIC processing module to external post-processing circuitry which, in turn, aligns and interprets the data and converts it to a usable form, such as an electronic representation of 3-D point cloud for display as an electronic image on a screen.
While the claimed apparatus and method has or will be described for the sake of grammatical fluidity with functional explanations, it is to be expressly understood that the claims, unless expressly formulated under 35 USC 112, are not to be construed as necessarily limited in any way by the construction of “means” or “steps” limitations, but are to be accorded the full scope of the meaning and equivalents of the definition provided by the claims under the judicial doctrine of equivalents, and in the case where the claims are expressly formulated under 35 USC 112 are to be accorded full statutory equivalents under 35 USC 112.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram of the present invention.
FIGS. 2a and 2b are perspective views of the individual layers and stacked configuration of the present invention
FIG. 3 is a cross-section the processing module of the present invention showing an exemplar T-connect and bump bond.
FIG. 4 is a circuit block diagram of the integrated circuit of the present invention.
FIG. 5 reflects a data set in a single FIFO shift register after the register is filled.
FIG. 6 illustrates selected circuit elements in a single layer ROIC.
FIG. 7 reflects a data set showing reference line insertion data in a set of range bins in a FIFO shift register over multiple clock cycles.
FIG. 8 is a histogram for a null image where no data are captured from the detector output.
FIGS. 9a, 9b, 9c and 9d show representative data sets in the FIFO of the invention in various states of data capture.
FIGS. 10a and 10b are exemplar output illustrations of pre-histogram processing and post-histogram processing of null image data in an eight layer module.
The invention and its various embodiments can now be better understood by turning to the following detailed description of the preferred embodiments which are presented as illustrated examples of the invention defined in the claims. It is expressly understood that the invention as defined by the claims may be broader than the illustrated embodiments described below.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
Turning now to the figures wherein like numerals designate like elements among the several views, FIG. 1 shows a block diagram of the present invention.
A photon source, such as a laser 1, generates a beam pulse 5, which is directed toward a scene or target 7, through suitable beam-shaping optics 10.
In a preferred embodiment, laser 1 is a 1064 nm, mechanically aligned, 300 micro-Joule, seed YAG laser capable of producing a pulse width of 500 picoseconds. In the preferred embodiment, a beam amplifier is provided (not shown), such as a master oscillator power amplifier, wherein a seed beam is fed into the amplifier. In this preferred embodiment, the seed beam passes through a Faraday rotator and enters a four-pass, thermally-controlled amplifier which includes two pumped YAG diodes.
In the preferred embodiment, beam-shaping optics 10 is a beam-shaping holographic lens suitable for projecting a rectangular beam area on a target. It has been determined that a rectangular beam area optimizes the ability of a detector array to receive and process reflected photons in its field of view.
Beam 5 is preferably amplified to generate a detectable photon reflection off of a target having 10% reflectivity at 100 meters.
Suitable collection optics 20 are provided, preferably a 75-mm, F1.4 lens with an appropriate spectral filter for optimization of the selected laser pulse wavelength. Beam collection optics 20 are oriented so as to receive photons that are reflected from the imaged target 7, (i.e., the collective obscuration, subject and ground) and focus them upon the detector array as is discussed below.
After beam amplification, beam 5 may optionally be pre-aligned with the detector array by passing beam 5 through Risley prisms (not shown) to steer and rotate beam 5 for “bore sighting” to the detector array. It has been determined that placing beam-shaping optics 10 in close proximity to and aligned with collection optics 20 minimizes parallax error with respect to the two optical subassemblies.
Further included is sensor means such as a detector assembly 30 comprising an electronic sensor, such as a detector array 40 having a detector output signal and a multilayer readout electronics processing module 50 comprising one or more integrated circuit layers.
Detector array 40 is preferably a focal plane array comprising individual photo-detector pixels suitable for detecting reflected photons from a target and generating an output signal in response thereto. In the preferred embodiment, an InGaAs detector array is used. Irvine Sensors Corp., the assignee herein, has demonstrated a 128×128 InGaAs detector array with 40-micron active detectors on 50-micron centers in the disclosed invention, wherein a 128×8 set of detectors on the array were accessed.
Turning now to FIGS. 2a, 2b and 3, processing module 50 includes a bonded stack of two or more layers 60, each layer containing one or more readout electronic integrated circuits or ROICs 70, preferably in the form of integrated circuit chips, for the processing of signals received from detector array 40. Each readout electronic circuit 70 comprises one or more channels 75 comprised of sensor-converting electronic circuitry for the converting of the electronic output of detector array 40 to a digitized data value comprised of one or more digital bits, a plurality of which comprise an image data set.
Individual layers 60 are formed so that appropriate input, output, power and ground paths of read-out electronic integrated circuits 70 are rerouted using metallized traces 73 to one or more edges of each layer 60 to form one or more access leads 80.
Layers 60 are bonded together using an appropriate adhesive 62 such as an epoxy to form a three-dimensional module and the lateral surfaces processed (e.g., ground or lapped) so as to expose conductive access leads 80. Layers 60 may optionally be thinned prior to bonding using techniques that are known in the field of semiconductor processing and packaging. Thinned active layers may also be embedded silicon cap chips for ease of handling.
Suitable interconnect pads or equivalent structures 90 are formed on one or more lateral surfaces of processing module 50 to form one or more “T-connects” 85 to allow the electrical connection of access leads 80 directly to the outputs of detector array 40 and to the image processing electronics in the stack such as by the use of conductive bumps 92.
FIG. 3 illustrates a cross-section of an exemplar T-connect 85 between a bump bond on a detector array 40 and a metallized access lead 80. The T-connect structures conveniently allow the interconnection of the high-density stack of layers to detector array 40 and external circuitry contained external of the processing module.
The preferred embodiment provides interconnect pads 90 formed on one or more processor module lateral surfaces so as to allow the electrical connection, such as by bump bonding, of detector array 40. Indium bump bonding is a preferred method of bump bonding in the disclosed invention.
After electrical connection of detector array 40, the volume occupied by perpendicularly disposed detector array 40 and processor module 50 is minimized yet the detector-to-readout circuitry lead length is optimized to increase speed while reducing parasitic capacitance. Such an interconnection orientation takes full advantage of the high circuit density of the processing module while providing very short circuit paths necessary for high-speed laser range resolution.
Referring back to FIG. 1, external electronic circuitry 100 for receiving and processing the output of processor module 50 may be connected to interconnect pads 90 through the use of compressible conductors such as “Fuzz Buttons” available from Technical Wire Products, Inc., as disclosed in U.S. patent application Ser. No. 11/499,403; filed Aug. 4, 2006, entitled High Density Interconnect Scheme, to Kennedy, which application is fully incorporated herein by reference.
Alternative means for connection of external electronic circuitry 100 to interconnect pads 90 include solder, conductive epoxies, wirebonding or other suitable means formed on the accessible conductive surfaces of processing module 50.
In system operation, beam 5 expands outward from beam-shaping optics 10 and is imaged upon the desired target. Photons from the pulsed beam 5 which are not absorbed or scattered by the target, are reflected and returned to collection optics 20 from the target surface. Because the distance between the differing target surfaces and collection optics varies, so will the time that the reflected photons are received by the detector assembly. Note that each individual detector in the detector array may receive multiple signal returns from photon reflections off of scene surfaces at different distances from the detector array.
Beam 5 is swept across the target area in a scan to attain, by way of example, 256 vertical samples. In this example, the laser is pulsed at 94 Hz to attain 256 horizontal samples while the channel data records any received laser echo in suitable storage circuit means such as a FIFO shift register for the storing of a digitized data value. In a preferred embodiment, storage circuit means comprises 1024 FIFO shift register range bin circuits. Each range bin circuit has a variable logic state and a unique position in the storage circuit.
An exemplar block diagram of a preferred embodiment of a channel on the ROIC chip is shown in FIG. 4. As indicated above, each ROIC chip is preferably comprised of multiple channels. As illustrated in FIGS. 1 and 4, each channel 75 provides a dedicated detector signal input circuit and includes circuitry that receives a T0 trigger signal (initial laser pulse trigger) to provide the FIFO shift register with a reference for the time of the laser echo's arrival at detector array 40. The preferred laser pulse is short relative to the capture interval (e.g.−500 picoseconds vs. 500 nanoseconds) and multiple returns can be detected by a single detector pixel.
Each channel 75 is preferably comprised of an analog section that operates with a relatively high bandwidth. Each channel further comprises a digital section operating at, in a preferred embodiment, up to 2 GHz during the time the FIFO shift register is filling with data. A digital to analog converter is included in channel 75 as well as circuitry to allow a user to adjust an included comparator's threshold. Each ROIC is preferably provided with multiple channels (e.g., 128 channels).
Received laser echoes are integrated and filtered by the channel during an image capture interval. The integrated signals are then amplified and differentiated using suitable electronic circuitry for converting received echoes into electronic pulses with an amplified signal strength sufficient to allow detection by the comparator circuitry. The comparator circuitry detects whether the pulses are above or below a programmable predetermined threshold. In the preferred embodiment, the comparator's output is sampled at a 2 GHz rate, resulting in a one bit, analog to digital converted signal.
In the preferred embodiment, each channel 75 includes a 1024 deep, FIFO shift register (i.e., a 1024 bin shift register) which receives and stores the comparator's output from T0 to T0 plus approximately 500 nanoseconds. In a FIFO shift register bin set of 1024, this results in each bin representing a period of about 500 picoseconds, (i.e., 500 ns/1024 bins). Each bin in the FIFO has a position in the shift register and is capable of having a variable logic state (e.g., 0V=logical “0”, +1.8V=logical “1”). A “1” or high bit in a FIFO register may be regarded as marker representing the arrival time of a laser echo with respect to the T0 signal, as determined by the bit's location in the register. With the FIFO clocking at an exemplar 2 GHz rate, each bin of the shift register represents about 500 picoseconds of time history, which in turn, equates to about 7.5 cm of range history based on the laser echo traveling at about the speed of light.
The initiation point of the T0 trigger signal is preferably variable by a user to allow system adjustment to accommodate for longer return times for laser echoes returning from more distant targets.
After the 1024 bins of the FIFO shift registers in the channels in the preferred embodiment are filled, the captured FIFO contents are read out to external processing circuitry 100. A preferred method of reading out the FIFO register data is by multiplexing the digitized FIFO data out to external processing circuitry 100 using conventional frame and line signals at a 20 MHz rate. The multiplexed data may be processed at this point using appropriate external circuitry to create a 3-D point cloud defined by the target and further processed to create a 3-D electronic image or may be further processed to minimize jitter as discussed below.
The 256×256×1024 data is recorded as a frame. Multiple frames of FIFO image data are electronically registered and coalesced into a 3-D point cloud in suitable external electronic circuitry for display as a 3-D image for classification of target objects.
FIG. 5 illustrates an exemplar set of image data that have been captured on a channel in a set of 0-n FIFO register bins on an ROIC layer X from a returning laser echo. It can be seen that during the capture cycle in which the bins in layer X were filling, a laser echo was received in bin 5 as a logical 1. The logical 1 at bin 5 can be used to calculate the time of flight of the initial laser pulse and its return, which in turn is used to calculate the distance of the surface from which the laser was reflected.
Because multiple ROIC layers in a high-speed LADAR system (e.g., GHz clock speeds) do not necessarily begin acquiring data at precisely the same instant, the individual layers may capture data asynchronously with respect to the other layers in the stack. In an alternative preferred embodiment, layer-to-layer asynchronous data capture or jitter resulting from variances in layer-to-layer operation is minimized by inserting a digital alignment reference point or “brick wall” at about the end of each image data capture cycle at the same point in time in each selected channel during range bin filling. A reference insert circuit means is provided for inserting one or more user-defined reference points at one or more predetermined positions in the image data set.
Layer-to-layer jitter can then be minimized by realigning the inserted alignment reference points on each layer to each of the other layers, thus shifting the images of the layers (i.e., image data sets) into alignment.
When a 3 D image is being captured, a solid electronic “wall” of logical “1”s is asserted on the reference point insert line which is common to all FIFO inputs in each of the layers, and is seen in the set of selected bins as an electronic alignment point. This method and device rely on the fact that when the bulk of the data have been sampled, the alignment reference point insert line is activated and, from that data capture point forward, all inputs into the FIFO shift registers are registered as logical “1”s.
The alignment reference points are inserted using appropriate reference insert circuit means in electrical connection with at least one preselected bin circuit (e.g. a data input port or TESTIN herein) in the FIFO shift registers for selectively asserting a user-defined predetermined bin logic state in at least one preselected bin circuit in the FIFO shift register in one or more ROIC channels. Using a common input into the FIFO in each layer in the stack allows the insertion of the alignment reference point into the incoming image data at a user defined point in time. Upon readout of the image data from the FIFO in which the alignment reference point has been inserted, a histogram on each of the layers' data is performed to determine the alignment reference point position of each layer to define the realignment point for all data sets.
Turning now to FIG. 6, a block diagram is shown reflecting an exemplar single layer of the ROIC inside a 3 D laser imaging system. Each detector is in electrical connection with an amplifier, which detector output undergoes an analog to digital conversion.
The digitized output is received by the FIFO at the data input port of the shift register, which, in this example, can store 1024 consecutive samples of data. In the illustrated embodiment, the resulting image size is (number ofpixels)(1)(1024). Multiple layers are stacked to form a three-dimensional image, i.e., eight layers of an ROIC would provide an image of 68×8×1024 pixels.
FIG. 7 shows a basic illustration of the operation of the reference insert line in a FIFO shift register having 0-n bins in LADAR system in an exemplar 13 clock cycles.
In FIG. 7, the Xs represent “don't care” states that are unfilled bins in a FIFO shift register during an image capture cycle.
At clock cycle 0, the digitized output of a detector is received by the input bin in the FIFO shift register. In this instance, the input is a logical 0. In this embodiment, the logical 0 represents no laser echo captured by the detector in clock cycle 0.
At clock cycle 1, a digitized laser echo is received, resulting in a logical 1 being received into the FIFO bin set.
At clock cycles 2-8, the FIFO shift register bins continue to fill with and shift data from the output of the detector in the system.
Turning to clock cycle 9, a predetermined insert time has been selected, in this instance, at about bin 3, wherein at the point in time where detector data (either 0 or 1) has been received by bin 3, the reference insert line is activated such that alignment reference points are fed into and received by the FIFO shift register bins in each of the layers. Logical “1”s from the reference insert line will continue to be received by the bins in the shift register until the entire set of bins is full (e.g., clock cycle 12 in this example) at which time, the entire data set from the channel is read out for further data aligning and processing as discussed further below.
Because the data sets that are read out contain reference points for use in aligning other data sets therewith, it is possible to remove jitter associated with the asynchronous operation of each layer by performing a histogram calculation such as a simple cumulative histogram. Reference point synchronization circuit means is provided for performing such histogram calculations which means can desirably be implemented in a field programmable gate array.
With respect to FIG. 8, a histogram for a null image is shown, where no data are captured from the detector output. In FIG. 8, 950 FIFO bins of valid data have been captured, the reference point insert activated and the remaining bins 951 to 1023 now contain a “brick wall” of logical “1”s. As is reflected, the bin count ramps up to the maximum at the alignment reference point insertion, i.e., 950.
Turning to FIGS. 9a-9d, FIG. 9a shows an exemplar data set prior to reference point correction where the laser echo return indicated by a “1” physically occurs at bin 150 but due to layer jitter, the individual layers acquire the echoes as offset in varying amounts with respect to each other.
The corresponding alignment reference point data would then appear as that illustrated in FIG. 9b where the predetermined reference insert time is designated as occurring at about the time bin 950 receives digitized detector data. In other words, the user selectively defines a time period or number of clock cycles at which point the reference line is asserted so as to insert the brick wall into the bins in each of the layers or channels. For example, where each bin represents about 500 picoseconds in a 2 GHz clock system, a user can define the reference insert line to assert at, for instance, 475 nanoseconds, which translates to a brick wall insert at about bin number 950.
The correction offset table is shown in FIG. 9c, showing the number of bins a layer's data must be moved to align the data set to the other layers' data sets.
FIG. 9d illustrates how the jitter correction operates where the acquired image data is located around bin 150 once the images in each layer have been moved by the number of bins calculated as a correction amount. The brick wall is shown from bin 950 to the remaining number of bins in the register.
Exemplar outputs of pre-histogram processing and post-histogram processing of null image data in an eight layer module are shown in FIGS. 10a and 10b respectively. As is seen, the bin count increase at the alignment reference point of bin 950 is much steeper in the corrected figure indicating the data alignment shift.
Appropriate reference point synchronization circuit means, such as an FPGA is incorporated with output circuit means, such as multiplexing circuitry for the receiving of and synchronizing of the outputted digitized data values of the FIFO shift registers on one or more of said layers using the inserted reference points for generating a plurality of synchronized digitized data values.
The digitized and aligned data values are further processed by suitable post-processing circuitry 100 to create a 3 D point cloud for producing an image.
Many alterations and modifications may be made by those having ordinary skill in the art without departing from the spirit and scope of the invention. Therefore, it must be understood that the illustrated embodiment has been set forth only for the purposes of example and that it should not be taken as limiting the invention as defined by the following claims. For example, notwithstanding the fact that the elements of a claim are set forth below in a certain combination, it must be expressly understood that the invention includes other combinations of fewer, more or different elements, which are disclosed in above even when not initially claimed in such combinations.
The words used in this specification to describe the invention and its various embodiments are to be understood not only in the sense of their commonly defined meanings, but to include by special definition in this specification structure, material or acts beyond the scope of the commonly defined meanings. Thus if an element can be understood in the context of this specification as including more than one meaning, then its use in a claim must be understood as being generic to all possible meanings supported by the specification and by the word itself.
The definitions of the words or elements of the following claims are, therefore, defined in this specification to include not only the combination of elements which are literally set forth, but all equivalent structure, material or acts for performing substantially the same function in substantially the same way to obtain substantially the same result. In this sense it is therefore contemplated that an equivalent substitution of two or more elements may be made for any one of the elements in the claims below or that a single element may be substituted for two or more elements in a claim. Although elements may be described above as acting in certain combinations and even initially claimed as such, it is to be expressly understood that one or more elements from a claimed combination can in some cases be excised from the combination and that the claimed combination may be directed to a subcombination or variation of a subcombination.
Insubstantial changes from the claimed subject matter as viewed by a person with ordinary skill in the art, now known or later devised, are expressly contemplated as being equivalently within the scope of the claims. Therefore, obvious substitutions now or later known to one with ordinary skill in the art are defined to be within the scope of the defined elements.
The claims are thus to be understood to include what is specifically illustrated and described above, what is conceptually equivalent, what can be obviously substituted and also what essentially incorporates the essential idea of the invention.

Claims (32)

1. An electronic circuit comprised of comprising:
a photon detector array means for convertingconfigured to convert photons tointo a detector output signal,;
a plurality of readout electronics integrated circuit chips, each readout electronics integrated circuit chip comprising including a plurality of channels for receiving configured to receive and processing process output signals generated by said the photon detector array, wherein said the plurality of readout electronics integrated circuit chips are arranged in a stacked configuration, with each of said the readout electronics integrated circuit chips forming one layer of said the stacked configuration, and wherein said the photon detector array is bonded to a lateral surface of said the stacked configuration perpendicular to said the stacked configuration and is connected to the plurality of channels via a plurality of T-connects disposed on said the lateral surface of said the stacked configuration,;
wherein at least two of said the plurality of channels comprise, include:
a converting circuit means for converting said configured to convert the detector output signal to into a digitized data value comprised of comprising one or more digital bits comprising including an image data set, wherein each digital bit has a position in said the image data set,;
a reference insert circuit means for inserting configured to insert one or more user-defined reference points at one or more predetermined positions in said the image data set,; and,
a reference point synchronization circuit means for synchronizingconfigured to synchronize at least one of saidthe user-defined reference points with at least one other of saidthe user-defined reference points in eachthe image data sets of at least two of said channels' image data setsthe plurality of channels.
2. The electronic circuit of claim 1, further comprising at least one FIFO a first-in, first out (FIFO) shift register for storing said configured to store the image data set.
3. The electronic circuit of claim 1, where said wherein the reference point synchronization means circuit is a field programmable field-programmable gate array.
4. The electronic circuit of claim 1, wherein said the at least two of the plurality of channels are each defined upon one integrated circuit chip.
5. The electronic circuit of claim 1, wherein each of said the at least two of the plurality channels are defined upon separate integrated circuit chips.
6. An electronic circuit comprised of comprising:
an electronic sensora photon detector array having an output,;
a plurality of readout electronics integrated circuit chips, each readout electronics integrated circuit chip comprising including a plurality of channels for receiving and processing configured to receive and process output signals generated by said the photon detector array, wherein said the plurality of readout electronics integrated circuit chips are arranged in a stacked configuration, with each of said the readout electronics integrated circuit chips forming one layer of said the stacked configuration, and wherein said the photon detector array is bonded to a lateral surface of said the stacked configuration perpendicular to said the stacked configuration and is connected to the plurality of channels via a plurality of T-connects disposed on said the lateral surface of said the stacked configuration,;
wherein at least two of said the layers comprise include:
sensor converting means for converting circuitry configured to convert the output of said the electronic sensor to into a digitized data value,;
a storage circuit means for the storing of said configured to store the digitized data value, said the storage circuit means comprising a plurality of individual bin circuit means circuits, each of said the bin circuit means circuits having a variable logic state and each having a storage circuit means position,;
a reference insert circuit for selectively asserting configured to selectively assert a predetermined bin logic state in said at least one a preselected bin circuit means, whereby a user-defined reference point may be selectively asserted at the respective storage circuit means positions on each of said the layers,;
an output circuit means for the outputting of said configured to output the digitized data value,; and,
a reference point synchronization circuit means for the receiving of and synchronizing of saidconfigured to receive and synchronize the outputted digitized data values using saidthe reference points for generatingto generate a plurality of synchronized digitized data values.
7. The electronic circuit of claim 6, wherein said the storage circuit means is comprised of comprises a FIFO first-in, first-out (FIFO) shift register.
8. The electronic circuit of claim 6, wherein said synchronizing the reference point synchronization circuit is performed in a field programmable field-programmable gate array.
9. An electronic circuit comprising:
a photon detector array configured to convert photons into a detector output signal; and
a processing module including a plurality of readout electronics integrated circuit chips (ROICs), wherein each ROIC comprises a plurality of channels configured to receive and process the detector output signal generated by the photon detector array, and wherein the plurality of channels are further configured to convert the detector output signal into an image data set including at least one digital bit;
wherein the plurality of ROICs are arranged in a stacked configuration with each ROIC forming a layer of the stacked configuration; and
wherein the photon detector array is bonded to a lateral surface of the stacked configuration substantially perpendicular to the stacked configuration and is connected to each channel of the plurality of channels via a respective T-connect disposed on the lateral surface of the stacked configuration.
10. The electronic circuit of claim 9, wherein a channel of the plurality of channels comprises a converting circuit configured to convert the detector output signal into an image data set including a digital bit, and wherein the digital bit has a position in the image data set.
11. The electronic circuit of claim 10, wherein a channel of the plurality of channels comprises:
a reference insert circuit configured to insert a user-defined reference points at a predetermined position in the image data set; and
a reference point synchronization circuit configured to synchronize a first user-defined reference point of a first image data set with a second user-defined reference point of a second image data set.
12. The electronic circuit of claim 11, wherein the reference point synchronization circuit is further configured to perform a histogram calculation based on at least two inserted user-defined reference points.
13. The electronic circuit of claim 11, wherein the reference point synchronization circuit is field-programmable gate array.
14. The electronic circuit of claim 9, further comprising metallized traces configured to electrically connect electronic contacts of the plurality of ROICs to access leads of the respective layers of the stacked configuration.
15. The electronic circuit of claim 9, wherein the respective electrical connection disposed on the lateral surface of the stacked configuration is a T-connect structure.
16. The electronic circuit of claim 9, further comprising comparator circuitry configured to determine if the detector output signal is above or below a predetermined threshold.
17. The electronic circuit of claim 9, further comprising a storage circuit configured to store the image data set.
18. The electronic circuit of claim 17, wherein the storage circuit comprises a first-in, first-out (FIFO) shift register.
19. The electronic circuit of claim 17, wherein the storage circuit comprises a plurality of bin circuits, each bin circuit having a variable logic state and a unique position in the storage circuit.
20. The electronic circuit of claim 19, further comprising a reference insert circuit configured to assert a predetermined bin logic state in at least one of the plurality of bin circuits and to insert a reference point into the image data set.
21. The electronic circuit of claim 20, further comprising an output circuit configured to output the image data set.
22. The electronic circuit of claim 21, further comprising a reference point synchronization circuit configured to receive and synchronize digitalized data values of the outputted image data set using the inserted reference points.
23. The electronic circuit of claim 9, wherein the plurality of channels are defined upon one integrated circuit chip.
24. The electronic circuit of claim 9, wherein a first subset of the plurality of channels are defined upon a first integrated circuit chip, and wherein a second subset of the plurality of channels are defined upon a second integrated circuit chip.
25. A method comprising:
converting received photons into a detector output signal using a photon detector array;
receiving and processing the detector output signal generated by the photon detector array at a first channel of a readout electronics integrated circuit (ROIC) of a processing module;
converting the output signal into an image data set at the first channel of the ROIC of the processing module, wherein the image data set includes a digital bit, wherein the ROIC is arranged in a stacked configuration with a plurality of other ROICs, and wherein the photon detector array is bonded to a lateral surface of the stacked configuration and is connected to each channel of a plurality of channels via a respective T-connect disposed on the lateral surface of the stacked configuration;
inserting a first user-defined reference point at a position in the image data set, wherein said inserting a first user-defined reference point includes asserting a predetermined bin logic state in a bit circuit of a storage circuit configured to store the image data set; and
synchronizing the first user-defined reference point with a second user-defined reference point associated with an integrated circuit layer of the processing module.
26. The method of claim 25, further comprising receiving photons reflected from a target.
27. The method of claim 25, further comprising performing a histogram calculation at a field-programmable gate array (FPGA) to determine positions of the first and second user-defined reference points and to define a realignment point for the image data set.
28. The method of claim 25, further comprising storing the image data set in the storage circuit.
29. The method of claim 28, wherein the storage circuit is a first-in, first-out (FIFO) shift register.
30. The method of claim 28, further comprising multiplexing the stored image data set to external processing circuitry.
31. The method of claim 25, further comprising determining via a comparator circuit if the detector output signal exceeds a predetermined threshold.
32. The method of claim 25, wherein said synchronizing comprises realigning the inserted first user-defined reference point with the second user-defined reference point to shift the image data set corresponding to the ROIC into alignment with another image data set corresponding to another ROIC of the plurality of other ROICs.
US12/607,253 2003-03-28 2009-10-28 Three-dimensional ladar module with alignment reference insert circuitry Expired - Lifetime USRE43722E1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12/607,253 USRE43722E1 (en) 2003-03-28 2009-10-28 Three-dimensional ladar module with alignment reference insert circuitry

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US46267703P 2003-03-28 2003-03-28
US10/806,037 US7180579B1 (en) 2003-03-28 2004-03-22 Three-dimensional imaging processing module incorporating stacked layers containing microelectronic circuits
US78513506P 2006-03-24 2006-03-24
US11/706,724 US7436494B1 (en) 2003-03-28 2007-02-15 Three-dimensional ladar module with alignment reference insert circuitry
US12/607,253 USRE43722E1 (en) 2003-03-28 2009-10-28 Three-dimensional ladar module with alignment reference insert circuitry

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US11/706,724 Reissue US7436494B1 (en) 2003-03-28 2007-02-15 Three-dimensional ladar module with alignment reference insert circuitry

Publications (1)

Publication Number Publication Date
USRE43722E1 true USRE43722E1 (en) 2012-10-09

Family

ID=39828322

Family Applications (2)

Application Number Title Priority Date Filing Date
US11/706,724 Ceased US7436494B1 (en) 2003-03-28 2007-02-15 Three-dimensional ladar module with alignment reference insert circuitry
US12/607,253 Expired - Lifetime USRE43722E1 (en) 2003-03-28 2009-10-28 Three-dimensional ladar module with alignment reference insert circuitry

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US11/706,724 Ceased US7436494B1 (en) 2003-03-28 2007-02-15 Three-dimensional ladar module with alignment reference insert circuitry

Country Status (1)

Country Link
US (2) US7436494B1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130230206A1 (en) * 2012-03-01 2013-09-05 Exelis, Inc. Foliage penetration based on 4d lidar datasets
US9165383B1 (en) 2011-11-21 2015-10-20 Exelis, Inc. Point cloud visualization using bi-modal color schemes based on 4D lidar datasets
US11933967B2 (en) 2019-08-22 2024-03-19 Red Creamery, LLC Distally actuated scanning mirror

Families Citing this family (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8119971B2 (en) 2008-01-17 2012-02-21 Ball Corporation Pulse data recorder in which a value held by a bit of a memory is determined by a state of a switch
US8077294B1 (en) 2008-01-17 2011-12-13 Ball Aerospace & Technologies Corp. Optical autocovariance lidar
US7961301B2 (en) * 2008-05-09 2011-06-14 Ball Aerospace & Technologies Corp. Flash LADAR system
US9041915B2 (en) 2008-05-09 2015-05-26 Ball Aerospace & Technologies Corp. Systems and methods of scene and action capture using imaging system incorporating 3D LIDAR
US9559987B1 (en) * 2008-09-26 2017-01-31 Tellabs Operations, Inc Method and apparatus for improving CAM learn throughput using a cache
US7929215B1 (en) 2009-02-20 2011-04-19 Ball Aerospace & Technologies Corp. Field widening lens
US8279420B2 (en) * 2009-08-06 2012-10-02 ISC8 Inc. Phase sensing and scanning time of flight LADAR using atmospheric absorption bands
US8125367B2 (en) * 2009-08-06 2012-02-28 Irvine Sensors Corp. AM chirp LADAR readout circuit and module
US20110084212A1 (en) * 2009-09-22 2011-04-14 Irvine Sensors Corporation Multi-layer photon counting electronic module
US20110285981A1 (en) * 2010-05-18 2011-11-24 Irvine Sensors Corporation Sensor Element and System Comprising Wide Field-of-View 3-D Imaging LIDAR
US8306273B1 (en) 2009-12-28 2012-11-06 Ball Aerospace & Technologies Corp. Method and apparatus for LIDAR target identification and pose estimation
US8736818B2 (en) 2010-08-16 2014-05-27 Ball Aerospace & Technologies Corp. Electronically steered flash LIDAR
RU2480712C2 (en) * 2011-04-28 2013-04-27 Общество С Ограниченной Ответственностью "Научно-Техническое Объединение "Ирэ-Полюс" Optical system for multifrequency laser location and method for implementation thereof
US8744126B1 (en) 2012-03-07 2014-06-03 Ball Aerospace & Technologies Corp. Morphology based hazard detection
US20130250272A1 (en) * 2012-03-21 2013-09-26 David Ludwig Threshold Detection Method, Module and Readout Integrated Circuit Layer for LIDAR Time of Flight System Using Differentiated Gaussian Signal
US20130250273A1 (en) * 2012-03-21 2013-09-26 ISC8 Inc. Threshold Detection Method and Device for LIDAR Time of Flight System Using Differentiated Gaussian Signal
US9297889B2 (en) * 2012-08-14 2016-03-29 Microsoft Technology Licensing, Llc Illumination light projection for a depth camera
US10203399B2 (en) 2013-11-12 2019-02-12 Big Sky Financial Corporation Methods and apparatus for array based LiDAR systems with reduced interference
US9360554B2 (en) 2014-04-11 2016-06-07 Facet Technology Corp. Methods and apparatus for object detection and identification in a multiple detector lidar array
US10386464B2 (en) 2014-08-15 2019-08-20 Aeye, Inc. Ladar point cloud compression
US10036801B2 (en) 2015-03-05 2018-07-31 Big Sky Financial Corporation Methods and apparatus for increased precision and improved range in a multiple detector LiDAR array
US10105040B2 (en) * 2015-05-08 2018-10-23 Nanosurgery Technology Corporation Imaging needle apparatus
WO2017058901A1 (en) 2015-09-28 2017-04-06 Ball Aerospace & Technologies Corp. Differential absorption lidar
US9933513B2 (en) 2016-02-18 2018-04-03 Aeye, Inc. Method and apparatus for an adaptive ladar receiver
US20170242104A1 (en) 2016-02-18 2017-08-24 Aeye, Inc. Ladar Transmitter with Induced Phase Drift for Improved Gaze on Scan Area Portions
US10761196B2 (en) * 2016-02-18 2020-09-01 Aeye, Inc. Adaptive ladar receiving method
US10042159B2 (en) 2016-02-18 2018-08-07 Aeye, Inc. Ladar transmitter with optical field splitter/inverter
US9866816B2 (en) 2016-03-03 2018-01-09 4D Intellectual Properties, Llc Methods and apparatus for an active pulsed 4D camera for image acquisition and analysis
JP7206206B2 (en) 2017-02-17 2023-01-17 エイアイ インコーポレイテッド LADAR pulse interference avoidance method and system
CN111344647A (en) 2017-09-15 2020-06-26 艾耶股份有限公司 Intelligent laser radar system with low-latency motion planning update
US10921245B2 (en) 2018-06-08 2021-02-16 Ball Aerospace & Technologies Corp. Method and systems for remote emission detection and rate determination
US10656252B1 (en) 2018-10-25 2020-05-19 Aeye, Inc. Adaptive control of Ladar systems using spatial index of prior Ladar return data
US10656272B1 (en) 2019-04-24 2020-05-19 Aeye, Inc. Ladar system and method with polarized receivers
US11630188B1 (en) 2021-03-26 2023-04-18 Aeye, Inc. Hyper temporal lidar with dynamic laser control using safety models
US20230044929A1 (en) 2021-03-26 2023-02-09 Aeye, Inc. Multi-Lens Lidar Receiver with Multiple Readout Channels
US11619740B2 (en) 2021-03-26 2023-04-04 Aeye, Inc. Hyper temporal lidar with asynchronous shot intervals and detection intervals
US20220308187A1 (en) 2021-03-26 2022-09-29 Aeye, Inc. Hyper Temporal Lidar Using Multiple Matched Filters to Determine Target Retro-Reflectivity
US11474213B1 (en) 2021-03-26 2022-10-18 Aeye, Inc. Hyper temporal lidar with dynamic laser control using marker shots
US11486977B2 (en) 2021-03-26 2022-11-01 Aeye, Inc. Hyper temporal lidar with pulse burst scheduling
US11635495B1 (en) 2021-03-26 2023-04-25 Aeye, Inc. Hyper temporal lidar with controllable tilt amplitude for a variable amplitude scan mirror

Citations (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US644968A (en) 1899-06-26 1900-03-06 Holman Crawford Wheel.
US3897150A (en) 1972-04-03 1975-07-29 Hughes Aircraft Co Scanned laser imaging and ranging system
US4574331A (en) 1983-05-31 1986-03-04 Trw Inc. Multi-element circuit construction
US4659931A (en) 1985-05-08 1987-04-21 Grumman Aerospace Corporation High density multi-layered integrated circuit package
US5198657A (en) 1992-02-05 1993-03-30 General Atomics Integrated imaging and ranging lidar receiver
US5214274A (en) 1992-07-24 1993-05-25 President And Fellows Of Harvard College Image sensor array with threshold voltage detectors and charged storage capacitors
US5262837A (en) 1992-10-21 1993-11-16 Norm Pacific Automation Corp. Laser range finder
US5274430A (en) 1991-09-09 1993-12-28 Dornier Luftfahrt Gmbh Process and a system for taking distance images
US5426566A (en) 1991-09-30 1995-06-20 International Business Machines Corporation Multichip integrated circuit packages and systems
US5608514A (en) 1995-04-19 1997-03-04 The United States Of America As Represented By The Secretary Of The Army High range resolution ladar
US5682229A (en) 1995-04-14 1997-10-28 Schwartz Electro-Optics, Inc. Laser range camera
US5875158A (en) 1995-01-25 1999-02-23 Discovision Associates Servo control system for information storage device
US5925924A (en) 1995-07-26 1999-07-20 International Business Machines Corporation Methods for precise definition of integrated circuit chip edges
US5953110A (en) 1998-04-23 1999-09-14 H.N. Burns Engineering Corporation Multichannel laser radar
US5953588A (en) 1996-12-21 1999-09-14 Irvine Sensors Corporation Stackable layers containing encapsulated IC chips
JPH11355813A (en) 1998-06-04 1999-12-24 Honda Motor Co Ltd Device for deciding internal parameters of camera
US6031601A (en) 1998-07-08 2000-02-29 Trimble Navigation Limited Code-space optical electronic distance meter
US6042388A (en) 1999-01-19 2000-03-28 Unisys Corporation Electromechanical module having a thin springy plate for establishing pressed electrical connections
US6133989A (en) 1993-02-09 2000-10-17 Advanced Scientific Concepts, Inc. 3D imaging laser radar
US6246468B1 (en) 1996-04-24 2001-06-12 Cyra Technologies Integrated system for quickly and accurately imaging and modeling three-dimensional objects
US6264476B1 (en) 1999-12-09 2001-07-24 High Connection Density, Inc. Wire segment based interposer for high frequency electrical connection
US6384519B1 (en) * 1996-10-30 2002-05-07 Nanosciences Corporation Micro-dynode integrated electron multiplier
US6414746B1 (en) 1999-11-24 2002-07-02 Advanced Scientific Concepts, Inc. 3-D imaging multiple target laser radar
US6444968B1 (en) 1997-03-22 2002-09-03 Eev Ltd CCD imager with separate charge multiplication elements
US20020171133A1 (en) 2000-06-20 2002-11-21 Sammy Mok Systems for testing and packaging integrated circuits
US6522396B1 (en) 2002-01-08 2003-02-18 Raytheon Company Dual mode adaptive threshold architecture for 3-D ladar FPA
US6650404B1 (en) * 2002-05-28 2003-11-18 Analog Modules, Inc. Laser rangefinder receiver
US20030221313A1 (en) 2001-01-26 2003-12-04 Gann Keith D. Method for making stacked integrated circuits (ICs) using prepackaged parts
US6849388B2 (en) 2002-10-22 2005-02-01 Northrop Grumman Corporation Self-aligning contacts for stacked electronics
US7098999B1 (en) 2004-03-05 2006-08-29 Raytheon Company High-speed readout circuit and system incorporating same
US7199872B2 (en) * 2004-05-18 2007-04-03 Leica Geosystems Ag Method and apparatus for ground-based surveying in sites having one or more unstable zone(s)
US7265993B1 (en) 2005-05-16 2007-09-04 Sun Microsystems, Inc. Dispersive interconnect system for EMI reduction
US7647199B2 (en) 2007-09-24 2010-01-12 Leica Geosystems Ag Method for determining positions of points to be measured
US7895015B2 (en) 2008-12-04 2011-02-22 Parker David H Method for measuring the structural health of a civil structure
US7941759B2 (en) 2007-09-27 2011-05-10 Trimble Navigation Limited Interactive analysis of network adjustment results

Patent Citations (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US644968A (en) 1899-06-26 1900-03-06 Holman Crawford Wheel.
US3897150A (en) 1972-04-03 1975-07-29 Hughes Aircraft Co Scanned laser imaging and ranging system
US4574331A (en) 1983-05-31 1986-03-04 Trw Inc. Multi-element circuit construction
US4659931A (en) 1985-05-08 1987-04-21 Grumman Aerospace Corporation High density multi-layered integrated circuit package
US5274430A (en) 1991-09-09 1993-12-28 Dornier Luftfahrt Gmbh Process and a system for taking distance images
US5426566A (en) 1991-09-30 1995-06-20 International Business Machines Corporation Multichip integrated circuit packages and systems
US5198657A (en) 1992-02-05 1993-03-30 General Atomics Integrated imaging and ranging lidar receiver
US5214274A (en) 1992-07-24 1993-05-25 President And Fellows Of Harvard College Image sensor array with threshold voltage detectors and charged storage capacitors
US5262837A (en) 1992-10-21 1993-11-16 Norm Pacific Automation Corp. Laser range finder
US6133989A (en) 1993-02-09 2000-10-17 Advanced Scientific Concepts, Inc. 3D imaging laser radar
US5875158A (en) 1995-01-25 1999-02-23 Discovision Associates Servo control system for information storage device
US5682229A (en) 1995-04-14 1997-10-28 Schwartz Electro-Optics, Inc. Laser range camera
US5608514A (en) 1995-04-19 1997-03-04 The United States Of America As Represented By The Secretary Of The Army High range resolution ladar
US5925924A (en) 1995-07-26 1999-07-20 International Business Machines Corporation Methods for precise definition of integrated circuit chip edges
US6246468B1 (en) 1996-04-24 2001-06-12 Cyra Technologies Integrated system for quickly and accurately imaging and modeling three-dimensional objects
US6384519B1 (en) * 1996-10-30 2002-05-07 Nanosciences Corporation Micro-dynode integrated electron multiplier
US5953588A (en) 1996-12-21 1999-09-14 Irvine Sensors Corporation Stackable layers containing encapsulated IC chips
US6444968B1 (en) 1997-03-22 2002-09-03 Eev Ltd CCD imager with separate charge multiplication elements
US5953110A (en) 1998-04-23 1999-09-14 H.N. Burns Engineering Corporation Multichannel laser radar
JPH11355813A (en) 1998-06-04 1999-12-24 Honda Motor Co Ltd Device for deciding internal parameters of camera
US6031601A (en) 1998-07-08 2000-02-29 Trimble Navigation Limited Code-space optical electronic distance meter
US6042388A (en) 1999-01-19 2000-03-28 Unisys Corporation Electromechanical module having a thin springy plate for establishing pressed electrical connections
US6414746B1 (en) 1999-11-24 2002-07-02 Advanced Scientific Concepts, Inc. 3-D imaging multiple target laser radar
US6264476B1 (en) 1999-12-09 2001-07-24 High Connection Density, Inc. Wire segment based interposer for high frequency electrical connection
US6791171B2 (en) 2000-06-20 2004-09-14 Nanonexus, Inc. Systems for testing and packaging integrated circuits
US20020171133A1 (en) 2000-06-20 2002-11-21 Sammy Mok Systems for testing and packaging integrated circuits
US20030221313A1 (en) 2001-01-26 2003-12-04 Gann Keith D. Method for making stacked integrated circuits (ICs) using prepackaged parts
US6522396B1 (en) 2002-01-08 2003-02-18 Raytheon Company Dual mode adaptive threshold architecture for 3-D ladar FPA
US6650404B1 (en) * 2002-05-28 2003-11-18 Analog Modules, Inc. Laser rangefinder receiver
US6849388B2 (en) 2002-10-22 2005-02-01 Northrop Grumman Corporation Self-aligning contacts for stacked electronics
US7098999B1 (en) 2004-03-05 2006-08-29 Raytheon Company High-speed readout circuit and system incorporating same
US7199872B2 (en) * 2004-05-18 2007-04-03 Leica Geosystems Ag Method and apparatus for ground-based surveying in sites having one or more unstable zone(s)
US7265993B1 (en) 2005-05-16 2007-09-04 Sun Microsystems, Inc. Dispersive interconnect system for EMI reduction
US7647199B2 (en) 2007-09-24 2010-01-12 Leica Geosystems Ag Method for determining positions of points to be measured
US7941759B2 (en) 2007-09-27 2011-05-10 Trimble Navigation Limited Interactive analysis of network adjustment results
US7895015B2 (en) 2008-12-04 2011-02-22 Parker David H Method for measuring the structural health of a civil structure

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Non-final Office Action issued in U.S. Appl. No. 12/287,691 and dated Jun. 28, 2011.
Non-Final Office Action mailed on Apr. 15, 2009 in U.S. Appl. No. 11/499,403.
Notice of Allowance issued in U.S. Appl. No. 12/287,691 and mailed Jan. 11, 2012.

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9165383B1 (en) 2011-11-21 2015-10-20 Exelis, Inc. Point cloud visualization using bi-modal color schemes based on 4D lidar datasets
US20130230206A1 (en) * 2012-03-01 2013-09-05 Exelis, Inc. Foliage penetration based on 4d lidar datasets
US8670591B2 (en) * 2012-03-01 2014-03-11 Exelis, Inc. Foliage penetration based on 4D LIDAR datasets
US11933967B2 (en) 2019-08-22 2024-03-19 Red Creamery, LLC Distally actuated scanning mirror

Also Published As

Publication number Publication date
US7436494B1 (en) 2008-10-14

Similar Documents

Publication Publication Date Title
USRE43722E1 (en) Three-dimensional ladar module with alignment reference insert circuitry
US8198576B2 (en) Three-dimensional LADAR module with alignment reference insert circuitry comprising high density interconnect structure
US6414746B1 (en) 3-D imaging multiple target laser radar
US7180579B1 (en) Three-dimensional imaging processing module incorporating stacked layers containing microelectronic circuits
Kumagai et al. 7.3 A 189× 600 back-illuminated stacked SPAD direct time-of-flight depth sensor for automotive LiDAR systems
US5696577A (en) 3D imaging underwater laser radar
US10401147B2 (en) Dimensioning system
TWI544232B (en) Calibration circuitry and method for a time of flight imaging system
US20130250273A1 (en) Threshold Detection Method and Device for LIDAR Time of Flight System Using Differentiated Gaussian Signal
CN111758047B (en) Single chip RGB-D camera
EP2758801B1 (en) Time of flight sensor with subframe compression and method
US20130107243A1 (en) Fast, High Resolution 3-D Flash LADAR Imager
US8829408B2 (en) Sensor pixel array and separated array of storage and accumulation with parallel acquisition and readout wherein each pixel includes storage sites and readout nodes
Elkhalili et al. A 4/spl times/64 pixel CMOS image sensor for 3-D measurement applications
US20130250272A1 (en) Threshold Detection Method, Module and Readout Integrated Circuit Layer for LIDAR Time of Flight System Using Differentiated Gaussian Signal
Gyongy et al. Direct time-of-flight single-photon imaging
Keränen et al. $256\times8 $ SPAD Array With 256 Column TDCs for a Line Profiling Laser Radar
US20040188596A1 (en) Three-dimensional imaging device incorporating stacked layers containing microelectronic circuits
US10444330B2 (en) LADAR receiver with enhanced signal to noise ratio and method
EP1637903A1 (en) Three-dimensional imaging processing module incorporating stacked layers containing microelectronic circuits
JP2006098190A (en) Photon detector module and imaging device
US20240121535A1 (en) An image sensing device
US20230236297A1 (en) Systems and Methods for High Precision Direct Time-of-Flight Lidar in the Presence of Strong Pile-Up
WO2022016448A1 (en) Indirect tof sensor, stacked sensor chip, and method for measuring distance to object using the same
Stettner et al. Staring underwater laser radar (SULAR) 3D imaging

Legal Events

Date Code Title Description
AS Assignment

Owner name: APROLASE DEVELOPMENT CO., LLC, DELAWARE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:IRVINE SENSORS CORPORATION;REEL/FRAME:024183/0800

Effective date: 20090316

CC Certificate of correction
AS Assignment

Owner name: NYTELL SOFTWARE LLC, DELAWARE

Free format text: MERGER;ASSIGNOR:APROLASE DEVELOPMENT CO., LLC;REEL/FRAME:037406/0200

Effective date: 20150826

FPAY Fee payment

Year of fee payment: 8

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 12

AS Assignment

Owner name: INTELLECTUAL VENTURES ASSETS 190 LLC, DELAWARE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NYTELL SOFTWARE LLC;REEL/FRAME:065015/0690

Effective date: 20230925

AS Assignment

Owner name: AI-CORE TECHNOLOGIES, LLC, TEXAS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTELLECTUAL VENTURES ASSETS 190 LLC;REEL/FRAME:065227/0107

Effective date: 20231005

AS Assignment

Owner name: LONGHORN AUTOMOTIVE GROUP LLC, TEXAS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AI-CORE TECHNOLOGIES, LLC;REEL/FRAME:066911/0261

Effective date: 20231006