WO2014007869A9 - Methods and apparatus for coaxial imaging of multiple wavelengths - Google Patents
Methods and apparatus for coaxial imaging of multiple wavelengths Download PDFInfo
- Publication number
- WO2014007869A9 WO2014007869A9 PCT/US2013/032682 US2013032682W WO2014007869A9 WO 2014007869 A9 WO2014007869 A9 WO 2014007869A9 US 2013032682 W US2013032682 W US 2013032682W WO 2014007869 A9 WO2014007869 A9 WO 2014007869A9
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- hyperspectral
- optical
- imaging device
- multispectral imaging
- detector
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims description 142
- 238000003384 imaging method Methods 0.000 title claims description 58
- 230000003287 optical effect Effects 0.000 claims abstract description 546
- 238000004891 communication Methods 0.000 claims abstract description 225
- 238000001914 filtration Methods 0.000 claims abstract description 21
- 238000000701 chemical imaging Methods 0.000 claims description 415
- 230000003595 spectral effect Effects 0.000 claims description 121
- 238000005286 illumination Methods 0.000 claims description 118
- 238000012545 processing Methods 0.000 claims description 47
- 230000005855 radiation Effects 0.000 claims description 45
- 238000003860 storage Methods 0.000 claims description 31
- XUMBMVFBXHLACL-UHFFFAOYSA-N Melanin Chemical compound O=C1C(=O)C(C2=CNC3=C(C(C(=O)C4=C32)=O)C)=C2C4=CNC2=C1C XUMBMVFBXHLACL-UHFFFAOYSA-N 0.000 claims description 24
- 230000001427 coherent effect Effects 0.000 claims description 20
- 208000025865 Ulcer Diseases 0.000 claims description 18
- 231100000397 ulcer Toxicity 0.000 claims description 18
- 239000004065 semiconductor Substances 0.000 claims description 16
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 claims description 15
- 229910052760 oxygen Inorganic materials 0.000 claims description 15
- 239000001301 oxygen Substances 0.000 claims description 15
- 230000035939 shock Effects 0.000 claims description 14
- INGWEZCOABYORO-UHFFFAOYSA-N 2-(furan-2-yl)-7-methyl-1h-1,8-naphthyridin-4-one Chemical compound N=1C2=NC(C)=CC=C2C(O)=CC=1C1=CC=CO1 INGWEZCOABYORO-UHFFFAOYSA-N 0.000 claims description 13
- 108010002255 deoxyhemoglobin Proteins 0.000 claims description 13
- 230000015572 biosynthetic process Effects 0.000 claims description 12
- 206010012601 diabetes mellitus Diseases 0.000 claims description 11
- 230000005540 biological transmission Effects 0.000 claims description 10
- 230000001351 cycling effect Effects 0.000 claims description 10
- 108010064719 Oxyhemoglobins Proteins 0.000 claims description 9
- 108010054147 Hemoglobins Proteins 0.000 claims description 8
- 102000001554 Hemoglobins Human genes 0.000 claims description 8
- 230000000295 complement effect Effects 0.000 claims description 8
- 208000028867 ischemia Diseases 0.000 claims description 7
- 238000005516 engineering process Methods 0.000 claims description 6
- 230000010412 perfusion Effects 0.000 claims description 6
- 230000035945 sensitivity Effects 0.000 claims description 6
- 102000008186 Collagen Human genes 0.000 claims description 5
- 108010035532 Collagen Proteins 0.000 claims description 5
- 229920001436 collagen Polymers 0.000 claims description 5
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 5
- 230000036571 hydration Effects 0.000 claims description 5
- 238000006703 hydration reaction Methods 0.000 claims description 5
- 230000000717 retained effect Effects 0.000 claims description 5
- 230000001131 transforming effect Effects 0.000 claims description 5
- 238000003491 array Methods 0.000 claims description 4
- 239000013043 chemical agent Substances 0.000 claims description 4
- 238000005534 hematocrit Methods 0.000 claims description 4
- 208000015181 infectious disease Diseases 0.000 claims description 4
- 238000003032 molecular docking Methods 0.000 claims description 4
- 230000008054 signal transmission Effects 0.000 claims description 4
- 206010007559 Cardiac failure congestive Diseases 0.000 claims description 3
- 208000005156 Dehydration Diseases 0.000 claims description 3
- 206010019280 Heart failures Diseases 0.000 claims description 3
- 208000032843 Hemorrhage Diseases 0.000 claims description 3
- 206010020772 Hypertension Diseases 0.000 claims description 3
- 206010021137 Hypovolaemia Diseases 0.000 claims description 3
- 208000004756 Respiratory Insufficiency Diseases 0.000 claims description 3
- 206010040047 Sepsis Diseases 0.000 claims description 3
- 208000000558 Varicose Ulcer Diseases 0.000 claims description 3
- 239000003124 biologic agent Substances 0.000 claims description 3
- 230000000747 cardiac effect Effects 0.000 claims description 3
- 230000018044 dehydration Effects 0.000 claims description 3
- 238000006297 dehydration reaction Methods 0.000 claims description 3
- 230000028709 inflammatory response Effects 0.000 claims description 3
- 201000004193 respiratory failure Diseases 0.000 claims description 3
- 201000002282 venous insufficiency Diseases 0.000 claims description 3
- 210000001519 tissue Anatomy 0.000 description 77
- 210000003491 skin Anatomy 0.000 description 59
- 239000000306 component Substances 0.000 description 44
- 238000004458 analytical method Methods 0.000 description 31
- 238000001228 spectrum Methods 0.000 description 26
- 210000002615 epidermis Anatomy 0.000 description 21
- 238000004422 calculation algorithm Methods 0.000 description 15
- 239000010410 layer Substances 0.000 description 15
- 210000004207 dermis Anatomy 0.000 description 13
- 230000006870 function Effects 0.000 description 13
- 238000001514 detection method Methods 0.000 description 12
- 238000005259 measurement Methods 0.000 description 12
- 238000011282 treatment Methods 0.000 description 12
- 238000007405 data analysis Methods 0.000 description 11
- 238000010521 absorption reaction Methods 0.000 description 9
- 206010028980 Neoplasm Diseases 0.000 description 8
- 238000003745 diagnosis Methods 0.000 description 8
- 201000001441 melanoma Diseases 0.000 description 8
- 230000037311 normal skin Effects 0.000 description 8
- 238000002059 diagnostic imaging Methods 0.000 description 7
- 230000010287 polarization Effects 0.000 description 7
- 239000000126 substance Substances 0.000 description 7
- 206010004146 Basal cell carcinoma Diseases 0.000 description 6
- 208000008960 Diabetic foot Diseases 0.000 description 6
- 230000008901 benefit Effects 0.000 description 6
- 201000011510 cancer Diseases 0.000 description 6
- 238000011156 evaluation Methods 0.000 description 6
- 230000003902 lesion Effects 0.000 description 6
- 238000004611 spectroscopical analysis Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 238000012512 characterization method Methods 0.000 description 5
- 238000004590 computer program Methods 0.000 description 5
- 230000003247 decreasing effect Effects 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 5
- 238000004393 prognosis Methods 0.000 description 5
- 238000002834 transmittance Methods 0.000 description 5
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 5
- 208000004210 Pressure Ulcer Diseases 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 4
- 210000004027 cell Anatomy 0.000 description 4
- 230000001413 cellular effect Effects 0.000 description 4
- 230000001419 dependent effect Effects 0.000 description 4
- 239000000835 fiber Substances 0.000 description 4
- 239000000203 mixture Substances 0.000 description 4
- 230000002829 reductive effect Effects 0.000 description 4
- 206010041823 squamous cell carcinoma Diseases 0.000 description 4
- 238000001356 surgical procedure Methods 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 208000024780 Urticaria Diseases 0.000 description 3
- 239000000654 additive Substances 0.000 description 3
- 230000000996 additive effect Effects 0.000 description 3
- 150000001875 compounds Chemical class 0.000 description 3
- 208000035250 cutaneous malignant susceptibility to 1 melanoma Diseases 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000005670 electromagnetic radiation Effects 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 230000036541 health Effects 0.000 description 3
- 229910052500 inorganic mineral Inorganic materials 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000003211 malignant effect Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 239000011707 mineral Substances 0.000 description 3
- 239000003921 oil Substances 0.000 description 3
- 210000005166 vasculature Anatomy 0.000 description 3
- 230000035899 viability Effects 0.000 description 3
- BPYKTIZUTYGOLE-IFADSCNNSA-N Bilirubin Chemical compound N1C(=O)C(C)=C(C=C)\C1=C\C1=C(C)C(CCC(O)=O)=C(CC2=C(C(C)=C(\C=C/3C(=C(C=C)C(=O)N\3)C)N2)CCC(O)=O)N1 BPYKTIZUTYGOLE-IFADSCNNSA-N 0.000 description 2
- 241000196324 Embryophyta Species 0.000 description 2
- 208000007985 Erythema Infectiosum Diseases 0.000 description 2
- 208000035874 Excoriation Diseases 0.000 description 2
- 208000032456 Hemorrhagic Shock Diseases 0.000 description 2
- 206010024648 Livedo reticularis Diseases 0.000 description 2
- 208000035346 Margins of Excision Diseases 0.000 description 2
- 206010027145 Melanocytic naevus Diseases 0.000 description 2
- 206010049771 Shock haemorrhagic Diseases 0.000 description 2
- 208000000453 Skin Neoplasms Diseases 0.000 description 2
- 208000027418 Wounds and injury Diseases 0.000 description 2
- 210000000270 basal cell Anatomy 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- 210000004369 blood Anatomy 0.000 description 2
- 239000008280 blood Substances 0.000 description 2
- 239000012503 blood component Substances 0.000 description 2
- 210000001124 body fluid Anatomy 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 239000010408 film Substances 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 210000004209 hair Anatomy 0.000 description 2
- 208000014674 injury Diseases 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000000302 ischemic effect Effects 0.000 description 2
- 238000007477 logistic regression Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000007620 mathematical function Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 210000002752 melanocyte Anatomy 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 239000000049 pigment Substances 0.000 description 2
- 239000004033 plastic Substances 0.000 description 2
- 238000000513 principal component analysis Methods 0.000 description 2
- 239000002689 soil Substances 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 210000000434 stratum corneum Anatomy 0.000 description 2
- 230000004083 survival effect Effects 0.000 description 2
- 239000003440 toxic substance Substances 0.000 description 2
- 238000001429 visible spectrum Methods 0.000 description 2
- OZFAFGSSMRRTDW-UHFFFAOYSA-N (2,4-dichlorophenyl) benzenesulfonate Chemical compound ClC1=CC(Cl)=CC=C1OS(=O)(=O)C1=CC=CC=C1 OZFAFGSSMRRTDW-UHFFFAOYSA-N 0.000 description 1
- 241000502561 Acacia irrorata Species 0.000 description 1
- 206010063409 Acarodermatitis Diseases 0.000 description 1
- 201000004384 Alopecia Diseases 0.000 description 1
- 206010002199 Anaphylactic shock Diseases 0.000 description 1
- 206010003694 Atrophy Diseases 0.000 description 1
- 238000012935 Averaging Methods 0.000 description 1
- 208000001054 Basilar Skull Fracture Diseases 0.000 description 1
- 206010005913 Body tinea Diseases 0.000 description 1
- 229910001369 Brass Inorganic materials 0.000 description 1
- 101100421200 Caenorhabditis elegans sep-1 gene Proteins 0.000 description 1
- 208000003732 Cat-scratch disease Diseases 0.000 description 1
- 208000032064 Chronic Limb-Threatening Ischemia Diseases 0.000 description 1
- 206010059547 Cutaneous larva migrans Diseases 0.000 description 1
- 201000004624 Dermatitis Diseases 0.000 description 1
- 206010012442 Dermatitis contact Diseases 0.000 description 1
- 208000008334 Dermatofibrosarcoma Diseases 0.000 description 1
- 206010057070 Dermatofibrosarcoma protuberans Diseases 0.000 description 1
- 206010056340 Diabetic ulcer Diseases 0.000 description 1
- 206010061818 Disease progression Diseases 0.000 description 1
- 206010058314 Dysplasia Diseases 0.000 description 1
- 206010014080 Ecchymosis Diseases 0.000 description 1
- 206010014970 Ephelides Diseases 0.000 description 1
- 206010015218 Erythema multiforme Diseases 0.000 description 1
- 206010051814 Eschar Diseases 0.000 description 1
- 206010016936 Folliculitis Diseases 0.000 description 1
- 208000007212 Foot-and-Mouth Disease Diseases 0.000 description 1
- 241000710198 Foot-and-mouth disease virus Species 0.000 description 1
- 229910000530 Gallium indium arsenide Inorganic materials 0.000 description 1
- 201000004331 Henoch-Schoenlein purpura Diseases 0.000 description 1
- 206010019617 Henoch-Schonlein purpura Diseases 0.000 description 1
- 208000009889 Herpes Simplex Diseases 0.000 description 1
- 208000007514 Herpes zoster Diseases 0.000 description 1
- 208000001953 Hypotension Diseases 0.000 description 1
- 206010021138 Hypovolaemic shock Diseases 0.000 description 1
- 208000031814 IgA Vasculitis Diseases 0.000 description 1
- 206010021531 Impetigo Diseases 0.000 description 1
- 206010061218 Inflammation Diseases 0.000 description 1
- 208000006877 Insect Bites and Stings Diseases 0.000 description 1
- 208000003456 Juvenile Arthritis Diseases 0.000 description 1
- 206010059176 Juvenile idiopathic arthritis Diseases 0.000 description 1
- 206010023256 Juvenile melanoma benign Diseases 0.000 description 1
- 208000007766 Kaposi sarcoma Diseases 0.000 description 1
- 208000011200 Kawasaki disease Diseases 0.000 description 1
- 208000002260 Keloid Diseases 0.000 description 1
- 206010066295 Keratosis pilaris Diseases 0.000 description 1
- 206010048649 Koebner phenomenon Diseases 0.000 description 1
- 201000005099 Langerhans cell histiocytosis Diseases 0.000 description 1
- 208000004204 Larva Migrans Diseases 0.000 description 1
- 206010066945 Lichen striatus Diseases 0.000 description 1
- 206010024438 Lichenification Diseases 0.000 description 1
- 201000005505 Measles Diseases 0.000 description 1
- 206010058858 Meningococcal bacteraemia Diseases 0.000 description 1
- 229910000661 Mercury cadmium telluride Inorganic materials 0.000 description 1
- 208000002030 Merkel cell carcinoma Diseases 0.000 description 1
- 241000565667 Mesodon inflectus Species 0.000 description 1
- 206010027476 Metastases Diseases 0.000 description 1
- 206010029266 Neuroendocrine carcinoma of the skin Diseases 0.000 description 1
- 208000009905 Neurofibromatoses Diseases 0.000 description 1
- 206010058119 Neurogenic shock Diseases 0.000 description 1
- 208000007256 Nevus Diseases 0.000 description 1
- 208000032452 Nevus, Epithelioid and Spindle Cell Diseases 0.000 description 1
- 206010034576 Peripheral ischaemia Diseases 0.000 description 1
- 208000018262 Peripheral vascular disease Diseases 0.000 description 1
- 208000009077 Pigmented Nevus Diseases 0.000 description 1
- 229920001609 Poly(3,4-ethylenedioxythiophene) Polymers 0.000 description 1
- 208000003251 Pruritus Diseases 0.000 description 1
- 201000004681 Psoriasis Diseases 0.000 description 1
- 241000447727 Scabies Species 0.000 description 1
- 206010039587 Scarlet Fever Diseases 0.000 description 1
- 206010039793 Seborrhoeic dermatitis Diseases 0.000 description 1
- 206010039796 Seborrhoeic keratosis Diseases 0.000 description 1
- 206010040070 Septic Shock Diseases 0.000 description 1
- 206010042033 Stevens-Johnson syndrome Diseases 0.000 description 1
- 231100000168 Stevens-Johnson syndrome Toxicity 0.000 description 1
- 206010051495 Strawberry tongue Diseases 0.000 description 1
- 206010043189 Telangiectasia Diseases 0.000 description 1
- 208000002474 Tinea Diseases 0.000 description 1
- 206010043866 Tinea capitis Diseases 0.000 description 1
- 208000026911 Tuberous sclerosis complex Diseases 0.000 description 1
- 108010067390 Viral Proteins Proteins 0.000 description 1
- 238000001793 Wilcoxon signed-rank test Methods 0.000 description 1
- 206010048214 Xanthoma Diseases 0.000 description 1
- 206010048215 Xanthomatosis Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000005299 abrasion Methods 0.000 description 1
- 238000002835 absorbance Methods 0.000 description 1
- 239000006096 absorbing agent Substances 0.000 description 1
- 231100000360 alopecia Toxicity 0.000 description 1
- 238000002266 amputation Methods 0.000 description 1
- 238000000540 analysis of variance Methods 0.000 description 1
- 208000003455 anaphylaxis Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000037444 atrophy Effects 0.000 description 1
- 230000001580 bacterial effect Effects 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 238000013477 bayesian statistics method Methods 0.000 description 1
- 239000011230 binding agent Substances 0.000 description 1
- 208000002352 blister Diseases 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 239000010839 body fluid Substances 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 239000010951 brass Substances 0.000 description 1
- 206010007625 cardiogenic shock Diseases 0.000 description 1
- 230000010261 cell growth Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 231100000481 chemical toxicant Toxicity 0.000 description 1
- 208000019069 chronic childhood arthritis Diseases 0.000 description 1
- 229920001940 conductive polymer Polymers 0.000 description 1
- 239000004020 conductor Substances 0.000 description 1
- 208000010247 contact dermatitis Diseases 0.000 description 1
- 208000030381 cutaneous melanoma Diseases 0.000 description 1
- 208000017763 cutaneous neuroendocrine carcinoma Diseases 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 230000005750 disease progression Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000009429 electrical wiring Methods 0.000 description 1
- 229940124645 emergency medicine Drugs 0.000 description 1
- 238000004134 energy conservation Methods 0.000 description 1
- 231100000333 eschar Toxicity 0.000 description 1
- 235000002864 food coloring agent Nutrition 0.000 description 1
- 238000004374 forensic analysis Methods 0.000 description 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical group [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 1
- 239000010931 gold Substances 0.000 description 1
- 229910052737 gold Inorganic materials 0.000 description 1
- 208000024908 graft versus host disease Diseases 0.000 description 1
- 230000012010 growth Effects 0.000 description 1
- 239000003721 gunpowder Substances 0.000 description 1
- 206010018797 guttate psoriasis Diseases 0.000 description 1
- 229910052736 halogen Inorganic materials 0.000 description 1
- 150000002367 halogens Chemical class 0.000 description 1
- 230000035876 healing Effects 0.000 description 1
- XMBWDFGMSWQBCA-UHFFFAOYSA-N hydrogen iodide Chemical compound I XMBWDFGMSWQBCA-UHFFFAOYSA-N 0.000 description 1
- 238000002675 image-guided surgery Methods 0.000 description 1
- 208000015446 immunoglobulin a vasculitis Diseases 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000012535 impurity Substances 0.000 description 1
- 238000011065 in-situ storage Methods 0.000 description 1
- AMGQUBHHOARCQH-UHFFFAOYSA-N indium;oxotin Chemical compound [In].[Sn]=O AMGQUBHHOARCQH-UHFFFAOYSA-N 0.000 description 1
- 230000004054 inflammatory process Effects 0.000 description 1
- 238000002329 infrared spectrum Methods 0.000 description 1
- 201000002215 juvenile rheumatoid arthritis Diseases 0.000 description 1
- 210000001117 keloid Anatomy 0.000 description 1
- 208000032839 leukemia Diseases 0.000 description 1
- 230000031700 light absorption Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 208000012866 low blood pressure Diseases 0.000 description 1
- 206010025226 lymphangitis Diseases 0.000 description 1
- 230000036244 malformation Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000028161 membrane depolarization Effects 0.000 description 1
- 208000022089 meningococcemia Diseases 0.000 description 1
- QSHDDOUJBYECFT-UHFFFAOYSA-N mercury Chemical compound [Hg] QSHDDOUJBYECFT-UHFFFAOYSA-N 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 230000009401 metastasis Effects 0.000 description 1
- 238000002493 microarray Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 239000002991 molded plastic Substances 0.000 description 1
- 208000008588 molluscum contagiosum Diseases 0.000 description 1
- 229910021421 monocrystalline silicon Inorganic materials 0.000 description 1
- 208000001725 mucocutaneous lymph node syndrome Diseases 0.000 description 1
- 201000004931 neurofibromatosis Diseases 0.000 description 1
- 238000011369 optimal treatment Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000003647 oxidation Effects 0.000 description 1
- 238000007254 oxidation reaction Methods 0.000 description 1
- 206010033675 panniculitis Diseases 0.000 description 1
- 244000052769 pathogen Species 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 239000011241 protective layer Substances 0.000 description 1
- 102000004169 proteins and genes Human genes 0.000 description 1
- 108090000623 proteins and genes Proteins 0.000 description 1
- 238000012887 quadratic function Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 238000012892 rational function Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000002271 resection Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 208000005687 scabies Diseases 0.000 description 1
- 231100000241 scar Toxicity 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 208000008742 seborrheic dermatitis Diseases 0.000 description 1
- 201000003385 seborrheic keratosis Diseases 0.000 description 1
- 230000036303 septic shock Effects 0.000 description 1
- 206010040400 serum sickness Diseases 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 206010040560 shock Diseases 0.000 description 1
- 239000002356 single layer Substances 0.000 description 1
- 210000004927 skin cell Anatomy 0.000 description 1
- 206010040882 skin lesion Diseases 0.000 description 1
- 231100000444 skin lesion Toxicity 0.000 description 1
- 230000036555 skin type Effects 0.000 description 1
- 238000010183 spectrum analysis Methods 0.000 description 1
- 208000011584 spitz nevus Diseases 0.000 description 1
- 210000004003 subcutaneous fat Anatomy 0.000 description 1
- 210000004304 subcutaneous tissue Anatomy 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 208000009056 telangiectasis Diseases 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000001931 thermography Methods 0.000 description 1
- 239000010409 thin film Substances 0.000 description 1
- 201000003875 tinea corporis Diseases 0.000 description 1
- 231100000167 toxic agent Toxicity 0.000 description 1
- 208000002419 toxicodendron dermatitis Diseases 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- 230000008733 trauma Effects 0.000 description 1
- 208000009999 tuberous sclerosis Diseases 0.000 description 1
- 238000002211 ultraviolet spectrum Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
- 210000002700 urine Anatomy 0.000 description 1
- 230000003612 virological effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 229910052724 xenon Inorganic materials 0.000 description 1
- FHNFHKCVQCLJFQ-UHFFFAOYSA-N xenon atom Chemical compound [Xe] FHNFHKCVQCLJFQ-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0075—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/44—Detecting, measuring or recording for evaluating the integumentary system, e.g. skin, hair or nails
- A61B5/441—Skin evaluation, e.g. for skin disorder diagnosis
- A61B5/443—Evaluating skin constituents, e.g. elastin, melanin, water
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/44—Detecting, measuring or recording for evaluating the integumentary system, e.g. skin, hair or nails
- A61B5/441—Skin evaluation, e.g. for skin disorder diagnosis
- A61B5/447—Skin evaluation, e.g. for skin disorder diagnosis specially adapted for aiding the prevention of ulcer or pressure sore development, i.e. before the ulcer or sore has developed
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/48—Other medical applications
- A61B5/4869—Determining body composition
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/30—Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/361—Image-producing devices, e.g. surgical cameras
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/02—Details
- G01J3/0205—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
- G01J3/0208—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows using focussing or collimating elements, e.g. lenses or mirrors; performing aberration correction
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/02—Details
- G01J3/0205—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
- G01J3/021—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows using plane or convex mirrors, parallel phase plates, or particular reflectors
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/02—Details
- G01J3/0205—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
- G01J3/0224—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows using polarising or depolarising elements
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/02—Details
- G01J3/0291—Housings; Spectrometer accessories; Spatial arrangement of elements, e.g. folded path arrangements
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/02—Details
- G01J3/10—Arrangements of light sources specially adapted for spectrometry or colorimetry
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/12—Generating the spectrum; Monochromators
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2803—Investigating the spectrum using photoelectric array detector
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2823—Imaging spectrometer
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/30—Measuring the intensity of spectral lines directly on the spectrum itself
- G01J3/36—Investigating two or more bands of a spectrum by separate detectors
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/447—Polarisation spectrometry
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
- G01N21/255—Details, e.g. use of specially adapted sources, lighting or optical systems
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
- G01N21/31—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/47—Scattering, i.e. diffuse reflection
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/50—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/30—Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
- A61B2090/309—Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure using white LEDs
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/361—Image-producing devices, e.g. surgical cameras
- A61B2090/3618—Image-producing devices, e.g. surgical cameras with a mirror
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2560/00—Constructional details of operational features of apparatus; Accessories for medical measuring apparatus
- A61B2560/04—Constructional details of apparatus
- A61B2560/0406—Constructional details of apparatus specially shaped apparatus housings
- A61B2560/0425—Ergonomically shaped housings
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
- A61B2562/02—Details of sensors specially adapted for in-vivo measurements
- A61B2562/0233—Special features of optical sensors or probes classified in A61B5/00
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
- A61B2562/02—Details of sensors specially adapted for in-vivo measurements
- A61B2562/028—Microscale sensors, e.g. electromechanical sensors [MEMS]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/12—Generating the spectrum; Monochromators
- G01J2003/1213—Filters in general, e.g. dichroic, band
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2823—Imaging spectrometer
- G01J2003/2826—Multispectral imaging, e.g. filter imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/47—Scattering, i.e. diffuse reflection
- G01N2021/4704—Angular selective
- G01N2021/4709—Backscatter
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10036—Multispectral image; Hyperspectral image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
- G06T2207/30064—Lung nodule
Definitions
- This application generally relates to systems and methods for
- Hyperspectral/multispectral spectroscopy is an imaging technique that integrates multiples images of an object resolved at different narrow spectral bands (e.g., narrow ranges of wavelengths) into a single data cube, referred to as a hyperspectral/multispectral data cube.
- Data provided by hyperspectral/multispectral spectroscopy allow for the identification of individual components of a complex composition through the recognition of
- Hyperspectral/multispectral spectroscopy has been used for a variety of applications, ranging from geological and agricultural surveying to military surveillance and industrial evaluation.
- satellite hyperspectral/multispectral imaging has been used in mineral exploration, environmental monitoring and military surveillance.
- Hyperspectral/multispectral spectroscopy has also been used in medical applications to assist with complex diagnosis and predict treatment outcomes.
- medical hyperspectral/multispectral imaging has been used to accurately predict viability and survival of tissue deprived of adequate perfusion, and to differentiate diseased (e.g. tumor) and ischemic tissue from normal tissue.
- Hyperspectral/multispectral imaging can also suffer from poor temporal and spatial resolution, as well as low optical throughput, due to the complex optics and taxing
- Certain aspects of the application provide methods and systems for co-axial hyperspectral/multispectral imaging.
- the methods and systems are based on a specific architectural arrangement of the internal hardware of a
- hyperspectral/multispectral imaging system which serves to direct and resolve light of multiple wavelengths in a co-axial fashion.
- a beam steering element having a plurality of operating modes directs light of different wavelengths to distinct optical detectors from a common point of origin, thus maintaining co-axial alignment between images captured by the respective optical detectors.
- the present disclosure provides a hyperspectral/multispectral imaging device.
- a hyperspectral/multispectral imaging device comprising a housing having an exterior and an interior and at least one objective lens attached to or within the housing.
- the at least one objective lens is disposed in an optical communication path comprising an originating end and a terminating end.
- the hyperspectral/multispectral imaging device further comprises a beam steering element within the interior of the housing.
- the beam steering element is in optical communication with the at least one objective lens and is positioned at the terminating end of the optical communication path.
- the beam steering element is characterized by a plurality of operating modes. Each respective operating mode in the plurality of operating modes causes the beam steering element to be in optical communication with a different optical detector.
- the hyperspectral/multispectral imaging device further comprises a plurality of optical detectors offset from the optical communication path. Each respective optical detector in the plurality of optical detectors is in optical communication with a corresponding operating mode of the beam steering element.
- the hyperspectral/multispectral imaging device further comprises a plurality of detector filters within the housing. Each respective detector filter in the plurality of detector filters covers a corresponding optical detector in the plurality of optical detectors thereby filtering light received by the corresponding optical detector from the beam steering element.
- the hyperspectral/multispectral imaging device further comprises at least one processor within the interior of the housing.
- the at least one processor is in electrical communication with the beam steering element and the plurality of optical detectors.
- the at least one program comprises instructions for (i) switching the beam steering element between operating modes in the plurality of operating modes, and (ii) controlling each optical detector in the plurality of optical detectors.
- the hyperspectral/multispectral imaging device further comprises at least one light source for illuminating a target object or subject.
- the present disclosure provides a hyperspectral/multispectral camera for medical imaging.
- a hyperspectral/multispectral imaging device comprising a housing having an exterior and an interior and at least one light source disposed on the exterior of the housing.
- the hyperspectral/multispectral imaging device comprises at least one objective lens attached to or within the housing.
- the at least one objective lens is disposed in an optical communication path.
- the optical communication path comprises an originating end and a terminating end.
- the at least one light source is offset from the optical communication path and is positioned so that light from the at least one light source is (i) first backscattered by a surface of a patient positioned at the originating end of the optical communication path and (ii) then passed from the originating end of the optical communication path, through the at least one objective lens, and to the terminating end of the optical communication path.
- the hyperspectral/multispectral imaging device further comprises a beam steering element within the interior of the housing.
- the beam steering element is in optical communication with the at least one objective lens and is positioned at the terminating end of the optical communication path.
- the beam steering element is characterized by a plurality of operating modes. Each respective operating mode in the plurality of operating modes causes the beam steering element to be in optical communication with a different optical detector.
- the hyperspectral/multispectral imaging device further comprises a plurality of optical detectors offset from the optical
- Each respective optical detector in the plurality of optical detectors is in optical communication with a corresponding operating mode of the beam steering element.
- the hyperspectral/multispectral imaging device further comprises a plurality of detector filters within the housing. Each respective detector filter in the plurality of detector filters covers a corresponding optical detector in the plurality of optical detectors thereby filtering light received by the corresponding optical detector from the beam steering element.
- At least one processor is within the interior of the housing. The at least one processor is in electrical communication with the one or more light source, the beam steering element, and the plurality of optical detectors.
- There is a memory within the interior of the housing. At least one program is stored in the memory and is executable by the at least one processor.
- the at least one program comprises instructions for operating the at least one light source, switching the beam steering element between operating modes in the plurality of operating modes, and controlling each optical detector in the plurality of optical detectors. Further, in some embodiments, there is a communication interface in electrical communication with the at least one processor.
- the light source comprises a first incoherent light source.
- the first incoherent light source is configured to emit near-infrared radiation.
- the first incoherent light source is a light-emitting diode.
- the first incoherent light source is a broadband light source, where the broadband light source emits radiation of wavelengths spanning at least 200 nm.
- the broadband light source emits radiation of wavelengths spanning at least 500 nm.
- the broadband light source is a white light- emitting diode.
- the device further comprises a first illumination lens covering the first incoherent light source thereby focusing the light emitted by the incoherent light source.
- the device further comprises a first light source filter covering the first incoherent light source thereby filtering light emitted by the incoherent light source.
- the device further comprises a first light source polarizer covering the first incoherent light source thereby polarizing light emitted by the first incoherent light source.
- the device further comprises a first homogenizer covering the first incoherent light source thereby homogenizing light emitted by the first incoherent light source.
- the first incoherent light source is in a first plurality of incoherent light sources disposed on the exterior of the housing.
- the first incoherent light source is configured to emit radiation of a first wavelength and a second incoherent light source in the plurality of incoherent light sources is configure to emit radiation of a second wavelength, where the first and second wavelengths are different.
- the plurality of incoherent light sources comprises a first sub-plurality of incoherent light sources capable of emitting radiation of a first wavelength.
- the first sub-plurality of incoherent light sources is arranged in a first pattern on the exterior of the housing, for instance the first pattern having radial symmetry with respect to the at least one objective lens.
- the plurality of incoherent light sources further comprises a second sub-plurality of incoherent light sources capable of emitting radiation of a second wavelength.
- the second sub- plurality of incoherent light sources is arranged in a second pattern on the exterior of the housing, for instance the second pattern having radial symmetry with respect to the at least one objective lens.
- the plurality of incoherent light sources comprises a plurality of light-emitting diodes. In some embodiments of the devices described above, the plurality of incoherent light sources comprises a plurality of broadband light sources.
- the device further comprises a first plurality of illumination lenses disposed on the exterior of the housing, each illumination lens in the first plurality of illumination lenses covering a corresponding incoherent light source in the plurality of incoherent light sources thereby focusing light emitted by the corresponding incoherent light source.
- the plurality of illumination lenses is removable from the exterior of the housing.
- the device further comprises a first plurality of light source filters disposed on the exterior of the housing, each respective light source filter in the first plurality of light source filters covering a corresponding incoherent light source in the plurality of incoherent light sources thereby filtering light emitted by the corresponding incoherent light source.
- the plurality of light source filters is removable from the exterior of the housing.
- the device further comprises a first plurality of light source polarizers, each respective light source polarizer in the first plurality of light source polarizers covering a corresponding incoherent light source in the plurality of incoherent light sources thereby polarizing light emitted by the corresponding incoherent light source.
- the plurality of light source polarizers is removable from the exterior of the housing.
- the device further comprises a first plurality of light homogenizers, each respective light homogenizer in the first plurality of light homogenizers covering a corresponding incoherent light source in the plurality of incoherent light sources thereby homogenizing light emitted by the corresponding incoherent light source.
- the plurality of light homogenizers is removable from the exterior of the housing.
- the at least one light source comprises a first coherent light source.
- the first coherent light source is configured to emit near-infrared radiation.
- the first coherent light source is a laser (e.g. , a laser diode).
- the first coherent light source is in a plurality of coherent light sources disposed on the exterior of the housing.
- the plurality of incoherent light sources comprises a plurality of laser diodes.
- the at least one objective lens comprises a fixed focus lens or a variable focus lens. In some embodiments of the devices described above, the at least one objective lens is a variable focus lens that is focused manually. In some embodiments of the devices described above, the objective lens is a variable focus lens that is an autofocus lens. In some embodiments of the devices described above, the variable focus lens is a zoom lens (e.g., a manual zoom lens or an auto-zoom lens).
- the beam steering element comprises a mirror mounted on an actuator.
- the actuator has a plurality of operating modes.
- the mirror is a single-surface mirror. In some embodiments of the devices described above, the mirror is mounted on an actuator, the actuator having a plurality of operating modes.
- the mirror is a two-axis micro electrical (MEMS) mirror.
- the beam steering element comprises an array of micromirrors.
- the array of micromirrors comprises a first and second plurality of micromirrors. Each respective micromirror in the first plurality of micromirrors is in a first orientation with respect to the optical
- each respective micromirror in the second plurality of micromirrors is in a second orientation with respect to the optical communication path.
- the first and second orientations comprise different operating modes.
- the array of micromirrors comprises a digital micromirror device.
- the array of micromirrors is mounted on an actuator that has a plurality of operating modes.
- the array of micromirrors is mounted on a two-axis micro electromechanical (MEMS) device.
- MEMS micro electromechanical
- the beam steering element comprises a two-axis scanning device.
- the two-axis scanning device is mounted on an actuator.
- the actuator has a plurality of operating modes.
- the two-axis scanning device is mounted on a two-axis micro electro-mechanical (MEMS) device.
- MEMS micro electro-mechanical
- the plurality of optical detectors comprises at least four optical detectors.
- each respective optical detector in the plurality of optical detectors is arranged in the interior of the housing and is positioned to receive reflected light from the beam steering element.
- each respective optical detector in the plurality of optical detectors is selected from the group consisting of a charge-coupled device (CCD), a complementary metal-oxide-semiconductor (CMOS), a photo-cell, and a focal plane array.
- the plurality of optical detectors comprises a plurality of charge-coupled devices (CCDs).
- the plurality of optical detectors comprises a plurality of complementary metal-oxide-semiconductors (CMOSs). In some embodiments of the devices described above, the plurality of optical detectors comprises a plurality of photo-cells. In some embodiments of the devices described above, the plurality of optical detectors comprises a plurality of focal plane arrays. In some embodiments of the devices described above, each respective optical detector in the plurality of optical detectors is used for detecting a different frequency of radiation.
- CMOSs complementary metal-oxide-semiconductors
- At least one optical detector in the plurality of optical detectors is not covered by a detector filter in the plurality of detector filters. In some embodiments of the devices described above, the at least one optical detector that is not covered by a detector filter is configured for capturing a color image of the surface of the patient. In some embodiments of the devices described above, the at least one optical detector that is not covered by a detector filter is configured for focusing an image of the surface of the patient acquired by at least one respective optical detector in the plurality of optical detectors. In some embodiments of the devices described above, at least one optical detector is covered by a detector filter in the plurality of detector filters and is configured for focusing an image of the surface of the patient acquired by at least one respective optical detector in the plurality of optical detectors.
- the plurality of detector filters comprises at least one bandpass filter, at least one longpass filter, and/or at least one shortpass filter.
- the plurality of detector filters is removable from the interior of the housing.
- the at least one light source is a plurality of light sources disposed on the exterior of the housing, each respective light source in the plurality of light sources corresponding to an optical detector in the plurality of optical detectors, and each respective detector filter in the plurality of detector filters allows radiation emitted from a corresponding light source in the plurality of light sources to pass through to the corresponding optical detector.
- the device further comprises a plurality of detector polarizers in the interior of the housing, each respective polarizer in the plurality of detector polarizers covering a corresponding optical detector in the plurality of optical detectors thereby polarizing light received by the corresponding optical detector.
- the plurality of detector polarizers is removable from the interior of the housing.
- a processor in the at least one processor is selected from the group consisting of a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a stream processor, a microprocessor, and a digital signal processor.
- FPGA field programmable gate array
- ASIC application-specific integrated circuit
- stream processor a stream processor
- microprocessor a microprocessor
- digital signal processor a digital signal processor
- the memory is selected from the group consisting of a random-access memory (RAM), a read-only memory (ROM), and a flash memory.
- the at least one program stored in the memory and executable by the at least one processor comprises instructions for turning on a first light source in the at least one light source, and placing the beam steering element in an operating mode in the plurality of operating modes that causes the beam steering element to be in optical communication with a corresponding optical detector in the plurality of optical detectors.
- the at least one program further comprises instructions for sequentially turning on and off each respective light source in the plurality of light sources.
- the at least one program further comprises instructions for sequentially switching the beam steering element to each respective operating mode in the plurality of operating modes such that each respective optical detector in the plurality of optical detectors is in optical communication with the beam steering element when said corresponding light source is turned on.
- each respective light source in the plurality of light sources is turned on for less than one second. In some embodiments of the devices described above, each respective light source in the plurality of light sources is turned on for less than one-half of a second. In some embodiments of the devices described above, each respective light source in the plurality of light sources is turned on for less than one-quarter of a second.
- the at least one light source is a plurality of light sources
- the beam steering element comprises a digital micromirror device having a plurality of micromirrors.
- Each respective micromirror in the plurality of micromirrors has at least a first operating mode in optical communication with a first corresponding optical detector in the plurality of optical detectors and a second operating mode in optical communication with a second corresponding optical detector in the plurality of optical detectors.
- the at least one program further comprises instructions for turning on a first and a second respective light source in the plurality of light sources and toggling each respective micromirror in the plurality of micromirrors between the first and second operating modes when the first and second respective light sources are turned on.
- each respective micromirror in the plurality of micromirrors further has at least a third operating mode in which the respective micromirrors are in optical communication with a third corresponding optical detector in the plurality of optical detectors, and a fourth operating mode in which the respective micromirrors are in optical communication with a fourth corresponding optical detector in the plurality of optical detectors.
- the at least one program further comprises instructions for turning off the first and second respective light sources, turning on a third and a fourth respective light source in the plurality of light sources, and toggling each respective micromirror in the plurality of micromirrors between the third and fourth operating modes when the third and fourth respective light sources are turned on.
- the at least one light source in the plurality of light sources comprises a plurality of light sources and the beam steering element comprises a digital micromirror device comprising a plurality of micromirrors.
- each respective micromirror in the plurality of micromirrors comprises a first operating mode in which the respective micromirror is in optical communication with a first optical detector in the plurality of optical detectors, and a second operating mode in which the respective micromirror is in optical communication with a second optical detector in the plurality of optical detectors.
- the at least one program further comprises instructions for turning on a first and a second respective light source in the plurality of light sources, placing a first sub-plurality of micromirrors in the plurality of micromirrors to the first operating mode when the first and second respective light sources are turned on, and placing a second sub-plurality of micromirrors in the plurality of micromirrors to the second operating mode when the first and second respective light sources are turned on.
- each respective micromirror in the plurality of micromirrors further comprises a third operating mode in which the respective micromirror is in optical communication with a third corresponding optical detector in the plurality of optical detectors, and a fourth operating mode in which the respective micromirror is in optical communication with a fourth corresponding optical detector in the plurality of optical detectors.
- the at least one program further comprises instructions for turning off the first and second light sources, turning on a third and a fourth respective light source in the plurality of light sources, placing a third sub-plurality of micromirrors in the plurality of micromirrors to the third operating mode when the third and fourth respective light sources are turned on, and placing a fourth sub-plurality of micromirrors in the plurality of micromirrors to the fourth operating mode when the third and fourth respective light sources are turned on.
- the at least one program further comprises instructions for processing a digital image acquired by the plurality of optical detectors.
- the instructions for processing a digital image acquired by the plurality of optical detectors comprises instructions for performing at least one of adjusting the brightness of the acquired digital image, adjusting the contrast of the acquired digital image, removing an artifact from the acquired digital image, cropping the acquired digital image, processing one or more sub- pixels of the acquired digital image, compressing the size of the acquired digital image, assembling a plurality of acquired digital images into a spectral hypercube, transforming a spectral hypercube assembled from the plurality of acquired digital images, formatting data contained within the acquired digital image; and encrypting data contained within the acquired digital image.
- the communication interface comprises a wireless signal transmission element.
- the wireless signal transmission element is selected from the group consisting of a bluetooth transmission element, a ZigBee transmission element, a Wi-Fi transmission element, a cellular transmission element, or an IEEE 802.1 lb, 802.1 la, 802.1 lg, or 802.1 In communication element.
- the communication interface comprises a communication bus.
- the communication bus is a universal serial bus (USB), a Fire Wire serial bus, a serial advanced technology attachment (Serial ATA) bus, a secure digital bus (SD), or an Ethernet bus.
- the communication bus is configured to interface with a removable storage media.
- the communication interface comprises a docking station for a mobile device having a mobile device display.
- the mobile device is a smart phone, a personal digital assistant (PDA), an enterprise digital assistant, a tablet computer, a digital camera, or a portable music player.
- PDA personal digital assistant
- enterprise digital assistant enterprise digital assistant
- tablet computer a tablet computer
- digital camera a digital camera
- portable music player portable music player
- the device further comprises a housing display disposed on the exterior of the housing.
- the housing display is in electronic communication with the at least one processor.
- the at least one program further comprises instructions for displaying an image captured by a respective optical detector in the plurality of optical detectors on the housing display.
- the housing display is a touch screen display.
- the housing display is used for focusing an image of the surface of the patient acquired by at least one respective optical detector in the plurality of optical detectors.
- the imaging device has a maximum power consumption of less than 15 watts, less than 10 watts, or less than 5 watts. In some embodiments of the devices described above, the imaging device is battery powered.
- Another aspect of the instant disclosure provides a method of acquiring a set of coaxial images for hyperspectral/multispectral analysis.
- the method comprises resolving a plurality of images of an object or subject using a
- hyperspectral/multispectral imaging device comprising a housing having an exterior and an interior and at least one objective lens attached to or within the housing.
- the at least one objective lens is disposed in an optical communication path comprising an originating end located at the object or subject and a terminating end.
- the hyperspectral/multispectral imaging device further comprises a beam steering element within the interior of the housing.
- the beam steering element is in optical communication with the at least one objective lens and is positioned at the terminating end of the optical communication path.
- the beam steering element is characterized by a plurality of operating modes. Each respective operating mode in the plurality of operating modes causes the beam steering element to be in optical communication with a different optical detector.
- the hyperspectral/multispectral imaging device further comprises a plurality of optical detectors offset from the optical communication path.
- Each respective optical detector in the plurality of optical detectors is in optical communication with a corresponding operating mode of the beam steering element.
- Each respective detector filter in the plurality of detector filters covers a corresponding optical detector in the plurality of optical detectors thereby filtering light received by the corresponding optical detector from the beam steering element.
- At least one processor is in electrical communication with the beam steering element and the plurality of optical detectors.
- a non-transitory memory is in electrical communication with the at least one processor.
- each respective image in the plurality of images is resolved at a different wavelength of light by a respective optical detector in the plurality of optical detectors.
- Each respective optical detector brought into optical communication with the objective lens by cycling the beam steering element through a plurality of operating modes.
- the present disclosure provides a method of
- hyperspectral/multispectral medical imaging for providing information on a medical condition of a subject.
- One embodiment provides a method for providing information on a medical condition of a subject using a hyperspectral/multispectral imaging device with a plurality of light sources as described herein. The method comprising illuminating a tissue of the subject with a plurality of lights. Each respective light in the plurality of lights characterized by a different wavelength. Each respective light in the plurality of lights emitted from a corresponding light source in the plurality of light sources and is
- the method further comprises identifying a spectral signature within the plurality of digital images that corresponds with a medical condition, thereby providing information on the medical condition of the subject.
- the step of providing information on a medical condition comprises diagnosing a medical condition.
- the plurality of lights illuminates the tissue in a sequential fashion.
- this sequential fashion characterized in that only one light in the plurality of lights is illuminating the tissue of the subject at a time.
- the step of illuminating a tissue of a subject comprises illuminating the tissue with a first respective light and a second respective light in the plurality of lights off the surface of the subject concurrently or nearly concurrently.
- the first respective light is characterized by a first wavelength and the second respective light is characterized by a second wavelength.
- the step of resolving the plurality of backscattered lights comprises resolving the first backscattered light by a first respective optical detector in the plurality of optical detectors.
- the first respective optical detector is covered by a first respective detector filter in the plurality of detector filters.
- the first detector filter is configured to allow light characterized by the first wavelength to pass through, and not allow light characterized by the second wavelength to pass through.
- the step of resolving the plurality of backscattered lights further comprises resolving the second backscattered light by a second respective optical detector in the plurality of optical detectors.
- the second optical detector is covered by a second respective detector filter in the plurality of detector filters.
- the second optical detector filter is configured to: allow light characterized by the second wavelength to pass through, and to not allow light characterized by the first wavelength to pass through.
- the first and second detector filters are characterized by corresponding first and second central wavelengths, and wherein the first and second central wavelengths are separated by at least 10 nm or by at least 25 nm.
- the first and second detector filters are bandpass filters. In some embodiments of the devices and methods described above, the first detector filter is a shortpass filter and the second detector filter is a longpass filter. In some embodiments of the devices and methods described above, the first and second wavelengths are separated by at least 50 nm or by at least 100 nm.
- the plurality of lights comprises at least three respective lights, each respective light characterized by a different wavelength. In some embodiments of the devices and methods described above, at least one respective light in the plurality of lights is characterized by a near infrared wavelength. In some embodiments of the devices and methods described above, at least one respective light in the plurality of lights is characterized by a visible wavelength. In some embodiments of the devices and methods described above, the plurality of lights includes at least one light characterized by a near infrared wavelength and at least one light characterized by a visible wavelength.
- the step of illuminating the tissue comprises illuminating the tissue of the subject with each respective light in the plurality of lights for no longer than two seconds, or for no longer than one second, or for no longer than a half second, or for no longer than a quarter second.
- the step of identifying a spectral signature within the plurality of digital images that corresponds with a medical condition comprises processing the plurality of digital images of the tissue of the subject to determine a set of values corresponding to the tissue of the subject.
- the set of values is selected from deoxyhemoglobin levels, oxyhemoglobin levels, total hemoglobin levels, oxygen saturation, oxygen perfusion, hydration levels, total hematocrit levels, melanin levels, and collagen levels.
- the step of identifying a spectral signature within the plurality of digital images that corresponds with a medical condition comprises processing the plurality of digital images of the surface of the subject to identify a pattern of oxidation or hydration in a tissue associated with the surface of the subject.
- processing the plurality of digital images comprises performing at least one of adjusting the brightness of at least one of the respective digital images, adjusting the contrast of at least one of the respective digital images, removing an artifact from at least one of the respective digital images, cropping at least one of the respective digital images, processing one or more sub- pixels of at least one of the respective digital images, compressing the size of at least one of the respective digital images, assembling the plurality of digital images into a spectral hypercube, transforming a spectral hypercube assembled from the plurality of digital images, formatting data contained within at least one of the respective digital images, and encrypting data contained within at least one of the respective digital images.
- At least a first processing step is performed by the at least one processor of the hyperspectral/multispectral imaging device.
- at least a second processing step is performed external to the hyperspectral/multispectral imaging device.
- the at least second processing step is performed by a mobile device, where the mobile device is configured to be in electrical communication with the hyperspectral/multispectral imaging device.
- the mobile device is selected from the group consisting of a smart phone, a personal digital assistant (PDA), an enterprise digital assistant, a tablet computer, a digital camera, and a portable music player.
- the at least second processing step is performed by an external server.
- the external server is a cloud computing environment.
- the external server is a server hosted within a hospital where the imaging takes place.
- the medical condition is selected from the group consisting of tissue ischemia, ulcer formation, ulcer progression, venous stasis, venous ulcer disease, infection, shock, cardiac decompensation, respiratory insufficiency, hypovolemia, the progression of diabetes, congestive heart failure, sepsis, dehydration, hemorrhage, hypertension, exposure to a chemical or biological agent, and an inflammatory response.
- the method further comprises focusing a preliminary image of the tissue of the subject prior to resolving the plurality of backscattered lights.
- hyperspectral/multispectral imaging device further comprises a housing display disposed on the exterior of the housing, the housing display is in electronic communication with the at least one processor, where the preliminary image is displayed on the housing display.
- the communication interface of the hyperspectral/multispectral imaging device is in electronic communication with a mobile device having a mobile device display, where the preliminary image is displayed on the mobile device display.
- the mobile device is selected from the group consisting of a smart phone, a personal digital assistant (PDA), an enterprise digital assistant, a tablet computer, a digital camera, and a portable music player.
- the present disclosure provides a method for acquiring a
- the method includes resolving the plurality of images of a subject using a
- hyperspectral/multispectral imaging device comprising: a housing having an exterior and an interior; at least one objective lens attached to or within the housing, the at least one objective lens disposed in an optical communication path, the optical communication path comprising an originating end located at the object or subject and a terminating end; a beam steering element within the interior of the housing, the beam steering element in optical
- the beam steering element characterized by a plurality of operating modes, each respective operating mode in the plurality of operating modes causing the beam steering element to be in optical communication with a different optical detector; a plurality of optical detectors offset from the optical communication path, each respective optical detector in the plurality of optical detectors in optical communication with a corresponding operating mode of the beam steering element; a plurality of detector filters within the housing, each respective detector filter in the plurality of detector filters covering a corresponding optical detector in the plurality of optical detectors thereby filtering light received by the corresponding optical detector from the beam steering element; at least one processor in electrical communication with the beam steering element and the plurality of optical detectors; a memory in electrical communication with the at least one processor, wherein at least one program is stored in the memory and executable by the at least one processor, the at least one program comprising instructions for: switching said beam steering element between operating modes in the plurality of operating modes and controlling each optical detector in said plurality of optical detector
- each respective image in the plurality of images is resolved at a different wavelength of light by a respective optical detector in the plurality of optical detectors, each respective optical detector brought into optical communication with the objective lens by cycling the beam steering element through a plurality of operating modes.
- the present disclosure provides a non-transitory computer readable storage medium storing at least one program for collecting co-axially aligned images of a subject at a plurality of narrowband wavelengths configured for execution by at least one processor of a computer system.
- the at least one program includes instructions for: turning on a first illumination source configured to emit narrowband light having a first wavelength or first wavelength band; placing a beam steering element in a first operating mode in optical communication with a first optical detector configured to resolve narrowband light having the first wavelength or the first wavelength band; capturing an image of the subject illuminated by the light emitted by the first illumination source using the first optical detector; turning off the first illumination source; turning on a second illumination source configure to emit narrowband light having a second wavelength or a second wavelength band; placing the beam steering element in a second operating mode in optical communication with a second detector configured to resolve narrowband light having the second wavelength or the second wavelength band; capturing an image of the subject illuminated by the light emitted by the second illumination source using the second optical detector; and optionally repeating the fourth through seventh steps a first predetermined number of times to collect a second predetermined number of images at distinct wavelengths or wavelength bands, thereby collecting co-axially aligned images of the subject.
- the present disclosure provides a method for directing an imager to acquire a hyperspectral/multispectral image of a tissue of a subject, the
- the imager includes: a housing having an exterior and an interior; at least one light source disposed on the exterior of the housing; at least one objective lens attached to or within the housing, the at least one objective lens disposed in an optical communication path, the optical communication path comprising an originating end and a terminating end, where light from the at least one light source is first backscattered by a tissue of a subject and then passed from the originating end of the optical communication path, through the at least one objective lens, and to the terminating end of the optical communication path; a beam steering element within the interior of the housing, the beam steering element in optical communication with the terminating end of the optical communication path, the beam steering element characterized by a plurality of operating modes; a plurality of optical detector
- the method includes identifying a plurality of baseline exposure times, each respective baseline exposure time in the plurality of baseline exposure times representing an exposure time for resolving a respective sub-image, in the plurality of sub-images, of the tissue of the subject at the wavelength or wavelength band of the respective sub-image, wherein a first baseline exposure time for a first sub-image is different than a second baseline exposure time of a second sub-image in the plurality of sub-images.
- the method further includes cycling the beam steering element through the plurality of operating modes, wherein the beam steering element is retained in each respective operating mode for the baseline exposure time corresponding to the wavelength or wavelength band collected by the optical filter corresponding to the respective operating mode so that a sub- image is recorded on the optical detector corresponding to the respective operating mode, thereby collecting the hyperspectral/multispectral image of the tissue.
- the present disclosure provides a non-transitory computer readable storage medium storing one or more programs executable by a hyperspectral/multispectral imaging device with a central processing unit configured to execute the one or more programs and an optical acquisition subsystem configured to acquire a
- the hyperspectral/multispectral imaging device comprising: a housing having an exterior and an interior; at least one light source disposed on the exterior of the housing; at least one objective lens attached to or within the housing, the at least one objective lens disposed in an optical communication path, the optical communication path comprising an originating end and a terminating end, wherein light from the at least one light source is first backscattered by a tissue of a subject and then passed from the originating end of the optical communication path, through the at least one objective lens, and to the terminating end of the optical communication path; a beam steering element within the interior of the housing, the beam steering element in optical
- the beam steering element characterized by a plurality of operating modes; a plurality of optical detectors offset from the optical communication path, each respective optical detector in the plurality of optical detectors in optical communication with a corresponding operating mode of the beam steering element; and a plurality of detector filters within the housing, each respective detector filter in the plurality of detector filters covering a corresponding optical detector in the plurality of optical detectors thereby filtering light received by the
- the one or more programs comprising instructions for identifying a plurality of baseline exposure times, each respective baseline exposure time in the plurality of baseline exposure times representing an exposure time for resolving a respective sub-image, in the plurality of sub-images, of the tissue of the subject at the wavelength or wavelength band of the respective sub-image, wherein a first baseline exposure time for a first sub-image is different than a second baseline exposure time of a second sub-image in the plurality of sub-images and cycling the beam steering element through the plurality of operating modes, wherein the beam steering element is retained in each respective operating mode for the baseline exposure time corresponding to the wavelength or wavelength band collected by the optical filter corresponding to the respective operating mode so that a sub-image is recorded on the optical detector corresponding to the respective operating mode, thereby collecting the hyper
- a baseline exposure time for a sub-image in the plurality of sub- images is determined by a factor affecting a baseline illumination of the tissue of the subject and a sensitivity of the corresponding optical detector in the optical acquisition subsystem used to acquire the sub-image at the wavelength or wavelength band corresponding to the sub-image.
- the factor is an amount of illumination of the tissue of the subject provided by an illumination subsystem of the hyperspectral/multispectral imaging device, an amount of ambient light, or a concentration of melanin in the tissue of the subject.
- Figure 1 is a schematic illustration of the internal hardware of a co-axial
- hyperspectral/multispectral camera according to some embodiments.
- Figure 2A is a schematic illustration of the internal hardware of a co-axial
- FIG. 2B is a schematic illustration of the internal hardware of a co-axial hyperspectral/multispectral camera, according to some embodiments.
- the illustration shows a cross-section down the barrel of the camera with a perspective view of the optical detectors 206.
- Figure 3A is a schematic illustration of the internal hardware of a co-axial hyperspectral/multispectral camera, according to some embodiments.
- the illustration shows an exploded view of the components of the internal hardware with a perspective view of the beam steering element 204.
- Figure 3B is a schematic illustration of the internal hardware of a co-axial hyperspectral/multispectral camera, according to some embodiments.
- the illustration shows a cross-section down the barrel of the camera with a perspective view of the imager subsystem 210.
- Figure 4 is a schematic illustration of the internal hardware of a co-axial
- hyperspectral/multispectral camera according to some embodiments.
- the illustration shows a view looking down the barrel of the camera.
- Figure 5 is a schematic illustration of the internal hardware of a co-axial
- hyperspectral/multispectral camera according to some embodiments.
- the illustration shows a cross-section through the barrel of the camera with a perspective view of the imager subsystem 210.
- Figure 6 is a schematic illustration of the illumination subsystem of a co-axial hyperspectral/multispectral camera, according to some embodiments. The illustration shows an exploded view of the components of the illumination subsystem.
- Figure 7 is a schematic illustration of the imager subsystem of a co-axial
- hyperspectral/multispectral camera according to some embodiments.
- the illustration shows an exploded view of the components of the imager subsystem.
- Figure 8 is a series of illustrations of a first housing for a co-axial
- FIG. 8A is a front elevational view of the first housing; Fig. 8B is a back elevational view thereof; Fig. 8C is a top plan view thereof; Figure 8D is a bottom plan view thereof; Fig. 8E is a left side elevational view thereof; and Fig. 8F is a right side elevational view thereof.
- Figure 9 is a series of illustrations of a second housing for a co-axial hyperspectral/multispectral camera, according to some embodiments: Fig. 9A is a front elevational view of the second housing; Fig. 9B is a back elevational view thereof; Fig. 9C is a top plan view thereof; Figure 9D is a bottom plan view thereof; Fig. 9E is a left side elevational view thereof; and Fig. 9F is a right side elevational view thereof.
- Figure 10 is a series of illustration of a third housing for a co-axial
- Fig. 10A is a front elevational view of the second housing; Fig. 10B is a back elevational view thereof; Fig. IOC is a top plan view thereof; Figure 10D is a bottom plan view thereof; Fig. 10E is a left side elevational view thereof; and Fig. 1 OF is a right side elevational view thereof.
- Figures 11 A and 1 IB are perspective views of a first housing for a co-axial hyperspectral/multispectral camera, according to some embodiments.
- Figures 11C and 1 ID are perspective views of a second housing for a co-axial hyperspectral/multispectral camera, according to some embodiments.
- Figures 1 IE and 1 IF are perspective views of a third housing for a co-axial hyperspectral/multispectral camera, according to some embodiments.
- Figure 12 is a schematic illustration of the internal hardware of a co-axial hyperspectral/multispectral camera mounted in a housing, according to some embodiments.
- the illustration shows a cross-section down the barrel of the camera with a perspective view of the beam steering element 204.
- Figure 13 schematically illustrates a processing subsystem for a co-axial
- hyperspectral/multispectral system according to some embodiments.
- Figure 14 illustrates a method for providing information regarding the medical condition of a subject and optionally treating the subject accordingly, according to some embodiments.
- Figure 15 is a schematic illustration of the light path for a captured
- hyperspectral/multispectral image according to some embodiments.
- Figure 16 is a schematic illustration of instructions for collecting and optionally processing hyperspectral/multispectral images, according to some embodiments.
- Figure 17 is a schematic illustration of instructions for optimizing exposure times and collecting a hyperspectral/multispectral image, according to some embodiments.
- Hyperspectral and multispectral imaging are related techniques in larger class of spectroscopy commonly referred to as spectral imaging or spectral analysis.
- hyperspectral imaging relates to the acquisition of a plurality of images, each image representing a narrow spectral band collected over a continuous spectral range, for example, 20 spectral bands having a FWHM bandwidth of 20 nm each, covering from 400 nm to 800 nm.
- multispectral imaging relates to the acquisition of a plurality of images, each image representing a narrow spectral band collected over a discontinuous spectral range.
- the terms “hyperspectral” and “multispectral” are used interchangeably and refer to a plurality of images, each image representing a narrow spectral band (having a FWHM bandwidth of between 10 nm and 30 nm, between 5 nm and 15 nm, between 5 nm and 50 nm, less than 100 nm, between 1 and 100 nm, etc.) , whether collected over a continuous or discontinuous spectral range.
- narrowband spectral range or “narrowband” are used interchangeably and refer to a continuous span of wavelengths, typically consisting of a FWHM spectral band of no more than about 100 nm.
- narrowband radiation consists of a FWHM spectral band of no more than about 75 nm, 50 nm, 40 nm, 30 nm, 25 nm, 20 nm, 15 nm, 10 nm, 5 nm, 4 nm, 3 nm, 2 nm, or 1 nm.
- Hyperspectral deals with imaging narrow spectral bands over a continuous spectral range, and produce the spectra of all pixels in the scene. So a sensor with only 20 bands can also be hyperspectral when it covers the range from 500 to 700 nm with 20 bands each 10 nm wide. (although a sensor with 20 discrete bands covering the visible spectrum, near-infrared, short-wavelength infrared, mid-wavelength infrared, and long-wavelength infrared would be considered multispectral.)
- the application provides methods and systems that enable the capture of co-axial hyperspectral/multispectral images using a beam steering element having a plurality of operating modes and a plurality of optical detectors, each of which is configured to resolve light at a specific wavelength(s).
- the architecture of the internal hardware of the hyperspectral/multispectral systems provided herein e.g., beam steering element and optical detectors, provides several advantages over systems and methods of
- hyperspectral/multispectral imaging known in the art.
- the systems and methods provided herein allow for true co-axial alignment of images captured at a range of wavelengths without the need for a tunable filter, shorter subject illumination times, shorter image exposure times, more rapid hyperspectral/multispectral imaging, lower power consumption, and lower manufacturing costs, to name a few benefits.
- the present application provides hyperspectral/multispectral systems and methods that use a beam steering element (e.g. , a mirror or array of micromirrors) having a plurality of operating modes capable of directing light of different wavelengths to distinct optical detectors from a common point of origin, thus maintaining co-axial alignment between images captured by the respective optical detectors.
- a beam steering element e.g. , a mirror or array of micromirrors
- each of the optical detectors is covered by a filter, to select a specific wavelength of light. This allows for the resolution of images of an object at multiple wavelengths without the need for expensive optics, such as tunable filers (e.g., liquid crystal tunable filters).
- the systems described herein can capture true two-dimensional co-axial images by sequentially resolving images of the object at different wavelengths using respective optical detectors in a plurality of optical detectors.
- very minor pixel field alignment between wavelength images may be necessary to account for edge effects and minor mechanical misalignment between the optical detectors and beam steering element.
- the optical detectors and beam steering element may be necessary to account for edge effects and minor mechanical misalignment between the optical detectors and beam steering element.
- hyperspectral/multispectral imaging systems described herein will contain software operable by the central processing unit(s) 1308 to perform minor image alignment between digital images 1332 acquired by different optical detectors 206, for example, during post processing of the hyperspectral/multispectral data cube 1336.
- minor alignment may be performed by an external device or server after the digital images 1332 and/or hyperspectral/multispectral data cube 1336 is transmitted from the imaging system.
- hyperspectral/multispectral have matching narrow band illumination sources and detector filters.
- the use of matching illumination and detection elements dramatically increases the signal-to-noise ratio of the resolved images (e.g. , the amount of light having a desired wavelength as compared to the amount of light having undesired wavelengths).
- the signal- to-noise ratio of the resolved images can be further increased by including matching polarizers over corresponding illumination and detection elements.
- hyperspectral/multispectral imaging systems can effectively capture images of a target object, for example, at five different wavelengths in a single second. This is accomplished by sequentially illuminating the object with narrowband light sources of various wavelengths and capturing images of the object with optical detectors configured to resolve light of the respective broadband wavelengths
- Hyperspectral/multispectral systems that capture images at different wavelengths using detectors that are slightly offset from one another (e.g. , at two or more detectors positioned next to each other at the end of an optical path) require extensive computational power to index and align the images with one another.
- images captured using imaging systems that rely on co-boring of multiple objective lenses e.g., where the center of each objective lens points to a common target
- the imaging systems provided herein provide true co-axial aligned images, reducing the computation burden and thus increasing the speed at which hyperspectral/multispectral data can be processed.
- matching illumination and detection elements also eliminates wasted light in bands other than those desired for imaging (e.g., out of band light), thereby improving the efficiency of the imaging system.
- the increased efficiency provided by sequential illumination with narrowband radiation, the absence of tunable filters, and decreased processor requirements results in an imaging system with a low power
- Table 1 provides an estimated power budget, for one embodiment of a hyperspectral/multispectral camera, based on a state of the art mechanical/electrical elements.
- a hyperspectral/multispectral imaging device e.g., a camera
- the disclosure provides a hyperspectral/multispectral camera that is battery operated.
- the camera has a maximum power requirement of less than 15 watts, less than 10 watts, or less than 5 watts.
- the camera has a maximum power requirement of less than 20 W, 19 W, 18 W, 17 W, 16 W, 15 W, 14 W, 13 W, 12 W, 1 1 W, 10 W, 9 W, 8 W, 7 W, 6 W, 5 W, or fewer watts.
- a maximum power requirement of less than 20 W, 19 W, 18 W, 17 W, 16 W, 15 W, 14 W, 13 W, 12 W, 1 1 W, 10 W, 9 W, 8 W, 7 W, 6 W, 5 W, or fewer watts.
- Table 1 Estimated power budget for one embodiment of a hyperspectral/multispectral camera.
- a beam steering element and plurality of optical detectors can be mounted inside a camera housing, for easy manipulation by a healthcare professional.
- the systems and methods provided herein are useful for other hyperspectral/multispectral applications such as satellite imaging (e.g., for geological sensing of minerals, agricultural imaging, and military surveillance), remote chemical imaging, and environmental monitoring.
- satellite imaging e.g., for geological sensing of minerals, agricultural imaging, and military surveillance
- remote chemical imaging e.g., remote chemical imaging
- environmental monitoring e.g., remote chemical imaging, and environmental monitoring.
- a beam steering element and plurality of optical detectors can be mounted inside a satellite or other telescopic apparatus for remote hyperspectral/multispectral imaging.
- FIG. 1 shows a schematic illustration of the internal hardware 100 of a co-axial hyperspectral/multispectral camera, according to some embodiments.
- the camera includes an illumination subsystem 102 comprising a housing 1 14 for one or more light source(s) 104, which are controlled by a central processing unit (CPU; not shown).
- the lights are in electronic communication with a motherboard 1 10, housing the CPU, by way of a flex circuit or wire 112 that connects the illumination subsystem to the motherboard 110.
- the camera further includes a lens subsystem, comprising an objective lens 106, and optionally a complex lens assembly (not shown), which is mounted inside a chassis 108 that appropriately positions the complex lens assembly with respect to an optical communication path originating at the surface of a subject, and terminating at a beam steering element (not shown) situated in the interior of the camera assembly.
- An optional stray light shield 116 functions to decrease stray light and reflections within the internal camera assembly.
- Figures 2A and 2B show a cross-section down the barrel of the internal hardware 100 of a co-axial hyperspectral/multispectral camera, according to some embodiments.
- Figure 2A provides a perspective view of the beam steering element 204
- Figure 2B provides a perspective view of the imager subsystem 210.
- the coaxial hyperspectral/multispectral camera includes: an illumination subsystem 102 containing one or more light sources 104; an objective lens assembly 202 housed in a chassis 108 that anchors the lens assembly with respect to other components of the optical assembly; an optional stray light shield 116; a beam steering element 204 in electrical communication, and optionally mounted on, a motherboard 110 in electrical communication with one or more CPU(s) (not shown); and an imager subsystem 210 comprising a plurality of optical detectors 206 in electrical communication with the motherboard 110 by way of a flex circuit or wire 208.
- an optical communication path is created when radiation emitted from one or more of the lights 104 of the illumination subsystem 102 illuminates a tissue of the subject (not shown) and is backscattered to an objective lens assembly 202, which focuses the light on a beam steering element 204 having a plurality of operating modes.
- the beam steering element 204 When positioned in a respective operating mode, the beam steering element 204 reflects the light onto one of the plurality of optical detectors 206, which is configured to capture an image of the surface of the subject at one or more specific wavelengths.
- Each optical detector 206 in the imager subsystem 210 is optionally covered by an optical filter (e.g., a detector filter), which allows light of a predetermined wavelength to pass through to the detector.
- an optical filter e.g., a detector filter
- one or more of the light sources 104 is matched to a filter covering an optical detector 206, e.g., the light emits radiation at wavelength that is capable of passing through the corresponding filter.
- the beam steering element 204 functions to direct radiation emitted by a respective light source 104 to the corresponding optical detector 206 covered by a matching filter.
- the beam steering element 204 is configured to have a plurality of operating modes, each of which directs light backscattered from the tissue of the subject to a different optical detector 206.
- Figure 12 shows the same cross-section as Figure 2 A of the internal hardware 100 of a co-axial hyperspectral/multispectral camera mounted in housing 1200, according to some embodiments.
- housing 1200 includes dock 1202 for attaching portable device 1204 to housing 1200.
- portable device 1204 contains a display, preferably a touch-screen display, for displaying images acquired by internal hardware 100 of a co-axial hyperspectral/multispectral camera.
- Figures 3 A and 3B show exploded views of the internal hardware of a co-axial hyperspectral/multispectral camera, according to some embodiments.
- Figure 3A provides a perspective view of the light directing subsystem 302
- Figure 2B provides a perspective view of the imager subsystem 210.
- the exploded views show the various components of the illumination subsystem 102, lens subsystem 300, light directing subsystem 302, and imager subsystem 210.
- FIG. 4 shows a view looking down the barrel of the lens subsystem of a co-axial hyperspectral/multispectral camera, according to some embodiments.
- the illumination subsystem comprises a plurality of lights 104 in a radial arrangement about the objective lens 106.
- the lights 104 are arranged having radial or approximate radial symmetry with respect to the objective lens 106.
- the light directing subsystem 302, comprising the beam steering element 204 and optionally an actuator (not shown) for moving the beam steering element between various operating modes, is disposed at the distal end of the barrel of the camera.
- the beam steering element 204 is capable of switching between operating modes to reflect light to the various optical detectors 206 mounted on the backside of the imager subsystem 210.
- Figure 5 illustrates a perspective view looking up the barrel of the lens subsystem of a co-axial hyperspectral/multispectral camera, according to some embodiments.
- the illustration shows a plurality of optical detectors 206 mounted on the back side of the imager subsystem 210, in a radial arrangement about the objective lens assembly 202.
- the optical detectors 206 are arranged having radial or near radial symmetry with respect to the objective lens assembly 202.
- the optical detectors 206 are offset from an optical communication path originating at a surface of an object being imaged and terminating at the beam steering element 204, yet in optical communication with corresponding operating modes of the beam steering element 204.
- FIG. 6 shows an exploded view of the illumination subsystem 102 of a co-axial hyperspectral/multispectral camera, according to some embodiments.
- the illumination subsystem includes a plurality of lights 104 mounted on a printed circuit board (PCB; 602) in electrical communication with a CPU, the CPU optionally mounted or otherwise in electrical communication with a motherboard 110, by means of a flex circuit or wire 112.
- the lights 104 are disposed on the surface of the hyperspectral/multispectral imaging system ⁇ e.g., camera), for example, by being mounted in an illumination subsystem housing 114.
- the lights may be completely contained within the camera housing ⁇ e.g., shielded from the external environment), they will be considered to be disposed on the exterior of the housing so long as they are in optical communication with the external environment.
- lights disposed on the exterior of the camera housing may be behind one or more layers of glass, plastic, or film to protect the lights from the external environment.
- the one or more layers of glass, plastic, or film may have filtering, polarizing, homogenizing, or focusing properties.
- one or more of the light sources 104 may be covered by any combination of a focusing lens, a homogenizer, a polarizer, and a wavelength filter.
- different lights may be covered by different combinations of filtering elements.
- a plurality of lights 104 disposed on the surface of a hyperspectral/multispectral imaging system ⁇ e.g., a co-axial camera
- includes one or more narrow-band light source ⁇ e.g. , a mono-wavelength LED) for capturing images that will be assembled into a hyperspectral/multispectral data cube and at least one broad-band light source ⁇ e.g. , a white light) for capturing color images used, e.g.
- the mono-wavelength light sources may be covered by a light homogenizer and/or light polarizer, as desired for uniform illumination of the subject and improved signal-to-noise ratio for the resolved image, while the broad-band light source need not be homogenized or polarized.
- the illumination sources may be arranged in any fashion on the imaging system.
- lights 104 will be arranged in one or more concentric circles about the objective lens 106 of an imaging system.
- two or more lights 104 emitting radiation of the same wavelength(s) may be illuminated at the same time.
- the two or more lights emitting radiation of the same wavelength(s) are positioned with radial symmetry about the objective lens 106 to provide homogenous illumination of the target.
- one or more broadband illumination source(s) may also be included, for example, for use in capturing a broadband image of the object or subject. Since a broadband image of the object or subject may not necessarily be used in the analysis of collected hyperspectral/multispectral data, the broadband source(s) need not be positioned about or in radial symmetry with the objective lens.
- an imaging device e.g., a camera
- hyperspectral/multispectral camera capable of illuminating an object at eight wavelengths can have two light sources (104-4 and 104-12), positioned symmetrically about the objective lens 106, which emit radiation having the same wavelength(s), e.g., two identical narrowband LEDs.
- four lights (104-4, 104-8, 104-12, and 104-16), emitting radiation of the same wavelength(s), can be positioned symmetrically about the objective lens.
- the imaging system includes 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 35, 40, 45, 50, or more light sources.
- the imaging system includes at least 2, 3, 4, 5, 6, 7, 8, 9, 10, or more lights emitting radiation at the same wavelength(s), preferably arranged in a symmetrical pattern about the objective lens.
- two or more lights 104 may be configured to emit broadband radiation.
- all of the broadband light sources 104 may be illuminated at the same time.
- two or more lights 104 may be configured to emit broadband radiation.
- the various lights 104 can then be covered by illumination filters rated for different wavelengths, which match filters covering the optical detectors 206.
- the illumination filters are removable from the exterior of the housing.
- the detector filters are removable from the interior of the housing. In this fashion, different illumination and/or detector filter sets can be provided for use in evaluating different medical conditions. For example, in some
- a user positions a first illuminator and/or detector filter set on the imaging system to aid in the evaluation of a first medical condition (e.g., ulcer formation) and a second illuminator and/or detector filter set on the imaging system to aid in the evaluation of a second medical condition (e.g., melanoma).
- a first medical condition e.g., ulcer formation
- a second illuminator and/or detector filter set on the imaging system to aid in the evaluation of a second medical condition (e.g., melanoma).
- the first filter set includes filters rated for wavelengths used, for example, to quantify oxyhemoglobin and
- the second filter set includes filters rated for wavelengths used, for example, to discriminate between melanoma and other pigmented legions (e.g., Spitz nevus, seborrheic keratosis, basal cell carcinoma, and melanocytic nevus; see, Nagaoka T. et al, Skin Res Technol. (2011) Aug 31).
- filters rated for wavelengths used for example, to discriminate between melanoma and other pigmented legions (e.g., Spitz nevus, seborrheic keratosis, basal cell carcinoma, and melanocytic nevus; see, Nagaoka T. et al, Skin Res Technol. (2011) Aug 31).
- light sources emitting radiation in the ultraviolet spectrum (wavelengths from about 10 nm to about 400 nm), visible spectrum (wavelengths from about 400 nm to about 760 nm), and/or near- infrared spectrum (wavelengths from about 760 nm to about 2000 nm) are used in the hyperspectral/multispectral imaging systems and methods provided herein.
- the hyperspectral/multispectral imaging systems described herein will generally include at least one light source for illuminating a region of interest on a subject.
- the light source includes a single broadband light source, a plurality of broadband light sources, a single narrowband light source, a plurality of narrowband light sources, or a combination of one or more broadband light source and one or more narrowband light source.
- the light source includes a plurality of coherent light sources, a single incoherent light source, a plurality of incoherent light sources, or a combination of one or more coherent and one or more incoherent light sources.
- broadband it is meant light that includes component wavelengths over a substantial portion of at least one band, e.g., over at least 20%, or at least 30%, or at least 40%, or at least 50%, or at least 60%, or at least 70%, or at least 80%, or at least 90%, or at least 95% of the band, or even the entire band, and optionally includes component wavelengths within one or more other bands.
- a “white light source” is considered to be broadband, because it extends over a substantial portion of at least the visible band.
- broadband light includes component wavelengths across at least 100 nm of the electromagnetic spectrum.
- broadband light includes component wavelengths across at least 150 nm, 200 nm, 250 nm, 300 nm, 400 nm, 500 nm, 600 nm, 700 nm, 800 nm, or more of the electromagnetic spectrum.
- narrowband it is meant light that includes components over only a narrow spectral region, e.g., less than 20%, or less than 15%, or less than 10%, or less than 5%, or less than 2%, or less than 1%, or less than 0.5% of a single band.
- Narrowband light sources need not be confined to a single band, but can include wavelengths in multiple bands.
- a plurality of narrowband light sources may each individually generate light within only a small portion of a single band, but together may generate light that covers a substantial portion of one or more bands, e.g., may together constitute a broadband light source.
- broadband light includes component wavelengths across no more than 100 nm of the electromagnetic spectrum (e.g., has a spectral bandwidth of no more than 100 nm).
- narrowband light has a spectral bandwidth of no more than 90 nm, 80 nm, 75 nm, 70 nm, 60 nm, 50 nm, 40 nm, 30 nm, 25 nm, 20 nm, 15 nm, 10 nm, 5 nm, or less of the electromagnetic spectrum.
- the "spectral bandwidth" of a light source refers to the span of component wavelengths having an intensity that is at least half of the maximum intensity, otherwise known as “full width at half maximum” (FWHM) spectral bandwidth.
- FWHM full width at half maximum
- Many light emitting diodes (LEDs) emit radiation at more than a single discreet wavelength, and are thus narrowband emitters. Accordingly, a narrowband light source can be described as having a “characteristic wavelength” or “center wavelength,” i.e., the wavelength emitted with the greatest intensity, as well as a characteristic spectral bandwidth, e.g., the span of wavelengths emitted with an intensity of at least half that of the characteristic wavelength.
- coherent light source it is meant a light source that emits electromagnetic radiation of a single wavelength in phase.
- a coherent light source is a type of narrowband light source with a spectral bandwidth of less than 1 nm.
- coherent light sources include lasers and laser-type LEDs.
- an incoherent light source emits electromagnetic radiation having a spectral bandwidth of more than 1 nm and/or is not in phase.
- incoherent light can be either narrowband or broadband light, depending on the spectral bandwidth of the light.
- suitable broadband light sources 104 include, without limitation, incandescent lights such as a halogen lamp, xenon lamp, a hydrargyrum medium-arc iodide lamp, and a broadband light emitting diode (LED).
- incandescent lights such as a halogen lamp, xenon lamp, a hydrargyrum medium-arc iodide lamp, and a broadband light emitting diode (LED).
- a standard or custom filter is used to balance the light intensities at different wavelengths to raise the signal level of certain wavelength or to select for a narrowband of wavelengths.
- Broadband illumination of a subject is particularly useful when capturing a color image of the subject or when focusing the hyperspectral/multispectral imaging system.
- suitable narrowband, incoherent light sources 104 include, without limitation, a narrow band light emitting diode (LED), a superluminescent diode (SLD) (see, Redding B., arVix: 1110.6860 (2011), the content of which is hereby incorporated herein by reference in its entirety for all purposes), a random laser, and a broadband light source covered by a narrow band-pass filter.
- suitable narrowband, coherent light sources 104 include, without limitation, lasers and laser-type light emitting diodes.
- coherent illumination is less well suited for full-field imaging due to speckle artifacts that corrupt image formation (see, Oliver, B.M., Proc IEEE 51, 220-221 (1963)).
- coherent illumination is particularly well suited for embodiments comprising a scanning detection device.
- light emitted from a light source 104 passes through an illumination lens, which focuses the light onto a target (e.g., the skin of a subject).
- a target e.g., the skin of a subject
- light emitted from a light source 104 passes through an optical diffuser or light homogenizer, that modifies the focal properties of the light so that it illuminates selected regions of a target in a substantially uniformly fashion. That is, the intensity of light at one point in the region is substantially the same as the intensity of light at another point in the region. In other embodiments, the intensity of the light varies from one point within the region to the next.
- a light source 104 is covered by both an illumination lens and a homogenizer.
- the light also passes through an optional illumination polarizer, which removes any light that does not have a selected polarization.
- the polarizer can be, for example, a polarizing beamsplitter or a thin film polarizer. The polarization can be selected, for example, by rotating the polarizer appropriately.
- a matching detector polarizer should be placed in front of the corresponding optical detector 206 to depolarize the light prior to resolving an image of the backscattered polarized light.
- the illumination subsystem 102 comprises one or more light emitting diodes (LEDs; 104) for illuminating the surface of a patient.
- LEDs provide several advantages over other light sources for illuminating the surface of a subject. For example, because LEDs generate very little heat, illumination with LEDs does not affect the surface temperature of a subject's skin, which results in physiological changes that can alter hyperspectral/multispectral analysis. LEDs can be constructed to emit light having a narrow and defined spectral band, negating a requirement to filter the illumination light, improving the overall efficiency of the imaging system, improving the signal-to-noise ratio of detection at a desired wavelength, and decreasing the illumination and exposure times necessary to produce a high quality digital image of the target.
- LEDs reach maximum intensities very rapidly, for example a red LED can achieve full brightness in under a microsecond, allowing for faster cycling through a plurality of illumination wavelengths. LEDs have a long useful lifetime, as compared to incandescent and fluorescent light sources. LEDs can be manufactured to be very small ⁇ e.g., from about 2 mm to about 8 mm), allowing for many LEDs to be mounted in one or more concentric circles about an objective lens 106.
- LEDs consist of a chip of semiconducting material doped with impurities, which create the formation of a p-n junction.
- Current applied across the semiconducting material readily flows from the p-side (anode) to the n-side (cathode), creating electromagnetic radiation (light) when an electron meets a hole and falls from the conduction band to the valence band.
- the wavelength of the emitted radiation is dependent on the size of the band gap formed between the conduction band and the valence band, and thus the semiconducting material used to form the LED.
- Non-limiting examples of semiconductor materials that can be used to produce an LED emitting light at a specific wavelength are provided in Table 2.
- the LEDs 104 are packaged LEDs, such as the LUXEON ® Rebel (Philips Lumileds Lighting). Other types of packaged LEDs may also be used, such as those manufactured by OSRAM (Ostar package), Luminus Devices (USA), or Tridonic (Austria). As defined herein, a packaged LED is an assembly of one or more LED die that contains electrical connections, such as wire bond connections or stud bumps, and possibly includes an optical element and thermal, mechanical, and electrical interfaces.
- the LEDs 104 may include a lens over the LED chips. Alternatively, LEDs without a lens may be used. LEDs without lenses may include protective layers, which may include phosphors. The phosphors can be applied as a dispersion in a binder, or applied as a separate plate.
- the LED semiconductor chips are typically potted in a clear or colored molded plastic shell, which: facilitates mounting of the LED semiconductor chip in the illumination subsystem housing 114, provides structural protection for the semiconductor chip and associated electrical wiring, and acts as a refractive intermediary between the relatively high- index semiconductor and low-index open air, boosting the light emission from the
- the hyperspectral/multispectral imaging systems provided herein include narrowband width LED lights.
- the narrowband LED lights have a FWHM spectral bandwidth or less than about 100 nm, preferably less than about 50 nm, more preferably less than 25 nm.
- the imaging system includes at least one LED source that emits radiation in the infrared, preferably near-infrared, spectrum. The used of near-infrared LED illumination in is commonly found in closed circuit security cameras.
- the hyperspectral/multispectral imaging system includes a single broadband light source 104 used for illuminating a tissue on the subject.
- each respective image corresponds to the backscattering of a different wavelength of light from the tissue
- each of the optical detectors 206 are covered by a narrow band-pass filter.
- wavelengths of light over a large band that includes all the wavelengths of interest are used simultaneously to illuminate the tissue and detector filters are used as a means of wavelength selection, where the optical detectors are exposed to backscattered light either simultaneously or sequentially.
- the imaging system illuminates a selected region (e.g., tissue) of the subject with the broadband light source 104, and light backscattered from the illuminated region of the subject is focused by the objective lens assembly 202 onto the beam steering element 204. While the selected region is illuminated, the imaging system's CPU 1308 cycles the beam steering element 204 through a plurality of operating modes, where each operating mode places the beam steering element 204 in optical communication with a corresponding optical detector 206.
- a selected region e.g., tissue
- the imaging system's CPU 1308 cycles the beam steering element 204 through a plurality of operating modes, where each operating mode places the beam steering element 204 in optical communication with a corresponding optical detector 206.
- the hyperspectral/multispectral imaging system includes a plurality of broadband light sources 104 used for illuminating a region (e.g., tissue) on the subject.
- Each of the broadband light sources 104 is covered by a narrow band-pass filter so that each light source illuminates the selected region with narrowband light.
- each of the optical detectors 206 used to resolve images for assembly into a hyperspectral/multispectral data cube are covered by a narrow band-pass filter that corresponds with a narrow band-pass filter covering a respective light source.
- Light sources 104 and optical detectors 206 that are covered by identical or substantially similar filters and/or polarizers are termed "matching" elements herein.
- one wavelength at a time is used to illuminate the subject and backscattered light at that wavelength is detected using an optical detector 206 covered by a matching filter. That wavelength is then turned off and the same is done for the next selected wavelength, in a stepwise process to capture a co-axial
- hyperspectral/multispectral image e.g., hyperspectral/multispectral data cube.
- more than one wavelength of light is used to illuminate the subject at a time and backscattered light is resolved using matching filtered optical detectors 206, either simultaneously, nearly simultaneously, or sequentially.
- the light sources 104 are turned on and off in sequence, illuminating a selected region on a subject with one type of filtered light at a time.
- the operating modes of the beam steering element 204 are coordinated with the light sources such that when a light source 104 covered by a particular illumination filter is turned on, the beam steering element 204 is placed in optical communication with an optical detector 206 covered with a matching detector filter.
- the hyperspectral/multispectral imaging system includes a plurality of narrowband light sources 104 used for illuminating a region of skin on the subject, preferably without wavelength illumination filters.
- each of the optical detectors 206 used to resolve images for assembly into a hyperspectral/multispectral data cube are covered by a narrow band-pass filter matching the narrowband light emitted by a corresponding light source.
- the light sources 104 are turned on and off in sequence, illuminating a selected region on a subject with one wavelength at a time.
- the operating modes of the beam steering element 204 are coordinated with the light sources such that when a light source 104 emitting a first wavelength is turned on, the beam steering element 204 is placed in optical communication with an optical detector 206 covered with a matching detector filter.
- hyperspectral/multispectral imaging system includes a combination of both types of light sources 104.
- a broadband light source may be present to provide illumination for, e.g. , acquiring a color image of the selected region or focusing the system.
- illumination filters are present in several of the embodiments provided herein, the use of non-filtered light, particularly non-filtered, narrowband light, is preferred.
- unused light is generated by the hyperspectral/multispectral imaging system, which is inefficient and unnecessarily increases the power requirements of the system.
- an illumination lens is placed over one or more lights 104 disposed on the surface of a hyperspectral/multispectral imaging system to shape the light toward the target and focus it as a spot that is appropriate for the focal distance of the objective imaging lens.
- the focusing lens will also improve the uniformity of light directed onto the target.
- a diffuser may also be placed over one or more lights 104 to cause the incident illumination on the subject or object to be diffuse.
- Illumination lenses are well known in the art.
- a light homogenizer is placed over one or more lights 104 disposed on the surface of a hyperspectral/multispectral imaging system to provide uniform illumination of an area on a target object or subject.
- An exemplary light homogenizer that can be used are the light homogenizers disclosed in Agilent Technologies Application Brief 1-003, Light Guide Techniques Using LED Lamps, Agilent Technologies (2001), the content of which is hereby incorporated herein by reference in its entirety for all purposes. The reference describes the use of light guides to direct and homogenize light emitted from LED lamps.
- Commercial LED light homogenizers are available, for example, from RPC
- polarizing lens is placed over one or more lights 104 disposed on the surface of a hyperspectral/multispectral imaging system to polarize light illuminating a target object or subject.
- Polarized illumination is advantageous because it eliminates surface reflection from the skin and helps to eliminate stray light reflection from off axis imaging directions.
- a second polarizer is placed in front of a corresponding optical detector 206 to depolarize the light prior to resolution of an image by the detector.
- a polarizing lens is placed over the objective lens or otherwise upstream of the beam steering element, such that a single polarizer is used to depolarize the image.
- individual polarizers are placed between the beam steering element and individual optical detectors.
- the detector polarizing lens is selected from a linear polarizer, a circular polarizer, and an elliptical polarizer.
- a surface of a turbid medium is imaged by illuminating the surface of the turbid medium with light, whereby light is backscattered from the illuminated surface of the turbid medium, detecting a pair of complementary polarization components of the
- the illuminating light is preferably polarized ⁇ e.g., linearly polarized, circularly polarized, elliptically polarized), where, for example, the illuminating light is linearly polarized, the pair of complementary polarization components are preferably the parallel and perpendicular components to the polarized illuminating light, and the image is formed by subtracting the perpendicular component from the parallel component, by taking a ratio of the parallel and perpendicular components or by using some combination of a ratio and difference of the parallel and perpendicular components.
- a wavelength filter is placed over the broadband light to increase the percentage of a desired wavelength of light.
- the illumination filter is matched to a corresponding detector filter, to increase the signal-to-noise ratio of an image captured at the desired wavelength.
- the illumination filter is selected from a bandpass filter, a longpass filter, and a shortpass filter.
- filters can be described in terms of their spectral "bandpass,” e.g., the span of component wavelengths allowed to pass through the filter.
- the bandpass of a filter is defined as the span of component wavelengths at which the filter is at least half as transparent as compared to the characteristic or center wavelength (FWHM).
- FWHM characteristic or center wavelength
- the spectral bandpass of a filter that is 100% transparent with respect to at least one component wavelength is the span of consecutive component wavelengths at which the filter is at least 50% transparent.
- the bandpass of a filter can be equivalently expressed in terms of the component wavelengths (e.g., 450-480 nm) or as the width of the bandpass at the central wavelength (e.g., 30 nm at 465 nm or ⁇ 15 nm at 465 nm).
- a bandpass filter can also be described in terms of its "characteristic wavelength,” e.g., the wavelength at which the filter is most transparent, or its “center wavelength,” e.g., the component wavelength at the midpoint of the spectral bandpass.
- the bandpass filter is characterized by both its characteristic or center wavelength and its spectral bandwidth. For example, a bandpass filter with a center wavelength of 340 ⁇ 2 nm, a FWHM bandwidth of 10 ⁇ 2, and a peak transmission (e.g., the maximum percentage transmission within the passband) of 50%, allows at least 25% of each component light having a wavelength from 330 ⁇ 4 nm to 350 ⁇ 4 nm to pass through.
- the illumination wavelength filter is a bandpass filter, e.g., a filter that allows only radiation having a wavelength in a certain range to pass, while blocking passage of other wavelengths.
- the FWHM spectral bandpass of an illumination filter e.g., the size of the passband transmitted through the filter
- the FWHM spectral bandpass of an illumination filter is no more than about 100 nm, preferably no more than about 50 nm, more preferably no more than about 25 nm.
- the FWHM spectral bandwidth of the filter is no more than 250 nm, 200 nm, 200 nm, 175 nm, 150 nm, 150 nm, 125 nm, 100 nm, 90 nm, 80 nm, 75 nm, 70 nm, 65 nm, 60 nm, 55 nm, 50 nm, 45 nm, 40 nm, 35 nm, 30 nm, 25 nm, 20 nm, 15 nm, 10 nm, 9 nm, 8 nm, 7 nm, 6 nm, 5 nm, 4 nm, 3 nm, 2 nm, or 1 nm.
- the bandpass filter is a narrow pass filter.
- the narrow pass filter has a FWHM spectral bandwidth of no more than 25 24 nm, 23 nm, 22 nm, 21 nm, 20 nm, 19 nm, 18 nm, 17 nm, 16 nm, 15 nm, 14 nm, 13 nm, 12 nm, 11 nm, 10 nm, 9 nm, 8 nm, 7 nm, 6 nm, 5 nm, 4 nm, 3 nm, 2 nm, or 1 nm.
- the plurality of bandpass illumination filters have central wavelengths that are separated by at least 10 nm, or at least 15 nm, 20 nm, 25 nm, 30 nm, 35 nm, 40 nm, 45 nm, 50 nm, 55 nm, 60 nm, 65 nm, 70 nm, 75 nm, 80 nm, 85 nm, 90 nm, 95 nm, 100 nm, or more.
- the hyperspectral/multispectral imaging system comprises a matching set of illumination bandpass filters and detector bandpass filters.
- respective illumination and detector bandpass filters having the same or substantially similar central or characteristic wavelengths and spectral bandpasses are used to provide in-band illumination and imaging.
- in-band illumination and imaging substantially reduces background noise while decreasing the required illumination and detector exposure times.
- the lens 106 may comprise a simple convex lens. In other words,
- the lens subsystem will include a compound lens.
- the compound lens can be a fixed focus lens or a variable focus lens.
- a fixed focus lens refers to a lens whose focus is set at the time of manufacture.
- the focus of a fixed focus lens will be set to the hyperfocal distance, which s a distance beyond which all objects can be brought into an acceptable focus.
- the aperture and the focal length of a fixed focus lens are reduced, so that the hyperfocal distance is small. This allows the depth of field to extend from infinity to a short distance. Because a fixed focus lens requires no moving parts, no electrical power is required to operate the lens. This is advantageous for building a low power
- hyperspectral/multispectral camera especially one that operates on battery power.
- the disadvantage of a fixed focus lens is that it cannot produce images that are as sharp as those obtained using a variable focus lens.
- variable focus lens also known as a zoom lens, refers to a compound lens assembly that can be manipulated to change the focal length. Variable focus is achieved by moving one or more lens relative to one or more additional lenses present in the lens assembly.
- variable focus lenses are more expensive to manufacture. However, the image quality obtainable by a variable focus lens is higher than that of a fixed focus lens.
- Variable focus lenses can be parfocal or varifocal.
- a parfocal lens also referred to as a true focus lens, maintains focus when the focal length is changes, whereas a varifocal lens losses focus during zooming.
- a variable focus lenses is a manual focus lens, e.g., one that requires the user to focus an image.
- Manual focus lenses can be designed to be controlled by rotating the barrel of a lens assembly, causing lenses within the lens assembly to move on tracks, or by instructing motors within the camera to move the lenses, for example with control buttons or input on a touch screen.
- variable focus lens is an autofocus lens, one that is focused electronically by a sensor, control system, and motor mounted within the camera.
- an autofocus lens can be controlled by a lens software module stored in the system memory 1314 of the processor subsystem 1300, which provides instructions to a motor mounted within the housing of the camera to move one or more focusing lenses within the object lens assembly 202 to achieve a desired focus.
- a hyperspectral/multispectral variable focus lens has settings for both automatic and manual focus adjustment modes.
- U.S. Patent No. 5,675,441 the content of which is hereby incorporated herein by reference in its entirety for all purposes, describes a focusing lens control apparatus that has an electric motor connected to a focusing lens in electrical communication with an automatic focus adjustment control, as well as a manual operation device which operates the motor mechanically in a manual operation mode.
- the light 1506 passes through the objective lens assembly (not shown) and is directed by a beam steering element 204, positioned in a first operating mode in a plurality of operating modes, towards an optical detector 206 configured to resolve light of the first wavelength ( ⁇ ).
- the beam steering element is positioned in its respective operating modes through the use of an actuator 1504 capable of adjust tip and tilt angles of the beam steering element.
- the actuator 1504 is a tip-tilt actuator that rotates the beam steering element 204 on two-axes. Through the movement of the actuator 1504, the beam steering element 204 is placed in optical communication with respective optical detectors 206.
- the actuator 1504 is a MEMS actuator, e.g., a comb type MEMS actuator (see, U.S. Patent No. 7,295,726 to Milanovic et ah, the content of which is hereby incorporated by reference in its entirety for all purposes) or a tip-tilt-piston MEMS actuator(see, Milanovic et ah, IEEE Journal of Selected Topics in Quantum Electronics, Vol. 10, No. 3, May/June 2004:462-471, the content of which is hereby incorporated by reference in its entirety for all purposes).
- the actuator 1504 is a gimbal device capable of tilting off of the vertical and horizontal axes.
- Texas Instruments Digits, Texas
- dual-axis analog MEMS pointing mirrors e.g., the TALP1000B
- the mirror is constructed of single crystal silicon, preferably having no grain boundaries.
- the mirrored surface is gold coated.
- the MEMS device is electromagnetically driven, allowing operation at low-voltage and low- power.
- the mirror is a micromirror or array thereof.
- the surface of the micromirror is no more than 0.01 mm 2 , 0.02 mm 2 , 0.03 mm 2 , 0.04 mm 2 , 0.05 mm 2 , 0.06 mm 2 , 0.07 mm 2 , 0.08 mm 2 , 0.09 mm 2 , 0.1 mm 2 , 0.2 mm 2 , 0.3 mm 2 , 0.4 mm2 , 0.5 mm2 , 0.6 mm2 , 0.7 mm2 , 0.8 mm2 , 0.9 mm2 , or 1.0 mm 2.
- the surface of the micromirror is no more than 0.01 mm 2 , 0.02 mm 2 , 0.03 mm 2 , 0.04 mm 2 , 0.05 mm 2 , 0.06 mm 2 , 0.07 mm 2 , 0.08 mm 2 , 0.09 mm 2 , 0.1 mm 2 , 0.2 mm 2 , 0.3 mm
- the surface of the mirror is at least 1 mm , 2 mm , 3 mm , 4 mm , 5 mm , 6 mm 2 , 7 mm2 , 8 mm2 , 9 mm2 , 10 mm 2 , 11 mm 2 , 12 mm 2 , 13 mm 2 , 14 mm 2 , 15 mm 2 , 16 mm 2 , 17 mm 2 , 18 mm 2 , 19 mm 2 , 20 mm 2 , 21 mm 2 , 22 mm 2 , 23 mm 2 , 24 mm 2 , 25 mm 2 , 30 mm 2 , 35 mm 2 , 40 mm 2 , 45 mm 2 , 50 mm 2 , or larger.
- control modules stored in the system memory 1314 control: the illumination, via an illumination control module 1324, the direction of the beam towards one or more optical detectors 206 via a beam steering control module 1326, and the image exposure time and optical detectors themselves via an optical detector control module 1328.
- the beam steering control module 1326 directs actuator 1504 to place the beam steering element 204 in various operating modes, each of which is in optical communication with one of the optical detectors 206.
- the illumination control module 1324 turns on a first light 104-4, emitting light 1506-4 at a first wavelength ( ⁇ ), illuminating a region of interest (ROI) 1502 on the subject 1500.
- Reflected or backscattered light 1506-4 from the subject 1500 enters the objective lens or assembly thereof (not shown) and hits the beam steering element 204, placed in a first operating mode by an actuator 1504 controlled by the beam steering control module 1326, which redirects the light onto an optical detector 206-4 configured to resolve light of wavelength ⁇ .
- the illumination control module 1324 then turns off the first light 206-4 and turns on a second light 206-5, emitting light 506-5 at a second wavelength ( ⁇ '), illuminating the ROI 1502. Simultaneously, the beam steering control module 1328 instructs the actuator 1504 to place the beam steering element 204 in a second operating mode, which is in optical communication with a second optical detector 206-5 configured to resolve light of wavelength ⁇ ' .
- the beam steering control module 1328 instructs the actuator 1504 to place the beam steering element 204 in a second operating mode, which is in optical communication with a second optical detector 206-5 configured to resolve light of wavelength ⁇ ' .
- the beam steering element 204 can be one or more reflective elements capable of redirecting the incident beam in one or more directions toward the detector(s).
- the beam steering element 204 is an element that reflects light in one or more directions (e.g., a mirror element).
- the beam steering element is a plain mirror capable of reflecting light over a wide range of wavelengths.
- the beam steering element is an array of mirrors, for example an array of micromirrors.
- the beam steering element consists of more than one element and is capable of simultaneously directing lights of different wavelengths in different directions.
- the beam steering element includes a first hot mirror and a second mirror positioned behind the hot mirror.
- the hot mirror is suitably coated to reflect light above or below a certain wavelength, while being transparent to light with lower or higher wavelengths, respectively.
- a hot mirror is coated to reflect infrared radiation, while allowing visible light to pass through.
- a hyperspectral/multispectral imaging system concurrently illuminates an OI 1502 with a first light source 104-4 emitting infrared radiation 1506-4 and a second light source 104-5 emitting visible light 1506-5.
- Reflected or backscattered infrared 1506-4 and visible 1506-5 light passing through the objective lens or system thereof (not shown) is projected onto the hot mirror (not shown), which reflects the infrared light 1506-4 onto a first optical detector 206-) configured to resolve the infrared light 1506-4.
- the visible light 1506-5 passes through the hot mirror and is reflected by a second mirror (not shown) placed in optical communication with a second optical detector 206-5 configured to resolve the visible light.
- the beam steering element is a chip comprising a micro electromechanical system (MEMS).
- MEMS micro electromechanical system
- the MEMS chip generally includes a two-dimensional array of tiltable mirrors which may be separately controlled.
- WSS wavelength selective switch
- U.S. Pat. No. 6,097,859 to Solgaard et ah the content of which is hereby incorporated herein by reference in its entirety for all purposes, describes the functional configuration of such a MEMS wavelength selective switch (WSS), which incorporates a wavelength from an incoming fiber and is capable of switching wavelength(s) to any one of multiple outgoing fibers.
- WSS wavelength selective switch
- the entire switching array of several hundred micro electromechanical system (MEMS) mirrors can be fabricated on a chip having dimension of less than one centimeter by techniques well developed in the
- the MEMS device is a digital light processing (DLP) technology, for example, a digital micromirror device (DMD) chip.
- DLP digital light processing
- a DMD chip consists of hundreds to hundreds of thousands of micromirrors arranged in a rectangular array.
- each of the mirrors on a DMD chip will correspond one-to-one with a pixel resolved by an optical detector.
- the mirrors in a DMD array are individually rotatable ⁇ 10-12 degrees about a single axis, allowing for a first and a second operating mode.
- DMDs used in light projectors operate each micromirror individually, with the two operating modes corresponding to a first "on" state, in which light is projected out through an aperture, and a second "off state, in which light is diverted elsewhere internal to the projector, for example, into a heat sink.
- Commercial DMD chips are sold, for example, by Texas Instruments (Dallas, Texas).
- the two rotatable states of a DMD mirror are each in optical communication with two optical detectors, respectively. Accordingly, two images, collected at two different wavelengths of light, can be collected simultaneously by either toggling a micromirror between its two rotatable states, or by positioning alternating micromirrors in optical communication with different optical detectors.
- the DMD chip is further positioned on an actuator having a plurality of operating modes. Thus, when the actuator is in a first operating mode, the DMD mirrors can be in optical communication with a first or a second optical detector, while in a second operating mode, the DMD mirrors can be in optical communication with a third or a fourth optical detector.
- the use of a MEMS chip having a two-dimensional array of tiltable mirrors allows for line scanning at one or more wavelengths simultaneously.
- a scanning device refers to line scanning (row or column scanner) or scanning the beam across the field of view of a detector.
- the imaging system line scans to a plurality of small optical detectors (e.g. , a single pixel sensor or photocell) so as to further decrease the size and power consumption of the device.
- a scanning mirror is used for line scanning across a field of view.
- the scanning mirror reflects light from one row of an OI toward an optical detector, which captures the image. After obtaining light from one row, the scanning mirror then rotates or otherwise moves in order to obtain light from a different row. The scanning mirror continues this rotation until light has been sequentially obtained from each row of the ROI.
- Mechanisms other than scan mirrors can be used for line scanning of an ROI, such as the focal plane scanner described in Yang et al., "A CCD Camera-based Hyperspectral Imaging System of Stationary and Airborne Applications," Geocarto International, Vol. 18, No. 2, June 2003, the content of which is hereby incorporated herein by reference in its entirety for all purposes.
- FIG. 7 shows an exploded view of the imager subsystem 210 of a co-axial hyperspectral/multispectral camera, according to some embodiments.
- the imager subsystem includes a plurality of optical detectors 206 mounted on a rigid flex printed circuit assembly 706 via rigid printed circuit board (PCB) islands 704 in electrical communication with a CPU (note shown), the CPU optionally mounted or otherwise in electrical communication with a motherboard (not shown), by means of a flex circuit or wire 208.
- the flex circuit-PCB island assembly is affixed onto a molded or machined fixture 700, optionally having individual slots 702 for mounting the respective PCB islands 704, to provide optical alignment with the various operating modes of the beam steering element 204, rigid mounting, and to provide vibration and shock resistance.
- the optical detectors 206 are arranged at a fixed distance from the beam steering element 204 and thus the objective lens 106 or objective lens assembly 202. In certain embodiments, all of the optical detectors 206 in the imager subassembly 210 are arranged at the same fixed distance from the beam steering element 204. The distance between the optical detector 206 and objective lens 106 or objective lens assembly 202, together with the size of the sensor elements that make up the optical detector 206, determines (in part) the spectral resolution of the imager subsystem.
- the spectral resolution which is the width (e.g., FWHM) of the component wavelengths collected by the optical detectors, is configured, e.g., by covering the optical detector 206 with an objective filter, so as to be sufficiently small to capture spectral features of medical conditions of interest.
- the sensed intensity of component wavelengths depends on many factors, including the light source intensity, the sensor element sensitivity at each particular component wavelength, the reflectance or transmittance of different sensor components such as filters, polarizers, and lenses, and the exposure time of the sensor element to the component wavelength. These factors are selected such that the imager subsystem 210 is capable of sufficiently determining the intensity of component wavelengths that can distinguish the spectral features of medical conditions of interest.
- wavelength filters are placed over the plurality of optical detectors to increase the percentage of a desired wavelength or spectral band of light reaching the optical detector.
- the detector filter is matched to a corresponding illumination filter, to increase the signal-to-noise ratio of an image captured at the desired wavelength.
- the detector filter is selected from a bandpass filter, a longpass filter, and a shortpass filter.
- the illumination wavelength filter is a bandpass filter, e.g., a filter that allows only radiation having a wavelength in a certain range to pass, while blocking passage of other wavelengths.
- the FWHM spectral bandpass of an illumination filter (e.g., the size of the passband transmitted through the filter) is no more than about 100 nm, preferably no more than about 50 nm, more preferably no more than about 25 nm.
- the FWHM spectral bandwidth of the filter is no more than 250 nm, 200 nm, 200 nm, 175 nm, 150 nm, 150 nm, 125 nm, 100 nm, 90 nm, 80 nm, 75 nm, 70 nm, 65 nm, 60 nm, 55 nm, 50 nm, 45 nm, 40 nm, 35 nm, 30 nm, 25 nm, 20 nm, 15 nm, 10 nm, 9 nm, 8 nm, 7 nm, 6 nm, 5 nm, 4 nm, 3 nm, 2 nm, or 1 nm.
- the bandpass filter is a narrow pass filter.
- the narrow pass filter has a FWHM spectral bandwidth of no more than 25 nm, 24 nm, 23 nm, 22 nm, 21 nm, 20 nm, 19 nm, 18 nm, 17 nm, 16 nm, 15 nm, 14 nm, 13 nm, 12 nm, 11 nm, 10 nm, 9 nm, 8 nm, 7 nm, 6 nm, 5 nm, 4 nm, 3 nm, 2 nm, or 1 nm.
- the plurality of bandpass illumination filters have central wavelengths that are separated by at least 10 nm, or at least 15 nm, 20 nm, 25 nm, 30 nm, 35 nm, 40 nm, 45 nm, 50 nm, 55 nm, 60 nm, 65 nm, 70 nm, 75 nm, 80 nm, 85 nm, 90 nm, 95 nm, 100 nm, or more.
- polarizing lenses are placed over one or more of the plurality of optical detectors 206 to de-polarize light backscattered from a tissue of a patient.
- a detector polarizing lens is matched to a corresponding illumination polarizing lens, to increase the signal-to-noise ratio of an image captured at the desired wavelength.
- the detector polarizing lens is selected from a linear polarizer, a circular polarizer, and an elliptical polarizer.
- the optical detector may consist of one pixel or a multiplicity of pixels such as a mega pixel array sensor. Suitable optical detectors are well known in the art, and include without limitation, a charge-coupled device (CCD), a complementary metal-oxide- semiconductor (CMOS), a focal plane array, and a photo-cell. Focal plane arrays and photocells are particularly well suited for a line scanning device, in which the image of the OI is scanned to one or more detectors simultaneously in rows and columns to create an image pixel-by-pixel.
- CCD charge-coupled device
- CMOS complementary metal-oxide- semiconductor
- Focal plane arrays and photocells are particularly well suited for a line scanning device, in which the image of the OI is scanned to one or more detectors simultaneously in rows and columns to create an image pixel-by-pixel.
- Examples of CCD in accordance with the present disclosure include, but are not limited to a Si CCD, InGaAs, and HgCdTe detectors.
- a suitable spectral range in some embodiments is 0.3 microns to 1 micron, 0.4 micron to 1 micron, 1 micron to 1.7 microns, or 1.3 microns to 2.5 microns.
- the detector contains between 320 and 1600 spatial pixels.
- the CCD has more or less spatial pixels.
- the detector has a field of view between 14 degrees and 18.4 degrees.
- the CCD samples at a rate of between 3 nm and 10 nm.
- the optical detectors are covered by detector filters, typically pass-band filters, configured for a wavelength of interest.
- detector filters typically pass-band filters
- one or more optical detectors mounted in the imaging system is not be covered by a filter and can be used to acquire a broadband image of the subject, used to focus an image of the subject, and/or used to zoom in or out on a OI.
- in-band illumination allows for greater signal-to-noise ratio and reduced exposure times, which in turn results in lower power consumption, reduced misalignment due to movement of the subject, and reduced computational burden when processing the resulting
- the hyperspectral/multispectral imaging systems described herein advantageously reduces the total amount of time required to collect a complete
- hyperspectral/multispectral imaging by determining the specific exposure time needed to resolve each sub-image of the hyperspectral/multispectral image.
- Each of the sub-images is collected at a different wavelength or wavelength band and, because of this, the amount of time needed to resolve each sub-image will vary as a function of wavelength.
- this variance is advantageously taken into account so that sub-images that require less time, because of their acquisition wavelengths or wavelength bands, are allotted shorter exposure times whereas sub-images that require more time because of their acquisition wavelengths or wavelength bands, are allotted shorter exposure times.
- non- transitory instructions encoded by the imager in non-transient memory determine the minimal exposure time required for image acquisition at each wavelength or each wavelength band used by the hyperspectral/multispectral imaging system.
- the exposure time used for image acquisition is the time period in which an optical detector 206 is actively resolving light received from the beam steering element 204. Accordingly, exposure time can be manipulated by controlling the amount of time the beam steering element is in optical communication with the particular optical detector, controlling the time an optical detector 206 is actively resolving light, or a combination thereof.
- One aspect provides a non-transitory computer readable storage medium storing one or more programs executable by a hyperspectral/multispectral imaging device with a central processing unit configured to execute the one or more programs and an optical acquisition subsystem configured to acquire a hyperspectral/multispectral image of a tissue of a subject.
- the hyperspectral/multispectral imaging device comprises a housing having an exterior and an interior and at least one light source attached to the housing (e.g. , disposed on the exterior of the housing.
- the hyperspectral/multispectral imaging device further comprises at least one objective lens attached to or within the housing, the at least one objective lens disposed in an optical communication path, the optical communication path comprising an originating end and a terminating end.
- the hyperspectral/multispectral imaging device further comprises a beam steering element within the interior of the housing, the beam steering element in optical communication with the terminating end of the optical communication path, the beam steering element characterized by a plurality of operating modes.
- hyperspectral/multispectral imaging device further comprises a plurality of optical detectors offset from the optical communication path, each respective optical detector in the plurality of optical detectors in optical communication with a corresponding operating mode of the beam steering element.
- the hyperspectral/multispectral imaging device further comprises a plurality of detector filters within the housing, each respective detector filter in the plurality of detector filters covering a corresponding optical detector in the plurality of optical detectors thereby filtering light received by the corresponding optical detector from the beam steering element so that each optical detector in the plurality of optical detectors is configured for recording a respective sub-image in the plurality of sub-images at the wavelength or wavelength band of the respective sub-image.
- the one or more programs comprising instructions for identifying a plurality of baseline exposure times, each respective baseline exposure time in the plurality of baseline exposure times representing an exposure time for resolving a respective sub-image, in the plurality of sub-images, of the tissue of the subject at the wavelength or wavelength band of the respective sub-image.
- a first baseline exposure time for a first sub-image is different than a second baseline exposure time of a second sub- image in the plurality of sub-images.
- the one or more programs further comprise
- a method (1700) for acquiring a hyperspectral/multispectral image of a tissue of a patient, the method comprising selecting (1702) a plurality of wavelengths or discrete wavelength bands for acquiring a
- hyperspectral/multispectral image of a tissue identifying (1704) minimum exposure times for resolving an image of the tissue at each wavelength or discrete wavelength band; identifying (1710) at least one factor affecting one of more minimum exposure times; adjusting (1718) the minimum exposure times based on the identified factors; and acquiring a
- the minimum exposure times are based on baseline
- the factor affecting the minimal exposure time is one or more of illumination provided by a device used to acquire the hyperspectral/multispectral image, ambient light, and concentration of melanin in the tissue.
- FIG. 13 schematically illustrates an exemplary embodiment of the processor subsystem 1300.
- the processor subsystem 1300 includes one or more central processing units (CPU) 1308, an optional main non-volatile storage unit 1340, for example a memory card, for storing software and data, the storage unit 1340 optionally controlled by storage controller 1342, and a system memory 1314 for storing system control programs, data, and application programs, including programs and data loaded from the non-volatile storage unit 1340.
- CPU central processing units
- main non-volatile storage unit 1340 for example a memory card
- storage controller 1342 optionally controlled by storage controller 1342
- system memory 1314 for storing system control programs, data, and application programs, including programs and data loaded from the non-volatile storage unit 1340.
- the processor subsystem 1300 optionally includes a user interface 1302 including one or more input devices 1306 ⁇ e.g., a touch screen, buttons, or switches) and/or a display 1304, although in certain embodiments, the hyperspectral/multispectral imaging system may be controlled by an external device such as a handheld device, laptop computer, or desktop computer.
- the processor subsystem 1300 includes one or more communication interfaces 1312 for connecting to any wired or wireless external device ⁇ e.g., a handheld device, laptop computer, or desktop computer) or communication network ⁇ e.g. , a wide area network such as the Internet) 1313.
- the processor subsystem 1300 includes a power source 1307 to power the aforementioned elements and an internal bus 1310 for interconnecting the aforementioned elements of the system.
- the communication bus 1310 may include circuitry (sometimes called a chipset) that interconnects and controls communications between system
- the processor subsystem 1300 communicates with a communication network 1313, thereby enabling the processor subsystem 1300 to transmit and/or receive data between mobile communication devices over the communication network, particularly one involving a wireless link, such as cellular, WiFi, ZigBee, BluTooth, IEEE 802.1 lb, 802.1 la, 802.1 lg, or 802.1 In, etc.
- the communication network can be any suitable communication network configured to support data transmissions. Suitable communication networks include, but are not limited to, cellular networks, wide area networks (WANs), local area networks (LANs), the Internet, IEEE 802.1 lb, 802.1 la, 802.1 lg, or 802.1 In wireless networks, landline, cable line, fiber-optic line, etc.
- the imaging system depending on an embodiment or desired functionality, can work completely offline by virtue of its own computing power, on a network by sending raw or partially processed data, or both simultaneously.
- Memory 1314 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid state memory devices; and typically includes nonvolatile memory flash memory devices, or other non-transitory solid state storage devices. Memory 1314 optionally includes one or more storage devices remotely located from the CPU(s) 1308. Memory 1314, or alternately the non-transitory memory device(s) within memory 1314, comprises a non-transitory computer readable storage medium.
- operation of the processor subsystem 1300 is controlled primarily by an operating system 1320, which is executed by a central processing unit 1308.
- the operating system 1320 can be stored in a system memory 1314.
- processor subsystem 1300 is not controlled by an operating system.
- the system memory 1314 includes one or more of a file system 1322 for controlling access to the various files and data structures described herein, an illumination software control module 1324 for controlling the light source(s) described herein, a beam steering software control module 1326 for controlling the operating modes of the beam steering element, an optical detector software control module 1328 for controlling the optical detectors and reading digital images acquired thereby, a digital image data store 1331 for storing digital images 1332 acquire by the optical detectors, a data processing software module 1334 for manipulating an acquired image or set of images, a file system 1322 for controlling access to the various files and data structures described herein, an illumination software control module 1324 for controlling the light source(s) described herein, a beam steering software control module 1326 for controlling the operating modes of the beam steering element, an optical detector software control module 1328 for controlling the optical detectors and reading digital images acquired thereby, a digital image data store 1331 for storing digital images 1332 acquire by the optical detectors, a data processing software module 1334 for manipulating an acquired image or set of images,
- hyperspectral/multispectral data cube data store 1335 for storing hyperspectral/multispectral data cubes 1336 assembled from a plurality of hyperspectral/multispectral data planes, and a communication interface software control module 1338 for controlling the communication interface 1312 that connects to an external device (e.g., a handheld device, laptop computer, or desktop computer) and/or communication network (e.g. , a wide area network such as the Internet.
- an external device e.g., a handheld device, laptop computer, or desktop computer
- communication network e.g., a wide area network such as the Internet.
- the acquired digital images 1332 and hyperspectral/multispectral data cubes 1336 can be stored in a storage module in the system memory 1314, and do not need to be concurrently present, depending on which stages of the analysis the processor subsystem 1300 has performed. In fact, in some embodiments, prior to imaging a subject and after communicating acquired digital images or processed data files thereof, the
- hyperspectral/multispectral imaging system contains neither acquired digital images 1332 nor hyperspectral data cubes 1336.
- the hyperspectral/multispectral imaging system contains neither acquired digital images 1332 nor hyperspectral data cubes 1336.
- hyperspectral/multispectral imaging system retains acquired digital images 1332 and/or hyperspectral data cubes 1336 for a period of time (e.g., until storage space is needed, for a predetermined amount of time, etc.).
- the programs or software modules identified above correspond to sets of instructions for performing a function described above.
- the sets of instructions can be executed by one or more processors, e.g., a CPU(s) 1308.
- the above identified software modules or programs (e.g. , sets of instructions) need not be implemented as separate software programs, procedures, or modules, and thus various subsets of these programs or modules may be combined or otherwise re-arranged in various embodiments.
- a memory 1314 stores a subset of the modules and data structures identified above.
- a memory 1314 may store additional modules and data structures not described above.
- the system memory 1314 optionally also includes one or more of the following software modules, which are not illustrated in Figure 13: a spectral library which includes profiles for a plurality of medical conditions, a spectral analyzer software module to compare measured hyperspectral/multispectral data to a spectral library, control modules for additional sensors; information acquired by one or more additional sensors, an image constructor software module for generating a hyperspectral/multispectral image, a
- hyperspectral/multispectral image assembled based on a hyperspectral/multispectral data cube and optionally fused with information acquired by an additional sensor, a fusion software control module for integrating data acquired by an additional sensor into a hyperspectral/multispectral data cube, and a display software control module for controlling a built-in display.
- a physician can optionally provide input to processor subsystem 1300 that modifies one or more parameters upon which a hyperspectral/multispectral image and/or diagnostic output is based. In some embodiments, this input is provided using input device 1306.
- the processor subsystem 1300 can be instructed to modify the spectral portion selected by a spectral analyzer (e.g., to modify a threshold of analytical sensitivity) or to modify the appearance of the image generated by an image constructor (e.g., to switch from an intensity map to a topological rendering).
- a spectral analyzer e.g., to modify a threshold of analytical sensitivity
- an image constructor e.g., to switch from an intensity map to a topological rendering
- the processor subsystem 1300 can be instructed to communicate instructions to the imager subsystem 210 to modify the sensing properties of one of the optical detectors (e.g., an exposure setting, a frame rate, an integration rate, or a wavelength to be detected). Other parameters can also be modified. For example, the processor subsystem 1300 can be instructed to obtain a wide-view image of the subject for screening purposes, or to obtain a close-in image of a particular region of interest.
- the processor subsystem 1300 can be instructed to obtain a wide-view image of the subject for screening purposes, or to obtain a close-in image of a particular region of interest.
- processor subsystem 1300 does not include a controller 1342 or memory 1340.
- memory 1314 and CPU 1308 are, in fact, one or more application-specific integrated circuit chips (ASICs).
- ASICs application-specific integrated circuit chips
- an ASIC includes the instructions of illumination control module 1324, beam steering control module 1326, optical detector control module 1328, data processing module 1334 and/or communication interface control module 1338.
- the ASIC further includes storage space for the acquired digital image data store 1331 and the digital images 1332 stored therein and/or the hyperspectral/multispectral data cube data store 1335 and the hyperspectral/multispectral data cubes 1336 stored therein.
- memory 1314 include a spectral library and spectral analyzer for comparing hyperspectral/multispectral data acquired by the imaging system to known spectral patterns associated with various medical conditions.
- analysis of the acquired hyperspectral/multispectral data is performed on an external device such as a handheld device, tablet computer, laptop computer, desktop computer, an external server, for example in a cloud computing environment.
- a spectral library includes profiles for a plurality of medical conditions, each of which contain a set of spectral characteristics unique to the medical condition.
- a spectral analyzer uses the spectral characteristics to determine the probability or likelihood that a region of the subject corresponding to a measured
- each profile includes additional information about the condition, e.g., information about whether the condition is malignant or benign, options for treatment, etc.
- each profile includes biological information, e.g., information that is used to modify the detection conditions for subjects of different skin types.
- the spectral library is stored in a single database. In other embodiments, such data is instead stored in a plurality of databases that may or may not all be hosted by the same computer, e.g. , on two or more computers addressable by wide area network.
- the spectral library is electronically stored in a non-volatile storage unit 1340 and recalled using a storage controller 1342 when needed.
- a spectral analyzer analyzes a particular spectra derived from a hyperspectral/multispectral data cube, the spectra having pre-defined spectral ranges (e.g., spectral ranges specific for a particular medical condition), by comparing the spectral characteristics of a pre-determined medical condition to the subject's spectra within the defined spectral ranges. Performing such a comparison only within defined spectral ranges can both improve the accuracy of the characterization and reduce the computational power needed to perform such a characterization.
- pre-defined spectral ranges e.g., spectral ranges specific for a particular medical condition
- the spectral characteristics of a medical condition can be determined, for example, by first identifying an actual condition of that type on another subject, for example using conventional visual examination, and then obtaining the wavelength-dependent backscattering RMC( ⁇ ) of a representative region of the skin affected by the medical condition.
- the backscattering of the affected skin RMC -) can then be spectrally compared to the wavelength-dependent backscattering of that subject's normal skin in the same area of the lesion, R N ⁇ X), by normalizing the backscattering of the affected skin against the backscattering of normal skin as follows: where RMCN -) is the normalized backscattering of the affected skin.
- RMQNQ * [RMC(A-)-RNS(A-)]/[RMC -)+RN )] .
- Other types of normalization are possible. Note that if there are multiple representative regions of affected skin, there will be as many normalized backscatterings of the affected skin. These normalized backscatterings can be averaged together, thus accounting for the natural spectral variation among different regions of the affected skin.
- the spectral analyzer in order to determine whether the subject has the type of medical condition characterized by RMCN -), obtains the skin reflectance of each region, R re gion(fy, from a measured hyperspectral cube or data plane (1336). The spectral analyzer then normalizes the backscattering R re gion(fy from that region against the
- RNS wavelength-dependent backscattering of the subject's normal skin in the same area
- RNS sub j ectify, as follows: where R reg ion , N(fy is the normalized backscattering of the region.
- R reg ion is the normalized backscattering of the region.
- Other types of normalization are possible.
- the spectral analyzer analyzes the subjects' spectra by comparing R reg ion,N(h) to I n one simple example, the comparison is done by taking the ratio R re gion,N )IRMC,N »), or the difference RMCN *)- Rre g ion,N fy-
- the magnitude of the ratio or difference indicates whether any region has spectral characteristics that match that of affected skin.
- ratios and differences are simple calculations, the result of such a calculation is complex and requires further analysis before a diagnosis can be made.
- the ratio or subtraction of two spectra, each of which has many peaks generates a calculated spectrum that also has many peaks. Some peaks in the calculated spectrum may be particularly strong (e.g. , if the subject has the medical condition characterized by
- peaks may also be present (e.g., due to noise, or due to some particular characteristic of the subject).
- a physician in the examination room would typically find significantly more utility in a simple "yes/no" answer as to whether the subject has a medical condition, than he would in a complex spectrum.
- One method of obtaining a "yes/no" answer is to calculate whether a peak in the calculated spectrum has a magnitude that is above or below a predetermined threshold and is present at a wavelength that would be expected for that medical condition.
- Another way to obtain a "yes/no" answer is to treat as vectors, and to determine the "angle" between the vectors.
- the angle represents the degree of overlap between the vectors, and thus represents how likely it is that the subject has the medical condition. If the angle is smaller than a threshold value, the subject is deemed have the medical condition; if the angle does not exceed a threshold value, the subject is deemed not to have the medical condition. Alternately, based on the value of the angle between the vectors, a probability that the subject has the medical condition can be determined.
- a spectral library may comprise a personalized database of spectral information that is collected for a particular subject.
- the personalized database can then be used to monitor changes in the subject over time. Changes over time in the spectral characteristics of a region on the subject can be used to provide information, for example, on the progression or regression of a medical condition, the efficacy of treatment, and the appearance of a new medical condition (e.g., the formation of an ulcer).
- the provided information can be used to inform the medical treatment of the subject.
- U.S. Patent Publication No. 2009/0326383 the content of which is hereby incorporated herein by reference in its entirety for all purposes.
- the spectral analyzer includes a trained data analysis algorithm for identifying a region on the subject's skin of biological interest using an image obtained by the apparatus and/or for determining a portion of a hyperspectral/multispectral data cube that contains information about a biological insult in the subject's skin.
- a trained data analysis algorithm for identifying a region on the subject's skin of biological interest using an image obtained by the apparatus and/or for determining a portion of a hyperspectral/multispectral data cube that contains information about a biological insult in the subject's skin.
- such pattern classification techniques and/or statistical techniques can be used to (i) assist in identifying a medical condition of a subject, (ii) assist in characterizing a medical condition of a subject, and (iii) assist in analyzing the progression of a medical condition of a subject (e.g., detect changes in tissue composition or a wound on the skin of a patient over time).
- assist in identifying a medical condition of a subject e.g., identify changes in tissue composition or a wound on the skin of a patient over time.
- Pattern classification is used to mine a spectral library to identify and characterize medical conditions (e.g., ischemia, an ulcer, diabetes, etc.) that are characterized by observable hyperspectral/multispectral signatures.
- medical conditions e.g., ischemia, an ulcer, diabetes, etc.
- hyperspectral/multispectral signatures are values of specific pixels in an image of a subject's skin, patterns of values of specific groups of pixels in an image of a subject's skin, values of specific measured wavelengths or any other form of observable data that is directly present in the spectral data and/or that can be derived from the spectral data taken of a subject's skin.
- pattern classification techniques such as artificial intelligence are used to analyze hyperspectral/multispectral data cubes, the output of other sensors or cameras, and/or hyperspectral/multispectral images themselves (which may or may not be fused with other information), further details, see: U.S. Patent Publication No. 2009/0326383; National Research Council; Panel on Discriminant Analysis Classification and Clustering,
- Relevant algorithms for decision rules include, but are not limited to: discriminant analysis including linear, logistic, and more flexible discrimination techniques (see, e.g. , Gnanadesikan, 1977, Methods for Statistical Data Analysis of Multivariate Observations, New York: Wiley 1977; tree-based algorithms such as classification and regression trees (CART) and variants (see, e.g., Breiman, 1984, Classification and Regression Trees, Belmont, California: Wadsworth International Group); generalized additive models (see, e.g. , Tibshirani , 1990, Generalized Additive Models, London: Chapman and Hall); neural networks (see, e.g., Neal, 1996, Bayesian Learning for Neural Networks, New York:
- data analysis algorithms include, but are not limited to, ANOVA and nonparametric equivalents, linear discriminant analysis, logistic regression analysis, nearest neighbor classifier analysis, neural networks, principal component analysis, quadratic discriminant analysis, regression classifiers and support vector machines.
- ANOVA and nonparametric equivalents linear discriminant analysis
- logistic regression analysis nearest neighbor classifier analysis
- nearest neighbor classifier analysis neural networks
- principal component analysis principal component analysis
- quadratic discriminant analysis regression classifiers and support vector machines.
- Such algorithms may be used to construct a decision rule and/or increase the speed and efficiency of the application of the decision rule and to avoid investigator bias, one of ordinary skill in the art will realize that computer-based algorithms are not required to carry out the disclosed methods.
- suitable data analysis algorithms operable by the central processing unit(s) 1308 of the hyperspectral/multispectral imaging systems described herein, or by external devices or servers are used, for example, to detect the location and/or severity of diabetic foot ulcers or pressure ulcers.
- suitable data analysis algorithms are used to predict the possible formation of diabetic foot ulcers or pressure ulcers.
- suitable data analysis algorithms for these purposes are found in Yudovsky D. et al, J Diabetes Sci Technol. (2010) Sep 1; 4(5): 1099-113; Yudovsky D. et al, J Biomed Opt. (2011) Feb; 16(2):026009, and Yudovsky D. et al, J Biophotonics (2011) Aug; 4(7-8):565-76, the contents of which are hereby incorporated herein by reference in their entireties for all purposes.
- the hyperspectral/multispectral imaging system comprises a display 1304 which receives an image ⁇ e.g., a color image, mono-wavelength image, or hyperspectral/multispectral image) from a display control module, and displays the image.
- the display subsystem also displays a legend that contains additional information.
- the legend can display information indicating the probability that a region has a particular medical condition, a category of the condition, a probable age of the condition, the boundary of the condition, information about treatment of the condition, information indicating possible new areas of interest for examination, and/or information indicating possible new information that could be useful to obtain a diagnosis, e.g., another test or another spectral area that could be analyzed.
- a housing display is built into the housing of a
- a video display in electronic communication with the processor 1308 is mounted on the back-side of a co-axial camera, as described herein.
- the housing display is a touchscreen display that is used to manipulate the displayed image and/or control the hyperspectral/multispectral system.
- the communication interface 1312 comprises a docking station for a mobile device having a mobile device display.
- a mobile device such as a smart phone, a personal digital assistant (PDA), an enterprise digital assistant, a tablet computer, an IPOD, a digital camera, or a portable music player, can be connected to the docking station, effectively mounting the mobile device display onto the hyperspectral/multispectral imaging system (e.g., camera).
- the mobile device is used to manipulate the displayed image and/or control the hyperspectral/multispectral system.
- the processor subsystem 1300 is configured to be in wired or wireless communication with an external display, for example, on a handheld device, tablet computer, laptop computer, desktop computer, television, IPOD, or projector unit, on which the image is displayed.
- an external display for example, on a handheld device, tablet computer, laptop computer, desktop computer, television, IPOD, or projector unit, on which the image is displayed.
- a user interface on the external device is used to manipulate the displayed image and/or control the hyperspectral/multispectral system.
- an image can be displayed in real time on the display.
- the realtime image can be used, for example, to focus an image of the subject, to select an appropriate region of interest, and to zoom the image of the subject in or out.
- the real-time image of the subject is a color image captured by an optical detector that is not covered by a detector filter.
- the imager subsystem comprises an optical detector dedicated to capturing true color images of a subject.
- the real-time image of the subject is a monowavelength, or narrow-band (e.g., 10-50 nm), image captured by an optical detector covered by a detector filter.
- any optical detector covered by a detector filter in the imager subsystem may be used for: (i) resolving digital images of the subject for integration into a hyperspectral/multispectral data cube; and (ii) resolving narrow-band images for focusing, or otherwise manipulating the optical properties of the imaging system.
- a hyperspectral/multispectral image constructed from data collected by the imaging system is displayed on an internal housing display, mounted housing display, or external display.
- Assembled hyperspectral/multispectral data (e.g., present in a hyperspectral/multispectral data cube) is used to create a two-dimensional representation of the imaged object or subject, based on one or more parameters.
- An image constructor module stored in the imaging system memory or in an external device, constructs an image based on, for example, an analyzed spectra. Specifically, the image constructor creates a representation of information within the spectra.
- the image constructor constructs a two-dimensional intensity map in which the spatially-varying intensity of one or more particular wavelengths (or wavelength ranges) within the spectra is represented by a corresponding spatially varying intensity of a visible marker.
- the image constructor fuses a hyperspectral image with information obtained from one or more additional sensors.
- suitable image fusion methods include: band overlay, high-pass filtering method, intensity hue- saturation, principle component analysis, and discrete wavelet transform.
- the display subsystem includes a touchscreen video display that can be manipulated by the user, for example, to focus an image, zoom in or out within an image, select a region of an image for further analysis, change the contrast of the image, change a parameter of a hyperspectral/multispectral image (e.g., the mode, spectral bands represented, artificial coloring, etc.).
- the display subsystem comprises a touchscreen overlaying a second video display, such as an LCD display, the touchscreen and second video display having their own circuitry and patterns.
- Touchscreens use various technologies to sense touch from a finger or stylus, such as resistive, capacitive, infrared, and acoustic sensors. Resistive sensors rely on touch to cause two resistive elements overlaying the display to contact one another completing a resistive circuit, while capacitive sensors rely on the capacitance of a finger changing the capacitance detected by an array of elements overlaying the display device. Infrared and acoustic touchscreens similarly rely on a finger or stylus to interrupt infrared or acoustic waves across the screen, indicating the presence and position of a touch.
- Resistive sensors rely on touch to cause two resistive elements overlaying the display to contact one another completing a resistive circuit
- capacitive sensors rely on the capacitance of a finger changing the capacitance detected by an array of elements overlaying the display device.
- Infrared and acoustic touchscreens similarly rely on a finger or stylus to interrupt infrared or acoustic waves
- Capacitive and resistive touchscreens often use transparent conductors such as indium tin oxide (ITO) or transparent conductive polymers such as PEDOT to form an array over the display image, so that the display image can be seen through the conductive elements used to sense touch.
- ITO indium tin oxide
- PEDOT transparent conductive polymers
- the size, shape, and pattern of circuitry have an effect on the accuracy of the touchscreen, as well as on the visibility of the circuitry overlaying the display.
- a single layer of most suitable conductive elements is difficult to see when overlaying a display, multiple layers can be visible to a user, and some materials such as fine line metal elements are not transparent but rely on their small size to avoid being seen by users.
- the hyperspectral/multispectral imaging system is mountable on a tripod or other fixed structure.
- the tripod is a fixed sensor tripod or a fixed sensor tripod on wheels.
- the hyperspectral sensor is mountable on a mobile or fixed rack.
- a mobile hyperspectral/multispectral imaging device may be mounted on a rack or other permanent fixture in an examination room.
- Figure 14 illustrates an overview of a method (1400) for providing information on the medical condition of a subject, using hyperspectral/multispectral imaging.
- a digital image of an ROI on a subject is captured (1402) at a first wavelength using a first matching illumination source and optical detector included in a hyperspectral/multispectral imaging system as described herein.
- the particular area of the subject comprises several different contiguous or non-contiguous regions of the subject's skin.
- this image is obtained by illuminating an ROI with a first narrowband light source disposed on the imaging system and directing light backscattered from the ROI to a first optical detector configured to resolve the narrowband light ⁇ e.g., an optical detector covered by a filter matching the narrowband illumination source) using a beam steering element positioned in a first operating mode in optical communication with the first optical detector.
- a first narrowband light source disposed on the imaging system and directing light backscattered from the ROI to a first optical detector configured to resolve the narrowband light ⁇ e.g., an optical detector covered by a filter matching the narrowband illumination source
- the quantity and identity of required images resolved at unique wavelengths depends upon the medical condition(s) being evaluated. In certain embodiments, for example, where an ischemic condition is being evaluated by measuring oxyhemoglobin and deoxyhemoglobin levels at an ROI, as little as three wavelengths are required for the analysis. In other embodiments, for example, where potentially cancerous skin legions are being discriminated from other pigmented legions, many more images may be required (see, Nagaoka T. et al, supra).
- data contained with the acquired digital images is then pre-processed (1406) by a central processing unit (CPU) present in the hyperspectral/multispectral imaging system.
- the digital images may be processed, for example, to adjust the brightness of the acquired digital image(s), adjust the contrast of the acquired digital image(s), remove an artifact from the acquired digital image(s), crop the acquired digital image(s), processing one or more sub-pixels of the acquired digital image(s), compress the size of the acquired digital image(s), assemble a plurality of acquired digital images into a spectral hypercube, transform a spectral hypercube assembled from the plurality of acquired digital images, format data contained within the acquired digital image(s); and/or encrypt data contained within the acquired digital image.
- Non-limiting examples of external devices suitable for processing hyperspectral/multispectral data include: handheld devices such as smart phone, personal digital assistants (PDA), IPHONES, enterprise digital assistants, tablet computers, desktop computers, laptop computers, servers containing multiple CPUs, and cloud computing environments. In certain embodiments, processing and analysis is broken- up between multiple devices.
- a hyperspectral/multispectral imaging system transmits processed or unprocessed digital images to a handheld device (e.g. , a smart phone).
- the smart phone may perform additional processing steps and/or initial analysis of the data and then transmit intermediate data files to a second device or server, for example, a cloud computing environment, to perform computationally taxing analysis.
- the external device or server determines whether the data were transmitted in a format suitable for hyperspectral/multispectral analysis, e.g., a pre-assembled spectral data cube. If the data are not present in a suitable format 1410-No, the external device or server format the data accordingly in step 1412, for example, by assembling the data into a spectral data cube.
- Measurements from the plurality of digital images may be processed according to methods such as clustering analysis, principal component analysis or any analytic methods that are suitable for processing such measurements. In some embodiments, the measurements are normalized before any analytical processing is performed.
- the hyperspectral/multispectral data is compared, in step 1414, to one or more spectral signatures in a library of spectral signatures in which each spectral signature in the library corresponds to one or more spectral characteristics of a medical condition.
- a medical condition is identified when the comparison results in identification of a spectral signature from the spectral signature library that most resembles a spectrum in the measured hyperspectral/multispectral data. Results from such a comparison can be represented by a similarity metric, such as a ratio or a similarity matrix.
- a region can be identified as a suspect region when the similarity metric between the region and a spectral signature for a medical condition is, e.g., a ratio of 0.4 or above, 0.5 or above, 0.6 or above, 0.7 or above, 0.8 or above, 0.9 or above, where 1.0 represents complete identity between the region and the spectral signature of the medical condition.
- the threshold for the similarity metric can differ with respect to the type of tissues under analysis. For example, it may be more difficult obtaining signals for a certain type of tissues such that the signal to noise ratio for the particular tissue may be low; thus ultimately resulting in a low similarity metrics value.
- measurements of a plurality of regions are resolved into spectra where each spectrum corresponds to a different region.
- the spectrum corresponding to a particular region is then compared to a signature spectrum corresponding to a medical condition.
- the signature spectrum is obtained by taking measurements of a region that is known to have the medical condition. It will be understood by one of skill in the art that any suitable analytical methods can be used for obtaining the spectral signature of the medical condition.
- a similarity metric e.g., a correlation coefficient
- a similarity metric e.g., a correlation coefficient
- a plurality of features can be calculated from the measurements of a region in the plurality of regions. These same features can be calculated from the spectral signatures.
- One example of the feature is an observed or recorded intensity at a particular wavelength.
- Another example of a feature is the ratio of one observed or recorded intensity at one particular wavelength divided by another observed or recorded intensity at another particular wavelength.
- Another example of a feature is some mathematical function of two or more observed or recorded intensities at unique
- additional information collected from the ROI on the subject is used to assist in the identification of a medical condition.
- the additional data is integrated with the hyperspectral/multispectral data measured from the plurality of images acquired of the ROI and the integrated data is compared to a library of spectral signatures containing integrated spectral and non-spectral signatures.
- the additional data is analyzed separately from the hyperspectral/multispectral data, for example compared to a separate library of like signatures, for further characterize or confirm a particular medical condition.
- Additional data can be collected using any combination of sensor, for example, a LIDAR sensor, a thermal imaging sensor, a millimeter- wave (microwave) sensor, a color sensor, an X-ray sensor, a UV sensor, a NIR sensor, a SWIR sensor, a MWIR sensor, and a LWIR sensor.
- the additional sensor is integrated into the hyperspectral/multispectral imaging system.
- the additional data is collected using a separate device.
- hyperspectral/multispectral data are provided in U.S. Patent Application Publication No. 2009/0326383 to Barnes et ah, the content of which is hereby incorporated herein by reference in its entirety for all purposes.
- information on the medical condition is provided to a healthcare professional (1416).
- This information can be, for example, information that provides or aids in providing a diagnosis for the patient, information that provides a likelihood for a particular diagnosis, information that provides or aids in providing a prognosis for disease progression, information that provides or aids in providing a prognosis for a particular treatment course, information that provides or aids in providing an optimal treatment course, where more than one treatment course is available, and information that provides a likelihood of remission or disease-free survival after treatment.
- a medical professional e.g., a physician
- the treatment plan may call for the excision of the lesion based on the boundaries shown in the hyperspectral/multispectral medical image.
- hyperspectral/multispectral images of a subject, or OI thereof are taken at a single time point, to evaluate the subject at that particular point in time.
- multiple hyperspectral/multispectral images of a subject, or ROI thereof are taken over a period of time, for example, separated by a minute, hour, day, week, month, year, or decade, to monitor, for example, the overall health of the subject, progression of a medical condition (e.g., progression of a disease), regression of a medical condition, the efficacy of a treatment plan, or to proactively monitor the subject for a medical condition.
- a medical condition e.g., progression of a disease
- regression of a medical condition e.g., the efficacy of a treatment plan
- the feet of a subject diagnosed with diabetes are periodically imaged by hyperspectral/multispectral techniques described herein to monitor for initial signs of the formation of a diabetic foot ulcer, which occur in fifteen percent of all diabepatients (Brem and Tomic-Canic, J Clin Invest. 2007 May; 1 17(5): 1219-22).
- the patient's feet are imaged at least once a week, at least once a month, at least once every three months, at least once every six months, or at least once a year.
- the present disclosure provides systems and methods for hyperspectral/multispectral medical imaging. These methods are based on distinguishing the different interactions that occur between light at different wavelengths and components of the human body, especially components located in or just under the skin. For example, it is well known that deoxyhemoglobin absorbs a greater amount of light at 700 nm than does water, while water absorbs a much greater amount of light at 1200 nm, as compared to
- deoxyhemoglobin By measuring the absorbance of a two-component system consisting of deoxyhemoglobin and water at 700 nm and 1200 nm, the individual contribution of deoxyhemoglobin and water to the absorption of the system, and thus the concentrations of both components, can readily be determined.
- the individual components of more complex systems e.g., human skin
- hyperspectral/multispectral imaging measures hyperspectral/multispectral signature, when the data is constructed into a hyperspectral/multispectral data cube.
- regions e.g., different OI on a single subject or different ROI from different subjects
- the region physiological structure of the region, and/or the presence of a chemical in the region.
- a chemical in the region For example, fat, skin, blood, and flesh all interact with various wavelengths of light differently from one another.
- a given type of cancerous lesion interacts with various wavelengths of light differently from normal skin, from non-cancerous lesions, and from other types of cancerous lesions.
- a given chemical that is present e.g., in the blood, or on the skin
- the light obtained from each illuminated region of a subject has a spectral signature based on the characteristics of the region, which signature contains medical information about that region.
- the structure of skin while complex, can be approximated as two separate and structurally different layers, namely the epidermis and dermis. These two layers have very different scattering and absorption properties due to differences of composition.
- the epidermis is the outer layer of skin. It has specialized cells called melanocytes that produce melanin pigments. Light is primarily absorbed in the epidermis, while scattering in the epidermis is considered negligible. For further details, see G.H. Findlay, "Blue Skin,” British Journal of Dermatology 83(1), 127-134 (1970), the content of which is incorporated herein by reference in its entirety for all purposes.
- the dermis has a dense collection of collagen fibers and blood vessels, and its optical properties are very different from that of the epidermis. Absorption of light of a bloodless dermis is negligible. However, blood-born pigments like oxy- and deoxy-hemoglobin and water are major absorbers of light in the dermis. Scattering by the collagen fibers and absorption due to chromophores in the dermis determine the depth of penetration of light through skin.
- Light used to illuminate the surface of a subject will penetrate into the skin.
- the extent to which the light penetrates will depend upon the wavelength of the particular radiation. For example, with respect to visible light, the longer the wavelength, the farther the light will penetrate into the skin. For example, only about 32% of 400 nm violet light penetrates into the dermis of human skin, while greater than 85% of 700 nm red light penetrates into the dermis or beyond (see, Capinera J.L., Encyclopedia of Entomology, 2 nd Edition, Springer Science (2008) at page 2854, the content of which is hereby incorporated herein by reference in its entirety for all purposes).
- a tissue when referring to “illuminating a tissue,” “reflecting light off of the surface,” and the like, it is meant that radiation of a suitable wavelength for detection is backscattered from a tissue of a subject, regardless of the distance into the subject the light travels. For example, certain wavelengths of infra-red radiation penetrate below the surface of the skin, thus illuminating the tissue below the surface of the subject.
- light from the illuminator(s) on the systems described herein penetrates the subject's superficial tissue and photons scatter in the tissue, bouncing inside the tissue many times. Some photons are absorbed by oxygenated hemoglobin molecules at a known profile across the spectrum of light. Likewise for photons absorbed by de-oxygenated hemoglobin molecules.
- the images resolved by the optical detectors consist of the photons of light that scatter back through the skin to the lens subsystem. In this fashion, the images represent the light that is not absorbed by the various chromophores in the tissue or lost to scattering within the tissue.
- light from the illuminators that does not penetrate the surface of the tissue is eliminated by use of polarizers. Likewise, some photons bounce off the surface of the skin into air, like sunlight reflecting off a lake.
- wavelengths of light may be used to examine different depths of a subject's skin tissue.
- high frequency, short-wavelength visible light is useful for investigating elements present in the epidermis
- lower frequency, long-wavelength visible light is useful for investigating both the epidermis and dermis.
- certain infra-red wavelengths are useful for investigating the epidermis, dermis, and subcutaneous tissues.
- VNI visible and near- infrared
- major light-tissue interactions include reflection, refraction, scattering and absorption.
- the regular reflection of the skin at the air-tissue interface is typically only around 4%-7% in the 250- 3000 nanometer (nm) wavelength range.
- the steady state VNIR skin reflectance can be modeled as the light that first survives the absorption of the epidermis, then reflects back toward the epidermis layer due the isotropic scattering in the dermis layer, and then finally emerges out of the skin after going through the epidermis layer again.
- R ⁇ 1) T E 2 ⁇ )R D ⁇ )
- ⁇ ⁇ ( ⁇ ) is the transmittance of epidermis and ⁇ ( ⁇ ) is the reflectance of dermis.
- the transmittance due to the epidermis is squared because the light passes through it twice before emerging out of skin. Assuming the absorption of the epidermis is mainly due to the melanin concentration, the transmittance of the epidermis can be modeled as:
- T E (A) Qxp(d E c m m(A))
- the value of a tissue's (e.g. , skin) backscattering as a function of wavelength, ⁇ ( ⁇ ), can be used to obtain medical information about the tissue and its underlying structures.
- a tissue's e.g. , skin
- ⁇ ( ⁇ ) a tissue's
- ⁇ ( ⁇ ) a tissue's backscattering as a function of wavelength
- ⁇ ( ⁇ ) a tissue's (e.g. , skin) backscattering as a function of wavelength
- BCC basal cell carcinoma
- SCC squamous cell carcinoma
- MM malignant melanoma
- Malignant melanoma is a cancer that begins in the melanocytes present in the epidermis layer.
- Most melanoma cells produce melanin that in turn changes the backscattering characteristics as a function of wavelength ⁇ ( ⁇ ) of the affected skin.
- Squamous and basal cells are also present in the epidermis layer.
- the outermost layer of the epidermis is called the stratum corneum.
- layers of squamous cells below it are layers of squamous cells.
- the lowest part of the epidermis, the basal layer, is formed by basal cells.
- Both squamous and basal cell carcinomas produce certain viral proteins that interact with the growth-regulating proteins of normal skin cells.
- the abnormal cell growth then changes the epidermis optical scattering characteristics and consequently the skin backscattering properties as a function of wavelength ⁇ ( ⁇ ).
- information about different skin conditions e.g., normal skin, benign skin lesions and skin cancers
- the systems and methods described herein can be used to diagnose and characterize a wide variety of medical conditions.
- the concentration of one or more skin or blood component is determined in order to evaluate a medical condition in a patient.
- components useful for medical evaluation include: deoxyhemoglobin levels, oxyhemoglobin levels, total hemoglobin levels, oxygen saturation, oxygen perfusion, hydration levels, total hematocrit levels, melanin levels, collagen levels, and bilirubin levels.
- the pattern, gradient, or change over time of a skin or blood component can be used to provide information on the medical condition of the patient.
- hyperspectral/multispectral imaging include: tissue ischemia, ulcer formation, ulcer progression, venous stasis, venous ulcer disease, infection, shock, cardiac decompensation, respiratory insufficiency, hypovolemia, the progression of diabetes, congestive heart failure, sepsis, dehydration, hemorrhage, hypertension, exposure to a chemical or biological agent, and an inflammatory response.
- the systems and methods described herein are used to evaluate tissue oximetery and correspondingly, medical conditions relating to patient health derived from oxygen measurements in the superficial vasculature.
- the systems and methods described herein allow for the measurement of oxygenated hemoglobin, deoxygenated hemoglobin, oxygen saturation, and oxygen perfusion. Processing of these data provide information to assist a physician with, for example, diagnosis, prognosis, assignment of treatment, assignment of surgery, and the execution of surgery for conditions such as critical limb ischemia, diabetic foot ulcers, pressure ulcers, peripheral vascular disease, surgical tissue health, etc.
- the systems and methods described herein are used to evaluate diabetic and pressure ulcers.
- Development of a diabetic foot ulcer is commonly a result of a break in the barrier between the dermis of the skin and the subcutaneous fat that cushions the foot during ambulation. This rupture can lead to increased pressure on the dermis, resulting in tissue ischemia and eventual death, and ultimately manifesting in the form of an ulcer (Frykberg .G. et ah, Diabetes Care 1998;21(10): 1714-9).
- Measurement of oxyhemoglobin, deoxyhemoglobin, and/or oxygen saturation levels by hyperspectral/multispectral imaging can provide medical information regarding, for example: a likelihood of ulcer formation at an ROI, diagnosis of an ulcer, identification of boundaries for an ulcer, progression or regression of ulcer formation, a prognosis for healing of an ulcer, the likelihood of amputation resulting from an ulcer.
- Further information on hyperspectral/multispectral methods for the detection and characterization of ulcers, e.g., diabetic foot ulcers are found in U.S. Patent Application Publication No. 2007/0038042, and Nouvong A. et ah, Diabetes Care. 2009 Nov; 32(11):2056-61, the contents of which are hereby incorporated herein by reference in their entireties for all purposes.
- the systems and methods described herein are used to evaluate shock in a subject.
- Clinical presentation of shock is variable from subject to subject. While common indicators for a state of shock include low blood pressure, decreased urine output, and confusion, these symptoms do not manifest in all subjects (Tintinalli J.E., "Emergency Medicine: A Comprehensive Study Guide,” New York: McGraw-Hill Companies, pp. 165- 172). However, it was found that changes in cutaneous oxygen saturation, an underlying cause of shock, present as pronounced hyperspectral mottling patterns in subjects
- oxyhemoglobin, deoxyhemoglobin, and/or oxygen saturation levels by hyperspectral/multispectral imaging can provide medical information regarding, for example: a likelihood of a subject entering a state of shock, diagnosis of a state of shock, progression or regression of a state of shock, and a prognosis for the recovery from a state of shock.
- the shock is hemorrhagic shock, hypovolemic shock, cardiogenic shock, septic shock, anaphylactic shock, or neurogenic shock.
- abrasion alopecia, atrophy, av malformation, battle sign, bullae, burrow, basal cell carcinoma, burn, candidal diaper dermatitis, cat-scratch disease, contact dermatitis, cutaneous larva migrans, cutis marmorata, dermatoma, ecchymosis, ephelides, erythema infectiosum, erythema multiforme, eschar, excoriation, fifth disease, folliculitis, graft vs.
- guttate guttate psoriasis
- hand foot and mouth disease
- Henoch-Schonlein purpura herpes simplex, hives, id reaction, impetigo, insect bite, juvenile rheumatoid arthritis, Kawasaki disease, keloids, keratosis pilaris, Koebner phenomenon, Langerhans cell histiocytosis, leukemia, lichen striatus, lichenification, livedo reticularis, lymphangitis, measles, meningococcemia, molluscum contagiosum, neurofibromatosis, nevus, poison ivy dermatitis, psoriasis, scabies, scarlet fever, scar, seborrheic dermatitis, serum sickness, Shagreen plaque, Stevens- Johnson syndrome, strawberry tongue, swimmers' itch, telangiectasia, tinea capitis, tinea corporis, tuberous sclerosis, ur
- tissue viability e.g., whether tissue is dead or living, and/or whether it is predicted to remain living
- tissue ischemia e.g., whether tissue is dead or living, and/or whether it is predicted to remain living
- malignant cells or tissues e.g., delineating malignant from benign tumors, dysplasias, precancerous tissue, metastasis
- tissue infection and/or inflammation e.g., bacterial or viral counts.
- pathogens e.g., bacterial or viral counts.
- Some embodiments include differentiating different types of tissue from each other, for example, differentiating bone from flesh, skin, and/or vasculature.
- the systems and methods provided herein can be used during surgery, for example to determine surgical margins, evaluate the appropriateness of surgical margins before or after a resection, evaluate or monitor tissue viability in near-real time or real-time, or to assist in image-guided surgery.
- hyperspectral/multispectral imaging see, Holzer M.S. et al, J Urol. 2011 Aug; 186(2):400-4; Gibbs-Strauss S.L. et al, Mol Imaging. 2011 Apr; 10(2):91-101; and Panasyuk S.V. et al, Cancer Biol Ther. 2007 Mar; 6(3):439-46, the contents of which are hereby incorporated herein by reference in their entirety for all purposes.
- the present disclosure is based, at least in part, on an arrangement of optical elements in a hyperspectral/multispectral imaging system that allow for true co-axial imaging of multiple wavelengths. While the methods and systems for co-axial
- hyperspectral/multispectral imaging provided herein are particularly well suited for medical imaging and diagnostics, other fields may also benefit from the particular arrangement of optical element described herein.
- the present disclosure provides a hyperspectral/multispectral imaging system for remote sensing.
- an imaging system comprising an high power objective lens, beam steering element having a plurality of operating modes, and a plurality of optical detectors, most or all of which are covered by an appropriate filter, can be mounted within a satellite.
- the hyperspectral/multispectral satellite can then be used in, for example: geological surveying, e.g., in the mining and oil industries to search for oil seeps (Ellis J., "Searching for oil seeps and oil-impacted soil with hyperspectral imagery", Earth Observation Magazine, Jan 2001) or pockets of other mineral resources; agricultural surveying, e.g., monitoring of crops or identification of suitable soil; surveillance, e.g., in military reconnaissance; chemical imaging, e.g., for detecting harmful or toxic agents or chemical emissions; and environmental monitoring; e.g., monitoring levels of chemicals in the atmosphere and sub-atmospheric regions.
- geological surveying e.g., in the mining and oil industries to search for oil seeps (Ellis J., "Searching for oil seeps and oil-impacted soil with hyperspectral imagery", Earth Observation Magazine, Jan 2001) or pockets of other mineral resources
- agricultural surveying e.g., monitoring of crops or identification of suitable soil
- surveillance e.g
- hyperspectral/multispectral imaging systems described herein make these devices well suited for other portable applications. For example, as a handheld device used on the battlefield to quickly determine the status of a wounded soldier or detect the presence of a chemical agent.
- hyperspectral/multispectral imaging for triage or other battlefield embodiments, see, U.S. Patent Publication No. 2007/0024946, the content of which is hereby incorporated by reference in its entirety for all purposes.
- a portable hyperspectral/multispectral device as described herein, can be used to detect harmful emissions, identify chemical spills, or otherwise identify unsafe working conditions, e.g., at a factory, refinery, or chemical plant.
- a hyperspectral/multispectral imaging system may be affixed on a wall, ceiling, post, etc., at a plant or factory to continuously monitor conditions of the atmosphere for safe working conditions.
- a portable hyperspectral/multispectral device as described herein, can be used for forensic analysis.
- the portable hyperspectral/multispectral device as described herein, can be used for forensic analysis.
- hyperspectral/multispectral methods and systems can be used, for example: to determine the time of death based on change in cellular chemistry analyzed by the imager; evaluate the proximity of a gunshot based on residue left on target; determine the severity of blunt trauma; determine whether oxygen deprivation occurred pre- or post-mortem; evaluate drug status; identify the location and composition of body fluids present at a crime scene; determine if an injury is old or new; make field assessments; locate evidence and provide in- situ evaluation ⁇ e.g., the identification of brass casings over a large area); determine the location of man-made objects; evaluate machined surfaces having varying polarization and spectral responses, analyze bodily fluids spread over a large area; identify a point of impact; evaluate an entire incident scene (as opposed to sampling of individual points within the scene), identify different hairs for DNA analysis; locate and separate out of hairs in a carpet; and analyze chemical residues present on a surface or subject ⁇ e.g., gun powder).
- aspects of the disclosed methodologies can be implemented as a computer program product that includes a computer program mechanism embedded in a non-transitory computer-readable storage medium. Further, any of the methods disclosed herein can be implemented in one or more computers or other forms of apparatus. Further still, any of the methods disclosed herein can be implemented in one or more computer program products. Some embodiments disclosed herein provide a computer program product that encodes any or all of the methods disclosed herein. Such methods can be stored on a CD-ROM, DVD, magnetic disk storage product, or any other computer-readable data or program storage product. Such methods can also be embedded in permanent storage, such as ROM, one or more programmable chips, or one or more application specific integrated circuits (ASICs). Such permanent storage can be localized in a server, 802.1 1 access point, 802.11 wireless bridge/station, repeater, router, handheld mobile device, laptop computer, desktop computer, or other electronic devices.
- ASICs application specific integrated circuits
- Some embodiments provide a computer program product that contains any or all of the program modules shown Figure 13. These program modules can be stored on a CD- ROM, DVD, magnetic disk storage product, or any other computer-readable data or program storage product.
- the program modules can also be embedded in permanent storage, such as ROM, one or more programmable chips, or one or more application specific integrated circuits (ASICs).
- ASICs application specific integrated circuits
- Such permanent storage can be localized in a server, 802.11 access point, 802.11 wireless bridge/station, repeater, router, mobile phone, or other electronic devices.
- the present disclosure provides a non-transitory computer readable storage medium storing one or more programs executable by a computer for collecting co- axially aligned hyperspectral/multispectral images of an object or subject at a plurality of narrowband wavelengths.
- the program contains instructions for: turning on a first illumination source capable of emitting narrowband light having a first wavelength (1602), placing a beam steering element in a first operating mode in optical communication with a first optical detector configured to resolve narrowband light having the first wavelength (1604); capturing an image of an object or subject illuminated by the light emitted by the first illumination source using the first optical detector (1606); turning off the first illumination source (1608); turning on a second illumination source capable of emitting narrowband light having a second wavelength (1610); placing the beam steering element in a second operating mode in optical communication with a second detector configured to resolve narrowband light having the second wavelength (1612); capturing an image of an object or subject illuminated by the light emitted by the second illumination source using the second optical detector (1614); repeating steps 1608 to 1614 as needed to collect a desired number of images at distinct wavelengths; optionally assembling data contained in each captured image into a hyperspectral data cube; and optionally comparing the data in the hyperspectral data cube to
- the program further comprises instructions for performing at least one of: adjusting the brightness of an acquired image, adjusting the contrast of an acquired image, removing an artifact from an acquired image, cropping an acquired image, processing one or more sub-pixels of an acquired image, compressing the size of an acquired image, assembling a plurality of acquired images into a spectral hypercube, transforming a spectral hypercube assembled from the plurality of acquired digital images, formatting data contained within an acquired image, and encrypting data contained within an acquired image.
- additional programs are provided for operating hyperspectral imaging systems and collecting hyperspectral data, as described herein.
- the programs include instructions for illuminating multiple light sources simultaneously, collecting multiple images using multiple optical detectors simultaneously, controlling MEMS mirror or MEMS micromirror arrays, collecting images in line scanning operational modes, etc.
Abstract
Description
Claims
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CA2875650A CA2875650A1 (en) | 2012-06-05 | 2013-03-15 | Methods and apparatus for coaxial imaging of multiple wavelengths |
US14/372,711 US9107624B2 (en) | 2012-06-05 | 2013-03-15 | Methods and computer readable storage media for hyperspectral or multispectral imaging of multiple wavelengths |
EP13813563.7A EP2856107A4 (en) | 2012-06-05 | 2013-03-15 | Methods and apparatus for coaxial imaging of multiple wavelengths |
US14/333,406 US9354115B2 (en) | 2012-06-05 | 2014-07-16 | Methods and apparatus for coaxial imaging of multiple wavelengths |
US15/143,399 US10448836B2 (en) | 2012-06-05 | 2016-04-29 | Methods and apparatus for coaxial imaging of multiple wavelengths |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201261655800P | 2012-06-05 | 2012-06-05 | |
US61/655,800 | 2012-06-05 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/333,406 Continuation US9354115B2 (en) | 2012-06-05 | 2014-07-16 | Methods and apparatus for coaxial imaging of multiple wavelengths |
Publications (3)
Publication Number | Publication Date |
---|---|
WO2014007869A2 WO2014007869A2 (en) | 2014-01-09 |
WO2014007869A9 true WO2014007869A9 (en) | 2014-02-27 |
WO2014007869A3 WO2014007869A3 (en) | 2014-04-24 |
Family
ID=49882567
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2013/032682 WO2014007869A2 (en) | 2012-06-05 | 2013-03-15 | Methods and apparatus for coaxial imaging of multiple wavelengths |
Country Status (5)
Country | Link |
---|---|
US (3) | US9107624B2 (en) |
EP (1) | EP2856107A4 (en) |
CA (1) | CA2875650A1 (en) |
RU (1) | RU2616653C2 (en) |
WO (1) | WO2014007869A2 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10939823B2 (en) * | 2013-12-16 | 2021-03-09 | Sony Corporation | Image analysis device, image analysis method, and illumination device |
Families Citing this family (99)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9117133B2 (en) | 2008-06-18 | 2015-08-25 | Spectral Image, Inc. | Systems and methods for hyperspectral imaging |
JP6313211B2 (en) | 2011-10-25 | 2018-04-18 | デイライト ソリューションズ、インコーポレイテッド | Infrared imaging microscope |
FR2985307B1 (en) * | 2012-01-03 | 2015-04-03 | Centre Nat Etd Spatiales | METHOD OF CALIBRATING THE BANDS OF ALIGNMENT OF AN EARTH OBSERVATION SYSTEM UTILIZING SYMMETRICAL SIGHTS |
RU2616653C2 (en) * | 2012-06-05 | 2017-04-18 | Хайпермед Имэджинг, Инк. | Methods and device for coaxial image forming with multiple wavelengths |
NL2010788C2 (en) * | 2013-03-11 | 2014-09-15 | Symae Technologies Holding B V | Improved skin and scalp diagnosis device and method. |
US10285624B2 (en) * | 2013-03-12 | 2019-05-14 | University Of Washington | Systems, devices, and methods for estimating bilirubin levels |
WO2014146053A2 (en) * | 2013-03-15 | 2014-09-18 | Hypermed Imaging, Inc. | Systems and methods for evaluating hyperspectral imaging data using a two layer media model of human tissue |
US10182757B2 (en) * | 2013-07-22 | 2019-01-22 | The Rockefeller University | System and method for optical detection of skin disease |
US9945722B2 (en) * | 2013-09-20 | 2018-04-17 | The United States Of America, As Represented By The Secretary Of The Navy | Ultra low power solid state spectral radiometer |
WO2015109274A1 (en) * | 2014-01-18 | 2015-07-23 | Daylight Solutions, Inc. | Low-noise spectroscopic imaging system |
US9606002B2 (en) * | 2014-01-18 | 2017-03-28 | Daylight Solutions, Inc. | Spectral imaging of a sample using a plurality of discrete mid-infrared wavelengths |
US10437033B2 (en) * | 2014-01-18 | 2019-10-08 | Daylight Solutions, Inc. | Modulating spectroscopic imaging system using substantially coherent illumination |
US11803044B2 (en) | 2014-01-18 | 2023-10-31 | Daylight Solutions, Inc. | Low-noise spectroscopic imaging system with steerable substantially coherent illumination |
US9655519B2 (en) | 2014-03-21 | 2017-05-23 | Hypermed Imaging, Inc. | Systems and methods for performing an imaging test under constrained conditions |
US9526427B2 (en) * | 2014-03-21 | 2016-12-27 | Hypermed Imaging, Inc. | Compact light sensors with symmetrical lighting |
CN106461462B (en) | 2014-03-21 | 2019-03-26 | 海佩尔梅德影像有限公司 | Compact optical sensor |
CN106471354A (en) * | 2014-07-08 | 2017-03-01 | 三美电机株式会社 | Biological component information measurement device |
US10772541B2 (en) | 2014-08-21 | 2020-09-15 | I. R. Med Ltd. | System and method for noninvasive analysis of subcutaneous tissue |
US20160051147A1 (en) * | 2014-08-21 | 2016-02-25 | Irmed | System and method for noninvasive analysis of subcutaneous tissue |
US10709365B2 (en) * | 2014-08-21 | 2020-07-14 | I. R. Med Ltd. | System and method for noninvasive analysis of subcutaneous tissue |
US9977961B2 (en) * | 2014-10-21 | 2018-05-22 | Bae Systems Information And Electronic Systems Integration Inc. | Method for maintaining detection capability when a frame in a multispectral image is corrupted |
KR20160144006A (en) * | 2015-06-07 | 2016-12-15 | 김택 | Portable hyper-spectral camera apparatus having semiconductor light emitting devices |
US10542961B2 (en) | 2015-06-15 | 2020-01-28 | The Research Foundation For The State University Of New York | System and method for infrasonic cardiac monitoring |
JP6664909B2 (en) * | 2015-09-01 | 2020-03-13 | レボックス株式会社 | Imaging optical device |
WO2017044130A1 (en) * | 2015-09-11 | 2017-03-16 | The Ip Law Office Of Guy Levi, Llc | Multi-modal optoelectronic vision system and uses thereof |
US9648255B2 (en) | 2015-09-11 | 2017-05-09 | General Starlight Co., Inc. | Multi-modal optoelectronic vision system and uses thereof |
US9671281B2 (en) | 2015-09-18 | 2017-06-06 | Sensors Unlimited, Inc. | Imaging systems with digital micromirror devices (DMD) |
US10921186B2 (en) * | 2015-09-22 | 2021-02-16 | Hypermed Imaging, Inc. | Methods and apparatus for imaging discrete wavelength bands using a mobile device |
DE102015117940A1 (en) * | 2015-10-21 | 2017-04-27 | Osram Opto Semiconductors Gmbh | Optical sensor |
US10799280B2 (en) * | 2015-10-22 | 2020-10-13 | Medtronic Cryocath Lp | Post ablation tissue analysis technique |
AU2016381563B2 (en) | 2015-12-31 | 2022-02-10 | Wear2B Ltd | Device, system and method for non-invasive monitoring of physiological measurements |
US10088468B2 (en) | 2016-02-04 | 2018-10-02 | Nova Biomedical Corporation | Analyte system and method for determining hemoglobin parameters in whole blood |
US10088360B2 (en) | 2016-02-04 | 2018-10-02 | Nova Biomedical Corporation | Spectroscopic analyte system and method for determining hemoglobin parameters in whole blood |
US10151630B2 (en) | 2016-02-04 | 2018-12-11 | Nova Biomedical Corporation | Analyte system and method for determining hemoglobin parameters in whole blood |
US9933411B2 (en) * | 2016-02-04 | 2018-04-03 | Nova Biomedical Corporation | Analyte system and method for determining hemoglobin parameters in whole blood |
WO2017176380A1 (en) * | 2016-02-22 | 2017-10-12 | The General Hospital Corporation | Systems and methods for selective targeting of structural features in treating skin conditions |
US11134848B2 (en) | 2016-04-25 | 2021-10-05 | Samsung Electronics Co., Ltd. | Mobile hyperspectral camera system and human skin monitoring using a mobile hyperspectral camera system |
US10341565B2 (en) * | 2016-05-10 | 2019-07-02 | Raytheon Company | Self correcting adaptive low light optical payload |
US10798310B2 (en) | 2016-05-17 | 2020-10-06 | Hypermed Imaging, Inc. | Hyperspectral imager coupled with indicator molecule tracking |
EP3458835B1 (en) | 2016-05-18 | 2022-03-23 | MultiSensor Scientific, Inc. | Hydrocarbon leak imaging and quantification sensor |
US11137519B2 (en) * | 2016-07-21 | 2021-10-05 | Rosemount Aerospace Inc. | Multi-fiber optical sensor for icing |
US11883128B2 (en) | 2016-08-24 | 2024-01-30 | Mimosa Diagnostics Inc. | Multispectral mobile tissue assessment |
JP6353145B2 (en) | 2016-09-15 | 2018-07-04 | ハイパーメツド・イメージング・インコーポレイテツド | Tissue oxygenation measurement system and method |
US9897484B1 (en) * | 2016-09-29 | 2018-02-20 | Intel Corporation | Measuring wideband spectrum information in mobile devices via an integrated optical system that uses multiple spectral sensors, multiple light sources and MEMS actuation |
KR20180036150A (en) | 2016-09-30 | 2018-04-09 | 삼성전자주식회사 | Spectrometric sensing apparatus in electronic device and method thereof |
CA3048632A1 (en) * | 2016-12-27 | 2018-07-05 | Urugus S.A. | Hyper-spectral imaging when observed object is still |
US10502621B2 (en) * | 2016-12-27 | 2019-12-10 | Urugus S.A. | Dynamic hyper-spectral imaging of objects in apparent motion |
US10650225B2 (en) * | 2016-12-27 | 2020-05-12 | Canon Kabushiki Kaisha | Image processing apparatus which determines category to which object in image belongs, method therefor, and object discrimination apparatus |
KR102581945B1 (en) * | 2017-02-07 | 2023-09-25 | 삼성전자주식회사 | Method for providing skin information and electronic device supporting the same |
WO2018170438A1 (en) | 2017-03-16 | 2018-09-20 | MultiSensor Scientific, Inc. | Scanning ir sensor for gas safety and emissions monitoring |
EP3641634A4 (en) * | 2017-06-20 | 2021-03-10 | I.R Med Ltd | System and method for noninvasive analysis of subcutaneous tissue |
JP6834842B2 (en) * | 2017-08-10 | 2021-02-24 | オムロン株式会社 | Setting support device, image processing system and setting support program |
WO2019058308A1 (en) * | 2017-09-21 | 2019-03-28 | Elucid Labs, Inc. | Imaging biological tissue or other subjects |
RU180103U1 (en) * | 2017-10-27 | 2018-06-04 | Федеральное государственное казенное образовательное учреждение высшего образования "Волгоградская академия Министерства внутренних дел Российской Федерации" (Волгоградская академия МВД России) | PORTABLE MULTI-FUNCTIONAL DEVICE FOR INSTALLING GROUP ACCESSORIES OF PAINTS FOR COLOR TONE, SATURATION AND BRIGHTNESS OF COLOR IN DIGITAL EQUIVALENT |
CA3076483A1 (en) * | 2017-11-16 | 2019-05-23 | MultiSensor Scientific, Inc. | Systems and methods for multispectral imaging and gas detection using a scanning illuminator and optical sensor |
JP6947021B2 (en) * | 2017-12-27 | 2021-10-13 | 株式会社リコー | Image processing method, image processing device and image processing system |
US10752494B2 (en) | 2017-12-29 | 2020-08-25 | Advanced Semiconductor Engineering, Inc. | Semiconductor device package |
KR102005832B1 (en) * | 2018-02-21 | 2019-08-01 | 주식회사 올리브헬스케어 | Signal processing device for bio-signal analysing and bio-signal analysing apparatus using the same |
CN108876770B (en) * | 2018-06-01 | 2021-06-25 | 山东师范大学 | Fundus multispectral image joint registration method and system |
EP3801211A1 (en) | 2018-06-08 | 2021-04-14 | East Carolina University | Determining peripheral oxygen saturation (spo2) and hemoglobin concentration using multi-spectral laser imaging (msli) methods and systems |
KR20220066192A (en) | 2018-06-29 | 2022-05-23 | 할로 스마트 솔루션즈, 인크. | Sensor device and system |
DE102018212858A1 (en) * | 2018-08-01 | 2020-02-06 | Siemens Healthcare Gmbh | Magnetic resonance device, magnetic resonance system and associated operating method |
WO2020081159A2 (en) * | 2018-09-05 | 2020-04-23 | East Carolina University | Systems for detecting vascular and arterial disease in asymptomatic patients and related methods |
FR3086758B1 (en) * | 2018-09-28 | 2020-10-02 | Commissariat Energie Atomique | METHOD AND DEVICE FOR OBSERVING A SAMPLE UNDER AMBIENT LIGHT |
US11808887B2 (en) * | 2018-11-02 | 2023-11-07 | Waymo Llc | Methods and systems for mapping retroreflectors |
US11313968B2 (en) * | 2018-12-14 | 2022-04-26 | Texas Instruments Incorporated | Interference signal rejection in LIDAR systems |
CN113260303A (en) | 2018-12-14 | 2021-08-13 | 光谱Md公司 | Machine learning systems and methods for assessing, healing predicting, and treating wounds |
JP7186298B2 (en) * | 2018-12-14 | 2022-12-08 | スペクトラル エムディー,インコーポレイテッド | Systems and methods for high-precision multi-aperture spectral imaging |
US10976245B2 (en) | 2019-01-25 | 2021-04-13 | MultiSensor Scientific, Inc. | Systems and methods for leak monitoring via measurement of optical absorption using tailored reflector installments |
CN111487223A (en) | 2019-01-29 | 2020-08-04 | 三星电子株式会社 | Optical device using reflection geometry |
JP2022525113A (en) | 2019-03-26 | 2022-05-11 | イースト カロライナ ユニバーシティ | Near-infrared fluorescence imaging and related systems and computer program products for blood flow and perfusion visualization |
IT201900004861A1 (en) * | 2019-04-01 | 2020-10-01 | Univ Degli Studi Di Pavia | System for the determination of the surface and mechanical characteristics of filamentous structures, in particular skin appendages, associated structures, natural or synthetic fibers and their aggregates |
US11359966B2 (en) * | 2019-04-17 | 2022-06-14 | Westboro Photonics Inc. | System, method and apparatus for wide wavelength range imaging with focus and image correction |
US10656272B1 (en) | 2019-04-24 | 2020-05-19 | Aeye, Inc. | Ladar system and method with polarized receivers |
CN109907735B (en) * | 2019-04-25 | 2022-06-21 | 北京峰云视觉技术有限公司 | Multispectral tongue picture collecting device |
WO2020230319A1 (en) * | 2019-05-16 | 2020-11-19 | 三菱電機株式会社 | Image processing device and method, image reading device, program, and recording medium |
US11793399B2 (en) | 2019-06-20 | 2023-10-24 | Cilag Gmbh International | Super resolution and color motion artifact correction in a pulsed hyperspectral imaging system |
US11360028B2 (en) * | 2019-06-20 | 2022-06-14 | Cilag Gmbh International | Super resolution and color motion artifact correction in a pulsed hyperspectral, fluorescence, and laser mapping imaging system |
TWI708089B (en) * | 2019-09-04 | 2020-10-21 | 群光電子股份有限公司 | Digital imaging device |
CN112525854A (en) * | 2019-09-18 | 2021-03-19 | 大连兆晶生物科技有限公司 | Method for identifying components |
EP4022082A1 (en) | 2019-09-30 | 2022-07-06 | Akoya Biosciences, Inc. | Multiplexed imaging with enzyme mediated amplification |
TWI825999B (en) * | 2020-02-14 | 2023-12-11 | 新加坡商兆晶生物科技股份有限公司(新加坡) | Spectrometer and spectral detection method |
US11885718B2 (en) * | 2020-03-18 | 2024-01-30 | Rosemount Aerospace Inc. | Multi-fiber single lens optical ice detector |
US20230157549A1 (en) * | 2020-04-23 | 2023-05-25 | Hypermed Imaging, Inc. | Portable hyperspectral imaging device |
US11002671B1 (en) * | 2020-05-28 | 2021-05-11 | Soter Technologies, Llc | Systems and methods for mapping absorption spectroscopy scans and video frames |
US11393182B2 (en) | 2020-05-29 | 2022-07-19 | X Development Llc | Data band selection using machine learning |
JP2022016851A (en) * | 2020-07-13 | 2022-01-25 | キヤノン株式会社 | Imaging apparatus and imaging system |
US11606507B1 (en) | 2020-08-28 | 2023-03-14 | X Development Llc | Automated lens adjustment for hyperspectral imaging |
US11651602B1 (en) | 2020-09-30 | 2023-05-16 | X Development Llc | Machine learning classification based on separate processing of multiple views |
CN112254820B (en) * | 2020-10-14 | 2021-04-27 | 中国科学院空天信息创新研究院 | Discrete forest scene thermal infrared radiation transmission simulation method |
US11079278B1 (en) * | 2020-12-22 | 2021-08-03 | Illuscio, Inc. | Systems and methods for using multispectral imagery for precise tracking and verification |
US20220202325A1 (en) * | 2020-12-31 | 2022-06-30 | Bioxytech Retina, Inc. | Methods and devices for measuring structural and functional properties of tissue |
RU204680U1 (en) * | 2021-02-09 | 2021-06-04 | Общество с ограниченной ответственностью "БИОСПЕК" | DEVICE FOR CONTACT INFRARED VISUALIZATION OF BLOOD VESSELS BY BACKSCATTERED LASER RADIATION |
JP2024512973A (en) * | 2021-03-31 | 2024-03-21 | クリア ラブズ, インコーポレイテッド | portable hyperspectral system |
CN113487592B (en) * | 2021-07-22 | 2023-11-17 | 四川九洲电器集团有限责任公司 | Hyper-spectral or multispectral image overexposure detection method and system based on statistics |
WO2023022964A1 (en) * | 2021-08-16 | 2023-02-23 | Netzvision, Llc | Multispectral and lidar detector using light field optics, and systems using same |
US11302174B1 (en) | 2021-09-22 | 2022-04-12 | Halo Smart Solutions, Inc. | Heat-not-burn activity detection device, system and method |
WO2023067611A1 (en) * | 2021-10-19 | 2023-04-27 | Ezerx Health Tech Pvt. Ltd. | An apparatus and a method for determining characteristics of a fluid |
US20230126975A1 (en) * | 2021-10-22 | 2023-04-27 | Samsung Electronics Co., Ltd. | Hyperspectral image sensor and operating method thereof |
Family Cites Families (73)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE3773381D1 (en) * | 1987-07-06 | 1991-10-31 | Agfa Gevaert Nv | SILVER COMPLEX DIFFUSION TRANSFER PROCESS. |
US5276321A (en) * | 1991-04-15 | 1994-01-04 | Geophysical & Environmental Research Corp. | Airborne multiband imaging spectrometer |
US5769792A (en) * | 1991-07-03 | 1998-06-23 | Xillix Technologies Corp. | Endoscopic imaging system for diseased tissue |
JP3411737B2 (en) * | 1995-03-03 | 2003-06-03 | ペンタックス株式会社 | Biological fluorescence diagnostic equipment |
US6081612A (en) * | 1997-02-28 | 2000-06-27 | Electro Optical Sciences Inc. | Systems and methods for the multispectral imaging and characterization of skin tissue |
US20030036855A1 (en) * | 1998-03-16 | 2003-02-20 | Praelux Incorporated, A Corporation Of New Jersey | Method and apparatus for screening chemical compounds |
US6504943B1 (en) * | 1998-07-20 | 2003-01-07 | Sandia Corporation | Information-efficient spectral imaging sensor |
US20050270528A1 (en) * | 1999-04-09 | 2005-12-08 | Frank Geshwind | Hyper-spectral imaging methods and devices |
US6975898B2 (en) * | 2000-06-19 | 2005-12-13 | University Of Washington | Medical imaging, diagnosis, and therapy using a scanning single optical fiber system |
WO2002028273A2 (en) * | 2000-10-06 | 2002-04-11 | Yang Victor X D | Multi-spectral fluorescence imaging and spectroscopy device |
FR2815750B1 (en) | 2000-10-20 | 2002-12-06 | Claude Ricard | METHOD FOR AVOIDING FRAUD ON A TAXI EQUIPPED WITH AN ELECTRONIC TAXIMETER |
US6826424B1 (en) * | 2000-12-19 | 2004-11-30 | Haishan Zeng | Methods and apparatus for fluorescence and reflectance imaging and spectroscopy and for contemporaneous measurements of electromagnetic radiation with multiple measuring devices |
US6736507B2 (en) * | 2001-01-22 | 2004-05-18 | Alexis Kudryashov | High resolution, multispectral, wide field of view retinal imager |
US7808633B2 (en) * | 2007-02-14 | 2010-10-05 | Chemimage Corporation | Spectroscopic system and method for predicting outcome of disease |
WO2003021853A2 (en) * | 2001-09-05 | 2003-03-13 | Genicon Sciences Corporation | Apparatus for reading signals generated from resonance light scattered particle labels |
EP1499231A4 (en) * | 2002-03-08 | 2007-09-26 | Sensys Medical Inc | Compact apparatus for noninvasive measurement of glucose through near-infrared spectroscopy |
AU2003248559A1 (en) * | 2002-05-22 | 2003-12-12 | Beth Israel Deaconess Medical Center | Device for wavelength-selective imaging |
US6950088B2 (en) * | 2002-06-17 | 2005-09-27 | Koninklijke Philips Electronics N.V. | Synchronizing optical scan and electrical addressing of a single-panel, scrolling color LCD system |
US6735507B2 (en) | 2002-06-28 | 2004-05-11 | General Motors Corporation | Four wheel steering system and method for trailering |
US8285015B2 (en) * | 2002-07-05 | 2012-10-09 | Lawrence Livermore Natioonal Security, LLC | Simultaneous acquisition of differing image types |
US20050154277A1 (en) * | 2002-12-31 | 2005-07-14 | Jing Tang | Apparatus and methods of using built-in micro-spectroscopy micro-biosensors and specimen collection system for a wireless capsule in a biological body in vivo |
US7420156B2 (en) * | 2003-08-06 | 2008-09-02 | University Of Pittsburgh | Metal nanowire based bandpass filter arrays in the optical frequency range |
EP1661182A4 (en) * | 2003-08-06 | 2011-03-23 | Univ Pittsburgh | Surface plasmon-enhanced nano-optic devices and methods of making same |
WO2005031433A1 (en) * | 2003-09-26 | 2005-04-07 | Tidal Photonics, Inc. | Apparatus and methods relating to color imaging endoscope systems |
US7103402B2 (en) * | 2003-10-02 | 2006-09-05 | Ut-Battelle, Llc | Advanced synchronous luminescence imaging for chemical and medical diagnostics |
US7711159B1 (en) * | 2004-02-11 | 2010-05-04 | Intermec Ip Corp. | Multi-technology information capture system and method |
WO2005088264A1 (en) * | 2004-03-06 | 2005-09-22 | Plain Sight Systems, Inc. | Hyper-spectral imaging methods and devices |
WO2005104932A1 (en) * | 2004-04-30 | 2005-11-10 | Koninklijke Philips Electronics N. V. | Probe head for spectroscopic analysis of a fluid |
US8827899B2 (en) * | 2004-09-24 | 2014-09-09 | Vivid Medical, Inc. | Disposable endoscopic access device and portable display |
US7850305B2 (en) * | 2004-12-03 | 2010-12-14 | Topcon Corporation | Apparatus and method for measuring spectrum image data of eyeground |
CA2604829C (en) * | 2005-04-04 | 2018-05-15 | Hypermed, Inc. | Hyperspectral imaging in diabetes and peripheral vascular disease |
US20060262304A1 (en) * | 2005-04-22 | 2006-11-23 | Keith Carron | Apparatus for automated real-time material identification |
US7366365B2 (en) * | 2005-11-23 | 2008-04-29 | Princeton Lightwave, Inc. | Tissue scanning apparatus and method |
US7329860B2 (en) * | 2005-11-23 | 2008-02-12 | Illumina, Inc. | Confocal imaging methods and apparatus |
US8320650B2 (en) * | 2005-11-30 | 2012-11-27 | Lawrence Livermore National Security, Llc | In vivo spectral micro-imaging of tissue |
US7508508B2 (en) * | 2006-09-19 | 2009-03-24 | Seethrough Ltd. | Device and method for inspecting a hair sample |
RU2344383C2 (en) * | 2006-11-10 | 2009-01-20 | ФГУП "Государственный оптический институт им. С.И. Вавилова" | Broadband fourier hyperspectrometer |
FR2913788B1 (en) * | 2007-03-14 | 2009-07-03 | Sagem Defense Securite | METHOD AND APPARATUS FOR IDENTIFYING AN INDIVIDUAL BY OPTICAL CAPTURE OF AN IMAGE OF A BODY FOOTPRINT |
WO2008123927A1 (en) | 2007-04-05 | 2008-10-16 | The Board Of Trustees Of The University Of Illinois | Biosensors with porous dielectric surface for fluorescence enhancement and methods of manufacture |
US20080306337A1 (en) | 2007-06-11 | 2008-12-11 | Board Of Regents, The University Of Texas System | Characterization of a Near-Infrared Laparoscopic Hyperspectral Imaging System for Minimally Invasive Surgery |
FR2918467B1 (en) * | 2007-07-06 | 2009-11-20 | Thales Sa | MULTISPECTRAL IMAGING DEVICE WITH MOEMS TYPE FILTER FOR SATELLITE OBSERVATION |
WO2009063322A2 (en) | 2007-08-15 | 2009-05-22 | Malvern Instruments Ltd | Broad-range spectrometer |
GB0717967D0 (en) * | 2007-09-14 | 2007-10-24 | Cascade Technologies Ltd | Polarimetric hyperspectral imager |
WO2009042862A1 (en) * | 2007-09-28 | 2009-04-02 | Illumina, Inc | Fluorescence excitation and detection system and method |
JP5231625B2 (en) * | 2008-03-18 | 2013-07-10 | ノヴァダク テクノロジーズ インコーポレイテッド | Imaging system for acquiring NIR and full color images and method of operation thereof |
DE102008019600B4 (en) * | 2008-04-18 | 2021-03-04 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Stacked optical device and method of making the same |
GB0809252D0 (en) * | 2008-05-21 | 2008-06-25 | Ntnu Technology Transfer As | Underwater hyperspectral imaging |
US20090318815A1 (en) * | 2008-05-23 | 2009-12-24 | Michael Barnes | Systems and methods for hyperspectral medical imaging |
US9117133B2 (en) * | 2008-06-18 | 2015-08-25 | Spectral Image, Inc. | Systems and methods for hyperspectral imaging |
WO2010019515A2 (en) * | 2008-08-10 | 2010-02-18 | Board Of Regents, The University Of Texas System | Digital light processing hyperspectral imaging apparatus |
DE102009000528B4 (en) * | 2009-01-30 | 2011-04-07 | Nanophotonics Ag | Inspection device and method for the optical examination of object surfaces, in particular of wafer surfaces |
JP5572326B2 (en) * | 2009-03-26 | 2014-08-13 | オリンパス株式会社 | Image processing apparatus, imaging apparatus, image processing program, and image processing method |
JP2011104199A (en) * | 2009-11-19 | 2011-06-02 | Fujifilm Corp | Endoscope apparatus |
US8619237B2 (en) * | 2009-12-04 | 2013-12-31 | The Trustees Of Columbia University In The City Of New York | Laser-scanning intersecting plane tomography such as for high speed volumetric optical imaging |
GB0921477D0 (en) * | 2009-12-08 | 2010-01-20 | Moor Instr Ltd | Apparatus for measuring blood parameters |
EP2359745A1 (en) * | 2010-02-12 | 2011-08-24 | Helmholtz Zentrum München Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH) | Method and device for multi-spectral photonic imaging |
JP5303012B2 (en) * | 2010-12-15 | 2013-10-02 | 富士フイルム株式会社 | Endoscope system, processor device for endoscope system, and method for operating endoscope system |
US20130015370A1 (en) * | 2011-07-15 | 2013-01-17 | Huron Technologies International In | Confocal fluorescence slide scanner with parallel detection |
US8970706B2 (en) * | 2011-09-22 | 2015-03-03 | Basil Henry Scott | Dual pixel pitch imaging array with extended dynamic range |
WO2013078412A1 (en) * | 2011-11-22 | 2013-05-30 | California Institute Of Technology | Systems and methods for noninvasive analysis of retinal health and function |
US20150044098A1 (en) * | 2012-01-30 | 2015-02-12 | Scanadu Incorporated | Hyperspectral imaging systems, units, and methods |
RU2616653C2 (en) * | 2012-06-05 | 2017-04-18 | Хайпермед Имэджинг, Инк. | Methods and device for coaxial image forming with multiple wavelengths |
US8913241B2 (en) * | 2012-07-23 | 2014-12-16 | Corning Incorporated | Hyperspectral imaging system and method for imaging a remote object |
US9526427B2 (en) * | 2014-03-21 | 2016-12-27 | Hypermed Imaging, Inc. | Compact light sensors with symmetrical lighting |
CN106461462B (en) * | 2014-03-21 | 2019-03-26 | 海佩尔梅德影像有限公司 | Compact optical sensor |
US9551616B2 (en) * | 2014-06-18 | 2017-01-24 | Innopix, Inc. | Spectral imaging system for remote and noninvasive detection of target substances using spectral filter arrays and image capture arrays |
EP3206567A1 (en) * | 2014-10-13 | 2017-08-23 | Glusense, Ltd. | Analyte-sensing device |
US9395293B1 (en) * | 2015-01-12 | 2016-07-19 | Verily Life Sciences Llc | High-throughput hyperspectral imaging with superior resolution and optical sectioning |
US9931040B2 (en) * | 2015-01-14 | 2018-04-03 | Verily Life Sciences Llc | Applications of hyperspectral laser speckle imaging |
EP3347700A4 (en) * | 2015-09-07 | 2019-04-17 | Mobileodt Ltd. | Handheld mobile device for detection of biological evidence |
EP3361950A4 (en) * | 2015-10-13 | 2019-03-20 | HyperMed Imaging, Inc. | Compact light sensors for surgical applications and shock detection |
US9824295B2 (en) * | 2016-02-18 | 2017-11-21 | L-3 Communications Cincinnati Electronics Corporation | Imaging systems and methods including obscurant characterization |
US10580129B2 (en) * | 2016-05-27 | 2020-03-03 | The Florida International University Board Of Trustees | Hybrid spectroscopy imaging system for intraoperative epileptic cortex detection |
-
2013
- 2013-01-31 RU RU2013104205A patent/RU2616653C2/en not_active IP Right Cessation
- 2013-03-15 EP EP13813563.7A patent/EP2856107A4/en not_active Withdrawn
- 2013-03-15 CA CA2875650A patent/CA2875650A1/en not_active Abandoned
- 2013-03-15 WO PCT/US2013/032682 patent/WO2014007869A2/en active Application Filing
- 2013-03-15 US US14/372,711 patent/US9107624B2/en not_active Expired - Fee Related
-
2014
- 2014-07-16 US US14/333,406 patent/US9354115B2/en active Active
-
2016
- 2016-04-29 US US15/143,399 patent/US10448836B2/en not_active Expired - Fee Related
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10939823B2 (en) * | 2013-12-16 | 2021-03-09 | Sony Corporation | Image analysis device, image analysis method, and illumination device |
Also Published As
Publication number | Publication date |
---|---|
US20160345835A1 (en) | 2016-12-01 |
EP2856107A2 (en) | 2015-04-08 |
CA2875650A1 (en) | 2014-01-09 |
EP2856107A4 (en) | 2016-03-16 |
WO2014007869A3 (en) | 2014-04-24 |
US9354115B2 (en) | 2016-05-31 |
US9107624B2 (en) | 2015-08-18 |
US20150051498A1 (en) | 2015-02-19 |
US10448836B2 (en) | 2019-10-22 |
WO2014007869A2 (en) | 2014-01-09 |
RU2616653C2 (en) | 2017-04-18 |
US20150308896A1 (en) | 2015-10-29 |
RU2013104205A (en) | 2014-08-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10448836B2 (en) | Methods and apparatus for coaxial imaging of multiple wavelengths | |
US11346714B2 (en) | Methods and apparatus for imaging discrete wavelength bands using a mobile device | |
JP6672406B2 (en) | Single sensor hyperspectral imaging device | |
US9526427B2 (en) | Compact light sensors with symmetrical lighting | |
US10652481B2 (en) | Compact light sensor | |
JP7261883B2 (en) | Machine learning system for wound assessment, healing prediction and treatment | |
WO2017201093A1 (en) | Hyperspectral imager coupled with indicator molecule tracking | |
US9480424B2 (en) | Systems and methods for measuring tissue oxygenation | |
JP2022513847A (en) | Systems and methods for precision multi-aperture spectral imaging | |
US10470694B2 (en) | Systems and methods for measuring tissue oxygenation | |
EP3298963A1 (en) | Systems and methods for measuring tissue oxygenation | |
US20210140822A1 (en) | Hyperspectral imaging method and device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 13813563 Country of ref document: EP Kind code of ref document: A2 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14372711 Country of ref document: US |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 13813563 Country of ref document: EP Kind code of ref document: A2 |
|
ENP | Entry into the national phase |
Ref document number: 2875650 Country of ref document: CA |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2013813563 Country of ref document: EP |