Long Wave Infrared and Multimodal Stereo

People: Philip Saponaro, Scott Sorensen, Stephen Rhein and Chandra Kambhamettu

site currently under construction

Long Wave Infrared

Scott as seen in LWIR.

Thermal, Long Wave Infrared, or LWIR cameras detect light in the 4-18μm range. Many materials at typical environmental temperatures emit thermal radiation at these wavelengths, and this allows the cameras to detect the heat given off by objects. Though there are many differences in the sensor and optics design, the cameras function in essentially the same way traditional visible band cameras do. LWIR can be used in stereo and multi camera systems and their unique abilities can complement other imaging modalities.

Material Classification

Since LWIR cameras can detect thermal radiation from different materials, they can more directly observe various properties of those materials. We have explored this in work of classifying materials based on LWIR and visbile imagery. We have looked at two material properties which are readily apparent in LWIR, namely water permeation and how the material responds to heating and cooling cycles. To do this we have created an experimental setup where we expose materials to controlled amounts of water and controlled heating and cooling cycles ins separate experiments.

A pipette is used to drop a small amount of water on the surface of the material, and a video is recorded. A 3D model is created by observing the water boundary over time, and this is used to generate a binned spherical mapping for classification.

A heatlamp connected to an arduino microcontroller is used to expose each sample to a uniform heating and cooling cycle while video is recorded. Samples on the image are taken and the laplacian of the intensity of these samples over time is used to solve for α and β from the heat equation. These represent physical properties of the material and aid in classification.

Calibration of LWIR

LWIR cameras operate similarly to visible band cameras, and the pinhole camera model applies equally well to them. For stereo reconstruction camera calibration is needed to fit a mathematical model to the lens parameters and orientations of the cameras. This process typically involves photographic a calibration pattern, typically a checkerboard, a number of times at different poses. This can be done in LWIR as well, however a standard printed calibration pattern is not readily visible in this image modality. To facilitate calibration in LWIR we have developed a simple inexpensive technique involving a ceramic tile backed printed pattern and a pre-processing technique which allows for off-the-shelf calibration calibration techniques to be used on these images. Furthermore this technique can be extended to allow for calibration between modalities of image, allowing calibration between visible band cameras and LWIR. This technique uses a calibration board backing that retains heat. The pattern is not heated uniformly however, so a model is fit to the pattern and normalized to remove artefacts from inconsistent heating.

A calibration board in the visible band, the LWIR band, and results of our preprocessing technique.

Multi Modal Stereo

Calibration between modalities allows for multi camera systems in different modalities. We have constructed a 4 camera system consisting of two visible band cameras and two long wave infrared cameras operating in the same coordinate system. The cameras are synchronized via software trigger to within a few milliseconds, and real time reconstructions in both modalities can be done simultaneously in our framework.

Ray Trace Reflection Reconstruction

Using multiple stereo imaging modalities it is possible to reconstruct a reflected scene, a problem that traditional stereo approaches are ill suited for. To accomplish this we reconstruct the reflecting surface in one image modality and the reflected scene is ray traced in the complementary modality. To elaborate, the diagram below shows the visible cameras in blue being used to reconstruct the reflecting surface, and the LWIR cameras in red reconstructing a reflected scene.

This technique allows us to even reconstruct our camera system, as it gives off heat which is visible in a LWIR reflection in a piece of aluminum.

The reflection of our camera system, and the reconstructed result.

Related Publications

Scott Sorensen, Philip Saponaro, Stephen Rhein, Chandra Kambhamettu. Multimodal Stereo Vision For Reconstruction In The Presence Of Reflection. Accepted at The British Machine Vision Conference, BMVC 2015.
Philip Saponaro, Scott Sorensen, Stephen Rhein, Chandra Kambhamettu. Improving Calibration of Thermal Stereo Cameras Using Heated Calibration Board. Accepted at The International Conference on Image Processing, ICIP 2015 (Oral presentation).
Philip Saponaro, Scott Sorensen, Abhishek Kolagunda, Stephen Rhein, Chandra Kambhamettu. Material Classification with Thermal Imagery. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015

Normal View Video/Image Modeling and Synthesis Lab Home
VIMS Home People Research Publications Workshops Achievements Courses Software Data Photos login	Long Wave Infrared and Multimodal Stereo People: Philip Saponaro, Scott Sorensen, Stephen Rhein and Chandra Kambhamettu site currently under construction Long Wave Infrared Scott as seen in LWIR. Thermal, Long Wave Infrared, or LWIR cameras detect light in the 4-18μm range. Many materials at typical environmental temperatures emit thermal radiation at these wavelengths, and this allows the cameras to detect the heat given off by objects. Though there are many differences in the sensor and optics design, the cameras function in essentially the same way traditional visible band cameras do. LWIR can be used in stereo and multi camera systems and their unique abilities can complement other imaging modalities. Material Classification Since LWIR cameras can detect thermal radiation from different materials, they can more directly observe various properties of those materials. We have explored this in work of classifying materials based on LWIR and visbile imagery. We have looked at two material properties which are readily apparent in LWIR, namely water permeation and how the material responds to heating and cooling cycles. To do this we have created an experimental setup where we expose materials to controlled amounts of water and controlled heating and cooling cycles ins separate experiments. A pipette is used to drop a small amount of water on the surface of the material, and a video is recorded. A 3D model is created by observing the water boundary over time, and this is used to generate a binned spherical mapping for classification. A heatlamp connected to an arduino microcontroller is used to expose each sample to a uniform heating and cooling cycle while video is recorded. Samples on the image are taken and the laplacian of the intensity of these samples over time is used to solve for α and β from the heat equation. These represent physical properties of the material and aid in classification. Calibration of LWIR LWIR cameras operate similarly to visible band cameras, and the pinhole camera model applies equally well to them. For stereo reconstruction camera calibration is needed to fit a mathematical model to the lens parameters and orientations of the cameras. This process typically involves photographic a calibration pattern, typically a checkerboard, a number of times at different poses. This can be done in LWIR as well, however a standard printed calibration pattern is not readily visible in this image modality. To facilitate calibration in LWIR we have developed a simple inexpensive technique involving a ceramic tile backed printed pattern and a pre-processing technique which allows for off-the-shelf calibration calibration techniques to be used on these images. Furthermore this technique can be extended to allow for calibration between modalities of image, allowing calibration between visible band cameras and LWIR. This technique uses a calibration board backing that retains heat. The pattern is not heated uniformly however, so a model is fit to the pattern and normalized to remove artefacts from inconsistent heating. A calibration board in the visible band, the LWIR band, and results of our preprocessing technique. Multi Modal Stereo Calibration between modalities allows for multi camera systems in different modalities. We have constructed a 4 camera system consisting of two visible band cameras and two long wave infrared cameras operating in the same coordinate system. The cameras are synchronized via software trigger to within a few milliseconds, and real time reconstructions in both modalities can be done simultaneously in our framework. Ray Trace Reflection Reconstruction Using multiple stereo imaging modalities it is possible to reconstruct a reflected scene, a problem that traditional stereo approaches are ill suited for. To accomplish this we reconstruct the reflecting surface in one image modality and the reflected scene is ray traced in the complementary modality. To elaborate, the diagram below shows the visible cameras in blue being used to reconstruct the reflecting surface, and the LWIR cameras in red reconstructing a reflected scene. This technique allows us to even reconstruct our camera system, as it gives off heat which is visible in a LWIR reflection in a piece of aluminum. The reflection of our camera system, and the reconstructed result. Related Publications Scott Sorensen, Philip Saponaro, Stephen Rhein, Chandra Kambhamettu. Multimodal Stereo Vision For Reconstruction In The Presence Of Reflection. Accepted at The British Machine Vision Conference, BMVC 2015. Philip Saponaro, Scott Sorensen, Stephen Rhein, Chandra Kambhamettu. Improving Calibration of Thermal Stereo Cameras Using Heated Calibration Board. Accepted at The International Conference on Image Processing, ICIP 2015 (Oral presentation). Philip Saponaro, Scott Sorensen, Abhishek Kolagunda, Stephen Rhein, Chandra Kambhamettu. Material Classification with Thermal Imagery. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015 Page last modified on July 27, 2015, at 02:18 PM