Long-term monitoring of the Earth-reflected
solar spectrum is necessary for discerning and attributing changes in
climate. High radiometric accuracy enables such monitoring over decadal
timescales with non-overlapping instruments, and high precision enables
trend detection on shorter timescales. The HyperSpectral Imager for Climate
Science (HySICS) is a visible and near-infrared spatial/spectral
imaging spectrometer intended to ultimately achieve
The 2007 NRC Decadal Survey for Earth Science (NRC, 2007) calls for shortwave spatial/spectral Earth-scene measurements with radiometric accuracy and SI-traceability of better than 0.2 % for Earth-climate studies on decadal timescales. These accuracies, being nearly ten times better than current on-orbit capabilities, will establish benchmark measurements of solar radiation scattered by the Earth, provide reference calibrations for other on-orbit instruments, and initiate a climate-data record to be used for future climate-policy decisions.
Current space-based imaging systems have radiometric uncertainties of
The HyperSpectral Imager for Climate Science (HySICS) is a prototype
instrument to demonstrate a new means of achieving
In this article, we provide an overview of the HySICS instrument and describe the solar cross-calibration approach relying on precisely characterized attenuation methods (Sect. 2), summarize the two completed high-altitude balloon flights (Sect. 3), detail the data-analysis methods and estimated uncertainties (Sect. 4), and present resulting data cubes of Earth ground scenes and the Moon acquired during Flight 2 (Sect. 5).
Optical layout of the HySICS shows the 4MA telescope followed by a grating-based Offner spectrometer that images onto a full-spectral-range HgCdTe focal-plane array with a three-region order-sorting filter on the back surface of its vacuum entrance window. The Offner and 4MA have nearly orthogonal optical-axis planes to reduce polarization sensitivity. The main picture shows a top view of the entire optical path, while a side view of the 4MA itself is shown in the upper right inset. The physical entrance aperture is positioned at the system's aperture stop. The spectrometer entrance slit is shown in its correct (albeit unconventional) orientation.
An eventual spaceflight instrument to achieve the 2007 Decadal Survey's
solar-reflected Earth-radiance measurement requirements, needed for climate
studies, would likely be designed to achieve desired ground-scene
characterizations having a 0.5 km spatial resolution and 100 km cross-track
field of view (FOV) while spanning the 350 to 2300 nm spectral range with
6 nm spectral resolution. Acquiring such measurements from low Earth orbit
formed the driving requirements for the HySICS spatial/spectral imager,
mandating a 10
The optical design of the pushbroom HySICS imaging-spectrometer is representative of state-of-the-art hyperspectral imagers, featuring a four-mirror anastigmat (4MA) telescope followed by an Offner spectrometer. The instrument-performance parameters are shown in Table 1, and a schematic of the optical layout, which is an evolution of that described by Espejo et al. (2011), is shown in Fig. 1.
HySICS performance specifications.
A precision NIST-calibrated aperture is the first element in the optical
train, precisely determining the collecting area for the light entering the
instrument. This front-most aperture location allows the most accurate
radiometry by reducing uncertainties in estimates of scatter and diffraction
effects, which must be corrected to provide low radiometric uncertainties.
Diffraction from the precision-aperture's knife edge is well understood
theoretically, but scatter is surface dependent and must be measured for the
actual optics. Both have been characterized to reduce uncertainties and
correct for light losses at the detector. There are no view-limiting baffles
in front of the aperture as these can cause additional diffractive,
scattering, and glint effects that are difficult to model and correct. A
six-element rotatable aperture wheel allows selection of any of the HySICS's
six circular apertures. A 20 mm diameter aperture is used to acquire
sufficient signal for Earth-scene radiances, while a 0.5 mm diameter
solar-calibration aperture provides a relative attenuation of 10
Immediately following the aperture wheel is a similar wheel containing
attenuation filters. These share a common filter-wheel thermistor. A
Hg
The compact 4MA telescope following the aperture and filter wheels uses
aspherical diamond-turned aluminum mirrors with electroless-nickel coatings.
A protected-aluminum topcoat is magneto-rheological finish (MRF)
post-polished for reduced scatter from each element. The fully reflective
system eliminates the need for chromatic corrections over the HySICS's broad
spectral range. The 4MA mirrors and housing incorporate precision-machined
mounting tabs and alignment pins for mechanical robustness and low
sensitivity to thermal distortions. This telescope is designed to produce a
distortion-free image of a spatial scene onto a 0.028 mm wide slit,
providing a slit-width-limited spatial resolution of 0.02
The precision 0.028 mm
The Offner spectrometer uses independent primary and tertiary mirrors. The
secondary element, a convex reflective 100-ln mm
A three-region order-sorting filter prevents overlap of different orders of
diffraction. Region 1, for wavelengths less than 634 nm, is clear; Region 2
passes wavelengths
The 480
All optics, as well as the aperture and filter wheels, are mounted to a thick
aluminum baseplate. Three independently controlled thermoelectric coolers
(TECs) reduce thermal gradients of the near-ambient-temperature optics. The
entire instrument is encased in a thick aluminum housing for contamination
control and thermal stability during integration and test with the balloon
gondola as well as during flight. A small door opens for flight observations,
which are performed at flight altitude ambient pressures of
A separate electronics box contains all the controlling components for the HySICS optical module. This 1-atmosphere nitrogen-pressurized enclosure is maintained during flight since not all off-the-shelf electronic components are intended for near-vacuum operations. The FPA electronics are mounted in this box in close proximity to the FPA for reduced noise. Five-hundred gigabytes of solid-state memory arranged as a redundant array of independent disks store all data redundantly during flight, allowing up to 8 h of continual, uncompressed, 14 Hz imagery from the FPA.
Three methods collectively provide the required 10
Changing from an entrance-aperture diameter of 20 mm for viewing
Earth scenes to 0.5 mm for viewing the Sun provides a geometric
attenuation level of 10
The HySICS apertures are diamond-turned nickel-coated aluminum, providing a
very sharp aperture edge with nearly negligible scatter. The six installed
apertures have entrance diameters of 20, 10, and 0.5 mm, with two each of
the largest and smallest. All are calibrated by NIST/Gaithersburg for
geometric area using a non-contact optical technique to achieve the desired
attenuation uncertainties, with the limiting factor being the 0.06 to
0.08 % (1
Shorter integration times are used for solar viewing than Earth-scene measurements. These are enabled by the FPA electronics, reproducible detector linearity, and an electronic global shutter to avoid spatial smear during image integration.
The FPA's controlling electronics demonstrate
Spectral filters capable of roughly 10
The three ionically colored Schott glass filters in the HySICS were polished
to 0.1 nm RMS surface roughness and NG4 (a neutral-density filter with NG5 (a neutral-density filter with BG25 (a high-transmittance filter in the UV and IR).
The HySICS was flown on two high-altitude balloon flights to demonstrate its
ability to cross-calibrate Earth-scene radiances to the spectral solar
irradiance. Each of the
The HySICS was mounted on a two-axis gimballed pointing system able to track
the Sun and Moon for calibrations and able to maintain a fixed-angle nadir
view for scanning along the ground as the balloon drifted. The pointing
system was mounted near the center of a large rectangular-frame gondola that
was suspended from the balloon itself. A rotator mechanism between the
balloon and gondola provided coarse azimuthal pointing (
The Wallops Arc Second Pointer (WASP) is a two-axis altitude-azimuth
gimbal-based pointing system designed to achieve nearly arc-second
accuracy levels for balloon payloads (Stuchlik, 2015a, b). This system was
provided courtesy of HySICS co-investigators D. Stuchlik's and J. Lanzi's
team at NASA's Wallops Flight Facility (WFF). With the HySICS center-of-mass
aligned within the WASP gimbal-axes to
The WASP generally provided
The WASP was also able to inertially track the Moon using an on-board ephemeris. This new pointing-system capability enabled flat-fielding calibrations using the Moon while operating with the same 20 mm aperture (and thus optical paths) and integration-time parameters as used for Earth-scene observations.
The balloon gondola is a rectangular-frame structure that houses the entire payload, consisting of the HySICS instrument, the WASP, 27 lead-acid batteries to supply power, all telemetry and tracking equipment, thermal enclosures, several crush pads for landing, and ballast. The net mass of the payload and gondola is 2300 kg (5000 lb), including 540 kg (1200 lb) of ballast.
The gondola was designed and built at the University of Colorado's Laboratory for Atmospheric and Space Physics (LASP), using a combination of 80–20 aluminum and square aluminum tubing. The structure is 3 m in height and contained within a 4.3 m diameter region when the crush pads are installed on all but the top of the gondola's six rectangular sides. During flight, the entire structure is suspended by the azimuthal rotator that provides coarse pointing.
The WASP and HySICS are centrally located in the gondola such that the HySICS
can view nadir for observing the Earth and greater elevation angles for
solar and lunar measurements. Once expanded at altitude, the overhead
Helium-filled balloon restricts viewing to elevation angles
Both high-altitude balloon flights were performed out of Fort Sumner, NM, and supported by the Columbia Scientific Balloon Facility (CSBF). Upper-atmosphere winds limit Fort Sumner balloon flights to a few weeks in the springtime and fall, while CSBF schedules limit support at Fort Sumner to only the fall launch season. Ground winds generally limit launches to early mornings. Upper-atmosphere wind speeds determine flight duration and allow only a narrow timeframe of a couple of weeks for lengthy flights needed for many other programs' nighttime viewing. HySICS observations allow a more extended launch window, since the Sun and Earth are the primary targets and both can be viewed shortly after the morning launches; nighttime observations are not needed.
Lunar observations, however, are needed, as they allow flat-fielding using
the same optics as for ground viewing. While low lunar phases are beneficial
for the higher radiances provided near full moon, such nighttime-acquired
flat fields would be separated temporally from the Earth-ground scenes and
would also require longer flight durations. Instead, higher lunar-phase
angles were chosen to acquire the flat-field calibrations at similar
instrument temperatures and times to the acquired ground scenes. Launch
windows at less than 90
Flight 1 occurred on 29 September 2013, with launch at 13:30 UT and landing at 22:13 UT. A float altitude of 37 100 m (121 800 ft) was reached for this engineering flight, during which the HySICS and WASP attempted all needed measurements. The gondola was recovered and returned to LASP for refurbishment. No damage to the instrument occurred during this flight or landing. Flight 2 launched at 15:36 UT on 18 August 2014, reached a float altitude of 37 200 m (122 000 ft) at 17:52 UT, was powered off at 23:52 UT, and landed early on the following day. Despite a rough landing, post-recovery checkout revealed that the instrument was unharmed and all optical alignments were maintained, validating the HySICS's robust design.
The HySICS has three primary observation targets, each containing various observation-modes as well as several internal-instrument calibrations.
These cross-track scans, with the ground track and speed determined by the balloon velocity from the aloft winds, provide samples of the desired data from an eventual flight instrument. During Flight 2, four ground scans were acquired. The two in the morning included a mix of the New Mexico high desert with broken clouds, while the two in the afternoon were predominantly of high, thin clouds. Three-dimensional data cubes of these scans were created in ground processing after all radiometric calibrations were applied.
Several scans of the Earth limb were also obtained on this flight. These scans provide spatial–spectral information through the vertical extent of the Earth's atmosphere. The Earth limb itself was largely occulted by the tops of bright cumulus clouds at the near-horizontal look-angle for these scans. Some such scans also included the Moon as it was setting.
Along-slit scans enable flat-fielding of the FPA by placing the same portion
of the Sun on each spatial element of the array. Cross-slit scans build up an
entire data cube of the Sun, enabling the spatially integrated
solar irradiance to be determined and allowing SI-traceability to SSI
(provided on an absolute scale by other measurements or models), as detailed
in Sect. 4.4. Since demonstrating the solar cross-calibration method was the
primary purpose of these flights, solar scans dominated the flight
observation time. Near local noon the Sun's elevation was greater than
60
Similar to those done with the Sun, along-slit scans enable flat-fielding of the FPA by placing the same portion of the Moon on each spatial element of the array. The lunar scans can be done with the larger Earth-viewing aperture, potentially providing a more appropriate flat field to be applied to ground scans than those obtained from solar scans. Additionally, spectral-filter transmission is calibrated during flight by quick successive measurements with each filter in and out of the optical path while tracking a fixed position of the Moon.
Internal-instrument calibrations and diagnostics helped track instrument
functionality, stability, and performance in flight. Spectral calibrations
were made intermittently throughout the flights by briefly illuminating the
instrument's Hg
The intent of Flight 2 was to quantify the radiometric uncertainties to which
HySICS-acquired Earth scenes could be related to known spectral solar
irradiances. The HySICS spatial/spectral ground images,
Since the factors in Eq. (1) are independent, their individual uncertainties are evaluated separately and root-sum-squared for each final scene-dependent uncertainty. These correction factors and their uncertainties are derived from component- and instrument-level characterizations from both pre-flight laboratory-based calibrations and in-flight calibrations of the instrument, which are described in this section.
The initial data-analysis step is to apply corrections to the raw
data images. Applying all such corrections gives
Non-responsive pixels and badly fluctuating pixels, defined as those with a
measurement-to-measurement standard deviation of more than 5
Read noise for the Teledyne sensor is determined using a traditional
photon-transfer measurement (Janesick, 2001) of a constant radiant-power
source provided by blackbody radiation from a uniform, warm, temperature-stabilized target.
This target is measured at various exposure
levels by varying the integration time from 33.6
With a gain of
Both dark-noise and thermal-background signals from the surrounding instrument scale with integration time and are dependent on instrument or FPA temperature. Thermistors monitor the FPA and several of the nearby instrument-components. Laboratory characterizations of the dark signal enable corrections for both internal-FPA and background-thermal effects.
The HySICS FPA's inherent dark signal is sufficiently low that it is
difficult to detect in the presence of any background light. A cold target
placed in front of the imager while keeping the sensor housing at
Background signals were also corrected during flight. Following all data acquisitions, the blanked aperture wheel position blocked incoming light for 100 exposures. These consist only of dark current, instrument-thermal-background contributions, and imager-fixed-pattern noise. They are acquired at the same integration time and nearly the same temperatures as the data frames themselves. The average of these dark exposures is subtracted from the data frames, thereby removing background offsets with the exception of possible thermal offsets caused by temperature differences between when the data and the dark measurements where acquired. These temperature dependencies are in turn corrected via in-flight thermal-background measurements using portions of the array viewing dark space during solar and lunar scans. From multiple such scans, FPA sensitivities to instrument thermal effects are determined as a function of surrounding instrument-component temperatures. All raw HySICS data images are thus corrected for thermal background based on the instrument temperatures at the actual time of data acquisition, using the instrument-temperature dependencies determined from these dark-space observations.
Although these thermal-background signals are largest at the longer-wavelength portion of the FPA's sensitivity, they influence the entire array uniformly, since the FPA has no long-wave rejection filter over the portions used only for shorter-wavelength readout, making the above corrections necessary for all portions of the spectrum. While the dark current is very small and contributes nearly insignificantly to the net HySICS uncertainties, the thermal-background signal contributes to shot noise (described in Sect. 4.2.1).
Deviations from linearity are determined individually for each FPA pixel in laboratory testing using varying levels of incident-light intensity and integration times. If temporally stable, non-linearities can be corrected once characterized. These corrections are applied to the images after the bad-pixel, dark, and thermal-background corrections.
Sensor linearity is measured in two steps: (1) The electronically determined integration time is measured directly using timing pulses from the sensor's field-programmable gate array's digital output signal and (2) the response of the FPA itself is measured using a stable light source while varying the now-known electronically controlled integration time. The former verifies the timing of the controlling electronics, which are, as expected for oscillator-based signals, very linear and stable. The latter step includes the effects of FPA pixel-well or amplifier-signal saturation and is a function of the net signal on each pixel. To characterize these non-linearities, the sensor is illuminated by a stable FEL lamp while the electronically controlled integration time is varied and the resulting signal levels are measured. A linear curve-fit is used to determine the expected signal level on each pixel, and deviations from that fit with signal level are considered non-linearities in that pixel's response. The curve fit uses only the most linear portion of the data at less than 50 % of the FPA's full well. Repetition of this measurement using various FEL-lamp intensities ensures that the deviation from linearity has an FPA signal-level dependence rather than an integration-time dependence.
The resulting non-linearities and uncertainties are detailed in Sect. 4.3.2, where the non-linearity corrections, uncertainties, and intensity range and the resulting dominant determinants of the overall instrument attenuation uncertainty based on the integration-time method are discussed.
Sensor gain, or the conversion [e
Flat-fielding the HySICS sensor requires a full-system calibration, since it is affected by the collective efficiencies of all upstream optics. This calibration therefore needs to be performed separately for the smaller solar-viewing aperture and the larger Earth-viewing aperture, as light passing through the two apertures interacts with different portions of the downstream optical elements in the instrument. These differences are accounted for via the flat-field calibrations and are corrected in post-processing of the data.
Flat-fielding uncertainties.
Although different apertures are used for the two scans, the flat-fielding
procedure for both is to use a stable light source that can be swept across
every pixel on the sensor, enabling a measurement of the relative response,
or gain, of each. In space, the only available sufficiently stable
light sources are the Sun and the Moon, which are used for the small- and
large-aperture flat-field calibrations respectively. In both cases, a slice
near the center of the solar or lunar disk is scanned in the along-slit
direction from one edge of the imager to the other, while images are
continuously captured at the instrument's nominal 14 Hz cadence used for
ground-scene measurements. The flat-field calibration scans
As with read noise, since the flat-field calibrations are acquired using static sources, they can benefit from multi-acquisition scans to reduce random uncertainties and at different integration times to improve signal in spectral regions having lower sensitivity such as the visible. These approaches, described in more detail in Sect. 4.2.6, were not performed for the flat-field calibrations of either the Sun or the Moon during Flight 2 and, as a result, the flight-acquired flat-field uncertainties dominate all others at the shorter wavelengths where instrument sensitivity is low. For flat-field calibrations using the Sun, cross-slit scans of which did benefit from multi-acquisition scans at different integration times and thus have low uncertainties for most other parameters, the flat-field uncertainties dominate at the shorter wavelengths and are comparable to diffraction at the longer wavelengths, so would greatly benefit from multi-acquisition scans. Lunar flat-field calibrations were marginal because of the high lunar phase during the time of the flight, giving low lunar signal and small spatial extent. These along-slit lunar scans are not only low in signal, but very sensitive to pointing, particularly since the large aperture used for lunar flat-field calibrations does not spatially blur the lunar image due to diffraction as the smaller aperture does to the solar image. Where the acquired flat-field uncertainties exceed the array's intrinsic 3.3 % pixel-to-pixel variations, such as in the shorter-wavelength portion of the visible, they were clipped at this intrinsic value.
Multi-acquisition flat-field calibrations at different integration times for
both the Sun and the Moon and a lower lunar phase-angle would greatly improve
the uncertainties demonstrated by Flight 2. Nevertheless, in spectral regions
having high signal, the flat-field uncertainties acquired during this flight
are
Flat-field corrections are applied to ground scenes and cross-slit solar scans, and thus these uncertainties directly affect those data. Measurements that rely purely on relative measurements, such as calibrations of aperture ratio (Sect. 4.3.1) and filter transmission (Sect. 4.3.3), are not affected by these flat-field uncertainties.
Section 4.1 discussed corrections from the FPA and their associated
uncertainties. Further contributions to
Shot noise arises from photon-counting statistics and varies as the reciprocal square root of the signal, including that from any thermal background. As with read noise, it is reduced via multiple-image acquisitions for calibrations of static sources, namely the Sun and the Moon, but cannot be similarly reduced for single-acquisition images of the ground. Shot noise is the dominant source of uncertainty for ground scenes across the majority of the spectrum.
HySICS directly measures outgoing Earth-reflected shortwave radiances and
incoming solar radiances. By spatially integrating radiances from the entire
solar disk, which are acquired from cross-slit scans of the Sun, the HySICS
measurements are calibrated to the independently known incoming SSI. To
provide an accurate spatial integration, HySICS data analysis needs to
correct for radiative losses, such as due to stray light and diffraction,
that may cause differences in the amount of light reaching the FPA when
viewing the Sun as opposed to ground scenes. Losses from diffraction are
higher for the solar-viewing configuration than for ground viewing because of
the smaller aperture used for solar observations. (Figure 2 illustrates the
noticeably larger diffraction that must be accounted for when using the
0.5 mm solar aperture compared to the 20 mm Earth-viewing aperture. At
1000 nm, the diffraction limit from each is
These scans of a lab FEL-lamp filament show the blurring caused by diffraction when using the 0.5 mm aperture (left panel) vs. the 20 mm aperture (right panel). These effects must be accounted for in spatially integrated spectral solar irradiance determinations.
Lab scatter- and diffraction-characterization setup (upper schematic) gives the 2-D pattern shown in the lower image when using a 2 mm beam block behind the 0.5 mm solar-viewing aperture. This beam block occults the un-diffracted and un-scattered light incident on the aperture from a distant, nearly collimated light source (left side of upper schematic). Most light that is diffracted or scattered by the aperture edges passes around the beam block to be reimaged onto the camera, helping to quantify the intensity and spatial pattern of that light. The innermost Airy rings from typical aperture-edge diffraction are visible in the lower image. Since incident sunlight will diffract and scatter similarly, thus spreading some light beyond the edges of the solar-disk image, these effects must be corrected when determining net solar-irradiance values via spatial integrations from HySICS's cross-slit scans of the solar disk. Lab measurements such as these, combined with diffraction models to account for wavelength sensitivity and extend the spatial extent, help reduce uncertainties for those corrections. (The nearly horizontal radially extending dark region to the left of image center in the lower image is due to the support for the beam block.)
Diffraction can be modeled well with a NIST-quoted uncertainty of
An idealized model estimating the net amount of light falling outside of various angles due to diffraction alone (black dashed curve) is scaled to match lab measurements including both scatter and diffraction (red diamonds) at 528 nm. The measurements match the angular dependence expected from diffraction, indicating that the majority of this measured light loss is mainly due to diffraction rather than scatter, which would have a less-well-modeled relation to angle. Correcting for light losses via this validated diffraction model reduces the uncertainties in solar-radiance measurements, improving the HySICS's determination of spectral solar irradiance.
By modeling the diffracted light and verifying the model with lab
measurements, the expected light losses are accounted for when
spatially integrating the solar disk to obtain a value that can be correctly
calibrated to the independently known SSI. A 1.8 % uncertainty on this
correction is allocated as per NIST diffraction-estimate uncertainties. Because
diffraction scales with wavelength, these corrections begin to dominate the
solar-calibration uncertainties at the longer wavelengths but never greatly
exceed the contributions from read and shot noise. The HySICS's small
solar-viewing aperture was chosen such that uncertainties due to diffraction
may be the limiting uncertainty at the longest wavelengths but would not
dominate across the spectrum, effectively balancing desirable
greater-attenuation capabilities afforded by smaller apertures with the
increased uncertainties expected from them. This balance established the
HySICS attenuation levels achievable via aperture ratios to
Since radiometric uncertainties are dependent on the product of the instrument's spectral accuracy and the derivative of the measured spectrum, knowledge of, and corrections for, the spectral scale (or wavelength position) are characterized and applied.
Intermittent measurements using the HySICS's internal pen-ray lamp throughout
the flight allow spectral calibrations based on this narrow-band source to
verify spectral-scale accuracy or correct for possible wavelength-position
fluctuations due to thermal or mechanical changes. Independent control of the
three TECs regulating optical-bench temperature reduced thermal gradients
during Flight 2 and thus reduced variations in the spectral scale. The
spectral scale when at altitude shifted by only 3 nm, with variations across
all wavelengths being maintained to
The spectral corrections were interpolated to the times of observations. Of particular importance are the corrections at the times of solar calibrations, as the Sun has more abrupt spectral variations than ground scenes, and uncertainties in the spectral scale near the edge of a large spectral variation can give a correspondingly large radiometric uncertainty at wavelengths near spectral lines. Since the HySICS uses an FPA that spectrally bins the incident light from the spectrometer into 3 nm regions defined by the size of the FPA pixels, small potential spectral shifts in the incident light coupled with large spectrally dependent changes in signal near the sharp edges of these pixel-defined spectral bins can affect radiometric uncertainties.
Both the effects of this pixel-delineated spectral binning and those from thermal or mechanical instrument distortions are included in estimates of the HySICS's wavelength-position uncertainties, which are plotted as a function of wavelength in Fig. 6 for solar observations and in Fig. 7 for ground-scene measurements. Because of the in-flight spectral-calibration corrections via the internal pen-ray lamp, wavelength-position uncertainties are rarely the dominant contributor to the net radiometric uncertainties, although they do increase at the shortest wavelengths, where the Sun has more spectral-absorption lines, as well as near 820 nm, where the Sun has several absorption lines and the HySICS has low sensitivity.
For a spaceflight instrument regularly acquiring Earth observations, the
spectral scale determined by the pen-ray calibrations could be validated by
select Earth-atmospheric spectral lines under certain viewing conditions to
help distinguish them from surrounding spatial or spectral features, such as
by observing these lines from uniform bright background clouds or dark
oceans or viewing them near the Earth limb by off-pointing from nadir.
Oxygen molecules provide some such possible spectral lines, with one HySICS
balloon flight even showing an O
Diattenuation of the HySICS at the integrated-instrument level is limited by high grating-induced polarization, which is as large as 4 % at wavelengths above 1000 nm. A Zemax model based on the flight-grating measurements (black, long dashes) shows the lower instrument diattenuations expected using a less-polarization-sensitive grating (gray, short dashes).
The FPA has a background-level offset that varies linearly with the measured
signal. This offset is detectable by observing the extreme-most ultraviolet
spectral column of the sensor, which, at 320 nm, is below the reflectivity
cutoff for the instrument mirrors and, therefore, is effectively a dark
column. All pixel values in this column should remain nearly constant,
showing mainly dark-current and fixed-pattern noise. Instead, they
consistently decrease by up to 120 DN when other portions of the array are
observing extremely bright signals. This background-level decrease is also
observed in all dark pixels during a solar scan, including columns
neighboring the dark column as well as regions of the sensor viewing dark
space up to 9.5
This “brightness offset” of the background level, as measured on the dark
column, is characterized using flight data. A matrix of background-level
reduction versus sensor signal is generated from all large power-level
transitions during the flight, such as when the solar disk moves out of the
instrument FOV during a flat-field scan or when it comes into- or out-of-view
during an irradiance scan. The amount of background-level reduction is linear
with the amount of light detected by the sensor, regardless of its spectral
distribution or spatial location, with the background level changing by
The resulting brightness-offset corrections, which are dependent on the total signal on the sensor as well as the integration time, are applied to each image acquired. The relative uncertainties in this correction are greatest for measurements having low signals, so they predominantly affect Earth ground scenes, where they are generally the second-largest contributor to net uncertainties.
Contributions to net relative uncertainty (black) when observing the Sun during Flight 2 are shown as a function of wavelength. With the exception of the flat-field uncertainties, these plotted uncertainties are the result of two consecutive cross-slit solar scans acquired using specific integration times for the short- and long-wavelength spectral regions to reduce the uncertainties within each. Flat-field uncertainties due to low signal levels dominate across the spectrum but could be reduced with similar multi-image, dual-scan techniques applied to those calibrations.
Accurate radiometric measurements of scenes having unknown polarization rely on the instrument having low polarization sensitivity (Lukashin et al., 2015). The HySICS was designed to reduce polarization sensitivity by orienting the optical plane of the 4MA perpendicularly to that of the spectrometer, such that reflection-induced diattenuation in the former is nearly offset by that in the latter. This was effective with the exception of the custom-ruled grating, the primary HySICS optical component that did not meet expected performance. Along with having low efficiency in the visible, polarization tests of this grating showed a much larger sensitivity than anticipated, with the net instrument-diattenuation results plotted in Fig. 5. Despite the orthogonal orientation of the 4MA to the Offner optics, this grating limits the instrument's desired low polarization sensitivity, particularly in the near infrared.
Contributions to net relative uncertainty (black) when observing a bright (top) and dark (bottom) Earth scene during Flight 2 are shown as a function of wavelength. The small, inset lower plots (red) indicate the signal strength from each scene relative to full scale of the instrument's FPA. Shot noise is generally the dominant uncertainty across the majority of the spectrum for ground scenes, which do not benefit from multiple-image or dual-scan acquisition techniques.
If measuring randomly polarized scenes, this internal-instrument polarization sensitivity has no effect on radiometric accuracy but, for scenes of unknown polarization amplitude and orientation, the radiometric uncertainties can potentially be as large as the instrument's diattenuation itself in the specific – albeit highly improbable – case of a 100 % polarized incident signal oriented along, or perpendicular to, the direction of the instrument's greatest polarization sensitivity.
Solar-irradiance scan uncertainties.
Integrated-instrument uncertainties showing the effects described above are
plotted in Fig. 6 and tabulated for select wavelengths in Table 3 for
spatially integrated cross-slit observations of the Sun and in Fig. 7 and
Table 4 for sample single-acquisition measurements of bright and dark
ground scenes. These two figures indicate the uncertainties on the
measurements
With the exception of the flat-field and diffraction uncertainties, the solar-scan uncertainties benefit from multiple-image acquisitions and a dual-scan approach. Multiple, repeated measurements of the same scene particularly reduce the effects of read and shot noise and from the brightness offset caused by the small vs. large apertures at the shorter wavelengths where the HySICS's response is lowest. Two back-to-back scans of the Sun, one using longer integration times to increase signals at wavelengths shorter than 850 nm and one with integration times better matched to the higher signals at longer wavelengths, followed by spectrally combining the scans in post-processing improves the signal in select portions of the spectrum. The improvements from multiple image acquisitions and the dual-scan approach are possible only because the Sun can be viewed repeatedly with the same instrument look-angles, so it provides a static in-flight calibration source. These techniques would also be applicable to reducing flat-field uncertainties but were not performed on Flight 2, so the solar-calibration results shown are dominated by the flat-field uncertainties.
Balloon-flight motions over the ground prevent applying these beneficial
uncertainty-reduction techniques to ground scenes, so uncertainties must be
based on single-image acquisitions. Despite the larger aperture and the
longer integration times for ground scenes, the lower radiances of these
single images have larger relative uncertainties than those from the Sun,
since they do not benefit from multi-image or dual-scan techniques. Typical
net uncertainties from representative bright (cloud-filled) and dark (desert-
and vegetation-filled) ground scenes are plotted in Fig. 7; these are the net
scene-dependent uncertainties in the measurement factor
Ground-scan uncertainties.
In addition to many of the instrument-level uncertainties for various
observation scenes and modes described in Sect. 4.1 and 4.2, characterizing
the radiometric uncertainties to which ground-scene radiances can be
referenced to the spectral solar irradiance also involves quantifying the
uncertainties from the three intensity-attenuation methods used to enable
solar vs. Earth viewing. These attenuation methods, represented by the
correction factor
The total attenuations demonstrated during Flight 2 were capable of a net
10
Different illuminations of the optical surfaces by the 0.5 and 20 mm apertures, mainly being affected by a boundary between grating blaze regions, cause the spectrally dependent attenuations that differ from the nominal geometric-ratio value due to the aperture-attenuation method, as shown in the left panel. The blue and the red curves are based on different lab light sources that provide peak power at shorter and longer wavelengths, respectively. The right panel gives the uncertainties in these attenuations. The large peak in uncertainties between the two light sources used is due to low signals from each and could be improved with additional calibration light sources.
Results and uncertainties from the individual attenuation-methods are detailed in the following subsections.
The baselined 10
Aperture attenuation-method uncertainties.
The small and large apertures respectively used for the Sun and Earth measurements illuminate different areal portions of the HySICS optical surfaces and thus have different throughput efficiencies that must be accounted for when transferring the solar-based radiometric scale to radiances from ground measurements. While most optical surfaces are sufficiently uniform or similarly illuminated to not be greatly affected by these different areal-illumination effects, the spectrometer grating is the dominant cause of current HySICS spectrally dependent efficiency variations between the two aperture-illumination regions.
To achieve the broad spectral range and high throughput efficiencies required
with a single-spectrometer design, varying grating-blaze-angles are needed.
The fabricated balloon-flight grating contains a sawtooth pattern of
four discrete regions, with the blaze-angle varying monotonically across
each. The small aperture used for solar measurements illuminates a boundary
between two such regions to a much greater proportional degree than the
larger aperture does, so it is more sensitive to symmetric alignment on this
boundary. A slight misalignment on the edge of this “tooth” in the
sawtooth grating pattern will preferentially favor the corresponding extreme-blaze-angle
at the edge of the region in that misalignment direction, thus
making the system more sensitive to either the shortest or longest
wavelengths. The relative throughput for the small and large apertures was
characterized in lab measurements with the results shown in Fig. 8. These
effects are accounted for in HySICS's radiometric results as part of the
aperture-ratio portion of the full attenuation-system correction,
These laboratory calibrations were performed using two light sources, with one peaking in the visible and the other in the NIR, to span the full spectral region. The intermediate visible-to-NIR spectral region had low intensity from both lamps. Combined with the strong increase in attenuation and resulting lower intensities at shorter wavelengths when using the small aperture, these low light-source intensities limited the relative uncertainties in this visible-to-NIR spectral region, resulting in the large uncertainty peak shown in Fig. 8. Further calibrations with a broader range of bright lamp sources, particularly near the visible-to-NIR transition, could improve the uncertainties shown. More significantly, reducing the large spectral dependence of this aperture-ratio correction by using smoothly varying but non-monotonic blaze-angles via a more expensive custom-made grating rather than the four-region sawtooth one used here, should reduce much of these aperture-ratio uncertainty issues in the first place and is planned for a future HySICS instrument.
The aperture-ratio attenuation technique is inherently nearly independent of wavelength. That HySICS demonstrated the technique to well less than the needed uncertainties through most of the NIR spectral region shows promise that this attenuation method would be equally applicable over the entire spectral range with a more uniformly blazed grating and further laboratory characterizations.
Correcting for non-linearities while varying the FPA's
electronically controlled integration times was more successful than
initially anticipated, achieving a demonstrated attenuation of
The electronic timing signals show deviations from linearity that are
Integration-time attenuation-method uncertainties.
The results from the characterizations of the FPA's response described in
Sect. 4.1.4, whereby the FPA's signal levels are determined from multiple,
repeated measurements of an input FEL-lamp source at different exposure
times, are shown in Fig. 9. Since the electronic shutter has nearly
negligible non-linearity across this range, these deviations from linearity
that manifest mainly at greater exposure times (i.e. greater signal levels)
are due to non-linearities in the detector response and/or readout-amplifier
electronics. The average of the deviations plotted in Fig. 9 (lower graph)
provides the applied non-linearity correction as a function of detector
signal, and the standard deviations about this average give the corresponding
uncertainties in the applied non-linearity correction. The corrections are
measured to be
Detector response is plotted vs. exposure time over a range
Filter-calibration attenuation-method uncertainties.
The standard deviations of the residuals from the average of the
residuals shown in Fig. 9 indicate the
uncertainties in the applied linearity correction as a function of signal
level (left panel). The semi-log plot of the same data demonstrates the
full
The greater-than-anticipated attenuation range and the lower-than-anticipated uncertainties due to this integration-time attenuation method allow flexibility in the attenuation amounts needed by the other two attenuation methods. The integration-time attenuation capabilities provided by this flight-capable FPA eliminate the need for attenuations via filters altogether, which reduces mass, cost, complexity, and power for a future flight instrument.
Spectral filters were calibrated during Flight 2 using both the Moon and the
Sun. The filter attenuation method demonstrated the desired attenuation range
of 10
Calibration of NG5 #2 filter during Flight 2. Measurements with the filter out dominate the net uncertainty, as the spectrally flatter filter-in measurements could be done at a longer integration time to achieve a higher overall signal.
Section 4.1 and 4.2 explain the intrinsic imaging-measurement uncertainties
from Sun and Earth scenes,
Contributions to net relative uncertainty (black) in the ratio of
a bright (top) and dark (bottom) Earth scene relative to the
HySICS-determined SSI during Flight 2 are shown as a function of wavelength.
The small, inset lower plot (red) indicates the signal strength from each
scene relative to the full scale of the instrument's FPA. Shown uncertainties
are for individual pixels and could be reduced with spatial or spectral
binning. Spectrally averaged uncertainties, being weighted by
globally averaged reflected-solar (RS) irradiance (gray), are given for both
the full (350 to 2300 nm) and partial (450 to 1900 nm) wavelength ranges.
Demonstrating a minimum uncertainty of
The solar cross-calibration techniques achieved a radiometric uncertainty of
nearly 0.3 % across a large spectral region longward of 1000 nm from a
bright ground scene, demonstrating a
A globally averaged, all-sky estimate of Earth-reflected irradiance over the
8-year period from 2003 to 2010, based on results from
observation-system simulation experiments
generated using SCIAMACHY data
(Y. S. Shea, personal communication, 2016), is plotted in Fig. 12 (gray) to indicate a typical,
realistic reflected-solar (RS) spectrum observed by a spaceflight
hyperspectral imager. Weighting the HySICS's net radiometric uncertainties by
this estimated RS spectral-irradiance gives the resulting spectrally averaged
radiometric uncertainties stated in the (black) figure text. These are higher
than ultimately desired, which is largely caused by low instrument
efficiencies and high flat-field uncertainties in the visible as well as
increased aperture-ratio attenuation uncertainties near the visible-to-NIR
transition. Improving these via the methods described in Sect. 4.3.1 and
extending the multiple-image acquisition and a dual-scan approach to
flat-field calibrations should reduce the weighted, Earth-reflected HySICS
uncertainties for a future instrument by another
The high-quality data from Flight 2 with all instrument-level and
attenuation-method corrections applied and with a final calibration factor,
Multiple images as the solar disk is scanned in the cross-slit direction are spatially integrated to give a net spectral solar irradiance with corrections to account for the spectrometer's NIST-calibrated slit width as well as image overlap during the cross-slit scan. This irradiance is corrected for the diffraction and scatter described in Sect. 4.2.2 as well as other instrument effects described above. At this stage, the spectral “irradiance” is in units of instrument data numbers (DNs) and has no traceability to normal physical units. Figure 13 shows the SSI determined from HySICS using a cross-slit scan of the solar disk during Flight 2.
The HySICS signal from spatially integrated cross-slit solar scans with all applied instrument-level corrections gives an instrument-level spectral solar “irradiance” (blue). The values are given in instrument DNs and have no traceability to SI at this stage. The traceability and physical units are provided by scaling to the NRLSSI2 model for the day of the flight. These values, plotted in red, are adjusted to the Sun-instrument distance at the time of Flight 2 to correctly indicate the actual SSI that should be measured at the HySICS's location. These, or direct solar measurements from space-borne instruments having high absolute accuracies, enable the SI-traceable cross calibration of the HySICS-measured SSI. (Note that the DN values exceed the 16-bit maximum values from individual pixels because the plotted HySICS SSI signal is the spatially integrated sum of the entire solar disk.)
By knowing what the actual SSI is, the HySICS instrument DNs can be converted to useful physical units and the overall instrument sensitivity can be determined. SSI values from Lean's NRLSSI2 model were applied to the HySICS Flight 2 data, since they were available prior to measurements from any on-orbit instrument. These values, which account for the solar activity state on that day, were adjusted from their as-provided 1-AU distance to the actual Earth–Sun distance on the date of the flight. They are plotted in Fig. 13 and provide the transfer to realistic physical units.
The ratio of this model-based “actual” SSI to the HySICS's measured
irradiance in Fig. 13 gives the instrument sensitivity via a conversion
from DNs to physical irradiance units via correction factor
Dividing the HySICS-measured spectral solar irradiance by the
modeled SSI in Fig. 13 gives the effective
end-to-end sensitivity of the instrument with SI-traceability via correction
factor
Applying the conversions,
This two-dimensional spatial ground scene at 1233 nm from Flight 2 is radiometrically calibrated using the conversion values from Fig. 14.
This representation of a three-dimensional data cube is from a ground-scene scan over mixed desert and water to show the generic HySICS data products of spatial/spectral imagery.
Built under a NASA ESTO Instrument Incubator Program, the HySICS uses direct radiance measurements of the Sun to cross-calibrate hyperspectral images of other scenes, such as of the ground, Earth's atmosphere, or the Moon, with improved radiometric accuracies over similar instruments relying on indirect or diffused solar observations, on-orbit light sources, pre-launch calibrations, or measurements of vicarious ground sites. This measurement technique, utilizing three precisely characterized intensity-attenuation methods, enables direct in-flight calibrations relative to the spectral solar irradiance, which is a more stable and better-known reference than other space-based light sources, and allows SSI measurements to benchmark Earth ground scenes with radiometric accuracy and long-term precision greatly exceeding the capabilities of current space-based ground-imaging instruments. The demonstrated improvements from the HySICS were accomplished using the instrument's broadband optical design, which covers the entire reflected-solar spectrum and is based on a single flight-capable FPA. This design reduces mass, volume, power, and cost for air- or spaceflight instrumentation compared to multi-focal-plane designs intended to cover this broad spectral region.
Data cubes from HySICS's in-flight lunar scans provide spectral-radiance as well as spatially integrated irradiance measurements of the Moon (as viewed from the Earth) with improved radiometric accuracy than has as of yet been obtained from Earth-based measurements.
The HySICS's solar cross-calibration methods have been applied to provide
radiometrically accurate, SI-traceable spatial/spectral data cubes of ground
scenes and the Moon from two high-altitude balloon flights. Using all three
of its intensity-attenuation systems, the HySICS achieved net radiometric
intensity reductions of 10
Radiometric uncertainties based on the HySICS's solar cross-calibration
approach were characterized as a function of wavelength for the
balloon-flight data. The largest uncertainties were identified as being due
to FPA and grating efficiencies in the visible, which cause dominant
flat-field uncertainties as well as high shot and read noise, limited light sources used in laboratory calibrations of attenuations due to the
aperture-ratio method, and the low lunar-signals during the times of the
balloon flights. The quantified HySICS uncertainties were not limited by any
intrinsic aspect of the solar cross-calibration approach, as demonstrated by
the minimum pixel-level uncertainty of
Versions of the HySICS have been designed for accommodation on free-flyer spacecraft as well as the International Space Station, with either platform offering future instrument opportunities and the acquisition of scientifically valuable data. Studies are currently underway to manifest the HySICS on the CLARREO Pathfinder mission to improve spaceflight technology readiness, demonstrate the ability to achieve eventual CLARREO-mission climate-benchmark measurement requirements (Wielicki et al., 2013), and provide inter-calibrations of other on-orbit sensors (Roithmayr et al., 2014). The CLARREO Pathfinder/HySICS is planned for launch to the International Space Station in 2020.
This paper discusses the corrections and uncertainties that have been characterized from a variety of lab- and flight-based data to ultimately provide improved radiometric accuracies for future spaceflight hyperspectral imagery. With that emphasis on acquiring high-quality calibration data, only a few such final representative images were acquired during the limited flight time. The corrections described in this paper are being applied to these final HySICS balloon-flight data products of representative ground and lunar scans, such as shown in Figs. 15 and 17, and corresponding uncertainties to those data cubes are being produced. Those data, which include a few ground-scan scenes of both desert and clouds, an Earth-limb scan, and a lunar scan, will be available via request to the authors when completed, but are auxiliary to and not the primary focus of this paper.
The actual calibration data themselves are diverse, with flight data coming from the instrument's FPA and internal sensors, the WASP pointing system, and numerous other balloon-flight sensors, and with laboratory data coming from several light-source monitors, temperature sensors, and hand-written lab notebooks recording specific conditions during tests. No attempt has been made to consolidate these data for simple online distribution. Instead, this paper details the processes utilized and the results achieved from a combination of those many sources over the few years of laboratory testing and characterizations subsequent to and after the balloon flight.
All manuscript authors actively contributed to the HySICS or WASP calibrations, operations, and/or data analysis during and following Flight 2. Greg Kopp was the principal investigator of the HySICS and did the majority of the writing as the primary author of the manuscript. Second author Paul Smith completed the data analyses presented here and contributed directly to the writing, designed the flight operations software, and performed ground calibrations. Ginger Drake was the project manager and coordinated efforts between the HySICS team and the Columbia Scientific Balloon Facility for Flight 2. Joey Espejo was the optical designer and verified instrument performance prior to launch and during flight. Chris Belting, Zach Castleman, and Karl Heuerman designed the electrical and mechanical interfaces and control systems for the flight instrument, verified operation prior to launch, and performed ground calibrations. James Lanzi and David Stuchlik led the NASA/WFF WASP team that enabled the pointing accuracies needed for tracking the Sun and Moon and performed WASP operations during flight. The listed authors are most directly responsible for the results presented here, although many others at LASP and WFF contributed to the success of the HySICS balloon flights.
The authors declare that they have no conflict of interest.
We greatly appreciate the support of the NASA/WFF WASP team led by David Stuchlik and James Lanzi that enabled the HySICS pointing capabilities needed to demonstrated the instrument's solar cross-calibration approach.
This effort was funded by NASA's Earth Science Technology Office's Instrument Incubator Project under contract NNG04HZ05C as IIP-10-0019, and their enabling support is also greatly appreciated. Edited by: M. Zribi Reviewed by: two anonymous referees