Deep Impact - Navigation Images Report by Brian Carcich (DI Science Data Center) August 31, 2006 Revised December 21, 2006 (Additions from Bob Werner, JPL/DI AutoNav) 1. Summary =========== Navigation (NAV) data came into the Deep Impact Science Data Center (SDC) in a different form than the science data. However, the SDC pipeline pre-processed NAV data to make it look enough like science FITS data that the existing science data processing pipeline could process the NAV data. This report describes the differences between the NAV and science data and the steps taken to make the science data pipeline accommodate the NAV data. NAV data were used for optical navigation (OpNav), where the navigation team analyzed the OpNav images to calculate trajectory corrections, and for autonomous navigation (AutoNav), where software onboard the spacecraft used AutoNav images to compute the brightness centroid of the target body for autonomous trajectory corrections. AutoNav took over control of the flyby spacecraft about two hours before impact. The impactor spacecraft was on autonomous navigation after its release by the flyby spacecraft. 2. Details =========== The raw NAV data files generated from telemetry in the Advanced Multimission Mission Operations Systems (AMMOS) are in a completely different form than the raw science data. Bob Werner (JPL/DI AutoNav team) provided the description of the raw file format in sections 2.4 and 2.5 below. The differences between the NAV and science data are described next. 2.1 Raw filenames ------------------ A typical raw filename delivered by AMMOS to the SDC was NAV_OM_9000385.bin_2005_185T03_38_58.000 where Prefix : NAV indicates NAV data NAV type : _O indicates OpNAV, and _A indicates AutoNAV Instrument : M_ indicates MRI, I_ indicates ITS, and H_ indicates HRIV ExposureID : 9000385, same as for science data; image number within an exposure ID was always 1 of 1 Suffix : .bin indicates raw data AMMOS Date: : _2005_185T03_38_58 in YYYY_DOYTHH_MM_SS format and nominally the ground-received time (GRT) Suffix2: : .000 provides uniqueness when two images were processed by AMMOS within the same second 2.2 Instruments ---------------- The NAV images only came from the visible CCD instruments: HRIV, ITS, and MRI. No data from the IR spectrometer (HRII) came through the NAV system. 2.3 Image Format ----------------- The NAV data excluded the serial- and parallel-overclock pixels. The nominal raw NAV image was 1008x1008 pixels in size. This means that quadrant bias and smear had to be compensated for elsewhere or not at all. The quadrants were integrated into a single image and not delivered separately as was done with the science data, so some of the packet snips (see below) crossed quadrant boundaries. Finally, the MRI and ITS NAV images needed to be rotated 180 degrees, and the HRI images needed to be inverted (vertically flipped) to be consistent with the science FITS files. The data were always 16-bit data, and a lossless compression method called ''Ed's Algorithm'' was available but never used in flight. 2.4 Packets ------------ A single NAV observation was made up of one or more ''snips'', each of which was made up of one or more ''packets''. A snip was a rectangular (usually square) region of an image. Each snip was intended to be centered on an interesting portion of the image, for example the comet or a background star. The length of the edge of the snip square was specified in the instrument command that exposed the image. AutoNav automatically cropped the square if the nominal square-shaped snip would have exceeded the image boundaries. As the snips in a single raw file did not have to cover the entire image area, not all pixels in a single image were being returned in all cases. Snips could overlap, resulting in duplicate pixels in the downlinked file. The number of snips per image was determined by AutoNav, based on its prediction of the number of interesting objects in the image. If the instrument command specified an overly large snip, then the entire 1008x1008 pixel image was encoded as a single snip. Pixels in a snip were transcribed in raster-scan order from the original image. The first pixel was from the upper-left corner of the snip. The next pixels proceeded to the right across the snip. Then the next lower line was transcribed, and so on down to the lower-right corner of the snip. Snips themselves were encoded as one or more self-contained ''packets''. (Note: These snip packets were not aligned with downlink telemetry packets. Perhaps a better term would have been snip ''records''.) If a snip packet was received completely, then its portion of the snip and the image could be reconstructed with confidence, even if adjacent snip packets were damaged or missing. Snip packets never exceeded 1002 bytes in length. If an image's snip required more than 1002 bytes, multiple snip packets were created. 2.5 Header ----------- The NAV data did not have the same 100-byte header inserted into the byte stream from the spacecraft that the science data had. The NAV data had instead an 80-byte header that was prepended onto each snip packet of the image data. The header contained the items listed below. - Length of the packet (header + data), in bytes (2-byte integer). Never exceeded 1002. - Pixel and line coordinates of brightness centroid determined by AutoNav (two 4-byte floats). - Pseudo-Ephemeris Time (ET), that is the spacecraft clock (SCLK) time plus an offset to compensate for SCLK drift relative to Barycentric Dynamical Time (TDB) (8-byte double). Pseudo-ET was an attempt to turn SCLK time directly into ET seconds past J2000. Pseudo-ET was calculated by dividing 256 into SCLK count (in ticks) to get seconds then an offset value (nominally 64.184 sec) was subtracted. This scheme would have been satisfactory if SCLK had run at the correct rate, but it was off by about 1 second per day. The offset value was updated at least twice during cruise, but the DI project insisted the offset value must not be modified during encounter. - RA, DEC, and TWIST (radian) of the instrument boresight (three 8-byte doubles). In the FITS headers, the values were converted to degrees. The twist angle type was that used for the Galileo mission, where this simple relationship exits between the values of twist angle and celestial North clock angle (also in the FITS headers): celestial North clock angle = (270 - twist angle) mod 360. - Spacecraft attitude rates (angular rotation rates); Incorrect AutoNav code caused the three rates to be identical (three 4-byte floats). - Commanded exposure duration excluding the 3.5-millisecond minimum exposure duration (4-byte float). - NAIF Target ID, such as 1000093 for comet 9P/Tempel 1 (4-byte integer). - Detector (1 = ITS, 2 = MRI, 3 = HRIV) (1-byte integer). - Compression algorithm (0 = none, 2 = ''Ed's algorithm'') (1-byte integer). Always 0 -- Ed's compression algorithm was never used. - Image number (2-byte integer). - Image type (2-byte integer). - Pixel and line coordinates of the snip rectangle within the image (four 2-byte integers): left, right, top, bottom. These values refer to an image displayed top-down and left-to-right. These values were not included in the raw NAV FITS data produced by the Deep Impact data pipeline. - Pixel and line coordinates of first pixel (two 2-byte integers). As a snip might require multiple packets, this tells where the packet's data begin in the overall snip. These values refer to an image displayed top-down and left-to-right. These values were not included in the raw NAV FITS data produced by the Deep Impact data pipeline. 2.6 Temperatures and Voltages ------------------------------ No temperatures or voltages were available in the NAV headers. 3. Raw NAV FITS data ===================== In order to make the raw NAV data enough like the FITS science data so the pipeline could process the raw data, the following manual steps were taken: - The exposure ID was obtained from the raw data's filename. - The Pseudo-ET, after adjustment or offset to SCLK per a table provided by the NAV team, was used as the image stop time. The applied offset value was stored in the FITS header (PSEUOFFS). - The instrument keyword in the FITS header, INSTRUME, was set to NAVHRI, NAVITS, or NAVMRI (for the HRIV, ITS, or MRI instrument, respectively). This version of the raw NAV FITS images was archived at PDS. 4. Pre-processing for Calibration =================================== In order to make the NAV data enough like the science data so the pipeline could calibrate the raw NAV FITS images, the following manual steps were taken to rewrite the raw FITS header and/or prepare for the calibration. The pre-processed raw NAV FITS images were not archived in PDS. 4.1 FILTER keyword ------------------- The filter was set (HRI and MRI only) from the sequencing spreadsheets using exposure ID and day-of-year (DOY) as a lookup. 4.2 INTTIME keyword -------------------- The exposure time was converted from seconds to milliseconds and increased by 3.5 milliseconds. 4.3 FNDOY keyword ------------------ The DOY was extracted from the raw file name. 4.4 IMGH001 keyword -------------------- The lookup table was set to zero to indicate no compression. 4.5 CCDT, DELTAT keywords -------------------------- The CCD on-chip temperature was selected from the nearest science image for the same detector and stored as the CCDT keyword in the FITS header. In practice, the smoothed temperatures generated as part of the science calibration procedures overrode this value. The time between the NAV image and the nearest science image was stored as the DELTAT keyword in the FITS header. 4.6 FITS Extension for the Quality Map --------------------------------------- A FITS image extension was generated for each NAV image to indicate missing data, that is data outside all snips. 4.7 Overclock pixels and BIAS ------------------------------ Overclock pixels were also added during pre-processing: - Zero values were used for the parallel overclock pixels (used for smear removal) - Fixed bias values calculated by science team member Olivier Groussin were used to fill in the serial overclock pixels (used for bias removal). One bias value was calculated for each of the four CCD quadrants for each instrument. The four bias values were stored as the NAVBIAS1, NAVBIAS2, NAVBIAS3, and NAVBIAS4 in the FITS headers. NAVBIAS1 applies to the upper-left quadrant, NAVBIAS2 to the lower-left, NAVBIAS3 to the upper-right, and NAVBIAS4 to the lower-right quadrant. Quadrant designations assume a NAV FITS image is displayed bottom-to-top and left-to-right (lines increasing up and samples to the right). 5. Calibration Steps for NAV images ==================================== Once the raw NAV images were pre-processed, the resulting FITS files were similar to the raw science data so that the standard calibration could be applied. Calibrated NAV FITS images were archived in PDS. The following calibration options were enabled: - Flag saturated pixels - Linearize DN values - Subtract dark frame - Normalize by flat field - Despiking - Flag bad pixels The segmented nature of the NAV images, due to the discrete snips and the absence of real parallel overclock pixel data, required skipping these calibration steps: - Crosstalk and smear removal - Gap filling - MTF correction Also, the following calibration steps were disabled: - Normalize quadrant gains (already included in flat fields) - (Re-)Calibrate temperatures and voltages (raw DNs were not available) - Geometric calibration - Remove random gaussian noise - Correct for uneven bit weighting (never implemented for science data)