VIRTIS
ISTITUTO NAZIONALE DI ASTROFISICA


           ROSETTA - VIRTIS
             Derived Data
To Planetary Science Archive Interface Control Document

VIR-INAF-IC-008
Issue 1.1
29-01-2020

Prepared by:
     Romolo Politi
     Gianrico Filacchione
Approved by:
     Fabrizio Capaccioni

1 Introduction
1.1 Purpose and Scope

The purpose of this EAICD (Experimenter to (Science) Archive Interface Control
Document) is two folds. First it provides users of the VIRTIS instrument with
detailed description of the derived products and a reference list of the
articles in which is explained how they were generated, including data
sources and destinations. Secondly, it is the official interface between our
instrument team and the archiving authority.

1.2 Archiving Authorities

The Planetary Data System Standard is used as archiving standard by

* NASA for U.S. planetary missions, implemented by PDS
* ESA for European planetary missions, implemented by the Research and
   Scientific Support Department (RSSD) of ESA



1.2.1 ESA's Planetary Science Archive (PSA)

ESA implements an online science archive, the PSA,

* to support and ease data ingestion
* to offer additional services to the scientific user community and science
   operations teams as e.g.

    o search queries that allow searches across instruments, missions and
      scientific disciplines
    o several data delivery options as

* direct download of data products, linked files and data sets
* ftp download of data products, linked files and data sets

The PSA aims for online ingestion of logical archive volumes and will offer the
creation of physical archive volumes on request.

1.3 Contents

This document describes the data flow of the VIRTIS instrument Level 5 data
for their insertion into the PSA. It includes information on how data were
processed, formatted, labeled and uniquely identified. The document discusses
general naming schemes for data volumes, data sets, and data files. Standards
used to generate the products are explained. The design of the data set
structure and the data product is given. Examples of the data labels are
given.

1.4 Intended Readership

The staff of the archiving authority (Planetary Science Archive, ESA, RSSD,
design team) and any potential user of the VIRTIS data in the scientific
community.

1.5 Applicable Documents

[AD 1]. Planetary Data System Preparation Workbook, February 1, 1995, Version
        3.1, JPL, D-7669, Part1

[AD 2]. Planetary Data System Standards Reference, August 3, 2003, Version 3.6,
        JPL, D-7669, Part 2

[AD 3]. Rosetta Archive Generation, Validation and Transfer Plan, January 10,
        2006, RO-EST-PL- 5011

[AD 4]. Rosetta Archive Conventions, January 10, 2006, RO-EST-TN-3372

[AD 5]. Planetary Science Data Dictionary, Revision D, JPL D-7116, July 15, 1996

[AD 6]. VIRTIS Data Archive Format, April 29, 2002, VIR-ORS-RS-1146, Version 3.4

[AD 7]. Update to VIRTIS Rosetta Archive Format, July 20, 2006, VIR-ORS-RS-2251,
        Issue 2.5

[AD 8]. VIRTIS EAICD, VIR-INAF-007, January, 2020, issue 4.8

1.6 Relationships to Other Interfaces

The Maps (Level 5 data) described in this document were generated from
calibrated data cubes [AD 8]; changes in the calibration process could affect
the Level 5 data. Any future reprocessing of the data, for whatever reason,
even if the labels are not affected, that will have an impact on the Level 5
data shall be described and new version of the Maps, when published, will be
added  and described in this document.

1.7 Acronyms and Abbreviations

EDR: Experimental Data Record
EGSE: Electrical Ground Support Equipment
MTP: Medium Term Plan
STP: Short Term Plan

1.8 Contact Names and Addresses

* INAF-IAPS, Romolo Politi, +39 06 4993 4710, romolo.politi@inaf.it

* INAF-IAPS, Gianrico Filacchione, +39 06 4993 4454,
   Gianrico.Filacchione@inaf.it



2 Overview of Data Handling Process and Product Generation

2.1 Data Handling Process

Table 2-1 describes the VIRTIS data processing levels as used in this document.
The archive contains the DDR (level 5) data.

|----------------|------------------------------------------------------|
| Processing     | PRODUCT_TYPE Description                             |
| Level ID       |                                                      |
| (CODMAC level) |                                                      |
|----------------|------------------------------------------------------|
| 1              | UDR   Raw data that have been separated by instrument|
|                |       and sorted by orbit number. Contains all TM    |
|                |       from VIRTIS (science and housekeeping data).   |
|----------------|------------------------------------------------------|
| 2              | EDR   PDS formatted data, including decompressed     |
|                |       science data, housekeeping selection, and      |
|                |       geometry data. Science data are still in DN.   |
|----------------|------------------------------------------------------|
| 3              | RDR   Science data calibrated in physical units      |
|                |       (radiance) + timing as OBT                     |
|----------------|------------------------------------------------------|
| 5              | DDR   High-level derived product such as             |
|                |       distribution maps...                           |
|----------------|------------------------------------------------------|
Table 2-1: Data level definition.

2.2 Data Description

In this data volume are present maps of selected spectral marker or Spectral
Indicators. These parameters helps considerably in reducing the dimensionality
of the data, hence their analysis, still preserving the relevant information
contained in the spectra. As all the maps that will be injected into PSA as
Level 5 products will only be those contained in published papers, the details
for the derivation of the Spectral Indicators will be provided in the paper
themselves. However, as we have identified a limited number of typology of
spectral markers which will be used, here below we provide a brief description
of the full set of the possible spectral markers (or Spectral Indicators) and
their potential relevance for the data interpretation. Should the need arise
for new Spectral Indicators the present document will be updated and their
description added.

  - TP Temperature; this represent the surface temperature as derived from the
    IR radiance in the 4-5 um range. Details of the Bayesian approach adopted
    for the derivation of the Surface Temperature are given in Tosi, F. et al.
    Thermal measurements of dark and bright surface features on Vesta as
    derived from Dawn/VIR. Icarus 240, 36 - 57 (2014).

  - EM Emissivity; this is the emissivity of the surface in the range 4-5 um;
    the emissivity is derived as a byproduct of the application of the
    algorithm adopted for the retrieval of the surface temperature.

  - BA Band Area; for a given identified absorption feature we can calculate
    4 descriptive parameters (Area, Depth, Centre wavelength and skewedness,
    or asymmetry) which are correlated to specific information; the Band Area
    is derived according to the formula in [IMAGE002].
    This in fact represent the area contained under the continuum, calculated
    as a linear fit of the reflectance between the two wavelengths located at
    the shoulders of the band indicated as a and b in the formula (see also
    Raponi et al. The temporal evolution of exposed water-ice-rich areas on
    the surface of 67P/CG: spectral analysis, MNRAS 2016).

  - BD Band Depth; the standard definition of the band depth follows the
    formula:

                               BD = 1 - Rb/Rc

    Where Rb is the reflectance of the band at the minimum of the absorption
    (after removal of the continuum), and Rc is the reflectance of the
    continuum at the same wavelength. The BD is a measure of the abundance of
    absorbers in the soil.

  - BC Band Centre; The Band Centre is calculated, after continuum removal, as
    the position of the minimum of a polynomial fit to the band shape. The
    continuum is calculated interpolating the reflectances at the two
    wavelengths located at the shoulders of the band. The BC is related to the
    composition of the surface.

  - BS band Skewness; the skewness is an evaluation of the asymmetry of the
    band using the  same polynomial fit used for the BC.

  - SL Spectral Slope; Spectral slopes are calculated as the linear fit to a
    portion of the spectrum between two specified start and stop wavelengths.

  - AL Albedo; this is the absolute reflectance (photometrically corrected)
    at a given wavelength.

  - BR Band Ratio; this is the ratio between the area of two different bands
    and is used to evaluate the abundance ratio of specific compounds.

  - WI Water Ice abundance; this is derived as a best fit, using a radiative
    transfer model, to the measured reflectance of a mixtures of various
    end-members (pure components of the mixtures) for which the optical
    constants are known.

All the geometric data are computed using the shape mode 5
(cg-scp-shap5-V1.1_Cedric.TPC)

The algorithms used to produce the maps and a deep discussion on the various
parameters represented in the maps and their scientific significance are
described in the scientific publication from which they were extracted. The
reference to the publication itself is reported in the label file. The full
set of references to the publication used is reported in the REF.CAT file in
the CATALOG folder.


2.3 Data Format

The maps in this volume are in table format. This is an unusual format for a
map, but the comet shape is also unusual. For the time being we used the
standard approach of cylindrical projection with lat/lon coordinates, however
due to the peculiar shape of the comet, we can have different point with the
same lat/lon coordinates, especially near the Neck region. For this reason,
we are working to generate maps not related to the lat/lon coordinate system
but expressed in terms of ortogonal coordinates X,Y,Z, relating each map to
the adopted shape model. For the above reason in the present dataset we
adopted a format which will be compatible with the final format of the maps.


3 Archive Format and Content

This section describes the features of the VIRTIS Derived Product Archive
volumes, including the file names, file contents, and file types, which apply to
VIRTIS Derived Data Sets. More details on the data sets are provided in Section
4.

3.1 Format and Conventions

3.1.1 Deliveries and Archive Volume Format

Delivery of data from the VIRTIS team to the PSA for archiving is done through
Internet, using the PVV tool, according to the release concept described in [AD
3]. In conformity with guidelines also provided in [AD 3], data are organized so
that one VIRTIS data set will coincide with a single logical volume. An example
is shown in Table 3-1.


|----------------------------------------|-----------------------|
| Data Set Name                          |  Volume ID            |
|----------------------------------------|-----------------------|
| ROSETTA-ORBITER 67P VIRTIS 5 MAPS V1.0 |  ROVIR_3007           |
|----------------------------------------|-----------------------|

Table 3-1: VIRTIS Data Sets and corresponding Volume ID's example.

3.1.2 Data Set ID Formation

The value of this keyword is formed following the PDS rules and the Rosetta
archive conventions as written in[AD 4]. For each delivery, the raw data from a
mission phase will be found in a dataset. An example is shown in Table 3-2


|----------------------------------------|-----------------------------|
| Data Set Name                          | Data Set ID                 |
|----------------------------------------|-----------------------------|
| ROSETTA-ORBITER 67P VIRTIS 5 MAPS V1.0 | RO-C-VIRTIS-5-67P-MAPS-V1.0 |
|----------------------------------------|-----------------------------|

Table 3-2: VIRTIS Data Set ID's example.

3.1.3 Data Directory Naming Convention

The /data directory is structured in subdirectories potentially containing
different types of products. At the moment only the MAP subdirectory is
implemented. The products are split in two sub folder to divide the product
coming from VIRTIS-M and from VIRTIS-H:

/data/MAP/VIRTIS_M: contains maps derived by VIRTIS-M Datasets;
/data/MAP/VIRTIS_H: contains maps derived by VIRTIS-H Datasets;

It must be noted that so far only VIRTIS-M L5 data are available and placed
in these directories

3.1.4 Filenaming Convention

The data are named using the following format:
<code>_<target>_<time>_<version>.map

The <code> field is composed of two parts: <type><subtype>.
The sub-field <type> is 2 alphabetic characters long, which describe the
spectral parameters that can be represented in the maps; their possible values
are reported in Table 3-3.

|------|----------------|
| Type | Description    |
|------|----------------|
| TP   | Temperature    |
| EM   | Emissivity     |
| BA   | Band Area      |
| BD   | Band Depth     |
| BC   | Band Center    |
| BS   | Band Skewness  |
| SL   | Spectral Slope |
| AL   | Albedo         |
| BR   | Band Ratio     |
| WI   | Water Ice      |
|------|----------------|

Table 3-3: Values for the sub-field type.

The sub-field <subtype> is 2 numeric character long and is optional. The admitted
combinations type subtypes are reported in Table 3-4.

|------|---------|----------------------------------------------------|
| Type | Subtype | Description                                        |
|------|---------|----------------------------------------------------|
| AL   | 01      | Albedo at 550 nm                                   |
| BA   | 01      | Band Area at 3.2 micron                            |
| BC   | 01      | Band center at 3200 nm                             |
|      | 02      | Band Area at 3.2 micron (different algorithm)      |
| BD   | 01      | Band depth at 3200 nm                              |
|      | 02      | Band depth at 2000 nm                              |
| SL   | 01      | Spectral slope 500-800 nm                          |
|      | 02      | Spectral slope 1000-2500 nm                        |
|      | 03      | Spectral slope 0,5-0,8 micron(different algorithm) |
|      | 04      | Spectral slope 1,0-2,35 micron                     |
| WI   | 01      | Water ice abundance                                |
|------|---------|----------------------------------------------------|

Table 3-4: Admitted combination of type and subtype fields.

The <target> field is 3 characters long and describe the scale of the map: if
it is the full nucleus (global map) is global, or if it is an area of the
comet or of it is a specific feature on the surface. The admitted values are
reported in Table 3-5 although so far only global maps are uploaded.

|--------|-------------------------------|
| Target | Description                   |
|--------|-------------------------------|
| GLB    | Global                        |
| LOC    | Local, specific comet area    |
| Tar    | Target Name, specific feature |
|--------|-------------------------------|

Table 3-5: Admitted values for the target field.

The time field if formed by 2 sub fields, <start_time>_<stop_time>, and
describe the start and stop phase, extremes included. Each field is formed by
1 char and 3 numbers. The first character could be M or S and indicate if the
start or stop field is a medium or a short-term plan period, the numbers are
the sequential number for the MTP and the STP.

E.g. SL01_GLB_M006_M006_V01.TAB: first version of the Nucleus Global Map
reporting the Spectral Slope in the 500-800 nm spectral range for the MTP006.


3.2 Standards Used in Data Product Generation

3.2.1 PDS Standards

All the data released by the VIRTIS Team for archiving are compliant with the
Planetary Data System (PDS3) standard. This standard imposes requirements on
several aspects of the data product generation process, among which the need for
a detailed documentation describing the origin, structure and processing
undergone by data, for their accurate location in space and time, and in general
for all auxiliary and ancillary data which are needed for the scientific use of
the data products. This information has to be provided in an Object Description
Language (ODL), in the format keyword = value, where keyword is a standard term
used to label a parameter (e.g. latitude), and value is any allowed information
quantifying that parameter.

3.2.1.1 START_TIME and STOP_TIME values formation

The PDS formation rule for dates and time in UTC is: YYYY-MM-DDThh:mm:ss.fff or

YYYY-DDDThh:mm:ss.fff, with:

* YYYY year (0000-9999);

* MM month (01-12);

* DD day of month (01-31);

* DDD day of year (001-366);

* T date/time separator;

* hh hour (00-23);

* mm minute (00-59);

* ss second (00-59);

* fff fractions of second (000-999) (restricted to 3 digits).

3.2.2 Reference Systems

During the cruise phase of Rosetta, always the planetocentric body-fixed
rotating coordinate system is used in order to compute geometric quantities
relative to targets in the solar system. The planetocentric latitude is the
angle between the equatorial plane and a vector connecting the point of interest
and the origin of the coordinate system. Latitudes are defined to be positive in
the northern hemisphere of the body, where north is in the direction of Earth's
angular momentum vector, i.e., pointing toward the northern hemisphere of the
solar system invariant plane. Longitudes increase toward the east, making the
planetocentric system right-handed.

The easternmost (rightmost) longitude of a target is the maximum numerical value
of longitude unless it crosses the Prime Meridian. For the Earth and the Moon,
PDS supports the traditional use of the range [-180,180] in which case the
easternmost (rightmost) longitude is the maximum numerical value of longitude
unless it crosses 180. As regards the longitude, for the Earth and the Moon, the
traditional use of the range [-180,180] is allowed. More information on the
subject can be found in VIRTISROS_GEOMETRY.PDF released within the data-set.

3.2.3 Volume Set

As the concept of a volume as defined in the PDS standard is based on physical
media, e.g. CD-Rs, the PSA does not use the name volume. Instead, the concept of
deliveries is defined for the PSA and the term delivery is used for the PSA.
However, here and in the following sections we will use the word "volume" to
refer to a standard PDS directory structure for a data set in which the entire
data set consists of a single (virtual) volume. Different VIRTIS data sets will
be organized as separate virtual volumes, and the concept of volume set will not
be used.

3.2.4 Data Set

The value of the keyword DATA_SET_NAME is formed following the PDS rules and the
Rosetta archive conventions as written in [AD 4], so, for example, the
DATA_SET_NAME for first data set delivery will have the value:

ROSETTA-ORBITER 67P VIRTIS 5 MAPS V1.0

3.2.5 Directories

VIRTIS data sets are organized into one data set on one virtual volume and use
the standard PDS volume structure. This standard structure is described in
Section 19.3 of [AD 2], and shown in the figure below. For the derived dataset
not all the directories have been implemented as described in chapter 3.1.3.
The content of each directory shown in Figure 3 1is detailed in the following
sections.


3.2.5.1 RootDirectory

Files in this directory are provided by the VIRTIS science team, and will remain
the same across different volumes.

   |--------------|----------------------------------------------------|
   | File Name    |                 File Contents                      |
   |--------------|----------------------------------------------------|
   | AAREADME.TXT | Volume content and format information              |
   |--------------|----------------------------------------------------|
   | VOLDESC.CAT  | Description of the contents of the volume in a PDS |
   |              | format readable by both humans and computers       |
   |--------------|----------------------------------------------------|

Table 3-6: Files located in the root directory of a VIRTIS data volume.

3.2.5.2 CALIBRATION Directory

The CALIBRATION directory is not required in this version of the dataset and
has not been populated.

3.2.5.3 Catalog Directory

Files in this directory are catalogue files, that is files containing PDS
catalogue objects. Such objects provide high-level information suitable for
loading into a database to facilitate searches across data sets, collections and
volumes. These files are provided by the VIRTIS science team, with the
concurrence of the PSA, and will remain the same across different volumes.


     |------------------|-----------------------------------------------|
     | File Name        | File Contents                                 |
     |------------------|-----------------------------------------------|
     | CATINFO.TXT      | Text description of the directory contents    |
     |------------------|-----------------------------------------------|
     | MISSION.CAT      | PDS catalogue object for the mission          |
     |------------------|-----------------------------------------------|
     | INSTHOST.CAT     | PDS catalogue object for the spacecraft       |
     |------------------|-----------------------------------------------|
     | INST.CAT         | PDS catalogue object for the instrument       |
     |------------------|-----------------------------------------------|
     | TARGET.CAT       | PDS catalogue object for the targets          |
     |------------------|-----------------------------------------------|
     | DATASET.CAT      | PDS catalogue object for the VIRTIS data set  |
     |------------------|-----------------------------------------------|
     | REF.CAT          | PDS catalogue object for references appearing |
     |                  | in the documentation                          |
     |------------------|-----------------------------------------------|

Table 3-7: Files located in the CATALOG subdirectory of a VIRTIS data volume.

3.2.5.4 Index Directory

This directory contains indexes, that is files with information that allows a
user to locate data of interest. Within the Planetary Science Archive (PSA),
index files fulfill two more purposes. First, some index files are read by
database software and allow the ingestion of additional parameters into the
database. Secondly, the PSA is using the index files to check for correct
deliveries of data set revisions into the PSA. Indexes are written as
INDEX_TABLE objects, that is a specific type of PDS ASCII TABLE objects, and are
provided with detached PDS label files.

The set of index files for VIRTIS, as required in [AD 3], is:

     |---------------|--------------------------------------------------|
     | File Name     | File Contents                                    |
     |---------------|--------------------------------------------------|
     | INDXINFO.TXT  | Text description of the directory contents       |
     |---------------|--------------------------------------------------|
     | INDEX.LBL     | Detached PDS label to describe INDEX.TAB         |
     |---------------|--------------------------------------------------|
     | INDEX.TAB     | PDS table, listing all files in the DATA         |
     |               | directory for the corresponding release and      |
     |               | revision.                                        |
     |---------------|--------------------------------------------------|

Table 3-8: Files located in the INDEX subdirectory of a VIRTIS data volume.

The Index File (.LBL and .TAB) contains a lists all data files on the volume,
the path to the directory containing the file and the name of the file in the
archive. For each file information is given about time at which data in the file
begin/end, number of rows and number of bytes in rows in the .TAB file.

3.2.5.5 BROWSE Directory and Browse Files

The BROWSE directory was not implemented in this version of the dataset.

3.2.5.6 GEOMETRY Directory

The GEOMETRY directory was not implemented in this version of the dataset.

3.2.5.7 Software Directory

The SOFTWARE directory was not implemented in this version of the dataset.

3.2.5.8 Gazetter Directory

The GAZETTER directory was not implemented in this version of the dataset.

3.2.5.9 Label Directory

The LABEL directory was not implemented in this version of the dataset.

3.2.5.10 Document Directory

Files in this directory are provided by the VIRTIS science team, and are the
same for all the volumes.

 |---------------------------------|------------------------------------|
 | File Name                       | File Contents                      |
 |---------------------------------|------------------------------------|
 | DOCINFO.TXT                     | Text description of the directory  |
 |                                 | contents                           |
 |---------------------------------|------------------------------------|
 | **********.LBL                  | PDS labels for all the documents   |
 |                                 | contained in the directory         |
 |---------------------------------|------------------------------------|
 | RO_VIRTIS_MAP_EAICD.AS          | VIRTIS EAICD (this document) in    |
 |                                 | ASCII text                         |
 |---------------------------------|------------------------------------|
 | RO_VIRTIS_MAP_EAICD.PDF         | VIRTIS EAICD (this document) in    |
 |                                 | PDF format                         |
 |---------------------------------|------------------------------------|

Table 3-9: Files located in the DOCUMENT subdirectory of a VIRTIS data volume.

3.2.5.11 Extras Directory

The EXTRA directory was not implemented in this version of the dataset.

3.2.5.12 Data Directory

The DATA directory is structured in subdirectories containing different types
of products. At the moment, only the MAP subdirectory is implemented. The
products are split in two sub-folders to distinguish the products coming
from the two VIRTIS channel, although no Level 5 data have been generated so
far for the VIRTIS-H channel:

/data/MAP/VIRTIS_M: contains maps derived by VIRTIS-M Datasets;
/data/MAP/VIRTIS_H: contains maps derived by VIRTIS-H Datasets (Not yet Used);

It must be noted that so far only VIRTIS-M L5 data are available and placed
in these directories