Menu: Skip within this page
How to Approach a PDS 3 Data Set
This page presents a basic description of what is included in a data set for those unfamiliar with the PDS3 archive structure. It also suggests a sequence of actions that new users seem to find helpful for getting familiar with the contents of a data set. You may work your way through a data set online or by downloading the entire collection.
Here is a typical sequence:
- Examine the root directory
- Read the catalog files
- Browse the documentation
- Check for data-browsing help
- Rummage through the data
- Inspect any remaining directories
1. Examine the root directory
Each data set is organized as a directory tree. In the root directory of the data set you will usually find an "AAREADME" file (plain text or possibly HTML) that should provide some basic info about what is included where in the data set. You'll probably also see a file called "VOLDESC.CAT", which you can safely ignore, and possibly an "ERRATA.TXT", which you should probably not ignore.
Most of the subdirectory names are dictated by the PDS Standards Requirements. The directories you will most likely encounter are:
- calib/: An optional directory that, when present, will contain calibration data files used to produce the data set.
- catalog/: A required directory containing files with very high-level information used to populate the central PDS catalog for searching.
- data/: In an SBN data set you will usually find the data files in a subdirectory called data/, though this is by convention rather than requirement. Some data sets will have multiple data directories under the root directory, usually named mnemonically for observing modes, mission phases, targets, or such. The "AAREADME" should make the subdirectory layout clear in these cases.
- document/: An optional directory containing document files that provide detailed descriptions and explanations of various topics related to the data, like calibration procedures, instrument descriptions, observatory logbooks, etc.
- index/: At a minimum, this required directory will contain files called index.tab, which contains the PDS manifest for the data set, and checksum.tab, which has MD5 checksums for everything in the data set. There might be additional table files containing scientific indices into the data (look for .lbl files describing the contents of the .tab files), or you might find scientifically interesting fields added to the index.tab file.
There may be additional optional subdirectories included with the data set, some of which are listed below. Each of these optional directories will include a plain text informational file with a name ending in info.txt. Look for the appropriate *info.txt file in these directories to learn more about their contents.
2. Read the catalog files
After you've dispensed with the "AAREADME", turn your attention to the catalog/ directory and the catinfo.txt file. This file will tell you what is in all the other files in the directory. You can find the instrument catalog file, for example, by looking up its name in the catinfo.txt file.
PDS catalog files provide overview information about various aspects of the data in a structured form that can be ingested directly into the PDS catalog database. There will always be a catalog file for the data set (usually called dataset.cat), which you should review carefully. Other catalog files will describe things like the mission, spacecraft and instrument, and you should review these as well - especially with regard to how they support the data set. Since these files are intended to provide high-level information, it is common for them to refer to other documents, either in the literature or included with the data in the document/ directory, that contain more detailed descriptions. You'll also usually find one catalog file containing a reference list. (We don't expect you to technically edit the references, but if you notice a mistake, please report it.)
Note for Windows Users
The files of interest in here have an extension of .cat and are simple text files. If you are a Windows user you may have to force Windows to open them with a specific file editor, because Windows has reserved the .cat extension for security catalog files and gets nervous when ordinary users start messing with .cat files. Use the "Open with" right-click option in your Windows Explorer window to select an editor for viewing the files. Notepad and Wordpad usually work well. If you're trying to look at a .cat file in Internet Explorer, you will be stymied by the browser. You will have to download the file (right-click and "Save target as") to your hard disk and then open it with any text editor. Alternately, you can use some browser other than IE. Google Chrome, for example, seems to be quite happy to display .cat files on Windows systems.
3. Browse the documentation
The docinfo.txt file provides a quick guide into the contents of the document/ directory. This is the place to look for things like detailed instrument descriptions, calibration instructions, observers logs, or anything that serves to describe or explain the data and isn't an observational record.
4. Check for data-browsing help
Very large data sets frequently have a top-level directory called browse/ which can provide a useful entry point for exploring the data. This directory may contain thumbnail representations of the data files, HTML files that can be opened in a browser for searching through the data, or specialized browsing indices (for use in Web browsers or spreadsheets, for example). There should be a browinfo.txt file in the directory to explain what has been provided.
5. Rummage through the data
At this point you should have enough information to dive into the data and know where to look if you have questions about what you find in the data files and PDS labels. The data/ directory itself may be subdivided in many ways, depending on the size of the data set. Small data sets may have no subdirectories; large mission data sets may have subdirectories for date or mission phase; and so on.
In the data directories you will likely see files with an .lbl extension. These are detached PDS label files; they are plain text and can be displayed in any text editor. If you don't see any .lbl files, the PDS labels will appear at the top of the data files themselves. The labels, attached or detached, will have internal pointers to the data, a structural description of the observational data (usually at the bottom), and a series of keyword = value statements that should provide all the information essential to understanding and working with the data. Definitions for the keywords used in PDS labels are either in the Planetary Science Data Dictionary data base, or in a local data dictionary provided by the mission which should be included in the document/ directory of the data set.
6. Inspect any remaining directories
Occasionally there are other directories included with the data that provide additional supporting information. For example:
- geometry/: This directory contains geometry data (pointing, target ephemeris, etc.). Sometimes this will be in the form of tables of text; sometimes this directory will contain kernel files for use with the NAIF SPICE system. The geominfo.txt file will provide content information.
- label/: This directory contains label include files - the repetitive parts of labels (the data structure, typically) extracted to simplify either the labels themselves or configuration management or both. If you see a line in a label that looks like this:
^STRUCTURE = "FILE.FMT"
then you should find the "FILE.FMT" file in the label directory. - software/: It is rare to see software delivered as software because PDS will not host, maintain or port mission software. A softinfo.txt file should list the contents of the directory. More often you will encounter software source code provided to document a calibration algorithm or as an example of how to read a particularly complex data format in the document/ directory.
- extras/: In fact, this directory is not considered to be part of the permanent archive. It is used to hold things like specialized display code that might be useful on a few systems for a few years while the data are still new, but which is not essential to the long-term viability of the data. An extrinfo.txt file should explain what is in the directory and what it might be good for.
One last helpful link
There are several file types within a data set. We do have a file types page that provides additional information on those file extensions, the information they contain, how to read them, and where they may be found in a data set.
