From TSDS

SPASE Numerical Data use cases

Contents Overview Outline Use 1. Use 1. continued Use 1. continued Use 2. Use 3. Use 4. Use 4. continued

1. Overview

• We need a clear vision for the intended use of SPASE Numerical Data records.
• I was not able to use them when creating catalog-level metadata for TSDS and so I wondered what they are meant to be used for.
• I came up with four possibilities inferred by inspection of a few records.
• As implemented, the SPASE Numerical Data records are not good at any of the four possibilities.
• We need to decide on the use-case for these SPASE records. Without agreement, they will not be useful for any of the four purposes. Some of the use-cases require a significant time investment for generation and maintenance. This should be considered.
• Before making a push for "completeness" of the Numerical Data SPASE record inventory, we need to decide if the completed inventory will be useful for at least one use-case.

2. Outline

Inspection of SPASE Numerical Data records indicates that it could be intended for use for:

1. To inform the user where data are available from. (A science use-case.)
2. As a replacement for a README. (A science use-case.)
3. For automated machine processing of ASCII files. (An automation use-case.)
4. For connecting a parameter ID from one system to another system. (An automation use-case.)

We need to decide on the use-case for these SPASE records so that they are useful for at least one thing.

In the following I justify my claim that existing SPASE records are not very useful for any of these four use cases.

3. Use 1.

1. To inform the user where these data are available from.

Consider this search on "OMNI2" [1], results of which are shown in the image.

Issues: The link SPDF CDF ftp area data is to files that have only a few parameters listed in the SPASE record - what does this mean? As a science user presented with this, I ask: Why are these data are located at five different places? Will I get identical data from each URL? If yes, why do we need five choices? If they are not identical, which one should I use? Which SPASE ID is associated with the data from each resource?

4. Use 1. continued

• This page http://spdf.gsfc.nasa.gov/research/visualization/visbard/download.html has a link to this file http://spdf.gsfc.nasa.gov/research/visualization/visbard/sample_data/Heliosphere.tar.gz which contains merged emphemeris data from COHOWeb.
• Should this very useful file show up in a search for COHOWeb (it does not)?

5. Use 1. continued

• This directory ftp://nssdcftp.gsfc.nasa.gov/spacecraft_data/omni/ has both yearly files and a merged file: ftp://nssdcftp.gsfc.nasa.gov/spacecraft_data/omni/omni2_all_years.dat
• The second file is a very useful file. Presented with five links here [2], would a science user find it?
• Richard Denton has created a very popular file [3] containing Tsyganenko inputs that includes some of the parameters from OMNI2. Should the OMNI2 SPASE record reference Denton's data set? Or vice-versa?
• The OMNI2 data are available from many other places - (SPIDR, Datashop, etc.). Should they appear in a VxO search for OMNI2?

6. Use 2.

2. As a replacement to a README.

• If yes, then lots of information in the original README has been lost (c.f., [4] versus [5] or [6]) is the README author OK with this?
• Why doesn't the SPASE record provide a direct link to the definitive "master" documentation? (A science user may not want to visit five links to make sure that they have found all of the documentation.)

7. Use 3.

3. For automated machine processing of the ASCII files.

• This OMNI2 SPASE record [7] notes that the cadence of the ap Index is PT3H. But in the OMNI2 files, the cadence is PT1H.

This would confuse an automated processor.

Parameter:
Name: ap Index
ParameterKey: Column 50
Description: ap index
Cadence: PT3H
Support:
SupportQuantity: Other

8. Use 4.

4. For connecting a parameter ID from one system to another system.

Issues:

• To make TSDS work, I need a parameter ID so that I can form a data request to CDAWeb. CDAWeb's web service uses IDs that are quite different than what is found in a typical SPASE record: [8]
• This SPASE record [9] seems to point to the same data product as [10]. Which one should I use?

9. Use 4. continued

• The 5th result of a search on "ap index" at VSPO points me to this (broken) link: [11]
• CDAWeb has a variable named PC_N_INDEX1800 in its master catalog. What does 1800 mean? How am I to figure out that this is the same parameter as listed in the OMNI2 SPASE record, which contains:

Parameter:
Name: Polar Cap (N) Index
ParameterKey: Column 52
Description: Polar Cap Index based on Thule
Cadence: PT1H
Support:
SupportQuantity: Other

Ideally TSDS would say "You are looking at PC_N_INDEX1800" and provide a link to the metadata. At present, there is no mapping between this variable name and the SPASE variable name. Even if there was, is the SPASE record the master documentation? I don't want to tell the user "to find metadata, look at this SPASE record and compare it to this README text file. I am not sure which is more up-to-date.".