[lug] MEAS Data file format

[Wayde Allen]

On Fri, 11 Aug 2000, Bruce Raup wrote:

> >    #CUSTOMER:      - Who these data belong to
> 
> If data are produced by one person (or group) and archived by another
> (which is an issue I face), then how about having, instead, the pair
> #DATA_HOME: and #DATA_PRODUCER: ?  If yet a 3rd party owns the data, you
> could have a #DATA_OWNER: as well, but perhaps that's too complicated.

No, that is a good idea and not particularly complicated.

> >    #FREQSCALE:     - The frequency units (Hz, KHz, MHz, GHz, etc.)
> 
> How about generalizing this to #UNITS, or even #UNITn:, where n is the
> column number?  Perhaps having #NUM_COLS: would also be useful.

That makes a LOT of sense.  I only chose the #FREQSCALE: tag since I was
struggling with frequency units the other day and made this one up.  I do
think that a generic units tag would be much more general.  I also
discussed a column description tag with a coworker yesterday, but need to
iron out the syntax.  It seems that the units could be specified for each
column.

> > #DATE: Tuesday, April 18, 2000
> 
> I suggest the ISO standards for date and time (e.g. 2000-04-18).  See
> http://www.cl.cam.ac.uk/~mgk25/iso-time.html.  Such a format is not tied
> to a particular human language, and is easier to handle by a computer.

I definitely agree!  

> [As a side issue, the text at the URL above doesn't address how to
> represent a range of dates.  I've begun to use things like 2000-09-17~22
> to express a range within a given month.  Has anyone seen a standard way
> to express date ranges?]

No, I'm afraid I can't help with that.

> In general I like this.  When I've had to come up with similar formats,
> I've never bothered to search for existing similar formats.  It looks like
> you've done some searching, i.e. you based this format on another
> one.  Are there other similar formats out there that might be directly
> usable?

Aside from the CITIFILE I've looked at the Common Data File format (CDF). 
The CITIFILE was designed for data exchange between network analyzers and
radio frequency simulation packages.  It is fine if you are exchanging
device S-parameters, but has some limitations for multidimensional
datasets.  It also has a 60 character line limit to "make it more humanly
readable", but that means that the columns are distributed down the file
rather than accross.  I don't particularly find that humanly readable, and
you usually end up writing a multipass algorithm to locate all the pieces
and reassemble them.  The CDF format
<http://nssdc.gsfc.nasa.gov/cdf/cdf_home.html> looks good, but is a set of
libraries that you compile into your code.  This results in a binary
structure that can hold very complex data structures (images, vectors,
etc.).  In our case, I wanted something that didn't require the use of a
compiler, and wouldn't result in a binary format.  There are almost
certainly others, but these are what I'm aware of.


- Wayde
  (wallen at boulder.nist.gov)