[lug] RE: [FRPythoneers] MEAS Data file format

[Wayde Allen]

On Fri, 11 Aug 2000, Vickroy, Jim wrote:

> Out of curiosity what are your reasons for steering away from XML or
> relational data bases?

Actually I have personally been interested in pursuing the use of XML for
exactly this purpose.  It isn't so much that I'm trying to steer away from
it.  The problem is more an issue of simplicity in the structure, and
familiarity with the design.  In this culture the use of the "flat file"
row and column data format as well as the CITIFILE is common.

As far as the relational database issue goes, we are moving in that
direction.  However, these files are designed to work in the intermediate
space between processes.  The idea is that a measurement system could
generate this kind of file and/or also enter data directly into a remote
database via the NIST network.  Basically, not all of our systems are
easily networked at this time.  This is especially true for systems
running on the older HP computers running Rocky Mountain Basic.  These
machines are pretty network ignorant.

The other thing is that we need to be able to allow the researchers to
develop their own analysis code however they want.  Not all of these
people are, or want to be, well versed in developing SQL queries or
database programming.  Using such an intermediate file format allows the
database programming to be separated from the data analysis routines.  In
other words you've got to get these data into and out of the database in
some form.  You probably don't want to do your uncertainty analysis
directly in the database tables, nor can we really tell a researcher what
software they should use.  That is really up to them.  We can tell them
what their datafile will look like though. 

- Wayde
  (wallen at boulder.nist.gov)