REAL TIME ACCESS TO GBT DATA

* SDFITS needs to run continuously during GBT data acquisition.  At the end of
  each scan, it must append the new SDFITS processed data to the output file,
  'SD.fits'.  (Obviously we shall use the GBT file naming protocol, but here
  for clarity I shall call this output file something rational.)

* Thus for any given GBT observing run (defined here as a sidereal pass), 
  a single SD.fits file will have all the data.  We shall focus 
  on the SDFITS  avg.acs mode which is the only thing we use for the 
  3-Helium project. 

* There are three relevant procedures:
  
  1. init_data,SDFITS_file_name
     
     defines the SDFITS data file !datafile

     scans !datafile  in order to determine the number of !data_points
     per spectral record.  gbt_idl assumes that this number does not change.
     this procedure *defines* !data_points which is then used to define
     the internal data structure {gbt_data}.
     defines !kount to be the number of records in !datafile

  2. make_ONLINE,records_per_scan

     takes !datafile and converts it to {gbt_data} format record by record. 
     writes each {gbt_data} record to the ONLINE data file !online_data
     !kon  counts number of ONLINE data records currently in ONLINE file
     recs_per_scan is the number of subcorrelators per scan#

3.   update

     uses !datafile to concatenate any new data in this file to the ONLINE file.
     !kon  counts number of ONLINE data records currently in ONLINE file

=================================================================================

The continuous, on-line version of SDFITS is called SDFITS-TEST.  It operates
at the scan level, i.e. appends new FITS files to the existing SDFITS output
file at the end of each scan.  As of 1 July 2004, this code operated very 
well for us, albeit with a one-scan 'pipeline' lag.  At the end of an
observing run this feature meant that one had to burn a dummy scan number
in order that SDFITS-TEST processed the last real data scan.

Syntax:

SDFITS-TEST -online -mode=avg  /home/gbtdata/AGBT041_021_01 /users/tbania/idl/data/jn04

This command runs SDFITS continuouly whilst observing, appending new scans
to the output file:  /users/tbania/idl/data/jn04.avg.acs.fits as they arrive.
It also automatically processes all other connected backends, but at this
juncture all are processed in avg mode.

===========================================================================================
REAL TIME DATA ACCESS DURING OBSERVING REQUIRES UPDATING THE ONLINE.GBT DATA FILE

To access data real time whilst observing:

1. In a unique X-window run:

sdfits-test -online -mode=avg /home/gbtdata/AGBT_your_project_file  /users/you/idl/data/fname

 SDFITS will run continuously while you are observing, making 
 /users/you/idl/data/fname.avg.XXX.fits files at the end of each scan.  
 These new data are appended to the existing .fits file above. Here XXX is the
 backend.  This package has been tested only with the 'acs' spectrometer.
 Note that 'fname' is the file you invoked with sdfits-test.  It is the input SDFITS
 file to the GBT_IDL package.

2. When running the package, when new SDFITS data are written to 
   /users/you/idl/data/fname.avg.XXX.fits files (indicated on the Xterm running sdfits-test)
   you can update the online.gbt data file with the command:

   UPDATE

It is as simple as that. 

Note that if you exit and rentler the GBT_IDL package and have set the input SDFITS
file to the ../fname.  above, then the program loads all the data currently in the file.
As new data comes in you must UPDATE, but it loads everything that is there, always.