How are the Level-0 FITS files named?
How do I access the Level-0 FITS files?
How does the software determine the name of the Level-0 file to read?
Is there an easy way I can tell what file the software is looking for?
Where does the software look for data files.
What is hessi_data_paths?
What should HSI_DATA_ARCHIVE be set to?
What is HSI_DATA_USER for?
What should HSI_DATA_USER be set to?
How do I enable automatically copying the files I need?
Can I select which archive to automatically copy from?
Can I retrieve all data files for a specific time interval?
How do I set the HSI_DATA_ARCHIVE and HSI_DATA_USER environment variables?
I copied some Level-0 FITS files last month, but now the software can't find them.
I copied some new Level-0 FITS files, but the software can't find them.
Why do I get the IDL dialog box asking me to browse to my data file?
Why isn't the software finding the data files?
How can I mirror the Observing Summary FITS files to my computer?
How are the Level-0 FITS files named?
The form for the Level-0 FITS file names is:
hsi_yyyymmdd_hhmmss_vvv.fits where
yyyy = year
mm = month (01 through 12)
dd = day of month
hhmmss = hour, minute, second
vvv = version number of the fileNote: the time in the name is the time of the beginning of the first packet in the file.
How do I access the Level-0 FITS files?
If you are on a network that has access to the entire RHESSI data archive (GSFC, HEDC, SSL, etc.) then you should NFS-mount (or Samba share or equivalent) the archive and point to it with the HSI_DATA_ARCHIVE environment variable.
If not, you can copy files to your computer manually, or you can enable the software to automatically copy the files over the network when it needs them.
Refer to 'Accessing the RHESSI Data' for complete instructions.
How does the software determine the name of the Level-0 file to read?
There is a file called hsi_filedb_yyyymm.fits (where yyyymm is the year and month of the times covered by the file) that is distributed with the SSW software (in $SSW/hessi/dbase). This file contains a cross-reference between time intervals and file names. An ASCII version of this file - hsi_filedb_yyyymm.txt - is available in the same directory for easy viewing. The software uses the hsi_filedb_filename function to convert a time interval to a file name.
Is there an easy way I can tell what file the software is looking for?
There are two ways:
Use the function hsi_filedb_filename. Pass the time interval of interest as an argument to the function and it returns the file name(s) containing that time interval. For example:
print, hsi_filedb_filename (['29-mar-2002 12:00', '29-mar-2002 14:00'])
View the $SSW/hessi/dbase/hsi_filedb_yyyymm.txt file.
Where does the software look for data files.
Once the software has determined the name of the file to read, it calls a routine called hsi_find_file to look in the following locations in the order listed:
HSI_DATA_ARCHIVE/yyyy/mm/dd
Your current directory
All directories under HSI_DATA_ARCHIVE
All directories under HSI_DATA_USER
Across network (if enabled)
The search path for Items 2-4 are created in the routine hessi_data_paths.
Exception: If your HSI_DATA_ARCHIVE points to the full RHESSI data archive and your HSI_ARCHIVE_MOUNTED is set to true, then only the software only looks in the file's expected location: HSI_DATA_ARCHIVE/yyyy/mm/dd.
What is hessi_data_paths?
Hessi_data_paths is an IDL routine that sets up a search path for the RHESSI software to search for files if they are not found otherwise.
You can display this search path by typing
prstr, hessi_data_paths()
After making a change in your setup (e.g. restoring network connection to data, redefining an environment variable dynamically) type
a = hessi_data_paths (/reset)
to force IDL to reset its list of RHESSI data locations.
On September 6, 2005, hessi_data_paths was changed to only include the following:
the current directory
HSI_DATA_ARCHIVE and subdirectories (if it's visible and HSI_ARCHIVE_MOUNTED isn't true)
HSI_DATA_USER and subdirectoriesBefore September 6, 2005, hessi_data_paths included the following:
the current directory followed by 'HSI_DATA_ARCHIVE', 'HSI_DATA_USER', 'SSW_HESSI', 'SSWDB_HESSI', 'HESSI_DBASE', 'HSI_USER_DATA', 'SPEX_USER_DATA', 'HSI_DOC' as well as all environment variables in $SSW/hessi/setup/setup.hessi_env
What should HSI_DATA_ARCHIVE be set to?
HSI_DATA_ARCHIVE is an environment variable that should point to the top-level directory under which the Level-0 data files are located. For example on Windows, HSI_DATA_ARCHIVE might be set to c:\hessidata and there might be Level-0 data files in that directory or they may be in subdirectories under that directory.
Note: Directory names may NOT contain spaces.
If your computer has direct access to the full RHESSI data archive, then HSI_DATA_ARCHIVE should point to it. (see 'Accessing the RHESSI Data' )
What is HSI_DATA_USER for?
HSI_DATA_USER is an environment variable that should point to the directory that Level-0 FITS files will be copied to when you have enabled the feature to automatically copy files across the network to your computer.
Or if you're copying files manually to your computer, copy them to the HSI_DATA_USER directory.
When the software is looking for a data file, this directory is part of the search path, so once a file is copied to this directory, the software should find it automatically.
What should HSI_DATA_USER be set to?
HSI_DATA_USER can be set to any directory local to your computer that you have permission to write in.
Note: Directory names may NOT contain spaces.
How do I enable automatically copying the files I need?
In your SSW IDL session, type
search_network, /enable
If a file can't be found on your computer, then it will automatically be ftp'd from either Goddard (east coast U.S.), SSL (west coast U.S.), or HEDC (Windisch, Switzerland) (depending on which is closest). (See next question to choose the server.) The files will be written in the directory you have specified in the HSI_DATA_USER environment variable.
Note: This is not advisable with a slow network connection. Most of the data files are ~100MB in size.
Note that this is in effect only for the current IDL session. To make it permanent, put that line in your idl startup file. To disable the automatic copy feature, type
search_network, /disable
Can I select which archive to automatically copy from?
Normally the search_network automatic copying feature will find the closest RHESSI archive to copy files from. You can choose which server the automatic copy will use by entering the following command:
hsi_server, /gsfc, /verbose ; select Goddard, Greenbelt, Maryland, U.S.
hsi_server, /ssl, /verbose ; select SSL, Berkeley, California, U.S.
hsi_server, /hedc, /verbose ; select FHNW, Windisch, Switzerland
hsi_server, /unset ; revert to using closest server
Can I retrieve all data files for a specific time interval?
Use the RHESSI data search tool.
This browser interface lets you specify the time range you're interested in. If necessary, it will automatically extend the time range to include enough aspect data for your time range (only necessary when the roll database isn't complete for your time interval). It will find both Level-0 and Observing Summary files in your time range. A second window will pop up giving you a chance to select among the files found.
A zip file will be created containing the files you select and an email will be sent to you giving you the URL of the zip file. Unpack this zip file (on WIndows use a program like WinZip, on Unix use the command unzip filename) into either the HSI_DATA_ARCHIVE or the HSI_DATA_USER directory on your computer. The RHESSI software will automatically find the files in either of those directories.
How do I set the HSI_DATA_ARCHIVE and HSI_DATA_USER environment variables?
Click here for instructions on setting RHESSI environment variables.
I copied some Level-0 FITS files last month, but now the software can't find them.
It is important to remember that the file containing the cross reference between filenames and time intervals is updated every time you update SSW. If a newer version of the data file that you copied a month ago has been written, then the hsi_filedb_yyyymm.fits file will point to the newer file and your older file is useless. Similarly, if you copy the newest version of a Level-0 data file, but don't update SSW to get the most recent hsi_filedb_yyyymm.fits file, the software won't find the data file.
Note that the version number of a Level-0 FITS file is encoded in the file name.
I copied some new Level-0 FITS files, but the software can't find them.
See answer above.
Why do I get the IDL dialog box asking me to browse to my data file?
If you get the dialog widget that asks you to browse to your data file, it means your setup is incorrect. With a correct setup, you should never get that dialog box when the software is looking for a data file. Review the information on this page.
Why isn't the software finding the data files?
Review the information on this page describing how the RHESSI software locates data files. Here are some additional reminders:
Are your environment variables set up correctly (remember that Unix is case-sensitive)? (Type hessi_var.)
If HSI_DATA_ARCHIVE is across a network, is the network path to the data archive up and running?
If you manually copied data files to your computer, did you use binary mode when ftp’ing the files?
Do the file names of local copies of files exactly match the names returned by the hsi_filedb_filename routine?
Do any of your directory or sub-directory names contain spaces in the names? They should not.
Is your hsi_filedb_yyyymm.fits file current? (Check that your SSW update is working correctly.)
If you changed your setup files, remember to restart SSW IDL (Windows), or rerun the Unix setup script and restart IDL (Unix).
How can I mirror the Observing Summary FITS files to my computer?
These instructions are for Windows. You can do something similar on Unix.
1. Create a .bat file in your $SSW/site/setup directory called mirror_hessi_obs_summ.bat and put the following lines into that file:
C:
cd C:\ssw\gen\mirror
perl.exe C:\ssw\gen\mirror\mirror.pl -d C:\ssw\site\setup\mirror_hessi_obs_summ.pkg > C:\ssw\site\setup\mirror_hessi_obs_summ.log(Notes: The line starting with perl.exe may have wrapped in your display, but it should be entered as one line. Also if perl isn't in your path, you have to enter the full path to perl.exe. On my computer it is C:/Perl/bin/perl.exe)
2. Create a .pkg file in your $SSW/site/setup directory called mirror_hessi_obs_summ.pkg and put the following lines into that file:
package=hessi_obs_summary
comment=hessi_obs_summary
site=128.183.173.219
remote_user=anonymous
remote_password=kim.tolbert@gsfc.nasa.gov
remote_dir=hessidata/metadata/catalog/
local_dir=C:\hessidata\metadata\catalog
mode_copy=true
use_timelocal=false
max_delete_files=99%
max_delete_dirs=99%
update_log=C:\ssw\site\setup\mirror_hessi_obs_summ_full.log
compress_patt=
force_times=yes
3. Run the .bat file created in Step 1 whenever you want to update your database of Observing Summary Files.
The names and directory of the .bat and .pkg file mentioned above are arbitrary, as long as you are consistent. In this example, the Observing Summary Files will be written to C:\hessidata\metadata\catalog (the local_dir field in the .pkg file); you can choose whatever directory you want. This example also writes two log files - a cumulative log file called mirror_hessi_obs_summ_full.log, and a log file for each update called mirror_hessi_obs_summ.log in your $SSW/site setup directory. You can choose not to write either log file ( by leaving off the > C:\ssw\site\setup\mirror_hessi_obs_summ.log in the .bat file, and by removing the update_log line in the .pkg file).
For the RHESSI SSW software to find these files, you must set HSI_DATA_ARCHIVE or HSI_DATA_USER to C:\hessidata (or the top level of wherever you put the data).
Last updated
12 January 2011
by
Kim Tolbert
,
301-286-3965