Using the Observing Summary and Quicklook Data

Overview
Using the HSI_OBS_SUMMARY Object
Observing Summary Count Rate Data
Retrieving Data
Retrieving Corrected Count Rates
Retrieving Times
INFO Parameters
Plotting with the PLOT or PLOTMAN methods
Plotting Manually
Extracting Individual Observing Summary Objects
Using the Flare List
Using the LIST method
Extracting Flag Information

Overview

The RHESSI observing summary and quicklook data are comprised of the Level-1 data products listed in Table 1 below.  As part of the RHESSI pipeline processing, these data were read from the Level-0 telemetry files, pre-binned by energy and time, and stored in daily files for quick access.  Most of the data types are stored in the daily hsi_obssumm_yyyymmdd_vvv.fits file (each data type in a separate extension), with the exception of the hsi_full_rate data, which is stored in a separate daily hsi_fullrate_yyyymmdd_vvv.fits file.  Both the obssumm and fullrate daily files are stored in $HSI_DATA_ARCHIVE/metadata/catalog. 

Most of these data types can be viewed through the Observing Summary widget in the RHESSI GUI (hessi).  This document deals with accessing them through the IDL command line interface.

Table 1.  A List of the Observing Summary and Quicklook Data Products

The hsi_obs_summary object provides access to all of the data types in the observing summary (via the keyword class_name).   The purpose of the hsi_obs_summary object is to provide a uniform interface to each individual object with access methods that are identical to the access methods in the other RHESSI objects (imaging, spectra, etc.), and to provide a container object for organization.  You can create any of the individual observing summary objects (e.g. hsi_obs_summ_rate) directly and use them to retrieve or plot data (described here), however some of the data access methods are not standard.  Click here for a complete description of the contents of each observing summary data type.

There are only three control parameters for the hsi_obs_summary object, shown in Table 2.  Usually the only parameter you need to set is the obs_time_interval; the appropriate file should be found automatically.

Table 2. HSI_OBS_SUMMARY Object Control Parameters

Table 3 lists the methods that are available in the hsi_obs_summary object, and the classes they currently apply to.

Table 3.  HSI_OBS_SUMMARY Object Methods

Using the HSI_OBS_SUMMARY Object

To create the object and select a time interval, type:

obj = hsi_obs_summary()
obj -> set, obs_time_interval=['29-mar-02 20:25', '29-mar-02 20:35']

or in one line:

obj = hsi_obs_summary( obs_time_interval=['29-mar-02 20:25', '29-mar-02 20:35'] )

or alternatively you can set a filename to read and the directory the file is in using the filename and datadir parameters.  You must set obs_time_interval or filename (and optionally datadir), but not both.  If you set the obs_time_interval, the software automatically finds the file containing that time interval (if you're not connected to or don't have a copy of the full archive, set search_network,/enable to enable copying the needed files to your computer automatically).

Observing Summary Count Rate Data

The observing summary count rate data are available in two forms, the hsi_obs_summ_rate class and the hsi_full_rate class.  In both, the rates are pre-binned to nine fixed energy bands and are provided at 4s time resolution.  The energy_edges field in the info structure contains the energy bins that were used.  In both cases, if you use the /corrected keyword on the call to getdata, plot, or plotman, the 'corrected' count rates are used.  The 'corrections' are approximate and attempt to remove the effects of attenuator and decimation state changes (NOT dead time or  pulse pileup effects). The difference between the hsi_obs_summ_rate class and the hsi_full_rate class is in how they handle the detectors.

In the hsi_obs_summ_rate class the rates are summed over front and rear pre-selected detectors.  The detectors used change with time and are chosen by the RHESSI science team based on the quality of their data.  The seg_index_mask field  in the info structure indicates which detector segment were used. For the 29-mar-02  times above, for example (note we don't have to specify the class because hsi_obs_summ is the default):

d1 = obj -> getdata()   ; hsi_obs_summ_rate is the default so don't have to specify class
help, d1.countrate
<Expression> FLOAT = Array[9, 150]  ; 9 energy bins, 150 time intervals
i1 = obj->get(/info)
print, i1.seg_index_mask
1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 0 1  ; means front and rear segments of detectors 1,3,4,5,6,9 were used
print, i1.energy_edges
3.00000 6.00000 12.0000 25.0000 50.0000 100.000 300.000 800.000 7000.00 20000.0

In the hsi_full_rate class, the 18 detector segments are returned separately.  For example,

d2 = obj->getdata(class='hsi_full_rate')
help,d2.countrate
<Expression> FLOAT = Array[9, 18, 150]  ; 9 energy bins, 18 detector segments, 150 time bins
i2 = obj->get(/info, class='hsi_full_rate')
print, i2.energy_edges
3.00000 6.00000 12.0000 25.0000 50.0000 100.000 300.000 800.000 7000.00 20000.0

Note that when using the plot or plotman method with these two classes, the units of the data plotted are counts s^-1 per detector segment.

Retrieving Data

The hsi_obs_summary object provides access to up to 20 individual obs summary object classes (those listed in Table 1 above).  To retrieve data from the different individual objects, use the class_name keyword.  If  the class_name keyword is not specified, then the default data type is the Observing Summary Rates (the internal hsi_obs_summ_rate object).

Hence, to retrieve data from the hsi_obs_summ_rate object, type:

rates_data = obj -> getdata()

whereas, for example, to retrieve data from the modulation variance data (from internal object hsi_mod_variance), type:

mod_data = obj -> getdata(class_name='hsi_mod_variance')

In fact, you can use any shorthand for the class_name that is unique, such as 'ephem' for 'hsi_ephemeris', or 'pointing' for 'hsi_qlook_pointing'.

The data returned by the getdata method is always a structure.  The contents of the structure returned for each class are described here.

Retrieving Corrected Count Rates

Use the corrected_countrate keyword to retrieve the obs_summ_rate or full_rate  corrected count rates:

corr_rates_data = obj -> getdata(/corrected)
corr_rates_sep_det = obj -> getdata(/corrected, class_name='full_rate')

The first example doesn't specify class_name, so the default data type (hsi_obs_summ_rate count rates) is returned.

Retrieving Times

Use getaxis(/ut) or getdata(/time_array) or getdata(/xaxis) to retrieve the time array.  Include the class_name keyword for data types other than the default.

For the two examples above, the time array can be retrieved by the following:

times_rate = obj -> getaxis(/ut)   ; no class implies hsi_obs_summ_rate
times_mod_variance = obj -> getaxis(/ut, class='mod_variance')
or
times_rate = obj -> getdata(/time_array)   ; no class implies hsi_obs_summ_rate
times_mod_variance = obj -> getdata(/time_array, class='mod_var')

INFO Parameters

There are no info parameters for the hsi_obs_summary object itself.  When you retrieve info parameters, they are specific to the class_name you specify (or to the default hsi_obs_summ_rate object if no class_name is specified).   So in the following examples, the first line lists the structure of info parameters for the hsi_obs_summ_rate object, and the second example does the same for the hsi_obs_summ_flag object.

help, obj -> get(/info), /structure   ; no class implies hsi_obs_summ_rate
help, obj -> get(/info, class_name='flag'), /structure

The info parameters specific to each object are described here.  Info parameter can be retrieved only after data has been retrieved for the object through one of getdata, plot, getaxis, etc.

Plotting with the PLOT or PLOTMAN methods

The observing summary classes that have a plot or plotman method are shown in Table 2.  The plot method sends the plot to a static IDL plot window.  The plotman method sends the plot to an interactive Plot Manager interface that lets you zoom, print, create plot  file, etc.  If you haven't explicitly already retrieved the data for your selected time interval, the plot and plotman methods do it automatically before plotting.  The primary keywords that can be used on the plot or plotman command are listed in Table 4.  In addition to these keywords, most standard plot keywords can be used.  To send multiple plots to the same plotman session, use the plotman_obj keyword.

Table 4.  Plot Keywords

The plotman method automatically defines an appropriate color table and plots in color.  The plot method plots in black and white with different line styles unless you set up a color table, and specify the dim1_colors keyword.

Examples:

• To plot the observing summary rates summed over energy, showing the flare, night, and SAA flags, the command is:

obj -> plot, /saa, /flare, /night, /dim1_sum

• To plot corrected count rates for front detectors 7,8,9 in plotman:

obj -> plotman, /corrected, class='full_rate', det=[6,7,8], plotman_obj=pobj

• To plot the ephemeris data with colors the commands are:

hsi_linecolors
obj -> plot, class='ephem', dim1_colors=indgen(3)+1

• To plot any of the observing summary flags, use class_name ='flag' and use the flag_name keyword to select which flag to show.  The allowed values for flag_name are listed here.   For example:

obj -> plot, class='flag', flag_name='attenuator_state'

• To plot corrected count rates from the hsi_obs_summ_rate class in plotman (in the same plotman session as above):

obj -> plotman, /corrected, plotman_obj=pobj

• To plot (in plotman) only the first 3 energy bands of the observing summary rates, and place the legend outside and to the right of the plot:

obj -> plotman, dim1_use=[0,1,2], legend_loc=6

If you use the plotman interface, some of the items listed in Table 4 can be controlled interactively using buttons in the plotman interface.

Plotting Manually

Example to plot the 6-12 keV band of corrected count rates from the hsi_obs_summ_rate class:

obj =hsi_obs_summary(obs_time= ['23-jul-2002 00:16','23-jul-2002 01:15'])
data = obj -> getdata(/corrected)
time = obj -> getdata(/time)
utplot, time - time[0], data.countrate[1,*], time[0], /ylog
or
utplot, anytim(time, /ext), data.countrate[1,*],/ ylog 

(The first utplot example above uses time in seconds relative to a base time (base time provided as the third argument).  The second utplot example above uses fully qualified times by converting the time array to the external format.)

Example to plot data from an individual detector segment (front detector 5 in this case) in the 12-25 keV band from  the hsi_full_rate class:

obj =hsi_obs_summary(obs_time= ['23-jul-2002 00:16','23-jul-2002 01:15'])
data = obj -> getdata(/corrected, class='full_rate')
time = obj -> getdata(/time, class='full_rate')
utplot, anytim(time, /ext), data.countrate[2,4,*], /ylog  ; energy bin 2, det/seg #4 is front detector 5

or plot data from three detector segments (front 7,8,9 here) in the 12-25 keV band from the hsi_full_rate class:

dets = [6,7,8]  ; means front detectors 7,8,9
d789 = total(data.countrate[2, dets , *], 2)   ; energy bin 2, sum over detectors
utplot, anytim(time, /ext), d789 / 3., /ylog   ; divide by 3 to get rate per detector

or plot data from three detector segments (front 7,8,9)  for 6 - 25 keV (combine 6-12 and 12-25 keV bands):

dets = [6,7,8]  ; means front detectors 7,8,9
d789 = total(data.countrate[1:2, dets , *] , 2)  ; energy bins 1 and 2, sum over detectors
utplot, anytim(time, /ext), total(d789, 1)/3., /ylog   ; sum over energy dimension, divide by 3 to get rate per detector

Example to plot the x coordinate of the spacecraft pointing data:

obj =hsi_obs_summary(obs_time= ['23-jul-2002 00:16','23-jul-2002 01:15'])
data = obj->getdata(class='hsi_qlook_pointing')
time = obj->getdata(/time,class='hsi_qlook_pointing')
utplot, anytim(time,/ext), data.pointing[0]

Extracting Individual Observing Summary Objects

To extract any of the individual observing summary objects out of the hsi_obs_summary object, use the get method with the object_reference keyword.  For example,

ephem_obj = obj -> get(/object_reference, class_name='ephem')

As mentioned above, each individual object can be used directly.

Using the Flare List

To extract the flare list from the hsi_obs_summary object, use the class_name keyword as above:

flare_list = obj -> getdata(class_name='flare_list')

This returns a structure containing start/end/peak times, peak count rates, etc. for each flare within the time range specified in the obs_time_interval control parameter.   For a description of all of the parameters in the flare list, click here.

The flare list is stored in separate monthly files (hessi_flare_list_yyyymm.fits) as well as in an extension of the daily observing summary files.  However, the version in the observing summary file may not be current - when the flare list is reprocessed, the separate monthly flare files are updated, but not the extension in the obs summ files.  Therefore we recommend not using the obs summ file for the flare list (hence the strike-through above).

Instead, use the flare list files directly through hsi_read_flarelist and other utilities that help to access the flare list easily, perform selections, and to format the flare parameters into strings.

Using the LIST method

Some of the individual objects have a list method (see Table 3).  The list method formats the data returned by the object into an ASCII table, and displays the results on the screen, saves the results in a file, or prints the results, as well as returning the string array as the function result.  Use the show_text, file_text, or print_text keywords respectively to specify these options.  The default is show_text.  An example that sends the hsi_ephemeris data to a file is:

text = obj -> list (/file_text, class='ephem')

Extracting Flag Information

As with the other types of summary data, the flag information can be extracted by the following:

flags = obj -> getdata(class='flag')
times = obj -> getaxis(/ut, class='flag')
info = obj -> get(/info, class='flag')

flags contains an array giving the state of all flags for every time in the times array, and info.flag_ids is a string array giving the names of the flags.

To determine when changes in state of any flag occurred, use the changes method:

flag_changes = obj -> changes()

flag_changes is a structure with a tag for every flag name.  Each flag name tag is itself a structure containing an array of start / end times and the state of the flag during those times.  Thus, flag_changes.eclipse_flag.start_times are the start times for when the eclipse flag was in the state given by flag_changes.eclipse_flag.state.

Last updated 08 December 2014 by Kim Tolbert , 301-286-3965