12-Apr-2007

Improvements to Spectroscopy Modules

I'm uploading new IDL modules into the atest directory for RHESSI.
These support some critical improvements to the datagap detection
and livetime integration algorithms.

1. Datagap Detetection and Post-Gap Deletion

Probably the most important changes have been made in the datagap
routines. Since mid-2005 some detectors have a spectral pedestal
extending up to ~30 keV. This was found to be due to a baseline
distortion at the tail of the datagaps for a subset of detectors. The
current software attempts to remove those pulses which are only found
in the 10 milliseconds after the end of a gap.  The datagap control
parameters have been expanded to allow for detector differences and
fine tuning of the minimum gap lengths. Users should not attempt
to manipulate these values, they should be fine-tuned only by David
Smith and Richard Schwartz.

Here are the new cutoff controls:

dp_cutoff_max: 0.0, $ all dropout cutoffs shorter than this in seconds
dp_cutoff_min: 0.0, $  all dropout cutoffs longer than this in seconds
dp_cutoff_coeff: 0.0, $ ;coefficient for function, normalization of 
	dropout curve
dp_cutoff_xp: 0.0, $ ;exponent

The dp_cutoff_xxx parameters are used with hsi_f_dp_cutoff()

dp_enable:0b , $ ;start with dropout disable
dp_extend_def:fltarr(9), $ ;extend datagap by this (seconds) regardless 
of date
dp_extend_sec:fltarr(9), $ ;extend datagap by this (seconds), allow for 
structure
;      for each front segment including time window and extension in 
seconds
dp_extend_utlim: dblarr(9), $

The default values can be found in hsi_eventlist_strategy_control()

In the future the post-gap pulse removal software will be enhanced
to look for even shorter gaps that exhibit the post-gap baseline
distortion as some of these still come through the current filter.

1a. Fewer livetime glitches

There is also an improvement to the continuity of the livetime
arrays because the transition between bunches of packets has had
some reworking.  This fixes some glitches that used to occur because
of the carryover of a datagap between bunches.



2. Semicalibrated Mode Restored 

When we made the change to the new control parameter defaults in
March 2007, we rediscovered another longstanding bug.  When the
full matrix option is selected it was found that the semicalibrated
option was affected since the two conditions are controlled by the
same parameters.  Because of the way most users worked through an
event it had not been problematic.  But with the full matrix enabled
at the beginning, any displays of the semicalibrated spectrum would
have been wrong.  This has been fixed by instantiating a second
hsi_srm object as a property of the first hsi_srm object but where
that object is devoted exclusively to the matrix diagonal response
need for the semicalibrated mode but it takes its other control
parameters from the containing object.

3. Averaging Correctly with Subset of Detectors Turned Off

There is also a fix for the spectrum when one or more of the N
selected detectors are off.  Previously we had summed the rates and
divided by the number of detectors.  When all the detectors are on
this method was fine, but it creates obvious discontinuities when one
or more detectors is off during the set.  Operationally, detector
8 is commanded off during certain transmission configurations and
those problems could be seen in the lightcurves. Now we will instead
divide by the number of live detectors in every time interval to
correct this.

4. LTIME field of the Data Structure from HSI_SPECTRUM

In the spectrum object when the SUM_FLAG is set we have changed
a field, LTIME, of the structure output, (/SP_DATA_STR) and in
the corresponding LIVETIME column of the exported FITS file for
use with OSPEX. This value now the average of the livetimes over
detectors instead of just the value of the livetime from the first
detector chosen.  This change in the reported livetime does not
change the critical data product which is the rate or the flux.
In OSPEX,  The livetime value is only used as a weighting factor
for integration and to construct error estimates in fitting; our
studies conclude this has only a marginal effect.  We don't expect
any material changes to meaningful scientific results.

Here's a list of the new and revised procedures:

hsi_dp_cutoff.pro
hsi_dp_cutoff__define.pro
hsi_eventlist_strategy_info__define.pro
hsi_eventlist_strategy_control__define.pr
hsi_f_dp_cutoff.pro
hsi_dropoutlist_noreset.pro
hsi_dropoutlist.pro
hsi_dropoutlist_remove.pro
hsi_spectrogram__find_detector_off.pro
hsi_srm_info__define.pro
hsi_srm__define.pro
hsi_eventlist_packet__define.pro
hsi_eventlist_strategy_control.pro
hsi_packet2eventlist.pro
hsi_spectrogram__define.pro
hsi_spectrum__define.pro
hsi_spectrum__spectra4spex.pro

Richard Schwartz