3-Mar-2017

On Friday, March 3, we put some new RHESSI data accumulation software online in the hessi atest directory.  For the most part, the changes should be transparent to almost every user.

The modifications include improvements that have been in development for more than a year when we discovered that the livetime in extremely small time bins (<512 binary microseconds) was not being reported correctly.  Fine time bins of that order are generally only necessary for real spatial structure on the order of a few arcseconds.  If you were imaging sources far from the spin axis with Grid 1 this might have affected your results. 

Additionally, a problem with the detector electronics for data after Jan 2014 is dealt with in this release.  This problem, which occurred at the end of datagaps, was pushing a small fraction of events to energies up to 100-150 keV higher than their actual energy.  This effect was getting worse with time. If you found higher energy emission than expected, you should look again with this new software. 

Details:

1. Packet_bunches -  we cannot expect to accumulate data requests by analyzing a single set (bunch) of packets. The boundary management was always a bit unreliable.  Datagaps and livetime procedures mean that you always have needed to look forward to complete any accumulation.  It's managed much more cleanly using a 2 second overlap for every bunch as we've been using a 2 second extension for many years.  Working with that has made the packet-eventlist software much cleaner. Also, there is a one-to-one relationship between the eventlist elements and the data longwords in the packets until just before the eventlist is returned to eventlist_strategy::process from the return in ::process_hook. This enables clearly knowing the originating packet of any event which greatly simplifies dealing with the time overlaps.


2. Livetime - There are a number of issues.  Livetime isn't reported in the datagaps. In the original telemetry schema that would have meant nearly 100% livetime. The maximum time resolution for livetime reporting is 512 binary microsec (busec).  High resolution imaging may require 256 busec or finer time bins. Livetime is only reported in 3 events following an interval or not at all so there has always been the issue of interpolating/integrating the livetime at both short and long time intervals (128 busec to thousands of seconds).
   
   The old software mapped the livetime poorly for time bins lt 512 busec.  The new software is robust at all timescales although there is still potential for modest improvement at sub-512 busec. Finally, the datagaps have to be integrated with the reported livetime. In the new software, the datagaps force the livetime during the gaps to be zero and it's a value that tells the follow-on software that it is a gap value of zero, different from a livetimecounter value of 0 which would mean 100% livetime.


3. Datagaps - The CSA reset method used in the first few years was unreliable and so now datagaps are always detected by the absence of events.  This new software doesn't support the old CSA event method at all.  The gap detection size limit varies with the rate. There are more possibilities for extending and prepending added time on the gap intervals.
   
   Over the last several years we have started to see anomalous higher energy events at the end of datagaps while the baseline was restoring to normal levels over 10s of milliseconds.  Because of their appearance in the time series we called this phenomenon "ramps." Now, the post-gap ramps are detected and removed from the spectrogram data (they're actually still in the eventlist as negative numbers and not used). Also, event detection for datagaps goes to channels down to 0 keV by default. Although there are no datagaps (maybe) in the rear segments this code is used to detect detector off intervals as well (SAA for example) and thus to control the livetime integration over large time intervals better.

New (and existing) control parameters for datagaps:

dp_extend_sec - fltarr(18), seconds to extend datagap for times after dp_extend_utlim for each detector segment

dp_extend_def - fltarr(18), seconds to extend datagap for times before dp_extend_utlim for each segment

dp_extend_utlim - dblarr(18), time boundaries for use of dp_extend_sec or dp_extend_def for each segment

dp_append_def - fltarr(18), same as dp_extend_def. Only effective if  > dp_extend_def (or ...sec depending on time)

dp_prepend_def - fltarr(18), seconds to prepend to datagap

dp_append_nvalid - fltarr(18), number of additional valid events after a gap to extend gap by

dp_prepend_nvalid - fltarr(18), number of additional valid events before a gap to extend gap by

Richard and Kim