RHESSI Data Analysis Software Change History


Last updated 12 October 2018 by Kim Tolbert

Notes:   
Some of the dates are links which will give you more information about the change.
Only the more significant changes are documented here.  Minor changes are ongoing.


12-Oct-2018 Small Error in Eventlist Accumulation Fixed

We include a 2-second extension at the beginning and end of the eventlist accumulation in order to allow the gap-finding s/w to work correctly. In March 2017, we also implemented a 2-second overlap between packet bunches to manage the boundaries between the bunches.  This change in March 2017 inadvertently disabled the 2-second extension at the end of the accumulation.  This is now fixed.  The result was that a few counts were lost in the last few milliseconds of the requested interval. The effect was to report slightly lower counts for an interval for both images and spectra, with very little scientific consequence.
24-Jul-2018 Total Counts are Now Displayed on Visibility-Based Algorithm Images

Previously didn't have total counts for vis-based algs.  Now new parameter, xx_n_event_used (where xx is the algorithm prefix, e.g. nj for mem_njit) saves number of counts in image for each detector (retrieved from count tag in vis structure.)
23-Jul-2018 Added Signal-to-Noise Ratio (SNR) Requirement to Make Images

SNR is computed from visibilities as follows:  Use coarse collimator visibilities.  If there are enough, compute mean of SNR (obsvis/sigamp) for those vis (after rejecting outliers).

Added 3 new control parameters:
snr_chk -      (Default is 2)
                     0 - don't check SNR
                     1 - try to check SNR. If can't (no vis, or not enough vis), DON'T make image.  If can check, make image if snr < snr_thresh
                     2 - try to check SNR. If can't (no vis, or not enough vis), MAKE image anyway. If can check, make image if snr < snr_thresh
snr_thresh -  Threshold of SNR to make image
snr_vis_file - For non-vis algorithms, can supply this file (must be on same time/energy bins as images) to read visibilities from

Added 2 new info parameters:
snr_value - The value of the SNR for each image
snr_msg -   The string error message if no image was made
8-June-2018 ALL GUIs and PLOTMAN

There was a problem viewing the GUI and PLOTMAN widget interfaces on high-resolution screens (the font was so small it was unreadable) on Windows platforms.  This has been fixed.  If anyone continues to have the problem, please let me know (kim.tolbert@nasa.gov).
6-Apr-2018 IMAGING Clean Option Change

The clean_regress_combine control parameter has been modified to have five choices (previously it was an on/off switch).  The options now are
  • disable - no regression, residual map added to component map or not, according to media mode (this is the default)
  • full_resid  - Regression of the component map expected counts against the observed. Scale the component map by the regression coefficient and add the unscaled residual map
  • scaled_resid - First do the steps for 'full_resid' but then take the counts minus the scaled expected counts and regress that against the expected counts from the residual map. Form the new map from the scaled component map added to the newly scaled residual map
  • old_scaled_resid - Method prior to April 2013. Regression on the expected counts from the component and residual maps.
  • no_resid - Perform the regression of 'full_resid', scale the component map by the regression coefficient and do not add any of the residual map, a media mode for regression.

These options are implemented in the routine hsi_clean_regress_combine.pro if you want to take a closer look.
6-Apr-2018 IMAGING Back Projection Change

The internal object class used to produce the back projection image is now called BPROJ_IMAGE.  Prior to this change, the BPROJ class had the dual role of producing and storing the back projection image, as well as internally computing back projection products for the other non-visibility algorithms.  Now the existing BPROJ class is used only for the internal functions, and BPROJ_IMAGE handles only the back projection image.

This change should be transparent to most users. You may be affected if you work with the internal objects directly instead of through the hsi_image object.
20-Mar-2018 IMAGE object now has option to work with modulation patterns in Cartesian space instead of ANNSEC space.

Use control parameter mpat_coord to choose between 'annsec' (the default for now) or 'cart'.  Using cartesian coordinates is more direct and simplifies the internal calculations used in making RHESSI images.
18-Mar-2018 IMAGE and SPECTRUM Object Parameter Defaults - Added options for user defaults.  Please see Item 6 in RHESSI OBJECT PARAMETER DEFAULTS for details.
29-Nov-2017
 IMAGING - Calibrated Eventlist Normalization Changes

(Note, calibrated eventlist normalization was first implemented in March 2017, but it was done incorrectly. Now it is corrected, and user has access to the correction factors.)
1-Nov-2017 IMAGING - Added VIS_WV image reconstruction algorithm
24-Oct-2017 IMAGING - Added VIS_CS image reconstruction algorithm
3-Oct-2017 VIS_FWDFIT Parameters Used to Construct Spectrum in OSPEX

Normally in OSPEX imaging spectroscopy, you define regions in your images, and construct spectra by summing the flux in the pixels in those regions.  You can now use the flux and flux error from the VIS FWDFIT fitted sources to construct the spectrum.  In the GUI, this is enabled in the Select Input widget, by clicking Select Regions / Use image algorithm parameters, and disabled by clicking Select Regions / Region Selection Widget.  At the command line, you can enable or disalbe this option via
o->set, /spex_image_use_alg_parms      or     o->set, spex_image_use_alg_parms=0
14-Sep-2017 VIS_FWDFIT Image Algorithm Changes
  • Can now fit multiple sources (up to 3 in the GUI), e.g. 2 circles or ellipses plus a loop
  • Can now choose which source parameters to fit
  • Can enable albedo correction, and choose which sources to apply it to
  • The vis_fwdfit options widget is now non-blocking; you can configure sources and make images without closing the widget
  • VF_SRCIN (the control parameter containing structure(s) defining input sources to vis_fwdfit) are now included in the image script generated by the GUI
7-Jun-2017 VISIBILITY CHANGES

The visibilities are now accumulated and stored for all time and energy bins at once.  (Originally, this is how it was done, but in 2012, it was changed to accumulate and save all energies for a single time bin, and lose those when moving to the next time bin.)  So now the entire visibility bag is available in the object, and if you are  using a visibility-based algorithm and want to reprocess your images, the s/w does not have to re-read the original raw files.
7-Jun-2017 IMAGE GUI CHANGES
  • Added buttons to send modulation profile plots to screen or PS file while generating image(s)
  • Added buttons to send visibility comparison plots to screen or PS file while generating image(s)
  • Added item to 'Display ->' pull-down menu to plot visibility comparison plot for most recent image made
  • Added button called 'Force Redo' to force image object to reprocess
  • The 'Write FITS File' button is now a pull-down for writing an image(cube) FITS file or a visibility FITS file.
7-Jun-2017 VISIBILITY COMPARISON PLOTS - new plots are available to show comparison between observed visibilities and those computed from image.

These plots are similar to the plot generated by the VIS_FWDFIT algorithm, but are now available for any algorithm (both visibility-based and not). In the GUI there are buttons to send the plot to the screen or a PS file while generating the image(s), as well as a button to show the plot for the most recent image on the screen. From the command line, you can call hsi_image_vis_plot directly, e.g. hsi_image_vis_plot, imobj=imobj, tb_index=2, eb_index=4
Please see Visibility Comparison Plots for more details.
2-Jun-2017 MEM_NJIT - Fixed half pixel offset problem.

Images made with the MEM_NJIT  algorithm with an even number of pixels were not correctly centered on the xyoffset value specified. They were off by a half a pixel.
3-Mar-2017 DATA ACCUMULATION - Fixed issues in three areas of data accumulation.

Electronic errors in the data since ~Jan 2014 moved events to energies 100-150 keV higher than their actual energy. These events are identified and removed.
Livetime accumulations at very small time bins (<512 busec)  were not handled correctly.
Accumulations across packet bunch boundaries were problematic.
15-May-2014 PACKET READER - unpacking bug related to bad timestamp fixed

Fixed a bug in the time unpacking code.  This bug was active when bad timestamps were detected in the science event packets.  When there are irresolvable timestamps some data is discarded in packets with the bad timestamps and that worked correctly.  The problem came in assigning times to all of the data in the group of 100 packets (up to 27,000 events) which contained the faulty timestamps.  It produced the effect seen in this figure with the gap and the spike.  Typically, about 4 seconds of data would show a dropout followed by a 4 second increase to twice the normal intensity. In the figure, the true rate is nearly constant. A new version of hsi_app100_unpack_time.pro that fixes this bug was uploaded to atest.
26-Sep-2013 IMAGING - Added EM image reconstruction algorithm

Find more details about the Expectation Maximization (EM) algorithm on the RHESSI web site under Software / Imaging Software / EM.
8-Apr-2013 IMAGING - Changes to CLEAN image algorithm software. 

Made improvements to CLEAN_REGRESS_COMBINE option (use old method by setting CLEAN_OLD_REGRESS_METHOD=1), and minimized reprocessing for the following parameters: CLEAN_BEAM_WIDTH_FACTOR, CLEAN_REGRESS_COMBINE, CLEAN_SHOW_N_MAPS, CLEAN_SHOW_N_CHI, CLEAN_PROGRESS_BAR.
9-Nov-2012 IMAGING - You can now make regularized electron and photon visibilities and images from the hessi GUI as well as from the command line using the RHESSI objects.

The visibility type is controlled by a new parameter, vis_type, which can be set to 'photon', 'regularized electron', and 'regularized photon'.  Previously the only type of visibility available was 'photon'.

Also, the software no longer automatically reads and writes visibility files when the vis_input_fits or  vis_out_filename parameter is set to a non-blank string; now you must call the visibility object READ or WRITE methods explicitly (in the CLI, not necessary in the GUI).

See Using Visibility Objects in the RHESSI Software for more details.
31-May-2012 SPECTRUM - A new control parameter, INSERT_KEDGE, was added to support better precision in the detector response around the instrumental K-edges for germanium, molybdenum (Grid 1 only), and tungsten.  Default is 0 (no effect) until after the June 2012 SPD meeting.
16-Mar-2012 IMAGING - For over-the-limb flares, two new parameters are available to ensure that the grids are precisely aligned to a REFERENCE_POSITION_ANGLE (in radians) for one roll bin sequence in the stacked eventlist and corresponding visibility.  Enable this feature by setting USE_REFERENCE_POSITION_ANGLE.
1-Nov-2011 IMAGING - In the hessi image widget, you can now set the image center graphically by clicking the 'Click Image to Set Map Center' button.
23-Sep-2011 IMAGING - In hessi image widget, more options were added to the Display-> Aspect Solution button to plot p_error, triangles, and distance from Sun center to image axis.  From the command line, these plots can produced via
asp_obj -> plot, /plot_dist   or   /plot_triangle   or   /plot_p_error
18-Apr-2011 OBSERVING SUMMARY - In the GUI, you now have the option to select which detectors to plot the quicklook count rate data for. Previously used only the obs_summ files, which have a fixed set of front and rear detectors already combined.  Now also use the full_rate files, which have data for the 18 separate detector segments.
1-Apr-2011 IMAGING - By setting CBE_MULTI_ATTEN_THRESHOLD, you can now make images in time intervals that span a change in attenuator state.
1-Apr-2011 ASPECT SOLUTION - The error term of the aspect solution is now checked and bad data points are not used.
28-Mar-2011 ASPECT SOLUTION - Improved quality of aspect solution where the pointing is just outside the nominal field of view.  Previously rejected all 3-limb solutions if some were isolated (which happened frequently). Now uses sets of contiguous 3-limb solutions.
1-Mar-2011 IMAGING - Removed a number of dangling pointer references, fixing the source of a number of memory leaks.
7-Feb-2011 BROWSER - SDO AIA Flare Cutout link added for some RHESSI flares.  Place red time arrow on a flare; if AIA cutouts are available for that flare, the 'AIA Flare Cutout' on the left side of Browser will become active.
18-Jan-2011 SPECTRUM - Added a button on spectrum widget to write separate detector FITS files with native energy bins (can be done from the command line using the hsi_spectrum_sep_det_files routine.
21-Jan-2011 Widgets in HESSI GUI, PLOTMAN, OSPEX, etc. should now work on small displays.  A vertical scroll bar is added if the widget would otherwise be too big for your screen. (please email kim.tolbert@nasa.gov if I've missed any big widgets)
29-Sep-2010 SIMULATIONS - A bug in the simulation software (in hsi_sim_dummy_livetime.pro) that caused the last 5% of
any simulation to have 0 livetime was fixed.
1-Sep-2010 RHESSI FLARE CATALOG - New version of the flare catalog is online that uses the 6-12 keV energy band (previously used 12-25 keV) and now includes ~24,000 microflares.   More details.
23-Aug-2010 OSPEX -added chi2_map method to compute uncertainties on fit parameters.  Chisquare mapping analysis consists of varying one parameter through a range of values surrounding the best-fit value, fitting the other parameters, and then examining the chisqr vs parameter value curve to determine 1- and 2-sigma error estimates. More details are in the full OSPEX documentation.
20-May-2010 BROWSER - Added daily and orbital Fermi GBM plots to BROWSER.  The orbital plots are on RHESSI orbit times, so the plots should line up with the RHESSI orbital plots.
13-May-2010 OSPEX -added albedo function component.
1-May-2010 IDL 6.1 is now the minimum version required to run RHESSI software.
16-Apr-2010 IMAGING - Added a new control parameter, CLEAN_REGRESS_COMBINE.  Defaults to 0.  Set to 1 to combine the component map with the residual map based on a regression of the count rate profiles from each of the two maps against the observed profile, resulting in a better match between the average observed and measured profiles.
15-Apr-2010 IMAGING -  There is a new tool for comparing observed modulation profiles with the expected profiles calculated from a reconstructed image, and for calculating the C-statistics and other fit parameters resulting from that comparison.  More details.
15-Mar-2010 OSPEX - added monte_carlo method to compute uncertainties on fit paramaters.  Monte Carlo analysis consists of doing many fits to Poisson random deviates of the model computed from the best-fit parameters, and determining the spread of the resulting fit parameters.  More details are in the full OSPEX documentation.
20-Jan-2010 SPECTRUM - For rear segments, implemented artifact mask to remove artifacts in fixed channels from specific detectors. Used in spectrum and SRM files export to OSPEX.
Jan-2010 OSPEX -FERMI GBM data can now be analyzed by OSPEX.  See the Fermi Solar Flare Observations page for more information..
3-Dec-2009 OSPEX - Corrected problem with sigma of fit parameters.  A new version of mcurvefit was onlined that provides more reliable, stable sigmas on fit parameters. A problem with the old version allowed sigmas to become unreasonably small in some cases.
30-Nov-2009 OSPEX - YOHKOH GRS data can now be analyzed by OSPEX
9-Sep-2009 SPECTRUM - Added rear_no_anti control parameter (defaults to 1) to remove anti-coincidence rejection for rear detectors.  Extended DRM to higher energies (150 MeV) and made DRM consistent with the non-anti-coincident condition.
24-Aug-2009 OSPEX - New RHESSI OSPEX User Guide online
4-Aug-2009 GOES - Added a button on the Image, Spectrum, and OSPEX widgets to plot a GOES time profile for the currently selected time interval.
16-Jul-2009 IMAGE - Added UV_SMOOTH image reconstruction algorithm.
7-May-2009 Object parameter tables updated to include parameters for visibility and vis-based image algorithm objects.
http://hesperia.gsfc.nasa.gov/ssw/hessi/doc/params/hsi_params_all.htm
4-May-2009 Improved visibility object software, visibility bags, image cubes from visibilities. Complete info at http://hesperia.gsfc.nasa.gov/ssw/hessi/doc/image/hsi_visibility_howto.html
cbe_digital_quality default changed to .95
3-May-2009 Change in notification of software modifications.
1-Feb-2009 Commonly used documentation now available offline.  Table of contents in $SSW/hessi/doc/index.html
6-Apr-2008 New post-anneal gain solution online
28-Mar-2008 Important: RHESSI gain calibration issue since anneal
16-Jan-2008 RHESSI detectors successfully annealed
19-Dec-2007 Release 9.0
5-Nov-2007 PLOTMAN plot options improved.  Users can now set their own preferences.
3-Oct-2007 Halfscale artifact filtering method implemented
13-Jul-2007 Support for harmonics and visibilities
10-May-2007 Release 8.9
12-Apr-2007 More about using old control parameter defaults and setting your own defaults.
12-Apr-2007_(a) Critical improvements to the datagap detection and livetime integration algorithms
26-Mar-2007 Change in RHESSI object control parameter default values
10-May-2006 Release8.8
24-Feb-2006 Release 8.7
20-Sep-2005 Release 8.6
6-Sep-2005 Changes in how the RHESSI software finds data files.
30-Aug-2005 Default RHESSI subcollimator response table has been updated.
2-Aug-2005 Improvements to pileup correction, offaxis response determination, and livetime calculation.
8-Jul-2005 Release 8.5.
22-Jun-2005 Improvement and bug fix for pileup correction.
10-Mar-2005 New image object that handles both multi-image cubes and single images.
 http://hesperia.gsfc.nasa.gov/ssw/hessi/doc/image/hsi_image_howto.html
http://hesperia.gsfc.nasa.gov/ssw/hessi/doc/gui/gui_changes_march2005.html
8-Mar-2005 Release 8.4.
23-Feb-2005 Fixed image blurring caused by culling bug.
16-Dec-2005 Release 8.3.
29-Nov-2004 Changes to Back Projection and Clean, new parameters for culling, rate-enabled back projection, local averaging and more.
7-Sep-2004 Fixed bug that caused incorrect source position in images in rare cases.
29-Jul-2004 Enabled writing multiple attenuator states in a single SRM FITS file.
16-Jul-2004 Release 8.2.
7-Jun-2004 Changed release/development version strategy.
19-Mar-2004 Release 8.1.
29-Sep-2003 Updated Roll database covering February 2002 through August 2003 online.
12-Sep-2003 Decimation, pileup, and flare position corrections implemented.
12-Aug-2003 Roll Aspect Data Base introduced.
14-Jul-2003 Livetime accumulation correction.
14-Jul-2003 (a) Detector 2 degraded.
1-Jul-2003 Improvements to SPEX and XRAY packages.
3-Mar-2003 More explanation about new daily observing summary files.
11-Feb-2003 Release 8 of HESSI Analysis Software online.
18-Nov-2002 Many changes to HESSI Analysis Software on ATEST (dropouts, deadtime, narrow spectrogram bins, SPEX export, error handling mechanism).
17-Nov-2002 Error handling mechanism (catch) added to HESSI software.
15-Nov-2002 SPEX thermal spectra now includes atomic line emission below 8 keV.
26-Oct-2002 Correction to image normalization at high energies - front/rear segment issues.
22-Oct-2002 Error in image and spectra normalization - assumed thin attenuator always in place.
1-Oct-2002 Spectral response matrix error for front segment of Detector 9.
19-Sep-2002 Daily observing summary files, daily instrument log files, and corrected count rate plots introduced.
29-Aug-2002 Fixes to dropouts, livetime. Detector gain a function of date.
24-Jul-2002 Interpolated roll solution for July 23 event.
16-Jul-2002 Imaging with Grids 1 and 2 fully supported.
11-Jul-2002 PMTRAS diagnostic plots.
8-Jul-2002 New multiple energy band parameter for imaging. Change in sum_flag, sum_coincidence parameters.
19-Jun-2002 Livetime corrections enabled for all classes (spectrum, lightcurve, and image).
22-May-2002 Preliminary subcollimator response table to enable imaging with Grids 1 and 2.