Object-Oriented Data Analysis Software Concept

This document describes the object-oriented concepts used to implement scientific data analysis tasks, mainly in the context of the HESSI data analysis software. It also describes the method to implement data analysis tasks using frameworks. This description assumes an IDL solarsoft environment.

Introduction

font-family : Myriad Web, Geneva, Arial, Helvetica,sans-serif; The HESSI data analysis software uses object classes to stepwise reconstruct images, spectra and lightcurves from raw data. Each step of the construction is associated with an object class. In the context discussed here, an object class includes a primary data type, a process method, control and information parameters, accessor methods and display methods.

• The primary data type defines the data managed by a given object class implementation.
• The process method implements or calls the algorithm which transforms a given input (source) data into the primary data type managed by the class.
• Control and information parameters pass values between the algorithm implemented in the process method and the object client (i.e. the user of the object). Control parameters are used to controls the behaviour of the algorithm. Information parameters are used to pass information (metadata) about the (primary) data generated by the algorithm.
• Accessor methods retrieve subsets of the primary data requested by the client object or user. They also retrieve control or information parameter values, and allow setting control parameter values.
• Display methods allow users to output to a specific media (screen, printer, image files) both primary data or parameters.

Object instance creation

An instance of an object class must be first created before it can be used:

IDL> o = Obj_New( 'class_name' )

An IDL object reference variable o is returned from the instance creation. Data and methods associated with the object can be accessed with this variable. During the instance creation, default values are set (i.e. assigned) to control parameters. Often, a constructor function is used to create the object reference:

IDL>o=class_name()

Primary data access

Primary data requests are handled by the accessor method GetData:

IDL>data = o->GetData( )

where o is the object reference. The object first checks whether its state is consistent, that is, if the primary data contained in the object corresponds to the values of the control parameters. If the object state is consistent, GetData returns (i.e. passes to the client that issued the request) the data that is contained in the "memory" of the object. If the object state is not consistent, the GetData method calls first the Process method, which puts the object back into a consistent state by running the transformation procedure associated with the class. After the transformation procedure has completed, the data is returned to the client.

Although the Process method can be defined fully arbitrarily, it usually consists of four steps.

1. The control parameters associated with the specific object (and thus associated with the algorithm) are loaded.
2. Data are read from a given source, usually (but not necessarily) by requesting data from another object, its source, of the same parent abstract class (abstract classes will be discussed below).
3. The algorithm runs for the specific control parameters and source data. It generates the primary data.
4. The primary data and the (optional) informational parameters are stored into the object's memory area.

Abstract and concrete classes

Object classes can be divided into abstract and concrete classes.

• Abstract classes define generic operations (most importantly accessor methods) that can be used, or implemented, by concrete object classes. Abstract classes are also called parent classes.
• Concrete classes implement an abstract class, i.e. they inherit an abstract class and define the data types and algorithms to use with this specific implementation of the abstract class. Concrete classes are also called child classes.

Abstract classes are never used alone. Their existence (as good parents!) makes sense only through concrete classes (their childs).

For HESSI, concrete classes are divided into several groups.

• Utility classes are "low-level" objects, and the largest group of objects. They allow reading data from files, simulating data, and generating intermediate and diagnostic data products used by higher-level objects. Most important utility classes are HSI_Packet, HSI_Eventlist, HSI_Calib_Eventlist, HSI_Modul_Pattern, HSI_Modul_Profile.
• Imaging classes are "high-level" objects, and the second largest group of objects. They define the imaging algorithms and the data products related to imaging tasks. Most important are HSI_BProj, HSI__PSF, HSI_Image, HSI_Clean, HSI_MEM_Sato, HSI_Pixon, HSI_Forwardfit.
• Spectroscopy classes define the data structures and algorithms used for spectroscopy tasks. Most important are HSI_Spectrum, HSI_Spectrogram.
• Lightcurve classes define the data structures and algorithms used for time series applications. Most important are HSI_Lightcurve and HSI_Binned_Eventlist.

All HESSI classes have a common parent class. This common parent class is called Framework. This class contains the code of the generic methods Get, GetData, Set, SetData, Plot, Write, Print. Because all classes inherit Framework, they implement the same interface. In other words, the same accessor and display methods are available for each class. As a result, and because each processing step reads data from a source object which implements the same parent class (a sibling object), the software actually reuses at all levels the same code of a single accessor method.

The data analysis process consist of a chain of transformation processed by sibling objects. A given class gets data from a source class, and passes data to a client class. The last client, of course, is the user.

Classes can be organized in well-defined design patterns. For HESSI, the imaging algorithms implement the template method pattern. An abstract class hsi_image_alg is common to all imaging algorithms. A (small) concrete class implements a so-called hook that contains the actual image algorithm (clean, mem, pixon, forward fit). All other processing steps are common and therefore implemented in the abstract class. In this way, the client object does not need to care what it uses for imaging algorithm. It only knows that it has for source an object with parent class of type hsi_image_alg

How to use a framework

This describes how you can use a framework for your own development. Let's assume you have the implementation of an algorithm and you want to manage it using a framework. First, why do you want that? There are several reasons.

1. The framework gives you a way of implementing a standard access interface consistent with the interface of other algoriths managed by a framework;
2. The framework gives you a way of keeping already generated data products in memory, avoiding excessive recalculations;
3. The framework provides you with a mechanism to check whether a data product is already available or whether it needs to be calculated
4. The framewrok provides you with a way of setting up a whole set of collaborations between independent classes.
5. The framework spares you development time of code that has already been developed

Let's assume you have implemented an algorithm algo that you want to manage with a framework. algo has the following delaration:

PRO Algo, input, output, param1, param2, out_param1, out_param2, KEYWORD1=keyword1, KEYWORD2=keyword2

where input is a variable containing the data to be processed by the algorithm (source data), output is the data generated by the algorithm, param1 and param2 are input (control) parameters, and out_param1 and out_param2 are output (info) parameters. Last, keyword1 and keyword2 are keyword values that can be set by users at run time.

There are a number of IDL templates ready to help you in the implementation. You can find these templates in your SSW installation in the directory $SSW/hessi/idl/objects, or you can get them here: framework_template__define.pro framework_template_control__define.pro, framework_template_control.pro, framework_template_info__define.pro.

Here is how you deal with it:

1. Configure the control parameter structure. all control parameters should be declared in a structure, e.g. with name algo_control__define.pro. You can use framework_template_control__define.pro to help you. The control structure {algo_control} would look like this:

   PRO Algo_Control__Define
   struct = {algo_control, param1: 0L, param2: 0.}
   END
   

   (the type of the parameters is arbitrary)
2. Configure the control parameter initialization function. In this function, you will assign to the variables the default values you want to provide. The function may have the same name as the control structure without the __define at the end, e.g. algo_control.pro. You can use framework_template_control.pro to help you. The initialization function for algo_control would look like this:

   FUNCTION Algo_Control
   var = {algo_control}
   var.param1 = 1002L
   var.param2 = !pi
   RETURN, var
   

3. Configure the information parameter structure. all information parameters should be declared in a structure, e.g. with name algo_info__define.pro. You can use framework_template_info__define.pro to help you. The information structure {algo_info} would look like this:

   PRO Algo_Info__Define
   struct = {algo_info, out_param1: 0B, out_param2: 0D}
   END
   

   (the type of the parameters is arbitrary)
   
4. Now you will need to make a copy of framework_template__define.pro to whatever your future class should be named, e.g. algo__define.pro Replace all occurences of framework_template with algo.
   
5. Configure the INIT procedure In this procedure, you will have to pass the control and information structures defined in steps above to the framework so that it knows what to manage. Moreoever, a source object can be passed to the framework of available. Note: inside of an object, methods and paameters are accessed using the self-reference of the object called self. For instance the init procedure for algo will look like this:

   FUNCTION Algo::INIT, SOURCE = source, _EXTRA=_extra
   
   IF NOT Obj_Valid( source ) THEN BEGIN
   source = obj_new( 'algo_source' )
   ENDIF
   
   ret=self->Framework::INIT( CONTROL = framework_template_control(), $
   INFO={framework_template_info}, $
   SOURCE=source, $
   _EXTRA=_extra )
   
   RETURN, ret
   
   END
   

6. Configure the Process procedure. According to the description above, process needs to read in the control parameters, call the function or procedure implementing the algorithm, passing any parameters and/or keywords requested by the algorithm, store the result of the transformation into the object memory, and store information parameters into the information structure. For instance, the Process for algo would look like this:

   PRO Algo::Process, KEYWORD1=keyword1, KEYWORD2=keyword2
   
   param1 = self->Get( /PARAM1 )
   param2 = self->Get( /PARAM2 )
   source = self->Get( /SOURCE )
   data = source->GetData( _EXTRA = _extra )
   
   Algo, input, output, param1, param2, $
   out_param1, out_param2, $
   KEYWORD1=keyword1, KEYWORD2=keyword2
   
   self->SetData, output
   self->Set, OUT_PARAM1 = out_param1
   self->Set, OUT_PARAM2 = out_param2
   
   END
   

   `
7. Configure the GetData function. This step is optional. If your algorithm outputs a complex data structure, it may be a good idea to provide some keywords to access subsets of the data, without involving a call to the Process method. This can be done by extending the functionality of the default GetData method. Lets assume the output of Algo is a 3D array, and you want to access slices of this "cube". You would define a set of keywords that would allow selecting the slices, prepending a THIS_ string to distinguish the accessor keywords from the control parameters. Thus the GetData method would look like this

   FUNCTION Algo::GetData, $
   THIS_X_SLICE=this_x_slice, $
   THIS_Y_SLICE=this_y_slice, $
   THIS_Z_SLICE=this_z_slice, $,br> _EXTRA=_extra
   
   data=self->Framework::GetData( _EXTRA = _extra )
   
   ; actually this could be done more nicely, but anyway...
   IF Keyword_Set( THIS_X_SLICE ) THEN BEGIN
   RETURN, data[this_x_slice,*,*]
   ENDIF ELSE IF Keyword_Set( THIS_Y_SLICE ) THEN BEGIN
   RETURN, data[*,this_y_slice,*]
   ENDIF ELSE IF Keyword_Set( THIS_Z_SLICE ) THEN BEGIN
   RETURN, data[*, *, this_z_slice ]
   ENDIF
   
   RETURN, data
   
   

8. Configure the Set procedure.This step is optional. It is sometimes necessary to do some specific action when a parameter is set, for instance when two parameters are dependent from each other. In this case the extended function woudl look like this:

   PRO Algo::Set, $
   PARAMETER=parameter, $
   _EXTRA=_extra
   
   IF Keyword_Set( PARAMETER ) THEN BEGIN
   
   ; first set the parameter using the original Set
   self->Framework::Set, PARAMETER = parameter
   
   ; then take some action that depends on this parameter
   Take_Some_Action, parameter
   
   ENDIF
   
   ; for all other parameters (included in _extra), just pass them to the
   ; original Set procedure in Framework
   
   IF Keyword_Set( _EXTRA ) THEN BEGIN
   self->Framework::Set, _EXTRA = _extra
   ENDIF
   
   END
   

9. Configure the Get function.This step is optional. The Get function needs to be modified only in very special cases, e.g. if you need to modify the value before passing in back to the user. This is not recommended, however. You should add two keyword variables NOT_FOUND and FOUND that must be passed to the Get function in Framework. Note also that you eventually need to retrieve parameter from the original Get function, otherwise most of the functionality of the Get function will not work (e.g. aggregation of requested parameters in a structure).

   FUNCTION Algo::Get, $
   NOT_FOUND=NOT_found, $
   FOUND=found, $
   PARAMETER=parameter, $
   _EXTRA=_extra
   
   IF Keyword_Set( PARAMETER ) THEN BEGIN
   parameter_local = self->Framework::Get( /PARAMETER )
   Do_Something_With_Parameter, parameter_local
   ENDIF
   
   RETURN, self->Framework::Get( PARAMETER=parameter, $
   NOT_FOUND = not_found, FOUND=found, _EXTRA = _extra )
   
   END