\documentclass[11pt]{article} \usepackage[dvips]{graphicx} %% FOR INCLUDING GRAPHICS \usepackage{natbib} %% ---------------------------------------------------------------------- %% Add specials here %% ---------------------------------------------------------------------- \def\ion#1#2{#1\,{\sc #2}\relax} %% %% ---------------------------------------------------------------------- %% Modify: %% Note number, version number and date %% Title %% Author and affiliation %% ---------------------------------------------------------------------- \setlength{\oddsidemargin}{0in} % 1 inch (0 + 1) from left. \setlength{\topmargin}{-0.5in} % 1 inch (1.5 - 0.5) from top. \setlength{\textwidth}{6.5in} % 1 inch (8.5 - 1 - 6.5) from right. \setlength{\textheight}{9in} % 1 inch (11 - 1 - 9) from bottom. \setlength{\parindent}{0pt} \addtolength{\parskip}{0.5\baselineskip} % The environment centerpage centers text vertically on the page. This is % useful within a titlepage environment. \newenvironment{centerpage}{\mbox{} \protect\vspace*{\fill}}{ \protect\vspace*{\fill} \mbox{} \protect\\ \mbox{}} \newcommand{\lam}{$\lambda$} \newcommand{\arcsec}{$^{\prime\prime}$} \begin{document} %\bibliographystyle{plain} % Define bibliography style. \begin{titlepage} \hrule \noindent\textbf{EUV IMAGING SPECTROMETER}\\ \vspace{-0.7\baselineskip}\\ {\noindent\Huge\bf Hinode} \vspace{2mm} \hrule \vspace{3mm} \centerline{\bf EIS SOFTWARE NOTE No. 19} \vspace{3mm} \hrule \noindent Version 2.2 \hfill 6 December 2019 \vspace{2mm} \hrule \begin{centerpage} \begin{center} {\Large\bf How to access and use the EIS databases}\\ \mbox{}\\ \mbox{}\\ Peter Young\\ NASA Goddard Space Flight Center\\ Greenbelt, MD 20771\\ U.S.A.\\ \end{center} \end{centerpage} \end{titlepage} %% ---------------------------------------------------------------------- %% The document! %% ---------------------------------------------------------------------- \section{Overview} The EIS project makes use of the same IDL database structures and software that were used for the SOHO/CDS mission. These in turn were developed by Don Lindler for the HST/GHRS and IUE astronomy missions, and are referred to as the ``UIT database system''. An important reference document on how the databases work and how they can be accessed is available as CDS Software Note \#10, available from the CDS homepage (http://solar.bnsc.rl.ac.uk) and from the soho/cds branch of SSWDB. The reader is referred to this document for more details. The present document contains examples on how information about EIS studies can be obtained from the EIS databases. A key feature of the UIT database system is that large amounts of information can be accessed very quickly, making it relatively easy to do complex searches on different parameters. \section{Summary of the EIS databases} There are several different EIS databases but this document will consider those databases that are most relevant to the general EIS user. \subsection{The as-run database}\label{sect.asrun} The two main databases are those that list the properties of the studies and rasters that were run by the instrument -- the ``as-run'' database. (Note: there is an ``as-planned'' database for those studies that were scheduled to be run by the EIS Chief Observer, but not all of these studies will have actually been run.) The as-run database actually consists of two distinct databases, called \verb|eis_main| and \verb|eis_experiment|. They are located in the directory \begin{verbatim} $SSW/hinode/eis/database/catalog \end{verbatim} \verb|eis_main| contains a list of all of the timeline entries, while \verb|eis_experiment| contains a list of all the individual rasters. To illustrate the difference, consider how an EIS operation is planned. The EIS Chief Observer (CO) places a study on the EIS planning. The study can consist of multiple rasters, although usually it contains only one raster. The CO can set the study to have multiple raster repeats. Consider the following example: \begin{tabular}{lll} Study & Raster 1 & 1 run\\ & Raster 2 & 10 runs\\ \end{tabular} where the study consists of two rasters, the first raster is run once, while the second raster is run 10 times. The study has a unique timeline ID (\verb|tl_id|) associated with it and will have a single entry in the \verb|eis_main| database. In the \verb|eis_experiment| database, there will be 11 entries that each have the same \verb|tl_id| value as the study in \verb|eis_main|. These correspond to the 11 rasters of the study. Note that each of the 11 rasters will end up as a separate FITS file. \subsection{The raster and line-list databases} The two as-run databases do not contain all the information about the rasters that have been run, and so it may be necessary to extract information from the raster and line-list databases. The former contains information such as the raster step size, field-of-view and exposure time, while the latter contains the list of emission lines observed by the raster. The raster (\verb|eis_raster|) and line-list (\verb|eis_linelist|) databases are stored in: % \begin{verbatim} $SSW/hinode/eis/database/planning_db/technical_db \end{verbatim} \subsection{The as-planned databases}\label{sect.asplanned} There are four databases related to the timeline, ``as-planned'' database, and they are stored in % \begin{verbatim} $SSW/hinode/eis/database/planning_db/timeline_db \end{verbatim} % Basically these store the information for the plans that the EIS COs prepare. Note that not all of the planned studies will actually run. For example, a study may be aborted if EIS hits its telemetry limit. Information about the rasters is stored in \verb|eis_science2_db|. One difference with the \verb|eis_experiment| database is that repeats of a raster will not have an entry. Consider the example from Sect.~\ref{sect.asrun}. In the as-planned database, this \verb|tl_id| will only have two entries: one each for the two different rasters. Although the second raster was run 10 times, only the first of the rasters has an entry in the as-planned database. \section{Examples} \subsection{Searching by time} The following gives an example of how the EIS catalog files can be searched. Suppose we want to find the raster that was running at 22:40~UT on 2007 January 20. The sequence of commands is as follows: \begin{verbatim} dbname=getenv('SSW')+'/hinode/eis/database/catalog/eis_experiment' dbopen,dbname t_obs='22:40 20-jan-2007' t_tai=anytim2tai(t_obs) t_tai_str=trim(t_tai,'(f15.3)') query='date_obs < '+t_tai_str+', date_end > '+t_tai_str list=dbfind(query) dbext,list,'filename,tl_id',filename,tl_id dbclose print,filename \end{verbatim} which will return the filename \verb|eis_l0_20070120_223207.fits.gz|. The reader should note the key parts of this query: \begin{enumerate} \item Open the database (dbopen). \item Create a query string. \item Perform the query (dbfind). \item Extract information from the database (dbext). \item Close the database (dbclose). \end{enumerate} When determining what fields to extract from the database, the user should do: \begin{verbatim} dbhelp \end{verbatim} (when the database is open) and a list of all of the fields will be displayed. The user can choose from any of these when making a query or extracting information. \subsection{Extracting study information} The eis\_experiment database only contains information from the raster database. Suppose that we want to extract some of the meta-data from the study database that is associated with the raster found in the previous example. The key database item that maps the raster database to the study database is the timeline ID number (TL\_ID). In the previous example we extracted the timeline ID, which we find to be 697. We now open the study database to extract information for this timeline ID. \begin{verbatim} dbname=getenv('SSW')+'/hinode/eis/database/catalog/eis_main' dbopen,dbname query='tl_id = 697' list=dbfind(query) dbext,list,'stud_acr,obstitle,obs_dec',stud_acr,obstitle,obs_dec dbclose print,stud_acr,obstitle,obs_dec \end{verbatim} You should find the following output: \begin{verbatim} PRY_loop_footpoints Cool lines for study of loop footpoints (e.g., near sunspot); 80" r& Cool lines for study of loop footpoints (e.g., near sunspot); 80" r& \end{verbatim} The latter two outputs are fields intended to give information about the observation that are written by the EIS Chief Observer at the time the EIS plan is created. \subsection{Extracting detailed raster properties} Some properties of a raster are not stored in the \verb|eis_experiment| catalog. Examples include exposure time(s) and the wavelength information for the data windows. These can be extracted from the raster and line list catalogs in the case of exposure times and wavelengths, respectively. An example for how to extract the exposure times defined for a raster is given below. The key link to the \verb|eis_experiment| catalog is the raster ID. \begin{verbatim} dbname=getenv('SSW')+'/hinode/eis/database/planning_db/technical_db/eis_raster_db dbopen,dbname query='id = 46' list=dbfind(query) dbext,list,'exposures',exposures dbclose print,exposures \end{verbatim} The raster ID here belongs to the raster of the PRY\_loop\_footpoints study. The output \verb|exposures| is an 8 element integer array that gives exposure times in milliseconds. The first entry is 30000 (30~s) and the rest are zeros. This indicates that there is only one exposure for this raster (a maximum of 8 are possible). The Solarsoft routine \verb|eis_get_exposure_info| makes use of the raster catalog to extract a number of pieces of information relating to exposure time, and the reader is referred to this for more details. For the line list catalog, each entry has an ID number that corresponds to the line list ID number (\verb|ll_id|) in the raster catalog. For the raster considered above, one has: \begin{verbatim} dbname=getenv('SSW')+'/hinode/eis/database/planning_db/technical_db/eis_raster_db dbopen,dbname query='id = 46' list=dbfind(query) dbext,list,'ll_id',ll_id dbclose print,ll_id \end{verbatim} We find the line list ID is 15, so we can now query the line list catalog to find the number of wavelength windows for the raster: \begin{verbatim} dbname=getenv('SSW')+'/hinode/eis/database/planning_db/technical_db/eis_linelist_db dbopen,dbname query='id = 17' list=dbfind(query) dbext,list,'n_lines',n_lines dbclose print,n_lines \end{verbatim} The number of wavelength windows used by this raster is thus 20. Use the \verb|dbhelp| routine to find the full list of items in the raster and line list catalogs. \section{The EIS\_OBS\_STRUCTURE routine}\label{sect.eos} The routine \verb|eis_obs_structure| (available in Solarsoft) serves as an example of how to extract information from the four databases described in this document. There are two main options for calling it. The first is a search for all observations between two times: % \begin{verbatim} IDL> str=eis_obs_structure(t0,t1) \end{verbatim} The second is a search for a particular raster (not study) acronym: % \begin{verbatim} IDL> str=eis_obs_structure(rast_acr='cool_lines') \end{verbatim} % If you know your study acronym, then you can find the corresponding raster acronym by running \verb|eis_xstudy| and searching for the study. Click on the study, then the bottom of the widget will show the raster acronym. Generally there is a one-to-one correspondence between rasters and studies, but multiple studies can use the same raster. You can add the additional keyword study\_acr to pick a specific study in the latter case. You can also use T0 and T1 to restrict the search to a particular time range. \verb|eis_obs_structure| identifies all rasters that were run during this time range and extracts meta-data, which goes into the output structure 'str'. The tags of this structure for one example are given below: % {\small \begin{verbatim} DATE_OBS STRING '2007-01-20T00:27:12.000' DATE_END STRING '2007-01-20T00:31:40.000' XCEN FLOAT 175.542 YCEN FLOAT 76.9085 FOVX FLOAT 40.9344 FOVY FLOAT 304.000 FILENAME STRING 'eis_l0_20070120_002712.fits' TL_ID LONG 677 STUD_ACR STRING 'PRY_fast_dens' RAST_ACR STRING 'Quick_AR_raster' LINK STRING 'http://tcrb.nrl.navy.mil/~iuu/eis/thumbnails/rasters/quick_ar_raster/eis'... RAST_ID INT 44 WAVELNTH STRING '19282 19512 20204 20383 25632' WAVEMIN STRING '19246 19476 20177 20347 25587' WAVEMAX STRING '19317 19547 20230 20418 25676' OBSTITLE STRING 'High cadence sit-and-stare in ARs; 5s exposure; Fe XIII dens; 40" c' OBS_DEC STRING 'High cadence sit-and-stare in ARs; 5s exposure; Fe XIII dens; 40" c' HOP_ID INT 0 SCI_OBJ STRING '' SLIT_WIDTH INT 1 SIT_STARE BYTE 0 STEP_SIZE FLOAT 1.00000 SLIT_INDEX INT 0 NSTEPS INT 40 NEXP INT 1 EXP_TIMES STRING '5.0' \end{verbatim} } \section{The uniqueness of TL\_ID}\label{sect.tlid} Each \emph{study} that is placed on the timeline is assigned a unique timeline ID, or TL\_ID. As the mission progresses, the TL\_ID numbers increase, although not necessarily monotonically as the EIS CO does not necessarily create the EIS plan by systematically adding studies from the beginning of the timeline onwards. If you know the TL\_ID, then you can retrieve information about the rasters by doing: % \begin{verbatim} IDL> str=eis_obs_structure(tl_id=tl_id) \end{verbatim} % Note that there may be multiple entries as a single study can consist of multiple rasters. There are three known problems with searching on the timeline ID, however, and these are discussed below. \subsection{Rasters before 23 November 2006} The EIS planning tool began being used from 13:14 UT on 23 November 2006. Prior to this EIS observations exist and were assigned TL\_ID values that were generally quite high, but the planning tool does not recognise their existence. Therefore eventually as time passed, the planning tool reached these TL\_ID values and reused them. An example is TL\_ID=37200. Therefore if you search for one of these TL\_ID values then you will find two entries widely separated in time. For the \verb|eis_obs_structure| routine, the pre-23 November studies are automatically filtered out, but you will see them if you use the UIT database software. \subsection{Response studies} Response studies are special cases that are assigned TL\_ID values of 1, 3 and 4. If you search on one of these values, then you will find many entries widely spaced in time. Please check the website below for more details. \begin{verbatim} http://solarb.mssl.ucl.ac.uk:8080/eiswiki/Wiki.jsp?page=TriggerStudies \end{verbatim} \subsection{TL\_ID values 16541 to 16603 (March 2010)} There is a period running from 07:33 on 20-Mar-2010 to 02:02 27-Mar-2010 for which the TL\_ID values were duplicated. This is because the EIS CO who began planning at 11~UT 27-Mar-2010 did not have the most recent copy of the database, and so the planning tool began assigning TL\_ID values that corresponded to the start of the previous week. The consequence is that most of the TL\_ID values between 16541 and 16603 are duplicated. Searching for one of these TL\_ID values will thus lead to two entries, separated by about a week. \newpage \appendix \section{Document modification history} \emph{Version 2.2}: Expanded information about \verb|eis_obs_structure| in Sect.~\ref{sect.eos}.\\ \emph{Version 2.1}: Added Sect.~\ref{sect.asplanned}.\\ \emph{Version 2}: Added Sect.~\ref{sect.tlid}. \end{document}