How To Use REF_SUPER Ver 2.0 MDM 9-Jun-92 Ver 2.1 JRL 15-Jul-92 1. Run the "ref_super" program A. Log into isass0 (IMPORTANT) B. Make sure that the data that was on the temporary directories /yd2/reformat_a1 and /yd1/reformat_a2 have been moved backed up to tape and copied to another disk. The "ref_super" program will delete all files there first! There also needs to be at least 400 megabytes free on /yd1 in order to reformat data taken after Nov-91. C. Make a program named "go_ref_super.pro" that has the following: ; ref_super, '92_12' end where the string '92_12' is the year and week number that is to be reformatted. See Appendix A for a description of the options available within the REF_SUPER call. E. Type the following at the Unix prompt idl_batch go_ref_super ~/Iref_super The procedure will create two files in your top directory, "IDL_BATCH" and "ref_super" with the starting date and time tagged onto the end of it. The IDL_BATCH file is all output from running the ref_super programs that would come to the screen. Error messages would be in that file (check the end of the file with the "tail" command). The "ref_super" file is a summary of the time it takes to run each stage. If a subdirectory "ref_super" exists on your top level directory on isass0, those summary files will be written in that directory. When the program has finished reformatting and making all of the data files, it should send you a mail message telling you that you can make the tapes. REF_SUPER is still running at that time and making the SXT log files and SFD files. BE AWARE. 2. Make two master tapes A. Log into isass0 (IMPORTANT) B. Get two blank exabyte tapes. Write the label on the tape. It should be of the form yy_ww.nn-v.vvv Where "yy_ww" is the year and week that was reformatted, "nn" is the archive number, and "v.vvv" is the reformatter version number. If the mail message arrived telling you that the reformatting finished successfully, then the tape name is contained in that message. It is also necessary to write the date the tape is written and if it is a master tape or copy tape. Below are three example tape labels ------------------------------------------ 92_10a.01-1.02 17-Mar-92 M ISAS ------------------------------------------ 92_11a.01-1.05 23-Mar-92 M LPARL ------------------------------------------ 91_49a.03-1.00 13-Mar-92 C1 NAOJ C. Move to the data directory where the script files are located cd /yd1/reformat_week D. Type the following at the Unix prompt idl_batch go_mktap ~/Imktap idl_batch go_mktap1 ~/Imktap1 3. Make one more copy of the tape. A. Log into isass0 (IMPORTANT) B. Put a master tape into drive 0 C. Write a label (see definition above) and stick it on a blank tape. Stick the blank tape in drive 1. Give this copy to the BCS team who will make one additional copy. ************************************************************* This does not seem to apply: (for one of the copies, get a special blank tape from the BCS team) ************************************************************* D. Type the following at the Unix prompt ;**** idl_batch acopy2 ~/Iacopy2 ;**** Doesn't work IDL>.run acopy And indicate that NAOJ is the recipient. 4. Make the Magneto-optical disk A. ?? B. ?? 5. Move the reformatted data to another disk A. Determine the amount of space required on the destination disk. Enter IDL and type move_sizes If the data is on the "a" directories, answer yes, otherwise no. The total required is on the last line to the right of the 0-6. B. Determine the data useage on the data disks. First, in IDL type IDL>.run yd_use2 ; Creates a summary file called yd_use2.txt and also at the Unix prompt type week_on ; Script file - shows were the weeks are located C. The oldest "n" weeks will have to be deleted to make space (generally only one week has to be deleted to make space for a new one). When the week to be deleted is determined, make sure that there are no big flares in that week. If there are, copy ALL datafiles for the flare period (generally only one orbit) to the /yd6/flares directory. Determine if there are flares by logging into the Hawaii SPAM account. This is done as follows: telnet koa.ifa.hawaii.edu spam ; Account name (no password) ; Question about terminal type (Vt100 =default) S ; for Seldads (no required) E ; for event listing (no required) And then you edit the time interval you want. Make sure you go from Sunday until Sunday of the next week. Enter X for the GOES flare classification. SPAM will then search and list all flares for the time period. D. Once the flare data has been coped, delete that directory. An example to delete /yd5/92_22a is: cd /yd5 rm -fr 92_22a E. Make the new directory. For example: cd /yd5 mkdir 92_30a F. Move the files over there. There is an IDL routine to do that which will tell you the transfer rates. An example is: dir_a = [finddir('reformat_a1'), finddir('reformat_a2')] move_files, dir_a, '/yd5/92_30a' The default is not to delete the files, however, the files can be deleted as they are moved by the /DELETE keyword (it uses a mv command instead of a cp command) IMPORTANT: Mons's routine gives you an update as it goes along. Watch for time-out messages such as: cp: /yd7/92_27a/./bda920704.1052: Connection timed out If this happens, you will have to manually copy the effected file. IMPORTANT: Once the files have been moved, make sure you do a chmod so that they can be deleted later by someone other than yourself. This is especially true of the FLARE machines in case they change the superuser password and not tell us. g. Delete the temporary KSC reformatted data from /yd3/reformat that corresponds to the week that was just completed. 6. Move the sfd and sfw files from /yd1/reformat_week to /yd1/sfd. Then follow Gary's procedure to write these data to the laser disk. 7. Check that the /yd1/sfd directory is not getting too full. If necessary, copy some files to a MO or tape and delete the old files. /yd1 should always have 400 megabytes free when starting a week of reformatting. If the SRSGET spills over onto /yd1, then it will reformat onto the "other" directory on /yd1. Therefore, it is wise to observe where the data is copied during stage 0 in case you need to clear up some space on /yd1. 8. Start the whole process over again 9. Once a week, mail the tapes to the U.S. A. Package the tapes that need to be sent to LPARL and NRL. Type the following commands to get an address label. /pslaser /ys/sxt/doc/tape_shipment.txt type10 B. Take the box to Prof Ogawara. (An alternate method is to call DHL to request a pickup. Phone (045) 201-2580) APPENDIX A: REF_SUPER calling options *. The /CHECK_DEL option will allow the user to start the SIRIUS copy (stage #0) of REF_SUPER and have the program check the output directory for stage #1 to see if it is empty before continuing. If there is still data there, the program stops and must be started again. Example: ref_super, '92_07', /check_del *. If the transfer stalls during SIRIUS copy (which is quite common), it is possible to transfer only select days by using the DAYS keyword option. The DAYS keyword option is currently used by both stage #0 and stage #1 so beware. Example: ref_super, '92_05', stages=[1], days=indgen(4)+3 *. Since the data is copied from SIRIUS to disk "a", reformatted in 24 hr chunks to disk "b", and then broken in orbit files back to disk "a", it is sometimes desirable to start the SIRIUS copy going and have it write to disk "b". The keyword option /DIR_REVERSE allows the a-b-a to switched to b-a-b. If this is used for the SIRIUS copy stage it should be used for ALL subsequent calls. Example: ref_super, '91_39', /dir_reverse, stages=[1] ref_super, '91_39', /dir_reverse, notstages=[1] *. The keyword NOTSTAGES is the opposite of stages. The following pairs are identical ref_super, '92_04', stages =[0,0,0,0,0,0,0,1,1,1,1] ref_super, '92_04', notstages=[1,1,1,1,1,1,1]