³ò kóýIc @sOddkZddkZddkZddkZddkZddkZddklZlZl Z ei di dƒdZ eiƒZ ddZdZd Zd „Zd „Zd „Zd „Zd„Zd„Zd„Zd„Zd„Zd„ZedjoDedeiƒZedjoe ddedƒneeƒndS(iÿÿÿÿN(thide_envtstarttsaytHOSTNAMEt.ii€Qii tskyd_c Cshd}tddgƒ}tit||ƒ}|ddjot|dd|ddƒn|id ƒ}|oíd t|d /skyd_ with a unique set # of characters (created by tempfile.mkstemp). # MODIFICATION HISTORY: # DEC-2005, Paul Hick (UCSD/CASS; pphick@ucsd.edu) #- t skyd_loadtnumberitmessagettEtcfs terminatedtconfs%dtpidtbosss %Y_%j_%H%M%Sttimetgroup_it cur_grouptmax_loaditgruntst,t reportdirtIt#sdoes not existtTUBsis report directoryt:N(Ri(smessageR (tdictt skyd_functskyd_read_conftcffileRtpopR RRtstrftimetkeystfindthas_keytinttsplittostpatht expandvarsRtisdirtenvironRtskyd_write_conftrangetlentmintnew_runstappendtskyd_empty_run( tclaimtcsaytstatusR RtkeyRRt grunts_loadtitruntgrunt((s skyd_wait.pyRsp)       # %  cCs¾d}t|dddƒt}tƒax3tiƒD]%}gt|_runs or _kill. # If a process is marked 'runs' then the matching # report file has name _done or _kill # If a process is marked 'dead' then no report # is expected for that process. # MODIFICATION HISTORY: # DEC-2005, Paul Hick (UCSD/CASS; pphick@ucsd.edu) #- t skyd_find_runtreportR3tdeadR R s,run %d is %s, but has a watermark terminatedRs-run %d is %s, but has no watermark terminateditrunst_itkilltdonetwmarkR8R7tresult( R@R R+R,RR%R&R$R?R( treportstlst_runsR2t run_foundt all_gruntsR8R7RER3tnameRLtrtn((s skyd_wait.pyRDs\-   %  !   !     cCs titƒ}g}xl|D]d}ti|ƒdjotdddd|ƒqn|itƒdjoqn|i|ƒqWt |t ƒ}|ddjo#ti|dƒd|d<|Snt |t ƒ}|ddjoti|dƒd |ds*got alarm signal, no activity for too longN(RR9Rtskyd_kill_runsR<R(RdReR2((s skyd_wait.pyRgKs cCs#d}tat|dddƒdS(sr #+ # NAME: # skyd_ctrlc # PURPOSE # MODIFICATION HISTORY: # DEC-2005, Paul Hick (UCSD/CASS; pphick@ucsd.edu) #- t skyd_ctrlcRXsctrl-cs"terminator set; coasting to finishN(tTrueR;R(RdReR2((s skyd_wait.pyRibs  c Csd}d}xm|tjo@tƒ}|d}|djoPntidƒ|d7}qWt|dddd tƒd S|d }|d }|d }|d} |d} t|dƒiƒ} yti|ƒt i|ƒWnt|ddd|ƒnX| djoat | ƒdjo't|dddt |ƒƒd} qpt | i dƒdi ƒdƒ} nt|dddt |ƒƒt|d|d||| | fd| ƒ|ddjo¼| djoCdt||d<| t||dsreceived SIGUSR1 signal,s& but no report found within %d secondsNRKR8R7R3RLtrs!ignoring os.remove() error for %sRRsempty report file, %siÿÿÿÿs s %s srun %d (%s): %s -> %ss %sRTR.RGR s %Y_%j_%H%M%SRRIR tstopsrats 2 terminateds)ignoring temination signal from old grunttSs!all processes finished terminatedR;s'is set; %d process has not finished yets*is set; %d processes have not finished yet(t report_delayRSRR`RtopentreadR%tremoveRVR,RR#R$R.RRR0RjR;tncountRAR<Rtskyd_count_runstskyd_show_runsR(RdReR2RCRRRERKR8R7R3RLRtgpidR?((s skyd_wait.pyRkzs€              '+          cCs tiitiitidƒdƒd}t|d|dƒtiƒa|a da tii t ƒpt|ddt dƒndti d d d d tt t fd }d|dad|dattƒtƒaxjtitƒD]Y}|itƒdjo=tiit|ƒ}t|dd|dƒti|ƒqóqóWtd'd(gƒ}tit t|ƒ}|ddjo't|ddt |ddƒdSn|idƒ}x9|iƒD]+}|idƒdjoqËn||}|idƒpqËnt |dƒpqËn|d}tii!|ƒ}tii |ƒpqËnti"t||ƒ}|ddjoqËn|idƒ} t |dƒ} t |dƒ} d} xì| D]ä} t#| ƒdjoqÆn| i$ƒoqÆn| idƒdjoqÆn| id ƒ}|d!joqÆn| iƒ}t |dƒ| joqÆnt |dƒ| joqÆn| d7} | d|!d"| | i%| ƒ= # = # .. # = # group_0: # <_name_1>= # .. # <_name_m>= # group_1: # <_name_1>= # .. # <_name_m>= # .. # .. # group_k: # <_name_1>= # .. # <_name_2>= # # The first block are 'global entries'. The following entries are separate # groups with instruction to run a specific group of orbits. skyd_orbit # calls by picking an orbit from group one, then group two, and so on, # cycling through all groups. # # Global entries: # # reportdir=reportdir directory where report files from skyd_orbit # runs are put. Default: $TUB # max_load=max_load max number of concurrent processes run on # each grund. Default: 2 # grunts=grunt1,grunt2 comma separated list of machines on which # to run skyd_orbit. Default: boss # # The following main entries are set when htm_wait is started. # # boss=boss machine controlling skyd_wait # pid=pid process id of skyd_wait # time=2005_354_162151 time at which skyd_wait started # cur_group=0 first group for which orbit is selected # (this field incremented in skyd_orbit) # # Group entries: # # Many of these correspond to keyword needed for the indexing program # smeidb_htm # # _camera=camera camera id (1,2,3); default: 1 # _mode=mode mode id (0,1,2); default: -1 (this effectively # select the main science mode for each camera # (mode 2 for cam 1 and 2; mode 1 for cam 3) # _min_orbit=min_orbit minimum orbit to run; zero means no restriction # on minimum orbit; default: 0 # _max_orbit=max_orbit maximum orbit to run; zero means no restriction # on maximum orbit; default: 1 # _source=source source for SMEI frames; default: SMEIDC? # _destination=destination # destination directory for sky maps # _level=level indexing level # _keepglare=0/1 0: subtract glare; 1; keep glare; default: 0 # _catalogue=catalogue user catalogue # _checkversion=0/1 0: don't check version number; 1: update only # if smeidb_skyd version is higher than version # in existing skymap # _overwrite=0/1 0: don't overwrite existing skymaps # 1: overwrite existing skymaps # # The user catalogue is an ascii file with one line for each orbit in the # following format: # # orbnr orbnr+1 YYYY_DOY_hhmmss status # # orbnr orbit number to be processed # orbnr+1 orbit number, plus one # YYYY_doy_hhmmss start time of orbit orbnr # status status of orbit; can be 'make','skip','busy','done' # # The catalogue can be constructed from a list of available skymaps # using the IDL procedure href=skyd_cat=. # MODIFICATION HISTORY: # DEC-2005, Paul Hick (UCSD/CASS) # NOV-2006, Paul Hick (UCSD/CASS) # The script sometimes crashed while executing the os.remove(report) # command claiming the file didn't exist even though the file # was just read succesfully, and the remove actually worked correctly. # Bracketed os.remove(report) command with try block to avoid crash. # Also added the reports_claimed dictionary to store the names of # report files that have been identified by skyd_find_run. The name # is deleted from the dictionary only if the os.remove() succeeds. # JAN-2007, Paul Hick (UCSD/CASS) # Moved initialization of global reports_claimed in front of the loop # starting the initial indexing runs. # MAR-2008, Paul Hick (UCSD/CASS) # Fixed bug in processing of -overwrite keyword. # Now if -overwrite is set for one or more groups then the # orbit catalogue for the range of orbits specified is updated # by changing the status for all "done" orbits to "make". # This way, skyd_orbit only needs to look at status "make" # if -overwrite is set. Without this change skyd_orbit will # keep processing the same "done" orbit over and over again. # NOV-2008, Paul Hick (UCSD/CASS; pphick@ucsd.edu) # The call to skyd_orbit is now prefixed with ". $HOME/.bashrc". # This did not use to be necessary. It is needed now for FC9; # not sure why. #- iiRs is now up R Rsdoes not exist terminateds!. $HOME/.bashrc; LD_LIBRARY_PATH=tLD_LIBRARY_PATHs; sexport LD_LIBRARY_PATH; s)skyd_orbit.py -pid=%d -boss=%s -cffile=%ss -report=%ssecho "s"" | at now+%d minutes 2> /dev/nullsecho \\\"export HOSTNAME; s%\\\" | at now+%d minutes 2> /dev/nullRXs ... cleanupRRR s terminatedNR Rt overwritet cataloguetcontentst min_orbitt max_orbitRJiÿÿÿÿsmake 0sdone -> make: %d orbitstwaits ... pid %d (%d;%.2f hr) ...i(snumberi(smessageR (:R%R&tsplitextR$tsystargvRtgetpidR RRstisfileR)RRZR[RRjRR<RURR!RWtjoinRrRRR@RR R"R#R't skyd_claimR,tisspacetindext skyd_releasetsignaltSIGUSR1RktSIGALRMRgtSIGINTRitSIGHUPRfRVR;R.RARuRtstimetalarmt long_pausetpause(t conf_fileR2R&RQR3R R4tgroupRytorbitsR{R|tmtorbitRCttmpR?R8((s skyd_wait.pyt skyd_waits¨£-  .         "  #  .  t__main__s-cffile=R R—R s4specify config file with -cffile= terminated(RR%RˆRR\RttinyRRRR)R$RRRRoRWRR9RDRSRARfRgRiRkR—t__name__R€R‘(((s skyd_wait.pyss8         } ~ v < z    – ÿC