Active Region Selective Extraction
There are two data sets available on solarmonitor.org. These are explained in detail below, along with the programs in each case. At the moment the files are all in individual day format. I'm going to tarball them into year format to make them easier to download.
The famous 10,000 region dataset
Peter and I extracted a large dataset from the MDI archive . This was one full disc MDI per day, cosine-corrected using Peter's program. A number of active region parameters were also extracted from SEC. These data are avaliable on solarmonitor here as gzipped .fits files. These data were projection-corrected using a cyclindrical equal area projection. These corrected data are available on solarmonitor here as gzipped .fits files. These data were used for the fractal paper (ApJ, accepted, september 2005) explained in more detail on the fractal webpage.
Since this paper, 3 problems with these data have been identified.
(1) The cosine correction was incorrect - see the header in mdi_cos_correct.pro
(2) The projection program was incorrect - see the header in map_carrington3.pro
(3) The program to identify the Mount Wilson class was incorrect - see the header in get_nar2.pro
While these three effects are expected to be small, they *do* exist in these data.
The Auto-magically extracted dataset
After fixing the problems outlined above, I wrote a routine to extract active regions auto-magically from full disc MDI. This routine is described in the active region extraction paper (Solar Physics Topical Issue, accepted, 2005) and on the region growing webpage.
I also pulled a lot more information on flare rates, active region sizes etc. from SEC (All the SEC information is available at the srs and events list for each year). These data are available on solarmonitor here as gzipped IDL .sav files. (ADDITION 15 July 05 I fixed another few bugs and reran the routines). Each .sav file contains arse (active region selective extraction; pronounced aka Father Jack ; an IDL structure) and map. In each element of the structure, the following information is available by
IDL>help,arse,/str
A description of each of these is avaliable - see the header in arse2struct.pro
IDL>plot_mdi,map
plots the full disc magnetogram with overlayed NOAA regions - check this against the ARM data. This is the original .fits file from the MDI archive, with no corrections. So if you're worried about exactly how the corrections have manipulated the data, compare it against this.
A few important things to note when working with these data are:
(1) MDI saturates and becomes non-linear around 1200G. This creates an artifical plateau in the data, which multiscale methods will inevitably discover. As I was interested in simple stats (e.g., total flux) I added in a 'false' multiplacative factor - see the header in mdi_cos_correct.pro.
(2) The ARSE_NONPIXELS part of the structure flags pixels which have already been identified as belonging to a previous active region.
(3) The program typically find ~20 regions of magnetic significance (as defined by the minimal neccessary number of arbitrary parameters used) per day. Some of these are real active regions (i.e., NOAA_AR has an active region number entry ), some are plage (i.e., NOAA_AR is '-1' and either ARSE_NEG_FLUX or ARSE_POS_FLUX is very small), some are data spikes (i.e., ARSE_TOTAL_SIZE, ARSE_TOTAL_FLUX are very small).
I'd prefer using the second dataset for future studies, as it is most correct (least wrong?). I'm going to run a bunch of batch files while I'm away, which will include repeating the fractal work on this larger, better dataset.
IDL PROGRAMS
In the interests of repeatable science, here are the programs used in extracting all these data. They are documented well in most cases. arse2struct.proplot_mdi.pro
get_nar2.pro
drot_nar2.pro
events2struct.pro
flare_stats.pro
mdi_cos_correct.pro
map_carrington3.pro
calc_pixel_indices.pro
calc_reg_growing3.pro
contig_pixels.pro
If these programs require other non-ssw programs, let me know and I'll make them available here. Email me with any other queries /ideas / suggestions.