# XMM-LSS: Validation Report (10% SUBSET)

XMM-LSS: Validation Report (10% SUBSET)

 
Master catalogue used: __master_catalogue_xmm-lss_RANDOM10PCSAMPLE_20180221.fits (10% of the master catalogue)__ <br>
Number of rows: 870,476
<br>
Surveys included:<br>
| Survey | Telescope / Instrument  | Filters (detection band in bold)  | Location        |
|--------|-------------------------|:---------------------------------:|-----------------|
| CANDELS-3D-HST | HST  | F140W, F160W, F606W, F814W, F125W            | dmu0_CANDELS-3D-HST |
| CANDELS-UDS    | HST  | ACS_F606W, ACS_F814W, WFC3_F125W, WFC3_F140W, WFC3_F160W                                                 | dmu0_CANDELS-UDS    |
| CFHTLS         | CFHT / Megacam  | u*, g', r', i', z'                | dmu0_CFHTLS         |
| CFHTLenS       | CFHT / Megacam  | u, g, r, i, z                     | dmu0_CFHTLenS       |
| VIPERS-MLS     | CFHT / Megacam / WIRCam| u, g, r, i, z, Y, Ks       | dmu0_VIPERS-MLS     |
| SpARCS         | CFHT / Megacam  | u, g, r, y, z                     | dmu0_SpARCS         |
| DES            | Blanco/DECam     | grizy                        | dmu0_DES            |
| DECaLS         | DEC             | g, r, z                           | dmu0_DECaLS         |
| HSC-SSP        | Subaru / Suprime| g, r, i, z, y                     | dmu0_HSC            |
| SXDS           | Subaru / Suprime| B, V, r, i, z                     | dmu0_SXDS           |
| PanSTARRS-3SS  | GPC1            | g, r, i, z, y                     | dmu0_PanSTARRS1-3SS |
| CFHT-WIRDS     | CFHT / WIRCAM   | J, H, Ks                          | dmu0_CFHT-WIRDS     |
| UKIDSS-DXS     | UKIRT / WFCAM   | J, H, K                           | dmu0_UKIDSS-DXS_DR10plus |
| UKIDSS-UDS     | UKIRT / WFCAM   | J, H, K                           | dmu0_UKIDSS-UDS    |
| VHS            | VISTA / VIRCAM  | Y, J, H, Ks                       | dmu0_VISTA-VHS      |
| VIDEO          | VISTA / VIRCAM  | Z, Y, J, H, Ks                    | dmu0_VISTA-VIDEO-private |
| VIKING         | VISTA / VIRCAM  | Z, Y, J, H, Ks                    | dmu0_VISTA-VIKING   |
| SERVS          | Spitzer / IRAC  | IRAC1, IRAC2                      | dmu0_DataFusion-Spitzer |
| SWIRE          | Spitzer / IRAC  | IRAC1, IRAC2, IRAC3, IRAC4        | dmu0_DataFusion-Spitzer |

Master catalogue used: master_catalogue_xmm-lss_RANDOM10PCSAMPLE_20180221.fits (10% of the master catalogue)
Number of rows: 870,476
Surveys included:

Survey Telescope / Instrument Filters (detection band in bold) Location
CANDELS-3D-HST HST F140W, F160W, F606W, F814W, F125W dmu0_CANDELS-3D-HST
CANDELS-UDS HST ACS_F606W, ACS_F814W, WFC3_F125W, WFC3_F140W, WFC3_F160W dmu0_CANDELS-UDS
CFHTLS CFHT / Megacam u*, g', r', i', z' dmu0_CFHTLS
CFHTLenS CFHT / Megacam u, g, r, i, z dmu0_CFHTLenS
VIPERS-MLS CFHT / Megacam / WIRCam u, g, r, i, z, Y, Ks dmu0_VIPERS-MLS
SpARCS CFHT / Megacam u, g, r, y, z dmu0_SpARCS
DES Blanco/DECam grizy dmu0_DES
DECaLS DEC g, r, z dmu0_DECaLS
HSC-SSP Subaru / Suprime g, r, i, z, y dmu0_HSC
SXDS Subaru / Suprime B, V, r, i, z dmu0_SXDS
PanSTARRS-3SS GPC1 g, r, i, z, y dmu0_PanSTARRS1-3SS
CFHT-WIRDS CFHT / WIRCAM J, H, Ks dmu0_CFHT-WIRDS
UKIDSS-DXS UKIRT / WFCAM J, H, K dmu0_UKIDSS-DXS_DR10plus
UKIDSS-UDS UKIRT / WFCAM J, H, K dmu0_UKIDSS-UDS
VHS VISTA / VIRCAM Y, J, H, Ks dmu0_VISTA-VHS
VIDEO VISTA / VIRCAM Z, Y, J, H, Ks dmu0_VISTA-VIDEO-private
VIKING VISTA / VIRCAM Z, Y, J, H, Ks dmu0_VISTA-VIKING
SERVS Spitzer / IRAC IRAC1, IRAC2 dmu0_DataFusion-Spitzer
SWIRE Spitzer / IRAC IRAC1, IRAC2, IRAC3, IRAC4 dmu0_DataFusion-Spitzer
 
## I. Caveats

I. Caveats

 
### I.a. Magnitude errors 

I.a. Magnitude errors

At faint magnitudes (mag > 24), some surveys have very large errors (> 10) on the magnitude. These objects may be unreliable for science puposes.<br>
This in includes __DECaLS aperture and total__ magnitudes (at m >  23), __PanSTARRS aperture and total__ magnitudes (at mag > 23), __VISTA aperture and total__ magnitudes (at mag > 22) and __HSC-SSP aperture and total__ magnitudes (at mag > 25, with magnitude as faint as 60 and error ap to 10$^{13}$). <br>
Also, the __CFHT (Megacam and WIRCam) aperture and total__ magnitude errors (at mag > 26), and the __UKIDSS aperture and total__ magnitude errors (at mag > 26) can be very large too but for few sources. <br>
<img src="help_plots/XMM-LSS_magVSmagerr_DECam_g_mag_total.png" />

At faint magnitudes (mag > 24), some surveys have very large errors (> 10) on the magnitude. These objects may be unreliable for science puposes.
This in includes DECaLS aperture and total magnitudes (at m > 23), PanSTARRS aperture and total magnitudes (at mag > 23), VISTA aperture and total magnitudes (at mag > 22) and HSC-SSP aperture and total magnitudes (at mag > 25, with magnitude as faint as 60 and error ap to 1013).
Also, the CFHT (Megacam and WIRCam) aperture and total magnitude errors (at mag > 26), and the UKIDSS aperture and total magnitude errors (at mag > 26) can be very large too but for few sources.

 
### I.b. Aperture corrections

I.b. Aperture corrections

xxxxxxxxxx
 
In most of the case when comparing the aperture magnitudes between surveys, we observed a two peak distribution in the difference between the magnitudes ($\Delta_{mag} = mag_{survey1} - mag_{survey2}$). We have one peak around 0 for point-source objects, with a small spread. And a second peak at higher $\Delta_{mag}$ with a larger spread for extended objects; implying a different aperture correction between surveys for these objects.<br>
That means that galaxies will not have the same aperture magnitude in different surveys. <br>
In the griz bands, for bright sources, there is a two peaks distribution when comparing Pan-STARRS, DES, Megacam and Suprime aperture magnitues. Except when comparing Suprime with DES magnitudes, $\Delta_{mag}$ is similar for point-sources and extended objects.<br>
<img src="help_plots/XMM-LSS_apcorrIssues_DECam_z_aperture_-_GPC1_z_aperture.png" />

In most of the case when comparing the aperture magnitudes between surveys, we observed a two peak distribution in the difference between the magnitudes (Δmag=magsurvey1magsurvey2). We have one peak around 0 for point-source objects, with a small spread. And a second peak at higher Δmag with a larger spread for extended objects; implying a different aperture correction between surveys for these objects.
That means that galaxies will not have the same aperture magnitude in different surveys.

In the griz bands, for bright sources, there is a two peaks distribution when comparing Pan-STARRS, DES, Megacam and Suprime aperture magnitues. Except when comparing Suprime with DES magnitudes, Δmag is similar for point-sources and extended objects.

 
## II. Flags

II. Flags

 
### II.a. Pan-STARRS aperture and total magnitudes

II.a. Pan-STARRS aperture and total magnitudes

 
Few Pan-STARRS sources have exactly the same error (of <font color='blue'>0.0010860000038519502</font>) on the __aperture and total__ magnitudes in all the grizy bands. The corresponding aperture magnitude should not be trusted for these objects.<br>
<img src="help_plots/XMM-LSS_gpc1Issues_GPC1_g_mag_aperture.png" />

Few Pan-STARRS sources have exactly the same error (of 0.0010860000038519502) on the aperture and total magnitudes in all the grizy bands. The corresponding aperture magnitude should not be trusted for these objects.

xxxxxxxxxx
 
### II.c IRAC aperture magnitude
Few IRAC sources have exactly the same aperture magnitude (of <font color='blue'>3.9000000001085695</font>) in the IRAC1 IRAC2 band (for the subset). These magnitudes also have extremely small errors (around 10$^{-8}$-10$^{-9}$). The corresponding magnitudes should not be trusted. <br>
<img src="help_plots/XMM-LSS_iracIssues_i1_i2.png" />

II.c IRAC aperture magnitude

Few IRAC sources have exactly the same aperture magnitude (of 3.9000000001085695) in the IRAC1 IRAC2 band (for the subset). These magnitudes also have extremely small errors (around 108-109). The corresponding magnitudes should not be trusted.

 
### II.d. Outliers

II.d. Outliers

By comparing magnitude in the same band between different surveys, we can see that some magnitudes are significanlty different could not be trusted. <br>
The outliers are identified to have a large weighted magnitude difference (equivalent of the $chi^2$).
$$chi^2 = \frac{(mag_{1}-mag_{2})^2}{magerr_{1}^2 + magerr_{2}^2}$$ 
<br>
We used the 75th and 25th percentile to flagged the objects 5$\sigma$ away on the large values tail of the $chi^2$ ditribution. (__NB:__ bright sources tend to have their errors underestimated with values as low as $10^{-6}$, which is unrealistic. So to avoid high $chi^2$ due to unrealistic small errors, we clip the error to get a minimum value of 0.1% (i.e. all errors smaller then $10^{-3}$ are set to $10^{-3}$).)
<br><br>
$$outliers == [chi^2 >  (75th \;percentile + 3.2\times (75th \;percentile - 25th \;percentile))]$$
<img src="help_plots/XMM-LSS_outliers_DECam_g_aperture_-_GPC1_g_aperture.png" />

By comparing magnitude in the same band between different surveys, we can see that some magnitudes are significanlty different could not be trusted.
The outliers are identified to have a large weighted magnitude difference (equivalent of the chi2).

chi2=(mag1mag2)2magerr12+magerr22

We used the 75th and 25th percentile to flagged the objects 5σ away on the large values tail of the chi2 ditribution. (NB: bright sources tend to have their errors underestimated with values as low as 106, which is unrealistic. So to avoid high chi2 due to unrealistic small errors, we clip the error to get a minimum value of 0.1% (i.e. all errors smaller then 103 are set to 103).)

outliers==[chi2>(75thpercentile+3.2×(75thpercentile25thpercentile))]