# NGP: Validation Report (FULL)

NGP: Validation Report (FULL)¶

labl
xxxxxxxxxx
 
Master catalogue used: __master_catalogue_ngp_20171207.fits__ <br>Number of rows: 4,924,329<br>Surveys included:<br>​| Survey        | Telescope / Instrument  | Filters (detection band in bold)  | Location        ||---------------|-------------------------|:---------------------------------:|-----------------|| DECaLS        | DEC                     | z                                 | dmu0_DECaLS     || PanSTARRS-3SS | GPC1                    | grizy                             | dmu0_PanSTARRS-3SS |  | Legacy SUrvey | BASS                    | grz                               | dmu0_LegacySurvey || UKIDSS-LAS    | UKIRT WFCAM             | YJHK                              | dmu0_UKIDSS-LAS |  ​__NB: DECaLS g- and r-bands missing. Only z-band available__​

Master catalogue used: master_catalogue_ngp_20171207.fits
Number of rows: 4,924,329
Surveys included:

Survey	Telescope / Instrument	Filters (detection band in bold)	Location
DECaLS	DEC	z	dmu0_DECaLS
PanSTARRS-3SS	GPC1	grizy	dmu0_PanSTARRS-3SS
Legacy SUrvey	BASS	grz	dmu0_LegacySurvey
UKIDSS-LAS	UKIRT WFCAM	YJHK	dmu0_UKIDSS-LAS

NB: DECaLS g- and r-bands missing. Only z-band available

## I. Caveats

I. Caveats¶

### I.a. Magnitude errors

I.a. Magnitude errors¶

Herschel-Stripe-82_magVSmagerr_SDSS_g_mag_total.png
xxxxxxxxxx
 
At faint magnitudes (mag > 24), some surveys have very large errors (> 10) on the magnitude. These objects may be unreliable for science puposes.<br>This in includes __PanSTARRS aperture and total__ magnitudes (at mag > 23), __DECaLS aperture and total__ z-band magnitudes (at m > 26), __Legacy Survey aperture and total__ magnitudes (at mag > 26) and __UKIDSS aperture and total__ magnitudes (at mag > 22).<br>​<img src="help_plots/NGP_magVSmagerr_BASS_g_mag_total.png" />

At faint magnitudes (mag > 24), some surveys have very large errors (> 10) on the magnitude. These objects may be unreliable for science puposes.
This in includes PanSTARRS aperture and total magnitudes (at mag > 23), DECaLS aperture and total z-band magnitudes (at m > 26), Legacy Survey aperture and total magnitudes (at mag > 26) and UKIDSS aperture and total magnitudes (at mag > 22).

### I.b. Aperture corrections

I.b. Aperture corrections¶

xxxxxxxxxx
 
In most of the case when comparing the aperture magnitudes between surveys, we observed a two peak distribution in the difference between the magnitudes ($\Delta_{mag} = mag_{survey1} - mag_{survey2}$). We have one peak around 0 for point-source objects, with a small spread. And a second peak at higher $\Delta_{mag}$ with a larger spread for extended objects; implying a different aperture correction between surveys for these objects.<br>That means that galaxies will not have the same aperture magnitude in different surveys. <br>​In the g- and r-bands, for bright sources, there is a two peaks distribution when comparing Pan-STARRS and Legacy Survey aperture magnitues. Buit not in th z-band when comparing Pan-STARRS, DECaLS magnitudes and Legacy Survey, $\Delta_{mag}$ is similar for point-sources and extended objects.<br>Also, in the y-band, we get again a two peak distribution on $\Delta_{mag}$ whgen comparing UKIDSS and Pan-STARRS magnitudes.​<img src="help_plots/NGP_apcorrIssues_BASS_g_aperture_-_GPC1_g_aperture.png" />​

In most of the case when comparing the aperture magnitudes between surveys, we observed a two peak distribution in the difference between the magnitudes ( $\Delta_{mag} = mag_{survey1} - mag_{survey2}$ ). We have one peak around 0 for point-source objects, with a small spread. And a second peak at higher $\Delta_{mag}$ with a larger spread for extended objects; implying a different aperture correction between surveys for these objects.
That means that galaxies will not have the same aperture magnitude in different surveys.

In the g- and r-bands, for bright sources, there is a two peaks distribution when comparing Pan-STARRS and Legacy Survey aperture magnitues. Buit not in th z-band when comparing Pan-STARRS, DECaLS magnitudes and Legacy Survey, $\Delta_{mag}$ is similar for point-sources and extended objects.
Also, in the y-band, we get again a two peak distribution on $\Delta_{mag}$ whgen comparing UKIDSS and Pan-STARRS magnitudes.

## II. Flags

II. Flags¶

### II.a. Pan-STARRS aperture and total magnitudes

II.a. Pan-STARRS aperture and total magnitudes¶

Herschel-Stripe-82_gpc1Issues_GPC1_r_mag_aperture.png
xxxxxxxxxx
 
Few Pan-STARRS sources have exactly the same error (of <font color='blue'>0.0010860000038519502</font>) on the __aperture and total__ magnitudes in all the grizy bands. The corresponding aperture magnitude should not be trusted for these objects.<br>​<img src="help_plots/NGP_gpc1Issues_GPC1_g_mag_aperture.png" />

Few Pan-STARRS sources have exactly the same error (of 0.0010860000038519502) on the aperture and total magnitudes in all the grizy bands. The corresponding aperture magnitude should not be trusted for these objects.

### II.b.DECaLS total magnitudes

II.b.DECaLS total magnitudes¶

Herschel-Stripe-82_decamIssues_DECam_z_mag_total.png
xxxxxxxxxx
 
Few DECaLS sources have excatly the same total magnitude (of <font color='blue'>14.999935</font>) in the g- and r-bands. These magnitudes should not be trusted. <br>Also, in the grz bands, few sources have very bright or negative total magnitude but no aperture magnitude (<font color='green'>$mag_{tot} < 7$</font>).​<img src="help_plots/NGP_decamIssues_DECam_z_mag_total.png" />

Few DECaLS sources have excatly the same total magnitude (of 14.999935) in the g- and r-bands. These magnitudes should not be trusted.
Also, in the grz bands, few sources have very bright or negative total magnitude but no aperture magnitude ( $mag_{tot} < 7$ ).

### II.c. Outliers

II.c. Outliers¶

Herschel-Stripe-82_outliers_GPC1_r_aperture_-_DECam_r_aperture.png
xxxxxxxxxx
 
By comparing magnitude in the same band between different surveys, we can see that some magnitudes are significanlty different could not be trusted. <br>The outliers are identified to have a large weighted magnitude difference (equivalent of the $chi^2$).​$$chi^2 = \frac{(mag_{1}-mag_{2})^2}{magerr_{1}^2 + magerr_{2}^2}$$ <br>We used the 75th and 25th percentile to flagged the objects 5$\sigma$ away on the large values tail of the $chi^2$ ditribution. (__NB:__ bright sources tend to have their errors underestimated with values as low as $10^{-6}$, which is unrealistic. So to avoid high $chi^2$ due to unrealistic small errors, we clip the error to get a minimum value of 0.1% (i.e. all errors smaller then $10^{-3}$ are set to $10^{-3}$).)<br><br>$$outliers == [chi^2 >  (75th \;percentile + 3.2\times (75th \;percentile - 25th \;percentile))]$$​<img src="help_plots/NGP_outliers_BASS_g_aperture_-_GPC1_g_aperture.png" />

By comparing magnitude in the same band between different surveys, we can see that some magnitudes are significanlty different could not be trusted.
The outliers are identified to have a large weighted magnitude difference (equivalent of the $chi^2$ ).

c h i^{2} = \frac{(m a g_{1} - m a g_{2})^{2}}{m a g e r r_{1}^{2} + m a g e r r_{2}^{2}}

$chi^2 = \frac{(mag_{1}-mag_{2})^2}{magerr_{1}^2 + magerr_{2}^2}$
We used the 75th and 25th percentile to flagged the objects 5

σ

$\sigma$ away on the large values tail of the

c h i^{2}

$chi^2$ ditribution. (NB: bright sources tend to have their errors underestimated with values as low as

10^{- 6}

$10^{-6}$ , which is unrealistic. So to avoid high

c h i^{2}

$chi^2$ due to unrealistic small errors, we clip the error to get a minimum value of 0.1% (i.e. all errors smaller then

10^{- 3}

$10^{-3}$ are set to

10^{- 3}

$10^{-3}$ ).)

o u t l i e r s == [c h i^{2} > (75 t h p e r c e n t i l e + 3.2 \times (75 t h p e r c e n t i l e - 25 t h p e r c e n t i l e))]

$outliers == [chi^2 > (75th \;percentile + 3.2\times (75th \;percentile - 25th \;percentile))]$

In [ ]: