ELAIS-N2 master catalogue¶
Preparation of Spitzer datafusion SWIRE data¶
The Spitzer catalogues were produced by the datafusion team are available in dmu0_DataFusion-Spitzer.
Lucia told that the magnitudes are aperture corrected.
In the catalouge, we keep:
We keep:
- The internal identifier (this one is only in HeDaM data);
- The position;
- The fluxes in aperture 2 (1.9 arcsec) for IRAC bands.
- The Kron flux;
- The stellarity in each band
A query of the position in the Spitzer heritage archive show that the ELAIS-N1 images were observed in 2004. Let's take this as epoch. Is ELAIS N2 different?
We do not use the MIPS fluxes as they will be extracted on MIPS maps using XID+.
In [1]:
from herschelhelp_internal import git_version
print("This notebook was run with herschelhelp_internal version: \n{}".format(git_version()))
In [2]:
%matplotlib inline
#%config InlineBackend.figure_format = 'svg'
import matplotlib.pyplot as plt
plt.rc('figure', figsize=(10, 6))
from collections import OrderedDict
import os
from astropy import units as u
from astropy.coordinates import SkyCoord
from astropy.table import Column, Table
import numpy as np
from herschelhelp_internal.flagging import gaia_flag_column
from herschelhelp_internal.masterlist import nb_astcor_diag_plot, remove_duplicates
from herschelhelp_internal.utils import astrometric_correction, flux_to_mag
In [3]:
OUT_DIR = os.environ.get('TMP_DIR', "./data_tmp")
try:
os.makedirs(OUT_DIR)
except FileExistsError:
pass
RA_COL = "swire_ra"
DEC_COL = "swire_dec"
I - Column selection¶
In [4]:
imported_columns = OrderedDict({
'internal_id': "swire_intid",
'ra_spitzer': "swire_ra",
'dec_spitzer': "swire_dec",
'flux_ap2_36': "f_ap_irac_i1",
'uncf_ap2_36': "ferr_ap_irac_i1",
'flux_kr_36': "f_irac_i1",
'uncf_kr_36': "ferr_irac_i1",
'stell_36': "swire_stellarity",
'flux_ap2_45': "f_ap_irac_i2",
'uncf_ap2_45': "ferr_ap_irac_i2",
'flux_kr_45': "f_irac_i2",
'uncf_kr_45': "ferr_irac_i2",
#'stell_45': "swire_stellarity_irac2",
'flux_ap2_58': "f_ap_irac_i3",
'uncf_ap2_58': "ferr_ap_irac_i3",
'flux_kr_58': "f_irac_i3",
'uncf_kr_58': "ferr_irac_i3",
#'stell_58': "swire_stellarity_irac3",
'flux_ap2_80': "f_ap_irac_i4",
'uncf_ap2_80': "ferr_ap_irac_i4",
'flux_kr_80': "f_irac_i4",
'uncf_kr_80': "ferr_irac_i4",
#'stell_80': "swire_stellarity_irac4",
})
catalogue = Table.read("../../dmu0/dmu0_DataFusion-Spitzer/data/DF-SWIRE_ELAIS-N2.fits")[list(imported_columns)]
for column in imported_columns:
catalogue[column].name = imported_columns[column]
epoch = 2004
# Clean table metadata
catalogue.meta = None
In [5]:
# Adding magnitude and band-flag columns
for col in catalogue.colnames:
if col.startswith('f_'):
errcol = "ferr{}".format(col[1:])
magnitude, error = flux_to_mag(
np.array(catalogue[col])/1.e6, np.array(catalogue[errcol])/1.e6)
# Note that some fluxes are 0.
catalogue.add_column(Column(magnitude, name="m{}".format(col[1:])))
catalogue.add_column(Column(error, name="m{}".format(errcol[1:])))
# Band-flag column
if "ap" not in col:
catalogue.add_column(Column(np.zeros(len(catalogue), dtype=bool), name="flag{}".format(col[1:])))
In [6]:
catalogue[:10].show_in_notebook()
Out[6]:
II - Removal of duplicated sources¶
We remove duplicated objects from the input catalogues.
In [7]:
SORT_COLS = ['ferr_ap_irac_i1', 'ferr_ap_irac_i2', 'ferr_ap_irac_i3', 'ferr_ap_irac_i4']
FLAG_NAME = "swire_flag_cleaned"
nb_orig_sources = len(catalogue)
catalogue = remove_duplicates(catalogue, RA_COL, DEC_COL, sort_col=SORT_COLS,flag_name=FLAG_NAME)
nb_sources = len(catalogue)
print("The initial catalogue had {} sources.".format(nb_orig_sources))
print("The cleaned catalogue has {} sources ({} removed).".format(nb_sources, nb_orig_sources - nb_sources))
print("The cleaned catalogue has {} sources flagged as having been cleaned".format(np.sum(catalogue[FLAG_NAME])))
III - Astrometry correction¶
We match the astrometry to the Gaia one. We limit the Gaia catalogue to sources with a g band flux between the 30th and the 70th percentile. Some quick tests show that this give the lower dispersion in the results.
In [8]:
gaia = Table.read("../../dmu0/dmu0_GAIA/data/GAIA_ELAIS-N2.fits")
gaia_coords = SkyCoord(gaia['ra'], gaia['dec'])
In [9]:
nb_astcor_diag_plot(catalogue[RA_COL], catalogue[DEC_COL],
gaia_coords.ra, gaia_coords.dec)
In [10]:
delta_ra, delta_dec = astrometric_correction(
SkyCoord(catalogue[RA_COL], catalogue[DEC_COL]),
gaia_coords
)
print("RA correction: {}".format(delta_ra))
print("Dec correction: {}".format(delta_dec))
In [11]:
catalogue[RA_COL] += delta_ra.to(u.deg)
catalogue[DEC_COL] += delta_dec.to(u.deg)
In [12]:
nb_astcor_diag_plot(catalogue[RA_COL], catalogue[DEC_COL],
gaia_coords.ra, gaia_coords.dec)
IV - Flagging Gaia objects¶
In [13]:
catalogue.add_column(
gaia_flag_column(SkyCoord(catalogue[RA_COL], catalogue[DEC_COL]), epoch, gaia)
)
In [14]:
GAIA_FLAG_NAME = "swire_flag_gaia"
catalogue['flag_gaia'].name = GAIA_FLAG_NAME
print("{} sources flagged.".format(np.sum(catalogue[GAIA_FLAG_NAME] > 0)))
V - Flagging objects near bright stars¶
VI - Saving to disk¶
In [15]:
catalogue.write("{}/SWIRE.fits".format(OUT_DIR), overwrite=True)