DP1 — shear & photo-z data products

This directory bundles the shape catalogs, photo-z inputs, spectroscopic calibration data, end-to-end simulations, and the example notebooks used to reproduce the DP1 weak-lensing / cluster-lensing analyses. The sub-directories below are organised by data type rather than by analysis step.

Directory layout

DirectoryContents
catalogs/ AnaCal shape catalogs on real DP1 data (FPFS moments + derivatives, multi-band fluxes, selection weights), one FITS per field, plus the spec-matched golden sample.
catalogs_sim/ Simulated AnaCal catalogs (same schema as catalogs/) used to validate the shape-measurement pipeline and to cross-check 2-D distributions against the real data (cf. notebook 1.3).
rail_data/ Photo-z inputs / outputs in tables_io / RAIL format: per-source feature tables, cached point estimates and per-source PDFs for ECDFS, EDFS, and Abell 360, plus the training / test split and the helper scripts that built them.
specz/ Spectroscopic redshifts used to calibrate / validate photo-z: the ECDFS spec-z compilation plus a small helper that sky-matches it to the AnaCal catalog and applies the golden-sample selection (mag + trace + confidence cuts).
notebooks/ Example notebooks demonstrating end-to-end analyses built on the directories above. See the dedicated notebook index for the full list and rendered HTML output.

Notebook index

The notebooks/ directory holds the rendered analyses, grouped by topic:

NotebookTopic
1_0_ecdfs_specz_sample.ipynb Build the ECDFS spec-z × AnaCal golden sample.
1_1_ecdfs_photoz_6bands.ipynb 6-band (ugrizy) photo-z evaluation on ECDFS.
1_2_ecdfs_photoz_4bands.ipynb 4-band (griz) photo-z evaluation on ECDFS.
1_3_ecdfs_sim_histogram.ipynb 2-D distribution comparison: real vs. simulated catalogs.
2_1_edfs_photoz_estimate.ipynb EDFS photo-z estimation (cached source PDFs for clusters).
2_2_edfs_erass_cluster1.ipynb EDFS-eRASS cluster 1: mass fit + aperture-mass S/N map.
2_3_edfs_erass_cluster2.ipynb EDFS-eRASS cluster 2: shear profile + aperture-mass S/N map.
3_1_a360_photoz_estimate.ipynb Abell 360 photo-z estimation.
3_2_abell360_cluster.ipynb Abell 360 shear profile vs. NFW.

Software environment

The notebooks are run under a single conda environment (dp1). To set up an equivalent environment from scratch:

conda create -n dp1 -c conda-forge --override-channels python=3.13
conda activate dp1

# Core scientific stack
conda install -c conda-forge \
    numpy scipy matplotlib astropy emcee \
    pandas pyarrow

# I/O helpers
conda install -c conda-forge \
    fitsio tables_io "qp-prob>=1.0.1" healsparse

# Cluster lensing
conda install -c conda-forge clmm pyccl

# Photo-z (RAIL stack — note the pz-rail-* prefix on conda-forge)
conda install -c conda-forge \
    pz-rail-base pz-rail-bpz pz-rail-flexzboost

# Jupyter / rendering
conda install -c conda-forge \
    jupyter notebook nbconvert ipykernel
python -m ipykernel install --user --name dp1 --display-name "Python-dp1"

See rail-hub.readthedocs.io for the full list of optional RAIL extensions and pip-only extras.

After conda is done, install xlens from source:

git clone https://github.com/mr-superonion/xlens.git
cd xlens
pip install -e .

The notebooks ship with a small importable notebooks/utils/ package that wraps the AnaCal shape combination, tangential-shear binning, aperture-mass map, and the spec-z sky-match helpers (the last is a re-export of specz/match.py). It is pure Python — no extra packages beyond the list above are needed.

Contact

Maintained by Xiangchong Li.