This topic is a placeholder for draft documentation on the origin and pre-processing of the atmosphere ancillary files that are listed in the Payu configurations of the ACCESS-ESM1.5 release.
From Martin Dix: An old document explaining how the ESM1.5 CMIP6 solar and volcanic forcing was set up, CMIP6_solar_volcanic.pdf (On ACCESS-NRI Sharepoint, so you will likely need to login to view it.)
This item documents the provenance of input files in atmosphere portion of the Payu configuration for the pre-industrial configuration of ACCESS-ESM1.5. Start dump is excluded.
At access-esm1.5-configs/config.yaml at release-preindustrial+concentrations · ACCESS-NRI/access-esm1.5-configs · GitHub on 2024-10-17
# Aerosols
- /g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/aerosol/global.N96/2020.05.19/OCFF_1850_ESM1.anc
- /g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/aerosol/global.N96/2020.05.19/BC_hi_1850_ESM1.anc
- /g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/aerosol/global.N96/2020.05.19/scycl_1850_ESM1_v4.anc
- /g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/aerosol/global.N96/2020.05.19/Bio_1850_ESM1.anc
- /g/data/vk83/configurations/inputs/access-esm1p5/modern/share/atmosphere/aerosol/global.N96/2020.05.19/biogenic_351sm.N96L38
- /g/data/vk83/configurations/inputs/access-esm1p5/modern/share/atmosphere/aerosol/global.N96/2020.05.19/sulpc_oxidants_N96_L38
- /g/data/vk83/configurations/inputs/access-esm1p5/modern/share/atmosphere/aerosol/global.N96/2020.05.19/DMS_conc.N96
# Forcing
- /g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/forcing/global.N96/2020.05.19/ozone_1850_ESM1.anc
- /g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/forcing/resolution_independent/2020.05.19/volcts_18502000ave.dat
# Land
- /g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/land/biogeochemistry/global.N96/2020.05.19/Ndep_1850_ESM1.anc
- /g/data/vk83/configurations/inputs/access-esm1p5/modern/share/atmosphere/land/soiltype/global.N96/2020.05.19/qrparm.soil_igbp_vg
- /g/data/vk83/configurations/inputs/access-esm1p5/modern/share/atmosphere/land/vegetation/global.N96/2020.05.19/cable_vegfunc_N96.anc
- /g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/land/biogeochemistry/resolution_independent/2020.05.19/modis_phenology_csiro.txt
- /g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/land/biogeochemistry/resolution_independent/2020.05.19/pftlookup_csiro_v16_17tiles_wtlnds.csv
- /g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/land/biogeophysics/resolution_independent/2020.05.19/def_soil_params.txt
- /g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/land/biogeophysics/resolution_independent/2020.05.19/def_veg_params.txt
# Spectral
- /g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/spectral/resolution_independent/2020.05.19/spec3a_sw_hadgem1_6on
- /g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/spectral/resolution_independent/2020.05.19/spec3a_lw_hadgem1_6on
# Grids
- /g/data/vk83/configurations/inputs/access-esm1p5/modern/share/atmosphere/grids/global.N96/2020.05.19/qrparm.mask
- /g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/grids/resolution_independent/2020.05.19/vertlevs_G3
# STASH
- /g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/stash/2020.05.19/
At access-esm/config.yaml at pre-industrial · coecms/access-esm · GitHub
/g/data/access/payu/access-esm/input/pre-industrial/atmosphere
At access-esm/config.yaml at 45c76c4ee46a4a9ab68ae1f27233b1714366e0d2 · coecms/access-esm · GitHub 2020-05-20
/scratch/p66/hxw599/packed_up/input/atmosphere
At Blaming access-esm/config.yaml at ef7914a40d5604ba4f40671c944da2d42ee001f5 · coecms/access-esm · GitHub 2020-05-20
/scratch/p66/hxw599/access-esm/input/atmosphere-2
At Blaming access-esm/config.yaml at 39daf2e4f468b85c8ec627bc77d423ad4ec2de10 · coecms/access-esm · GitHub 2019-12-04
/scratch/p66/hxw599/access-esm/input/atmosphere
At Blaming access-esm/config.yaml at 2fad536b437e34d5e63170d10b9483ca3f7855b6 · coecms/access-esm · GitHub 2019-02-14
/short/public/access-esm/payu/input/pre-industrial/atmosphere
At Blaming access-esm/config.yaml at fb52e216bd3f8cdc2c2aaf4aee9eaa5a803c53b9 · coecms/access-esm · GitHub 2019-02-11
/short/w35/saw562/scratch/access-esm/access-esm1.5-cmip6/payu/input/pre-industrial/atm
Known earliest copies of atomosphere input files on the NCI Gadi file system, found by @spencerwong. These files have identical contents as checked by sha256sum.
Aerosols
/g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/aerosol/global.N96/2020.05.19/OCFF_1850_ESM1.anc
/g/data/p66/txz599/data/ancil/CMIP6/OCFF_1850_ESM1.anc
Examining the history field of the equivalent NetCDF file:
-rw-r--r-- 1 txz599 p66 5.4M Dec 19 2018 OCFF_1850_ESM1.anc
-rw-r--r-- 1 txz599 p66 2.6M Dec 19 2018 OCFF_1850_ESM1.nc
:history = "Mon Jul 16 17:26:09 2018: ncks -d time,12,23 OCFF_1849_2015_ESM1.nc OCFF_1850_ESM1.nc" ;
/g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/aerosol/global.N96/2020.05.19/BC_hi_1850_ESM1.anc
/g/data/p66/txz599/data/ancil/CMIP6/BC_hi_1850_ESM1.anc
Examining the history field of the equivalent NetCDF file:
-rw-r--r-- 1 txz599 p66 5.4M Dec 19 2018 BC_hi_1850_ESM1.anc
-rw-r--r-- 1 txz599 p66 2.6M Dec 19 2018 BC_hi_1850_ESM1.nc
:history = "Mon Jul 16 17:25:45 2018: ncks -d time,12,23 BC_hi_1849_2015_ESM1.nc BC_hi_1850_ESM1.nc" ;
/g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/aerosol/global.N96/2020.05.19/scycl_1850_ESM1_v4.anc
/g/data/p66/txz599/data/ancil/CMIP6/scycl_1850_ESM1_v4.anc
Examining the history field of the equivalent NetCDF file and previous versions:
-rw-r--r-- 1 txz599 p66 9.3M Dec 19 2018 scycl_1850_ESM1.anc
-rw-r--r-- 1 txz599 p66 11M Dec 19 2018 scycl_1850_ESM1.nc
:history = "Mon Jul 16 17:26:51 2018: ncks -d time,12,23 scycl_1849_2015_ESM1.nc scycl_1850_ESM1.nc" ;
-rw-r--r-- 1 txz599 p66 9.3M Dec 19 2018 scycl_1850_ESM1_v2.anc
-rw-r--r-- 1 txz599 p66 5.2M Dec 19 2018 scycl_1850_ESM1_v2.nc
:history = "Fri Nov 2 17:05:15 AEDT 2018 - XCONV V1.92 16-February-2006" ;
-rw-r----- 1 txz599 p66 9.3M Dec 19 2018 scycl_1850_ESM1_v3.anc
-rw-r--r-- 1 txz599 p66 9.3M Dec 19 2018 scycl_1850_ESM1_v4.anc
-rw-r--r-- 1 txz599 p66 5.2M Dec 19 2018 scycl_1850_ESM1_v4.nc
/g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/aerosol/global.N96/2020.05.19/Bio_1850_ESM1.anc
/g/data/p66/txz599/data/ancil/CMIP6/Bio_1850_ESM1.anc
Examining the history field of the equivalent NetCDF file:
-rw-r--r-- 1 txz599 p66 6.7M Dec 19 2018 Bio_1850_ESM1.anc
-rw-r--r-- 1 txz599 p66 5.2M Dec 19 2018 Bio_1850_ESM1.nc
:history = "Mon Jul 16 17:26:32 2018: ncks -d time,12,23 Bio_1849_2015_ESM1.nc Bio_1850_ESM1.nc" ;
/g/data/vk83/configurations/inputs/access-esm1p5/modern/share/atmosphere/aerosol/global.N96/2020.05.19/biogenic_351sm.N96L38
/g/data/access/projects/access/data/ancil/HadGEM3_cal365/biogenic_351sm.N96L38
/g/data/vk83/configurations/inputs/access-esm1p5/modern/share/atmosphere/aerosol/global.N96/2020.05.19/sulpc_oxidants_N96_L38
/g/data/access/projects/access/data/ancil/HadGEM3_cal365/sulpc_oxidants_N96_L38
/g/data/vk83/configurations/inputs/access-esm1p5/modern/share/atmosphere/aerosol/global.N96/2020.05.19/DMS_conc.N96
/g/data/access/projects/access/data/ancil/HadGEM3_cal365/DMS_conc.N96
Forcing
/g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/forcing/global.N96/2020.05.19/ozone_1850_ESM1.anc
/g/data/p66/txz599/data/ancil/CMIP6/ozone_1850_ESM1.anc
Examining versions:
-rw-r--r-- 1 txz599 p66 5.8M Dec 19 2018 ozone_1850_ESM1.anc
-rw-r--r-- 1 txz599 p66 5.8M Dec 19 2018 ozone_1850_ESM1_v2.anc
/g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/forcing/resolution_independent/2020.05.19/volcts_18502000ave.dat
/g/data/access/projects/access/data/ancil/CMIP5/volcts_18502000ave.dat
Land
/g/data/vk83/configurations/inputs/access-esm1p5/modern/pre-industrial/atmosphere/land/biogeochemistry/global.N96/2020.05.19/Ndep_1850_ESM1.anc
/g/data/p66/txz599/data/ancil/CMIP6/Ndep_1850_ESM1.anc
Examining the history field of the equivalent NetCDF file:
-rwxr-xr-x 1 txz599 p66 5.4M Dec 19 2018 Ndep_1850_ESM1.anc
-rw-r--r-- 1 txz599 p66 1.3M Feb 8 2019 Ndep_1850_ESM1.nc
:history = "Fri Feb 8 15:04:33 AEDT 2019 - XCONV V1.92 16-February-2006" ;
This appears to show that the file Ndep_1850_ESM1.nc was created using xconv after the creation of Ndep_1850_ESM1.anc.
/g/data/vk83/configurations/inputs/access-esm1p5/modern/share/atmosphere/land/soiltype/global.N96/2020.05.19/qrparm.soil_igbp_vg
/g/data/access/payu/access-esm/input/pre-industrial/atmosphere/qrparm.soil_igbp_vg
/g/data/vk83/configurations/inputs/access-esm1p5/modern/share/atmosphere/land/vegetation/global.N96/2020.05.19/cable_vegfunc_N96.anc
/g/data/access/projects/access/data/ancil/access_v2/cable_vegfunc_N96.anc
/g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/land/biogeochemistry/resolution_independent/2020.05.19/modis_phenology_csiro.txt
/g/data/access/projects/access/data/ACCESS_CMIP5/CABLE-AUX-1.4/core/biogeochem/modis_phenology_csiro.txt
/g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/land/biogeochemistry/resolution_independent/2020.05.19/pftlookup_csiro_v16_17tiles_wtlnds.csv
/g/data/access/projects/access/data/ACCESS_CMIP5/CABLE-AUX-1.4/core/biogeochem/pftlookup_csiro_v16_17tiles_wtlnds.csv
/g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/land/biogeophysics/resolution_independent/2020.05.19/def_soil_params.txt
/g/data/access/projects/access/data/ACCESS_CMIP5/CABLE-AUX-1.4/core/biogeophys/def_soil_params.txt
/g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/land/biogeophysics/resolution_independent/2020.05.19/def_veg_params.txt
/g/data/access/projects/access/data/ACCESS_CMIP5/CABLE-AUX-1.4/core/biogeophys/def_veg_params.txt
Spectral
/g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/spectral/resolution_independent/2020.05.19/spec3a_sw_hadgem1_6on
/g/data/access/projects/access/data/ancil/HadGEM3_cal365/spec3a_sw_hadgem1_6on
/g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/spectral/resolution_independent/2020.05.19/spec3a_lw_hadgem1_6on
/g/data/access/projects/access/data/ancil/HadGEM3_cal365/spec3a_lw_hadgem1_6on
Grids
/g/data/vk83/configurations/inputs/access-esm1p5/modern/share/atmosphere/grids/global.N96/2020.05.19/qrparm.mask
/g/data/access/projects/access/data/ancil/access_v2/qrparm.mask
/g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/grids/resolution_independent/2020.05.19/vertlevs_G3
/g/data/access/projects/access/umdir/vn7.3/ctldata/vert/vertlevs_G3
STASH
/g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/stash/2020.05.19/stasets
/g/data/access/projects/access/umdir/vn7.3/ctldata/stasets
/g/data/vk83/configurations/inputs/access-esm1p5/share/atmosphere/stash/2020.05.19/STASHmaster
/g/data/access/projects/access/umdir/vn7.3/ctldata/STASHmaster
The ncks commands in history reveal that at least some of the *_1850_ESM1*.anc files were originally obtained by slicing the equivalent *_1849_2015_*.nc files by time to obtain 1850 data from these files.
See also /g/data/access/payu/access-esm/input/pre-industrial/atmosphere/SOURCES
For ACCESS-ESM1.5 CMIP5 inputs, the Subversion branch file:///g/data/access/access-svn/cmip5/branches/dev/saw562/a10_piControl dated 2017-11-07 contains the script a10_piControl, which contains the definitions
export UMDIR=/projects/access/data/ACCESS_CMIP5/umdir
export VN=7.3
export UM_SPECTRAL=$UMDIR/vn$VN/ctldata/spectral
export UM_ANCIL_A=$UMDIR/vn$VN/ancil/atmos/n96
export CENTRAL=$UMDIR/vn$VN
export ANCIL_ATMOS=/projects/access/data/ancil/HadGEM3_cal365
export CMIP5ANCIL=/projects/access/data/ancil/CMIP5
export ANCIL_LAND=/projects/access/data/ancil/access_v2
export HGPKG2=$CENTRAL/HGPKG2-365
export STASHMSTR=$UMDIR/vn$VN/ctldata/STASHmaster
export ANCILMSTR=$UMDIR/vn$VN/ctldata/ANCILmaster
The directories
/projects/access/data/ACCESS_CMIP5/umdir/vn7.3/ancil
/projects/access/data/ACCESS_CMIP5/umdir/vn7.3/HGPKG2-365
no longer exist.
The history fields in the equivalent *_1849_2015_ESM1*.nc files reveal some of the processing used to create these files:
$ for f in *_1849_*_ESM1*.nc; do echo; g=${f/.nc/.anc}; ls -lh $f $g; ncdump -ht $f | grep history; done 2>&1|more
-rw-r--r-- 1 txz599 p66 224M Jan 15 2019 BC_hi_1849_2015_ESM1.anc
-rw-r--r-- 1 txz599 p66 213M May 30 2019 BC_hi_1849_2015_ESM1.nc
:history = "Thu May 30 14:41:38 AEST 2019 - XCONV V1.92 16-February-2006" ;
-rw-r--r-- 1 txz599 p66 443M Jan 15 2019 Bio_1849_2015_ESM1.anc
-rw-r--r-- 1 txz599 p66 852M Dec 19 2018 Bio_1849_2015_ESM1.nc
-rw-r--r-- 1 txz599 p66 224M Jan 15 2019 OCFF_1849_2015_ESM1.anc
-rw-r--r-- 1 txz599 p66 426M Dec 19 2018 OCFF_1849_2015_ESM1.nc
-rw-r--r-- 1 txz599 p66 338M Jan 16 2019 ozone_1849_2015_ESM1.anc
-rw-r----- 1 txz599 p66 85M Jan 15 2019 ozone_1849_2015_ESM1.nc
o3:history = "From O3_CMIP6_1849-2015" ;
:history = "Thu Nov 8 12:30:17 2018: ncrename -v ozone_zonal,o3 ozone_hist.2d.nc\n",
ls: cannot access 'scycl_1849_2015_ESM1.anc': No such file or directory
-rw-r--r-- 1 txz599 p66 1.7G Dec 19 2018 scycl_1849_2015_ESM1.nc
-rw-r--r-- 1 txz599 p66 881M Jan 16 2019 scycl_1849_2015_ESM1_v4.anc
-rw-r--r-- 1 txz599 p66 1.7G Jan 15 2019 scycl_1849_2015_ESM1_v4.nc
$ ls -hl ozone_hist.2d.nc; ncdump -ht ozone_hist.2d.nc | grep history
-rw-r----- 1 txz599 p66 85M Jan 14 2019 ozone_hist.2d.nc
o3:history = "From O3_CMIP6_1849-2015" ;
:history = "Thu Nov 8 12:30:17 2018: ncrename -v ozone_zonal,o3 ozone_hist.2d.nc\n",
The file O3_CMIP6_1849-2015* is not in /g/data/p66/txz599/data/ancil/CMIP6/.
The configuration for the CMIP6 pre-industrial control for ACCESS-CM2 is documented at PI control configuration · Issue #330 · ACCESS-NRI/accessdev-Trac-archive · GitHub
Issue 330 includes the comments:
"Met Office PI control experiments are described in https://code.metoffice.gov.uk/trac/ukcmip6/ticket/8 with details in https://code.metoffice.gov.uk/trac/ukcmip6/wiki/ticket/8/SuiteSetup.
Standard suite is u-am927."
"The suite has
ukca_em_files='/data/d01/tkuhlbro/ancils/ukca_emiss/ukca_emiss_BC_biofuel_1850.nc',
='/data/d01/tkuhlbro/ancils/ukca_emiss/ukca_emiss_BC_fossil_1850.nc',
='/data/d00/hadtr/ancil/ukca_emiss/cmip5/1860/v2/ukca_emiss_DMS.nc',
='/data/d00/hadtr/ancil/ukca_emiss/cmip5/1860/v2/ukca_emiss_Monoterp.nc',
='/data/d01/tkuhlbro/ancils/ukca_emiss/ukca_emiss_OC_biofuel_1850.nc',
='/data/d01/tkuhlbro/ancils/ukca_emiss/ukca_emiss_OC_fossil_1850.nc',
='/data/d01/tkuhlbro/ancils/ukca_emiss/ukca_emiss_SO2_high_1850.nc',
='/data/d01/tkuhlbro/ancils/ukca_emiss/ukca_emiss_SO2_low_1850.nc',
='/data/d00/hadtr/ancil/ukca_emiss/andres_kasgnoc/v2/ukca_emiss_SO2_nat_r2428.nc',
='/data/d01/tkuhlbro/ancils/ukca_emiss/ukca_emiss_BC_biomass_high_1850.nc',
='/data/d01/tkuhlbro/ancils/ukca_emiss/ukca_emiss_BC_biomass_low_1850.nc',
='/data/d01/tkuhlbro/ancils/ukca_emiss/ukca_emiss_OC_biomass_high_1850.nc',
='/data/d01/tkuhlbro/ancils/ukca_emiss/ukca_emiss_OC_biomass_low_1850.nc'
There are files of the same name on JASMIN in directory CMIP6_ANCILS/users/bosley/ancils_N96e. For now this directory has also been copied to /g/data/access/TIDS/CMIP6_ANCILS.
TODO
- Check that the Bosley files are the ones actually being used
- Add netcdf attribute so they work with any calendar"