Hello
I am getting an error in making a datastore for processed ACESS-OM2 files. These are density-binned files procesed offline from the 01deg_jra55v140_iaf_cycle3_antarctic_tracers run. The error appears related to the fact that the files are not identified to have any standard intake catalog “realm”, presumably due to the offline density binning. Code and error message detalis are below. Anyone has an idea of how to deal with this?
Thanks,
Aviv
p.s.1 the 01deg_jra55v140_iaf_cycle3_antarctic_tracers run is in the process of integrated into the intake catalog.
p.s.2 the datastore making script
from access_nri_intake.source import builders
from access_nri_intake.experiment import use_datastore
FoldSigBinned = ‘/g/data/g40/access-om2/archive/01deg_jra55v140_iaf_cycle3_antarctic_tracers_isopyc/’
FoldDBs = ‘/scratch/v45/as2408/access-om2/01deg_jra55v140_iaf_cycle3_antarctic_tracers/DBs/’
BUILDER = builders.AccessOm2Builder
CATALOG_DIR = FoldDBs
DATASTORE_NAME = “Exp9tracersSigAvgs”
esm_ds = use_datastore(
builder=BUILDER,
experiment_dir=FoldSigBinned,
catalog_dir=CATALOG_DIR,
datastore_name=DATASTORE_NAME
)
esm_ds
p.s.3 The error message:
---------------------------------------------------------------------------
ParserError Traceback (most recent call last)
Cell In[112], line 5
2 CATALOG_DIR = FoldDBs# Path("~/catalog_dir").expanduser() # We'll save our datastore in a directory called catalog_dir in our home dir
3 DATASTORE_NAME = "Exp9tracersSigAvgs"
----> 5 esm_ds = use_datastore(
6 builder=BUILDER,
7 experiment_dir=FoldSigBinned,
8 catalog_dir=CATALOG_DIR,
9 datastore_name=DATASTORE_NAME
10 )
12 esm_ds
File /g/data/xp65/public/apps/med_conda/envs/analysis3-26.01/lib/python3.11/site-packages/access_nri_intake/experiment/main.py:122, in use_datastore(experiment_dir, builder, catalog_dir, builder_kwargs, open_ds, datastore_name, description)
120 builder_instance: Builder = builder(path=str(experiment_dir), **builder_kwargs)
121 print(f"{f_info}Building esm-datastore...{f_reset}")
--> 122 builder_instance.get_assets().build()
123 print(f"{f_success}Sucessfully built esm-datastore!{f_reset}")
124 print(
125 f"{f_info}Saving esm-datastore to {f_path}{str(catalog_dir.absolute())}{f_reset}"
126 )
File /g/data/xp65/public/apps/med_conda/envs/analysis3-26.01/lib/python3.11/site-packages/access_nri_intake/source/builders.py:237, in BaseBuilder.build(self)
232 def build(self):
233 """
234 Builds a datastore from a list of netCDF files or zarr stores.
235 """
--> 237 self.get_assets().validate_parser().parse().clean_dataframe()
239 return self
File /g/data/xp65/public/apps/med_conda/envs/analysis3-26.01/lib/python3.11/site-packages/access_nri_intake/source/builders.py:227, in BaseBuilder.validate_parser(self)
224 validate_against_schema(info, ESM_JSONSCHEMA)
225 return self
--> 227 raise ParserError(f"""Parser returns no valid assets.
228 Try parsing a single file with Builder.parser(file)
229 Last failed asset: {asset}
230 Asset parser return: {info}""")
ParserError: Parser returns no valid assets.
Try parsing a single file with Builder.parser(file)
Last failed asset: /g/data/g40/access-om2/archive/01deg_jra55v140_iaf_cycle3_antarctic_tracers_isopyc/passive_wilkes_tr_DensityBinned_monthly-mean-ym_1999_12.nc
Asset parser return: {'INVALID_ASSET': '/g/data/g40/access-om2/archive/01deg_jra55v140_iaf_cycle3_antarctic_tracers_isopyc/passive_wilkes_tr_DensityBinned_monthly-mean-ym_1999_12.nc', 'TRACEBACK': 'Traceback (most recent call last):\n File "/g/data/xp65/public/apps/med_conda/envs/analysis3-26.01/lib/python3.11/site-packages/access_nri_intake/source/builders.py", line 207, in _parser_catch_invalid\n return cls.parser(file)\n ^^^^^^^^^^^^^^^^\n File "/g/data/xp65/public/apps/med_conda/envs/analysis3-26.01/lib/python3.11/site-packages/access_nri_intake/source/builders.py", line 470, in parser\n raise ParserError(f"Cannot determine realm for file {file}")\naccess_nri_intake.source.builders.ParserError: Cannot determine realm for file /g/data/g40/access-om2/archive/01deg_jra55v140_iaf_cycle3_antarctic_tracers_isopyc/passive_wilkes_tr_DensityBinned_monthly-mean-ym_1999_12.nc\n'}