Regional Ancillary Suite no longer running (again)

Atmosphere Unified Model
1

Hi all.

This appears to the same problem that afflicted Sonya Fiddes back in August - see Regional Ancillary Suite no longer running

I checked out u-dg767 yesterday and it ran in the morning. I made some changes to the domain location in the afternoon, and hasn’t worked since.

The error is a repeat of Sonya’s error back in August)

/g/data/access/TIDS/UM/ancil/atmos/master/vegetation/cover/cci/v3/vegetation_fraction.nc does not exist.

These files no longer exist. Only v1 does.

$ ls -lt /g/data/access/TIDS/UM/ancil/atmos/master/vegetation/cover/cci/
total 4
drwxr-xr-x+ 2 mrd599 access.admin 4096 Feb 26  2020 v1

The request for v3 is provided in app/ancil_lct/rose-app.conf

[env]
source=${ANCIL_MASTER}/vegetation/cover/cci/v3/vegetation_fraction.nc

Is it possible that I deleted the files in /g/data/access/ ?

Cheers,

Paul

2

Not sure how the files got deleted, but trying the rsync from JASMIN again at the moment.

1 Like
3

Thanks Martin.

The files have returned.

 ls -lt /g/data/access/TIDS/UM/ancil/atmos/master/vegetation/cover/cci/
total 12
drwxrwxr-x+ 2 mrd599 p66          4096 Apr  4  2020 v3
drwxrwxr-x+ 2 mrd599 access.admin 4096 Mar  5  2020 v1
drwxrwxr-x+ 2 mrd599 p66          4096 Sep 29  2018 v2

EDIT. Spoke too soon. The directories have synced but the files haven’t yet.

$ ls -lt /g/data/access/TIDS/UM/ancil/atmos/master/vegetation/cover/cci/v3/
total 0
4

@MartinDix Any update on the rsync status? I was hoping by this morning the files would have synced, but there is still nothing in /g/data/access/TIDS/UM/ancil/atmos/master/vegetation/cover/cci/v3/

1 Like
5

All the master ancillary files should be restored now.

6

Wonderfull! Thanks very much Martin.

7

Hi Martin,
Thanks for your work on this! I’m unfortunately seeing similar issues in: /g/data/access/TIDS/UM/ancil/data/parameters/
(i.e. data rather than atmos/master)

Emma

8

Hi Emma

I can confirm the same issue. The job ancil_cap_vegfrac fails with

forrtl: severe (29): file not found, unit 38, file /scratch/gb02/pag548/cylc-run/u-dg767/work/1/Gippsland_era5_ancil_cap_vegfrac/fort.38

The underlying link to v3 TIDS ancillary data directory is missing.

$ ls -l /g/data/access/TIDS/UM/ancil/data/parameters/IGBP_to_MOSES/latest

lrwxrwxrwx 1 mrd599 access.admin 2 Dec 14 2015 /g/data/access/TIDS/UM/ancil/data/parameters/IGBP_to_MOSES/latest -> **v3**

I’m interested in why the install_cold task succeeds if this data is missing.

I will run through the task tomorrow to try and find out why.

9

This should be fixed now.

Still a mystery how these files got deleted but I’m changing the permissions so it shouldn’t happen again

2 Likes
10

Hi Martin - sorry, one more that seems to be missing now?
We need these ones for the global driving model…

/g/data/access/projects/access/umdir/ancil/atmos/n216e/orca025/land_sea_mask/etop01/v3/

It’s a real mystery! Hopefully once the permissions are changed it won’t happen again!

1 Like
11

Some of the n216 files went missing but no other resolutions :confused:

Downloading now

1 Like
12

Should be complete now.

13

Jasmin is now recommending globus for data transfers JASMIN Help Site - Migration to Rocky Linux 9 2024. I wonder if this could be used for the tids data sync, not knowing too much about how globus works

1 Like