Containers failing to start all of a sudden

I woke up this morning to the CORE failing once again to gracefully recover from a network outage. Long story short I manually got the network back and unplugged the CORE, plugged it back in and now a handful of containers are refusing to start citing something wrong with symlinks. The affected containers are Z2M, InfluxDB, and Uptime-Kuma.

This is an example log for InfluxDB. Very strange error after downloading the new image. Any guidance would be greatly appreciated @markus @april.brandt

...
Timestamp	Lvl	Message
2025-03-20 10:04:03 GMT−7	info	Finalizing preparing for the InfluxDB container as the podman user...
2025-03-20 10:04:03 GMT−7	info	Done!
2025-03-20 10:04:03 GMT−7	info	pam_unix(su-l:session): session closed for user oh-la
2025-03-20 10:05:50 GMT−7	info	INFO: Running command: prepare
2025-03-20 10:05:50 GMT−7	info	Executing the first part of prepare as root
2025-03-20 10:05:50 GMT−7	info	WARN: NOT replacing /home/oh-la/data/influxdb/influxdb.conf since there has been manual changes!
2025-03-20 10:05:50 GMT−7	info	failed adding user 'telegraf', data deleted
2025-03-20 10:05:50 GMT−7	info	Set permissions
2025-03-20 10:05:50 GMT−7	info	About to update telegraf.conf
2025-03-20 10:05:51 GMT−7	info	INFO: Overwriting /etc/telegraf/telegraf.conf since no manual changes have been made to it!
2025-03-20 10:05:51 GMT−7	info	Updated telegraf.conf
2025-03-20 10:05:51 GMT−7	info	Inserting the telegraf influxdb password into the telegraf config
2025-03-20 10:05:51 GMT−7	info	Done!
2025-03-20 10:05:51 GMT−7	info	INFO: We are root, becoming the oh-la user instead...
2025-03-20 10:05:51 GMT−7	info	INFO: Running "oll-influxdb --prepare" as user oh-la
2025-03-20 10:05:51 GMT−7	info	(to oh-la) root on none
2025-03-20 10:05:51 GMT−7	info	pam_unix(su-l:session): session opened for user oh-la(uid=2000) by (uid=0)
2025-03-20 10:05:51 GMT−7	info	INFO: Running command: prepare
2025-03-20 10:05:51 GMT−7	info	INFO: Running as required user oh-la
2025-03-20 10:05:51 GMT−7	info	Finalizing preparing for the InfluxDB container as the podman user...
2025-03-20 10:05:51 GMT−7	info	Done!
2025-03-20 10:05:51 GMT−7	info	pam_unix(su-l:session): session closed for user oh-la
2025-03-20 10:06:32 GMT−7	info	INFO: Running command: prepare
2025-03-20 10:06:32 GMT−7	info	Executing the first part of prepare as root
2025-03-20 10:06:32 GMT−7	info	WARN: NOT replacing /home/oh-la/data/influxdb/influxdb.conf since there has been manual changes!
2025-03-20 10:06:32 GMT−7	info	failed adding user 'telegraf', data deleted
2025-03-20 10:06:32 GMT−7	info	Set permissions
2025-03-20 10:06:32 GMT−7	info	About to update telegraf.conf
2025-03-20 10:06:32 GMT−7	info	INFO: Overwriting /etc/telegraf/telegraf.conf since no manual changes have been made to it!
2025-03-20 10:06:32 GMT−7	info	Updated telegraf.conf
2025-03-20 10:06:32 GMT−7	info	Inserting the telegraf influxdb password into the telegraf config
2025-03-20 10:06:32 GMT−7	info	Done!
2025-03-20 10:06:32 GMT−7	info	INFO: We are root, becoming the oh-la user instead...
2025-03-20 10:06:32 GMT−7	info	INFO: Running "oll-influxdb --prepare" as user oh-la
2025-03-20 10:06:32 GMT−7	info	(to oh-la) root on none
2025-03-20 10:06:32 GMT−7	info	pam_unix(su-l:session): session opened for user oh-la(uid=2000) by (uid=0)
2025-03-20 10:06:33 GMT−7	info	INFO: Running command: prepare
2025-03-20 10:06:33 GMT−7	info	INFO: Running as required user oh-la
2025-03-20 10:06:33 GMT−7	info	Finalizing preparing for the InfluxDB container as the podman user...
2025-03-20 10:06:33 GMT−7	info	Done!
2025-03-20 10:06:33 GMT−7	info	pam_unix(su-l:session): session closed for user oh-la
2025-03-20 10:07:14 GMT−7	info	INFO: Running command: prepare
2025-03-20 10:07:14 GMT−7	info	Executing the first part of prepare as root
2025-03-20 10:07:14 GMT−7	info	WARN: NOT replacing /home/oh-la/data/influxdb/influxdb.conf since there has been manual changes!
2025-03-20 10:07:14 GMT−7	info	failed adding user 'telegraf', data deleted
2025-03-20 10:07:14 GMT−7	info	Set permissions
2025-03-20 10:07:14 GMT−7	info	About to update telegraf.conf
2025-03-20 10:07:14 GMT−7	info	INFO: Overwriting /etc/telegraf/telegraf.conf since no manual changes have been made to it!
2025-03-20 10:07:14 GMT−7	info	Updated telegraf.conf
2025-03-20 10:07:14 GMT−7	info	Inserting the telegraf influxdb password into the telegraf config
2025-03-20 10:07:14 GMT−7	info	Done!
2025-03-20 10:07:14 GMT−7	info	INFO: We are root, becoming the oh-la user instead...
2025-03-20 10:07:14 GMT−7	info	INFO: Running "oll-influxdb --prepare" as user oh-la
2025-03-20 10:07:14 GMT−7	info	(to oh-la) root on none
2025-03-20 10:07:14 GMT−7	info	pam_unix(su-l:session): session opened for user oh-la(uid=2000) by (uid=0)
2025-03-20 10:07:14 GMT−7	info	INFO: Running command: prepare
2025-03-20 10:07:14 GMT−7	info	INFO: Running as required user oh-la
2025-03-20 10:07:14 GMT−7	info	Finalizing preparing for the InfluxDB container as the podman user...
2025-03-20 10:07:15 GMT−7	info	Done!
2025-03-20 10:07:15 GMT−7	info	pam_unix(su-l:session): session closed for user oh-la
2025-03-20 10:07:53 GMT−7	info	INFO: Running command: prepare
2025-03-20 10:07:53 GMT−7	info	Executing the first part of prepare as root
2025-03-20 10:07:53 GMT−7	info	WARN: NOT replacing /home/oh-la/data/influxdb/influxdb.conf since there has been manual changes!
2025-03-20 10:07:53 GMT−7	info	failed adding user 'telegraf', data deleted
2025-03-20 10:07:53 GMT−7	info	Set permissions
2025-03-20 10:07:54 GMT−7	info	About to update telegraf.conf
2025-03-20 10:07:54 GMT−7	info	INFO: Overwriting /etc/telegraf/telegraf.conf since no manual changes have been made to it!
2025-03-20 10:07:54 GMT−7	info	Updated telegraf.conf
2025-03-20 10:07:54 GMT−7	info	Inserting the telegraf influxdb password into the telegraf config
2025-03-20 10:07:54 GMT−7	info	Done!
2025-03-20 10:07:54 GMT−7	info	INFO: We are root, becoming the oh-la user instead...
2025-03-20 10:07:54 GMT−7	info	INFO: Running "oll-influxdb --prepare" as user oh-la
2025-03-20 10:07:54 GMT−7	info	(to oh-la) root on none
2025-03-20 10:07:54 GMT−7	info	pam_unix(su-l:session): session opened for user oh-la(uid=2000) by (uid=0)
2025-03-20 10:07:54 GMT−7	info	INFO: Running command: prepare
2025-03-20 10:07:54 GMT−7	info	INFO: Running as required user oh-la
2025-03-20 10:07:54 GMT−7	info	Finalizing preparing for the InfluxDB container as the podman user...
2025-03-20 10:07:54 GMT−7	info	Done!
2025-03-20 10:07:54 GMT−7	info	pam_unix(su-l:session): session closed for user oh-la
2025-03-20 10:31:42 GMT−7	info	INFO: Running command: prepare
2025-03-20 10:31:42 GMT−7	info	Executing the first part of prepare as root
2025-03-20 10:31:42 GMT−7	info	WARN: NOT replacing /home/oh-la/data/influxdb/influxdb.conf since there has been manual changes!
2025-03-20 10:31:42 GMT−7	info	failed adding user 'telegraf', data deleted
2025-03-20 10:31:42 GMT−7	info	Set permissions
2025-03-20 10:31:42 GMT−7	info	About to update telegraf.conf
2025-03-20 10:31:42 GMT−7	info	INFO: Overwriting /etc/telegraf/telegraf.conf since no manual changes have been made to it!
2025-03-20 10:31:42 GMT−7	info	Updated telegraf.conf
2025-03-20 10:31:42 GMT−7	info	Inserting the telegraf influxdb password into the telegraf config
2025-03-20 10:31:42 GMT−7	info	Done!
2025-03-20 10:31:42 GMT−7	info	INFO: We are root, becoming the oh-la user instead...
2025-03-20 10:31:42 GMT−7	info	INFO: Running "oll-influxdb --prepare" as user oh-la
2025-03-20 10:31:42 GMT−7	info	(to oh-la) root on none
2025-03-20 10:31:42 GMT−7	info	pam_unix(su-l:session): session opened for user oh-la(uid=2000) by (uid=0)
2025-03-20 10:31:43 GMT−7	info	INFO: Running command: prepare
2025-03-20 10:31:43 GMT−7	info	INFO: Running as required user oh-la
2025-03-20 10:31:43 GMT−7	info	Finalizing preparing for the InfluxDB container as the podman user...
2025-03-20 10:31:58 GMT−7	info	INFO: Running command: defaults
2025-03-20 10:31:58 GMT−7	info	INFO: Running as required user oh-la
2025-03-20 10:31:58 GMT−7	info	INFO: Ensuring the ~/.systemd folder exists
2025-03-20 10:31:58 GMT−7	info	INFO: Running command: run
2025-03-20 10:31:58 GMT−7	info	INFO: Running as required user oh-la
2025-03-20 10:31:59 GMT−7	info	INFO: Running the InfluxDB container... Should ONLY be run through a systemd service!
2025-03-20 10:31:59 GMT−7	info	INFO: Mapping ports: 8086:8086
2025-03-20 10:31:59 GMT−7	info	INFO: Mapping ports: 8089:8089/udp
2025-03-20 10:31:59 GMT−7	info	INFO: Mapping folders: /home/oh-la/data/influxdb:/var/lib/influxdb
2025-03-20 10:31:59 GMT−7	info	INFO: Mapping folders: /etc/ssl/certs:/etc/ssl/certs:ro
2025-03-20 10:31:59 GMT−7	info	INFO: Mapping folders: /usr/local/share/ca-certificates:/usr/local/share/ca-certificates:ro
2025-03-20 10:31:59 GMT−7	info	INFO: Mapping enviroment var: NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt
2025-03-20 10:31:59 GMT−7	info	INFO: Mapping enviroment var: INFLUXDB_LOGGING_FORMAT=json
2025-03-20 10:31:59 GMT−7	info	INFO: Mapping enviroment var: INFLUXDB_REPORTING_DISABLED=true
2025-03-20 10:31:59 GMT−7	info	INFO: Mapping enviroment var: INFLUXDB_LOGGING_SUPPRESS_LOGO=false
2025-03-20 10:31:59 GMT−7	info	INFO: Mapping enviroment var: INFLUXDB_LOGGING_LEVEL=warn
2025-03-20 10:31:59 GMT−7	info	INFO: Mapping enviroment var: INFLUXDB_CONFIG_PATH=/var/lib/influxdb/influxdb.conf
2025-03-20 10:31:59 GMT−7	info	time="2025-03-20T10:31:59-07:00" level=info msg="/usr/bin/podman filtering at log level info"
2025-03-20 10:31:59 GMT−7	info	time="2025-03-20T10:31:59-07:00" level=info msg="Error initializing configured OCI runtime kata: no valid executable found for OCI runtime kata: invalid argument"
2025-03-20 10:31:59 GMT−7	info	time="2025-03-20T10:31:59-07:00" level=info msg="Error initializing configured OCI runtime runc: no valid executable found for OCI runtime runc: invalid argument"
2025-03-20 10:31:59 GMT−7	info	time="2025-03-20T10:31:59-07:00" level=info msg="Setting parallel job count to 13"
2025-03-20 10:31:59 GMT−7	info	time="2025-03-20T10:31:59-07:00" level=info msg="/usr/bin/podman filtering at log level info"
2025-03-20 10:31:59 GMT−7	info	time="2025-03-20T10:31:59-07:00" level=info msg="Error initializing configured OCI runtime runc: no valid executable found for OCI runtime runc: invalid argument"
2025-03-20 10:31:59 GMT−7	info	time="2025-03-20T10:31:59-07:00" level=info msg="Error initializing configured OCI runtime kata: no valid executable found for OCI runtime kata: invalid argument"
2025-03-20 10:31:59 GMT−7	info	time="2025-03-20T10:31:59-07:00" level=info msg="Setting parallel job count to 13"
2025-03-20 10:31:59 GMT−7	info	Resolving "influxdb" using unqualified-search registries (/etc/containers/registries.conf)
2025-03-20 10:31:59 GMT−7	info	Trying to pull docker.io/library/influxdb:1.8...
2025-03-20 10:32:31 GMT−7	info	time="2025-03-20T10:32:31-07:00" level=warning msg="failed, retrying in 1s ... (1/3). Error: Error initializing source docker://influxdb:1.8: Error reading manifest 1.8 in docker.io/library/influxdb: toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit"
2025-03-20 10:33:03 GMT−7	info	time="2025-03-20T10:33:03-07:00" level=warning msg="failed, retrying in 2s ... (2/3). Error: Error initializing source docker://influxdb:1.8: Error reading manifest 1.8 in docker.io/library/influxdb: toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit"
2025-03-20 10:33:37 GMT−7	info	time="2025-03-20T10:33:37-07:00" level=warning msg="failed, retrying in 4s ... (3/3). Error: Error initializing source docker://influxdb:1.8: Error reading manifest 1.8 in docker.io/library/influxdb: toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit"
2025-03-20 10:34:13 GMT−7	info	toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit
2025-03-20 10:34:13 GMT−7	info	Error: Error initializing source docker://influxdb:1.8: Error reading manifest 1.8 in docker.io/library/influxdb: toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit
2025-03-20 10:34:13 GMT−7	info	time="2025-03-20T10:34:13-07:00" level=error msg="failed to kill 12596"
2025-03-20 10:34:14 GMT−7	info	Error: no arguments are needed with --latest or --cidfile
2025-03-20 10:34:44 GMT−7	info	INFO: Running command: prepare
2025-03-20 10:34:44 GMT−7	info	Executing the first part of prepare as root
2025-03-20 10:34:44 GMT−7	info	WARN: NOT replacing /home/oh-la/data/influxdb/influxdb.conf since there has been manual changes!
2025-03-20 10:34:44 GMT−7	info	failed adding user 'telegraf', data deleted
2025-03-20 10:34:44 GMT−7	info	Set permissions
2025-03-20 10:34:44 GMT−7	info	About to update telegraf.conf
2025-03-20 10:34:44 GMT−7	info	INFO: Overwriting /etc/telegraf/telegraf.conf since no manual changes have been made to it!
2025-03-20 10:34:44 GMT−7	info	Updated telegraf.conf
2025-03-20 10:34:44 GMT−7	info	Inserting the telegraf influxdb password into the telegraf config
2025-03-20 10:34:44 GMT−7	info	Done!
2025-03-20 10:34:44 GMT−7	info	INFO: We are root, becoming the oh-la user instead...
2025-03-20 10:34:44 GMT−7	info	INFO: Running "oll-influxdb --prepare" as user oh-la
2025-03-20 10:34:44 GMT−7	info	(to oh-la) root on none
2025-03-20 10:34:44 GMT−7	info	pam_unix(su-l:session): session opened for user oh-la(uid=2000) by (uid=0)
2025-03-20 10:34:45 GMT−7	info	INFO: Running command: prepare
2025-03-20 10:34:45 GMT−7	info	INFO: Running as required user oh-la
2025-03-20 10:34:45 GMT−7	info	Finalizing preparing for the InfluxDB container as the podman user...
2025-03-20 10:35:49 GMT−7	info	INFO: Running command: defaults
2025-03-20 10:35:49 GMT−7	info	INFO: Running as required user oh-la
2025-03-20 10:35:49 GMT−7	info	INFO: Ensuring the ~/.systemd folder exists
2025-03-20 10:35:49 GMT−7	info	INFO: Running command: run
2025-03-20 10:35:49 GMT−7	info	INFO: Running as required user oh-la
2025-03-20 10:35:58 GMT−7	info	INFO: Running the InfluxDB container... Should ONLY be run through a systemd service!
2025-03-20 10:35:58 GMT−7	info	INFO: Mapping ports: 8086:8086
2025-03-20 10:35:58 GMT−7	info	INFO: Mapping ports: 8089:8089/udp
2025-03-20 10:35:58 GMT−7	info	INFO: Mapping folders: /home/oh-la/data/influxdb:/var/lib/influxdb
2025-03-20 10:35:58 GMT−7	info	INFO: Mapping folders: /etc/ssl/certs:/etc/ssl/certs:ro
2025-03-20 10:35:58 GMT−7	info	INFO: Mapping folders: /usr/local/share/ca-certificates:/usr/local/share/ca-certificates:ro
2025-03-20 10:35:58 GMT−7	info	INFO: Mapping enviroment var: NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt
2025-03-20 10:35:58 GMT−7	info	INFO: Mapping enviroment var: INFLUXDB_LOGGING_FORMAT=json
2025-03-20 10:35:58 GMT−7	info	INFO: Mapping enviroment var: INFLUXDB_REPORTING_DISABLED=true
2025-03-20 10:35:58 GMT−7	info	INFO: Mapping enviroment var: INFLUXDB_LOGGING_SUPPRESS_LOGO=false
2025-03-20 10:35:58 GMT−7	info	INFO: Mapping enviroment var: INFLUXDB_LOGGING_LEVEL=warn
2025-03-20 10:35:58 GMT−7	info	INFO: Mapping enviroment var: INFLUXDB_CONFIG_PATH=/var/lib/influxdb/influxdb.conf
2025-03-20 10:35:59 GMT−7	info	time="2025-03-20T10:35:59-07:00" level=info msg="/usr/bin/podman filtering at log level info"
2025-03-20 10:35:59 GMT−7	info	time="2025-03-20T10:35:59-07:00" level=info msg="Error initializing configured OCI runtime kata: no valid executable found for OCI runtime kata: invalid argument"
2025-03-20 10:35:59 GMT−7	info	time="2025-03-20T10:35:59-07:00" level=info msg="Error initializing configured OCI runtime runc: no valid executable found for OCI runtime runc: invalid argument"
2025-03-20 10:35:59 GMT−7	info	time="2025-03-20T10:35:59-07:00" level=info msg="Setting parallel job count to 13"
2025-03-20 10:35:59 GMT−7	info	time="2025-03-20T10:35:59-07:00" level=info msg="/usr/bin/podman filtering at log level info"
2025-03-20 10:35:59 GMT−7	info	time="2025-03-20T10:35:59-07:00" level=info msg="Error initializing configured OCI runtime runc: no valid executable found for OCI runtime runc: invalid argument"
2025-03-20 10:35:59 GMT−7	info	time="2025-03-20T10:35:59-07:00" level=info msg="Error initializing configured OCI runtime kata: no valid executable found for OCI runtime kata: invalid argument"
2025-03-20 10:35:59 GMT−7	info	time="2025-03-20T10:35:59-07:00" level=info msg="Setting parallel job count to 13"
2025-03-20 10:35:59 GMT−7	info	Resolving "influxdb" using unqualified-search registries (/etc/containers/registries.conf)
2025-03-20 10:35:59 GMT−7	info	Trying to pull docker.io/library/influxdb:1.8...
2025-03-20 10:36:01 GMT−7	info	Getting image source signatures
2025-03-20 10:36:01 GMT−7	info	Copying blob sha256:76475c2689e229fac9e8ba4a02e64decb7fd62b2a3e4ad65ba97f8e1a35471f2
2025-03-20 10:36:01 GMT−7	info	Copying blob sha256:1deb2c2ef23607994f7238c8d97d113f5ebd868b8eb64a0c10d2e0983f036a39
2025-03-20 10:36:02 GMT−7	info	Copying blob sha256:8fa0738214bb9b9c99a4687a36d2f88b1da7bee68cfb099439210663bc99d7d6
2025-03-20 10:36:02 GMT−7	info	Copying blob sha256:71efbbe32e44303a3305fcc39de75a46e033b4ffd4176944c7e3c7eb7063148a
2025-03-20 10:36:02 GMT−7	info	Copying blob sha256:822229d5942c09beb2b6b9cca4f78a6f49f33025a1e30feea735942abb7fb6a5
2025-03-20 10:36:02 GMT−7	info	Copying blob sha256:b8ace858e7d3c6b17074be16ab9448e1241e907bef8fa72a4d53144e8a7f065a
2025-03-20 10:36:02 GMT−7	info	Copying blob sha256:54df75daf565fb6d2ed179ea7b170f9692df1301c9c1234823572e780a758f19
2025-03-20 10:36:09 GMT−7	info	Copying config sha256:7fe366b9a9af62c1d199e01e5898d650011bc52fc18be481851ff4f236aff284
2025-03-20 10:36:09 GMT−7	info	Writing manifest to image destination
2025-03-20 10:36:09 GMT−7	info	Storing signatures
2025-03-20 10:36:09 GMT−7	info	time="2025-03-20T10:36:09-07:00" level=warning msg="Can't read link \"/home/oh-la/.local/share/containers/storage/overlay/l/6W3NDAZU6I53V5TAQYOVN3E7VC\" because it does not exist. Going through storage to recreate the missing symlinks."
2025-03-20 10:36:09 GMT−7	info	time="2025-03-20T10:36:09-07:00" level=warning msg="Can't stat lower layer \"/home/oh-la/.local/share/containers/storage/overlay/l/6W3NDAZU6I53V5TAQYOVN3E7VC\" because it does not exist. Going through storage to recreate the missing symlinks."
2025-03-20 10:36:09 GMT−7	info	error recreating the missing symlinks: error reading name of symlink for &{"1c0de0b539a9c2a4b6dd5d33ec23867a408dcd8493227734f03c6ce9334f7ffc" 'က' %!q(os.FileMode=2147484096) {%!q(uint64=668000040) %!q(int64=63814361496) %!q(*time.Location=&{Local [{LMT -28378 false} {PDT -25200 true} {PST -28800 false} {PWT -25200 true} {PPT -25200 true}] [{-2717640000 2 false false} {-1633269600 1 false false} {-1615129200 2 false false} {-1601820000 1 false false} {-1583679600 2 true true} {-880207200 3 false false} {-769395600 4 false false} {-765385200 2 false false} {-687967140 1 false false} {-662655600 2 false false} {-620838000 1 false false} {-608137200 2 false false} {-589388400 1 false false} {-576082800 2 false false} {-557938800 1 false false} {-544633200 2 false false} {-526489200 1 false false} {-513183600 2 false false} {-495039600 1 false false} {-481734000 2 false false} {-463590000 1 false false} {-450284400 2 false false} {-431535600 1 false false} {-418230000 2 false false} {-400086000 1 false false} {-386780400 2 false false} {-368636400 1 false false} {-355330800 2 false false} {-337186800 1 false false} {-323881200 2 false false} {-305737200 1 false false} {-292431600 2 false false} {-273682800 1 false false} {-260982000 2 false false} {-242233200 1 false false} {-226508400 2 false false} {-210783600 1 false false} {-195058800 2 false false} {-179334000 1 false false} {-163609200 2 false false} {-147884400 1 false false} {-131554800 2 false false} {-116434800 1 false false} {-100105200 2 false false} {-84376800 1 false false} {-68655600 2 false false} {-52927200 1 false false} {-37206000 2 false false} {-21477600 1 false false} {-5756400 2 false false} {9972000 1 false false} {25693200 2 false false} {41421600 1 false false} {57747600 2 false false} {73476000 1 false false} {89197200 2 false false} {104925600 1 false false} {120646800 2 false false} {126698400 1 false false} {152096400 2 false false} {162381600 1 false false} {183546000 2 false false} {199274400 1 false false} {215600400 2 false false} {230724000 1 false false} {247050000 2 false false} {262778400 1 false false} {278499600 2 false false} {294228000 1 false false} {309949200 2 false false} {325677600 1 false false} {341398800 2 false false} {357127200 1 false false} {372848400 2 false false} {388576800 1 false false} {404902800 2 false false} {420026400 1 false false} {436352400 2 false false} {452080800 1 false false} {467802000 2 false false} {483530400 1 false false} {499251600 2 false false} {514980000 1 false false} {530701200 2 false false} {544615200 1 false false} {562150800 2 false false} {576064800 1 false false} {594205200 2 false false} {607514400 1 false false} {625654800 2 false false} {638964000 1 false false} {657104400 2 false false} {671018400 1 false false} {688554000 2 false false} {702468000 1 false false} {720003600 2 false false} {733917600 1 false false} {752058000 2 false false} {765367200 1 false false} {783507600 2 false false} {796816800 1 false false} {814957200 2 false false} {828871200 1 false false} {846406800 2 false false} {860320800 1 false false} {877856400 2 false false} {891770400 1 false false} {909306000 2 false false} {923220000 1 false false} {941360400 2 false false} {954669600 1 false false} {972810000 2 false false} {986119200 1 false false} {1004259600 2 false false} {1018173600 1 false false} {1035709200 2 false false} {1049623200 1 false false} {1067158800 2 false false} {1081072800 1 false false} {1099213200 2 false false} {1112522400 1 false false} {1130662800 2 false false} {1143972000 1 false false} {1162112400 2 false false} {1173607200 1 false false} {1194166800 2 false false} {1205056800 1 false false} {1225616400 2 false false} {1236506400 1 false false} {1257066000 2 false false} {1268560800 1 false false} {1289120400 2 false false} {1300010400 1 false false} {1320570000 2 false false} {1331460000 1 false false} {1352019600 2 false false} {1362909600 1 false false} {1383469200 2 false false} {1394359200 1 false false} {1414918800 2 false false} {1425808800 1 false false} {1446368400 2 false false} {1457863200 1 false false} {1478422800 2 false false} {1489312800 1 false false} {1509872400 2 false false} {1520762400 1 false false} {1541322000 2 false false} {1552212000 1 false false} {1572771600 2 false false} {1583661600 1 false false} {1604221200 2 false false} {1615716000 1 false false} {1636275600 2 false false} {1647165600 1 false false} {1667725200 2 false false} {1678615200 1 false false} {1699174800 2 false false} {1710064800 1 false false} {1730624400 2 false false} {1741514400 1 false false} {1762074000 2 false false} {1772964000 1 false false} {1793523600 2 false false} {1805018400 1 false false} {1825578000 2 false false} {1836468000 1 false false} {1857027600 2 false false} {1867917600 1 false false} {1888477200 2 false false} {1899367200 1 false false} {1919926800 2 false false} {1930816800 1 false false} {1951376400 2 false false} {1962871200 1 false false} {1983430800 2 false false} {1994320800 1 false false} {2014880400 2 false false} {2025770400 1 false false} {2046330000 2 false false} {2057220000 1 false false} {2077779600 2 false false} {2088669600 1 false false} {2109229200 2 false false} {2120119200 1 false false} {2140678800 2 false false}] PST8PDT,M3.2.0,M11.1.0 1741514400 1762074000 0x4000162160})} {'\x1f' '\U00010265' '䇀' '\x03' '\x00' '\x00' '\x00' '\x00' 'က' 'က' '\x00' '\b' {%!q(int64=1741588746) %!q(int64=92565444)} {%!q(int64=1678764696) %!q(int64=668000040)} {%!q(int64=1678764696) %!q(int64=668000040)} ['\x00' '\x00']}}: open /home/oh-la/.local/share/containers/storage/overlay/1c0de0b539a9c2a4b6dd5d33ec23867a408dcd8493227734f03c6ce9334f7ffc/link: no such file or directory
2025-03-20 10:36:09 GMT−7	info	Error: Error committing the finished image: error adding layer with blob "sha256:71efbbe32e44303a3305fcc39de75a46e033b4ffd4176944c7e3c7eb7063148a": error recreating the missing symlinks: error reading name of symlink for &{"1c0de0b539a9c2a4b6dd5d33ec23867a408dcd8493227734f03c6ce9334f7ffc" 'က' %!q(os.FileMode=2147484096) {%!q(uint64=668000040) %!q(int64=63814361496) %!q(*time.Location=&{Local [{LMT -28378 false} {PDT -25200 true} {PST -28800 false} {PWT -25200 true} {PPT -25200 true}] [{-2717640000 2 false false} {-1633269600 1 false false} {-1615129200 2 false false} {-1601820000 1 false false} {-1583679600 2 true true} {-880207200 3 false false} {-769395600 4 false false} {-765385200 2 false false} {-687967140 1 false false} {-662655600 2 false false} {-620838000 1 false false} {-608137200 2 false false} {-589388400 1 false false} {-576082800 2 false false} {-557938800 1 false false} {-544633200 2 false false} {-526489200 1 false false} {-513183600 2 false false} {-495039600 1 false false} {-481734000 2 false false} {-463590000 1 false false} {-450284400 2 false false} {-431535600 1 false false} {-418230000 2 false false} {-400086000 1 false false} {-386780400 2 false false} {-368636400 1 false false} {-355330800 2 false false} {-337186800 1 false false} {-323881200 2 false false} {-305737200 1 false false} {-292431600 2 false false} {-273682800 1 false false} {-260982000 2 false false} {-242233200 1 false false} {-226508400 2 false false} {-210783600 1 false false} {-195058800 2 false false} {-179334000 1 false false} {-163609200 2 false false} {-147884400 1 false false} {-131554800 2 false false} {-116434800 1 false false} {-100105200 2 false false} {-84376800 1 false false} {-68655600 2 false false} {-52927200 1 false false} {-37206000 2 false false} {-21477600 1 false false} {-5756400 2 false false} {9972000 1 false false} {25693200 2 false false} {41421600 1 false false} {57747600 2 false false} {73476000 1 false false} {89197200 2 false false} {104925600 1 false false} {120646800 2 false false} {126698400 1 false false} {152096400 2 false false} {162381600 1 false false} {183546000 2 false false} {199274400 1 false false} {215600400 2 false false} {230724000 1 false false} {247050000 2 false false} {262778400 1 false false} {278499600 2 false false} {294228000 1 false false} {309949200 2 false false} {325677600 1 false false} {341398800 2 false false} {357127200 1 false false} {372848400 2 false false} {388576800 1 false false} {404902800 2 false false} {420026400 1 false false} {436352400 2 false false} {452080800 1 false false} {467802000 2 false false} {483530400 1 false false} {499251600 2 false false} {514980000 1 false false} {530701200 2 false false} {544615200 1 false false} {562150800 2 false false} {576064800 1 false false} {594205200 2 false false} {607514400 1 false false} {625654800 2 false false} {638964000 1 false false} {657104400 2 false false} {671018400 1 false false} {688554000 2 false false} {702468000 1 false false} {720003600 2 false false} {733917600 1 false false} {752058000 2 false false} {765367200 1 false false} {783507600 2 false false} {796816800 1 false false} {814957200 2 false false} {828871200 1 false false} {846406800 2 false false} {860320800 1 false false} {877856400 2 false false} {891770400 1 false false} {909306000 2 false false} {923220000 1 false false} {941360400 2 false false} {954669600 1 false false} {972810000 2 false false} {986119200 1 false false} {1004259600 2 false false} {1018173600 1 false false} {1035709200 2 false false} {1049623200 1 false false} {1067158800 2 false false} {1081072800 1 false false} {1099213200 2 false false} {1112522400 1 false false} {1130662800 2 false false} {1143972000 1 false false} {1162112400 2 false false} {1173607200 1 false false} {1194166800 2 false false} {1205056800 1 false false} {1225616400 2 false false} {1236506400 1 false false} {1257066000 2 false false} {1268560800 1 false false} {1289120400 2 false false} {1300010400 1 false false} {1320570000 2 false false} {1331460000 1 false false} {1352019600 2 false false} {1362909600 1 false false} {1383469200 2 false false} {1394359200 1 false false} {1414918800 2 false false} {1425808800 1 false false} {1446368400 2 false false} {1457863200 1 false false} {1478422800 2 false false} {1489312800 1 false false} {1509872400 2 false false} {1520762400 1 false false} {1541322000 2 false false} {1552212000 1 false false} {1572771600 2 false false} {1583661600 1 false false} {1604221200 2 false false} {1615716000 1 false false} {1636275600 2 false false} {1647165600 1 false false} {1667725200 2 false false} {1678615200 1 false false} {1699174800 2 false false} {1710064800 1 false false} {1730624400 2 false false} {1741514400 1 false false} {1762074000 2 false false} {1772964000 1 false false} {1793523600 2 false false} {1805018400 1 false false} {1825578000 2 false false} {1836468000 1 false false} {1857027600 2 false false} {1867917600 1 false false} {1888477200 2 false false} {1899367200 1 false false} {1919926800 2 false false} {1930816800 1 false false} {1951376400 2 false false} {1962871200 1 false false} {1983430800 2 false false} {1994320800 1 false false} {2014880400 2 false false} {2025770400 1 false false} {2046330000 2 false false} {2057220000 1 false false} {2077779600 2 false false} {2088669600 1 false false} {2109229200 2 false false} {2120119200 1 false false} {2140678800 2 false false}] PST8PDT,M3.2.0,M11.1.0 1741514400 1762074000 0x4000162160})} {'\x1f' '\U00010265' '䇀' '\x03' '\x00' '\x00' '\x00' '\x00' 'က' 'က' '\x00' '\b' {%!q(int64=1741588746) %!q(int64=92565444)} {%!q(int64=1678764696) %!q(int64=668000040)} {%!q(int64=1678764696) %!q(int64=668000040)} ['\x00' '\x00']}}: open /home/oh-la/.local/share/containers/storage/overlay/1c0de0b539a9c2a4b6dd5d33ec23867a408dcd8493227734f03c6ce9334f7ffc/link: no such file or directory

I have the restic backups going, but based on that error I would have no clue what actually needs restoring. I have tried removing the image, redownloading, multiple reboots, but nothing has helped so far. The rest of my containers are fine which makes this even more puzzling.

Here is another datapoint that seems quite strange, if I podman rm --force the container ID, this goes away, but not liking the idea of that.

oh-la@Collective-Core:~$ podman ps
ERRO[0000] error joining network namespace for container a9b619aa191829efb56ee576784fc005930f2d62105be5880bc5b488c7579d52: error retrieving network namespace at /run/user/2000/netns/cni-803fe804-99de-e9d7-ce2b-f8929f57eb80: unknown FS magic on "/run/user/2000/netns/cni-803fe804-99de-e9d7-ce2b-f8929f57eb80": 1021994 
Error: error joining network namespace of container a9b619aa191829efb56ee576784fc005930f2d62105be5880bc5b488c7579d52: error retrieving network namespace at /run/user/2000/netns/cni-803fe804-99de-e9d7-ce2b-f8929f57eb80: unknown FS magic on "/run/user/2000/netns/cni-803fe804-99de-e9d7-ce2b-f8929f57eb80": 1021994

Trying to stay positive but going to bed with everything humming along, to waking up with everything broken is quite frustrating. Now some stuff is fixed but Z2M being broken is a pretty big deal.

Crisis averted for now. It seems the container cache got corrupted (probably my manual reboot by pulling the plug) and this solved my issue: sudo rm -r /home/oh-la/.local/share/containers

https://docs.oh-lalabs.com/en/user-guide/troubleshooting#errors

2 Likes