Game Server & Database Backups Failed — July 14, 2026

Incident Report for Hyper Layer

Update

Backups for game servers and game server databases are still failing. Our upstream storage provider has confirmed the outage is ongoing.

The issue was triggered when new hardware was added to their storage cluster, which unexpectedly degraded IO performance to the point of being unusable. Removing that hardware did not resolve the problem, as the cluster had already redistributed data onto the new disks. Their investigation points to a software incompatibility as the underlying cause.

Additional disks have since been added to relieve pressure on the cluster. While this has not yet fully resolved the issue, further capacity is being added today, which is expected to progressively restore the platform over the course of the day.

**Is my data safe?**

Yes. Our upstream provider has confirmed your data remains intact and is not at risk. This remains a performance and access issue, not a data-integrity issue.

**What we're doing**

We continue to stay in direct contact with our upstream provider, whose engineers are working around the clock alongside specialist storage experts to stabilise the platform. As soon as upstream access is fully restored, we will resume normal backup operations and run catch-up backups for the missed schedule.

**What you need to do**

No action is required. Your running game servers remain unaffected — only backup generation is impacted.

In the meantime, customers with game servers can still take manual backups using the snapshot feature, which is stored on a separate server unaffected by this outage. Please note that snapshots do not back up game server databases hosted on za.dbhost.hyperlayer.net. We continue to recommend avoiding major world resets or high-risk changes until backups resume.

**Next update**

We will post again once upstream service is fully restored or as soon as further information becomes available. We apologise for the continued inconvenience and appreciate your patience.
Posted Jul 15, 2026 - 09:56 SAST

Identified

We are currently experiencing a backup failure affecting all game servers and game server databases. Backups scheduled for July 14, 2026 have not completed successfully.

This issue originates with our upstream storage provider, who identified a storage-layer performance issue on their platform and temporarily disabled client access while they work to resolve it. As a direct result, our backup jobs against that storage platform are failing.

Is my data safe?

Yes. Our upstream provider has confirmed this is a performance/access issue, not a data-integrity issue — no data has been lost or corrupted on their end. Your live game server data is unaffected; it is only the backup process itself that is currently unable to complete.

What we're doing

We are in direct contact with our upstream storage provider, who has their engineering team working alongside specialist Ceph storage experts to restore access.
We are monitoring the situation closely and will resume normal backup operations as soon as upstream access is restored.
We will attempt to run catch-up backups once service is restored.


What you need to do

No action is required. This does not affect your running game servers — only backup generation is impacted for today. We recommend avoiding major world resets or high-risk changes until backups resume, out of an abundance of caution.

Next update

We will post an update as soon as we have further information from our upstream provider, or once backups have resumed successfully.


We apologize for the inconvenience and appreciate your patience while our upstream provider works through this issue.
Posted Jul 14, 2026 - 15:02 SAST
This incident affects: ZA NODES (OFFSITE BACKUP).