[PHX] Node issue

Resolved
Resolved

This incident has been resolved.

Recovering

The original VM is back online post hardware issue and will continue to operate. We're updating DR snapshots and prepping a new node to migrate to in another method.

In any case, we do not expect any further service disruptions

Updated

Unfortunately we are experiencing network issues due to the original machine having the hardware issue.

We are working on this now

Updated

We are finalizing the live migration now, there may be a brief 5-10 minute window of slowness/inaccessibility.

Updated

We have completed an up to date DR snapshot as of 9:40AM and are in the process of live migrating the VM to a new hardware node.

Updated

The node is back online at the moment, we're working to move it to a new machine to prevent any future issues.

Identified

The data center has acknowledged that they are going to swap the RAM on the server as it appears to be having an issue preventing the server from booting.

Investigating

We are currently investigating a boot issue with phx23 in Phoenix.

We will have updates ASAP

Began at:

Affected components
  • Rocket.net - Data Centers
    • US - Phoenix