Unplanned Network Outage

Incident Report for Anchor Systems

Postmortem

On Friday, 12th of March at 3:39 am, one of two routers in a redundant pair failed. Partial failover to the backup router took place but due to invalid configuration on the backup router, some redundant gateway (VRRP) interfaces did not come online.

By 4:43 am the problem was identified and a fix to the redundant router configuration was put in place.

Emergency network maintenance was scheduled on Saturday morning to permanently remediate the problem and restore full redundancy. During this maintenance, gateways were moved to a new pair of routers.

Posted Mar 16, 2021 - 09:17 AEDT

Resolved

This incident has been resolved. Emergency network maintenance will be scheduled early tomorrow morning (13/03/2021) to permanently remediate the root cause. A post-mortem will be supplied to the affected customers by Tuesday morning 16/03/2021.

Posted Mar 12, 2021 - 08:02 AEDT

Monitoring

A fix has been implemented and we are monitoring the results.

Posted Mar 12, 2021 - 05:43 AEDT

Identified

Our network engineers have identified the issue and a fix is being implemented.

Posted Mar 12, 2021 - 04:34 AEDT

Investigating

An unplanned short-duration network outage has affected some internal services and our Network engineers are currently investigating the cause.

Posted Mar 12, 2021 - 04:03 AEDT