Origin One - Issues with AWS N. Virginia Infrastructure – Incident details

All systems operational

Issues with AWS N. Virginia Infrastructure

Resolved
Major outage
Started 9 months agoLasted about 14 hours

Affected

Platform

Major outage from 9:15 AM to 9:45 AM, Operational from 9:45 AM to 12:18 PM, Major outage from 12:18 PM to 10:44 PM, Operational from 10:44 PM to 11:05 PM

Updates
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Monitoring
    UTC
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Update
    UTC
    Update
    We are continuing to work on a fix for this incident.
  • Update
    UTC
    Update
    We are continuing to work on a fix for this incident.
  • Update
    UTC
    Update
    We are continuing to work on a fix for this incident.
  • Update
    UTC
    Update

    We're still continuing to setup alternative infrastructure running in a different region.

    More updates to follow

  • Update
    UTC
    Update

    Efforts to getting the new infrastructure up is still on-going, ETA is still unknown. Another update to be sent as soon as possible.

  • Update
    UTC
    Update

    We are continuing to work on a fix for this incident to restore the system functionality.

  • Update
    UTC
    Update
    We are continuing to work on a fix for this incident.
  • Update
    UTC
    Update

    We continue working to resolve the issue and return service to normal.

    More updates to follow.

  • Identified
    UTC
    Identified

    The platform is again experiencing an issue: any user opening the platform will see an error page "Sorry, we can’t find that page"

    We are working on the issue and will send an update again shortly.

  • Update
    UTC
    Update

    No changes, but we continue to monitor the situation.

  • Update
    UTC
    Update

    Systems are not fully recovered and the incident is still open on AWS however we don't detect any impacted functionality.

    We will continue to monitor the situation until AWS resolves the issue on their status page. Updates will now only follow every hour unless there's a significant update in the mean time.

  • Monitoring
    UTC
    Monitoring

    We are seeing recovery, with users now able to log into the Origin One platform.

    We will continue to monitor the situation.

  • Identified
    UTC
    Identified

    Our infrastructure provider is experiencing platform wide issues in the N. Virginia region and is affecting our system in the following ways:

    • Unable to log into the platform

    • Some test results communicated to Origin One from labs can fail

    We will provide another update shortly.