Some customers are not able to login to our UI

Postmortem

Major impact

Started

Lasted 54m

View official incident ↗

Updates

  1. Postmortem

    Start Time: Thursday, October 7, 2021, at 17:52 UTC End Time: Thursday, October 7, 2021, at 18:46 UTC Duration: 0:54:00 ## What happened: Our Web UI returned the error message “This site can’t be reached” when some users tried to login or load pages. The ingestion of logs was unaffected. ## Why it happened: The Telia carrier service in Europe experienced a major network routing outage caused by a faulty configuration update. The routing policy contained an error that impacted traffic to our service hosting provider, Equinix Metal. The Washington DC data center that houses our services was impacted. During this incident the [app.logdna.com](http://app.logdna.com) site was unreachable for some customers, depending on their location. ## How we fixed it: No remedial action was possible by LogDNA. We waited until the incident from Equinix Metal, our service hosting provider, was resolved. ## What we are doing to prevent it from happening again: For this type of incident, LogDNA cannot take proactive preventive measures.

  2. Resolved

    Logins are working for all customers. All services are operational.

  3. Monitoring

    Logins to our UI appear to be working again for all customers. We are monitoring for any further failures.

  4. Identified

    Some customers are not able to login to our UI. It appears this is due to an incident with our cloud provider Equinix. See their status page https://status.equinixmetal.com/incidents/wgg6kl862tl6.