incident.ioincident.io is unavailable
Between 10:43 and 10:46 GMT our app was unavailable. This will have meant both the web UI and Slack app were unusable. Sorry for any inconvenience this caused.
Resolved
Outages and recoveries from the services you depend on.
40 incidents
Between 10:43 and 10:46 GMT our app was unavailable. This will have meant both the web UI and Slack app were unusable. Sorry for any inconvenience this caused.
Resolved
This incident has been resolved.
Resolved
We believe this issue has been resolved. Thanks for your patience!
Resolved
Final update to say Slack officially closed their incident at 7:24 PM GMT, while we saw our last errors at 3:45 PM GMT. We don't expect any further interruption to incident.io products as a result of this incident.
Resolved
Slack's incident has been closed, and we're no longer seeing a high error rate.
Resolved
This incident has been resolved.
Resolved
We think the impact from the issue is over.
Resolved
Our API, dashboard, Slack app and any background services have experienced an outage. This incident is being logged retrospectively, as we are now back to full service.
Resolved
Fivetran is back up and running, so Insights are now up-to-date.
Resolved
We think the impact from the issue is over. Heroku have closed their incident, and our monitoring hasn't spotted any issues in a few hours.
Resolved
We believe that the outage is now over, and the website and Slack app are available to everyone.
Resolved
This incident has been resolved.
Resolved
We think the impact from the issue is over.
Resolved
The upstream service we use for data visualisations has now resolved their issue, and insights are available once again.
Resolved
We've not seen any errors for 10 minutes, and we are once again able to consistently create Slack channels. We believe that this incident is now resolved.
Resolved
We’ve shared a technical write-up about the causes of this incident and steps we’ve taken to improve reliability here: [https://incident.io/blog/intermittent-downtime](https://incident.io/blog/intermittent-downtime)
Resolved
We were seeing infrequent bursts of errors due to running out of resources when talking with our database. This is now resolved, with the team having increased our capacity to handle these requests. We've also made changes to improve the performance of the feature that created this pressure, and don't expect the problem to reoccur.
Resolved
A fix has been deployed and issue has been resolved. Insights is now available again.
Resolved
We're no longer seeing errors from the PagerDuty endpoint and have confirmed escalations are polling successfully, meaning escalation messages (created in response to `/inc escalate`) are working once more. We've followed up with PagerDuty to understand what caused this, and improved our retrying logic to make us more robust to future flakiness.
Resolved
Our application is back up, and we're serving traffic as usual. We believe the incident has been fully resolved.
Resolved
The incident has been resolved
Resolved
We're no longer seeing any issues when making requests to Slack. From our logs, we see a period of time between 9:55-10:05am BST during which requests to Slack were failing and we received no webhooks. We expect the incident.io app was functional outside this period, and it appears Slack have now fixed the issue on their side so don't expect this to reoccur.
Resolved
Slack have resolved the incident and we’re no longer seeing errors. We don’t expect any further disruption, thanks for your patience.
Resolved
We think the impact from the issue is over. We believe there was momentary downtime from Slack with their shortcuts while they worked on fixing an ongoing issue on their side.
Resolved
This incident has been resolved.
Resolved
Following the mitigation that we deployed, our monitoring and testing has shown that we're back to full performance across the globe. We don't expect to see any further issues.
Resolved
We've identified the cause of the issue and have deployed a change that ensures the dashboard now loads correctly. We do not expect any further disruption.
Resolved
This incident has been resolved.
Resolved
This incident has been resolved
Resolved
This incident has now been resolved on Linear's side. We'll continue to monitor this to spot any issues.
Resolved
A recent code change has been rolled back
Resolved
We’ve now contacted everyone we believe were affected by this incident. If you’re still having connection issues with Atlassian integrations, you can reconnect by visiting Settings > Integrations (https://app.incident.io/~/settings/integrations) and following the instructions. If you require further support, please get in touch via https://help.incident.io/
Resolved
We've deployed a fix for this issue, and the affected pages within the Dashboard are now loading again.
Resolved
We have resolved the issue and don't expect any further issues with Insights.
Resolved
Slack has been stable for a while now, so we're resolving this incident. If you have any questions, or you are still seeing issues, please get in touch.
Resolved
We are confident that this issue is now resolved. Our app was in a degraded state between 13:38 and 13:52. During this time, the following product areas were affected: * Workflows did not run * Incidents were not auto-created from external sources * Incidents weren’t announced through Slack * Slack messages from our incident.io bot were not posted * Postmortem exports failed Workflows, incident autocreation, and Slack messages will now have processed in the background, so you do not need to retry these actions. During this time, the Slack app and the dashboard were both working as expected.
Resolved
We've deployed a fix which means logos and footers are now displaying correctly on all public status pages.
Resolved
New user registrations are now permitted.
Resolved
AWS RDS cluster instances were automatically rebooted at approximately 09:27 Central (04:27 UTC). We are investigating the issue with the AWS team and will update our statuspage with the findings.
Resolved
This incident has been resolved.
Resolved