Resolved incident

Occasional anomalously-high `messages.inbound.rest.messages.refused` stat when a rest publish is refused

Ably · None · Started

Duration
2h 8m
Severity
None
Status
Resolved

Summary

None · Resolved

The fix for this has been fully rolled-out.

View official incident ↗

Event timeline

How this incident unfolded, from the official status page.

  1. Identified

    Ably

    Since Monday 7th, a REST publish that was refused by Ably's servers (for example, for being above the maximum message size) had a small probability of contributing an incorrect, anomalously high message inbound refused stat (key `messages.inbound.rest.messages.refused`) to app and account stats. Since refused messages do not count towards a customer's chargeable message quota, this bug did not result on anyone being charged more or having their quota used up. However, in a couple of cases it did momentarily trigger a customer's account-wide instantaneous message rate limit (`messages.maxRate`), for which we apologise. We are rolling out a fix for this currently.

  2. Resolved

    Ably

    The fix for this has been fully rolled-out.