Occasional anomalously-high `messages.inbound.rest.messages.refused` stat when a rest publish is refused

Resolved

None impact

Started

Lasted 2h 8m

View official incident ↗

Updates

  1. Resolved

    The fix for this has been fully rolled-out.

  2. Identified

    Since Monday 7th, a REST publish that was refused by Ably's servers (for example, for being above the maximum message size) had a small probability of contributing an incorrect, anomalously high message inbound refused stat (key `messages.inbound.rest.messages.refused`) to app and account stats. Since refused messages do not count towards a customer's chargeable message quota, this bug did not result on anyone being charged more or having their quota used up. However, in a couple of cases it did momentarily trigger a customer's account-wide instantaneous message rate limit (`messages.maxRate`), for which we apologise. We are rolling out a fix for this currently.