OnlineOrNot https://onlineornot.com OnlineOrNot monitors your websites and APIs so you get notified instantly when they go down. Fri, 04 Sep 2026 10:35:05 GMT https://validator.w3.org/feed/docs/rss2.html https://github.com/jpmonette/feed en OnlineOrNot https://onlineornot.com/onlineornot-brand-og.png https://onlineornot.com All rights reserved 2026, Max Rozen <![CDATA[7 best Atlassian Statuspage alternatives for 2026]]> https://onlineornot.com/atlassian-statuspage-alternative https://onlineornot.com/atlassian-statuspage-alternative Wed, 02 Sep 2026 00:00:00 GMT Atlassian Statuspage is one of the best-known tools for communicating outages and scheduled maintenance. It gives customers a place to check service health instead of asking your support team whether something is down.

But it is not the right fit for every team.

You may want an Atlassian Statuspage alternative that includes monitoring, costs less as your audience grows, is easier to configure, or can be self-hosted. The difficult part is comparing complete incident workflows rather than screenshots and starting prices.

This guide compares seven alternatives, including hosted, free, and open-source options.

Table of contents

Atlassian Statuspage alternative: the short answer

OnlineOrNot is a strong Atlassian Statuspage alternative for SaaS companies and small engineering teams that want uptime monitoring and status pages in one product.

OnlineOrNot provides:

  • Public and private status pages
  • Uptime, API, heartbeat, DNS, TCP, and browser monitoring
  • Automatic incidents connected to monitoring
  • Manual incidents and scheduled maintenance
  • Custom domains with automatic HTTPS
  • Unlimited email subscribers on paid plans
  • A limited free status page and a 14-day trial of paid features

Paid plans start at $12/month with annual billing or $15/month billed monthly. The first public status page is included, without per-subscriber fees.

Create your status page or compare OnlineOrNot pricing.

Quick comparison

| Tool | Monitoring included | Free option | Private pages | Best for | |------|---------------------|-------------|---------------|----------| | OnlineOrNot | Yes | Yes | Yes | SaaS teams wanting monitoring and status pages together | | Better Stack | Yes | Yes | Yes | Teams wanting a broader incident-response platform | | Instatus | Yes | Yes | Plan-dependent | Teams prioritizing a status-page-first workflow | | OpenStatus | Yes | Yes / open source | Hosted add-on or DIY | Developers wanting open source with managed hosting | | StatusPal | Via integrations | Trial | Yes | Teams wanting dedicated incident communication | | UptimeRobot | Yes | Yes | Plan-dependent | Existing UptimeRobot users and basic monitoring | | Uptime Kuma | Yes | Self-hosted | DIY | Home labs and teams comfortable operating infrastructure |

Features and limits change. Use this table to build a shortlist, then confirm that the current plan includes the page count, subscriber limit, access controls, and notification channels you need.

What does Atlassian Statuspage do?

Atlassian Statuspage is incident communication software. It publishes the health of services and components, active incidents, scheduled maintenance, metrics, and previous incidents.

Atlassian offers three broad page types:

  • Public pages for customer-facing services
  • Private pages for employees or selected users
  • Audience-specific pages that show different components to different customers

Statuspage is primarily a communication product. Teams typically connect monitoring and incident-management systems through integrations or APIs to automate component states and incident updates.

That model works well for large organizations with established Atlassian workflows. Smaller teams may prefer a product where monitoring, alerts, and customer communication share the same data from the start.

Why consider an alternative?

Teams usually compare Atlassian Statuspage alternatives for four reasons.

Monitoring and communication live in separate products

A status page is only useful when it reflects what is actually happening.

If monitoring lives somewhere else, your team must maintain an integration or update incidents manually. A combined platform can detect an outage, alert the team, change affected components, and publish an incident from the same source of truth.

Automation should not remove human control. Monitoring can report that an API is unavailable, but a person still needs to explain customer impact, available workarounds, and when the next update will arrive.

The full price is higher than the starting price

The cheapest plan rarely represents a growing team's actual configuration.

Compare costs after adding:

  • Public and private pages
  • Team members or responders
  • Email subscribers
  • Custom domains and branding
  • SSO, IP allowlists, or passwords
  • SMS and phone notifications
  • Monitoring and incident-management tools

A tool with a higher starting price can still cost less if it replaces separate monitoring and status-page subscriptions.

The workflow is more complex than the team needs

Large incident-management platforms support approval processes, audience segmentation, and enterprise controls. Those features are valuable when you need them and extra administration when you do not.

A small SaaS team may only need reliable monitoring, automatic incidents, manual updates, and subscriber notifications.

You want control over hosting

Open-source tools let you control infrastructure and data. The tradeoff is responsibility for deployment, upgrades, security, backups, email delivery, and availability.

A self-hosted status page should run independently from the application it reports on. Otherwise, the page may disappear during the exact outage it is supposed to explain.

1. OnlineOrNot

OnlineOrNot combines hosted status pages with uptime, API, heartbeat, DNS, TCP, and Playwright browser monitoring.

Connect a monitor to a status-page component and OnlineOrNot can automatically create and resolve incidents when its state changes. Your team can still post manual incidents, progress updates, retrospective incidents, and scheduled maintenance.

The status page is served separately from your application on Cloudflare's global edge network, so your incident communication does not depend on your production application remaining available.

What OnlineOrNot does well:

  • Monitoring and status pages share the same data - Reduce integration work and avoid conflicting service states.
  • Public and private pages - Publish a customer-facing page or protect an internal page with a password or IP allowlist.
  • Custom domains with automatic HTTPS - Use a domain such as status.example.com without managing certificates yourself.
  • Unlimited subscribers on paid plans - Customers receive incident updates without per-subscriber fees.
  • Automatic and manual incidents - Automate clear outages while keeping people in control of customer communication.
  • Third-party components - Show the health of important providers alongside services you monitor directly.
  • API and webhooks - Connect status-page workflows to your existing tooling.

Where it may not fit:

  • OnlineOrNot does not provide log aggregation or distributed tracing.
  • Atlassian may suit large organizations better when they require complex audience-specific pages or deeply integrated Atlassian approval workflows.

Pricing: A limited public status page is available on the free plan. Paid plans start at $12/month annually or $15/month monthly and include the first public status page. Additional public and private pages are priced separately.

Best for: SaaS companies, developers, startups, and small teams that want monitoring and customer communication in one product.

2. Better Stack

Better Stack combines status pages with uptime monitoring, on-call scheduling, incident management, logs, and other observability products.

It is a strong option when a status page is one part of a broader response workflow. The same vendor can detect an incident, alert the on-call engineer, and communicate with customers.

What it does well:

  • Built-in monitoring and incident response
  • On-call schedules and escalation policies
  • Public and private status pages
  • Integrations with existing monitoring tools
  • A broader logs and observability platform

Tradeoffs: The wider platform may introduce more product and pricing complexity than a team needs for straightforward monitoring and customer communication.

Best for: Teams that want monitoring, on-call response, logs, and status pages from one vendor.

3. Instatus

Instatus is a status-page-first product that also offers uptime monitoring and incident-response features.

Its focus is creating polished customer-facing pages quickly. Teams can start with a free public page and add custom domains, faster monitoring, private pages, and more notification options on paid plans.

What it does well:

  • Fast status-page setup
  • Polished customer-facing design
  • Built-in monitoring
  • Incident and maintenance communication
  • Free entry point

Tradeoffs: Custom domains, subscriber limits, private pages, monitoring frequency, and SSO depend on the selected plan. Teams requiring advanced browser or API monitoring may still need another product.

Best for: Startups and software teams that prioritize a simple, status-page-first workflow.

4. OpenStatus

OpenStatus is an open-source uptime monitoring and status page platform with a managed cloud service.

It gives developers a choice: use hosted plans to avoid operational work or self-host for more control. The product includes global monitoring, incidents, subscribers, themes, and status pages.

What it does well:

  • Open-source codebase
  • Managed and self-hosted options
  • Monitoring and status pages together
  • Developer-focused workflow

Tradeoffs: Monitor intervals, regions, page counts, white-labeling, and access controls vary by hosted tier or add-on. Self-hosting makes your team responsible for availability and maintenance.

Best for: Developers who want open-source software without giving up the option of managed hosting.

5. StatusPal

StatusPal is a dedicated status page and incident communication product. It supports public and private pages, custom domains, subscriber notifications, and integrations with monitoring and incident-management tools.

It is closer to the standalone Statuspage model than a monitoring-first product. That can be an advantage when your team already has detection and response tools it wants to keep.

What it does well:

  • Dedicated incident communication workflow
  • Public and private pages
  • Custom branding and domains
  • Monitoring and incident-management integrations
  • Subscriber notifications

Tradeoffs: Teams buying both monitoring and a status page should compare the combined cost and integration work with an all-in-one option.

Best for: Organizations that want a dedicated communication layer connected to their existing monitoring stack.

6. UptimeRobot

UptimeRobot is best known for uptime monitoring and also provides hosted status pages.

It is a practical starting point for personal projects and small teams that already use UptimeRobot monitors. Monitor state and uptime history can appear on a public page without connecting another vendor.

What it does well:

  • Free way to start
  • Monitoring and status pages together
  • Familiar, straightforward interface
  • Public incident and maintenance communication

Tradeoffs: Check the current plan limits for custom domains, branding, subscribers, team access, and private pages. The free monitoring interval is more appropriate for low-risk projects than critical production services.

Best for: Personal projects, small websites, and existing UptimeRobot customers who need a basic page.

7. Uptime Kuma

Uptime Kuma is a popular open-source monitoring tool with built-in status pages.

There is no SaaS subscription or per-monitor charge. You run the software on your own infrastructure and control its data and configuration.

What it does well:

  • Free and open source
  • Many monitor types
  • Built-in public status pages
  • Large community
  • Full control over hosting and data

Tradeoffs: Your team owns hosting, upgrades, backups, security, TLS, access control, and notification delivery. A single self-hosted instance also provides less geographic verification than a managed multi-region monitoring service.

Best for: Home labs, internal services, and technical teams comfortable operating their own monitoring stack.

Free and open-source Atlassian Statuspage alternatives

Searches for a "free Atlassian Statuspage alternative" often mix two different categories.

Free hosted plans

OnlineOrNot, Better Stack, Instatus, OpenStatus, and UptimeRobot offer ways to start without paying. These plans normally limit some combination of subscribers, monitors, team members, check frequency, private access, or custom domains.

A free hosted plan is useful when you want to test the incident workflow without operating infrastructure.

Free open-source software

Uptime Kuma and OpenStatus can be self-hosted. Other projects can be found in GitHub collections such as awesome-status-pages.

The software may be free, but operating it is not. Include server costs and the time required for:

  • Upgrades and security patches
  • Backups and recovery testing
  • Email or SMS delivery
  • DNS and TLS configuration
  • Monitoring the monitoring system
  • Keeping the page available during primary infrastructure failures

Reddit discussions about Statuspage alternatives often recommend self-hosting because the license costs nothing. That can be the right decision for a home lab or infrastructure-focused team. For a small SaaS, a managed page may cost less than the engineering time needed to run one reliably.

How to choose an Atlassian Statuspage alternative

Choose based on what happens during an outage, not only what the finished page looks like.

1. Decide whether monitoring should update the page

If you already have reliable monitoring, a standalone status page with strong integrations may be enough.

If you are buying both, a product with built-in monitoring can reduce setup and eliminate conflicting incident states. Look for the ability to automate component changes while preserving manual control over customer-facing messages.

2. Keep the status page independent

Your status page should not share a likely failure mode with the application it reports on.

For self-hosted products, consider the cloud account, region, DNS provider, deployment pipeline, and network dependencies. For hosted products, understand which dependencies remain under your control, especially custom-domain DNS.

3. Compare the complete price

Calculate the price for the configuration you expect to use, including:

  • Status pages
  • Monitors and check frequency
  • Team members
  • Subscribers
  • Private access
  • Custom domains
  • Notification channels
  • SSO and audit logs
  • Data retention

4. Test the incident workflow

Use the trial to rehearse a real incident:

  1. A monitor fails
  2. The team receives an alert
  3. An incident opens
  4. Affected components change state
  5. Subscribers receive an update
  6. An operator adds context
  7. The service recovers
  8. The incident resolves and remains in history

Also test scheduled maintenance and a manual incident. The interface your team uses under pressure matters more than the marketing page.

5. Check access and subscriber requirements

Public pages work for most customer-facing services. Internal systems and enterprise customers may require a password, IP allowlist, email authentication, or SSO.

Do not assume every product uses the same definition of "private page." Confirm that the access method and subscriber limits fit your actual audience.

How to migrate from Atlassian Statuspage

A migration does not need to interrupt customer communication.

  1. Audit the existing page. List components, groups, incidents, maintenance windows, metrics, subscribers, integrations, and access controls.
  2. Create the replacement page. Match the current component structure before trying to improve it.
  3. Connect monitoring. Map monitors to components and decide which incidents can be automated safely.
  4. Configure branding and access. Set the logo, colors, custom domain, password, IP allowlist, or SSO requirements.
  5. Test notifications. Confirm that incidents, updates, maintenance, and resolutions reach the intended subscribers.
  6. Rehearse an outage. Test both automatic and manual publishing with the people who will operate the page.
  7. Move the custom domain. Lower DNS TTL in advance, change the record, and verify HTTPS before announcing the migration.
  8. Retire the old page carefully. Keep a record of historical incidents and redirect old links where possible.

For OnlineOrNot, start with the guide to create a status page, then link uptime monitoring to the page.

FAQ

What are the best alternatives to Atlassian Statuspage?

OnlineOrNot is a strong choice for teams that want monitoring and status pages together. Better Stack suits broader incident-response requirements, Instatus and StatusPal focus on status-page workflows, OpenStatus offers open-source and hosted options, UptimeRobot provides a simple free starting point, and Uptime Kuma is popular for self-hosting.

Is Atlassian Statuspage free?

Atlassian offers a limited free plan for public status pages. Paid public plans increase limits and customization, while private and audience-specific pages use separate pricing. Confirm current features and prices on Atlassian's official site before purchasing.

Who competes with Atlassian Statuspage?

Competitors include OnlineOrNot, Better Stack, Instatus, OpenStatus, StatusPal, UptimeRobot, Status.io, Hyperping, and open-source projects such as Uptime Kuma and Cachet.

What is the best free Atlassian Statuspage alternative?

OnlineOrNot, Better Stack, Instatus, OpenStatus, and UptimeRobot all provide hosted ways to start for free, with different limits. Uptime Kuma is a strong free self-hosted option when your team is prepared to operate it.

The best choice depends on whether you need monitoring, a custom domain, private access, faster checks, multiple team members, or more subscribers.

What is the best open-source Statuspage alternative?

Uptime Kuma is a popular option when you want monitoring and a public page in one self-hosted product. OpenStatus combines an open-source codebase with managed hosting. Cachet is another option when you want a dedicated status page and already have monitoring.

Should monitoring and status pages use the same platform?

They do not have to, but combining them reduces integration work and makes automatic component and incident updates easier.

The status page itself should still be delivered independently from the application being monitored. A shared product workflow is useful; shared failure infrastructure is not.

How much does an OnlineOrNot status page cost?

OnlineOrNot includes a limited public status page on its free Hobby plan. Paid plans start at $12/month with annual billing or $15/month billed monthly and include the first public status page. Paid plans include unlimited email subscribers.

When should I keep Atlassian Statuspage?

Keep Atlassian Statuspage when it already meets your requirements, your team depends on Atlassian integrations, or you need mature audience-specific communication and enterprise governance.

Switching tools introduces migration and training work. An alternative should solve a concrete pricing, monitoring, access, or workflow problem rather than simply offering a different interface.

Create a status page connected to monitoring

OnlineOrNot gives customers one place to check service health while helping your team detect outages and publish updates from the same product.

Start a 14-day trial or learn more about OnlineOrNot status pages.

]]> <![CDATA[10 best status page tools for 2026 (hosted and open source)]]> https://onlineornot.com/status-page-tools https://onlineornot.com/status-page-tools Wed, 02 Sep 2026 00:00:00 GMT Status page tools give customers one place to check whether your service is working, what went wrong, and when they can expect another update. The best status page software also connects that communication to monitoring, incident response, and subscriber notifications.

The page itself is the easy part. The differences show up during a real incident: whether monitoring can update the page automatically, how quickly your team can publish an explanation, who gets notified, and whether the page stays available when your own infrastructure does not.

This guide compares the status page tools worth considering in 2026, with a focus on practical tradeoffs: built-in monitoring, incident workflows, custom domains, subscribers, private pages, pricing, and who each tool is best for.

Table of contents

Quick comparison table

| Tool | Free option | Monitoring included | Custom domain | Private pages | Best for | |------|-------------|---------------------|---------------|---------------|----------| | OnlineOrNot | Yes | Yes | Yes | Yes | SaaS teams wanting monitoring and status pages together | | Atlassian Statuspage | Yes | Via integrations | Paid plans | Yes | Large organizations in the Atlassian ecosystem | | Better Stack | Yes | Yes | Yes | Yes | Teams wanting monitoring, on-call, and incident response | | Instatus | Yes | Yes | Paid plans | Higher tiers | Teams prioritizing a polished status-page workflow | | UptimeRobot | Yes | Yes | Paid plans | Plan-dependent | Personal projects and basic monitoring | | Uptime Kuma | Self-hosted | Yes | DIY | DIY | Home labs and teams comfortable self-hosting | | OpenStatus | Yes / open source | Yes | Paid hosted plans or DIY | Paid add-on or DIY | Developers wanting open source with hosted plans | | Cachet | Self-hosted | Via API | DIY | DIY | Teams wanting a dedicated open-source status page | | Status.io | Trial | Via integrations | Yes | Higher tiers | Organizations with complex communication needs | | Hyperping | Yes | Yes | Paid plans | Paid plans | Small teams wanting monitoring, on-call, and status pages |

Features and plan limits change. Use this table as a shortlist, then confirm that the current plan includes the subscriber count, access controls, and notification channels you need.

What is a status page tool?

A status page tool publishes the current health of your website, API, application, or infrastructure. It normally displays components, active incidents, planned maintenance, uptime history, and a way for users to subscribe to updates.

A useful status page answers four questions:

  1. Is the service working?
  2. Which components are affected?
  3. Does the team know about the problem?
  4. When will the next update arrive?

There are three common approaches.

Standalone status page software

A standalone tool focuses on incident communication. Your team posts updates manually or connects monitoring and incident-management tools through integrations.

This works well when the people detecting incidents and the people communicating with customers use different systems.

Status pages with built-in monitoring

These tools monitor your services and can automatically open or resolve incidents when a check changes state.

This reduces the chance that your team fixes an outage but forgets to update the status page. It also means one product can own uptime checks, alerts, incident history, and customer communication.

Self-hosted status pages

Open-source tools give you control over hosting, data, and customization. The tradeoff is operational responsibility: deployment, upgrades, backups, email delivery, security, and availability are now your problem.

A status page should be independent from the application it reports on. If both fail together, customers lose the place they were supposed to check for updates.

Best for most teams: OnlineOrNot

OnlineOrNot combines hosted status pages with uptime, API, browser, and heartbeat monitoring.

It is built for developers and small software teams that want monitoring to drive incident communication without adopting a larger enterprise incident-management suite.

What it does well:

  • Monitoring and status pages share the same data - Connect an uptime check or heartbeat to a component, then automatically create and resolve incidents when its state changes.
  • Public and private pages - Publish a customer-facing page or protect an internal page with a shared password or IP allowlist.
  • Custom domains and automatic HTTPS - Host the page on your own domain without managing certificates yourself.
  • Unlimited subscribers - Customers can subscribe to incident updates by email without creating an OnlineOrNot account. RSS feeds are also available.
  • Third-party components - Display the health of dependencies alongside services you monitor directly, so customers get more context during provider outages.
  • Manual control when it matters - Publish manual incidents, updates, retrospective incidents, and scheduled maintenance alongside automated incidents.
  • API and webhooks - Manage pages and incident workflows programmatically instead of relying only on the dashboard.

What it does not do:

  • No log aggregation or distributed tracing
  • Less suitable than a dedicated enterprise platform if you need complex audience-specific pages or heavily customized approval workflows

Pricing: A limited public status page is available on the free plan. Paid plans start at $12/month with annual billing or $15/month billed monthly and include the first public status page. Additional public and private pages are priced separately.

Best for: SaaS companies, developers, startups, and small teams that want uptime monitoring and customer communication in one product.

Best for large enterprises: Atlassian Statuspage

Atlassian Statuspage is one of the best-known dedicated status page products. It offers public, private, and audience-specific pages, with integrations across the Atlassian ecosystem.

What it does well:

  • Mature incident communication - Components, incidents, scheduled maintenance, metrics, and subscriber notifications are established parts of the product.
  • Several page types - Public, private, and audience-specific pages cover customer-facing and internal use cases.
  • Atlassian integrations - A natural fit for teams already coordinating incidents in Jira Service Management, Opsgenie, and other Atlassian products.
  • Enterprise controls - Higher tiers support the governance and access requirements larger organizations expect.
  • Free public-page entry point - The free plan is enough to test the basic workflow with limited subscribers and team members.

Where it falls short:

  • Monitoring is not the main product - Most teams connect separate monitoring or incident-management systems to automate component status.
  • Costs rise quickly - Subscriber limits, team members, private pages, and enterprise requirements can make it expensive.
  • More workflow than small teams need - A two-person SaaS team may not benefit from the extra administration.

Pricing: Free public plan available. Paid public plans start at $29/month, with private and audience-specific pages using separate pricing.

Best for: Larger organizations, especially teams already using Atlassian for service management and incident response.

For a closer look at migration tradeoffs, compare the best Atlassian Statuspage alternatives.

Best for incident response: Better Stack

Better Stack combines status pages with uptime monitoring, on-call scheduling, incident management, logs, and observability products.

This makes it a strong choice when a status page is one part of a broader response workflow rather than a standalone communication page.

What it does well:

  • Incident response built in - Monitoring alerts, escalation policies, on-call schedules, and status updates can live in one platform.
  • Built-in uptime monitoring - The same system that detects a problem can update component status and start the incident workflow.
  • Custom domains on the free plan - Small teams can publish a branded page without immediately paying for a status-page-only plan.
  • Private status pages - SSO, password, and IP protection support internal teams and enterprise customers.
  • Broad integrations - Connect existing monitoring and observability tools if Better Stack is not your only source of incident data.

Where it falls short:

  • A broad platform can add complexity - If you only want a simple status page, logs, tracing, and on-call features may be more than you need.
  • Pricing depends on the wider stack - Compare the combined cost of monitors, responders, data, and status-page features you will use.

Pricing: Free plan available. Paid pricing depends on monitoring, incident management, team, and observability usage.

Best for: Teams that want status pages, monitoring, and on-call incident response under one vendor.

Best status-page-first option: Instatus

Instatus is a status-page-first product that has expanded into uptime monitoring, alerts, and on-call workflows.

It suits teams that care most about quickly publishing a polished page and want basic monitoring included rather than connected from another vendor.

What it does well:

  • Fast setup - The product is centered on creating and maintaining status pages rather than configuring a broad observability stack.
  • Useful free plan - The free tier includes a public status page, monitoring, team members, and a limited subscriber count.
  • Built-in monitoring - Paid tiers provide faster checks and more monitors, reducing the need for a separate uptime product.
  • Incident notifications - Email, SMS, calls, and on-call features are available as you move through the plans.
  • Polished customer-facing pages - A strong choice when design and simple communication workflows are the priority.

Where it falls short:

  • Important features are plan-dependent - Custom domains, private pages, SAML SSO, subscriber limits, and monitor counts depend on the tier.
  • Monitoring depth is lighter than a monitoring-first platform - Teams needing browser checks, richer API assertions, or broader observability may still need another tool.

Pricing: Free plan available. Paid plans add faster monitoring, custom domains, more subscribers, and private pages.

Best for: Startups and software teams wanting a polished, status-page-first product with monitoring included.

Best free starting point: UptimeRobot

UptimeRobot combines its well-known uptime monitoring with hosted status pages for sharing service health, incidents, and maintenance updates.

It is a practical starting point for personal projects and small teams that already use UptimeRobot monitors and want a basic public page without adopting a separate incident communication product.

What it does well:

  • Free way to start - Create a basic status page alongside free uptime monitors.
  • Monitoring is already connected - Display monitor state and uptime history without wiring together separate products.
  • Simple public communication - Share incidents and planned maintenance from a familiar dashboard.
  • Customizable pages - Paid plans add more branding and status page features.

Where it falls short:

  • Less incident workflow depth - Dedicated status page and incident-response products offer richer subscriber, access-control, and collaboration features.
  • Important features depend on the plan - Check the current limits for custom domains, branding, subscribers, and private access.
  • Free monitoring is slower - The free monitoring interval is better suited to personal and low-risk projects than critical production services.

Pricing: Free plan available. Paid plans add faster monitoring and more advanced status page features.

Best for: Personal projects, small websites, and existing UptimeRobot users who need a straightforward public page.

Best open-source and self-hosted status page tools

Open-source status page software gives you control over deployment, data, and customization. It also makes your team responsible for availability, upgrades, security, backups, and notification delivery.

Uptime Kuma

Uptime Kuma is a popular open-source monitoring tool with built-in status pages.

It has a friendly interface, supports many monitor types, and does not charge per monitor. The tradeoff is that your team runs the infrastructure behind both monitoring and incident communication.

What it does well:

  • Free and open source - There is no SaaS subscription or per-monitor fee.
  • Monitoring included - Supports HTTP, TCP, DNS, Docker, ping, keyword, and other monitor types.
  • Simple public status pages - Group monitors and show their current state and history without adopting another tool.
  • Data control - Useful for home labs, private networks, and teams that prefer to host their own operational data.
  • Large community - It is widely used and actively maintained.

Where it falls short:

  • You own reliability - Hosting, upgrades, backups, security, and notification delivery are your responsibility.
  • Independence takes planning - Hosting Uptime Kuma beside the application it monitors creates a shared failure point.
  • Advanced communication workflows are limited - Dedicated products usually offer richer subscriber management, private access, incident editing, and enterprise controls.
  • Custom domains and access control are DIY - Expect to configure your own DNS, reverse proxy, TLS, and authentication where required.

Pricing: Free software. You pay for infrastructure and the time required to operate it.

Best for: Self-hosters, home labs, internal services, and technical teams comfortable maintaining their own monitoring stack.

OpenStatus

OpenStatus is an open-source uptime monitoring and status page platform with a managed cloud service.

The hosted product combines global monitoring, incidents, subscribers, custom themes, and status pages. Self-hosting gives technical teams more control, while paid hosted plans remove much of the operational work.

The main tradeoff is plan structure: monitor intervals, region coverage, page counts, white-labeling, and access controls vary by tier or add-on.

Best for: Developers who value open source but still want the option of a managed service.

Cachet

Cachet is a dedicated open-source status page system. It supports components, incidents, planned maintenance, metrics, multiple languages, and API-based automation.

Unlike Uptime Kuma, Cachet is status-page-first rather than monitoring-first. You normally connect your monitoring through its API or other automation.

You are responsible for hosting, upgrades, email delivery, backups, and keeping the page available during incidents.

Best for: Technical teams that want a self-hosted, dedicated status page and already have monitoring.

Other hosted status page tools to consider

Status.io

Status.io is a hosted incident communication platform aimed at organizations that need detailed components, subscriber notifications, metrics, automation, and private-page controls.

It supports integrations with existing monitoring tools rather than trying to replace the rest of your observability stack. White-labeling, access controls, audit trails, and subscriber compliance tools make it more relevant to larger organizations.

There is no permanent free plan, and entry pricing is higher than lightweight status page products.

Best for: Organizations with complex public or private incident communication requirements.

Hyperping

Hyperping combines uptime monitoring, status pages, incident management, and on-call scheduling.

Its free plan is useful for basic monitoring and a simple page. Paid plans add faster checks, custom domains, more subscribers, browser checks, and escalation policies.

Pricing grows with monitors, seats, and page requirements, so compare the full configuration rather than only the starting plan.

Best for: Small teams wanting a modern monitoring and status page product with on-call features.

How to choose a status page tool

The best tool is not necessarily the one with the longest feature list. It is the one your team can update accurately while an incident is unfolding.

1. Decide whether monitoring should update the page

Manual updates give your team complete control, but someone has to remember to publish them.

Automatic updates work well for clear failures such as an unreachable API or a missed heartbeat. They are less useful when an incident needs context before customers see it.

A practical setup combines both:

  • Monitoring changes the affected component state
  • Automation creates or resolves straightforward incidents
  • A human writes context, scope, workarounds, and the next update time
  • Your team can pause or override automation when needed

If you already have reliable monitoring, a standalone status page with good integrations may be enough. If you are buying both, a monitoring-first tool usually requires less setup.

2. Keep the status page outside your own infrastructure

A status page is most valuable when your application is unavailable. Hosting both in the same account, region, cluster, or deployment pipeline creates a shared failure mode.

For self-hosted pages, use separate infrastructure and test what happens when your primary systems fail. For hosted products, consider which infrastructure and DNS dependencies the page shares with your application.

3. Compare subscriber limits and notification channels

Some tools look inexpensive until you add customers.

Check the limits and extra costs for:

  • Email subscribers
  • SMS or phone notifications
  • Webhooks
  • Slack and Microsoft Teams
  • RSS or Atom feeds
  • Per-component subscriptions
  • Subscriber imports and exports

If your status page is public but nobody sees updates, it only solves half the problem.

4. Choose the right access model

Public pages work for most customer-facing products. Internal services and enterprise accounts may need more control.

Common access options include:

  • Shared password
  • IP allowlist
  • Email authentication
  • SAML or OIDC SSO
  • Separate pages for specific customers or audiences

Do not assume that a tool offering “private pages” supports your preferred authentication method. These features are also frequently limited to expensive plans.

5. Test the incident workflow, not just the page design

A beautiful page does not help if publishing an update takes ten steps.

During a trial, rehearse an incident from start to finish:

  1. A monitor fails
  2. The on-call person is alerted
  3. An incident is opened
  4. Affected components change state
  5. Subscribers receive an update
  6. Your team publishes progress
  7. The service recovers
  8. The incident is resolved and kept in history

Test scheduled maintenance and retrospective incidents too. The dashboard used under pressure matters more than the marketing screenshot.

6. Calculate the full cost

Status page pricing can depend on more than the number of pages.

Include:

  • Monitors and check frequency
  • Team members or responders
  • Subscribers
  • Public, private, and audience-specific pages
  • SMS and phone credits
  • Custom domains and white-labeling
  • SSO and audit logs
  • Data retention

A free page can be ideal for a small project. For a growing SaaS, predictable subscriber and team pricing often matters more than the cheapest entry tier.

FAQ

What is the best status page tool?

For most small SaaS teams, the best status page tool combines reliable monitoring, automatic incident updates, manual control, custom domains, and subscriber notifications without enterprise complexity.

OnlineOrNot is a good fit for that use case. Atlassian Statuspage is strong for large organizations, Better Stack is strong for incident response, Instatus is a polished status-page-first option, and Uptime Kuma is a good self-hosted choice.

Is Atlassian Statuspage free?

Yes. Atlassian Statuspage has a free public-page plan with limited subscribers, team members, components, and metrics. Paid public plans add higher limits and more customization, while private and audience-specific pages use separate pricing.

What should a status page include?

A useful status page should include current service health, affected components, active incidents, scheduled maintenance, incident history, uptime history, and a way to subscribe to updates.

During an incident, every update should explain what is affected, what the team is doing, and when customers should expect the next update.

How do you manage a status page?

Connect the page to monitoring for component health, assign someone to own customer communication, and use a repeatable incident update cadence. During an outage, publish the affected components, customer impact, current response, and next update time. Resolve the incident when service recovers, then preserve the timeline for future reference.

Schedule maintenance in advance, review subscriber and access settings regularly, and rehearse the full publishing workflow before a real incident.

Are free status page tools good enough?

Free status page tools are often good enough for personal projects, open-source projects, and small products with a limited audience.

A paid plan becomes worthwhile when you need a custom domain, more subscribers, faster monitoring, private access, multiple team members, SMS notifications, SSO, audit logs, or guaranteed support.

Should I self-host a status page?

Self-host if you need full control, have strict data requirements, or already operate the infrastructure needed to keep it reliable.

Use a hosted service if you want the page to remain independent from your application and do not want to manage upgrades, email delivery, TLS, backups, and availability during an outage.

What is the difference between a public and private status page?

A public status page is available to anyone and is normally used for customer-facing services. A private status page restricts access using a password, IP allowlist, email authentication, or SSO and is normally used for internal systems or specific enterprise customers.

Should status pages update automatically?

Automate clear component changes and routine incidents, but keep a human in control of customer communication.

Monitoring can say that an API is unavailable. It cannot always explain which customers are affected, whether data is safe, what workaround exists, or when the next meaningful update will arrive.

How is a status page different from uptime monitoring?

Uptime monitoring detects whether a website, API, or service is working and alerts your team when it fails.

A status page communicates service health and incident updates to customers or internal stakeholders. Many modern tools combine both, but they solve different parts of the incident: detection and communication.

Related guides

A status page earns trust when it is accurate, available, and updated before customers need to ask what is happening.

If you want monitoring and incident communication in one place, OnlineOrNot includes uptime checks, automatic incident updates, custom domains, public and private pages, and unlimited subscribers without requiring a full enterprise observability platform.

]]> <![CDATA[OnlineOrNot updates from May 2026]]> https://onlineornot.com/onlineornot-updates-from-2026-may https://onlineornot.com/onlineornot-updates-from-2026-may Wed, 03 Jun 2026 00:00:00 GMT Last month I focused on adding Telegram alerts to OnlineOrNot, making scripted browser checks generally available, and making it clearer to understand why checks fail.

Table of contents

What's new

Features for Alerts

Telegram alerting

OnlineOrNot now supports Telegram alerts.

You can connect Telegram from the dashboard and receive alerts for uptime checks and heartbeat checks, alongside the existing Slack, Discord, email, and webhook options.

If your team already coordinates incidents in Telegram, OnlineOrNot can now send downtime and recovery alerts there too. It supports private chats, groups/supergroups, and channels. See the docs to get started.

Features for Browser Checks

Scripted browser checks

OnlineOrNot now supports scripted browser checks via Playwright.

Regular uptime checks are still the right default for APIs, health endpoints, and simple pages. Browser checks are for the flows where "the URL returned HTTP 200" isn't enough.

They're useful for monitoring things like:

  • Login flows
  • Checkout and payment pages
  • Dashboards behind JavaScript-heavy frontends
  • Forms and customer portals
  • "The app loads, but the important action is broken" failures

Scripted browser checks are powered by Playwright scripts, so they're best suited for technical teams right now. If you already know the flow you care about, you can script it directly, draft it with an LLM, or use this as a reason to finally turn your most painful manual smoke test into monitoring.

You can create a scripted browser check here.

Better failure details

Scripted browser checks also include clearer failure details when something goes wrong.

Instead of only seeing that a browser check failed, alerts can now include more useful context about the failure, helping you understand what broke faster.

Features for the Dashboard

Better check results explorer

The check results explorer has been improved to make recent check activity easier to inspect.

This should make it simpler to understand what happened during a failure, retry, or recovery.

Features for Status Pages

Better custom domain handling

Status page custom domains are now easier to manage and more reliable to visit.

OnlineOrNot now handles custom domains consistently whether you enter example.com, https://example.com, or include a trailing slash. Dashboard links now open the right public status page URL, and visitors see clearer messages if a custom-domain status page is unavailable.

]]> <![CDATA[Best Pingdom alternatives for 2026 (free and paid)]]> https://onlineornot.com/best-pingdom-alternatives https://onlineornot.com/best-pingdom-alternatives Fri, 29 May 2026 00:00:00 GMT Pingdom is one of the best-known website monitoring products. It has been around for years, is now part of SolarWinds, and remains a familiar choice for teams that want uptime monitoring, page speed monitoring, and transaction checks.

But it is not the only option. If you are comparing Pingdom alternatives in 2026, you are probably looking for one of three things: simpler pricing, faster uptime checks, or monitoring that covers more than public URLs.

This guide compares practical Pingdom alternatives for engineering teams, startups, agencies, and developers.

If you specifically want to compare OnlineOrNot and Pingdom side by side, read the Pingdom alternative comparison.

Table of contents

Quick comparison table

| Tool | Check frequency | Free plan | Browser checks | Cron monitoring | Status pages | Best for | |------|----------------|-----------|----------------|-----------------|--------------|----------| | OnlineOrNot | 30 seconds | Yes | Yes | Yes | Yes | Engineering teams that want uptime, browser checks, cron monitoring, and status pages together | | UptimeRobot | 5 min free, 1 min paid | Yes | Limited | Heartbeats | Yes | Simple uptime checks and hobby projects | | Better Stack | 30 seconds | Yes | Yes | Heartbeats | Yes | Teams that want monitoring plus incident response | | Uptime Kuma | Configurable | Self-hosted | Limited | Push monitors | Yes | Self-hosters and private infrastructure | | Datadog Synthetic Monitoring | 1 minute+ | No | Yes | No | No | Enterprises already using Datadog | | StatusCake | 5 min free, 30s on higher tiers | Yes | Yes | No | Yes | Website monitoring plus page speed and domains | | Uptime.com | 1 minute | Trial | Yes | Yes | Yes | Larger teams with broad monitoring needs | | Checkly | Usage-based | Trial/free developer usage | Yes | No | No | Code-first synthetic monitoring | | Hyperping | Plan-dependent | Yes | No | Heartbeats | Yes | Lightweight uptime monitoring and status pages |

Best overall Pingdom alternative: OnlineOrNot

OnlineOrNot is built for developers and small teams that want practical production monitoring without buying a full enterprise observability suite.

It covers the core workflows teams usually assemble across several tools: uptime checks, API monitoring, Playwright browser checks, cron job monitoring, alerts, and hosted status pages.

What it does well:

  • 30-second uptime checks - OnlineOrNot checks production services quickly enough for SaaS apps, APIs, and customer-facing workflows where several minutes of downtime matters.

  • Multi-location verification - Failures are verified from multiple regions before alerting, which helps reduce false positives caused by regional network issues.

  • Playwright browser checks - If your login, signup, checkout, or dashboard flow breaks while the homepage still returns 200 OK, browser checks can catch it.

  • Cron job monitoring - Backups, billing jobs, queue workers, and data pipelines can check in on a schedule. If they miss their window, OnlineOrNot alerts you.

  • Status pages included - Hosted status pages are included with paid plans, so you can communicate incidents without adding another vendor.

  • Team-friendly pricing - Pro starts at $15/month for 10 monitors and includes unlimited team members.

Where it falls short:

  • No APM or distributed tracing
  • No log aggregation
  • Not designed to replace Datadog, New Relic, or a full observability platform

Best for: SaaS teams, startups, agencies, and engineering teams that want uptime checks, browser checks, cron monitoring, and status pages in one lightweight product.

Best free starting point: UptimeRobot

UptimeRobot is a popular Pingdom alternative for teams that want simple uptime monitoring at a low starting cost.

It is especially common for personal projects, side projects, and small sites where the monitoring setup needs to be quick and inexpensive.

What it does well:

  • Generous free plan - Useful for personal projects and low-risk sites.
  • Fast setup - Add a URL, choose alert channels, and start monitoring.
  • Low-cost paid plans - A reasonable fit when you mostly need basic HTTP checks.
  • Familiar product - Many developers have used UptimeRobot before.

Where it falls short:

  • The free tier checks less frequently than many paid production setups need.
  • It is less focused on browser-based user journeys.
  • Teams that need cron monitoring, status pages, alert routing, and browser checks may outgrow it.

Best for: Hobby projects, simple marketing sites, and teams that mainly need basic uptime checks.

If you are comparing OnlineOrNot directly with UptimeRobot, read the UptimeRobot alternative comparison.

Best for incident response: Better Stack

Better Stack combines uptime monitoring, incident management, on-call scheduling, logs, and status pages.

That makes it useful if your team wants more than “send an alert when a check fails.” Better Stack can help assign incidents, escalate alerts, and communicate with customers from the same broader platform.

What it does well:

  • Monitoring plus incident response - Good fit for teams that want on-call workflows with monitoring.
  • Status pages - Public incident communication is part of the product.
  • Logs and broader platform features - Useful if you want more than uptime monitoring.
  • Team workflows - Better fit for operational teams than simple personal monitoring tools.

Where it falls short:

  • More product than some teams need if they only want lightweight uptime monitoring.
  • Pricing can increase as you adopt more of the suite.
  • Teams that already use another incident-management workflow may not need the full platform.

Best for: Teams that want uptime monitoring and incident response in one place.

Best self-hosted option: Uptime Kuma

Uptime Kuma is an open-source monitoring tool you run yourself.

It is popular because it has a friendly interface, supports many monitor types, and does not charge per monitor. The tradeoff is that you are responsible for keeping the monitoring system itself online.

What it does well:

  • Free and open source - No SaaS subscription.
  • Flexible monitor types - Supports HTTP, TCP, DNS, Docker containers, and more.
  • Good UI - More approachable than many self-hosted monitoring tools.
  • Private network support - Useful when you need monitoring inside your own infrastructure.

Where it falls short:

  • You maintain the server, backups, updates, and alert delivery.
  • Single-location monitoring is the default unless you design around it.
  • If the server running Uptime Kuma fails, your monitoring can fail with it.

Best for: Self-hosters, home labs, internal services, and teams comfortable maintaining their own monitoring stack.

Best enterprise option: Datadog Synthetic Monitoring

Datadog Synthetic Monitoring is strongest when your company already uses Datadog for logs, metrics, traces, dashboards, and incident investigation.

For simple Pingdom replacement use cases, Datadog is usually more than you need. For larger teams that want synthetic checks connected to the rest of their observability data, it can be powerful.

What it does well:

  • Deep observability integration - Failed checks can connect to traces, logs, services, and infrastructure metrics.
  • API and browser tests - Useful for complex synthetic monitoring.
  • Enterprise controls - Good fit for larger organizations.
  • Global locations - Checks can run from many locations around the world.

Where it falls short:

  • Pricing can be hard to forecast for small teams.
  • It is not lightweight if all you need is uptime monitoring.
  • Public status pages are not the main product.

Best for: Enterprises already standardized on Datadog.

Other Pingdom alternatives to consider

StatusCake

StatusCake offers uptime monitoring, SSL monitoring, domain monitoring, page speed checks, and status pages.

It is a practical Pingdom alternative if you want a broader website monitoring suite and do not mind feature access changing by plan.

Best for: Teams that want uptime monitoring plus page speed, SSL, and domain checks.

Uptime.com

Uptime.com is a broader website monitoring platform with uptime checks, transaction checks, API monitoring, status pages, and reporting.

It is positioned more toward businesses with larger monitoring requirements than individual developers.

Best for: Larger teams that need multiple monitoring types and reporting.

Checkly

Checkly focuses on code-first synthetic monitoring. It is strongest when your team wants to write checks as code and integrate monitoring into development workflows.

Best for: Engineering teams that want programmable API and browser checks.

Hyperping

Hyperping is a lightweight uptime monitoring and status page product.

It is worth considering if you want a simple Pingdom alternative focused on uptime checks, incident communication, and a clean interface.

Best for: Small teams that want lightweight uptime monitoring and status pages.

How to choose a Pingdom alternative

The best Pingdom alternative depends on why you are leaving Pingdom.

1. If you need faster alerts

Look for 30-second or 1-minute checks on production plans. A 5-minute check interval may be fine for side projects, but it can be too slow for production SaaS apps and APIs.

2. If you get too many false positives

Prioritize multi-location verification. A single regional network issue should not wake up your on-call team if customers are not affected.

3. If public URLs are not enough

Choose a tool that supports browser checks, API checks, and cron job monitoring. Your homepage can be online while signup, billing, or a background job is broken.

4. If customers need outage updates

Choose a monitoring tool with built-in status pages or a clean status page integration. During incidents, communication matters as much as detection.

5. If your team is growing

Check how pricing changes with users, monitors, integrations, and status pages. A tool that is cheap for one person can become awkward for a team.

FAQ

What is the best Pingdom alternative?

For engineering teams that want uptime checks, browser checks, cron job monitoring, alerting, and status pages in one product, OnlineOrNot is a strong Pingdom alternative. UptimeRobot is a good free starting point, Better Stack is good for incident response, and Datadog is best for enterprises already using Datadog.

Is there a free Pingdom alternative?

Yes. UptimeRobot, Uptime Kuma, Better Stack, Hyperping, and OnlineOrNot all have free or low-cost ways to start, depending on whether you want hosted monitoring or self-hosted monitoring.

Why do teams switch away from Pingdom?

Common reasons include pricing, needing faster checks, wanting fewer false positives, needing browser checks, needing cron job monitoring, or wanting status pages and team workflows in one product.

Is UptimeRobot better than Pingdom?

UptimeRobot can be a better fit for simple uptime monitoring and low-cost monitoring. Pingdom can be a better fit for teams that already use SolarWinds products or want a mature website monitoring vendor. For teams that need browser checks and cron monitoring, OnlineOrNot may be a better fit than either.

Should I use a self-hosted Pingdom alternative?

Use a self-hosted tool like Uptime Kuma if you want control and are comfortable maintaining the monitoring server. Use a hosted tool if you want monitoring to remain independent from your own infrastructure.

Related guides

The best Pingdom alternative depends on what you need next. If your team wants fast uptime checks, multi-location verification, browser checks, cron monitoring, and status pages without enterprise complexity, try OnlineOrNot.

]]> <![CDATA[Best UptimeRobot alternatives for 2026 (free and paid)]]> https://onlineornot.com/best-uptimerobot-alternatives https://onlineornot.com/best-uptimerobot-alternatives Fri, 29 May 2026 00:00:00 GMT UptimeRobot is one of the most popular uptime monitoring tools because it is simple, recognizable, and easy to start with.

UptimeRobot now covers multi-location, API, and heartbeat monitoring on paid plans. Teams still compare alternatives when they need Playwright scripts, different check-interval packaging, more team access, or a different incident workflow.

This guide compares the UptimeRobot alternatives worth considering in 2026.

If you specifically want to compare OnlineOrNot and UptimeRobot side by side, read the UptimeRobot alternative comparison.

UptimeRobot details were verified on July 27, 2026 against its public pricing and feature table, multi-location documentation, and heartbeat monitoring documentation. Plans can change, so verify current limits before purchasing.

Table of contents

Quick comparison table

| Tool | Check frequency | Free plan | Browser checks | Cron monitoring | Status pages | Best for | |------|----------------|-----------|----------------|-----------------|--------------|----------| | OnlineOrNot | 30 seconds | Yes | Yes | Yes | Yes | Production teams that want uptime, browser checks, cron monitoring, and status pages together | | Better Stack | 30 seconds | Yes | Yes | Heartbeats | Yes | Monitoring plus incident response | | Uptime Kuma | Configurable | Self-hosted | Limited | Push monitors | Yes | Self-hosters and internal services | | Datadog Synthetic Monitoring | 1 minute+ | No | Yes | No | No | Enterprises already using Datadog | | Pingdom | 1 minute | No | Transaction monitoring | No | Via related offering | Teams that want a mature SolarWinds monitoring product | | StatusCake | 5 min free, 30s on higher tiers | Yes | Yes | No | Yes | Website monitoring plus SSL, page speed, and domains | | Uptime.com | 1 minute | Trial | Yes | Yes | Yes | Larger teams with broad monitoring needs | | Hyperping | Plan-dependent | Yes | No | Heartbeats | Yes | Lightweight uptime monitoring and status pages | | Checkly | Usage-based | Trial/free developer usage | Yes | No | No | Code-first synthetic monitoring |

Best overall UptimeRobot alternative: OnlineOrNot

OnlineOrNot is built for developers and small teams that need production monitoring without a heavy enterprise observability stack.

It covers uptime checks, API monitoring, Playwright browser checks, cron job monitoring, alerts, and hosted status pages in one product.

What it does well:

  • 30-second uptime checks - Faster checks are useful when downtime affects revenue, customer trust, or support volume.

  • Multi-location verification - OnlineOrNot verifies failures from multiple regions before alerting, which helps reduce false positives from regional network problems.

  • Playwright browser checks - Uptime monitoring can tell you a URL responds. Browser checks can tell you whether login, signup, checkout, or a dashboard flow still works.

  • Cron job monitoring - Scheduled jobs, queue workers, backups, billing jobs, and data pipelines can check in on a schedule. If a job misses its window, OnlineOrNot alerts you.

  • Status pages included - Hosted status pages help communicate incidents without adding a separate status page vendor.

  • Unlimited team members on paid plans - Helpful when monitoring is owned by a team, not one person.

Where it falls short:

  • No log aggregation
  • No APM or distributed tracing
  • Not designed to replace a full observability platform

Best for: SaaS companies, agencies, startups, and production teams that have outgrown basic uptime checks.

Best for incident response: Better Stack

Better Stack combines uptime monitoring with incident management, on-call scheduling, logs, and status pages.

It is a strong UptimeRobot alternative if your team wants alerting and incident response in the same product.

What it does well:

  • 30-second checks - Fast enough for most production websites and APIs.
  • Incident response workflows - On-call scheduling, escalation, and incident timelines are built in.
  • Status pages - Public communication is part of the platform.
  • Broader suite - Useful if logs and incident workflows are part of the buying decision.

Where it falls short:

  • More product than some small teams need.
  • Pricing can increase as you adopt more of the suite.
  • If you already have incident management elsewhere, the overlap may be unnecessary.

Best for: Teams that want monitoring and incident response together.

Best self-hosted option: Uptime Kuma

Uptime Kuma is an open-source uptime monitoring tool you run yourself.

It is one of the most common UptimeRobot alternatives for self-hosters because it has a friendly UI and supports many monitor types.

What it does well:

  • Free and open source - No SaaS subscription.
  • Flexible monitor types - HTTP, TCP, DNS, Docker containers, and more.
  • Good UI - Approachable for self-hosted monitoring.
  • Private infrastructure - Useful when checks need to run inside your network.

Where it falls short:

  • You maintain updates, backups, hosting, and alert delivery.
  • Monitoring can fail with your infrastructure if you host it in the same environment.
  • Multi-region monitoring requires extra work.

Best for: Self-hosters, home labs, private services, and teams comfortable maintaining their own monitoring infrastructure.

Best for enterprise monitoring: Datadog Synthetic Monitoring

Datadog Synthetic Monitoring is a good fit for organizations already using Datadog for metrics, logs, traces, and dashboards.

It is usually too much if you only need simple uptime monitoring, but it is powerful when synthetic checks need to connect to the rest of your observability stack.

What it does well:

  • Deep observability integration - Failed checks can connect to traces, logs, services, and infrastructure data.
  • API and browser tests - Useful for complex synthetic monitoring.
  • Enterprise controls - Better suited to large organizations than lightweight uptime-only products.
  • Global test locations - Checks can run from many locations.

Where it falls short:

  • Pricing can be hard to forecast.
  • It is not lightweight for teams that only need uptime alerts.
  • Public status pages are not the core product.

Best for: Enterprises already standardized on Datadog.

Best Pingdom-style enterprise option: Pingdom

Pingdom is a long-running website monitoring product from SolarWinds.

It offers uptime monitoring, page speed monitoring, transaction monitoring, and alerting. If your team wants to move from UptimeRobot to a more established website monitoring vendor, Pingdom is one option to consider.

What it does well:

  • Mature product - Pingdom has been in the category for a long time.
  • Website monitoring focus - Uptime, page speed, and transaction monitoring are core use cases.
  • Recognizable vendor - Useful for teams that prefer established vendors.

Where it falls short:

  • It may be more product and vendor complexity than smaller teams want.
  • Cron job monitoring and modern developer workflows may require another tool.
  • Pricing and plan details should be checked carefully against current public pricing.

Best for: Teams that want a mature SolarWinds website monitoring product.

If you are comparing OnlineOrNot directly with Pingdom, read the Pingdom alternative comparison.

Other UptimeRobot alternatives to consider

StatusCake

StatusCake offers uptime monitoring, SSL monitoring, domain monitoring, page speed checks, and status pages.

It is a practical alternative if you want a broader website monitoring suite and do not mind plan-based feature access.

Best for: Teams that want uptime monitoring plus SSL, domain, and page speed checks.

Uptime.com

Uptime.com is a broader monitoring platform with uptime checks, transaction checks, API monitoring, status pages, and reporting.

It is positioned more toward businesses with larger monitoring needs than individual developers.

Best for: Larger teams that need multiple monitoring types and reporting.

Hyperping

Hyperping is a lightweight uptime monitoring and status page product.

It is worth considering if you want a simple hosted alternative focused on uptime checks and customer communication.

Best for: Small teams that want lightweight uptime monitoring and status pages.

Checkly

Checkly focuses on code-first synthetic monitoring.

It is strongest when your team wants checks defined in code and integrated into engineering workflows.

Best for: Engineering teams that want programmable API and browser monitoring.

How to choose an UptimeRobot alternative

The best UptimeRobot alternative depends on what made you start looking.

1. If check frequency is the issue

Choose a tool with 30-second or 1-minute checks on the plan you will actually use. Free tiers often check less frequently than production teams need.

2. If false positives are the issue

Look for multi-region verification. A single monitoring location can create noisy alerts when the problem is regional routing rather than a real outage.

3. If basic uptime checks are not enough

Pick a tool that supports browser checks, API checks, content assertions, and cron job monitoring. A service can respond with 200 OK while the customer experience is broken.

4. If incident communication matters

Choose a tool with hosted status pages or a clean status page integration. During outages, customers want to know whether you know about the problem.

5. If your team is growing

Check how pricing changes with users, monitors, alert channels, status pages, and synthetic checks. Team pricing matters once monitoring is no longer owned by one person.

FAQ

What is the best UptimeRobot alternative?

For engineering teams that need uptime checks, browser checks, cron monitoring, status pages, and multi-location verification, OnlineOrNot is a strong UptimeRobot alternative. Better Stack is strong for incident response, Uptime Kuma is strong for self-hosting, and Datadog is best for enterprises already using Datadog.

Is there a free UptimeRobot alternative?

Yes. OnlineOrNot, Better Stack, Hyperping, and StatusCake have free or low-cost ways to start. Uptime Kuma is free and open source if you are comfortable hosting it yourself.

Why do teams switch away from UptimeRobot?

Common reasons include needing Playwright scripts, 30-second checks outside an Enterprise tier, different team-seat pricing, more flexible alerting, or a different incident workflow. UptimeRobot already offers paid multi-location and heartbeat monitoring, so those features alone are not reasons to switch.

Is Uptime Kuma better than UptimeRobot?

Uptime Kuma can be better if you want self-hosted monitoring and control over your infrastructure. UptimeRobot is easier if you want hosted monitoring without maintaining a server. For production teams that want hosted monitoring plus browser checks and cron monitoring, OnlineOrNot is usually a better fit than self-hosting.

Is Pingdom better than UptimeRobot?

Pingdom can be a better fit for teams that want a mature website monitoring product from SolarWinds. UptimeRobot can be a better fit for a large free monitor allowance or its current paid feature set. OnlineOrNot is a better fit when the comparison is about Playwright scripts, 30-second Pro checks, and unlimited team access.

Related guides

UptimeRobot has grown beyond basic uptime checks. If you want Playwright scripts, 30-second Pro checks, cron monitoring, status pages, and unlimited team members in one product, try OnlineOrNot.

]]> <![CDATA[10 Best Uptime Monitoring Tools for 2026]]> https://onlineornot.com/uptime-monitoring-tools https://onlineornot.com/uptime-monitoring-tools Thu, 21 May 2026 00:00:00 GMT Uptime monitoring tools check your website, API, or app on a schedule and alert you when something stops working.

That sounds simple, but the details matter. A tool that checks every 5 minutes from one region will catch different problems than a tool checking every 30 seconds from multiple regions. A tool built for hobby projects will feel very different from an enterprise observability platform.

This guide compares the uptime monitoring tools worth considering in 2026, with a focus on practical tradeoffs: check frequency, false positives, alerting, status pages, pricing, and who each tool is actually best for.

Best uptime monitoring tools: short answer

  • OnlineOrNot is the best fit for small software teams that want 30-second checks, multi-region verification, browser and cron monitoring, and status pages in one product.
  • UptimeRobot is a practical free starting point for basic website checks.
  • Better Stack is the strongest fit when uptime monitoring needs to share a workflow with incident management and on-call scheduling.
  • Uptime Kuma is the best fit for teams that want to self-host and are prepared to operate their own monitoring infrastructure.
  • Datadog Synthetic Monitoring is the best fit for enterprises that already use Datadog logs, metrics, and traces.

There is no universal winner. Shortlist tools based on the check interval, verification method, alert delivery, monitor types, status-page requirements, and total price for your production setup.

Table of contents

Quick comparison table

| Tool | Check frequency | Free plan | Multi-region checks | Status pages | Best for | |------|----------------|-----------|---------------------|--------------|----------| | OnlineOrNot | 30 seconds | Yes | Yes | Yes | Developers, startups, SaaS teams | | UptimeRobot | 5 min free, 1 min paid | Yes | Limited | Yes | Hobby projects, simple sites | | Better Stack | 30 seconds | Yes | Yes | Yes | Teams wanting monitoring + incident response | | Uptime Kuma | Configurable | Self-hosted | DIY | Yes | Self-hosters, home labs | | Pingdom | 1 minute | No | Yes | No | Legacy enterprise environments | | StatusCake | 5 min free, 30s on higher tiers | Yes | Yes | Yes | Teams needing uptime + page speed | | Uptime.com | 1 minute | Trial | Yes | Yes | Larger teams with broader monitoring needs | | HetrixTools | 1 minute | Yes | Yes | Yes | Server and blacklist monitoring | | Datadog Synthetic Monitoring | 1 minute+ | No | Yes | No | Enterprises already using Datadog | | Sentry Uptime Monitoring | 1 minute | Included with Sentry plans | Yes | No | Teams already using Sentry |

What is uptime monitoring?

Uptime monitoring is the practice of regularly checking whether a website, API, server, or application is reachable and responding correctly.

At its simplest, an uptime monitor sends a request to your URL every few minutes. If the request fails, times out, or returns the wrong status code, the tool sends an alert.

Good uptime monitoring goes further:

  • Checks from multiple regions so one network blip does not create a false alarm
  • Verifies response content so a broken page returning 200 OK is still caught
  • Tracks response time so you can spot slowdowns before a full outage
  • Routes alerts correctly so the right person hears about the right incident
  • Publishes status pages so customers can see what is happening during an outage

If you are new to the category, start with the broader website monitoring guide. If you need browser journeys or scripted API tests, compare synthetic monitoring tools instead.

Best for most teams: OnlineOrNot

OnlineOrNot is built for developers and small teams that want fast, practical uptime monitoring without buying a full enterprise observability stack.

It covers the basics most teams actually need: uptime checks, API monitoring, cron job monitoring, browser checks, alerts, and hosted status pages.

What it does well:

  • 30-second checks on paid plans - Fast checks are useful when downtime costs money. Waiting 5 minutes to find out your checkout or API is down is too long for most production apps.

  • Multi-location verification - OnlineOrNot checks from multiple regions before alerting, which helps reduce false positives caused by regional network issues.

  • Simple pricing - Pro starts at $15/month and includes 10 uptime monitors, 3,000 browser check runs, one public status page, and unlimited team members. Annual billing reduces the base price to $12/month.

  • Status pages included - Public status pages are included, so you can communicate incidents without paying for a separate status page product.

  • More than basic ping checks - You can monitor websites, APIs, cron jobs, and browser checks in the same product.

What it does not do:

  • No APM or distributed tracing
  • No log aggregation
  • Not designed to replace Datadog, New Relic, or a full observability platform

Pricing: The free Hobby plan includes 3 monitors. Pro starts at $15/month, or $12/month billed annually, with 10 uptime monitors included. See the current OnlineOrNot pricing.

Best for: SaaS companies, developers, startups, and small teams that need reliable uptime monitoring without enterprise complexity.

Best free starting point: UptimeRobot

UptimeRobot is one of the best-known uptime monitoring tools. Many developers start there because it is free, simple, and quick to set up.

What it does well:

  • Generous free plan - The free tier is useful for personal projects and low-stakes sites.
  • Very easy setup - Add a URL, choose alert channels, and you are done.
  • Cheap paid plans - Paid plans are inexpensive compared with enterprise tools.
  • Recognizable product - It has been around for a long time, so many teams already know how it works.

Where it falls short:

  • 5-minute checks on the free tier - A 5-minute interval can mean several minutes of downtime before you know anything happened.
  • Limited depth - It is good for basic uptime checks, but less useful if you need richer API assertions, browser monitoring, or more advanced team workflows.
  • Free plan restrictions - If you are using monitoring commercially, check the current terms before relying on the free tier.

Pricing: Free plan available. Paid plans start at low monthly pricing depending on monitor count and features.

Best for: Hobby projects, side projects, personal websites, and teams that only need basic uptime checks.

Best for incident management: Better Stack

Better Stack combines uptime monitoring with incident management, status pages, logs, and on-call scheduling.

This is useful if you do not just want to know that something broke. You also want to assign incidents, escalate alerts, and keep a public status page updated from the same workflow.

What it does well:

  • 30-second checks - Fast enough for most production websites and APIs.
  • Incident response built in - On-call scheduling, escalation, incident timelines, and status pages are part of the product.
  • Good team workflows - Better fit for teams than tools aimed at individual developers.
  • Broad monitoring suite - Useful if you also want logs and incident management under one brand.

Where it falls short:

  • More product than some teams need - If you just want a few uptime checks, the incident management layer may be unnecessary.
  • Pricing rises as you adopt more of the suite - It can be good value if you use the whole platform, but less compelling if you only need uptime monitoring.

Pricing: Free plan available. Paid monitoring plans are typically more expensive than lightweight uptime-only tools.

Best for: Teams that want uptime monitoring and incident response in one place.

Best self-hosted option: Uptime Kuma

Uptime Kuma is an open-source uptime monitoring tool you run yourself.

It is popular because it has a friendly UI, supports many monitor types, and does not charge per monitor. The tradeoff is that you are responsible for keeping your monitoring system online.

What it does well:

  • Free and open source - No monthly SaaS subscription.
  • Nice interface - Much more polished than many self-hosted monitoring tools.
  • Flexible monitor types - Supports HTTP, TCP, DNS, Docker containers, and more.
  • Good for private infrastructure - Useful when you need monitoring inside a private network.

Where it falls short:

  • You run the infrastructure - Updates, backups, security, uptime, and alert delivery are your responsibility.
  • Single-location by default - Unless you deploy multiple instances, you only monitor from wherever your server runs.
  • Monitoring can fail silently - If the machine running Uptime Kuma dies, your monitoring dies with it.

Pricing: Free, but you pay for hosting and maintenance time.

Best for: Self-hosters, home labs, internal services, and teams comfortable maintaining their own monitoring infrastructure.

Best for enterprises: Datadog

Datadog Synthetic Monitoring is a strong option for companies already using Datadog for logs, metrics, traces, and dashboards.

For simple uptime monitoring, Datadog is usually overkill. For large teams that need to connect external checks to backend traces and infrastructure telemetry, it can be powerful.

What it does well:

  • Deep observability integration - A failing uptime check can connect to traces, logs, services, and infrastructure metrics.
  • API and browser tests - Useful for complex synthetic monitoring, not just simple uptime checks.
  • Enterprise controls - Strong fit for larger organizations with existing Datadog workflows.
  • Global locations - Checks can run from many locations around the world.

Where it falls short:

  • Pricing can be hard to predict - Usage-based synthetic test pricing requires more planning than per-monitor pricing.
  • Not lightweight - If all you need is “tell me when my site is down,” Datadog is more tool than you need.
  • No built-in public status page - You will likely use another product for customer-facing incident communication.

Pricing: Usage-based synthetic monitoring pricing. API and browser tests are priced differently.

Best for: Enterprises already standardized on Datadog.

Other uptime monitoring tools to consider

Pingdom

Pingdom is a long-running website monitoring product now owned by SolarWinds.

It offers uptime monitoring, page speed monitoring, transaction monitoring, and real user monitoring. It is reliable and recognizable, but the product can feel more legacy than newer developer-focused tools.

Best for: Teams already using SolarWinds or companies that want a familiar enterprise vendor.

StatusCake

StatusCake offers uptime monitoring, SSL monitoring, domain monitoring, page speed checks, and status pages.

It is a good middle-ground option if you want a broader website monitoring suite and do not mind feature access changing by tier.

Best for: Teams that want uptime monitoring plus page speed and domain checks.

Uptime.com

Uptime.com is a broader website monitoring platform with uptime checks, transaction checks, API monitoring, status pages, and reporting.

It is positioned more toward businesses with larger monitoring needs than individual developers.

Best for: Larger teams that need multiple monitoring types and reporting features.

HetrixTools

HetrixTools combines uptime monitoring with server monitoring and blacklist monitoring.

The blacklist monitoring angle makes it especially useful for teams that care about email deliverability or IP reputation.

Best for: Teams monitoring servers, IP reputation, and uptime together.

Sentry Uptime Monitoring

Sentry now offers uptime monitoring alongside its error monitoring and performance products.

This can be convenient if your team already uses Sentry every day. The value is less about replacing dedicated uptime monitoring tools and more about connecting downtime to errors and traces.

Best for: Engineering teams already using Sentry for errors and performance.

How to choose an uptime monitoring tool

The best uptime monitoring tool depends less on the logo and more on how you operate.

1. Pick the right check frequency

Check frequency determines how quickly you find out something is wrong.

  • 30 seconds: Good for production SaaS apps, APIs, checkout flows, and customer-facing services
  • 1 minute: Good default for most business websites and APIs
  • 5 minutes: Acceptable for hobby projects or low-risk pages

If downtime costs money or breaks customer trust, avoid relying on 5-minute checks for critical services.

2. Decide whether you need multi-region verification

Single-region monitoring is simple, but it can create false positives. A network issue between the monitoring server and your website can look like an outage even if customers are fine.

Multi-region verification reduces noise by checking from more than one location before alerting.

This matters most if:

  • Your customers are global
  • You use a CDN
  • Your team ignores alerts after a few false alarms
  • You have had “site down” alerts that turned out to be regional routing problems

3. Check more than the homepage

Monitoring only your homepage is better than nothing, but it misses many real failures.

For a production SaaS, consider monitoring:

  • Homepage
  • Login page
  • Signup page
  • API health endpoint
  • Billing or checkout flow
  • Public status page
  • Cron jobs and background jobs
  • SSL certificate expiry

The goal is not to monitor every URL. It is to monitor the paths that matter when customers are trying to use or buy your product.

4. Verify correctness, not just reachability

A page can return 200 OK and still be broken.

Common examples:

  • Your app serves a blank page because JavaScript failed to load
  • Your backend returns a branded error page with a 200 status code
  • Your CDN serves stale maintenance content
  • Your API returns a valid status code with invalid data

At minimum, use content checks for key pages. For APIs, use assertions against status codes, response bodies, and response time thresholds. For JavaScript-heavy apps, consider browser checks.

5. Make sure alerting matches severity

The tool matters less if alerts go to the wrong place.

Use different alerting rules for different services:

  • Critical production services: SMS, phone call, PagerDuty, or on-call escalation
  • Important but non-critical pages: Slack or Teams
  • Low-priority marketing pages: Email or low-urgency Slack channels

If every alert is urgent, none of them are urgent. Alert fatigue is one of the most common ways monitoring setups fail.

6. Consider status pages early

During an outage, customers want to know two things: whether you know about the problem, and when they should expect an update.

A public status page helps you communicate without replying to every support ticket manually.

If you already know you need a status page, choose a monitoring tool that includes one or integrates cleanly with the one you use.

FAQ

What is the best uptime monitoring tool?

For most small teams and SaaS companies, the best uptime monitoring tool is one that offers fast checks, reliable alerting, multi-region verification, and status pages without enterprise complexity.

OnlineOrNot is a good fit for that use case. UptimeRobot is a good free starting point. Better Stack is a good choice if you also want incident management. Datadog is best if your company already uses Datadog heavily.

What is uptime monitoring software?

Uptime monitoring software checks whether your website, API, or service is reachable. It runs checks on a schedule, records availability and response time, and alerts you when a check fails.

What is the difference between uptime monitoring and website monitoring?

Uptime monitoring focuses on whether a service is up or down. Website monitoring is broader and can include uptime, page speed, SSL certificates, broken pages, real user monitoring, and synthetic browser tests.

In practice, people often use the terms interchangeably.

What is the difference between uptime monitoring and synthetic monitoring?

Uptime monitoring usually means simple scheduled checks: is this URL reachable, did it return the expected status code, and did it respond fast enough?

Synthetic monitoring is broader. It can include scripted API checks, browser-based user journeys, login flows, checkout tests, and multi-step workflows.

How often should I check my website uptime?

For production websites and APIs, every 30 seconds to 1 minute is a good default. For low-risk personal sites, every 5 minutes may be enough.

Use faster intervals for services where downtime affects revenue, customer trust, or incident response.

Are free uptime monitoring tools good enough?

Free uptime monitoring tools are good enough for personal projects, early-stage side projects, and low-risk websites.

For business-critical websites and APIs, paid tools are usually worth it because they offer faster checks, better alerting, multi-region verification, team features, and more reliable incident communication.

Should I use a self-hosted uptime monitoring tool?

Use a self-hosted tool like Uptime Kuma if you want control, need to monitor private infrastructure, or enjoy maintaining your own systems.

Use a managed tool if you want your monitoring to remain independent from your own infrastructure. If your self-hosted monitoring server goes down at the same time as your app, you may not get alerted.

Related comparisons

Uptime monitoring is one of the simplest reliability improvements you can make. Start with your most important URL, add alerts you will actually notice, then expand to APIs, signup flows, cron jobs, and status pages.

If you want a practical starting point, compare OnlineOrNot plans or start a 14-day trial with 30-second uptime checks, multi-region verification, and status pages without adopting a full observability platform.

]]> <![CDATA[Best synthetic monitoring tools for 2026 (tested and compared)]]> https://onlineornot.com/synthetic-monitoring-tools https://onlineornot.com/synthetic-monitoring-tools Tue, 10 Mar 2026 00:00:00 GMT Synthetic monitoring tools check your websites and APIs before your users do. They run automated tests from multiple locations, catch outages, and alert you when something breaks.

The problem is that there are dozens of options, ranging from free open-source tools to enterprise platforms costing thousands per month.

This guide compares the ones worth considering in 2026, based on actual usage - not vendor marketing.

Table of contents

What is synthetic monitoring?

Synthetic monitoring means running automated tests against your websites and APIs on a schedule. Unlike real user monitoring (RUM), which tracks actual user sessions, synthetic monitoring proactively checks your services from the outside.

Think of it as a robot that visits your site every 30 seconds and tells you if something breaks.

The basic flow:

  1. You configure a check (URL, expected response, acceptable response time)
  2. The tool runs that check at regular intervals from multiple locations
  3. If something fails, you get an alert

Most tools also track response times over time, so you can spot performance degradation before it becomes an outage.

When you need synthetic monitoring:

  • You want to know about outages before users report them
  • You need to verify that deployments didn't break production
  • You're monitoring APIs that external services depend on
  • You want to track uptime for SLA reporting

When RUM is better:

  • You need to understand real user experience across different devices
  • You're debugging performance issues specific to certain browsers or regions
  • You want to correlate frontend errors with user sessions

Most teams benefit from both, but synthetic monitoring is the baseline everyone should have.

Quick comparison table

| Tool | Check frequency | Pricing | Multi-region | Browser tests | Best for | |------|----------------|---------|--------------|---------------|----------| | OnlineOrNot | 30 seconds | From $12/mo | Yes (17 regions) | Yes | Developers, startups, growing teams | | Datadog Synthetics | 1 minute+ | ~$5+ per 10k tests | Yes | Yes | Enterprise with existing Datadog stack | | Better Stack | 30 seconds | From $24/mo | Yes | No | Teams wanting incident management included | | UptimeRobot | 5 min (free), 1 min (paid) | Free / $7+/mo | Limited | No | Hobbyists, basic monitoring | | Pingdom | 1 minute | From $15/mo | Yes | Yes | Legacy enterprise environments | | StatusCake | 5 min (free), 30s (Business) | From €20/mo | Yes | No | Teams needing page speed monitoring | | Uptime Kuma | Configurable | Free (self-hosted) | DIY | No | Self-hosters, home labs | | Checkly | 1 minute | From $30/mo | Yes | Yes | Teams with complex browser test needs |

Best for most teams: OnlineOrNot

OnlineOrNot is built for developers who want reliable monitoring without enterprise complexity.

What it does well:

  • 30-second checks on all paid plans - Most competitors reserve fast checks for expensive tiers. OnlineOrNot checks every 30 seconds starting at $12/month.

  • Multi-location verification - Every check runs from multiple regions before alerting. This eliminates false positives from network blips or regional CDN issues.

  • Simple pricing - $2.40 per monitor per month (monthly billing). No confusing tiers, no "contact sales" for basic features.

  • Status pages included - Branded status pages on your domain are included, not a paid add-on.

  • Cron job monitoring - Track scheduled tasks and background jobs alongside your uptime checks.

  • Browser checks - Run checks in real Chrome browsers to verify JavaScript apps load and work correctly.

What it doesn't do:

  • No APM or distributed tracing
  • No log aggregation

Pricing: Free tier with 3 monitors. Paid plans from $12/month for 5 monitors.

Best for: Development teams, SaaS companies, and anyone who wants fast, reliable monitoring without paying for features they won't use.

Best for enterprise: Datadog Synthetic Monitoring

Datadog is the default choice for enterprises already using Datadog for APM and logging.

What it does well:

  • Deep integration with APM - Correlate synthetic test failures with traces, logs, and infrastructure metrics. When a test fails, you can see exactly what broke in the backend.

  • Browser testing - Record user journeys and replay them as synthetic tests. Good for testing complex flows like checkout processes.

  • Mobile app testing - Test mobile apps alongside web properties.

  • Global coverage - Run tests from dozens of locations worldwide.

What it doesn't do well:

  • Pricing is complex - You pay per test execution, which can add up quickly. Running a check every minute from 5 locations is 5 × 60 × 24 × 30 = 216,000 test executions per month.

  • Overkill for simple monitoring - If you just want to know if your API is up, you don't need Datadog's full observability platform.

  • Slow minimum interval - Fastest check interval is 1 minute, compared to 30 seconds elsewhere.

Pricing: Approximately $5 per 10,000 test executions for API tests, $12 per 1,000 for browser tests. No free tier for synthetics.

Best for: Large organizations already invested in the Datadog ecosystem who need synthetic monitoring integrated with their existing observability stack.

Best budget option: UptimeRobot

UptimeRobot is the tool everyone starts with. It's free for basic monitoring and has been around forever.

What it does well:

  • Generous free tier - 50 monitors with 5-minute checks, completely free
  • Simple interface - No learning curve, just add URLs and go
  • Cheap paid plans - Pro plans start at $7/month

What it doesn't do well:

  • Slow check intervals - Free tier is 5 minutes, which means up to 5 minutes before you know about an outage
  • Single-location checks - No multi-location verification means more false positives from network issues
  • Basic alerting - Limited options for routing alerts to different team members
  • Recent ToS changes - Free tier can no longer be used for commercial projects

Pricing: Free for 50 monitors (personal use). Pro from $7/month.

Best for: Personal projects, hobby sites, and teams with minimal monitoring budgets who can tolerate slower detection.

Best self-hosted: Uptime Kuma

Uptime Kuma is an open-source monitoring tool you run on your own infrastructure.

What it does well:

  • Completely free - No per-monitor costs, no monthly fees
  • Full control - Your data stays on your servers
  • Feature-rich - Supports HTTP, TCP, DNS, Docker, and more
  • Nice UI - Clean, modern interface that's easy to use
  • Active development - Regular updates and community contributions

What it doesn't do well:

  • Single location - You're monitoring from wherever you host it. No multi-region verification unless you set up multiple instances.
  • Self-hosting overhead - You need to maintain the server, handle updates, and ensure it stays online. If your monitoring server goes down, you won't know about outages.
  • No team features - Limited collaboration tools compared to managed services
  • No managed alerting infrastructure - You're responsible for ensuring SMS/email delivery works

Pricing: Free (you pay for hosting)

Best for: Home labs, self-hosting enthusiasts, teams with strict data residency requirements, or anyone comfortable managing their own infrastructure.

Other options worth considering

Better Stack

Better Stack (formerly Better Uptime) combines uptime monitoring with incident management and on-call scheduling.

  • 30-second checks
  • Built-in incident management
  • On-call scheduling included
  • From $24/month

Good if you want monitoring + incident response in one tool. More expensive than pure monitoring solutions.

Pingdom

Pingdom is owned by SolarWinds and has been around since 2007. Reliable but dated.

  • 1-minute minimum check interval
  • Strong brand recognition
  • From $15/month for 10 monitors

Good for enterprises who already use SolarWinds products. Feels dated compared to modern alternatives.

StatusCake

StatusCake offers uptime, page speed, and server monitoring.

  • 30-second checks require €70/month Business plan
  • Page speed monitoring included
  • From €20/month

Good if you need page speed testing alongside uptime monitoring. The tiered pricing with monitor caps can get expensive.

Checkly

Checkly focuses on API and browser testing with a developer-first approach.

  • Playwright-based browser checks
  • Monitoring as code
  • From $30/month

Good for teams with complex browser testing needs or who want to version control their monitoring configuration.

How to choose the right tool

What check frequency do you need?

If downtime costs you significant money, you want the fastest detection possible. 30-second checks catch issues twice as fast as 1-minute checks.

  • 30 seconds: OnlineOrNot, Better Stack, StatusCake (expensive tier)
  • 1 minute: Datadog, Pingdom, Checkly
  • 5 minutes: UptimeRobot (free)

What's your budget?

Be honest about what you're willing to pay:

  • Free: UptimeRobot (limited), Uptime Kuma (self-hosted)
  • $10-30/month: OnlineOrNot, StatusCake, Pingdom
  • $50-100/month: Better Stack, Checkly
  • $100+/month: Datadog, enterprise tiers of others

Do you need multi-location verification?

Single-location checks trigger false positives when there's a network issue between the monitoring location and your server. Multi-location verification only alerts when multiple regions confirm the outage.

Most paid tools offer this. Free tools and self-hosted options typically don't.

Do you need browser testing?

If you need to verify that JavaScript apps load and render correctly, or test complex user flows (login, checkout, multi-step forms), you need a tool with browser testing capabilities:

  • OnlineOrNot (real Chrome browsers)
  • Datadog Synthetic Monitoring
  • Checkly
  • Pingdom
  • Playwright/Puppeteer + your own infrastructure

For simple HTTP/API monitoring, browser testing is overkill.

What integrations do you need?

Most tools integrate with Slack, PagerDuty, and webhooks. Check that your specific tools are supported:

  • On-call platforms: PagerDuty, Opsgenie, Incident.io
  • Chat: Slack, Discord, Microsoft Teams
  • Ticketing: Jira, Linear, GitHub Issues

FAQ

What is an example of synthetic monitoring?

A synthetic monitor might check https://api.example.com/health every 30 seconds. It sends an HTTP GET request, verifies the response is a 200 status code with {"status": "ok"} in the body, and confirms the response time is under 500ms. If any of those conditions fail, you get an alert.

What is a synthetic monitoring agent?

A synthetic monitoring agent is software that runs the actual checks. Managed services run agents in data centers around the world. Self-hosted solutions like Uptime Kuma require you to run the agent on your own server.

What are the different types of synthetic monitors?

The main types are:

  • HTTP/API monitors - Check that URLs respond correctly
  • Browser monitors - Simulate user interactions like clicking and form submission
  • TCP/UDP monitors - Check that ports are open
  • DNS monitors - Verify DNS resolution works
  • SSL monitors - Track certificate expiration

Synthetic monitoring vs RUM - which do I need?

Synthetic monitoring proactively tests your services on a schedule. It catches outages and degradation before users are affected.

Real User Monitoring (RUM) passively collects data from actual user sessions. It shows you what real users experience across different devices and network conditions.

Most teams should start with synthetic monitoring because it's simpler and catches the most critical issues. Add RUM when you need to understand real user experience in detail.

How much does synthetic monitoring cost?

Costs range from free (UptimeRobot free tier, self-hosted Uptime Kuma) to hundreds of dollars per month for enterprise tools.

For a typical SaaS with 20-50 monitors:

  • Budget option: $7-15/month (UptimeRobot Pro)
  • Mid-range: $40-100/month (OnlineOrNot, Better Stack)
  • Enterprise: $200+/month (Datadog, based on usage)

Can I use multiple synthetic monitoring tools?

Yes, and some teams do. Running two independent monitoring services provides redundancy - if one has an outage, the other still alerts you.

The downside is managing multiple dashboards and paying for both services. For most teams, one reliable tool is enough.

Related comparisons

The best synthetic monitoring tool is the one you'll actually use. Start with something simple, get it running on your critical endpoints, and expand from there.

If you're not sure where to start, OnlineOrNot offers a 14-day free trial with 30-second checks - no credit card required.

]]> <![CDATA[API monitoring: A practical guide]]> https://onlineornot.com/api-monitoring-guide https://onlineornot.com/api-monitoring-guide Tue, 03 Mar 2026 00:00:00 GMT Your API is the backbone of your product. Mobile apps, integrations, webhooks - they all depend on it working.

When it goes down, everything goes down with it.

The frustrating part is that API failures are often invisible. Your website might load fine, but the API returning data to your mobile app? Down for hours without anyone noticing.

Until customers start complaining.

Table of contents

What is API monitoring?

API monitoring is checking that your API endpoints are working correctly - not just that they respond, but that they respond with the right data, fast enough.

At its simplest, you're making requests to your API at regular intervals and checking:

  1. Does it respond at all?
  2. Does it return the correct status code?
  3. Is the response what we expect?
  4. How long did it take?

If any of those checks fail, you get an alert.

It's basically the same idea as uptime monitoring for websites, but with a few extra considerations for APIs.

Why you need it

There are a few scenarios where API monitoring saves you:

Silent failures - Your API starts returning 500 errors, but your website still loads because it degrades gracefully. Users see broken features, but you don't see errors in your logs because the frontend swallows them.

Slow degradation - Response times creep up from 200ms to 2 seconds over a few days. Not enough to trigger an error, but enough to make your mobile app feel sluggish. Users churn, and you don't know why.

Third-party dependencies - You call another service's API, and they have an outage. Your API technically works, but returns empty data or errors for certain requests.

Regional issues - Your API works fine from your office, but users in Europe are timing out because of a CDN misconfiguration.

In all these cases, monitoring catches the problem before your users do.

What to monitor

Not every endpoint needs the same level of monitoring. Here's how I think about it:

Critical endpoints

These are the endpoints that, if they break, your business stops working:

  • Authentication (login, token refresh)
  • Core business logic (checkout, payments, data submission)
  • Public APIs your customers integrate with

Monitor these frequently (every 30 seconds to 1 minute) from multiple regions. Set up aggressive alerting - phone calls, not just Slack messages.

Important endpoints

Endpoints that matter, but won't immediately break everything:

  • User profile and settings
  • Search and filtering
  • Non-critical data fetches

Monitor these every few minutes. Slack alerts are usually fine.

Everything else

Dashboard data, analytics, internal tools. If these break, it's annoying but not catastrophic.

Monitor less frequently, or don't monitor at all. Focus your attention on what matters.

Setting up API monitoring

Here's how to set up basic API monitoring with OnlineOrNot:

1. Start with your most critical endpoint

Pick one endpoint - probably your main API health check or authentication endpoint. Don't try to monitor everything at once.

2. Create the check

You'll need:

  • The URL to monitor
  • The HTTP method (GET, POST, etc.)
  • Any required headers (like Authorization or Content-Type)
  • The expected response (status code, maybe specific JSON fields)

3. Set the check interval

For critical APIs, every 30 seconds is a good starting point. You can always adjust later.

4. Add assertions

This is where API monitoring gets more useful than basic uptime checks. You can verify:

  • Status code is 200
  • Response contains specific JSON fields
  • Response time is under a threshold
  • Response body matches a pattern

For example, if you're monitoring a /health endpoint that returns {"status": "ok"}, you'd assert that the response contains that exact JSON.

5. Configure alerts

Choose where alerts go. For critical endpoints, I recommend at least two channels:

  • Immediate: Phone/SMS/PagerDuty for wake-you-up urgency
  • Awareness: Slack/Email for the rest of the team

6. Add more endpoints gradually

Once your first check is running smoothly, add the next most critical endpoint. Build up your coverage over time.

Going beyond uptime

Basic "is it up?" monitoring is table stakes. Here are some things worth tracking as you mature:

Response time trends

A slow API is almost as bad as a down API. Track your p50, p95, and p99 response times over time. If your p95 suddenly jumps from 500ms to 2 seconds, something changed.

Error rate

What percentage of requests are returning errors? Even if the API is "up", a 5% error rate means 1 in 20 users is having a bad time.

Multi-region checks

Your API might work perfectly from AWS us-east-1 but be slow or broken from Europe. Running checks from multiple locations catches regional issues.

Authenticated checks

Your public health endpoint might return 200, but what about authenticated requests? Sometimes auth middleware breaks while the rest of the API is fine.

Dependency health

If your API depends on a database or third-party service, monitor those too. When something breaks, you want to know if it's your code or a dependency.

Common mistakes

A few things I've seen teams get wrong:

Only monitoring the health endpoint

/health returning 200 doesn't mean your API works. I've seen health checks succeed while the actual API was completely broken because the database was down.

Monitor real endpoints that exercise real functionality.

Ignoring response content

Checking for a 200 status isn't enough. An API can return 200 with an error message in the body, or with empty data when there should be results.

Use assertions to verify the response makes sense.

Too many alerts

If every alert goes to the same place with the same priority, your team will start ignoring them. Differentiate between "wake someone up" and "look at this tomorrow".

I wrote more about this in saving your team from alert fatigue.

Not testing from where users are

If all your monitors run from the same region as your servers, you'll miss regional issues. Users in Australia don't care that your API is fast from Virginia.

Forgetting about dependencies

Your API might be working perfectly, but if Stripe is down, your checkout is broken. Consider monitoring critical third-party APIs too, or at least having alerts for when they have issues.

API monitoring isn't complicated, but it does require some thought about what matters most to your business.

Start with your most critical endpoint. Get that working well. Then expand from there.

The goal isn't to monitor everything - it's to make sure you find out about problems before your users do.

]]> <![CDATA[Cron job monitoring: How to know when your scheduled tasks fail]]> https://onlineornot.com/cron-job-monitoring-guide https://onlineornot.com/cron-job-monitoring-guide Tue, 03 Mar 2026 00:00:00 GMT Cron jobs have a nasty habit of failing silently.

Your backup script stops running. Your daily report never gets sent. Your database cleanup job crashes halfway through. And you don't find out until someone asks "hey, why haven't we had a backup in three weeks?"

The problem is that cron doesn't care if your job succeeds or fails. It just runs the command and moves on. No alerts, no notifications, nothing.

Let's fix that.

Table of contents

Why cron jobs fail silently

Cron was designed in the 1970s. Back then, the assumption was that a sysadmin would check the server regularly and notice if something was wrong.

That's not how most of us work today.

When a cron job fails, a few things might happen:

  1. Nothing - The job crashes, cron shrugs, and nobody knows
  2. An email gets sent - Cron can email output to root, but who checks that?
  3. A log entry appears - Somewhere, buried in /var/log/syslog

None of these are great for catching problems quickly.

The real killer is when a job doesn't run at all. Maybe the server rebooted and cron didn't start. Maybe someone accidentally deleted the crontab. Maybe the disk filled up and cron couldn't write its lockfile.

In these cases, there's nothing to log. The job just... doesn't happen.

How cron job monitoring works

The solution is a "dead man's switch" (also called heartbeat monitoring).

The idea is simple:

  1. At the end of your cron job, you ping a URL
  2. A monitoring service tracks these pings
  3. If a ping doesn't arrive when expected, you get an alert

It's called a dead man's switch because the alert triggers on absence of activity, not presence. If your job stops running for any reason - crash, server down, crontab deleted - you'll know.

Here's what it looks like in practice:

# Before: Your cron job
0 2 * * * /home/user/backup.sh

# After: With monitoring
0 2 * * * /home/user/backup.sh && curl -fsS --retry 3 https://oonchk.com/abc123

The && is important - the curl only runs if backup.sh exits successfully. If your script fails, the ping doesn't get sent, and you get an alert.

Setting up cron job monitoring

Here's how to set it up with OnlineOrNot:

1. Create a heartbeat monitor

Give it a name (like "nightly-backup") and set the expected schedule. If your job runs daily at 2am, tell the monitor to expect a ping every 24 hours.

2. Add a grace period

Jobs don't always run at exactly the same time. A backup might take 5 minutes one day and 20 minutes the next. Set a grace period that accounts for normal variation.

For a daily job, 30-60 minutes of grace is usually fine. For an hourly job, maybe 10 minutes.

3. Add the ping to your cron job

You'll get a unique URL. Add it to the end of your cron command:

0 2 * * * /home/user/backup.sh && curl -fsS --retry 3 https://oonchk.com/your-unique-id

The flags:

  • -f - Fail silently on HTTP errors
  • -s - Silent mode (no progress output)
  • -S - Show errors if they occur
  • --retry 3 - Retry a few times if the network hiccups

4. Configure your alerts

Choose where you want to be notified: Slack, email, PagerDuty, SMS, etc.

For critical jobs (like backups), I'd recommend at least two channels. Slack for awareness, phone/SMS for wake-you-up urgency.

That's it. If your job stops running, you'll know within minutes instead of weeks.

Common issues with cron jobs

Over the years, I've seen the same problems come up again and again:

The job runs but fails

Your script starts, hits an error, and exits with a non-zero status. If you're using && before your ping (as above), this is caught automatically.

If you're not using &&, your monitoring will think everything's fine when it isn't.

The job takes too long

A job that usually takes 10 minutes suddenly takes 2 hours. This might be fine, or it might mean something's wrong.

Good monitoring tools let you track job duration, not just completion. If a job starts taking significantly longer than usual, that's worth investigating.

The job overlaps with itself

Your hourly job takes 90 minutes to run. Now you've got two instances running at once, possibly fighting over the same resources.

This isn't strictly a monitoring problem, but it's something to watch for. Use a lockfile or flock to prevent overlapping runs:

0 * * * * flock -n /tmp/myjob.lock /home/user/myjob.sh && curl ...

Environment differences

This one's classic: the job works when you run it manually, but fails under cron.

Cron runs with a minimal environment. Your $PATH is different, your shell config isn't loaded, and environment variables you take for granted aren't set.

Always use full paths in cron jobs, and set any required environment variables explicitly.

The server rebooted

Cron usually starts automatically on boot, but not always. And even if cron is running, your job might depend on other services that aren't ready yet.

This is where heartbeat monitoring really shines - if the job doesn't run after a reboot, you'll know.

What to monitor

Not every cron job needs monitoring. Here's how I think about it:

Always monitor:

  • Backups (you really don't want to find out these stopped working when you need them)
  • Jobs that affect money (billing, invoices, payments)
  • Data pipelines and syncs
  • Security-related jobs (certificate renewal, log rotation)

Probably monitor:

  • Reports and notifications
  • Cleanup jobs
  • Health checks

Maybe skip:

  • Low-importance jobs you'd barely notice if they stopped
  • Jobs that have other visible effects when they fail

The rule of thumb: if you'd want to know within an hour that this job stopped running, monitor it.

Silent failures are the worst kind of failures. By the time you notice something's wrong, the damage is already done.

Adding a simple ping to your cron jobs takes about 30 seconds per job. It's one of those small investments that pays off enormously when things go wrong.

And things always go wrong eventually.

]]> <![CDATA[MTTR: What it means and how to improve it]]> https://onlineornot.com/mttr https://onlineornot.com/mttr Tue, 03 Mar 2026 00:00:00 GMT It's 2am. Your phone buzzes with an alert. Your site is down.

You fumble for your laptop, try to remember where the logs are, SSH into the server, and eventually figure out the problem. By the time you've fixed it, it's 4am.

The next morning, someone asks: "How long did it take to resolve?"

That's your MTTR.

Table of contents

What is MTTR?

MTTR stands for Mean Time to Resolution (or Recovery, or Repair - more on that in a second). It measures the average time it takes your team to resolve incidents.

The idea is simple: the faster you can get your service back online, the less pain for your customers and your business.

If you had three incidents last month that took 30 minutes, 45 minutes, and 15 minutes to resolve, your MTTR would be 30 minutes.

The four types of MTTR

Here's where things get confusing. MTTR can actually mean four different things:

Mean Time to Resolution - The total time from incident start to "everything's back to normal". This includes detection, diagnosis, fixing, and verification. This is the most common definition for software teams.

Mean Time to Recovery - How long until service is restored. You might not know why it broke, but users can use the product again.

Mean Time to Repair - Just the time spent actively fixing. Doesn't include waiting around or diagnosis. More common in hardware/manufacturing contexts.

Mean Time to Respond - How quickly someone acknowledges the incident after an alert fires. Sometimes called MTTA (Mean Time to Acknowledge).

For most of us running web services, Mean Time to Resolution is what we care about - it captures the full customer impact.

The key thing is to pick one definition and stick with it. Otherwise your metrics become meaningless.

How to calculate MTTR

The formula is straightforward:

MTTR = Total downtime / Number of incidents

So if you had 4 incidents last month with a combined downtime of 2 hours:

MTTR = 120 minutes / 4 incidents = 30 minutes

That's it. Nothing fancy.

One thing to watch out for: make sure you're measuring from when the incident actually started, not when you first noticed it. If your site was down for an hour before anyone noticed, that hour counts.

This is why monitoring your uptime matters - it reduces the gap between "incident started" and "someone's working on it".

What's a good MTTR?

It depends on what you're building, but here are some rough benchmarks from DORA's research:

| Performance | MTTR | |-------------|------| | Elite | Less than 1 hour | | High | Less than 1 day | | Medium | Less than 1 week | | Low | More than 1 week |

If you're running a SaaS product, you probably want to aim for under an hour. For critical infrastructure like payments, under 15 minutes.

But here's the thing: don't obsess over the number.

I've seen teams game their MTTR by closing incidents early, or not counting certain outages as "real" incidents. That's worse than having a high MTTR, because now you're lying to yourself.

Focus on actually getting better at resolving incidents, and the number will follow.

MTTR vs MTBF

You'll often see MTTR mentioned alongside MTBF (Mean Time Between Failures).

MTTR tells you how fast you fix things.

MTBF tells you how often things break.

Both matter. A 5-minute MTTR is great, but if you're having incidents every day, something's fundamentally broken.

Ideally you want:

  • High MTBF (things rarely break)
  • Low MTTR (when they do break, you fix them fast)

If you're having lots of incidents, work on reliability first. If incidents are rare but take forever to resolve, work on your incident response.

How to improve your MTTR

Here are practical things you can do today:

Detect incidents faster

You can't fix what you don't know about. The time between "something broke" and "someone's looking at it" is often the biggest chunk of your MTTR.

Set up uptime monitoring that checks your site frequently (every 30 seconds is ideal) from multiple locations. When something goes wrong, you want to know within minutes, not hours.

Write runbooks

At 3am, you won't remember exactly how to restart that one service, or where the logs are, or who to escalate to.

Write it down. Keep a document for each service with:

  • What does this service do?
  • How do I check if it's working?
  • Common problems and how to fix them
  • Who to escalate to if you're stuck

I've written more about this in writing your first runbooks.

Reduce noise

If your team is drowning in alerts, the important ones get lost. When everything's urgent, nothing is.

Review your alerts regularly. If an alert fires and there's no action to take, delete it or change its threshold. I wrote a whole article on saving your team from alert fatigue if you want to dig deeper.

Use status pages

During an incident, your support team gets flooded with "is it down?" messages. This takes time away from actually fixing the problem.

A status page lets customers check the status themselves. Less noise for your team, faster resolution.

Run postmortems

After every significant incident, figure out what happened and why. Not to blame anyone, but to learn.

Ask:

  • What broke?
  • How did we find out?
  • What slowed us down?
  • How do we prevent this from happening again?

The goal is to get a bit better each time. Over months, these small improvements compound.

Keep it simple

This one's harder to act on, but worth mentioning: complex systems have more ways to fail, and take longer to debug.

If you're a small team and your architecture looks like a distributed systems textbook, maybe it's time to simplify. I wrote about this more in monitoring your web application as a small team.

MTTR isn't the only metric that matters, but it's a good one to track. It keeps you honest about how your team handles incidents.

Start measuring it, pick one thing to improve, and iterate from there.

]]> <![CDATA[OnlineOrNot updates from February 2026]]> https://onlineornot.com/onlineornot-updates-from-2026-february https://onlineornot.com/onlineornot-updates-from-2026-february Tue, 03 Mar 2026 00:00:00 GMT February was all about making OnlineOrNot better for dev teams. I shipped audit logs for teams, added support for two-factor authentication and passkeys, unified the webhooks system across uptime checks, heartbeats, and status pages, and made it possible to connect multiple Discord channels.

Table of contents

What's new

Features for the platform

Audit logs

If you're on a team, at some point someone's going to change a check or update a status page and nobody will remember who did it, or when. You end up asking around in Slack, or worse, guessing.

OnlineOrNot now has an audit log. Every action taken in your organisation is logged: who did it, what they did, when, and from where. Creating checks, updating status pages, deleting maintenance windows, inviting team members - it's all there.

Each entry shows the actor (user or API token), the action performed, the IP address and location, and a timestamp. If the action had extra context (like which fields changed), you can expand the row to see the full metadata.

You'll find the audit log in the sidebar. It's paginated, so even if your team is busy, it stays fast.

The audit log also works through the REST API, so you can pull events programmatically if you need to feed them into your own logging pipeline or compliance tooling.

Two-factor authentication and passkeys

OnlineOrNot accounts have always been secured with email-based one-time passwords. Not great when your email service is down in the middle of an incident.

As of February 16th, OnlineOrNot supports two-factor authentication (2FA) using authenticator apps like 1Password, Authy, or Google Authenticator. You can enable it from your account settings: if your account has a password, scan the QR code with your authenticator app, and you're done. You'll also get backup codes in case you lose access to your authenticator.

On top of that, you can now register passkeys for your account. Passkeys let you log in with your fingerprint, face, or a hardware security key instead of typing a password. If you've used passkeys with GitHub or Google, it's the same idea. There's a "Login with passkey" button on the login page, and you can manage your passkeys in account settings.

Both features are optional. If you're happy with email-based login, nothing changes on your end.

Webhooks API and custom API token expiry

You've been able to create webhooks in the dashboard for a while now, but a few folks have asked about managing them through the API. Maybe you're spinning up new environments and don't want to click through the UI each time, or you'd rather have your webhooks defined alongside the rest of your infrastructure.

So I've added webhook management to the REST API. You can now list, create, update, and delete webhooks programmatically. Each webhook can subscribe to status page incident events (started, updated, resolved) and be scoped to specific status pages. You'll find the details in the API docs.

API token expiry

While I was in there, I also added customisable API token expiry. You can now pick an expiration date when creating API tokens - handy for CI tokens or anything you'd rather not have floating around forever.

Sidebar showing Webhooks and API Tokens

Both API tokens and Webhooks also have their own section in the sidebar now, which should make them easier to find.

Features for Checks

Unified webhooks

When I first added webhooks to OnlineOrNot, they only worked with status page incidents. Uptime checks and heartbeats had their own separate webhook alert system tucked away in the integrations settings, which meant you had to configure things in two different places.

I've now unified this. The webhooks you create in the Webhooks page can be attached to uptime checks, heartbeats, and status pages all in one place. You pick which events you care about (uptime.down, uptime.up, heartbeat.down, heartbeat.up, plus the existing status page incident events), and which resources should trigger them.

If you had generic webhooks configured before, they've been automatically migrated over. Your existing webhook URLs, check associations, and heartbeat associations are all there. The payload format hasn't changed either, so anything receiving your webhooks will keep working.

On-call integrations like PagerDuty, Opsgenie, Grafana OnCall, and Spike now live under Settings > Integrations > On-call.

Multiple Discord channels

Up until now, you could only connect one Discord channel to OnlineOrNot. If you wanted alerts in different channels for different checks, you were out of luck.

This worked fine for smaller teams, but once you start shipping a bit, having a single channel for all of your services' alerts starts to get noisy.

You can now connect multiple Discord channels, and pick which ones should receive alerts for each uptime check and heartbeat. The flow is the same as before: click "Add to Discord", authorize in Discord, and give your integration a name so you can tell them apart later.

Discord channel naming

If you already had a Discord integration set up, nothing changes on your end. Your existing checks will keep alerting to the same channel. You can add more channels whenever you need them.

]]> <![CDATA[Website monitoring: What it is and why you need it]]> https://onlineornot.com/website-monitoring-guide https://onlineornot.com/website-monitoring-guide Tue, 03 Mar 2026 00:00:00 GMT Your website is down. You don't know it yet.

A customer tries to visit, gets an error, and leaves. Then another. And another. An hour passes before someone finally emails you: "Hey, is your site working?"

By then, the damage is done.

This is why website monitoring exists - to make sure you find out about problems in minutes, not hours.

Table of contents

What is website monitoring?

Website monitoring is exactly what it sounds like: regularly checking that your website is working.

At its simplest, a monitoring service visits your site every few minutes and checks if it loads. If it doesn't, you get an alert.

But "working" can mean different things:

  • Is it reachable? Can users get to it at all?
  • Is it fast? Does it load in a reasonable time?
  • Is it correct? Does it show the right content?

Good monitoring checks all three.

How it works

A monitoring service runs checks from servers around the world. Every 30 seconds (or whatever interval you set), it makes a request to your site and checks the response.

If your site returns a 200 status code and loads within your threshold, the check passes. If it returns an error, times out, or is missing expected content, the check fails.

When checks fail, you get notified. Most services let you configure multiple notification channels - email, Slack, SMS, phone calls - depending on how urgently you need to know.

The "from around the world" part matters. Your site might work perfectly from your office in London but be completely unreachable from Australia because of a CDN issue. Multi-region monitoring catches these problems.

What can go wrong

Websites go down for all sorts of reasons. Some common ones:

Server crashes - Your web server runs out of memory, hits 100% CPU, or just dies. The process stops, and requests go unanswered.

Deployment failures - You push a bad deploy. The new code crashes on startup, or has a bug that breaks the homepage.

SSL certificate expiry - Your certificate expires, and browsers refuse to load your site. Users see scary security warnings.

DNS issues - Your domain stops resolving. Maybe your registrar had an issue, maybe you forgot to renew, maybe someone misconfigured something.

Database problems - Your database runs out of connections, fills up its disk, or just becomes unreachable. Your app can't load data, so pages fail.

Third-party failures - A CDN, DNS provider, or cloud region goes down. Your code is fine, but users can't reach it.

Traffic spikes - You get featured on Hacker News or Reddit. Your server can't handle the load and starts dropping requests.

The frustrating thing is that many of these failures are silent. Your server might be happily serving errors while you have no idea anything's wrong.

Setting up monitoring

Here's how to get started with website monitoring:

1. Start with your homepage

Add your main URL. This is the most important page - if it's down, your whole site is effectively down.

2. Set a reasonable check interval

Every 30 seconds to 1 minute is good for most sites. Faster catches issues quicker but generates more traffic.

3. Choose your regions

Pick regions where your users actually are. If you're serving a US audience, check from US locations. If you're global, check from multiple continents.

4. Add content verification

Don't just check for a 200 status code. Add a text check to verify the page contains expected content.

For example, if your homepage always has "Welcome to Acme Corp", check for that text. This catches cases where your server returns 200 but serves an error page or blank content.

5. Set up alerts

Configure where notifications go. At minimum, email. For critical sites, also Slack and SMS.

6. Test it

Deliberately break something and verify you get an alert. You don't want to find out your alerting is broken during a real outage.

What to monitor

Start with the basics and expand from there:

Essential

  • Homepage - Your main entry point
  • Login/signup pages - Critical for user access
  • Core product pages - Whatever your users actually use

Important

  • API endpoints - If you have an API, monitor it separately
  • Payment flows - Checkout pages, subscription management
  • SSL certificate - Monitor expiry dates

Nice to have

  • Marketing pages - Blog, docs, landing pages
  • Third-party dependencies - Status pages of services you rely on
  • Performance baselines - Alert if response time degrades significantly

You don't need to monitor everything. Focus on pages where downtime actually hurts your business.

Alerting that works

The point of monitoring is to get alerts when things break. But alerting is easy to get wrong.

Too many alerts and your team ignores them. Every alert becomes noise.

Too few alerts and you miss real problems.

Wrong channels and the right people don't see the alert in time.

Here's what I recommend:

For critical pages

  • Check frequently (every 30 seconds)
  • Alert immediately on failure
  • Use aggressive channels: SMS, phone call, PagerDuty
  • Multiple team members should receive alerts

For important pages

  • Check every few minutes
  • Allow 2-3 failures before alerting (to filter out blips)
  • Use Slack or email
  • Route to the relevant team

For everything else

  • Check less frequently
  • Higher failure threshold before alerting
  • Email or a low-priority Slack channel

The key is matching the alert urgency to the business impact. Your marketing blog being slow for 5 minutes is not the same as your checkout being down.

I wrote more about this in saving your team from alert fatigue.

Website monitoring is one of those things that feels unnecessary until you need it.

The first time you catch an outage in 2 minutes instead of 2 hours, you'll wonder how you ever operated without it.

Start simple: one check on your homepage, alerts going to your email. You can always add more later.

]]> <![CDATA[OnlineOrNot updates from January 2026]]> https://onlineornot.com/onlineornot-updates-from-2026-january https://onlineornot.com/onlineornot-updates-from-2026-january Wed, 04 Feb 2026 00:00:00 GMT Hopefully this will be one of the last major "behind-the-scenes" updates for a while, because OnlineOrNot's frontend now runs on a React framework that's easy to deploy across multiple providers, and is fully off GraphQL, being powered by its own REST API.

Table of contents

What's new

Features for the platform

Expanded REST API

As of today, you can manage almost everything in OnlineOrNot programmatically:

  • Heartbeat checks - create, list, update, pause, mute, and delete heartbeat checks
  • Maintenance windows - create, update, list, and delete maintenance windows
  • Status page incidents - create and manage incidents and incident updates, including per-component status changes
  • Scheduled maintenance - schedule, update, and cancel maintenance events on your status pages
  • Status page subscribers - manage email subscribers for your status pages
  • Status page components - add, reorder, and remove components
  • Team management - invite, list, and remove team members

As I mentioned earlier, the OnlineOrNot dashboard itself has been rewritten to use the same REST API instead of GraphQL, so the API is a first-class citizen that can't go out of date or become unmaintained. As a side-effect, we get a faster and more reliable dashboard.

Every new endpoint follows the same conventions as the existing API, with consistent error responses and OpenAPI documentation.

Features for Checks

Per-region response time breakdown

You know your API is slow, but is it slow everywhere, or just from one region? Maybe it's a database replica lagging in Europe, or a CDN cache miss in Asia-Pacific.

OnlineOrNot used to average response times across all monitoring regions into a single number, which made it hard to tell at a glance.

OnlineOrNot regional response times

Your uptime check detail pages now show a per-region breakdown of response times, so you can see exactly where things are slow and narrow down the cause faster.

Features for Status Pages

They're even faster now

When your service goes down, the last thing you want is for your status page to be slow to load too. Your customers are already stressed, and a sluggish status page doesn't help.

I've spent some time optimizing how OnlineOrNot's public status pages are served. Behind the scenes, status pages now run with targeted placement to minimise round-trip time, lazy loading has been replaced with optimized server-side queries, and the overall page weight has been reduced.

The result: your status page loads faster, and incidents display more quickly during the moments that matter most.

Incident improvements

Speaking of incidents: after an incident gets resolved, one of the first things your customers want to know is "how long was it down?"

Previously, they'd have to do the mental math themselves by comparing timestamps.

Resolved incidents on your status pages now display how long they took to resolve (e.g., "Resolved after 7 minutes"), so your customers can see the impact at a glance.

OnlineOrNot Incidents view

I've also added incident timelines to the recent incidents section of your status page. Your customers can now see the full progression of an incident without having to click into each one individually.

]]> <![CDATA[Looking back on 2025, and what's next]]> https://onlineornot.com/2025 https://onlineornot.com/2025 Sun, 11 Jan 2026 00:00:00 GMT Continuing on with the tradition I started to wrap up 2024, in this article I'll go over what's new in OnlineOrnot from 2025.

Table of Contents

What went live in 2025

In 2025, I committed changes to OnlineOrNot 1347 times across 229 days (35% more than in 2024). The way I work remains unchanged - folks contact me about things they wish OnlineOrNot could do, I look at how that fits in with the existing roadmap, and I build it in the two hours a day I have before my workday starts.

It's always an odd feeling looking at these stats at the end of a year as a founder - you feel as though you didn't release nearly enough features that your customers ask for while simultaneously being impressed at the amount of work shipped.

General

Checks

Status Pages

  • migrated status page queries to ClickHouse for significantly faster load times
  • added incident resolution time display so subscribers can see how long incidents took to resolve
  • improved the subscriber modal experience
  • added an RSS feed for status page updates
  • aligned status page dark mode with the main app's theme

Under the hood

This section is a bit more technical than the others, and is mainly for folks that want to know how OnlineOrNot runs under the hood.

2025 was a big year for revisiting infrastructure decisions I made while building OnlineOrNot. While many decisions made sense for the product at the time they were made, the product has grown, and boring technologies that were hard to adopt in the past became much easier.

  • Authentication rewrite
    • Migrated from Auth.js (originally known as NextAuth) to Better Auth, simplifying the codebase significantly and unlocking the ability to add SSO, password auth, and more.
  • Migrated timeseries data from Postgres to Clickhouse
    • Despite being well-optimised with covering indexes, the queries powering many of OnlineOrNot's dashboards were slowing down when you had too many uptime checks. Some queries were taking 600ms on average, with spikes above 10 seconds. Moving that data into Clickhouse dropped that down to 60ms on average.
  • Dropped Next.js and Remix, adopted React Router 7
    • Both the frontend for status pages (what you see when you load a status page such as the Hacker News status page) and the frontend for OnlineOrNot now run React Router 7 and the latest version of React, significantly simplifying both frontends.
  • Dropping GraphQL in favor of REST
    • While GraphQL let me iterate rapidly on OnlineOrNot's dashboard, each feature I built in GraphQL then had to be rewritten for REST. As a result, priorities got in the way, and the REST API lagged behind.
    • The fix is simple: all new code is written in REST, and I got to work migrating off GraphQL. The migration is still in progress, but it'll finish early in 2026.
]]> <![CDATA[OnlineOrNot's lessons from Cloudflare's outage on 2025-11-18]]> https://onlineornot.com/onlineornot-lessons-from-cloudflare-outage-2025-11-18 https://onlineornot.com/onlineornot-lessons-from-cloudflare-outage-2025-11-18 Wed, 19 Nov 2025 00:00:00 GMT On 2025-11-18 at 11:48 UTC, Cloudflare declared an incident affecting the global network (that also affected OnlineOrNot). OnlineOrNot monitors websites, APIs, web apps, and cron jobs, while providing status pages as well.

While we partially mitigated the issue by enabling a fallback to AWS-based monitoring, between 13:00 UTC and 14:33 UTC failing checks went unreported, heartbeat checks over-reported, and status pages were unavailable. This postmortem outlines what went wrong, and what will be done to fix the issues.

Table of contents:

OnlineOrNot's dashboard and public API

OnlineOrNot's frontend (onlineornot.com) is currently deployed in a single place: AWS Lambda (through Vercel) in us-east-1, with the DNS proxied on Cloudflare. The public API on the other hand, runs purely on Cloudflare Workers.

OnlineOrNot itself paged me to say that something was wrong with the public API at 11:44 UTC. Clicking on the link in the alert showed me Cloudflare's typical error screen, with a key difference - my host was still working, Cloudflare was the one with the error.

As a means of working around it, I disabled DNS proxying in OnlineOrNot's Cloudflare DNS settings, and got access back to the frontend. I then noticed the API was still down.

In disabling DNS proxying, I discovered a bug in OnlineOrNot's frontend that relied on Cloudflare's proxying to function properly.

It would remain down until 15:41 UTC, when I was able to login to the Cloudflare dashboard again, and re-enable DNS proxying. Had I taken no action with the DNS settings, OnlineOrNot's dashboard would have been accessible at 14:33 UTC.

OnlineOrNot's uptime checking system

OnlineOrNot's uptime checking system is both multi-cloud and multi-region, running in AWS and Cloudflare Workers. Having learned our lesson from a large us-east-1 outage in 2021, the failover part of the system is frequently tested. Things still went wrong in new and different ways.

Each part of OnlineOrNot's Cloudflare system continuously reads config from KV, and writes health data to Workers Analytics Engine (WAE). There's also an autonomous Cloudflare Worker process that queries WAE every minute, and if the health of the system degrades, it updates the config in KV, and the system starts running on AWS.

During this incident, no part of this autonomous health system was functioning as expected, and I needed to manually trigger the CI process to redeploy the service to make it default to running in AWS. At 12:12 UTC, 28 minutes after noticing the disruption, OnlineOrNot was back up and running checks in AWS, and sending alerts.

These checks were however, less accurate than usual. The uptime checks in AWS relied on Cloudflare to double-check a URL is down. When the connection to Cloudflare was completely severed between 13:00 UTC and 14:33 UTC, the checks timed out in the queue, resulting in failing checks being skipped:

OnlineOrNot missing checks

(This chart displays time in UTC+1)

OnlineOrNot's heartbeat (cron) monitoring system

OnlineOrNot's heartbeat system suffered similar issues to the uptime checking system. Heartbeats provide healthcheck URLs (also hosted on Cloudflare), which were unreachable between 13:00 UTC and 14:33 UTC.

The other side of the system that looks for heartbeats to alert on successfully ran from AWS thanks to the failover.

As a result, the heartbeat system started sending alerts for cron jobs that were still running.

OnlineOrNot's status pages

OnlineOrNot's status pages are powered by a Cloudflare Worker serving the frontend, and a Cloudflare Worker proxying the private API.

During the outage, status pages were completely inaccessible between 13:00 UTC and 14:33 UTC.

For customers experiencing their own incidents at the same time as the Cloudflare outage, they had no way to update their status pages or show their users what was happening.

Remediation steps

Now that the systems are back online and functioning normally, I have begun work to harden them against failures like this in the future. In particular:

  • Adding multi-cloud fallback services to all APIs and frontends - one vendor is not enough for services like OnlineOrNot
  • Eliminating the cross-cloud double checking on uptime checks in favor of same-region retries
  • Implementing automatic DNS failover for status page domains to redirect to the fallback version during outages
  • Reviewing and improving the autonomous health monitoring system that failed to trigger AWS failover automatically

An outage like this is unacceptable for a monitoring system like OnlineOrNot. I've architected our systems to be multi-cloud and resilient, but this incident exposed gaps in our failover automation and dependencies I hadn't fully accounted for.

I apologize for the disruption this caused to your monitoring and incident response yesterday.

]]> <![CDATA[OnlineOrNot updates from August 2025]]> https://onlineornot.com/onlineornot-updates-from-2025-august https://onlineornot.com/onlineornot-updates-from-2025-august Wed, 03 Sep 2025 00:00:00 GMT A bit more behind-the-scenes work than usual this month, but I still managed to ship some public-facing features you might be interested in. Logging in, and clicking around the dashboard just got 60% faster, and we're just getting started.

Table of contents

What's new

General features for OnlineOrNot

One-time password login

To login to OnlineOrNot, you used to receive a magic link to let you sign-in on the same device. Various email-scanning providers broke this functionality from time to time, and unfortunately the experience wasn't as smooth as I'd have liked.

As a result, we now use one-time passwords:

OnlineOrNot's OTP login screen

The idea remains the same (you'll get an email containing a code, and need it to login), except you can check your email on one device, and login on another, and email-scanning technology won't interfere with your login attempt.

Features for Checks

Historical overview

Up until recently, you couldn't fetch more than 24 hours of uptime data about a check in the OnlineOrNot dashboard.

In August, I took the time to start migrating OnlineOrNot's timeseries data into a Clickhouse instance, and now for the first time, you can query up to 30 days worth of data in a few hundred milliseconds:

OnlineOrNot's uptime check overview

(every paginated screen in OnlineOrNot now remembers how many results you wanted to see too)

There's more work to be done to finish the migration (we'll be moving heartbeats over too, and you'll be able to see more detailed logs), but it's looking promising.

Additionally, we're now using OnlineOrNot's own public HTTP API to power this screen, and over time we should have 100% of the functionality provided by the public API.

Features for Status Pages

Links to external status pages

Sometimes your providers have incidents, and you want to give your users more context about the issue. We now link to your external provider's status page when you link a third-party component:

OnlineOrNot external status page linking

RSS feed

There's now an RSS feed available on each status page:

OnlineOrNot's status page RSS feed

Letting your users subscribe to your status page's updates without needing to provide their email address.

Thank you

OnlineOrNot only exists because of folks like you. I don't have a massive marketing budget, or time to do it all myself.

OnlineOrNot sustains itself from folks like you enjoying it enough to tell your friends and colleagues about it, and if you don't enjoy it - let me know where it's falling short of your expectations.

]]> <![CDATA[OnlineOrNot updates from June/July 2025]]> https://onlineornot.com/onlineornot-updates-from-2025-june-july https://onlineornot.com/onlineornot-updates-from-2025-june-july Mon, 04 Aug 2025 00:00:00 GMT In this latest update, I'll walk you through a few features I added that will make working with uptime checks less noisy, an alerts integration with Teams, and a few behind-the-scenes changes that will finally let me build mobile apps for OnlineOrNot.

Table of contents

What's new

Features for Checks

Alert muting

Sometimes you need to keep monitoring running, but temporarily silence the alerts. Maybe you're investigating an issue, or your service is in a known degraded state while you work on a fix.

OnlineOrNot now lets you manually mute individual uptime and browser checks, giving you the flexibility to silence alerts without stopping monitoring entirely.

When you mute a check, OnlineOrNot will:

  • Continue running the uptime check
  • Keep collecting response time and availability data
  • Stop sending alerts and status page updates about downtime or issues

Perfect for those times when you're aware of an issue and actively working on it, but don't want to be bombarded with alerts or confuse your users with status page notifications.

OnlineOrNot - alert muting

You can also use the API to mute or pause checks from your own codebase.

Confirmation and Recovery periods

Ever been in the middle of fixing an incident, and your service starts recovering but isn't quite 100% yet? Some requests succeed, others fail, and your monitoring keeps flip-flopping between "down" and "up" while you're still working on the fix?

You end up frustrated, and with noisy alerts and status pages that don't accurately reflect that the incident isn't fully resolved.

To solve this, OnlineOrNot now supports confirmation periods and recovery periods:

  • Confirmation period - wait for consecutive failures over a specified time period before marking a service as down and sending alerts
  • Recovery period - wait for consecutive successes over a specified time period before marking a service as recovered and closing incidents

For example, you can now configure a check to wait for five minutes of consecutive failures before alerting (avoiding false alarms), and wait for fifteen minutes of consecutive successes before marking the service as fully recovered (ensuring it's actually stable before closing the incident and confusing your users).

You can configure these settings in the "Advanced" section when creating or editing any uptime check.

OnlineOrNot - confirmation and recovery periods

Microsoft Teams integration

OnlineOrNot now integrates with Microsoft Teams, allowing you to receive uptime and heartbeat alerts directly in your Microsoft Teams channels. With this integration, you'll get clear, actionable alerts when services go down, and recovery notifications when they're back up, as well as reminder alerts (if configured).

To get started, head to your OnlineOrNot settings, click the Integrations tab, and connect your Microsoft Teams channel. You can configure different channels for different services, just like with our other integrations.

OnlineOrNot - Microsoft Teams integration

Behind-the-scenes updates no one will notice

I'm not one to publicize these types of updates too much (most folks don't care that the SQL query powering a commonly-visited page got 80% faster), but this one's worth it.

In July I took the time to rewrite OnlineOrNot's auth engine (the thing that lets you sign-up/login). The original system served its purpose, and was starting to block new features from being built.

The new system runs faster, and we finally have the ability to add:

  • Email + password login (for new users only for now, it will be possible to set a password for existing accounts soon)
  • SSO/SAML login (work in progress)
  • Mobile apps (work in progress)

New reddit community

In case you missed it, there's a bunch of places (LinkedIn, YouTube, Twitter, Discord) you can stay up to date on what I'm doing with OnlineOrNot.

As of this month, there's a new one: reddit.

]]> <![CDATA[OnlineOrNot updates from May 2025]]> https://onlineornot.com/onlineornot-updates-from-2025-may https://onlineornot.com/onlineornot-updates-from-2025-may Mon, 02 Jun 2025 00:00:00 GMT As OnlineOrNot has grown, I've been building features quickly to get them into your hands as fast as possible. However, this meant I ended up with multiple versions of similar pages that looked and worked differently from each other. This month, I focused on putting systems in place to create a consistent experience across all parts of the dashboard, making everything look and feel unified.

In terms of what's new to see: there's a new navigation sidebar (so there's room for new features in the menu), and the whole app supports dark mode now.

Table of contents

What's new

Features for the dashboard

Dark mode

While OnlineOrNot's status pages have supported dark mode for a long time now, I never found the time to update the dashboard to support it too (particularly when there were useful features to build).

Today, I'm happy to announce that the dashboard finally supports dark mode too:

OnlineOrNot main dashboard with dark mode

By default, OnlineOrNot will follow your system's dark mode setting, but you can also manually toggle it in the account dropdown in the sidebar:

OnlineOrNot main dashboard with dark mode, dropdown

While adding dark mode, I also took the time to give the oldest parts of OnlineOrNot some polish:

  • The main features are now in a mobile-friendly sidebar
  • We're now using a design system + component library to ensure a consistent user experience throughout the dashboard
  • Every date displayed in the dashboard now shows you how long ago it was, the date in your local timezone, as well as the date in UTC time

Heartbeats in greater detail

Finally, heartbeat checks (what you use to monitor cron jobs and scheduled tasks) now display status changes in a table, rather than just in a graph:

OnlineOrNot Heartbeats UI glow-up

]]> <![CDATA[OnlineOrNot updates from April 2025]]> https://onlineornot.com/onlineornot-updates-from-2025-april https://onlineornot.com/onlineornot-updates-from-2025-april Fri, 09 May 2025 00:00:00 GMT In April I worked on a behind the scenes refactor, webhooks for status pages, a new endpoint, and more.

Table of contents

What's new

Something entirely new

I spent a solid week or so this month rewriting how OnlineOrNot handles what each account can do. Before, OnlineOrNot would only let folks sign-up for a certain plan, and each plan would have predefined values for how many checks, status pages, and users they could use. Now, each account has its own separate limit, independent of the plan they're on.

In short, instead of forcing you to sign-up for a plan that gives you 100 checks when you only need 5, you can now just use 5 checks.

Alternatively, if you need thousands of checks, you can also just sign-up for thousands of checks without needing OnlineOrNot staff to set-up a new plan for you.

Features for checks

PATCH endpoint for Uptime Checks

OnlineOrNot has a new early-release HTTP API PATCH endpoint for updating uptime checks.

As of today, the endpoint only supports updating your uptime check's HTTP request headers, but will be updated to support more features over time.

PS: In case you weren't aware, OnlineOrNot has an API! You create an API token from the OnlineOrNot dashboard, by going to Settings > Developers and selecting Create Token.

Features for status pages

Webhooks for status pages

  • OnlineOrNot can now send you webhooks about your status page incidents.

    To start with, the new webhooks system is simple and only supports sending the following events:

    • status_page.incident.started
    • status_page.incident.updated
    • status_page.incident.ended

    OnlineOrNot status page webhooks

    In the longer term, OnlineOrNot will also automatically migrate existing uptime check webhooks to the new system, so that it's easier to setup webhooks for multiple resources.

Thank you

OnlineOrNot only exists because of folks like you. I don't have a massive marketing budget, or time to do it all myself.

OnlineOrNot sustains itself from folks like you enjoying it enough to tell your friends and colleagues about it.

As always: here's an ask:

I work on things that people care enough about to tell me about them. If you think there's something missing, or something that needs improving, I'd like to know more!

You can reply to this email/email me at max@onlineornot.com, if you'd like to chat.

]]> <![CDATA[How to get alerted when your EC2 instance shuts down]]> https://onlineornot.com/how-to-get-alerted-on-ec2-instance-shutdown https://onlineornot.com/how-to-get-alerted-on-ec2-instance-shutdown Tue, 29 Apr 2025 06:26:00 GMT Some of your most critical infrastructure runs on AWS EC2, so it's pretty damn important to know when your EC2 instances shut down.

Sure, chances are someone in your organisation will start kicking and screaming within 30 minutes of a particularly important instance shutting down, but we can do better than that.

When it comes to monitoring and customers (whether inside your org or outside), being proactive wins you a lot of points.

AWS Native

There are a few ways to track whenever your EC2 instance shuts down natively within AWS:

CloudWatch

Your first idea might be to set up a CloudWatch alarm whenever CPU Utilization is at 0% for 5 minutes, which is close, but doesn't actually do the job. An instance shutting down doesn't actually send data about its CPU Utilization.

What you actually want to do is to tell CloudWatch to treat missing data as breaching the threshold.

You can see the official AWS CloudWatch docs for how to do that.

EventBridge

Using AWS EventBridge, you can get notified directly when your EC2 instance changes state.

Basically, you create an SNS topic with a subscription for however you want to receive the notification, then add an EventBridge rule to react to when your EC2 instance changes state, and point it at your SNS topic.

You can see the official AWS EC2/EventBridge docs for how to do that.

(don't use) CloudTrail

You might be tempted to use AWS CloudTrail for this, but it's not the right tool for the job. CloudTrail is an audit log. It's more for tracking who in your organization ran which API call against your AWS resources, a while after the fact.

It won't catch your instance deciding to shut itself down, and it'll take a while to get data from it.

Why you shouldn't focus on server status

It doesn't directly answer your question (how do you get alerted when an EC2 instance shuts down), but you should probably reconsider your approach. Ideally, a human shouldn't get involved unless you actually need human intervention to restart your application.

What you should do instead is set up EC2 autoscaling with health checks so that the system recovers automatically whenever an instance shuts down (EC2 has docs for this).

Basically: stop looking for infrastructure issues, and actually worry about application availability instead.

Once your application can heal itself by adding resources as needed, you can start to worry about getting alerts when your application becomes unreachable.

External monitoring services

I'm not going to sugar-coat it, this article was made possible because the author runs an external monitoring service, but even if I didn't, I would still recommend this route by default.

If your application runs an HTTP service, this is the quickest win by far.

Set up an external check to visit your application's HTTP endpoint/URL, and you'll receive an SMS, Email, Slack notification, Pager alert, and more if the application stops being available on the internet.

You can see a quick guide on how to set that up.

Health checks and heartbeat monitors

If your application doesn't expose a HTTP server, you can still use an external monitoring service.

You would just need to set up your application to ping a healthcheck endpoint on a regular basis (either at the instance-level using something like cron, or from within your application)

There's also a quick guide for setting that up.

]]> <![CDATA[OnlineOrNot updates from 2025 Q1]]> https://onlineornot.com/onlineornot-updates-from-2025-first-quarter https://onlineornot.com/onlineornot-updates-from-2025-first-quarter Tue, 08 Apr 2025 00:00:00 GMT I can't believe it's April already. OnlineOrNot now lets you automatically pause checks on a recurring basis with maintenance windows, there's better support for timezones, loading a lot of data got faster, and more.

Table of contents

What's new

Features for checks

Maintenance windows

  • OnlineOrNot defaults to checking your URLs and waiting for your cron jobs twenty-four hours a day, without a break. But what if every weeknight, you had a 30 minute window where you knew your service might go down?

    OnlineOrNot now lets you pause your uptime checks and heartbeats for a given period of time, on a recurring basis:

    OnlineOrNot maintenance windows

    Once you've set a recurring maintenance window, OnlineOrNot will update your checks and heartbeats to be in "maintenance" mode for the duration of the maintenance period.

Alerts in your timezone

  • OnlineOrNot now lets you select a timezone preference for the email alerts you receive.

    This is particularly useful if you have team members in different timezones, and you want to make sure they receive alerts that are easy to understand without having to convert the timezone.

    To update your timezone preference, click into your account settings and select your timezone from the dropdown:

    OnlineOrNot custom timezone

Features for status pages

Better timezone display

  • Status pages used to default to UTC time for all updates. That's probably fine when you're in Europe, but your users could be from anywhere. Every time now displays:

    • How long ago something happened (relative time)
    • When something happened in UTC
    • When something happened in their local timezone

    OnlineOrNot status page timezone display

External status pages

Bug fixes and improvements

  • Fixed a bug to ensure OnlineOrNot can keep checking even during an incident with our main service provider
  • Rewrote how OnlineOrNot shows multiple pages of data for checks, heartbeats, and status pages. It's now a lot faster to load a lot of data.
  • To help folks monitoring their cron jobs, I built a cron command generator

Thank you

OnlineOrNot only exists because of folks like you. I don't have a massive marketing budget, or time to do it all myself.

OnlineOrNot sustains itself from folks like you enjoying it enough to tell your friends and colleagues about it.

As always: here's an ask:

I work on things that people care enough about to tell me about them. If you think there's something missing, or something that needs improving, I'd like to know more!

You can reply to this email/email me at max@onlineornot.com, if you'd like to chat.

]]> <![CDATA[Monitoring your web application as a small team]]> https://onlineornot.com/how-to-handle-monitoring-small-team https://onlineornot.com/how-to-handle-monitoring-small-team Wed, 22 Jan 2025 00:52:00 GMT When you're part of a small team running a system with thousands of users or more, it can be pretty daunting to think about going on holiday, or even relaxing for a weekend.

"What if it goes down, and I'm not there to fix it?!" you ask yourself.

While you can never really guarantee that nothing will go wrong, you can take some steps to minimise your risk of things going wrong.

Table of contents

Before you start

Choose boring technology

This mainly comes to mind for databases, but can also apply to the frameworks you choose to use: pick boring technology that has been around for ages, with a community behind it. That way when things go wrong, you won't be the only person in the world trying to solve your problem.

An early startup I worked on used a NoSQL database for clearly very relational data. Even if you ignore the fact that we weren't using the right tool for the job - we would constantly run into issues that you couldn't find on StackOverflow/Google.

In fact, the only result on Google for our problems was often an article written by us.

Since that project, I've pushed for using Postgres (released in 1996) as much as possible, and in the last five years the only outage I've seen was due to under-provisioning (we tried to use a lower RAM server to save on running costs, and it backfired spectacularly).

Choose technology you already know

In a similar vein to picking boring technology, using technology you already know leaves you with just the "simple" task of building your product.

For example, if every project your team has worked on in the last year was built in Rails + React, maybe just use Rails + React to build your project. Building your product is time-intensive as it is, without having to worry about whether you're doing it the $INSERT_LANGUAGE_HERE way.

Opt for simplicity over complexity, especially considering your team size

By simplicity, I mean perhaps running your service on a sharded multi-region database isn't the best idea for your team of two devs - a single large database instance with redundancy would work just as well (up to a certain point).

While it may technically be "better" or "correct", when you have fewer resources to investigate problems, sometimes it's easier to just bump your database up a few resource tiers (scale vertically), rather than to scale onto multiple servers (scaling horizontally). Of course, if it looks like your service needs to handle hundreds of thousands of users, perhaps then you should consider horizontal scaling.

If you find yourself constantly getting alerted and having to fight fires to keep your service running, it might be time to simplify your architecture.

While running the business

Use managed services where possible

Sure, you can run your database on any VPS hosting provider in the world for cheap, but then it's on you to handle:

  • Keeping the server updated
  • Monitoring the server
  • Ensuring the backup script runs
  • Keeping backups, deleting old ones
  • Fighting fires when things go wrong

Alternatively, you can outsource the actual running of your database to AWS. When things go wrong, you then have access to their immense support resources to resolve the issue.

I take a similar approach with payments (Stripe), email (Postmark/ConvertKit), and tracking errors (Sentry/Bugsnag/Rollbar).

Deploy at a good time

Generally speaking, deploying before heading to lunch, dinner, a holiday, or a weekend trip isn't a great idea. You need to ensure you've got time to rollback, or hotfix the change you made.

Some people prefer only releasing at quiet times of the day, when the system doesn't have many users. The upside of this is that subsequent outages would impact the least number of users, but the downside is that it probably won't be the best time for you.

Depending on the number of users you have, it might be worth looking into feature flags. Feature flags let you separate your deployments, from releases, when it comes to delivering features.

In other words, you "deploy" your change with the feature flag turned off, then once you've checked that it works on a small subset of users, you can roll out the change to your whole userbase.

Write two-way database migrations

Sometimes the fastest way to resolve an outage is to rollback the service to the last known "good version". A key part of this is to ensure your database migrations work in both directions - both when deploying the new version, and when tearing down the new version to release a previous version.

The alternative here is to roll forward with new database migrations when there's an issue, but I prefer being able to revert my database to the last known "good version".

Write good error pages

Communication is key. Good error pages let the user know that it's not their fault. The last thing you want is for your user to feel stupid after trying to submit a form and your service not responding to the request.

On top of that, a bit of communication goes a long way towards reducing the number of users messaging you when things go wrong.

Avoid alert fatigue

I previously wrote about saving your team from alert fatigue, but the gist of it is that you should tailor your alerting to the business impact of the outage.

If you've got a well-tested application where you know nothing should go wrong, you should monitor your uptime every minute, and send an alert immediately via phone call/SMS when the app becomes unresponsive.

On the other hand, if your business's legacy app becomes unreachable at the same time each day due to a database backup job running, you can setup your alerts to send after several minutes of downtime.

For more general alerts, like disk space reaching 75% on your server, or a database backup job failing, send the alerts to a Slack/Discord/Microsoft Teams channel as an FYI.

Write yourself a runbook

Keep a list of steps to run through when your service has issues. It can act as a checklist so you don't miss anything when you inevitably try to fix things after getting an alert at 3am.

As an added bonus, when you start building your dev team, you'll already have a procedure in place for what to do when the service goes down - they won't have to relearn the same mistakes again.

Mike Julian's Practical Monitoring recommends the following contents for a runbook:

  • What is this service, what does it do?
  • Who is responsible for it?
  • What dependencies does it have?
  • What does the infrastructure for it look like?
  • What metrics and logs does it emit, and what do they mean?
  • What alerts are set up for it, and why?

As well as that, make sure your alerts contain links to your runbook, so you're not franctically searching for it.

Automate your recovery

If parts of your runbook involve running a series of commands in a terminal, chances are you've got yourself a script you can automatically run without waking up a human.

It's worth keeping in mind - if the system can automatically recover, it's not worth waking up a human to check what happened.

Keep track of what goes wrong

Over time, you'll get a chance to observe your system in action. You'll find some parts of the codebase will cause more outages than others, which will nudge you to write better tests for it, or refactor to a better implementation, or better exception handling or validation.

Schedule yourself some time for these fixes, or at least add tasks to the backlog that let you track your effort to fixing these hot zones.

Run post-mortem meetings

You need to know what caused an issue to ensure you don't repeat the mistake again.

After you resolve incidents, be sure to give yourself some time (it doesn't have to be immediately after the incident) to review root causes, and come up with actions to ensure the incident doesn't happen again.

As your team grows, you want to foster a blameless culture around post-mortems. If your team fears being in trouble for mistakes, they'll either try to hide them or downplay their impact. Google's SRE book has some handy tips on building a postmortem culture.

]]> <![CDATA[What is the curl command?]]> https://onlineornot.com/curl-command https://onlineornot.com/curl-command Wed, 08 Jan 2025 00:00:00 GMT curl is one of those programs that feels like its always been there for you in a pinch, like when you're trying to debug what your API is doing, and yet we never take the time to actually learn how to use it (I only ever used it via "copy as curl" from my browser's devtools, prior to writing this article).

It's insanely powerful too, run curl --help all to see what I mean.

In this article, we're going to take the time to learn what we can do with a tiny subset of curl's options, so we don't have to look them up every time.

Not a fan of reading? Generate a curl command instead.

Table of contents

What is curl?

curl is an open-source program that lets you transfer data to, or from a server, using URLs. If you've used it before, chances are you were just checking if an HTTP API was responding with the data you expected it to - but it supports a lot more than just HTTP.

Excerpt from man curl:

curl is a tool for transferring data from or to a server using URLs. It supports these protocols: DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET, TFTP, WS and WSS.

That being said, in this article we're just going to learn how to use curl with HTTP, because that's already a challenge.

curl examples

Here are some examples of how to make requests with curl.

We'll be using https://echo.onlineornot.com - a free API that returns your request data back to you (useful for verifying your curl command does what you expect).

How to make a curl request

At its simplest, you can give curl any URL, and it will show you what's at that URL via a GET request:

curl https://echo.onlineornot.com

(see this example in our curl generator) and you get a response:

{
  "method": "GET",
  "pathname": "/",
  "searchParams": {},
  "headers": {
    "accept": "*/*",
    "accept-encoding": "gzip, br",
    "connection": "Keep-Alive",
    "host": "echo.onlineornot.com",
    "user-agent": "curl/8.7.1"
  },
  "body": "",
  "powered-by": "https://onlineornot.com"
}

As I mentioned earlier, if you don't specify the HTTP method you want to use, by default curl will send a GET request. If you want to send a POST request, you'll need to provide an additional flag, as shown below.

How to save curl output to a file

There are two ways to save curl's output to a file, instead of displaying the result in your terminal:

  1. Using the -o flag

For example:

curl -o output.json https://echo.onlineornot.com

(see this example in our curl generator) this creates a file in the same directory as where I ran the command, with the contents:

{
  "method": "GET",
  "pathname": "/",
  "searchParams": {},
  "headers": {
    "accept": "*/*",
    "accept-encoding": "gzip, br",
    "connection": "Keep-Alive",
    "host": "echo.onlineornot.com",
    "user-agent": "curl/8.7.1"
  },
  "body": "",
  "powered-by": "https://onlineornot.com"
}
  1. Using STDIN redirection (the > character)

For example:

curl https://echo.onlineornot.com > output.json

This creates the same output.json as before.

How to make a POST request with curl

You can specify the HTTP method using the -X flag, like so:

curl -X POST https://echo.onlineornot.com

(see this example in our curl generator) which outputs the following in our terminal:

{
  "method": "POST",
  "pathname": "/",
  "searchParams": {},
  "headers": {
    "accept": "*/*",
    "accept-encoding": "gzip, br",
    "connection": "Keep-Alive",
    "host": "echo.onlineornot.com",
    "user-agent": "curl/8.7.1"
  },
  "body": "",
  "powered-by": "https://onlineornot.com"
}

POST requests are cool and all, but they're not much use to the APIs you're using if you can't authenticate, or tell the API which format the data you're sending it is in.

How to POST custom headers with curl

You can provide headers to the curl command using the -H or --header flag, like so:

# we've used the backslash (\) below to split
# the command across multiple lines
curl -H 'Content-Type: application/json' \
      -X POST https://echo.onlineornot.com

(see this example in our curl generator) which outputs the following in our terminal:

{
  "method": "POST",
  "pathname": "/",
  "searchParams": {},
  "headers": {
    "accept": "*/*",
    "accept-encoding": "gzip, br",
    "connection": "Keep-Alive",
    "content-type": "application/json",
    "host": "echo.onlineornot.com",
    "user-agent": "curl/8.7.1"
  },
  "body": "",
  "powered-by": "https://onlineornot.com"
}

To add multiple headers, you'll need to add additional -H flags, like so:

curl -H 'Content-Type: application/json' \
      -H 'Authorization: Bearer a_real_token_goes_here' \
      -X POST https://echo.onlineornot.com

Now that we've provided an Authorization header and a Content-Type header, we can send data to most APIs (that support JSON data, anyway).

How to POST JSON data with curl

In case you're wondering:

but when would I send data TO an API?

You would typically send data to an API when creating or updating records.

You can send data with your request using the -d or --data flag, like so:

curl -H 'Content-Type: application/json' \
      -H 'Authorization: Bearer a_real_token_goes_here' \
      -d '{"name": "My website", "url": "https://onlineornot.com"}' \
      -X POST https://echo.onlineornot.com

(see this example in our curl generator) which outputs the following in our terminal:

{
  "method": "POST",
  "pathname": "/",
  "searchParams": {},
  "headers": {
    "accept": "*/*",
    "accept-encoding": "gzip, br",
    "authorization": "Bearer a_real_token_goes_here",
    "connection": "Keep-Alive",
    "content-length": "56",
    "content-type": "application/json",
    "host": "echo.onlineornot.com",
    "user-agent": "curl/8.7.1"
  },
  "body": "{\"name\": \"My website\", \"url\": \"https://onlineornot.com\"}",
  "powered-by": "https://onlineornot.com"
}

How to POST a JSON file with curl

You can also send files with curl using the -d flag as in the previous example, and adding @ in front of the file path.

For example, if I was in a directory containing a file called my_data.json , I could use:

curl -H 'Content-Type: application/json' \
      -H 'Authorization: Bearer a_real_token_goes_here' \
      -d @my_data.json \
      -X POST https://echo.onlineornot.com

(in this case, I've put the data from the previous example into my_data.json), which outputs the following in our terminal:

{
  "method": "POST",
  "pathname": "/",
  "searchParams": {},
  "headers": {
    "accept": "*/*",
    "accept-encoding": "gzip, br",
    "authorization": "Bearer a_real_token_goes_here",
    "connection": "Keep-Alive",
    "content-length": "56",
    "content-type": "application/json",
    "host": "echo.onlineornot.com",
    "user-agent": "curl/8.7.1"
  },
  "body": "{\"name\": \"My website\", \"url\": \"https://onlineornot.com\"}",
  "powered-by": "https://onlineornot.com"
}

How to change the user-agent header curl uses

By default, curl will use the following user-agent header for sending requests: curl/{version_number_here}

For all of the requests in this article, curl has been sending the default user-agent (in my case, curl/8.7.1).

If you're building your own tool that fetches URLs on top of curl, you might want to customize the user-agent header curl sends. You can do that by passing a user-agent header, as in the how to post custom headers with curl example above:

curl -H 'user-agent: testing-echo' \
      https://echo.onlineornot.com

(see this example in our curl generator) which outputs the following in our terminal:

{
  "method": "GET",
  "pathname": "/",
  "searchParams": {},
  "headers": {
    "accept": "*/*",
    "accept-encoding": "gzip, br",
    "connection": "Keep-Alive",
    "host": "echo.onlineornot.com",
    "user-agent": "testing-echo"
  },
  "body": "",
  "powered-by": "https://onlineornot.com"
}

How to follow redirects with curl

By default, curl won't automatically follow redirects (when a website sends you to a different URL). You can enable this with the -L or --location flag:

curl -L https://echo.onlineornot.com/redirect

This is particularly useful when working with URLs that might redirect you to a login page or a different domain.

How to see the full HTTP request and response

When debugging APIs, it's often useful to see exactly what curl is sending and receiving. The -v (verbose) flag shows you the complete HTTP conversation:

curl -v https://echo.onlineornot.com

This will show you:

  • DNS lookup details
  • TLS/SSL handshake information
  • Request headers sent
  • Response headers received
  • The response body

How to handle cookies with curl

Building on our knowledge of headers, sometimes you need to handle cookies. You can:

  1. Send a cookie:
curl -H "Cookie: session=123abc" https://echo.onlineornot.com
  1. Save received cookies to a file:
curl -c cookies.txt https://echo.onlineornot.com
  1. Use saved cookies in a request:
curl -b cookies.txt https://echo.onlineornot.com

How to simulate different HTTP methods

We've covered GET and POST, but REST APIs often use other HTTP methods. Here's how to use them:

# PUT request
curl -X PUT -H "Content-Type: application/json" \
     -d '{"name": "Updated website"}' \
     https://echo.onlineornot.com

# DELETE request
curl -X DELETE https://echo.onlineornot.com/resource/123

# PATCH request
curl -X PATCH -H "Content-Type: application/json" \
     -d '{"name": "Partial update"}' \
     https://echo.onlineornot.com

How to handle authentication methods with curl

Building on our earlier Authorization header example, here's how to handle different auth methods:

  1. Basic Auth (username/password):
curl -u username:password https://echo.onlineornot.com
  1. Bearer token (as shown above):
curl -H "Authorization: Bearer a_real_token_goes_here" https://echo.onlineornot.com
  1. API Key in query string:
curl "https://echo.onlineornot.com?api_key=a_real_token_goes_here"

How to handle file uploads

For APIs that accept file uploads, you can use the -F flag for form data:

# Upload a single file
curl -F "file=@local-file.pdf" https://echo.onlineornot.com/upload

# Upload multiple files with additional form fields
curl -F "files[]=@file1.pdf" \
     -F "files[]=@file2.pdf" \
     -F "description=My files" \
     https://echo.onlineornot.com/upload

How to time your requests

When working with APIs, you often want to know how long requests take. The -w flag lets you format timing data:

curl -w "\nTime taken: %{time_total}s\n" https://echo.onlineornot.com

You can get detailed timing metrics with this format string:

curl -w "\
    Time DNS lookup: %{time_namelookup}s\n\
    Time to connect: %{time_connect}s\n\
    Time to first byte: %{time_starttransfer}s\n\
    Total time: %{time_total}s\n" \
    https://echo.onlineornot.com

Putting it all together

Here's an example that combines multiple concepts we've learned:

curl -v -L \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer a_real_token_goes_here" \
     -d '{"name": "Complex request"}' \
     -w "\nTotal time: %{time_total}s\n" \
     https://echo.onlineornot.com/api/v1/resource

This command:

  • Shows verbose output (-v)
  • Follows redirects (-L)
  • Sets content type and auth headers
  • Sends JSON data
  • Shows timing information
  • Makes a POST request (implied by -d)

Common curl command patterns

To wrap up, here are some common patterns you might use in your day-to-day work:

  1. API testing:
curl -v -H "Authorization: Bearer a_real_token_goes_here" https://echo.onlineornot.com
  1. Download a file:
curl -L -o output.zip https://echo.onlineornot.com/file.zip
  1. API creation request:
curl -X POST \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer token" \
     -d @create-payload.json \
     https://echo.onlineornot.com/resource

Remember, you can always combine these patterns and flags to build the exact curl command you need for your use case.

]]> <![CDATA[2024, and what's next]]> https://onlineornot.com/2024 https://onlineornot.com/2024 Mon, 06 Jan 2025 00:00:00 GMT The other day I realized that while I write monthly updates on what's new in OnlineOrNot, and keep the changelog updated, I didn't have a yearly review of what was shipped, and what folks could expect from the coming year - so here we are!

Table of Contents

What went live in 2024

Taking a quick look through OnlineOrNot's code history for 2024 shows that I've made 1002 individual releases to OnlineOrNot's production environment over 235 working days, with quite a few bugfixes and UI tweaks too small to mention here.

The vast majority of these were implemented in the two hours I allow myself before full-time work, and sometimes I work on Saturday mornings.

Here are the highlights:

General

  • adopted OpenAPI for automated API documentation, and more
  • integrated with incident.io for alerting
  • rewrote OnlineOrNot for Cloudflare Workers, and backported the changes to AWS to allow the same code to be run redundantly across the two cloud providers

Checks

Status Pages

What's next

As I mention on the About page, OnlineOrNot is 100% backed by its customers, and I expect to still be running it in 10 years. It goes without saying that I'm going to continue to focus on reliability, and ensuring OnlineOrNot is a tool that software teams can trust.

More specifically, I'm going to wrap up some status page features I didn't get to in 2024 (subscribe webhooks to a status page, display data from third-party sources, show a component's uptime history, incident postmortems, custom templates for creating incidents).

Uptime Checking and Cron Job Monitoring will see some love too, with better data display options in the dashboard and UI improvements.

]]> <![CDATA[How OnlineOrNot more than halved its AWS bill]]> https://onlineornot.com/how-onlineornot-halved-aws-bill https://onlineornot.com/how-onlineornot-halved-aws-bill Mon, 28 Oct 2024 08:45:00 GMT Chances are, you've heard one of the main promises of serverless compute: "you only pay for what you use". While convenient when starting a new project, once you start to get continuous usage on that compute, you start to realize you're paying a hefty premium for that convenience.

If you're using AWS Lambda like I am, chances are that "for what you use" part isn't completely true either. If your function spends the majority of its time waiting for I/O to complete (fetching external URLs, querying DBs), you're paying a serverless premium mostly for your compute to sit around, waiting for something to do.

My solution to this was to migrate to Cloudflare Workers. With Workers, I went from paying up to 30000ms for a function to wait around while fetching a URL, to paying for the 1.4ms (p99) of CPU time to process the data before and after fetching.

Table of contents:

The project

OnlineOrNot is a bootstrapped business that provides status pages with uptime monitoring attached, for software teams. When I started out, it was the 200th alternative uptime monitor for the Internet. To save money on the off-chance no one other than me used it, I built it on AWS Lambda. It did indeed cost nothing while no one was using it.

Building an uptime checker is a simple matter of code, you need:

  • a process to query a database, find checks that are ready to be run, and queue them up in the right location
  • a process that picks up checks, runs them, and sends the results somewhere

On AWS Lambda, this worked out to be a single function for picking up checks, an SNS topic, another function for running checks, a queue for batching writes to the DB, and a final function for writing results to the DB.

Fast-forward to a few weeks ago, and the vast majority of OnlineOrNot's cloud spend came from paying for AWS Lambda, for functions that spent most of their time waiting for things to do.

"But why don't you just use VMs?"

I've tried moving off of AWS Lambda before, too.

More than two years ago, after running OnlineOrNot for a bit and seeing costs steadily grow, I tried moving the service to continuously running VMs. The particular vendors I picked at the time struggled to maintain more than a few nines of reliability, which is awkward when you're trying to be accurate at measuring the reliability of others.

Long story short, I moved back to AWS, happy to continue paying the premium, knowing it would at least be reliable.

Why Workers worked out to be significantly cheaper

If you're curious, there's a good explanation of how Workers works, but the gist of it is: classic serverless platforms run a chunky VM under the hood, which can't be easily paused and restarted while waiting for I/O to complete, while Workers is built on the V8 runtime where the isolates that run your code are so lightweight that they can be easily restarted.

The most expensive part of my AWS Lambda functions was waiting for DB queries to finish and fetching external URLs - Cloudflare doesn't bill for that.

So while Workers does provide significantly cheaper compute, what stopped me (until recently) from using them as the main compute for my own applications was the inability to talk directly to my existing database.

and then Workers started talking to Postgres

Cloudflare announced Hyperdrive back in late 2023, it became generally available in April 2024, and it was exactly what I was looking for to unlock building entire applications on Workers.

Hyperdrive lets you query Postgres/MySQL databases from Cloudflare Workers (it's a connection pooler with an optional query cache service built-in).

The approach to moving off AWS Lambda

I was initially worried about how much time I'd spend rewriting, but it turned out that all of my AWS Lambda code was compatible with Cloudflare Workers. In the move I decided to change my Postgres client library from pg-promise to Postgres.js (Hyperdrive's recommended library), and remove the queue for batching writes, since I now had a connection pooler.

I started by creating a new Durable Object with a single purpose: run an alarm every 15 seconds, and start the checker.

Next, I moved the check-fetcher function across. On AWS, this would talk to SNS topics to send checks to the right region. I needed a Cloudflare-native means of placing Workers in different regions around the world. To do this, I reached for Durable Objects, with their location hints.

So I created another new Durable Object cleverly named "placer" with a single purpose: run a Worker in the region you wake up in.

The Worker itself I copied over from AWS Lambda, and since it used native JavaScript, it didn't need rewriting.

Finally I moved over the function to upload results to the database, it too was fully compatible with Workers, but I chose to change from pg-promise's db.one/db.many syntax to Postgres.js's sql syntax because I liked it better.

Rolling out the change

To start with, every check in my database has a "version" field, that I use to control which runtime it should run on. This made for a nice feature-flag for opting customers into the new system.

I wasn't too worried, as I had been using Workers to double-check failing uptime checks from AWS for years now, but there are always edge-cases.

I started by running my personal checks on Cloudflare for a few days, before finding some early adopters willing to try it out, then moving over the free tier, followed by the paid tier of users.

It's faster, too?

While it's well documented that when you allocate less RAM to an AWS Lambda function, you also get less CPU, I didn't realize that also meant they were throttling network performance.

In moving OnlineOrNot's checks to run primarily from Cloudflare, I noticed response time dropped by up to 75%:

OnlineOrNot gets faster checks

Attempting to investigate further lead me to this answer on the AWS forums:

Yes, the network performance will increase with memory. There are no specifics as there is no definitive performance since Lambda compute is abstracted from you and it is also protocol and application specific. You should benchmark your function with various memory sized to determine where you meet your requirements.

Summary

To wrap it up, OnlineOrNot is now "multi-cloud": there are multiple replicas of the system ready to run (for free while no one uses it) on AWS, and a version of the system running natively on Cloudflare, where checks complete faster, with running costs of less than half of what I was paying for AWS Lambda.

As we are bootstrapped, this accelerates the timeline for hiring additional help for the business, and will improve our support and product overall in the long term.

]]> <![CDATA[OnlineOrNot updates from September 2024]]> https://onlineornot.com/onlineornot-updates-from-2024-september https://onlineornot.com/onlineornot-updates-from-2024-september Mon, 07 Oct 2024 00:00:00 GMT A quieter month this time around: you can now customize your status pages, there are additional overall statuses for status pages, and I fixed a few bugs.

Table of contents

What's new

Features for status pages

Custom logos and favicons for Status Pages

For a while now, OnlineOrNot's status pages all looked the same: a brand name, followed by some uptime bars:

Scheduled maintenance - OnlineOrNot Status Page

This worked fine if you wanted a bare-bones status page, but what if you wanted it to display your logo and favicon? Now you can!

In the Settings tab of your status page, you can now customize your favicon, and your light-mode and dark-mode logos:

Scheduled maintenance - OnlineOrNot Status Page

This way, your status page can match your website's design, and you'll keep your designers happy:

Scheduled maintenance - OnlineOrNot Status Page

Changing how overall status works

Before this update, status pages had two overall statuses: "All Systems Operational" and "There is an ongoing incident". They were linked to incidents, and the only way to update your overall status was by creating or resolving an incident.

With this update, overall status on a status page is now controlled by components. Changing an individual component's status to "Major outage", "Partial outage", "Degraded performance" or "Maintenance" will also change your page's status:

Additional statuses - OnlineOrNot Status Page

Bug fixes and improvements

  • Fixed a bug in the uptime percentage calculation on status pages. It would previously ignore incidents on the first day of a component's existence
  • Status pages now load approximately 500ms faster thanks to a performance optimization

What's next

Here's what is still planned for the coming months:

  • Ability to subscribe to status pages via webhook
  • Ability to push third-party metrics and plot graphs to showcase for customers
  • Display historical uptime stats for the past year in a calendar
  • Ability to create or update incident updates through APIs

Thank you

OnlineOrNot only exists because of folks like you. I don't have a massive marketing budget, or time to do it all myself.

OnlineOrNot sustains itself from folks like you enjoying it enough to tell your friends and colleagues about it.

As always: here's an ask:

I work on things that people care enough about to tell me about them. If you think there's something missing, or something that needs improving, I'd like to know more!

You can reply to this email/email me at max@onlineornot.com, if you'd like to chat.

]]> <![CDATA[OnlineOrNot updates from August 2024]]> https://onlineornot.com/onlineornot-updates-from-2024-august https://onlineornot.com/onlineornot-updates-from-2024-august Wed, 04 Sep 2024 00:00:00 GMT This month I continued adding features to OnlineOrNot's status page functionality, while also taking the time to fix bugs, and add an integration for Pingdom.

Table of contents

What's new

Features for status pages

Scheduled Maintenance

Sometimes you realise ahead of time that you need to take your service offline for a bit while you run updates. You already have enough on your plate, why worry about keeping the status page manually updated too?

You can now keep your customers updated on how your scheduled maintenance is progressing, and when they can use your service again, automatically:

Scheduled maintenance - OnlineOrNot Status Page

Custom favicon and logo

We now support custom favicons and logos in status pages:

Scheduled maintenance - OnlineOrNot Status Page

The process requires emailing OnlineOrNot's support, but I figured getting the feature into people's hands was more important than making it fully self-serve.

You can read more about it in the documentation.

Pingdom integration

You might already have uptime checks that you've been running for years in Pingdom. It does everything you need, and it's a total pain to move uptime monitoring when you've had it running for years, I get it.

What if you still wanted an OnlineOrNot status page for your Pingdom uptime checks?

You're in luck, because OnlineOrNot now integrates with Pingdom to update your status page:

Pingdom integration - OnlineOrNot Status Page

Bug fixes and improvements

  • External status page integrations now update every minute
  • Fixed bugs in the GitLab and Paddle external status page integration
  • Fixed a bug that impacted email deliverability when subscribing to a status page

What's next

Here's what I have planned for these coming months:

  • Upload your own favicon and logo to status pages at any time
  • Ability to push third-party metrics and plot graphs to showcase for customers
  • Display historical uptime stats for the past year in a calendar
  • Ability to subscribe to status pages via webhook
  • Ability to create or update incident updates through APIs

Thank you

OnlineOrNot only exists because of folks like you. I don't have a massive marketing budget, or time to do it all myself.

OnlineOrNot sustains itself from folks like you enjoying it enough to tell your friends and colleagues about it.

As always: here's an ask:

I work on things that people care enough about to tell me about them. If you think there's something missing, or something that needs improving, I'd like to know more!

You can reply to this email/email me at max@onlineornot.com, if you'd like to chat.

]]> <![CDATA[OnlineOrNot updates from June/July 2024]]> https://onlineornot.com/onlineornot-updates-from-2024-june-july https://onlineornot.com/onlineornot-updates-from-2024-june-july Tue, 06 Aug 2024 00:00:00 GMT If you remember the last update, I originally planned to ship quite a few features in June. Life and family had to take priority for me, so they didn't get done.

Thankfully, I'm building OnlineOrNot for the extremely long term, so there's no rush to ship as fast as possible.

A fair few things did get shipped though.

Table of contents

What's new

Features for status pages

Retrospective incidents

Ever internally declare an incident, have your team fix the issue, and in the heat of the moment completely forget to update your status page?

It happens more often than you'd think, which is why OnlineOrNot now supports adding incidents that happened in the past:

Retrospective incidents - OnlineOrNot Status Page

Descriptions and hiding from search engines

You can now add descriptions (in markdown) to status pages, and hide them from being indexed by search engines.

Incident history - OnlineOrNot Status Page

To update your status page's settings, click into your status page's dashboard and visit the "Settings" tab.

In the "Settings" tab, you'll find a new Description field, and a checkbox for disabling search engines from indexing your status page:

Incident history - OnlineOrNot Status Page

Incident history

Each status page now displays more than just the last fourteen days of incidents: you can now access your status page's entire incident history.

Incident history - OnlineOrNot Status Page

To access a status page's incident history, scroll down to "Recent Incidents" and click the "view incident history" link to the right.

IP allowlisting

You might want to restrict access to your status page, but not want to share a password around with your team.

For these cases, OnlineOrNot now supports IP allowlisting:

IP allowlisting - OnlineOrNot Status Page

Additional external status pages

OnlineOrNot can now display the uptime status of the following providers on your status page:

  • Ably
  • Radar

Bringing it to a total of thirty-two external services supported, with more to come!

What's next

Here's what I have planned for this coming month:

  • Ability to subscribe to status pages via webhook
  • Ability to create or update incident updates through APIs
  • Ability to push metrics and plot graphs to showcase for customers
  • Display historical uptime stats for the past year in a calendar

Thank you

OnlineOrNot only exists because of folks like you. I don't have a massive marketing budget, OnlineOrNot gets its users from folks like you enjoying it enough to tell your friends and colleagues about it.

So here's an ask:

I work on the things that people care enough about to tell me about them. If you think there's something missing, or something that needs improving, I'd like to know more!

You can reply to this email/email me at max@onlineornot.com, if you'd like to chat.

]]> <![CDATA[OnlineOrNot updates from May 2024]]> https://onlineornot.com/onlineornot-updates-from-2024-may https://onlineornot.com/onlineornot-updates-from-2024-may Tue, 04 Jun 2024 00:00:00 GMT In May I mostly took the time to plan out the rest of the year, as well as one big feature to make OnlineOrNot's incident management more realistic.

Table of contents

What's new

Fine-grained management for your incidents

There's no sugar-coating it: incidents can get complicated. To help, OnlineOrNot now supports updating components as your incident occurs, for components that recover before others.

Here's a concrete example of how this works. Say your AI provider goes down, impacting your API:

Incident Start - OnlineOrNot Status Page

Half an hour passes, your AI provider comes back online, but your API has degraded performance as your users rush to use your service again:

Incident Update - OnlineOrNot Status Page

Finally, your API scales up to meet demand, and your service is operating as normal, so you resolve the incident:

Incident End - OnlineOrNot Status Page

In short, OnlineOrNot now supports letting your users know on a per-update level, how your incident is progressing, and what the status of each component is.

Additional external status pages

OnlineOrNot can now display the uptime status of the following providers on your status page:

  • Paddle
  • GitLab
  • Bunny.net

Bringing it to a total of thirty external services supported, with more to come!

What's next

This past month I took the time to evaluate the backlog, and see what customers and prospects have been asking for.

Here's the minimum of what I have planned for this coming month:

  • Ability to restrict access to status pages via IP allowlisting
  • Ability to subscribe to status pages via webhook
  • Ability to create or update incident updates through APIs
  • Ability to push metrics and plot graphs to showcase for customers
  • Have historical details of the events + uptime percentage for the past year

Thank you

OnlineOrNot only exists because of folks like you. I don't have a massive marketing budget, OnlineOrNot gets its users from folks like you enjoying it enough to tell your friends and colleagues about it.

So here's an ask:

I work on the things that people care enough about to tell me about them. If you think there's something missing, or something that needs improving, I'd like to know more!

You can reply to this email/email me at max@onlineornot.com, if you'd like to chat.

]]> <![CDATA[OnlineOrNot updates from April 2024]]> https://onlineornot.com/onlineornot-updates-from-2024-april https://onlineornot.com/onlineornot-updates-from-2024-april Mon, 06 May 2024 00:00:00 GMT In April I kept focusing on making Status Pages better than ever, and added a new on-call integration: incident.io.

Table of contents

What's new

The free checker returns

OnlineOrNot originally started back in 2021 as a tiny form at https://onlineornot.com, letting you enter a URL, and the system would check the URL from around the world.

As I added more features, I needed space on the home page to describe all the functionality OnlineOrNot supported, and eventually I removed the free checker.

It's back now, and better than ever at Is this website down?

Free Website Checker

OnlineOrNot x Incident.io integration

Incident.io lets you manage incidents entirely within your Slack, handle on-call, manage runbooks, and learn from your incidents with post-incident reviews.

If you've dreamed of being able to raise an incident because your daily database backup didn't run (frankly, I get it), you'll want to check this out: Cron Job alerts in Incident.io

With the integration, OnlineOrNot supports sending alerts from all uptime checks, as well as cron job/scheduled task checks to incident.io. Once in incident.io, the member of your team currently on-call can be notified, an incident raised, and a post-mortem written, all from the comfort of your own Slack.

Status Page Component Groups

Your status page can get a little busy once you've added your own system components next to components from your vendors and other third parties.

Grouping your components lets you separate your system into logical parts, to make it easier to understand your status at a glance.

As a common example, let's say you've integrated deeply with OpenAI (and a few other vendors), and want to let your customers know when issues with their services may impact your own:

Ungrouped OnlineOrNot Status Page

Even with a single third-party vendor, it starts to become difficult to see where your service ends, and theirs begins.

Grouping simplifies things:

Grouped OnlineOrNot Status Page

Cron expression support

OnlineOrNot now supports monitoring even the most complicated cron schedules with heartbeat checks. Particular thanks to Martin for raising this feature request by replying to a monthly update email!

While getting an alert when your cron job fails to run "every day" is great, sometimes you have jobs you only want to run at midnight every weekday (0 0 * * 1-5): Cron Job alerting for OnlineOrNot Checks

What's next

This section is always difficult to write, if I'm totally honest: software estimation is always a challenge.

Regardless, here's the minimum that I aim to release this month:

  • Scheduled maintenance for status pages
  • Retrospective incidents for status pages
  • Additional external status pages

Thank you

OnlineOrNot only exists because of folks like you. I don't have a massive marketing budget, OnlineOrNot gets its users from folks like you enjoying it enough to tell your friends and colleagues about it.

So here's an ask:

I work on the things that people care enough about to tell me about them. If you think there's something missing, or something that needs improving, I'd like to know more!

You can reply to this email/email me at max@onlineornot.com, if you'd like to chat.

]]> <![CDATA[OnlineOrNot updates from March 2024]]> https://onlineornot.com/onlineornot-updates-from-2024-march https://onlineornot.com/onlineornot-updates-from-2024-march Tue, 02 Apr 2024 00:00:00 GMT March was a fun month for OnlineOrNot. I spoke to more customers than ever before, developed a long roadmap for OnlineOrNot's status page functionality (it's going to be an even better stand-alone product), heartbeat checks are going to support cron expressions, and more.

Table of contents

What's new

Up-to-date changelog

For about a year, the only way to know what was new in OnlineOrNot was to follow me on Twitter. I've realized not everyone is on Twitter these days, so I'm now also publishing the near-daily updates on Discord, and LinkedIn.

More substantial feature updates will be added to the changelog (and there's now an in-app notification when that happens). Once a month updates will continue coming on this blog and its newsletter.

External Status Pages

Web applications are more complicated than ever. Your service depends on other services, and a simple status page displaying "Website" and "API" as components isn't enough any more.

OnlineOrNot now lets you display the live status of 25 other services, including Cloudflare, OpenAI, DigitalOcean, Vercel, and more.

To get started, jump into your status page's list of components, click the "add a component" dropdown, and click "add a third-party component":

OnlineOrNot - External Status Pages

This will be a growing list, and I expect to have hundreds of services supported by the end of the year.

Manage status pages through an API

Manually setting up a single status page can be fine, but what if you have dozens, or even hundreds of pages to manage?

To make this easier, OnlineOrNot now provides API endpoints for managing status pages, status page components, and status page incidents.

I've also begun work on an official terraform provider for OnlineOrNot, soon you'll be able to manage status pages (and more) through an infrastructure-as-code setup.

Reminder alerts

You might get a notification from OnlineOrNot during a meeting, and not realize your website never came back online, or cron job never reported successful completion.

To remedy this, OnlineOrNot now sends reminder notifications until your website comes back online. You can tweak the frequency under a Check's alert settings:

OnlineOrNot - Reminder alerts

Thank you

OnlineOrNot only exists because of folks like you. I don't have a massive marketing budget, OnlineOrNot gets its users from folks like you enjoying it enough to tell your friends and colleagues about it.

So here's an ask:

I work on the things that people care enough about to tell me about them. If you think there's something missing, or something that needs improving, I'd like to know more!

You can reply to this email/email me at max@onlineornot.com, if you'd like to chat.

]]> <![CDATA[Improving your on-call schedule with runbooks]]> https://onlineornot.com/improving-oncall-schedule-with-runbooks https://onlineornot.com/improving-oncall-schedule-with-runbooks Tue, 12 Mar 2024 07:00:00 GMT Incidents are a stressful time for your team: your service isn't working the way you expect and your customers/stakeholders want to know what's going on. The last thing you want to do is let your team improvise everything when it comes to responding to incidents.

Google's own SRE book has great overall tips for incident management, part of which involves "develop(ing) and document(ing) your incident management procedures in advance", which this article dives into.

Table of Contents

What is a runbook

Depending on where you work, you'll hear different words describing relatively similar concepts around runbooks:

  • Standard Operating Procedure (or SOP): a standard set of steps required by the business to perform a process or task
  • Runbook: a set of steps required to respond to an incident or alert
  • Playbook: what we call the overall set of steps when responding to an incident. Can involve several runbooks

You shouldn't worry too much about terminology, just know we're going to be writing instructions on how to perform the manual process of resolving an incident with a system your team runs.

Your first runbook

When to write your first runbook

Ideally, you would write a runbook as you're deploying a service for the first time. You would go through the manual process you take to resolve any issues that come up, and write down each step, in detail, so anyone on your team can follow it while half-asleep.

Realistically though, chances are you'll be writing your first runbook after something goes wrong. For a lot of organisations, it takes an incident to really examine the way you work and learn where the potential improvements are.

Most commonly, something will go wrong, it'll take your on-call engineer a few minutes to figure out what happened, page the team members that know what to do, and it'll take the team a few hours to resolve the issue.

During the post-incident review, someone will ask "why did it take so long to fix?", and one of the inevitable answers will be "we didn't have a runbook in place". One of your team members will go off on a search for how to create a runbook, and they'll probably end up reading this article.

How to write a runbook

Honestly, anything is better than nothing. As time goes on, and your team gains operational experience, your runbooks will improve as gaps are found.

To start, you want to be documenting the manual steps necessary to get your service working again. You want to tell folks exactly what to do:

  1. Login to the Splunk dashboard at URL
    • Run this query: "QUERY GOES HERE"
    • In case of weird results, run this command: "COMMAND HERE"
      • Weird results look like this: "SCREENSHOT GOES HERE"

In other words, favour explicit steps over implicit steps.

If any of your runbook steps say "Do the usual troubleshooting procedure", you need to rewrite that step, following the advice above.

If any step of your runbook contains "If you need to do X, follow the procedure here...", you need to explain the decision making process that leads the reader towards figuring out if they need to do X.

Do not allow perfectionism to block you from releasing your runbooks

Depending on your organisation, you may have stakeholders that'll want every aspect of the system documented in the runbook before it's "ready".

Without feedback on using the runbook (such as during real or simulated incidents), you'll quickly start getting diminishing returns on the time you spend trying to perfect your runbooks.

You're better off spending that time resolving issues that cause common alerts to fire for your on-call engineers.

Use new team members as a starting point

New team members are a gift for getting started, and improving your runbooks. They aren't yet affected by the curse of knowledge, and tend to be curious about how things work.

Have them sit in on incidents, join incident response chat rooms, and have them write down any questions they have as the incident is worked on. The questions they come up with are perfect starting points for a runbook.

Do not worry about automation (yet)

Automating your runbooks comes later (chances are, it won't cover every single step of your runbook anyway).

To start with though, you're going to want to document the entire manual procedure before you begin to automate the easiest steps.

In short: write today, automate tomorrow.

If you're early in your incident response journey, your team isn't ready for automation. Your team likely has too much tribal knowledge, and needs to work on documenting their manual processes first.

Improving your existing runbooks

Now imagine it's 2am.

You're getting paged for the latest service your team built and deployed. You have no idea how to debug it, and it's the first time you've been paged for the service.

"No worries, I'll just go through the list of steps in the runbook, I'm sure it'll be fine... wait, runbook just says 'Ask Dave'?!"

You start paging the developers who built the service (especially Dave), and over a few anxious hours, you and the team manage to resolve the issue, and go back to bed.

In the rest of this article, we're going to improve our runbooks today, so no one in your team needs to experience the above scenario.

Optimize for editing

The first runbook your team writes is probably going to suck, but that's okay!

Getting started is the first step towards being good at something.

You're going to want to put your runbooks somewhere easy to edit. While a git repo is fine, something like Notion, Confluence or Google Docs is going to be easier to update.

Note that a self-hosted wiki/repo probably isn't the best idea - particularly if your team needs VPN/office network access to read the runbooks, and your VPN goes down.

Encourage newcomers to use your runbooks and make edits

As mentioned earlier, authors of your runbooks are affected by the curse of knowledge, and you will assume people will have the background knowledge to understand what to do.

Newcomers to your team aren't yet affected by this, and can help you uncover assumed knowledge in your runbooks. Have them pair with your on-call team members as they resolve incidents - they'll get more comfortable with on-call before their first roster, and will notice undocumented gaps that your team just knows.

This will also help the team feel comfortable with editing your runbooks - they're not perfect, and will need changing over time.

Keep track of how often you need to use your runbooks

As I mentioned earlier, while the initial goal is to document the manual process required to resolve an incident, a later goal is to automate those steps.

The more often you use a runbook, the more evidence you gather that perhaps parts of the runbook should be automated.

A simple table at the bottom of the runbook would help with this:

| Date last used | Used by | | -------------- | ------- | | 2022/01/01 | rozenmd | | ... | ... |

Then, at a regular frequency (say, once a month or so), review your runbooks for automation opportunities.

Explicitly tell the reader what to do

People shouldn't need to interpret what the steps in your runbook could mean. Write as though the person reading it just woke up at 2am, and just wants to go back to bed.

The steps should explicitly tell them what to do. If you're starting with existing runbooks, it's worth auditing them to ensure the content is explicit, rather than implicit.

Make your runbook easier to search

It helps to add key phrases from your alerts, or even the exact error message thrown by your system.

For example:

# How to fix: "Error: Query defined in resolvers, but not in schema"

1. If you see this error in environment X, you need to run this command:
   ...

It'll make it easier to find a solution to the exact problem the system is having (by including the searched phrase in your heading, you'll also game certain documentation search engines to rank the terms higher).

]]> <![CDATA[How to check if a website is online]]> https://onlineornot.com/how-check-website-online https://onlineornot.com/how-check-website-online Thu, 07 Mar 2024 00:00:00 GMT Ever wonder how to check if a website is still working, without having to load up the website and manually check every few minutes? In this article, we'll go over the various ways to check if a website is online.

If you just want a tool to check if a website is down, check out OnlineOrNot's website down checker.

I also have a video guide on how to check if a website is online.

Table of contents:

Why do websites go offline?

In its simplest form, a website is a collection of files that are hosted on a server. When you visit a website, your browser makes a request to the server, and the server responds with the files for the website. More recently, websites have started to use APIs to fetch data to generate webpages from other servers, but the core concept is the same.

When we say a website is online, we mean that a server is responding (with the expected files) to requests from the Internet.

Things aren't that simple when a website is offline, though. There are a few different ways a website can be offline:

1. The server is down

If the server is down (that is, not running), it can't respond to requests from the Internet, so the website will be unavailable.

2. The server takes too long to respond

If the server takes too long to respond, the browser will give up and show an error. This often happens when a server is overloaded with requests, or if the server makes a request to another server, which itself is overloaded.

Smaller companies tend to run into this issue by hosting their websites on small servers without a CDN in front, as the servers often don't have enough RAM/CPU to handle the number of requests they receive.

In my experience, this is the most common way a website can be offline.

3. The domain name is not pointing to the server

If the domain name is not pointing to the server, the server won't receive any requests from the Internet.

You might run into this issue when launching your website, as in the worst case, it can take a few hours for the domain name to start pointing to the server. This can also happen when changing domain name registrars.

4. The domain name has expired

If the domain name has expired, the domain name registrar will stop pointing the domain name to the server. This is an avoidable problem however, and I wrote about how to avoid losing your domain name a while back.

5. The SSL certificate has expired

This tends to happen a lot less these days with the advent of automatically renewing SSL certificates from LetsEncrypt and free SSL certificates from the likes of Cloudflare, but SSL certificates do have an expiry date.

If your server isn't configured to automatically renew its SSL certificate, modern browsers often won't even try to display your website, and users are trained to avoid sites that don't use HTTPS.

How to check if a website is online

There are a few ways to check if a website is online, some requiring more manual work than others:

1. Visit the website

The most obvious way to check if a website is online is to visit it using your browser. If you can see the website, it's online. If you can't, it's probably offline (assuming your internet connection is fine).

This method isn't bulletproof, as a website appearing online for you doesn't guarantee that it's online for everyone else.

2. Use a command line tool

If you're technically-minded, you can use a command line tool like curl in your Terminal to check if a website is online.

curl https://www.google.com

If you see a bunch of HTML, the website is online. If you see an error, the website is offline.

If you want to take it further, you can turn the above command into a script that runs every few minutes, and sends you an email if the website is offline:

#!/bin/bash

if curl -s https://www.google.com | grep -q "html"; then
  echo "Website is online"
else
  echo "Website is offline" | <command to send email goes here>
fi

3. Use an online tool

OnlineOrNot's website down checker is a free tool to check if a website is online.

All you need to do is enter a URL, such as https://www.google.com, and click "Check".

OnlineOrNot will then check the website from the US East Coast, US West Coast, Australia, Europe, and Asia Pacific regions to see if the website is online, and how long it takes to respond:

OnlineOrNot website down checker

4. Monitor your website

If you're running a business, you probably don't have time to check if your website is online every few minutes. You also don't want to have customers email/call/text you frantically that your website is down, and they're unable to use your product.

This is where website uptime monitoring services like OnlineOrNot come in. It checks if your website is online up to every 30 seconds, and if it's not, you'll immediately get a notification via email, SMS, Slack, or Discord.

How to monitor your website

To monitor a website with OnlineOrNot, all you need is the URL.

I tend to give my checks memorable names, so I can easily tell what they're for, and I set the check to run from around the world, so I can be sure the website is online for everyone:

OnlineOrNot landing page monitoring settings

After adding the check, you will start to see the results in the dashboard:

OnlineOrNot dashboard

How to monitor keywords on a website

Monitoring for keywords on OnlineOrNot requires one extra step.

In a check's advanced settings I set the 'Text to search for' to look for the keyword I'm interested in (it also supports HTML, if you want to check for a specific HTML element):

OnlineOrNot landing page monitoring settings

Once setup, OnlineOrNot will send you an alert whenever that keyword is not found on the website.

]]> <![CDATA[OnlineOrNot updates from January 2024]]> https://onlineornot.com/onlineornot-updates-from-2024-january https://onlineornot.com/onlineornot-updates-from-2024-january Sat, 03 Feb 2024 00:00:00 GMT Over December and January I focused on OnlineOrNot's public API, making it possible to see what OnlineOrNot saw when it detected a failing uptime check, as well as fixing bugs and cleaning up tech debt across all of OnlineOrNot (it's getting faster and faster to release updates).

Table of contents

What's new

Adopting OpenAPI and publishing new API docs

OnlineOrNot's public API was originally written to serve a single purpose: provide data for the CLI. The documentation for the API was handwritten, and lovingly placed alongside OnlineOrNot's other hand-written docs.

This made it time-consuming to update the API: first I would write code, then remember to context-switch into "content-mode" and decide where the new API endpoint docs should live, then actually write docs from scratch, and finally deploy the content.

In December, I automated this process (I wrote more about the technical details here), and now we now have a published OpenAPI schema and a new API documentation hub:

New API docs

The API itself (like OnlineOrNot) is still under active development, and new endpoints will be automatically published to both the documentation site, and the OpenAPI schema in the coming weeks.

OnlineOrNot, what did you see?

Up until now, OnlineOrNot would only ever tell you if your website or API was up or down, and roughly how long your website or API took to serve a request. Actually figuring out what was wrong was "left as an exercise for the reader", like an academic textbook.

As a user of OnlineOrNot, you would have one of two reactions to this:

  1. I better investigate my logs around that time, see if the database backup job isn't accidentally killing our website at the same time.

  2. Nah, that's a false alarm, no way the website goes down at exactly 2am every single day.

To remedy this, in January I focused on making every request result (HTTP status code, region it checked from, response time, date and time checked) available in the dashboard, as well as the request/response metadata (request headers, response headers and response body).

Every uptime check in OnlineOrNot now displays a list of recent checks:

Recent uptime checks

with an option to view all checks, and filter by failing checks:

All uptime results

and finally, the raw metadata involved in the check:

uptime result metadata

By the time you see these screens in OnlineOrNot, they will likely be even more featured - it was just so useful that I wanted to release and share an early preview as soon as possible.

In short, performing a root-cause analysis of why an uptime check failed with OnlineOrNot is now significantly easier.

Thank you

OnlineOrNot only exists because of folks like you.

I don't have a massive marketing budget, OnlineOrNot gets its users from folks like you enjoying it enough to tell your friends and colleagues about it.

So here's an ask: I work on the things that people care enough about to tell me about them. If you think there's something missing, or something that needs improving, reply to this email/email me at max@onlineornot.com - I'd like to know more!

]]> <![CDATA[I built my HTTP API docs from scratch]]> https://onlineornot.com/built-my-http-docs-from-scratch https://onlineornot.com/built-my-http-docs-from-scratch Sun, 14 Jan 2024 10:30:00 GMT You might be thinking “building HTTP API docs from scratch? in 2024? wtf?”, and you’re probably right. After all redoc has been around since 2016, and there are hundreds of “generate beautiful documentation from your OpenAPI spec” startups around, some even use AI now.

To be honest, I didn’t even know it was possible to do-it-yourself when I started looking into it. Every search turned up with instructions on how to convert my OpenAPI schema into beautiful documentation via …some third party.

The thing is, I don’t want AI to answer questions about my product. If I can’t guarantee that it will be 100% accurate, it is useless to me. Similarly, I don’t want to put a third party in charge of displaying my API’s documentation to my users.

Frankly, third parties have no skin in the game when generating beautiful documentation. They seem to be more worried about selling to you, the one with the API, than to your users. Who cares if the docs are unusable? They’re beautiful! Have you seen our dark mode? /s

As a developer that started out on the frontend before learning backend and infra/ops, I thought I could do better, so I gave it a shot.

It only took 8 hours in the end, and I'm pleased with the results.

There were two steps:

Converting OpenAPI schemas to markdown

I cheated a little bit, I didn’t handwrite an OpenAPI YAML schema from scratch (though I have in the past). My API of choice (HonoJS) provides a middleware for turning Zod validation into OpenAPI schemas, so I used that.

With an OpenAPI spec in hand, I needed a means of converting that data into markdown. Extensive searching came up with a weird word: widdershins.

Introducing widdershins

Widdershins was originally written to convert Swagger/OpenAPI specs to Slate/ReSlate compatible markdown using a templating language called doT.

I had no interest in using Slate/ReSlate, but since a templating language was involved, I knew I could grab the existing templates, fork them, and have them output MDX-compatible code instead.

Customizing widdershins output

Customizing the widdershins templates was a matter of cloning the widdershins repo, copying the files in templates/openapi3 into my own directory, and telling widdershins to use my templates instead of the default ones.

widdershins didn't do everything I wanted, but thankfully it was easy to customize the output.

Some hacks involved

  • By default, widdershins outputs a single markdown file, so I had to insert placeholder text in my templates, and later use that to split the markdown by resource
    • Using an OpenAPI resource tag in my template, I then write the desired filename (checks/index.mdx for example) as the first line of the split markdown, and use that as my filename
  • doT uses curly brackets for variables with no simple way of inserting curly brackets into output, so I had to use &lcub;&lcub; and &rcub;&rcub; and run a markdownOutput.replace() to fix up my MDX

Converting markdown to HTML

As a developer, you're spoiled for choice these days when it comes to turning markdown/MDX into HTML. I was already a paying customer of Tailwind UI, so I chose to use their Protocol template (which uses Next.js under the hood), and modify it to my needs (mainly changes to the color-scheme and icon sizing).

I used Cloudflare's next-on-pages project so I could host the website on Cloudflare Pages, and not worry about being billed for bandwidth.

** Summary **

In short, it took me 8 hours to go from OpenAPI YAML file to my own custom HTTP API documentation website, and I'm very pleased with the results.

My main aim with this project was being able to have a say over every single aspect of the docs, so that I can react to my own customer feedback. If folks write in asking for extra details in certain places, or that something doesn't make sense, I'm capable of modifying the docs from the OpenAPI spec to the pipeline generating markdown, to the markdown renderer. I don't have to write to a vendor asking them to fix something, and have them tell me it'll be scheduled some time next quarter, maybe.

]]> <![CDATA[How to get your first ten customers]]> https://onlineornot.com/how-to-get-your-first-ten-customers https://onlineornot.com/how-to-get-your-first-ten-customers Mon, 18 Dec 2023 06:52:00 GMT It'll soon be the third anniversary of publicly launching OnlineOrNot on Twitter, and I often get asked what I did to get my first paying customers - so I felt like sharing.

I assume when most folks ask this that they're looking for the one thing they can do to finally start getting paid customers.

Let me be clear: it's never just one thing.

Update: I wrote about my third year of running OnlineOrNot here.

Table of contents

What I actually did

Off the top of my head:

  • I tweeted every time I was thinking about a feature, building a feature, and released a feature
  • I tweeted random things I learned about running a business, as I learned them
  • I tapped my personal network on LinkedIn to find out what folks were currently using, and how they were dissatisfied
  • I charged way too low and attracted un-ideal customers
  • I made a free tier to compete with other providers that also charged way too low
  • I wrote changelogs for every big feature
  • I wrote docs for every feature
  • I kept a mailing list where I would update folks about the product's progress
  • I kept a mailing list where I would update folks about running the SaaS
  • I talked about OnlineOrNot on Reddit, Hacker News, in the aim of inspiring people to launch their idea in a highly competitive space
  • I shared OnlineOrNot on popular lists (Product Hunt, etc)
  • I made landing pages that resonate with folks looking specifically for my type of product
  • I built features customers asked for
  • I would spend one week marketing, followed by one week coding, and repeat
  • I was extremely responsive to customers via email and chat
  • I kept improving the product, adding features, and revisiting features

There's probably a lot more, but that's what comes to mind first.

Some context

OnlineOrNot (this website) started because I needed a weekly report for my contracting clients to prove their web host was unreliable, to the point where it was costing them significant money. They were paying for the cheapest possible tier of WordPress hosting at the time, and didn't believe me when I said random 5 minute blocks of downtime throughout the day were adding up.

I built a dirt-simple form that takes a URL and sends an email notification when the site goes down/up, with a weekly summary email.

Then, I kept adding features every day, 2 hours at a time, even after I stopped being a contractor.

I read, a lot

When starting off, I did a lot of reading to figure out what others did before me. While many folks tend to hold-off from starting a business until they feel like they've read everything, I started the business first and started reading as I needed help.

The following articles/books were particularly influential:

  • The Mom Test by Rob Fitzpatrick

  • Deploy Empathy by Michele Hansen

  • Traction by Gabriel Weinberg and Justin Mares

  • Your first 10 customers by Patio11

  • A Bootstrapped SaaS Journey to $10K MRR by Jon Yongfook

  • A random indiehackers forum comment that made me totally re-think marketing:

    • Don't believe for one moment that 5 hours on Product Hunt or anywhere else for that matter represents a serious marketing effort.

      If you want to run a business rather just create stuff, your work has only just begun. In the light of Facebook and other social media revelations, the idea of a truly disposable email address which means your entire life is not analysed and spammed to death has to be worth something.

      You haven't told anyone about it though. And I mean you shout from the rooftops every day and everywhere you can think of. You market. People are not going to come looking for you. You have to start approaching influencers, be seen and be heard everywhere you think your potential users might lurk.

      And, by the way, everyone sees a million new ideas a day so you have to be consistent, appear to be permanent and appear to be solid. No-one is going to entrust communications with you if they think you are a small, one-man band with an idea and little else.

      Time to start reading marketing articles and strategies and applying them.

      And expect it to take time.

On charging low and having a generous free tier

Charging low did help attract my first customers, but few of them are still subscribed years later, as I did not build OnlineOrNot with them in mind.

Of particular note is the type of customer that would overload their single server with hundreds of websites, and complain that OnlineOrNot did what they hired it to, whenever their server would inevitably crash under load.

Customers that only use your service because it is cheap are also the type of customer to cheap out in other parts of their business, making them unpleasant to deal with. At $9/mo, you can't afford to spend much time on support - and these are the customers who will ask for the most support.

I also initially made a free tier to attempt to compete with other players in the market, offering dozens of uptime checks for free. Again, while this attracted tons of free users, it was expensive with little return to the business, so I eventually had to trim down the free tier to a level that the business could support.

Rethinking the free tier

For a very long time, it wasn't possible to even trial OnlineOrNot's paid plans. You would start on the free tier, and if you needed more checks, you would upgrade. This made growth extremely slow and painful as a founder.

I saw this as starting the relationship with a customer like this: "Hey, this is a free tool, if you use it a bit more, you can pay me". I needed to flip the relationship to be more like "Hey, this is a paid tool, if you want slower checks for your personal projects, I can help you out".

It wasn't until I flipped the relationship - starting you off on a free trial of OnlineOrNot's best features - that OnlineOrNot started growing significantly faster.

Battling churn

When I think about OnlineOrNot's customers, there are five "journeys" to spend my time building features for:

  • Lead nurture (aka turning random folks browsing the site into free trials)
  • Trial conversion (turning free trials into paid customers)
  • Trial abandonment (what happens to the folks that don't subscribe)
  • Customer success (ensuring the first month of use is without drama)
  • Customer retention (communicating value over the long term, and keeping the product useful)

When launching a product, most folks (myself included) tend to only focus on lead nurture and trial conversion. This makes sense when you don't have customers yet, I'll admit.

It wasn't until I started balancing my feature development between these five points, rather than just the first two that I got churn down to an acceptable level.

Just keep shipping

One of the beautiful things about bootstrapping a business while working a full-time job is that it's default-alive.

There is near-zero cash burn.

The only thing stopping folks from reaching their first ten paying customers is persistence. Just keep shipping (features, marketing), and eventually you'll get there.

]]> <![CDATA[OnlineOrNot updates from November 2023]]> https://onlineornot.com/onlineornot-updates-from-november-2023 https://onlineornot.com/onlineornot-updates-from-november-2023 Fri, 01 Dec 2023 00:00:00 GMT Over October and November I focused on adding new features to Status Pages and making it easier to iterate on new features, as well as fixing bugs across all of OnlineOrNot.

Table of contents

What's new

New graphs for Status Pages

While compact, OnlineOrNot's old Status Page graphs were often difficult to read, and tried to pack too much information into a small space.

We now have new graphs:

New Graphs for Status Pages

The new graphs are:

  • interactive, with tooltips displaying each data point
  • visually larger, with y-axis grid lines for easier reading

You can add metrics graphs to your OnlineOrNot Status Pages by adding uptime checks to them, as described in this support article.

Dark Mode for Status Pages

A common ask from customers (and my customers' customers) is to make OnlineOrNot Status Pages less blinding when viewed on a system that has a dark-mode theme enabled.

So we now support dark mode:

Dark Mode for Status Pages

By default, OnlineOrNot Status Pages will use the user's system theme to decide whether to default to dark mode or not. Dark mode itself can be toggled by clicking the moon/sun icon at the bottom of each status page.

Documentation as a YouTube channel

While OnlineOrNot has a fairly comprehensive set of documentation, not everyone learns best by reading. To remedy this, I started a dedicated YouTube Channel for OnlineOrNot, and a Screencasts page, where folks can visually learn how to configure OnlineOrNot for their use case:

Screencasts page

Want a feature explained in a video? Let me know!

Thank you

OnlineOrNot only exists because of folks like you. I don't have a massive marketing budget, OnlineOrNot gets its users from folks like you, enjoying it enough to tell their friends about it.

So here's an ask: I work on the things that people care enough about to tell me about them. If you think there's something missing, or something that needs improving, reply to this email/email me at max@onlineornot.com - I'd like to know more!

]]> <![CDATA[OnlineOrNot updates from September]]> https://onlineornot.com/onlineornot-updates-from-september-2023 https://onlineornot.com/onlineornot-updates-from-september-2023 Mon, 02 Oct 2023 00:00:00 GMT Coming back from August holidays, I felt the need to take a hard look at what OnlineOrNot does, and keep improving it.

Table of contents

What's new

Cron job monitoring

I mainly spent September refining cron job monitoring functionality - the system handles millions of cron job checks per day, and is now production-ready.

Cron job monitoring flips how OnlineOrNot works in reverse: instead of checking your website to see if it's online, with cron job monitoring you tell OnlineOrNot if your cron job/scheduled task/IoT device/internet connection is still online via HTTP request, and if the time between requests is too long, you'll get an alert.

After a bit of feedback (particular thanks to Jens!) from OnlineOrNot's customers, quite a few things got built and fixed for cron job monitoring:

  • You can now display the status of your cron job on our status pages, and have OnlineOrNot automatically open and close incidents based on your cron job monitoring
  • You can now send alerts about your cron jobs to the following places:
    • SMS
    • Email
    • Discord/Slack
    • Webhooks
    • On-call integrations: PagerDuty/Opsgenie/Grafana OnCall/Spike.sh
  • Cron job monitoring now lives on its own dedicated infrastructure, redundantly across several AWS regions
  • Fixed several bugs in the heartbeat monitoring graphs

On-call integrations

On-call integrations aren't just for cron job monitoring by the way!

Back in late July I added support for uptime checks to alert based on your team's existing on-call roster for:

Automated free tier

OnlineOrNot has offered a free trial of its best features at the start for every new account for well over a year now, but what happens at the end of that free trial hasn't been particularly clear.

I used to think I was being clever by sending an email letting folks know they can pick between upgrading to a paid plan, or using the hobby plan for non-commercial projects. If they did nothing, OnlineOrNot would stop monitoring. In reality, many folks would miss that email, and not realize that OnlineOrNot stopped monitoring.

This goes against what I built OnlineOrNot for, so in September I automatically converted thousands of accounts with expired free trials to use the free tier. Going forward, folks that don't decide will automatically have their accounts converted to the free tier.

Thank you

OnlineOrNot only exists because of folks like you. I don't have a massive marketing budget, OnlineOrNot gets its users from folks like you, enjoying it enough to tell their friends about it.

So here's an ask: I work on the things that people care enough about to tell me about them. If you think there's something missing, or something that needs improving, reply to this email/email me at max@onlineornot.com - I'd like to know more!

]]> <![CDATA[How to monitor your uptime with OnlineOrNot]]> https://onlineornot.com/uptime-monitoring-best-practices https://onlineornot.com/uptime-monitoring-best-practices Wed, 27 Sep 2023 00:00:00 GMT Jumping into monitoring software for the first time can be pretty overwhelming. If you're not in an exploring mood it can be easy to get lost, and you're not sure what all the knobs and buttons do.

To help lighten this feeling for OnlineOrNot, I thought it might be useful to let folks know how I use OnlineOrNot, to monitor OnlineOrNot.

You might think it's silly to monitor your own site as an uptime monitoring service, however as I keep the monitoring infrastructure separate from the marketing website and web app, I actually get notified if anything goes wrong.

Table of Contents

What to monitor

To start with, I monitor:

As OnlineOrNot's marketing site is mainly static HTML (not powered by anything server-based like WordPress), monitoring the main landing page covers almost every page that could go down. I also monitor the sitemap as it's generated by a script at build time, and that script has failed in the past.

Settings for Landing Pages

I use the following settings for both my main landing page, and the sitemap.xml file.

To start with, I have OnlineOrNot check its own landing page every thirty seconds, from around the world: OnlineOrNot landing page monitoring settings

Things can get noisy on the internet, and it's possible for a website to fail uptime checks for a minute or two without it being a particular drama (assuming it's not a regular occurrence). As a result, I only want to be notified if my landing page check fails four times in a row (two minutes of continuous downtime):

OnlineOrNot landing page monitoring advanced settings

To be sure it's actually the page I expect that OnlineOrNot is checking, I also set the 'Text to search for' to look for text on the webpage.

Settings for APIs

For APIs, things are a little bit different. If the API check fails, I know something is wrong, and needs investigating immediately.

I have OnlineOrNot check its own public API every thirty seconds: OnlineOrNot API monitoring settings

To make OnlineOrNot actually check the API correctly, I have OnlineOrNot make an API request as a real (test) user, with a valid JWT token.

I set the following HTTP request settings:

OnlineOrNot API HTTP Request settings

It's important to test for correctness too, so I set assertions to check the data is coming back correctly:

OnlineOrNot API assertion settings

Finally, in advanced settings, I set the check to only alert me if two checks in a row fail.

As I'm already checking the response via Assertions, I don't set 'Text to search for' for my API check.

OnlineOrNot API monitoring advanced settings

Alert settings

As I check my phone (way too much), I find email notifications to work quite well when things go wrong (no additional settings required).

For added redundancy though, I also have alerts sent to Slack and Discord, which I've added as integrations for my account:

OnlineOrNot alert settings

]]> <![CDATA[Scaling AWS Lambda and Postgres to thousands of simultaneous uptime checks]]> https://onlineornot.com/scaling-aws-lambda-postgres-to-thousands-of-uptime-checks https://onlineornot.com/scaling-aws-lambda-postgres-to-thousands-of-uptime-checks Sat, 16 Sep 2023 09:52:00 GMT When you're building a serverless web app, it can be pretty easy to forget about the database. You build a backend, send some data to a frontend, write some tests, and it'll scale to infinity with no effort, right?

Not quite.

Especially not with a tiny Postgres server. As the number of users of your frontend increases, your app will open more and more database connections until the database is unable to accept any more.

That's just the frontend - it gets worse on the backend.

If you're using AWS Lambda functions as asynchronous "job runner" functions that increase in usage as your web app becomes more popular, you're going to run into scaling issues fast.

By revisiting my app's architecture, I removed the need for each worker function to talk to my database, and greatly improved scalability without needing to pay for a bigger database.

Curious? Read on.

Table of contents

First, some context

OnlineOrNot (where this article is hosted) started its life as an extremely simple uptime monitoring service. As a user you could sign in, add a URL you wanted to monitor, get alerts (only via email), and pay for a subscription.

OnlineOrNot's first form for adding URLs to monitor

Architecture-wise, it was pretty simple too.

It consisted of:

  • a single AWS RDS database instance running Postgres
  • a Lambda function that fired every minute to check the database for URLs to check
  • an SQS queue
  • a Lambda function that got triggered via SQS, which would check the URLs, and write the results to the database

The scalability issue

If you have a bit of experience with building on serverless, you can probably immediately see where my mistake was. As the web app grew in popularity, each URL that got added for monitoring would need its own database connection when writing results.

That might have been fine if I was developing an internal uptime checker at a small company (assuming you build for the problems you know you have now, rather than problems you might have), but I'm running OnlineOrNot as a growing bootstrapped business.

Shortly after thinking to myself "this is fine", I found myself needing to check up to 100 URLs at a time, every minute. Each time the jobs would finish, all 100 Checker Lambda functions would try to write to the database at (more or less) the same time, my database would refuse to open any more connections, including from my frontend!

To buy myself time, I scaled up the database a few tiers (the more memory your RDS Postgres instance has, the more simultaneous connections it can receive), and went looking for ways to fix the issue.

Fixing the issue

While implementing a fix for the issue, I managed to land my largest customer to that point. They wanted to monitor over a thousand URLs at a time, every minute.

I ran the numbers - if I kept upgrading database tiers every time I needed to scale, I would quickly run out of database tiers to upgrade to.

The fix in the end was to:

  • Instead of letting each checker Lambda write to the database, I had them return their result via Simple Notification Service (SNS)
  • Added a second queue, that batches results from SNS
  • Made a new AWS Lambda function to receive batched results, and write to the database from there

The architecture ended up looking like this:

As a result, the application now scales relative to how many resultfwdr functions are running, rather than the individual checker Lambda functions. This pattern is also known as fan-out/fan-in architecture.

With this fix in place, I was able to downgrade my database instance back to where I started, with only a minor percentage increase in CPU usage.

]]> <![CDATA[Guidelines for updating WordPress and its plugins, safely.]]> https://onlineornot.com/guidelines-for-updating-wordpress-plugins https://onlineornot.com/guidelines-for-updating-wordpress-plugins Sat, 16 Sep 2023 00:52:00 GMT Chances are, you already know how important it is to keep WordPress and its plugins up to date. If not, let this article be a wake-up call: you absolutely need to keep your system up to date.

All software has bugs and vulnerabilities (whether it's WordPress itself, themes/plugins, or Apache, Nginx, Linux, PHP) discovered every day. Updates patch those bugs to keep your website secure and functioning.

That's all well and good, but the problem is: how do you keep your WordPress installation updated, without an update taking your website completely offline without you noticing? It can be pretty challenging - you have a website that works for your business, you don't need to login to update the content more than a couple of times a year - the last thing on your mind is updating your WordPress plugins.

This article can help.

Table of contents:

Manually update your WordPress plugins

While automatic updates are best-practice in most other places, such as your phone and laptop, you definitely want to manually update each WordPress plugin one-by-one, on a regular basis (whether that's each week, or each month).

Why manually? So that you can observe the results, and hit "Rollback" in case your website stops showing content after you update the plugin.

Recovering a failed automatic plugin update

"What if it's too late?!" you ask?

You can recover from an automatic update taking down your WordPress website in a few steps, assuming you can log in to the server

  1. Log in to your WordPress host server
  2. Rename the existing wp-content/plugins folder, I'd call it something like plugins_temp
  3. Create a new, empty plugins folder
  4. One plugin at a time: copy your plugins back from the temporary folder, into the plugins folder, and refresh your WordPress website in your browser - repeat until the site breaks
  5. Once the website breaks, you know that's your bad plugin (or one of them)
  6. At that point, you can log into the WordPress admin, and either rollback the plugin, or if it's a premium plugin, download a fresh copy and install it

Monitor your website, effectively

While the author of this article does run a website monitoring service, this tip applies regardless of which uptime monitoring tool you use: it's important to monitor correctly.

The last thing you want is to find out your website has been down for months, while your uptime monitoring tool has been happily reporting your site as "up".

A simple "is my website online?" check will not always work for WordPress. Those famous "Critical Error" screens (below) can show up without sending a "down" HTTP status (4xx or 5xx).

WordPress Critical Error Screen

For this reason, I recommend uptime checks with "text to search for" configured. These checks will look for text that only shows up when your content has loaded, while also looking for "down" HTTP status codes.

]]> <![CDATA[Domain expiry: How to prevent your domain from expiring]]> https://onlineornot.com/guidelines-to-help-avoid-losing-your-domain https://onlineornot.com/guidelines-to-help-avoid-losing-your-domain Tue, 25 Jul 2023 00:52:00 GMT Imagine you're sitting in your office, and you start noticing emails coming in asking if you'd like to buy your domain.

"Huh, that's weird, I already own that domain" you think to yourself.

A few more emails come in, and they're getting past the spam filter, so you decide to double check your domain manager. Doubt starts creeping into your mind, you start panicking, and you frantically scroll down to where the domain should be, and...

It's gone.

The only option you have is to pay the person that grabbed your domain $3000 USD.

Hold up, rewind...

This sort of scenario can be avoided, yet an entire industry of domain squatters exists due to how commonly it occurs.

In this article, I'll provide advice you can do today to keep your domain secure in the long run.

Table of contents

Enable 2FA

If your domain registrar supports it, enable 2FA (two-factor authentication, also known as MFA/multi-factor authentication). It'll send you an email/SMS/push notification when logging into your domain manager.

While not a foolproof way of stopping hackers (hackers could still phish your employees for their 2FA code - YubiKey devices don't have this issue), it'll slow them down and alert you if your account has been compromised.

Check that your domain is set to auto renew

Some domain registrars don't enable auto renew by default, particularly when transferring domains.

Check your domain registrar to see that it's enabled. You can usually find it in settings when managing your domain.

Lock your domain from transfer

While you're checking your domain manager has auto renewal enabled, also double check that your domain's "transfer lock" is also enabled.

Enabling transfer lock for your domain is effectively like a car alarm for your domain. If someone manages to get into your domain manager account, and tries to transfer the domain, you'll receive quite a few emails about it.

Enabling transfer lock might slow you down in the future when you want to change domain registrars, but if you don't plan on moving any time soon, you may as well turn it on.

Check your payment details

I know this one sounds obvious, but if the payment fails, your domain doesn't get renewed, and you typically have about 30 days to notice before your domain disappears.

The most common mistake is that your credit card expires, and you forget to update the payment details your domain registrar has on file.

On the off-chance your domain registrar accepts PayPal (or similar), also double check that the payment details that PayPal have are also up to date.

Use a reputable domain registrar

There are a few ways to interpret "reputable" - I mean large companies trust them with their services, and the business itself is trustworthy. Certain domain registrars also own companies that "drop catch" domains that expire from their services. Would you want to use a domain registrar that's financially incentivized to let your domain expire?

Here are some reputable domain registrars that immediately come to mind:

  • AWS Route 53
    • Some of the largest internet companies trust AWS to host their services
  • Cloudflare Registrar
    • 20% of the internet's traffic runs through them, and the domain registrar service is provided at-cost
  • NameCheap
    • Decent reputation, has been around a very long time, used to make you pay for privacy, now doesn't
  • Gandi
    • Decent reputation, has been around a very long time

I've personally had negative experiences with GoDaddy and CrazyDomains (in Australia), and would strongly recommend to anyone reading this: transfer your domain ASAP to somewhere else.

Be sure you actually own your domain

If you purchased your domain through a third-party, like Wix, WordPress, or maybe the agency or contractor that helped build your site, chances are you're not fully in control of your domain.

Sure, it might be easier for you to have a third-party manage the domain for you, and pass the bill along each year, however this sort of arrangement can become problematic when you want to cancel the service, or move to another provider.

If you're having an agency or contractors build your site for you, and they become unresponsive, you risk losing the domain if you also let them manage it for you. By keeping the login to your domain manager to yourself, you can cut ties with rogue third-parties and move to a different provider.

Extend your domain registration

Most domain registrars will let you extend your registration for around $12 USD per year for .com domains. This lets you remove the risk of your automatic renewal not going through by manually renewing.

For example, AWS offers the following:

AWS Extend Domain Registration

Considering the amount of money domain squatters will try to get from you if you let your domain expire, it's a pretty good deal.

Be aware of any TLD-specific rules around renewals

While it's great fun to grab a domain from a country half way across the world from you so you can spell out your brand, different countries have different rules around domain renewals.

As an example, Spain (.es) charges a renewal fee on top of an annual fee. As well as that, if you let the domain expire, there's another renewal fee that ranges from 30 USD to hundreds of dollars (depending on your registrar).

Know when something goes wrong

Even with all these precautions, things can still go wrong. Payment failures, registrar issues, DNS misconfigurations - any of these can take your site offline.

Set up uptime monitoring so you'll know within minutes if your site goes down. Whether it's an expired domain, a DNS issue, or something else entirely, you want to find out before your customers do.

]]> <![CDATA[Our lessons from the latest AWS us-east-1 outage]]> https://onlineornot.com/our-lessons-from-the-latest-us-east-1-outage https://onlineornot.com/our-lessons-from-the-latest-us-east-1-outage Sun, 18 Jun 2023 00:00:00 GMT In case you missed it, AWS experienced an outage or "elevated error rates" on their AWS Lambda APIs in the us-east-1 region between 18:52 UTC and 20:15 UTC on June 13, 2023.

If this sounds familiar, it's because it's almost a replay of what happened on December 7, 2021, although that outage was significantly more severe and took longer to restore.

Table of contents

Some background on us-east-1

us-east-1 is the oldest and most used AWS region, it's where AWS EC2 was launched from in 2006. It has six availability zones (AZs), instead of the usual two or three.

Bugs that show up under high load tend to show up there first. If you look at the list of AWS outages, you'll notice almost every major outage since 2017 started in us-east-1:

a list of aws outages

The easiest mitigation strategy

While folks may say us-east-1 is "just another region", you could avoid being impacted by major outages with one simple strategy:

Migrate to us-east-2.

It's not 100% foolproof, as some AWS services host their dashboard exclusively in us-east-1, so even if your service isn't impacted by an outage, you might not be able to check out the dashboard during an outage.

Unfortunately for OnlineOrNot, as an uptime monitoring service, things aren't that simple.

The retrospective

Some context

Since OnlineOrNot's job is to measure uptime, every service is designed to run across multiple AWS regions. There's a high availability API that OnlineOrNot's services use to figure out which region to run from, and I use that to dodge major outages (something I learned to build back in 2021).

What went wrong

A couple of weeks before the most recent AWS outage as part of an architecture refactor, I spun up a new alerting service in us-east-1, without taking the time to implement the same multi-region failover that OnlineOrNot's other systems use.

As a result, during the us-east-1 outage while uptime checks were successfully being scheduled, and running - the service that collects the results from individual checks was not.

What went well

The only redeeming part of this incident, was that it proved the failover system works.

After the outage, I took the time to add failover to the alerting service, and now the entire OnlineOrNot system has the ability to hop AWS regions.

What I learned

  1. Don't run production workloads on services that don't have failover implemented
  2. Regularly test service failover, end-to-end
  3. It's almost never a good idea to build things that run only in us-east-1
]]> <![CDATA[No, the average cost of downtime is not $5600 per minute]]> https://onlineornot.com/average-cost-of-downtime-not-5600-minute https://onlineornot.com/average-cost-of-downtime-not-5600-minute Wed, 14 Jun 2023 00:00:00 GMT A fairly common claim among website uptime monitoring services is that downtime costs $5600 per minute. Chances are, you'll have one of two reactions to this claim:

  1. "Argh, that's a lot! We should use uptime monitoring!"
  2. "That's completely made up, no one running a business my size would tell you a number anywhere near this high"

The reality of what downtime costs your business lies somewhere in between.

As a company that runs 3.6 million uptime checks per week, we have a bit of insight into the cost of downtime, so if you're curious - read on.

Why it's not $5600 per minute

Let's start with the obvious calculation: take your annual revenue, divide it by the number of minutes in a year, and there's your cost per minute of downtime.

The formula for calculating your downtime cost per minute:

(annual revenue) / 60 * 24 * 365

This assumes 100% of your revenue comes from your website and your website being offline drops your revenue to 0, which are fairly strong assumptions to make.

For downtime to cost $5600 per minute by this measure, your business would need to be making $2.9 billion per year. That's almost enough to make it into the Fortune 500.

How much does downtime really cost your business

Let's say your business makes $1 million per year, and 100% of revenue comes from your website and web app, and for each minute your website is down, you're unable to generate revenue:

1000000 / 60 * 24 * 365 = 1.902

At this level of revenue, downtime costs you $1.90 per minute. If your website is offline for a couple of hours without you noticing, that's a couple hundred dollars in lost revenue.

Why this calculation is not enough

Unfortunately, things aren't this simple in the real world. Downtime also costs your business reputational damage from both prospective customers unable to pay you, and existing customers unable to use your service. For e-commerce businesses, the above calculation is a rough estimate at best.

For Software as a Service (SaaS) businesses, the above calculation doesn't make sense, as SaaS businesses often run their marketing websites separate to the systems that their customers pay them for (and will often have significantly more advanced monitoring to ensure everything is still running). So their marketing websites going offline might be a little embarrassing, but not the business-ending event some folks make it out to be.

]]> <![CDATA[OnlineOrNot updates from May]]> https://onlineornot.com/onlineornot-updates-from-may https://onlineornot.com/onlineornot-updates-from-may Thu, 08 Jun 2023 00:00:00 GMT It's a warm afternoon here in Toulouse, I feel like it's as good a time as any to update folks about what's new in OnlineOrNot.

What's new

In May I spent time making OnlineOrNot better for folks monitoring as a team.

The headline feature is that I've updated how Slack integrations work with OnlineOrNot: you can now disable Slack alerts for an uptime check, or even pick which channel alerts should go to. Folks on the Team plan and above can add multiple Slack channels, and send alerts directly to individual people in your Slack.

Not everyone uses Slack of course, there were other updates:

  • Admins of OnlineOrNot teams can now remove people from a team
  • You can now monitor entire HTML tags with the "Text to search for" setting in Uptime Checks
  • OnlineOrNot has been accepted as a "known bot" in Google Analytics 4 (GA4). This means once GA4 updates (I'm told this will happen by June 23), uptime checks won't appear in your GA4 analytics anymore
  • Cron Job Monitoring is now available to try out. The feature is still being refined in beta, but now anyone can try them out.

What's next

Almost all of my product ideas come from a very simple process:

  • When folks sign up, I ask them a single question about what brought them to OnlineOrNot
  • Some folks will respond with some of the pains they've experienced while monitoring their websites using other tools
  • I'll propose a solution, and those folks tend to stick around once it's built

June will bring a few things:

  • Making it possible to fetch historical uptime data
  • Making it possible to add multiple Discord integrations, and pick which alerts get sent to which Discord channel
  • Rethinking how OnlineOrNot handles alerts and incidents: at the moment you get one email when things go wrong - sometimes you want to keep getting reminded until you confirm that you've seen the alert
  • Rethinking how OnlineOrNot displays who gets alerts: as I add additional integrations, the "Alert Settings" section for Uptime Checks gets harder to understand

Thank you

OnlineOrNot only exists because of folks like you. I don't have a massive marketing budget, OnlineOrNot gets its users from folks like you, enjoying it enough to tell their friends about it.

So here's an ask: I work on the things that people care enough about to tell me about them. If you think there's something missing, or something that needs improving, reply to this email/email me at max@onlineornot.com - I'd like to know more!

]]> <![CDATA[On writing better error messages]]> https://onlineornot.com/write-better-error-messages https://onlineornot.com/write-better-error-messages Mon, 27 Mar 2023 00:52:00 GMT You're browsing your favorite website, clicking around, when suddenly, you're rudely interrupted by a white screen, proclaiming:

Error 503 Service Unavailable

Unhelpful Error Message (I don't mean to pick on Varnish cache here, It's just a screenshot I had handy)

As a developer, my eyes scan error messages like these for numbers - in this case, the "503" - indicating that the error isn't my fault, and I can move on with my life.

Unfortunately the majority of internet users aren't trained in the art of reading HTTP status codes, so this error message isn't particularly useful to them.

How to write better error messages

The majority of internet users aren't developers, so outputting the HTTP error code and its name (503 Service Unavailable) isn't good enough.

The Nielsen Norman Group back in 1998 (!) provided us with some basic guiding principles for writing better error messages:

  • Write in plain English (or whichever language you're supporting)
  • Tell the user exactly what went wrong
  • Tell the user how the problem can be fixed

More concretely, we can write better error messages by answering the following four questions:

  1. Who caused the error?
  2. What happened, and why?
  3. When will it be fixed?
  4. How can the user respond to the error?

If your error message covers those four points, then you can think about adding humour and some brand identity.

Note that humour and brand identity should never interfere with telling the user how to resolve the issue as briefly as possible:

Bad error messages like "Oopsie woopsie" are not recommended

Who caused the error?

The last thing you want to do is make your users feel dumb, or as though they're at fault for an issue with the service. Communicating who caused the error helps clear up any confusion.

Explicitly focus on "we" when the error is caused by an issue on your end (typically HTTP status codes in the 5xx range).

An error message such as

Our service is down for maintenance

is infinitely better than:

Uh-oh! Something went wrong!

For errors caused by the user (typically HTTP status codes in the 4xx range), be explicit about that too. For example, a 403 Forbidden error (where you know the user isn't authorized to view content) could be communicated as:

Access Denied. You do not have permission to view this page.

What happened, and why?

While users may not be technical, they still need an explanation of why they're seeing your error screen.

Take the classic 404 error message: "404 Not Found". You can make it significantly better for non-technical users with a single word:

Page not found.

adding a "why", makes it even better, giving them a way to fix the issue:

Page not found. You might have mistyped the URL.

When will it be fixed?

It's relatively difficult to keep an error message updated with details of your outage, and when you expect the service to become available again.

A better approach would be to link to either your status page, or Twitter account, or both, as in GitHub's case:

An example of GitHub's service error page

How can the user respond to the error?

In the case of a 404, you might want to list some steps the user can take to fix the issue, such as:

  • Going back to the home page
  • Using your search bar to find the page if it's been moved
  • Contacting support

Whereas in the case of a 5xx server error, you want to communicate to the user that there isn't much they can do, and that it's not their fault.

My favorite example of a company doing this well is Airbnb:

An example of Airbnb's service error page

They tell users:

  • there's definitely an issue, and they're working on it,
  • to check out their Twitter account for updates,
  • a way to get support for urgent issues,
  • and they set the expectation that they may be slow to respond while the site is experiencing downtime

Summary

We as developers need to improve our error messages. Try to be helpful, and explain:

  1. Who caused the error?
  2. What happened, and why?
  3. When will it be fixed?
  4. How can the user respond to the error?

Depending on the type of service you run, you'll likely still need request IDs and other diagnostics in your error message to help your support staff debug the issue.

A human-readable error message doesn't have to come at the expense of removing all technical information.

]]> <![CDATA[Monitoring our monitoring]]> https://onlineornot.com/monitoring-our-monitoring https://onlineornot.com/monitoring-our-monitoring Wed, 08 Mar 2023 08:52:00 GMT Last Saturday, our API went down. Not even a funny error message or slightly slower responses either, it just completely vanished off the internet for 18 minutes.

I'm not normally one to point fingers at my hosting provider when things go wrong (since ultimately, I chose to use them, so it's my problem to fix), but when fly.io publicly posts on their forums about their reliability issues, I may as well link to them.

That being said, my fix was "simple": our monitoring is multi-region and multi-cloud, our API can be too.

Since I use Cloudflare Workers in front of everything, I started using a Worker as a sort of load-balancer to send traffic to different providers, in the hope that it'll be more resilient in case of an individual host suffering another outage.

Suddenly, false-positives

Around the same time, I noticed the number of email alerts OnlineOrNot was sending was a bit high.

On a bad day, we'll send out maybe 300 emails. On Saturday, we sent out 4900.

This Saturday morning, I rolled out a change to OnlineOrNot to migrate from a system that checks from a single region by default (us-east), to one that checks around the world (us-east -> eu-west -> south-east asia -> us-west) significantly more frequently (up to every 30 seconds for paid plans).

As part of this change, a bad deployment to Singapore resulted in elevated error alerts for almost all of our uptime checks from Singapore. At first I tried finding another region that wasn't having reliability issues, before just rolling back the change entirely.

What I did about it

This weekend made me realize I didn't have a reliable way of knowing when OnlineOrNot's own monitoring was having issues. While OnlineOrNot does use OnlineOrNot to monitor OnlineOrNot quite successfully, it only covers its web presence (marketing website, web app, API), and not the services running the system.

Instead, I would rely on a live-tail of the logs to see how things were going (essentially, vibes-based monitoring), and for two years that worked quite well! I also have a sort of "deadman's switch" to automatically change hosting provider running the uptime checks in the case of total outage, but that didn't help here.

To start with, I've started storing the results of each check in Clickhouse and built a Grafana dashboard for a live view of the system:

OnlineOrNot's Grafana dashboard

While it doesn't sound like much, this sudden visibility into how the system is running (live) gives me the confidence to make improvements I wouldn't otherwise be able to make, such as quadrupling uptime check concurrency on a single VM, and tweaking default timeouts and retries to reduce false positives.

We also have distributed tracing in other parts of the system (status pages), which I'll be adding to the uptime check system to get a bit more of a detailed view into how each check runs (more than just console.log, at least).

]]> <![CDATA[Announcing OnlineOrNot's open-source uptime check CLI]]> https://onlineornot.com/announcing-onlineornot-uptime-check-cli https://onlineornot.com/announcing-onlineornot-uptime-check-cli Thu, 02 Mar 2023 00:52:00 GMT I'm excited to be launching onlineornot - OnlineOrNot's newly-built, open source (TypeScript) CLI for managing your uptime checks, all from the comfort of your terminal!

Getting started is a matter of:

You can also check out the docs to see what else you can do.

"Why a CLI?" you might ask - OnlineOrNot needed a way of testing the newly built public API.

To start with, the public API supports fetching data about uptime checks and API tokens, but I'll be supporting status pages soon too.

If you don't feel like using the API, you can always pass a --json flag to the CLI to get your output as JSON, and pass it to the next command you'd like to run.

]]> <![CDATA[What I learned running a SaaS for a second year]]> https://onlineornot.com/lessons-from-two-years-of-saas-operation https://onlineornot.com/lessons-from-two-years-of-saas-operation Mon, 20 Feb 2023 06:52:00 GMT Two years ago, OnlineOrNot started as a little toy app I built in an afternoon to see what it's like using the Next.js framework, to see if a URL is down from around the world.

I gave myself a week to turn that toy into a SaaS people could pay for. It looked like this when it went live:

OnlineOrNot after one week

It wasn't ready for real users, but that didn't matter. I had something out there, that people could sign-up for, tell me what they were expecting, and how OnlineOrNot fell short of their expectations.

Since then, I've been shipping features in two hour blocks, cutting down scope aggressively to ensure something goes out into the world, each time I write code for OnlineOrNot. I then get feedback on what goes out, and the app improves.

These days, OnlineOrNot can do a quite a bit more than just visit a page, and send an email alert when it's down:

OnlineOrNot, present day

OnlineOrNot Uptime Checks get used to monitor web apps, APIs, internet connections, residential power, IoT devices, blogs, and of course, regular websites. It's also no longer just an uptime monitor. OnlineOrNot is more of a status page service, with built-in uptime monitoring.

I learned quite a bit in my first year of running a SaaS, and this year's learning builds on top of that.

Table of Contents

OnlineOrNot runs on opportunity cost

I don't spend anything on customer acquisition apart from opportunity cost. I build features and write content for OnlineOrNot's customers at the expense of possibly other lucrative activities I could be doing.

Folks often look at my "2 hours per work day, every work day" rule, calculate roughly what they think my hourly rate is, and say "pffft I'd rather be doing nothing".

The thing is, I'd rather be doing this. To me, OnlineOrNot is like painting, and if years down the track I decide to start a new artwork, I don't have to learn to paint again.

No, running two SaaS businesses isn't a good idea

Early on in the year, I had the bright idea to spin out OnlineOrNot's internal feature flag service, and make it a SaaS project.

The thought process was essentially:

| Working on two projects at once, a tweet | | :---------------------------------------------------------------------------------------------------------------------: | | it was not different this time, dear reader. |

I spent a weekend building a prototype SaaS app, launched it, and nothing happened. I talked to potential customers throughout my professional network, and folks were just not that keen. It turns out, I wasn't that keen either. I worked a bit on content marketing, let the project sit for a few months, and eventually shut it down.

It cost me time I could have spent building features for OnlineOrNot, and was just a distraction, in the end.

It turns out no one cares if you came up with a really fast way to do feature flags over Cloudflare's shiny new developer platform - your service didn't exist a week ago and has almost no features, and there are feature flag services that have been around for a very long time, with an adequate feature set.

It made me realise OnlineOrNot's moat is that I don't plan on giving up on it, I'm genuinely interested in the problem space, and I keep myself employed full-time so that I can build OnlineOrNot the way I want, with zero risk to my livelihood.

Ship small, invalidate your assumptions

The trouble with holding off from releasing a feature because it "isn't ready", is that folks can't tell you if your assumptions are wrong, until you spend months building on top of your incorrect assumptions.

OnlineOrNot's Status Page feature came from a conversation with a customer trying to figure out how to sign up hundreds of email addresses to get notifications when an uptime check fails.

I built something as quickly as possible, showed the customer (and a few other customers, friends, and colleagues), realised where I went wrong, and tried again.

OnlineOrNot itself was released after only 7 days of part-time development, after all.

Talk to potential customers, even if you think you have nothing for them

I was contacted by a CTO at some point during the year asking if OnlineOrNot supported some particular feature.

My normal reaction would've been to just say "sorry, no" and leave it at that, but I got curious and started asking what they'd like to achieve with the feature, and told them how I figured I would build the feature.

They signed up to a paid plan the next day, they've been customers ever since, and the feature gets used by other customers too.

That's all for what I learned this year. Below is a sort of CHANGELOG of what I shipped in OnlineOrNot.

What I actually shipped

I recently wrote 2022: I just kept shipping, and while it's nice to tell folks "just keep shipping", I think it's valuable to point out just how much you can get done if you give yourself 2 hours, and ship ever day.

There were months where my focus was more on marketing/writing docs/helping and talking to customers/drinking wine by the lake/I just didn't feel like writing code, so not all months resulted in the same amount of feature development, but I still released my articles/docs/etc ASAP, and iterated on them while they were live.

January

  • The uptime dashboard now actually tracks the last 24h of uptime (used to be response time)
  • Added auto-refresh to the uptime dashboard so folks don't need to remember to refresh the page
  • Made the uptime dashboard mobile-friendly
  • Migrated the database from Intel to ARM
  • Made the rest of the web app mobile-friendly
  • Made it clearer why OnlineOrNot thinks a check is failing
  • Reached 50 million all-time uptime checks
  • Published Communicating to Users During Incidents
  • Cleaned up my sign-up form to remove distractions, halving the drop-off rate in the process
  • Built the first screen of a new onboarding flow and immediately released it
  • Added a second screen to the onboarding flow to enable people to join the mailing list
  • Added a third screen to the onboarding flow to figure out how people found out about OnlineOrNot (majority of folks come from me shitposting on Twitter, and commenting on hacker news) and what their name/company name is (so I finally stopped sending emails starting with "Hey there,")
  • Added a fourth screen to the onboarding flow to let folks add their whole team in one go (it's just a textfield, and I parse it to invite folks)
  • Added GitHub Auth to the login/signup screen
  • Added a free trial sign-up option to the onboarding flow (that's right, I only added free trials after 11 months of running the business)

February:

  • After shipping the free trial sign-up, I had 14 days to implement what happens at the end of a free trial, so I shipped that separately. In the meanwhile, I was manually sending folks their "getting the most out of your free trial" emails
  • Removed the paywall from a few uptime monitoring features
  • Wrote the first year's version of this blog post, around 33k people read it
  • Wrote How OnlineOrNot uses OnlineOrNot to run OnlineOrNot
  • Added a "you're on a free trial!" banner to OnlineOrNot
  • Wrote a pair of guides for the docs to help folks getting started with their free trial
  • Added human-sounding errors when uptime assertions fail
    • for example: "Looked for a value at $.data.viewer.sendWeeklyReport, found true, which we expected to be false"
  • Added the ability to report on SSL certificate validity
  • Worked on the deployment pipeline, getting a 7 minute build down to 1 minute 45 seconds

March

  • Got tired of forgetting to write docs after releasing features, so I moved the docs into my app's monorepo, and gave it the same look and feel as the rest of my web presence (I still get customers telling me they wish their app had this - but I'm not planning on spinning off a docs product)
  • Made it possible to BYO Twilio account to enable unlimited SMSes for free
  • Added a "home screen" to OnlineOrNot
  • Used my new docs as a template for an Incident Management mini-site. Before this, I had a few blog posts about on-call, run-books, and navigating between them was annoying.

April

  • Moved house, took the time to explore my new town/region, so didn't ship much
  • Tested and tweaked copy on my landing pages
  • Unified browser checks and regular uptime checks into a single dashboard

May

  • Moved most of my DNS management from AWS to Cloudflare (I got hired by Cloudflare for $DAY_JOB, and it's a lot faster than AWS)
  • Built and shipped the DNS system behind OnlineOrNot's Status Pages (what allows me to simultaneously display status pages at custom domains and on OnlineOrNot's subdomain)
  • Started showing the last 14 days of incident history on status pages
  • Actually made it possible for my paid customers to sign up for a status page at a custom domain

June

  • Made it possible to manually add incidents to status pages
  • Made it possible to manually update existing incidents on status pages
  • Started an Early Access Program so I didn't have to feature flag early features one account at a time
  • Migrated most of my core uptime check workload off AWS and onto fly.io, and wrote about it
  • Made it possible to add components to a status page
  • Made it possible to pick components affected by an incident
  • Published Writing your first runbooks
  • Published Guidelines for writing better runbooks

July

  • Made it possible to link existing uptime checks to a status page, and automatically start and resolve incidents based on uptime check data
  • Hacker News went down, 19k people ended up checking out https://hackernews.onlineornot.com, and I wrote about the experience
  • Made it possible to sign up to status page updates, without needing to sign up to OnlineOrNot
  • Rewrote the whole status page app from Next.js to Remix, because shipping 100KB of JavaScript for a basic HTML page felt ridiculous
  • Published Postmortem Templates
  • Edited and republished Improving your team's on-call experience

August

  • Summer holidays
  • Added a way to highlight active incidents on the status page
  • Made it possible to make a status page private (password-protection)
  • Added live system metrics to status pages

September

  • Used OnlineOrNot to acquire the @OnlineOrNot handle on Twitter
  • Worked on my landing pages
  • Fought off an attack from spammers looking to abuse OnlineOrNot
  • Added support for webhook events from Uptime Robot
  • Migrated the business from Australia to France

October

  • Holidays in Australia
  • Added Cloudflare Workers to the list of "backup uptime checkers" that OnlineOrNot uses to verify a website is actually down
  • Published Saving your team from alert fatigue

November

  • Made it even clearer why OnlineOrNot thinks a check is failing

December

  • Added support for HTTP Basic Auth in uptime checks
  • Hit 120 million uptime checks
  • Made it possible to duplicate an uptime check
]]> <![CDATA[Saving your team from alert fatigue]]> https://onlineornot.com/saving-your-team-from-alert-fatigue https://onlineornot.com/saving-your-team-from-alert-fatigue Tue, 01 Nov 2022 08:45:00 GMT It's a story as old as the web itself: someone on your team gets excited to install a new tool.

The tool promises to finally give you a clear view into the problems your users have with your product.

Your team agrees to give it a go.

The errors start coming...

...and they don't stop coming...

Soon enough, most of your team has either created an email filter to manage all the alerts, or has unsubscribed themselves entirely. Just like all the other tools.

Welcome to alert fatigue.

What this article covers:

What is Alert Fatigue

In short, alert fatigue happens when you start to become desensitized to alerts.

It tends to happen after receiving multiple alerts with similar content (often with limited context into the problem, so not enough information to fix the alert).

Alert fatigue can strike relatively quickly when there's a large number of alerts (often irrelevant to the developer), and there's not enough time to resolve all of the issues.

Beating Alert Fatigue

This section suggests a few ways you can beat alert fatigue.

It boils down to: making alert management a team effort, continuously cleaning up your alerts, and categorizing your alerts.

Make alert management a team effort

There is nothing worse than being paged for a system you didn't create, with an alert you have no autonomy over.

If your team has an on-call roster, it should have autonomy to create, modify, and remove alerts as it requires. Sometimes alerts are flaky, and need to be temporarily disabled or have its thresholds changed, and sometimes an alert no longer fits the team's needs, and should be removed entirely.

Clean up your alerts

Cleaning up your alerts doesn't have to be a big task to do in one go, you can continuously clean-up alerts as part of an on-call roster.

The idea is to look at every single alert as they fire, and figure out if there's a clear human response required. If an alert fires, and there's no human action required as a result, it shouldn't be taking your team's attention, and needs to be removed.

Getting paged constantly while on-call is a symptom of a broken system.

Either the system is too unstable, and needs time invested to make it resilient, or your alerts are too verbose and their thresholds need tweaking, or they require no action, and need to be deleted.

Categorize your alerts and notifications

The beauty of having an on-call roster within your team, is that your team agrees that one person gets to focus entirely on either fighting fires, or improving the on-call experience for the week.

Grooming your alerts greatly improves your team's on-call experience. Broadly speaking, sending every notification or alert your system generates to the same location is a mistake.

There are three types of alert that we care about, and they need to go to different places, as they help us do different things. Not every alert needs to wake someone up. Not every alert needs to go to a Slack channel.

While it's easy to sit on my armchair here and give you clear categories, in reality your team will often get high priority alerts that aren't really high priority, and low priority alerts that really should be looked at immediately. Part of your team's on-call responsibilities should be to review the alerts that fired each week, and re-categorize alerts as necessary.

High priority alert - immediate action required

This covers things like: your website being completely unreachable after several checks in a 5 minute period, your SSL certificate expiring, or customers being unable to pay for your product.

These are the types of alerts that need go to someone's phone (SMS, call or pager app under a high priority), and wake them up if they happen outside of business hours.

Once your team is aware of the alert, they start up an incident room, and start working through your team's runbook for the service affected.

Medium priority alert - awareness required

This covers things like: your system's database backup failing, and starting to run out of disk space on your database server.

They're still particularly bad things, but nothing that immediately impacts your customers.

These types of alerts that can go under a lower priority in your pager tool or to a team channel in Slack/Discord/Microsoft Teams. Since they're lower priority, your team should only investigate if there's no active incident, and generally only during business hours.

Low priority alert - things to fix during spare time/diagnostics

This covers things like: individual 5xx errors, server timeouts, and JavaScript errors.

Most commonly, these are the alerts your team gets from Sentry. It may be tempting to dump them into a Slack channel, but your team will quickly start to mute/ignore the channel.

]]> <![CDATA[The unreasonable effectiveness of shipping every day]]> https://onlineornot.com/unreasonable-effectiveness-shipping-daily https://onlineornot.com/unreasonable-effectiveness-shipping-daily Fri, 26 Aug 2022 06:52:00 GMT It's fairly common for folks in tech to dream of quitting their day job and working on their side projects. I find when you ask them how their projects are going, they tend to have 2-3 projects running at the same time, none of the projects are actually available for potential users to try out.

The question they seem to ask me most is "you seem to complete your projects, how do you stay motivated?"

My secret? It's a habit. I ship something every day.

Table of Contents

Some context

For context, since 2017 I have shipped:

I initially got started because I wanted to learn to use React and GraphQL better for my day job, but these days I enjoy running the business and learning about marketing.

Forming the habit

Actually forming the habit is the most difficult part, but I promise it gets easier.

To get started on each project, I prioritize getting an absolute minimal "viable" product live and able to accept customers as soon as possible. In the case of my most recent project (that I've now been working on continuously for two hours per workday since February 2021), this took one week.

I say "viable" because while users could login, subscribe to a paid plan, add pages to monitor, and receive downtime notifications, it wasn't an actually viable solution to their problem. It wasn't until I added significantly more features that users started paying me for the service.

Once a product/blog article/book is live, you can start getting users. Once you start getting users, their feedback helps inform what you should work on next.

How I work

I tend to work on my personal projects in 2 hour increments (before my work day starts), partly because that's how long I can focus at a time, and partly because I prefer to chip away at a problem, rather than making one big push.

Usually by the end of those two hours, I'll have something running live.

You'll find two hours isn't nearly enough to "complete" a feature. Get used to being embarrassed by your incomplete project.

In other words, limit scope as much as possible.

Practically, this means releasing forms that gather less data than needed, or only building the UI portion of a feature before the backend is ready. For books, it could mean releasing a chapter as an email to your newsletter subscribers. For SaaS, I'd recommend starting an early access program and let users that opt-in know when they're looking at incomplete features.

How this works in practice

It's all well and good for me to tell you to "ship more!", but it feels pointless without providing a concrete example of how it works in practice for a real SaaS product.

Here's roughly how OnlineOrNot's uptime monitoring was built:

  1. Released a simple form that accepted a URL to monitor, and a name for the uptime check
  2. Made it possible to receive email notifications when an uptime check is down
  3. Made it possible to search for text on the page while checking
  4. Made it possible to customize the uptime check frequency
  5. Made it possible to sign up for a Slack notification when your uptime check is down
  6. Made the Slack integration actually send a notification when the uptime check is down
  7. Made it possible to wait for several failed uptime checks before sending a notification
  8. Made it possible to edit existing uptime checks

All of these features were delivered while the service was live, accepting users, and those users were helping inform the direction of the product.

You'll find an Early Access Program helps a lot for cases like Steps 5 and 6 above - I'll often release forms to my early access users that only save their preferences, without actually taking the preference into account until a few days later when I fully complete the feature.

Caveat: the types of folks that opt-in to an early access program tend to be accepting of this kind of thing, I wouldn't recommend shipping incomplete product features to your entire user base unless you're extremely early in your product's development.

Summary

Consistent effort over a long time helps you accomplish things you think are impossible.

Get used to being embarrassed by your incomplete project.

I wish I realized this in high school/university.

]]> <![CDATA[How I accidentally told 19k people Hacker News was down]]> https://onlineornot.com/how-i-accidentally-told-19k-people-hacker-news-was-down https://onlineornot.com/how-i-accidentally-told-19k-people-hacker-news-was-down Fri, 12 Aug 2022 08:45:00 GMT Hacker News is a forum run by Y Combinator that tends to focus on topics relating to computer science and entrepreneurship. During an average month, Hacker News serves almost 10 million visitors.

Hacker News had a series of extremely rare outages in early July 2022:

OnlineOrNot downtime stats for Hacker News

On July 8, 06:20am UTC, the hard drives in Hacker News's primary server stopped working, and they switched over to the failover server in just over an hour.

The failover server chugged along for almost six and a half hours before the hard drives also failed at 12:47pm UTC.

At this point, it looks like all four hard disks reached 40,000 hours of uptime, and a bug in their firmware caused them to fail.

About OnlineOrNot

OnlineOrNot is a new kind of Incident Management software that runs on Cloudflare Pages. OnlineOrNot provides uptime monitoring, with integrated status pages that automatically update.

The architecture for our Status Page service is relatively simple: it's a Remix frontend app (we use Remix as a React framework that lets us write React components, and output HTML/CSS with minimal JS). We do not currently use a client-side framework. As a result of this approach, our average page weight is around 13KB (instead of 200KB).

Our Remix app runs on Cloudflare Pages, which itself uses Cloudflare's global network, without direct access to my database.

To workaround this limitation, we built an Express API server running on fly.io. While I usually opt for a serverless solution for early versions of features, I wanted to avoid cold-starts. Another advantage of using fly.io is that we can deploy replicas of the API server globally. However thanks to Cloudflare Pages' ability to access Cloudflare's cache, this won't be necessary as most requests under high load won't hit my origin server.

Letting folks know what happened

While testing the status page feature described above, I decided to monitor Hacker News. To my surprise, the first result came back saying Hacker News was down, and my system automatically created an incident on the status page.

When Hacker News came back up (running on the failover server), naturally, I created a thread on Hacker News, thinking no one would notice or care that Hacker News went down.

That thread quickly reached the front page, was in the number one spot for about half an hour, and thousands of users were sent to my Hacker News Status Page:

OnlineOrNot's Cloudflare stats

Eventually Hacker News went down again as the disks in their failover server also failed, and my automated monitoring kicked in again, creating a second incident page.

This second outage lasted roughly seven hours, with no updates from the Hacker News team for several hours (understandably so, the failover servers failed in the middle of the night). During this time, Google indexed the status page I built, and temporarily gave me the top search result on Google for "hacker news status page".

Now that I know the feature works (and at scale - I'm grateful for the load test!), I'll be releasing an embarrassingly under-featured MVP this week.

]]> <![CDATA[Postmortem Templates]]> https://onlineornot.com/postmortem-templates https://onlineornot.com/postmortem-templates Wed, 27 Jul 2022 00:52:00 GMT After things go wrong and the incident is resolved, it's time to learn.

Generally in tech, the act of reviewing the cause of an incident is known as a postmortem, though there may be other names used such as "Post-Incident Review (PIR)" or "Correction of Error (COE)"

Regardless of what you call this process, you want to use a template so that your team has a standardized way of documenting what went wrong, and what your organization will do differently in the future to avoid repeat issues. It also helps to store these documents in a centralized place so that your teams have the opportunity to learn from each other's mistakes (Google Docs, Confluence/Notion, etc).

A note on root causes in the cloud

It's common for postmortem templates to include a five whys analysis to determine a root cause.

Be aware that when building software in the cloud, there is rarely a "true" root cause due to the sheer complexity of building distributed software. It's still worth attempting to find improvements to your organization's processes to avoid repeat incidents.

Amazon's template

  • What happened?
  • What was the impact on customers and your business?
  • What was the root cause?
  • What data do you have to support this?
    • Especially metrics and graphs
  • What were the critical pillar implications, especially security?
    • When architecting workloads you make trade-offs between pillars based upon your business context. These business decisions can drive your engineering priorities. You might optimize to reduce cost at the expense of reliability in development environments, or, for mission-critical solutions, you might optimize reliability with increased costs. Security is always job zero, as you have to protect your customers.
  • What lessons did you learn?
  • What corrective actions are you taking?
    • Actions items
    • Related items (trouble tickets etc)

Source: https://wa.aws.amazon.com/wat.concept.coe.en.html

Atlassian's template

  • Incident summary (what happened, why, incident severity, how long did the incident last?)
  • Leadup (the series of events that lead to the incident)
  • Fault (describe how the software misbehaved)
  • Impact (determine how many users were impacted, over which time period)
  • Detection (when did the team detect the incident, and how)
  • Response (who responded to the incident, when, and what did they do?)
  • Recovery (how was the system restored, what steps were needed to restore the system to a functioning state?)
  • Timeline (an incident timeline, using the UTC timezone to standardize time)
  • Five whys (describe the incident, ask why it happened, then ask why that happened, recursively, five times)
  • Root cause (what needs to be changed to avoid this happening again)
  • Backlog check (review your backlog to see if any unplanned work could have prevented this issue)
  • Recurrence (use the root cause to see if other incidents had the same root cause. Ask why the incident happened again)
  • Corrective actions (describe the work needed to prevent the incident happening again, who is responsible for completing the work, and when)

Source: https://www.atlassian.com/incident-management/postmortem/templates

Summary

In short, it doesn't matter which template you pick (or if you create one from scratch), as long as it's consistently used by your organization.

]]> <![CDATA[Improving your team's on-call experience]]> https://onlineornot.com/improving-your-teams-on-call-experience https://onlineornot.com/improving-your-teams-on-call-experience Tue, 26 Jul 2022 00:52:00 GMT Your engineers probably dislike going on-call for your services.

Some might even dread it.

It doesn't have to be this way.

With a few changes to how your team runs on-call, and deals with recurring alerts, you might find your team starting to enjoy it (as unimaginable as that sounds).

I wrote this article as a follow-up to Getting over on-call anxiety. While that article handles what individual engineers can do to psychologically prepare themselves for on-call, this article describes what the team can do to improve the on-call experience overall.

You build it, you run it

This point is important, which is why it's the first tip in this article: you cannot hire an SRE/DevOps team to magically make your operational problems go away.

If your engineers build a service and it's business-critical, they need to be on-call for it.

Handling on-call this way incentivizes your team to improve the service's operations over time, and reduces the risk of "just throw it over the fence to the ops folks" mentality that can develop when a team isn't on the hook for their own code.

Continuously improve your service

On-call shouldn't just be an additional responsibility to respond to alerts on top of your team's regular scheduled work (no sprint work, no picking up tasks from the backlog, nothing).

Instead when folks are rostered on-call, they should be working on tasks that improve the quality of life for on-call.

This stream of work will come up naturally if you take each alert that fires as an opportunity to improve your service. When things break, take the time to ensure an alert never fires again.

Especially if the alert is unactionable.

A quick note on unactionable alerts: If your service has alerts that do not have a single, clear, human response, the alert should be deleted. If it's a flaky alert, your team should prioritize fixing the issue as soon as possible.

Getting paged constantly while on-call is a symptom of a broken system. Either the system is too unstable, and needs time invested to make it resilient, or your alerts are too verbose and their thresholds need tweaking, or they require no action, and need to be deleted.

Over time, applying long-term fixes to the causes of your alerts will make your service more resilient, and make your team's time on-call a lot less stressful.

Google make this possible for their own SRE's by blocking them from spending more than 50% of their time on operational work:

We cap the amount of time SREs spend on purely operational work at 50%; at minimum, 50% of an SRE’s time should be allocated to engineering projects that further scale the impact of the team through automation, in addition to improving the service.

— Google's SRE Book

Good processes make a good on-call experience

If your team relies on engineers to regularly improvise while on-call, they (and your customers) are going to have a bad time.

Runbooks, up-to-date documentation, and standard operating procedures give your on-call engineers direction on how to respond to, and escalate, incoming alerts. You don't want them to be figuring this stuff out by themselves at 2am.

If you've already got these in place, have your newest team members review them. Chances are, there are blindspots that you and your team can't see due to familiarity with the services and their docs.

If you're the new member of the team, take the initiative to improve the documentation yourself. Try going through runbooks and see if you have enough context to complete the steps. A common mistake senior engineers will make is assuming their colleagues know where to run commands, or where certain repos are (particularly when engineers give a service one name, while the codebase lives in a repo with a completely different name).

Your documentation should be as explicit as possible.

Have a primary on-call, and a secondary on-call

In order to help drive the continuous improvement aspect of on-call, it helps to have an engineer on secondary on-call.

While the primary on-call engineer answers pages at any time of the day for a week, the secondary on-call engineer's role is to make the primary on-call engineer's life easier. They can assist primary with investigations when a particularly bad incident occurs, but their main focus should be improving service reliability (by improving the codebase/writing tests, fixing flaky alerts, deleting unactionable alerts) and generally ensuring primary doesn't get paged each week for the same reason.

While in the past we've had secondary on-call for only one week at a time, we found that due to ramp up time, a month of secondary on-call worked better.

As secondary on-call doesn't answer pages overnight, you can typically assign newer engineers to the role so they get familiar with your team's documentation and runbooks, and can help improve them further.

Handover between your on-call engineers

You should hold a meeting to handover on-call responsibilities between engineers as one shift ends and another begins. We typically hold these in the middle of the week, as most major issues for the week typically surface by then.

In the meeting, the engineer whose on-call shift is ending talks through the alerts that were fired, whether they were resolved, or if they still require attention. Commonly recurring themes (types of alerts that fire often) should be added to the on-call backlog, and prioritized to be fixed by the on-call engineers.

Let anyone on your team attend these meetings (particularly new employees), as it helps them get familiar with the service that they'll eventually be on-call for.

Pay your staff

Going on-call overnight is additional work, whether or not an engineer gets paged.

You need to pay a bonus on top of their regular salary for the week your engineers go on-call. In many countries, this is now a legal requirement (you may want to double check this with your legal team).

Particularly if you expect them to do their job well.

Conclusion

I wrote this article as a former-employee of a very large tech company (thousands of engineers). In our case, we had our regular software engineers that build the service also handle on-call for it.

Be aware that blindly implementing recommendations will not automatically make your company successful. Large tech companies adopt these practices as a means of managing their success - not because these practices alone make them successful.

That being said, if your on-call staff are constantly fighting fires, or the on-call experience isn't generally improving week to week, hopefully some of these recommendations will help.

]]> <![CDATA[On moving over a million uptime checks per week onto fly.io]]> https://onlineornot.com/on-moving-million-uptime-checks-onto-fly-io https://onlineornot.com/on-moving-million-uptime-checks-onto-fly-io Wed, 29 Jun 2022 08:45:00 GMT The other day, a friend told me about fly.io's nice developer experience (DX). For my day job, I work on improving wrangler2's DX, so naturally it had me curious.

I went from "I'll just play around with it, maybe give it a toy workload" to "holy shit, what if I quickly rewrite my business's AWS Lambda + SQS stack to fit entirely within their free tier" in about 90 minutes.

It wasn't that simple in the end, but I did manage to migrate most of my active workload from AWS Lambda to fly.io.

Table of contents

The project

I built OnlineOrNot as the 200th uptime monitor for the Internet. The idea was: I hate all the other solutions, I'm a frontend developer that can handle infra and backend, let's solve the problem the way I would solve it, while listening to any customers I might get along the way.

On the backend, an uptime monitor is simple(ish):

  • A process to query the database, find the checks that are ready to run, and queue them up as a job
  • A queue to hold jobs while waiting to be run
  • A process to pick up jobs from the queue, and run the check (and do something with the result)

To save money on the off-chance no one other than me ever used OnlineOrNot, I built it on AWS Lambda. Costs nothing if no one is using it (apart from the database, which I kept within the free-tier while building anyway).

What that looks like on AWS Lambda

On AWS Lambda, the system I built looks like this:

  • One AWS Lambda function that runs every minute to query the database
  • A Simple Queue Service (SQS) queue to hold jobs
  • One AWS Lambda function to pick up jobs, invoke (yet another) AWS Lambda function that performs the check, get the result, and do something with it

The vast majority of my AWS bill was coming from that last function - every time there's an uptime check that hits a timeout, I get billed for the full 10 seconds it spends waiting, twice (once for the check, once for the function waiting for the check), even though the functions aren't doing anything in that time.

The naive approach to migrating off AWS Lambda

My initial idea was to take that last AWS Lambda function (the one that has to wait for every timeout) code, use the SQS Consumer library to pick jobs off the SQS queue, build a Dockerfile to run that code, and boom, easily moved my biggest Lambda cost into a free VM.

It turned out that over a million uptime checks per week is a lot, and the SQS Consumer library couldn't keep up. It also didn't process the jobs in order, which made some jobs never even run their uptime check!

Replacing SQS with Redis

With the main solution to processing SQS queues not being fit for my use, I spent a day thinking about how to re-architect my entire solution to run in VMs.

After a bit of research, I found BullMQ. BullMQ replaces my SQS queue with a Redis instance, and handles the job creating and running for me.

After re-architecting, my fly.io-based solution looks like this:

  • A permanently running VM running Redis with persistent storage (in case it goes down)
  • A permanently running VM that uses node-cron to run every minute, and queues up uptime checks to run
  • A permanently running VM that picks up jobs from Redis, runs the checks

Initially I gave each VM 256MB of RAM, flipped a feature flag to stop processing checks on AWS Lambda, and let fly.io start picking up checks to queue instead.

It worked!

Sort of.

Messing around with RAM and finding out

It turns out if you overload a VM with too many jobs, some jobs will "stall" - be neither in the queue ready for work, nor actively worked on by the job runner.

Thankfully I built around this possibility already with my AWS Lambda stack (I requeue if a job hasn't run when we expected it to), and the code held up when permanently running in VMs.

From my customer's perspective, some of their uptime checks would just run every 10 minutes instead of every 5 minutes. While better than failing to run entirely, it wasn't a great outcome.

It took me about 30 minutes each morning over a week to fine tune the number of worker VMs, RAM allocation, and concurrency before working out I just needed a single VM with 512MB RAM.

The cost-benefit calculation

I've known this day was coming for about a year, ever since my first customer with over a thousand websites came in, smashed my AWS bill, and made me rethink my unlimited pricing.

Don't make paid resources unlimited, kids. Not even once.

What happened was, my AWS Lambda bill jumped to over $100 USD/mo, and I started thinking "well, surely a continuously running VM would be cheaper", and it is! I roughly worked out how much it'd cost me to rewrite a serverless app to continuously run in a VM (about a week, 2 hours per day before work), and sat on the idea for a year, since building features for my customers was a better use of my time.

What I got wrong in my initial calculation was not factoring in the freedom of not having to worry about a surge of usage (whether from free trial accounts, or broken code). With a continuous running VM, your pricing is capped by:

amount of RAM provisioned * number of instances * amount of CPU provisioned

whereas with AWS Lambda, your pricing is uncapped:

amount of RAM provisioned * number of milliseconds your function takes * number of invocations

With AWS Lambda, if you get a sudden rush of curious trial users from a viral blog post, or you've deployed a function that infinitely calls itself, suddenly you're on the hook for a huge bill (entirely bound by how lucky/unlucky you are).

When the same thing happens to a continuously running VM processing jobs in a queue, the wait times increase. If you screwed up the programming, maybe your VM runs out of memory, and reboots. The UX degrades, your monitoring picks it up and lets you know, and you scale up the VM to handle the load (for a couple of bucks a month more).

What I kept on AWS Lambda

I'm keeping several full replicas of my stack still sitting on AWS Lambda (in a few AWS regions), ready to turn back on if a certain number of jobs don't run within a given timeframe.

I also still run Google Chrome/Puppeteer for Browser Checks on AWS Lambda, but now I'm invoking them from my fly.io VM.

]]> <![CDATA[Guidelines for writing better runbooks]]> https://onlineornot.com/guidelines-for-writing-better-runbooks https://onlineornot.com/guidelines-for-writing-better-runbooks Tue, 14 Jun 2022 00:52:00 GMT It's 2am.

You're getting paged for the latest service your team built and deployed. You have no idea how to debug it, and it's the first time you've been paged for the service.

"No worries, I'll just go through the list of steps in the runbook, I'm sure it'll be fine... wait, runbook just says 'Ask Dave'?!"

You start paging the developers who built the service (especially Dave), and over a few anxious hours, you and the team manage to resolve the issue, and go back to bed.

In this article, we're going to improve our runbooks today, so no one in your team needs to experience the above scenario.

Table of Contents

Guidelines

These aren't in any particular order, and come from a few years experience working at startups, scale-ups, and enterprise.

Optimize for editing

The first runbook your team writes is probably going to suck, but that's okay!

Getting started is the first step towards being good at something.

You're going to want to put your runbooks somewhere easy to edit. While a git repo is fine, something like Notion, Confluence or Google Docs is going to be easier to update.

Note that an internal wiki/repo probably isn't the best idea - particularly if your team needs VPN/office network access to read the runbooks, and your VPN goes down.

Encourage newcomers to use your runbooks and make edits

As the authors of your runbooks, you're affected by what's known as the curse of knowledge - essentially, you assume people will have the background knowledge to understand what to do.

Newcomers to your team aren't yet affected by this, and can help you uncover assumed knowledge in your runbooks. Have them pair with your on-call team members as they resolve incidents - they'll get more comfortable with on-call before their first roster, and will notice undocumented gaps that your team just knows.

Keep track of how often you need to use your runbooks

As I mentioned earlier, while the initial goal is to document the manual process required to resolve an incident, a later goal is to automate those steps.

The more often you use a runbook, the more evidence you gather that perhaps parts of the runbook should be automated.

A simple table at the bottom of the runbook would help with this:

| Date last used | Used by | | -------------- | ------- | | 2022/01/01 | rozenmd | | ... | ... |

Explicitly tell the reader what to do

People shouldn't need to interpret what the steps in your runbook could mean. Write as though the person reading it just woke up at 2am, and just wants to go back to bed.

The steps should explicitly tell them what to do.

For example, instead of writing:

  1. Investigate issues in Splunk

a better runbook step would be:

  1. Login to the Splunk dashboard at URL
    • Run this query: "QUERY GOES HERE"
    • In case of weird results, run this command: "COMMAND HERE"
      • Weird results look like this: "SCREENSHOT GOES HERE"

Make your runbook easier to search

It helps to add key phrases from your alerts, or even the exact error message thrown by your system.

For example:

# How to fix: "Error: Query defined in resolvers, but not in schema"

1. If you see this error in environment X, you need to run this command:
   ...

It'll make it easier to find a solution to the exact problem the system is having (by including the searched phrase in your heading, you'll also game certain documentation search engines to rank the terms higher).

]]> <![CDATA[Writing your first runbooks]]> https://onlineornot.com/writing-your-first-runbooks https://onlineornot.com/writing-your-first-runbooks Tue, 14 Jun 2022 00:52:00 GMT Incidents are a stressful time for your team: your service isn't working the way you expect and your customers want to know what's going on. The last thing you want to do is let your team improvise everything when it comes to responding to incidents.

Google's own SRE book has great overall tips for incident management, part of which involves "develop(ing) and document(ing) your incident management procedures in advance", which this article dives into.

Table of Contents

What is a runbook

Depending on where you work, you'll hear different words describing relatively similar concepts around runbooks:

  • Standard Operating Procedure (or SOP): a standard set of steps required by the business to perform a process or task
  • Runbook: a set of steps required to respond to an incident or alert
  • Playbook: what we call the overall set of steps when responding to an incident. Can involve several runbooks

You shouldn't worry too much about terminology, just know we're going to be writing instructions on how to perform the manual process of resolving an incident with a system your team runs.

When to write your first runbook

Ideally, you would write a runbook as you're deploying a service for the first time. You would go through the manual process you take to resolve any issues that come up, and write down each step, in detail, so anyone on your team can follow it while half-asleep.

Realistically though, chances are you'll be writing your first runbook after something goes wrong. For a lot of organisations, it takes an incident to really examine the way you work and learn where the potential improvements are.

Most commonly, something will go wrong, it'll take your on-call engineer a few minutes to figure out what happened, page the team members that know what to do, and it'll take the team a few hours to resolve the issue.

During the post-incident review, someone will ask "why did it take so long to fix?", and one of the inevitable answers will be "we didn't have a runbook in place".

How to write a runbook

Honestly, anything is better than nothing. As time goes on, and your team gains operational experience, your runbooks will improve as gaps are found.

To start, you want to be documenting the manual steps necessary to get your service working again. You want to tell folks exactly what to do:

  1. Login to the Splunk dashboard at URL
    • Run this query: "QUERY GOES HERE"
    • In case of weird results, run this command: "COMMAND HERE"
      • Weird results look like this: "SCREENSHOT GOES HERE"

In other words, favour explicit steps over implicit steps.

If any of your runbook steps say "Do the usual troubleshooting procedure", you need to rewrite that step, following the advice above.

If any step of your runbook contains "If you need to do X, follow the procedure here...", you need to explain the decision making process that leads the reader towards figuring out if they need to do X.

Do not allow perfectionism to block you from releasing your runbooks

Depending on your organisation, you may have stakeholders that'll want every aspect of the system documented in the runbook before it's "ready".

Without feedback on using the runbook (such as during real or simulated incidents), you'll quickly start getting diminishing returns on the time you spend trying to perfect your runbooks.

You're better off spending that time resolving issues that cause common alerts to fire for your on-call engineers.

Use new team members as a starting point

New team members are a gift for getting started, and improving your runbooks. They aren't yet affected by the curse of knowledge, and tend to be curious about how things work.

Have them sit in on incidents, join incident response chat rooms, and have them write down any questions they have as the incident is worked on. The questions they come up with are perfect starting points for a runbook.

Do not worry about automation (yet)

Automating your runbooks comes later (chances are, it won't cover every single step of your runbook anyway).

To start with though, you're going to want to document the entire manual procedure before you begin to automate the easiest steps.

In short: write today, automate tomorrow.

If you're early in your incident response journey, your team isn't ready for automation. Your team likely has too much tribal knowledge, and needs to work on documenting their manual processes first.

]]> <![CDATA[What I learned running a SaaS for a year]]> https://onlineornot.com/what-learned-running-saas-for-year https://onlineornot.com/what-learned-running-saas-for-year Mon, 21 Feb 2022 06:52:00 GMT This time last year, I showed the internet a little prototype uptime checker I built using Next.js as the frontend, with services running on AWS Lambda. I gave myself one week to put it together.

Update: I've been running this business for over three years now, if you're interested to see how it's going, check out lessons from my third year running a SaaS .

The gist of my approach is as follows:

I started with a single Lambda function that checks if static websites were still online, added an email alert if it's offline, wrapped authentication around it, integrated Stripe, and shipped it. Throughout the year, I kept adding features.

My trick for launching into 200 competitors providing the "same" service and still getting customers?

  • I work two hours a day, every weekday on OnlineOrNot, and no other side projects. I managed to keep this going for about 10 months this year (with a cheeky 2 month burnout in the middle).

  • I focus particularly on features that solve customer pain (and I ask my customers what that pain is)

  • I'm ruthlessly iterative. If I can't get a feature done in two hours, I figure out how to cut scope down to a two hour block, and ship that. Then iterate on it.

Here's what I learned after a year:

Table of Contents

You're solving a problem, not selling a SaaS subscription

When building a product, it helps to think from the perspective of your customers, rather than your own goal of selling subscriptions.

This takes you from a mindset of "I'm just going to keep building features, they'll all come eventually!" to "I should be helping my users solve this annoying problem".

SaaS is just one of many ways to solve a problem. You should be looking at all the ways you could be helping, whether that's screencasts, docs, articles, books, workshops, code samples or software.

Docs are part of your user experience

People often say "Developers don't read documentation" - I think that's only partially true.

They don't read, they skim the headings.

In my experience, some people would try to figure out how to do things themselves in OnlineOrNot's UI, get frustrated, check the docs, and either one of two things would happen:

  1. They wouldn't find a way to do what they were trying to do, and they'd immediately churn
  2. They'd find the page they were looking for, jump to the heading, and do the thing

Successful use of OnlineOrNot's docs drove retention - so I treat it as part of the core product now, not as an afterthought.

Build for Mobile

Contrary to popular belief (for B2B SaaS), people actually work from their phones.

Something like 50% of traffic to OnlineOrNot.com comes from folks on mobile. They tend to quickly create an account, add a few pages to monitor, then eventually get on their laptop/desktop to review their checks from time to time.

For about 6 months I didn't support mobile well, and folks that signed up on their phone churned rapidly. I eventually took the time to build responsive views for mobile, and now new mobile users are sticking around.

Ask people how they found you

One of the most valuable code changes I made this year was asking people as they signed up: "How did you find out about OnlineOrNot?"

You need to know where your users are finding you.

There are dozens of channels you could be using to attract potential customers, and it's useful to know whether to double-down on paid ads, content marketing, or shitposting on Twitter to bring in users.

Use analytics, set-up funnel tracking

Your marketing funnel helps you work out the health of your business. While it's nice to see how many people visit individual pages, it's even better to see how people flow across multiple pages.

When I say marketing funnel, I mean tracking the flow of people that visit your home page, eventually making it to your sign-up form, and finally making it inside the actual product. It helps you diagnose problems with your marketing copy, the sign-up form itself, as well as any onboarding you may have before the user actually sees your app.

You don't need gross privacy-invasive tracking to achieve this by the way - I use a basic privacy-first web analytics service (and have no cookie banner as a result), and it works fine.

Sometimes you need to make your own mistakes

I read quite a few business books, mainly from not wanting to repeat mistakes that others have made.

Sometimes though, you need to make mistakes for yourself.

As an example - it took me getting on the front page of Hacker News, having 6000 people visit my landing page, a few hundred people attempt to sign-up, and only single-digits making it through to the app for me to realise something might be wrong.

I had something like a 75% drop-off rate on my sign-up form alone. With a bit of A/B testing, I got it down to 50% just by adding an extra OAuth login provider.

Pricing is bloody hard to get right

Price too high, and you'll have churn from folks who expect your app does everything. Too low, and you'll have customers that demand you rewrite your app just because they gave you $9. Refund the difficult customers, raise your prices, and move on.

Be prepared to experiment a lot with pricing.

You probably focus too much on MRR

Tracking your MRR is a pretty lousy way to measure how you're doing as a business early on.

Things you did weeks (if not months) ago will affect your MRR today, so you won't really know if pricing changes work until you've already got a decent number of customers going through different stages of their customer journey.

I find measuring your daily active users, or some sort of "success metric" for your customers (like pages checked, images generated, etc) is more helpful than MRR. It lets you figure out if people are actually using your product, and whether it's bringing them value.

You still need a free trial for paid tiers

While a free tier is a great way to attract people and get them talking about your product, you still need a way to let them sample "the good stuff", especially if the free tier is significantly less useful than your paid tiers.

It took me 11 months to realise I should probably build an onboarding flow, and start offering free trials. 95% of new users pick a free trial of the pro tier, even though I offer a free tier.

It's hard to bring in more traffic, easy to change what your current traffic does

Getting noticed on the internet is a long, slow game.

Eventually over months (if not years), if you're consistent at quality content marketing, the number of readers on your articles will grow from 1-2 a day, to a few hundred per day.

Increasing the number of people landing on your site isn't particularly easy.

On the other hand, what people do once they land on your site is entirely within your influence, and something you can change today (such as adding an additional OAuth login provider to your sign-up form, that I mentioned earlier).

Content marketing buys you time

Investing in content marketing gives you the option of letting the business run itself/coast for a while.

Throughout the year, I'd have an occasional article go viral and bring in tens of thousands of visitors over a month, if I did absolutely nothing, about 1500 people would still visit the site organically for the articles I had written.

OnlineOrNot's traffic

This was particularly useful for when it's a pandemic and I was having a bit of a burnout while moving across the world to live in France.

Ship small, ship often

People will suggest you should build particular features to improve your product.

They'll probably never use those features.

They're probably just trying to be helpful, and saw a similar feature in another product. Because you're new to running a SaaS, you'll be excited that people are actually talking to you, and rush out to build that feature for them.

I'm not going to tell you not to build the feature (that's the advice I was given, and I built unused features anyway). You should ask how they would use the feature, ask other customers how they deal with the problem, build the smallest possible version of that feature, and see how the rest of your customers use it. You don't want to be building snowflake features only one person uses.

It stings a lot less to remove a feature no one wanted after spending a few hours on it, rather than a few months.

Ship first, worry about scale later

In the first iteration of OnlineOrNot, I didn't optimise the architecture at all.

Each uptime check would manage its own database connection, meaning as more users found the service, the harder it would be for additional users to use the app. I also didn't bother making decent error states, so new users would see this while the database was busy:

Old OnlineOrNot Error Screen

Not a great look.

At the same time, I prefer being embarrassed by incomplete UI than building things people don't need. There was never a guarantee that OnlineOrNot would attract thousands of users, it could have ended up as another SaaS I built only for myself.

I ended up reworking the architecture to handle millions of checks per week on the smallest AWS RDS instance, as well as cleaning up that error screen:

New OnlineOrNot Error Screen

You don't get to spend as much time working on the problem as you'd think

Of the time I spend programming this year, I'd say half of my time went to actually solving the problem I wanted to solve (knowing if a site is down, and alerting folks when that happens). The other half went to building a SaaS platform around that problem.

SaaS platform things you didn't even realise you'd need, like multiple types of authentication and user management, trials, onboarding, team management and invoice management, lifecycle emails, and more.

You can outsource a lot of this (and I do! If Stripe didn't exist, I probably wouldn't be selling a service nor using subscription-based billing), but there's always stuff you don't feel comfortable outsourcing, or that you handle differently, so you need to build it yourself.

]]> <![CDATA[Communicating to Users During Incidents]]> https://onlineornot.com/communicating-users-incidents https://onlineornot.com/communicating-users-incidents Fri, 14 Jan 2022 06:52:00 GMT Imagine you're having a regular day at work, opening up your browser, double checking something for a client in that web app your team built for them, when suddenly, you see this screen:

500 Error example

You hit refresh a few times, just to be sure.

Nope. Still down.

What happens next depends on how well your team has planned for incidents like this (some folks call it unplanned downtime).

Incident Response and Communication

Incidents can be a frustrating time for everyone.

For you, as someone on the operational side of the app, it's frustrating because you're trying to fix the problem as fast as possible, with as little distraction as possible.

On the user side of things, their work day is interrupted, and without good communication, they're not sure if someone is currently working on the issue, or when it'll be fixed.

The Incident Communication Scale

You can think of communicating with your users during an incident as a scale.

On the left, there's "ignore your users", or under-communicate. The vast majority of organisations are on this side of the scale.

On the right, there's "send each of your users an SMS every few minutes to let them know how it's going", or over-communicate.

Both of these extremes are annoying - the sweet spot is somewhere in the middle.

While it's rare for a company to intentionally ignore their users during an incident, even waiting to confirm exactly what's happening before acknowledging an issue can be extremely frustrating for users.

A famous example of under-communication is AWS's Service Health Dashboard. AWS regularly takes over an hour to even acknowledge anything is wrong, let alone describe the issue and whether it's being worked on.

Reducing Frustration for Everyone

As I mentioned earlier, chances are, your organisation is on the left hand side of the scale: under-communicating with your users during incidents.

To reduce frustration for everyone, you're going to want to do a few things during your incidents:

As soon as you're aware of an incident, tell your users!

You'll want to let your users know something is not right, and someone is investigating.

If you're only working with internal users within your organisation, send an SMS, email or a Slack message letting your users know something is wrong, and that someone is looking into the issue.

If you're working with external users, you're going to want a dashboard to display your product's uptime or status (on separate infrastructure to your app!) that users know to look when things aren't working properly.

You might be wondering whether or not you should notify your users about incidents before you've confirmed there's a user-facing impact. As an idealist, I'd suggest all incidents should be posted on your status dashboard, as the cost of getting it wrong is your user's trust - but certain organisations value other things more than user trust.

Send regular updates

When working at a previous employer, we would send an update as often as every 30 minutes, depending on how critical the system was to the business. In this case, "We're still looking into it" was a completely valid update.

For some audiences (particularly if you have a status dashboard), this might be too often, and you'll need to tweak how often you communicate.

General rule of thumb: if your users are reaching out to you (via Phone call, SMS, Tweets, etc), you've probably left it too long without an update.

Have a dedicated communications person for incidents

It shouldn't be up to one person to simultaneously communicate with stakeholders AND fix the issue.

At the very least, you'll want a dedicated person in charge of communicating updates from the team fixing the issue to the outside world. Having a separate person here can help improve how long it takes to resolve the issue, as you spend less time context switching between "Oh no, need to fix the problem! aaaahhhh!" and "We are aware of the issue with [SERVICE], and are working to restore it ASAP. We will notify you once we have any updates".

In some organisations this role is called "Communications Engineer" or "Communications Manager" - either way, the person doesn't have to be technical, they just need to be capable of talking to engineers and stakeholders.

Summary

In short, once you're aware of an incident, tell your users!

Once you've told your users, send regular updates.

Finally, having a separate person in charge of communication, between the team fixing the issue and the rest of the business, can help reduce your Mean Time to Resolution by reducing context switching.

]]> <![CDATA[What we learned from AWS's us-east-1 outage]]> https://onlineornot.com/onlineornot-aws-outage-retrospective https://onlineornot.com/onlineornot-aws-outage-retrospective Wed, 08 Dec 2021 00:00:00 GMT In case you missed it, for several hours on December 7, 2021, AWS's us-east-1 region had an outage impacting multiple AWS APIs, taking out various websites across the internet.

According to our own monitoring at OnlineOrNot, the outage started at 2021-12-07 15:32 UTC and began to recover well at 2021-12-07 22:48 UTC (with minor signs of life for a few minutes around 2021-12-07 20:08 UTC).

Had we relied solely on AWS to update their status page before reacting, we would have been waiting a while. In fact, AWS took approximately an hour to update their status page to indicate anything was happening at all.

After things got serious, they added a banner to the top of their status page, explaining what was happening:

AWS Status Page

The retrospective

Some context

OnlineOrNot is an uptime checker service provides uptime and correctness checks for websites, web apps, and APIs. While our infrastructure is hosted across 20 AWS regions, a single point of failure was hosted in us-east-1.

What went wrong

OnlineOrNot's uptime check service stopped running regularly when us-east-1 started having issues.

Every 60 seconds, our service is kicked off via an AWS Cloudwatch event rule hosted in us-east-1.

When that event fires, a single AWS Lambda in us-east-1 queries our database to determine which URLs need checking, and sends off messages via SQS queues to trigger our checkers in 20 AWS regions.

What went well

Mostly by luck, the database was unaffected and our web app remained online. This let us quickly deploy a banner letting our users know something was wrong:

OnlineOrNot displaying its warning banner

What we learned

Since the outage we've implemented redundancy and the ability to failover to other AWS regions in our uptime check service.

On top of this, we'll be looking at duplicating more of our stack in other cloud providers to ensure this doesn't happen to OnlineOrNot again on AWS, nor any other cloud service.

]]> <![CDATA[Dealing with Noisy Error Monitoring]]> https://onlineornot.com/dealing-with-noisy-error-monitoring https://onlineornot.com/dealing-with-noisy-error-monitoring Wed, 01 Dec 2021 00:52:00 GMT Say you've been tasked with monitoring an application, so you set up some alerts to let you know when errors are coming in.

The minutes roll by, the errors start coming...

...and they don't stop coming...

Oh my, there seems to be quite a few errors coming through. Alerting on each error isn't going to help, better report on changes in the error rate instead right?

Not quite.

While there's no shortage of vendors that'll sell you on the benefits of error rate alerting, you need to get back to basics first.

Instrument your application

If you haven't already, you'll want to instrument your individual API endpoints.

For each endpoint, you'll want to be monitoring the requests, errors, and how long each call takes. Doing so makes it easier to find regressions in each endpoint when things do go wrong.

There are a few services out there for instrumenting your app, it's worth looking at either Sentry or Honeycomb.

For example, tracking down the cause of an increase in 4xx errors becomes a matter of comparing the current week's requests/errors/duration to the previous week's for each endpoint.

Clean up your alerts

Go through each type of error you're seeing that triggers an alert, and figure out what the appropriate response would be.

If the error isn't actionable, then it shouldn't create an alert. Have your developers change it to an INFO or a WARN log instead.

If the error IS actionable, and your application has already attempted to self-heal the issue (assuming self-healing is possible), THEN you should alert.

If the error IS actionable, BUT only after a certain threshold, you might be dealing with errors caused by a retry loop. Errors that occur as part of a sequence that eventually succeeds aren't real errors. If after several retry attempts the action still fails, that whole sequence of events should be counted as one error. Have your developers modify the error reporting accordingly.

As part of your team's on-call schedule, you should be documenting every alert that gets made, and actions taken to resolve each issue (and your alerts should provide instructions on what to do via a runbook). Common alerts should be prioritised and fixed by your developers.

Summary

If you're struggling with the signal to noise ratio of your company's alerting, you need to reflect on what the point of all of those alerts is.

An alert should mean someone needs to do something NOW.

It should not mean "Hey Jordan, check it out, CPU is at 69%!"

Another way to think of this is - if you're getting an alert for an error, it should be a priority to fix that error. If it isn't, what's the point?

]]> <![CDATA[Getting over on-call anxiety]]> https://onlineornot.com/getting-over-on-call-anxiety https://onlineornot.com/getting-over-on-call-anxiety Thu, 22 Jul 2021 00:52:00 GMT You've joined a company, or worked there a little while, and you've just now realised that you'll have to do on-call. You feel like you don't know much about how everything fits together, how are you supposed to fix it at 2am when you get paged?

So you're a little nervous. Understandable. Here are a few tips to help you become less nervous:

It's not your job to fix everything

The first thing you need to internalise to do well on-call is: it's not your job to fix every possible thing that could break.

Your job is to make sure that someone fixes the issue.

You can always page someone else with more expertise to fix the problem (ideally, once you've investigated the alert, and can provide the next person you page with context to get them started).

Use a different ringtone for your pages

It's pretty common to have a loud, scary "nuclear alarm" style ringtone for your on-call pages. Or worse, using the same ringtone as the rest of your calls.

You're going to want to use a different ringtone than the rest of your calls - it lets you know immediately whether you're going to need to focus, or if you're about to have a chat with your Mum. On top of that, it helps you avoid associating your phone ringing with anxiety - a fair chunk of the anxiety comes from not knowing whether the call is you being paged, or just another telemarketer.

I'd also recommend using a gradual ringtone that gets louder and louder - this is more of a personal preference, but I find the loud alarm style ringtones create an unproductive sense of panic that gradual ringtones don't.

Know that it'll get better with time

Like most things, on-call (should) get easier as time goes on. Eventually, you learn the processes (and help improve the documentation as you discover gaps), who to escalate to when things get really bad, and so on.

While the anxiety over getting paged never completely goes away, you gain confidence that you can solve any issue that arises, making the anxiety much easier to manage.

Also know that your team is there to support you - particularly your manager. Chances are, they've guided your colleagues through starting on-call as well, and can help address any concerns you may have about going on-call.

]]> <![CDATA[Self-hosting vs Managed Services: Deciding how to host your database]]> https://onlineornot.com/self-hosting-vs-managed-services-deciding-how-host-your-database https://onlineornot.com/self-hosting-vs-managed-services-deciding-how-host-your-database Thu, 08 Jul 2021 00:00:00 GMT Like all good things in infrastructure, picking whether or not to self-host your database is full of trade-offs.

On the one hand, you have the absolute freedom to do whatever it is you want with your database - whether it's adding a useful Postgres extension, or experimenting with new technologies. On the other hand, you now have to dedicate resources to keeping your database reliably online.

This article aims to provide an unbiased dive into the benefits of each side, as well as tasks that need doing regardless of what you decide to do, to help you decide whether or not it's worth self-hosting in your circumstances.

Note that while I mention certain services, it's purely out of familiarity - I'm not paid for mentioning them.

You might even decide a traditional database isn't worth the hassle, and opt for an entirely different approach (like MongoDB or CockroachDB) - though that's for another article.

Table of Contents

What I mean by self-hosting and managed services

Not everyone means the same thing when they say "self-hosting" and "managed services", so I just want to be clear on what this article is talking about.

Self-hosting

When I say self-hosting, I'm talking about running database software on VMs that you have control over. In other words, running the database on AWS EC2, Google Cloud Engine, or Azure Virtual Machines or a similar VPS provider.

While you could take the term further, and have "run your own physical hardware" fall under self-hosting, I have no intention of discussing that here.

Managed Services

When I say managed services, I'm talking about a "database-as-a-service" type platform, such as AWS RDS, Cloud SQL, or Azure Database.

Benefits of self-hosting

Price

One of the biggest factors driving folks to self-host their database is price.

At the low end, the cheapest tier for AWS RDS (the AWS managed database service) is around $15 USD per month. In comparison, installing PostgreSQL or MySQL on your application server is "free". In doing so, you lose the ability to scale parts of your application independently, but for smaller projects this is fine.

Once your application needs a bit more CPU and RAM, and you start looking for say, a 4 vCPU 16 GB RAM server (db.m4.xlarge on AWS RDS), assuming you've got 1TB of data, you're looking at around $266 USD per month on AWS RDS. Over on AWS EC2, that configuration (t4g.xlarge) would cost you only around $98 USD per month.

Portability

By self-hosting, you also minimise vendor lock-in. Say one day you notice that your 4 vCPU/16GB RAM/1TB storage configuration costs only $80 USD per month at VPS hosts like Hetzner, moving your self-hosted setup is as simple as running your setup scripts again and restoring from backup.

Of course, using a managed database service doesn't prevent you from using another service, it's just a lot easier when your database setup can be quickly spun up on any commodity Linux VM.

Control

Pick your own adventure! By self-hosting, you get to decide:

  • Which OS to run (and ensuring your OS is configured correctly, so your database comes back up after a restart)
  • How often to update/patch your OS
  • Which other software runs on the same server as your database
  • Which database extensions to run (for example, TimescaleDB took years to be supported by major managed service providers)
  • How often to upgrade/update/patch your database software
  • How often to run backups
  • Which disk configuration to use (to get more performance), as well as how to log (ensuring your logs go to the right place, so they don't fill up your data directory)
  • and more!

Benefits of managed services

Focus

With managed services, you pick the version of the database you want to run, your desired instance size, click "Create", and that's it.

Backups, minor updates, maintenance tasks and more are all run automatically, leaving you to focus on building your application. You do of course pay for the convenience of having these tasks automated, but it frees up your engineers to work on features of your product, rather than keeping the lights on.

If your core business isn't running a database, why are you wasting your focus on it, when it can be outsourced?

Scalability

One of the biggest benefits of managed services is the ease of scaling your database up or down at a moment's notice.

With the press of a button (and a couple of minutes of downtime, depending on how you've set up your database), you can go from the smallest database tier to whichever size you need for your workload, and back down again once you understand the resource requirements of your workload.

You also only pay for the resources you've used - so if you decide you need a significantly beefier machine for 12 hours, you only pay for those 12 hours.

Support

When you decide to self-host, the best free support you're likely to get is on mailing lists and forums. Whereas when you use a managed service, part of the cost covers basic access to dedicated support staff that specialise in your database.

While they won't be able to give you free bespoke advice on how to architect your application, they can assist with root cause analysis when incidents happen, and provide general "this is how most people use our databases" advice.

Blame-as-a-service

Especially in larger organisations, being able to assign blame to the vendor when things go wrong can save a lot of stress.

In some organisations this doesn't give you a "free pass" for the blame - after all if you decide to outsource to a managed service and it goes down, since you made the decision, you still get the blame for making the decision. At the same time, plausible deniability via "nobody got fired for buying IBM" is still a thing: if you picked a reputable party, how were you to know that they'd screw up?

(While I personally wouldn't use a managed service provider as a scapegoat, there are those that would, so it is worth mentioning.)

Things you'll have to do regardless of which option you pick

Whether you decide to self-host your database, or use a managed service, you'll still need to:

  • Secure access to your database
  • Monitor basic metrics like CPU usage, RAM usage, disk usage, etc as well as slow queries
  • Ensure your backups are running, and tested for recovery (as well as off-site backups if you want to be particularly strict about your data)
  • Major updates
    • While most managed services will perform minor updates for you during your scheduled maintenance window, you still need to handle major updates yourself

Summary

Are you a startup, or a business where you'd prefer all of your engineers to be working on features, rather than keeping a database operational? Comfortable with paying extra to make the problem go away?

Then managed database services such as AWS RDS might be right for you.

Are you an established business with engineers working at a steady pace (i.e. not moving fast and breaking things)? Looking to build something cost efficient in the long-term, that's optimised specifically for the needs of your application?

Then self-hosting with your own employees managing and operating the database may be a better choice.

If you're a large enterprise with a stable product that uses managed services, it might be worth reflecting on whether it's worth bringing your database in-house by self-hosting, as the cost savings can be significant.

]]> <![CDATA[What the Fastly outage can teach us about writing error messages]]> https://onlineornot.com/what-fastly-outage-can-teach-about-writing-error-messages https://onlineornot.com/what-fastly-outage-can-teach-about-writing-error-messages Wed, 09 Jun 2021 00:00:00 GMT In case you missed it, for about 15 minutes on June 8, 2021, Fastly's CDN had an outage, taking some of the internet's largest websites down (including the BBC, UK government, Reddit, and the New York Times - Amazon.com also had its CSS fail to load).

If you happened to visit those websites during Fastly's outage, you saw the relatively unhelpful error message below:

Unhelpful Fastly Error Message

As a frontend developer, my eyes scan error messages like these for numbers - in this case, the "503" - indicating that the error isn't my fault, and I can move on with my life.

Unfortunately the majority of internet users aren't trained in the art of reading HTTP status codes, so this error message wasn't particularly useful to them. Particularly when a solid portion of the error message was an in-joke.

We can write better error messages

The majority of internet users aren't developers, so just writing the error code and its name (503 Service Unavailable) just isn't good enough.

The Nielsen Norman Group (back in 1998!) provided us with some basic guiding principles for writing better error messages:

  • Write in plain English (or whichever language you're supporting)
  • Tell the user exactly what went wrong
  • Tell the user how the problem can be fixed

More concretely, we can write better error messages by answering the following four questions:

  1. Who caused the error?
  2. What happened, and why?
  3. When will it be fixed?
  4. How can the user respond to the error?

If your error message covers those four points, then you can think about adding humour and some brand identity.

Who caused the error?

The last thing you want to do is make your users feel dumb, or as though they're at fault for an issue with the service. Communicating who caused the error helps clear up any confusion.

Explicitly focus on "we" when the error is caused by an issue on your end (typically HTTP status codes in the 5xx range).

An error message such as

Our service is down for maintenance

is infinitely better than:

Uh-oh!

For errors caused by the user (typically HTTP status codes in the 4xx range), be explicit about that too. For example, a 403 Forbidden error (where you know the user isn't authorized to view content) could be communicated as:

Access Denied. You do not have permission to view this page.

What happened, and why?

While users may not be technical, they still need an explanation of why they're seeing your error screen.

Take the classic 404 error message: "404 Not Found". You can make it significantly better for non-technical users with a single word:

Page not found.

adding a "why", makes it even better, giving them a way to fix the issue:

Page not found. You might have mistyped the URL.

When will it be fixed?

It's relatively difficult to keep an error message updated with details of your outage, and when you expect the service to become available again.

A better approach would be to link to either your status page, or Twitter account, or both, as in GitHub's case:

An example of GitHub's service error page

How can the user respond to the error?

In the case of a 404, you might want to list some steps the user can take to fix the issue, such as:

  • Going back to the home page
  • Using your search bar to find the page if it's been moved
  • Contacting support

Whereas in the case of a 5xx server error, you want to communicate to the user that there isn't much they can do, and that it's not their fault.

My favourite example of a company doing this well is Airbnb:

An example of Airbnb's service error page

They tell users:

  • there's definitely an issue, and they're working on it,
  • to check out their Twitter account for updates,
  • a way to get support for urgent issues,
  • and they set the expectation that they may be slow to respond while the site is experiencing downtime

Summary

We as developers should take the Fastly outage as an opportunity to improve our error messages. You never know when your witty in-joke might end up being seen by hundreds of millions of internet users, so just try to be helpful, and explain:

  1. Who caused the error?
  2. What happened, and why?
  3. When will it be fixed?
  4. How can the user respond to the error?

Of course, if you're running a CDN like Fastly is, you'll likely still need request IDs and other diagnostics in your error message to help your support staff debug the issue. A human-readable error message doesn't have to come at the expense of removing all technical information.

]]> <![CDATA[Picking a domain name, and which country to host your website from]]> https://onlineornot.com/picking-domain-name-which-country-to-host-website-from https://onlineornot.com/picking-domain-name-which-country-to-host-website-from Mon, 24 May 2021 00:52:00 GMT No matter where you live, if your business targets a global audience, the question of where to host your website comes up pretty often.

Table of Contents

Picking a domain name

A common point of confusion here is your domain name.

I often hear people asking, "If I have a .com.au, do I have to keep my server in Australia?"

In short, the answer is no - It's very common to purchase domains from across the world, whether that's .co.nz (New Zealand), .es (Spain) or .kz (Kazakhstan), and host your content in a different country.

Side note: When to get a country domain name, and when to get a .com

If your business is local (think locksmith, plumber, gardener etc) - it might make sense to go for a country-specific domain name.

On the other hand, if you have a global audience go for a .com - for example: OnlineOrNot.com (this site!) while based in Australia with me in Sydney, our customers are from across the world, so a country-specific domain name wouldn't make sense.

Hosting a country domain name anywhere in the world

Back to the topic at hand - you might be wondering, "How is it possible to have a country domain name anywhere in the world?"

Without going too technical, think of a domain name like a record in an address book. Your customers look up your domain name, and then your domain registrar tells your customer's browser where the server is, and the browser starts loading your page.

Of course, if you have a server in Sydney, Australia, and customers from the US are trying to visit your site, it's going to be slow. Unless...

Best Practice: Use a CDN!

Essentially, a CDN (Content Delivery Network) makes a copy of your website available in thousands of servers across the world, so when your customers visit your website, the request doesn't have to travel across the world to reach your server. Instead, the requests only have to travel to your customer's nearest CDN location.

Using a CDN is as simple as updating the records in your domain registrar to point at your CDN, instead of your server. It's also one of the quickest and cheapest ways to significantly speed up your site.

I personally use CloudFlare for my own website (MaxRozen.com) - their three pricing tiers are free, $20 per month, and $200 per month. Compared to paying a developer $100 per hour to figure out why your website is slow, it's a pretty good deal!

]]> <![CDATA[How to Improve First Contentful Paint]]> https://onlineornot.com/how-to-improve-first-contentful-paint https://onlineornot.com/how-to-improve-first-contentful-paint Sun, 02 May 2021 00:52:00 GMT Chances are, you've run a PageSpeed Insights test and noticed "First Contentful Paint" as one of the first numbers in the report.

I've covered most of the metrics before in my article on Understanding the Page Speed Metrics in Google Lighthouse, but in this article I wanted to dive deeply into First Contentful Paint - particularly what it is, what a good score is, and how to improve.

Table of Contents

What is First Contentful Paint (FCP)

First Contentful Paint is a performance metric measured by Google Lighthouse and PageSpeed Insights. It measures the amount of time it takes the browser to show the first visible (as in, above the fold - the portion of the webpage visible without scrolling) block of text, or image (including SVGs) on your page. Iframes aren't counted when measuring FCP.

What's a good First Contentful Paint (FCP) score?

Google considers FCP values of:

  • less than 2 seconds as fast
  • greater than 2 seconds, but less than 4 seconds as moderate
  • greater than 4 seconds as slow

Ways to reduce First Contentful Paint (FCP)

Reduce Time To First Byte (TTFB)

In case you're not familiar, TTFB measures the amount of time your server takes to respond to the browser's request. Since First Contentful Paint includes the amount of time it takes the server to send us our data, improving TTFB will also improve our First Contentful Paint.

There are two ways to improve your TTFB:

  1. Reduce the amount of time spent on the server processing the request (this includes the time your server spends on database queries, API calls, and load balancing).

    • The simplest way to reduce this is to upgrade your server's RAM and CPU specs, or pick a better hosting provider.
    • Alternatively, look at optimising your backend code, though this is significantly easier said than done, particularly if using a CMS like WordPress or Drupal.

  2. Reduce the amount of time spent sending the request from the server to the browser

    • The simplest way to reduce this is to use a CDN, and enable GZIP or Brotli compression.

Resize your images

This an extremely simple, yet often overlooked step. Some frameworks like WordPress have plugins that do this automatically, but some plugins fail to do this well.

There are two steps:

  1. Open your image in macOS Preview or Microsoft Paint, resize the image, making it smaller. For example, if you have a photo that's 4000x3000, I'd make it 1000x750.
  2. Now that the image is physically smaller, run it through an image optimiser. I use TinyJPG for this, but there are hundreds of options.

These two steps alone will save you 67% or more.

Lazy load your images below the fold

In other words - ensure your above the fold images aren't being lazy loaded. All other content should be lazy loaded.

In WordPress, lazy loading was typically enabled via plugin - as of WordPress 5.4, all images are lazy loaded by default.

If you're using a React framework like Next.js, and using the next/image package to load your hero/above the fold images, ensure you pass the priority prop. Next/image uses lazy loading by default, so you could be taking a hit on your First Contentful Paint without realising it.

]]> <![CDATA[Why VPS Plans are Cheaper than Shared Hosting]]> https://onlineornot.com/why-vps-plans-cheaper-shared-hosting https://onlineornot.com/why-vps-plans-cheaper-shared-hosting Sat, 24 Apr 2021 00:52:00 GMT You might be looking around for a new web host, or trying to get a better deal. You notice VPS plans are incredibly cheap - as low as $3-5 per month, while the cheapest shared hosting is around $10 per month.

You wonder, "How is this possible?!" - especially when people recommend moving to a VPS once a site becomes popular.

Let's discuss what these are first, before going into why cheaper doesn't necessarily mean better for you and your business.

Table of Contents

What a VPS provides

VPS stands for Virtual Private Server. You typically get allocated a certain amount of CPU, RAM, Disk space, and bandwidth. Most providers let you pick the operating system, and you get admin access on that operating system.

Everything else, is up to you. This means:

  • Operating System updates
  • Security (managing non-root users)
  • Configuring, and updating the software you want to run (for WordPress, this also means configuring the database)

Access to your VPS is via SSH, and you use terminal commands to install what you need.

Essentially, you're only paying for exclusive access to the hardware, and bandwidth. The provider only ensures that your VPS stays up, and the network is working as expected.

What Shared Hosting provides

Shared Hosting provides you with pre-configured software, and some disk space for file storage. Depending on the provider, they'll also bundle in 24/7 Customer Support, CDN, SSL certificate, and domain management.

Access to your shared hosting is typically via some sort of admin panel, such as cPanel, and there's only so much configuration that's accessible to you as a user.

Hosting providers will rarely guarantee a certain amount of CPU/RAM. As a result, you'll find the more dodgy providers will attempt to run as many shared hosting plans on a single server as possible, while more reputable providers that have a reputation for fast and reliable shared hosting cap the number of sites running on a single server.

Why is a VPS cheaper than shared hosting

VPS plans offer limited customer support compared to shared hosting. As a result, a hosting provider is able to generate cash flow solely from keeping their hardware online.

In contrast, for shared hosting, a hosting provider also needs to pay for people to provide quality customer support, monitor the web servers and databases, and much more "keeping the lights on" type work. They typically pass on the additional costs, which is why shared hosting is more expensive than a VPS.

How to pick: VPS vs shared hosting

If you know your way around SSH, terminal commands, Linux in general, and don't want the provider's software to get in your way, chances are you'll want a VPS.

On the other hand, if you just want to get a website online, and are looking for a "set and forget" type solution, shared hosting is what you're looking for.

]]> <![CDATA[Understanding the Page Speed Metrics in Google Lighthouse]]> https://onlineornot.com/understanding-page-speed-metrics-google-lighthouse https://onlineornot.com/understanding-page-speed-metrics-google-lighthouse Sat, 17 Apr 2021 00:00:00 GMT If your experience is anything like mine, you're probably pretty confused about all the acronyms involved in checking your page speed.

Chances are you probably also want to know which metrics matter, and if your score is good or bad.

Table of Contents

Metrics in a Google Lighthouse report

As an example, let's check out the performance section of a Google Lighthouse v7.3.0 report:

Google Lighthouse Performance Result

It helps to think of page speed metrics in a few categories:

  • How long did the server take to start sending the page
    • Time to First Byte (TTFB), or Server Response Time falls under this category
  • How long did it take for the page to show certain things on the page
  • How long did it take for the page to start responding to user interactions
    • Time to Interactive (TTI) and Total Blocking Time (TBT) fall under this category

What do all of these metrics even measure?

What is Time to First Byte (TTFB)?

Time to First Byte measures the time between requesting a page, and getting the first byte of data back from a server. Time to First Byte used to be featured as a main metric tracked by Google Lighthouse. In more recent versions, it's featured as an audit named "Reduce initial server response time".

Google considers values greater than 600ms to be bad.

Some people don't worry as much about this metric since they believe that they can't influence it, but that's not necessarily true.

There are two places you can fix your TTFB:

Time spent on the server

Your backend server does have some influence on TTFB, with the main contributors being server-side rendering, database queries, API calls, load balancing, your actual app's code, the server's load itself (particularly if you're using cheap shared hosting).

You can improve your TTFB on your server by:

  • Upgrading your server's specs (more RAM, more CPU), or paying more for a better hosting provider
  • Upgrading your database's specs (assuming it's on a different server to your app server)
  • Optimizing your backend code

Time spent being transmitted

The time spent transferring data from the server to the browser is often overlooked, but there are some easy wins here.

If you're not using a CDN, every request to your website (including images, CSS and JavaScript), gets routed across the world to your server.

As mentioned in Ten Ways to Improve WordPress Page Speed, using a CDN gives you hundreds of little servers across the world close to your users, that massively reduce the time it takes to load your page.

What is First Contentful Paint (FCP)?

First Contentful Paint measures the time it takes the browser to load the first visible block of text, or image on your page.

Google considers FCP values of:

  • less than 2 seconds as fast
  • greater than 2 seconds, but less than 4 seconds as moderate
  • greater than 4 seconds as slow

If you have a particularly high FCP value, it might be worth double checking how fonts load on your page. You want to ensure your text is visible while your fonts load to minimize FCP.

What is Speed Index?

Speed Index measures how quickly visible content loads over time. As a metric, it rewards pages that provide a good user experience (that is, show visible content as soon as possible), and punishes pages that download all of their JavaScript and CSS before displaying anything.

Essentially this metric penalizes websites that load just a header and footer, then take 10 seconds to load content, while rewarding sites that load all content gradually over the same period of time.

As a user, you typically visit a webpage because you want to see its content. Visiting a website that loads its content slowly is generally a frustrating experience, and we're more likely to abandon a page that takes ages to show anything.

Google considers Speed Index values of:

  • less than 4.3 seconds as fast
  • greater than 4.3 seconds, but less than 5.8 seconds as moderate
  • greater than 5.8 seconds as slow

What is Largest Contentful Paint (LCP)?

In contrast with FCP, Largest Contentful Paint measures the time it takes the browser to load the largest visible block of text, or image on your page.

This metric can be dramatically different between mobile and desktop, depending on how much content you show when the page first loads.

Google considers LCP values of:

  • less than 2.5 seconds as fast
  • greater than 2.5 seconds, but less than 4 seconds as moderate
  • greater than 4 seconds as slow

What is Time to Interactive (TTI)?

Time to Interactive measures the time it takes for a webpage to be fully interactive, which Google defines as:

  • The page displays content
  • Event handlers are registered for most visible elements on the page
  • User interactions are responded to within 50ms

TTI penalizes pages that optimize for content visibility at the expense of interactivity. That is, you can see the content, but you can't scroll or clicking on things has no effect.

Sites with slow TTI often use Server-Side Rendering (SSR) - a page is sent to the user with complete HTML and CSS, but effectively no scripts. It looks loaded, but actually isn't - not until we finish downloading jQuery, and 2.5MB of other JavaScript dependencies.

Google considers TTI values of:

  • less than 3.8 seconds as fast
  • greater than 3.9 seconds, but less than 7.3 seconds as moderate
  • greater than 7.3 seconds as slow
]]> <![CDATA[Ten Ways to Improve WordPress Page Speed]]> https://onlineornot.com/ways-to-improve-page-speed https://onlineornot.com/ways-to-improve-page-speed Tue, 13 Apr 2021 00:52:00 GMT Introduction

Page Speed is a pretty big deal these days. As of May 2021, Google will start combining Core Web Vitals (how Google measures page speed) with other UX-related signals to rank your page. In other words, Page Speed impacts your SEO.

Since Google changed Googlebot's algorithm to highly favour fast, mobile-friendly websites, it has become more important to have a fast website. If that's not reason enough to start caring, users typically spend less time, and spend less money, the slower your website's experience is.

What is Page Speed

Page Speed is the amount of time it takes to completely load content on your webpage.

The metric we use at OnlineOrNot to measure page speed is mainly Largest Contentful Paint (it's also one of Google's Core Web Vitals). It's a fancy way of saying "the amount of time it takes your page to show the largest image, or block of text, when loading the page".

There could be dozens of reasons, for any given user, for why your page is slow. Your users could be on the train, passing through a tunnel with a weak signal, or their internet could just be slow.

By following best practices, we can at least mitigate the issue on our end, by ensuring we've done the best job we can.

10 Ways to Improve your Page Speed

Now that you know what it is, I'm going to teach you what you need to look at to speed up your page.

Note: these are listed in order of difficulty. At some point, you will need a developer to help optimise your site.

Table of Contents

#1 - Use a CDN

CDN stands for Content Delivery Network. Using a CDN effectively gives you access to hundreds of little servers across the world that host a copy of your site for you, massively reducing the time it takes to fetch your site. If you're not using a CDN, every request to your website (including images, CSS and JavaScript), gets routed across the world, slowly, to your server.

According to 468 million requests in the HTTPArchive, 48% were not served from a CDN. That's more than 224 million requests that could have been more than 50% faster, if they spent a few minutes adding a CDN to their site.

Be sure to check you've configured your CDN correctly - high rates of cache misses in your CDN mean the CDN has to ask your origin server for the resource more often, which kind of defeats the purpose of using a CDN in the first place!

#2 - Enable GZIP compression

First of all, you'll want to enable GZIP compression on your CDN (if it isn't already enabled).

On some CDNs, GZIP compression will just be a checkbox labelled "enable compression". Enabling compression will roughly half the size of the files your users need to download to use your website, your users will love you for it.

Once your CDN has compression enabled, look into enabling compression on your server. Some servers will do this automatically, other need plugins (such as WordPress's WP Rocket).

Assuming you have a CDN, enabling compression on your server will save you bandwidth costs between the CDN and your server, and make your site faster for requests that experience cache misses.

#3 - Use smaller images

This means both reducing the resolution (such as from 4000x3000 pixels your camera outputs to 1000x750 for the web), and reducing the size by compressing the file.

There are WordPress plugins that will do this automatically for you as you upload images. In particular, try these:

You don't necessarily need plugins, or WordPress to optimise your images though. I personally use TinyJPG to compress images as I write blog posts, and upload the optimised images directly.

#4 - Reduce the number of requests your page makes

The goal is to reduce the number of requests necessary to load the top part of your page (known as "above the fold content").

You have a few options here:

  • Reduce the number of requests on the page as a whole, by removing fancy animations, plugins, and images that don't improve the site's experience
  • Or, you can defer loading content that isn't a high priority through the use of lazy loading

#5 - Avoid redirects where possible

Redirects slow down your site considerably. Instead of having special subdomain for mobile users, use responsive CSS and serve your website from one domain.

Some redirects are unavoidable, such as www -> root domain or root domain -> www, but the majority of your traffic shouldn't be experiencing a redirect to view your site.

#6 - Reduce Time to First Byte

Time to First Byte or Server Response Time, is the amount of time your browser spends waiting after a request for a resource is made, to receive the first byte of data from the server.

There are two parts:

Time spent on the server

You can improve time spent on the server by optimising your server-side rendering, database queries, API calls, load balancing, your app's actual code, and the server's load itself (particularly if you're using cheap, shared web hosting - this will impact your site's performance).

Side note - using cheap hosting is a false economy. If your site handles any sort of eCommerce, using cheap, shared web hosting is costing you significantly in lost conversions. Any money you save (and realistically, it'd be $50 per month at most), you lose in missed sales from frustrated users of your site.

Time spent sending data

You can greatly reduce time spent sending data by following the earlier steps in this guide:

  • Use a CDN
  • Enable compression
  • Send as little data as possible (smaller images, less JavaScript/CSS)

#7 - Reduce and remove render blocking JavaScript

External scripts (particularly those used for marketing) will often be written poorly, and block your page from loading until it is finished running.

You can reduce this effect by marking external scripts async:

<script async src="..."></script>

You can also delay the loading of your marketing scripts until your users start scrolling:

window.addEventListener(
  'scroll',
  () =>
    setTimeout(() => {
      //insert marketing snippets here
    }, 1000),
  { once: true }
);

In general, try to avoid using plugins and javascript snippets to add functionality to your site (such as pop-up banners, social media buttons, calendars, etc). In most cases, they'll slow your entire site down for little benefit.

Hire someone to add the required functionality to your theme, or use a different theme.

#8 - Minify and combine your CSS and JS

Minifying means using tools to remove spaces, newline characters, and shortening your variable names. Typically this would be done automatically as part of your theme's build process.

As you add plugins to your installation however, you start to accumulate JavaScript and CSS files, slowing your site down.

If you're using Wordpress, you can use a plugin to combine and minify your JavaScript and CSS files. Two worth checking out are Autoptimize and W3 Total Cache.

It's worth noting that these are band-aid fixes at best. The best time to make your theme fast was while it was under development.

#9 - Remove unused CSS

Since Chrome 59 (released in April 2017), it's been possible to see unused JS and CSS in Chrome DevTools.

To see this, open the DevTools, show the console drawer (the annoying thing that appears when you hit Esc), click the three dots on the bottom left hand side, and open "Coverage".

Hitting the button with a reload icon will then refresh your page, and audit the CSS and JS for usage.

Here's what it looks like when you audit the starting page in Google Chrome: Unused CSS in Chrome

#10 - Regularly track your site's speed

It's much easier to fix problems with your site's speed within moments of slowing your site down. On top of that, if you make reviewing your site's speed a habit, it becomes a much smaller task to fix things that are slow.

There are free tools to check your website's current speed, two of the most popular being WebPageTest and Google Lighthouse. The downside to these tools is that you need to remember to run them before and after you make a change.

]]> <![CDATA[Guidelines for picking where to send monitoring alerts]]> https://onlineornot.com/guidelines-for-picking-where-send-monitoring-alerts https://onlineornot.com/guidelines-for-picking-where-send-monitoring-alerts Sun, 14 Mar 2021 00:52:00 GMT If you've ever had to be on the receiving end of a monitoring system that uses email for alerts, you know how noisy things can get. Particularly if you're working in an agency or freelance-like environment, with dozens of client sites to maintain.

You get so many emails that you start looking into integrations with third-party services like Zapier, and coming up with more and more complex rules to try reduce the noise, such as:

  • wait until a site has been down for 5 minutes before sending an email
  • wait until a site has been down for 30 minutes before forwarding the email to your helpdesk

The trouble isn't so much with email - it's a pretty decent way to receive and store important information - but rather that the alerts are for humans, and humans tend to have limited attention spans. Every alert you get eats up more and more of your attention span, and the lower your attention span, the more likely you are to miss the real "huge business impact" alerts.

So let's take a look at what type of alerts we should care about, and where to send them.

So what should you be alerting on?

Basically, you should send an alert for anything that needs to be responded to immediately, such as your site going down (and your business's ability to make money with it). These alerts should be able to wake an on-call person up, so they can fix the issue ASAP.

If you can't trust that your site is actually down when receiving an alert, perhaps it's time to re-evaluate your monitoring service.

Alert types, and where to send them

When was the last time you woke up for an email? It's not particularly common, so here are some suggestions for where your alerts should go.

Mike Julian's Practical Monitoring categorises alerts into three groups, depending on their use-case:

  • Immediate action required
    • Events that should trigger these kinds of alerts: your site being unreachable, users being unable to pay for products, SSL certificate expired.
    • These are the alerts that should be going to Phone/SMS/Pager, and should wake people up (if you respond to outages overnight).
  • Awareness needed, but immediate action not required
    • Events that should trigger these kinds of alerts: your database backup failed, warnings that your server is starting to run out of disk space
    • These are the types of alerts that should go to a team channel, such as Slack, Discord, or Microsoft Teams
    • You could also send these types of alerts to email, though it might rapidly get too noisy for a single person
  • Record for historical/diagnostic purposes
    • Events that should trigger these kinds of alerts: your system returning a 5xx error for a request, server timeouts
    • These are the types of alerts your system should be sending to a logging service.

Summary

Don't use email as a "catch-all" for your monitoring system's alerts. You should split your alerts by use-case, and send them to different locations:

  • Alerts that require immediate action should go to Phone/SMS/Pager
  • Alerts that just need awareness, but no immediate action should go to a team channel in Slack/Discord/Microsoft Teams
  • General system errors should be going to your logging service
]]> <![CDATA[Building a SaaS in one week: How I built OnlineOrNot]]> https://onlineornot.com/building-saas-in-one-week-how-built-onlineornot https://onlineornot.com/building-saas-in-one-week-how-built-onlineornot Sat, 06 Mar 2021 00:52:00 GMT When I first started building SaaS apps for side projects, it would take me a solid weekend solely to build the Stripe integration. These days, It's possible to build the whole SaaS app in one week.

Update: I've been running this business for over three years now, if you're interested to see how it's going, check out lessons from my third year running a SaaS.

Table of contents

Background Info

Let's start with some background about the app: OnlineOrNot is a pretty standard uptime checker - it does have some fancy logic to ensure the page is actually down before notifying you, but essentially it's a login/sign-up page, a dashboard, a "new page" form, a checker service, a transactional email integration, and a payment integration. I built a similar tool in 2018 (under the same name) to snapshot test GraphQL queries, but that project never took off.

I recently migrated my blog from Gatsby to Next.js, and that got me thinking - How much effort could it possibly be to use Next.js as the core of a SaaS?

I decided to do a quick test, and built a webpage to manually check if any URL is available, in a single afternoon (I later made it part of my landing page, before removing it entirely). I was impressed by how seamlessly Next.js blended together React, server-side rendering, and API routes - compared to my regular approach of using Gatsby.js as a landing page, create-react-app as the SaaS app, and a ton of AWS Lambda functions to handle the backend.

Tech Stack

Keeping in mind that I prefer to choose boring technology - I could have used DynamoDB/MongoDB to keep running costs as low as possible, but I opted for a relational data model.

Anyway, here's the tech stack:

  • Core app: Next.js, hosted on Vercel
  • UI/CSS framework: Tailwind CSS
  • Auth: NextAuth.js - a Next.js library
  • Backend: GraphQL + Node.js, written in TypeScript, sitting on a Next.js API route
  • Database: Postgres
  • Worker functions: Node.js/TypeScript, hosted on AWS Lambda
  • Payment integration: Stripe
  • Transactional Emails: Mailgun - I'd normally prefer to use Postmark, but didn't want to blocked from shipping by having to wait around to get verified
  • Newsletter Emails: Convertkit (referral link)

Approach

My work hours for my employer are 9am to 5pm, I tend to wake up at 7am, and go to bed around 11pm. If I gave 100% of my own time to this build, I would have had around 40 hours to work with.

In this particular week, I still either went to the gym, or ran each day, watched some Netflix, had dinner, and went out one night, so I had roughly 20 hours to work with.

Building the SaaS

First, a blog

To begin with, OnlineOrNot started its life as a blog. In particular, I cloned this example: blog-starter-typescript from the Next.js repo.

To get most of the blog functionality I wanted (sitemap, RSS feed, and code snippets in particular), I followed these steps.

Uptime Checker Test

With the blog running smoothly, I built an additional page to manually check uptime on my landing page (since removed).

To get technical with you, the form would submit data to a page living under status/[...url].tsx in my Next.js pages folder. Once it got a request, it sent off a request to an API route, which then sent the request to one of ten serverless functions deployed around the world (I had to use AWS Lambda for this part).

Seeing app-like functionality work so seamlessly on a blog is what inspired me to see how long it would take me to build a whole SaaS around the AWS Lambda function I wrote.

On Auth

For auth I initially wanted to use Passport, and try out Max Stoiber's recently released passport-magic-login, but I didn't enjoy the user experience of logging into an app with your email only, checking your email, clicking a link, then having to find the original tab to be able to use the app.

After a bit of searching, I came across NextAuth.js, which boasted similar functionality (and heaps more Auth Providers supported), with a bit more polish (to be fair, it's been around longer). It only took me a couple of hours to have both email-only login, and Google Auth0 login working.

Using Tailwind for UI

My typical approach to building UI rapidly would still be to use Bootstrap. In React, I opt for the Reactstrap library. It's never going to win me any design awards, but it helps me get the job done fast.

Since rewriting my own blog from Gatsby to Next.js, I've started writing my CSS using Tailwind utility classes. I find that whereas with Reactstrap I would use the default Bootstrap styles, then manually override each component, Tailwind has me thinking "How can I make this a component I can re-use?" more often, and it makes the code feel cleaner as a result.

To make my use of Tailwind CSS even faster, I paid for Tailwind UI, which has removed the "oh crap, where do I even start?" step for me when working on the frontend. It's kind of like paying for a designer to tell you "here's what screens like this tend to look like" - I highly recommend it.

On SSR and Emotion

One pitfall I noticed with server-side rendering (SSR) and Emotion, is that some of the Tailwind classes use the :first-child selector, and Emotion's SSR support (a CSS in JS library) extracts its CSS to be the first sibling of your component.

As a result, your pages tend to flicker on first load. The fix is simple enough - in Next.js you need to extract the critical styles into the head of the page (I followed the instructions here since I used twin.macro to use Tailwind with Emotion).

Forms!

I used to hate building forms in React until I came across React Hook Form.

Importing the hook, passing ref={register({ required: true })} to a few of my inputs, and implementing a submit handler is all it takes to build awesome forms in minutes these days (though it helps that I've spent a couple of years building forms in React).

OnlineOrNot Add Page Form

Once the user submits the form, they get to see their page being monitored:

OnlineOrNot Your Pages List

Latency troubles

Two days into building the SaaS with login/signup working, and it being possible to add pages to monitor, I decided to ship a v0.0.1 for friends to check out. I'm very glad I did, because I immediately noticed doing almost anything on the production build of the site took 3-6 seconds per click.

I eventually realised that the issue was caused by where I host my database. I typically host my Postgres database in AWS's ap-southeast-2 (Sydney, Australia) region, and Vercel hosts their functions somewhere on the US East Coast - probably in AWS's us-east-1 region.

So each request would hit their CDN, travel half-way across the world to the US, travel half-way across the world back to Sydney, Australia, and back to the US to finish the request.

Shutting down my Sydney database and spinning up a new one in AWS's us-east-1 region greatly reduced the latency issues.

Docs

Before launching, I knew I needed documentation - even though initially I was just building OnlineOrNot as a test to see how much effort is involved in building a SaaS in 2021, as a developer, nothing pisses me off more than crap/incomplete/missing documentation.

I followed the instructions at Docusaurus' homepage, spun up a quick site in Vercel, and mounted it onto the /docs path of OnlineOrNot - a pretty nifty Vercel feature that I loved from AWS CloudFront.

Stripe integration

Integrating Stripe's Checkout requires adding products in the Stripe dashboard, and following the docs - there are also examples to follow.

Since I implemented my last Stripe integration, Stripe launched Billing - greatly reducing the amount of effort required to let users upgrade, change, or cancel their subscriptions.

One of my employer's core values is "Don't f**k the customer", and I strongly identify with it, carrying it with me even when building side-projects. So I knew I had to have Stripe Billing to let my customers cancel at any time.

Rather than build the whole integration myself, I paid for Divjoy (referral link), which conveniently includes a Stripe integration (with Checkout, Billing, and a webhook endpoint). While I only wanted the Billing portion of the codebase, it was money well-spent - it gave me ideas to refactor the integration I built by following the Stripe docs.

Shipped!

With the Stripe integration built and tested, I deployed everything, and here we are - one week after I started toying around with a simple manual uptime checker in Next.js, I have a SaaS with login/sign-up, an app that tracks page uptime, and a way to pay for it (there's also a free tier).

Where to next?

Glad you asked!

I keep a changelog and a product roadmap tracking what's on the radar, what's coming soon, and what's released - I intend to continue shipping features for OnlineOrNot, and continue running it as part of my side-business.

]]> <![CDATA[Migrating between WordPress hosts without downtime]]> https://onlineornot.com/migrating-wordpress-hosts-without-downtime https://onlineornot.com/migrating-wordpress-hosts-without-downtime Fri, 26 Feb 2021 00:52:00 GMT Sometimes you'll want to migrate WordPress hosts - maybe it's time for renewal, and you found a better deal elsewhere, or your hosting provider isn't as reliable as they promised.

Which is great for you, but your site's readers don't care that it's a better deal - they just want to see your content. So minimising downtime when transferring hosts is a pretty big deal.

Let's learn how to avoid downtime.

Step 1: Transfer your content

The first step is to transfer your content to your new host, without turning off the old host. Your site on your old hosting provider doesn't have to be shutdown for you to start standing up your site on the new hosting provider.

Whether that's via a plugin, and manually copying your files across using FileZilla, or by getting a developer to copy your database to the new host, you need to have two copies of your site running to avoid downtime.

At the end of this step you should have two WordPress sites:

  1. The old one, which your domain is still pointing to (as in, visiting https://yoursite.com still shows your WordPress site on your old hosting provider)
  2. The new one, with a weird URL (depending on your new provider, this could be an IP address, or just a testing URL)

Step 2: Test everything on your new site

The last thing you want is to realise you forgot to copy your images after your site's users start visiting the new site.

Click around your new WordPress site, ensure images work, pages and posts are all there with the URLs you expect.

Once you're satisfied the new site works as you expect, you're ready to update your DNS

Step 3: Update your DNS settings

This step will be different depending on your domain registrar (the company you bought the domain from), but the gist of it is that you're changing the address your domain points to from your old hosting provider, to the new one.

This step can take up to 72 hours for some old school providers, and can be as quick as a few minutes with AWS or CloudFlare. The time it takes is due to the DNS TTL values you use - you can read more about that here. You can speed this step up by setting your TTL to a low value (like 300 seconds or 5 minutes) before you start the migration.

You can test whether the new hosting provider is live by creating a dummy blog post or page using the same URL you tested your new hosting provider with - if the post shows up at https://yoursite.com, your new site is live!

Once you're absolutely sure your domain is displaying the WordPress site from your new hosting provider, then you can turn off the WordPress site running at your old hosting provider.

How does this prevent downtime?

Essentially by not turning off either WordPress site, you entirely avoid downtime. When you switch the DNS records from the old hosting provider over to the new one, people visiting your site just start seeing the new site (since you don't turn off the old site until the migration is complete).

]]>