Skip to content

Nginx fails to start after reboot due to DNS resolution of ap.ghost.org in config #26091

@sebastienfouss

Description

@sebastienfouss

Issue Summary

Ghost's ActivityPub nginx configuration uses a hardcoded external hostname (ap.ghost.org) in the proxy_pass directive. This causes nginx to fail startup if DNS isn't fully available yet — which happens during reboots or systemd daemon-reexec events.

What happened:
Unattended-upgrades triggered a systemd daemon-reexec. During the service restart sequence, systemd-resolved was stopped momentarily before nginx attempted to start. Nginx failed its config test because it couldn't resolve ap.ghost.org:
nginx: [emerg] host not found in upstream "ap.ghost.org" in /etc/nginx/sites-enabled/example.com-ssl.conf:24
The site remained down until manual intervention.

Root cause:
Nginx resolves hostnames in proxy_pass directives at startup, not at runtime. If DNS is unavailable at that moment, nginx refuses to start entirely.

Suggested fix:
Use a variable to force runtime DNS resolution:
location /.ghost/activitypub/ {
resolver 1.1.1.1 8.8.8.8 valid=300s;
resolver_timeout 5s;
set $ghost_ap https://ap.ghost.org;
proxy_pass $ghost_ap;
# ... rest of config
}

This pattern is the standard nginx workaround for external upstreams and ensures the service starts reliably regardless of DNS timing.

Steps to Reproduce

  1. Install Ghost with ghost-cli on Ubuntu 24.04
  2. Enable ActivityPub (Social features) in Ghost settings
  3. Confirm nginx config contains proxy_pass https://ap.ghost.org in the ActivityPub location blocks
  4. Trigger a systemd daemon-reexec or simulate DNS unavailability during nginx startup:
    sudo systemctl stop systemd-resolved && sudo systemctl restart nginx

Alternatively: Just wait — unattended-upgrades or any package that triggers systemctl daemon-reexec will eventually cause this on a long-running server.
In my case, the server had been running flawlessly for 3 months until an apt-daily-upgrade triggered a daemon-reexec at 06:11 UTC, which restarted services in an order where DNS wasn't available when nginx tried to start.

Ghost Version

Ghost 6.3.1 with ActivityPub enabled

Node.js Version

v18.19.1

How did you install Ghost?

Ubuntu 24.04, nginx installed via ghost-cli

Database type

MySQL 5.7

Browser & OS version

No response

Relevant log / error output

Code of Conduct

  • I agree to be friendly and polite to people in this repository

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs:triage[triage] this needs to be triaged by the Ghost team

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions