CVE-2026-98432 (CVSS 9.8): rebuilding a live server without losing it

White paper · Hosting operations · a 2-day project

How we moved every hosted site off a production server, wiped and reinstalled its operating system, and moved everything back, with one short maintenance window and no lost orders.

The bug: CVE-2026-98432, a Critical 9.8 Linux kernel flaw

CVE-2026-98432 is a use-after-free in the Linux kernel's IPv4 TCP code. An attacker who can reach any open TCP port, with no login and no password, sends a crafted connection request and can take over the server. It has been in the kernel since Linux 4.4, so almost every Linux server on the internet had it.

How fast we reacted

Two days from the first public exploit reports to every site on a patched kernel, and no lost orders. This is the white paper of how we did it.

Why we did it

Our main production server is a bare-metal machine with 125 GB of RAM. It is fast, it is paid for, and our hosting provider no longer offers that plan. If we gave it up, we could not get another one like it.

It was also running Debian 11, which had reached end of life. Then CVE-2026-98432 was published: a critical remote flaw in the Linux kernel's TCP stack, rated 9.8, that can be triggered before any login. Supported releases got a patched kernel. Debian 11 did not, and never would.

We had three options:

• Upgrade the OS in place, two major versions (11 → 12 → 13) on a live box serving 18 sites. If it fails halfway, every site is down and the server may not boot.
• Replace the server with a new one and lose the hardware we could not buy again.
• Do a round trip: rent a temporary server, move everything onto it, reinstall the original with a fresh OS, and move everything back.

We chose the round trip. It costs a few days of a second server and a lot of care, but at every step there is a working copy of every site, and a way back.

The temporary server stays on afterwards as a warm standby.

Phase 1, day 1: build a twin, then move the sites one at a time

The temporary server got a clean Debian 13 install and the same stack the sites expect: nginx 1.26 for TLS, Varnish 7.7 for page caching, PHP 8.3 and 8.4 (FPM), MySQL 8.4, Redis 8, OpenSearch 3.9, PostgreSQL 17 for analytics, and Node 20. The two machines talk over the provider's private network, so the large copies never touch the public internet.

A few decisions early on saved hours later:

• Same user and group IDs on both machines. The web user, the deploy user and the service users have identical numeric IDs, so copied files keep the right owners with no chown pass.
• Databases copied with their users. Each database went over as a mysqldump stream through the private network. Database users were recreated from their stored password hashes, so no site needed a password change. MySQL 8.4 renamed one privilege (SET_USER_ID became SET_ANY_DEFINER), and the grant scripts were rewritten to match.
• Search indexes moved as snapshots. OpenSearch took a snapshot to a local repository, the repository folder was copied, and the other side restored it, leaving out the system indices that already existed there.
• The old server forwards traffic. When a site moved, its virtual host on the old server was replaced with a proxy to the new one over the private network. Visitors whose DNS still pointed at the old address never noticed, and the new server still saw their real IP address.

Each site moved the same way: a short maintenance page, a final copy of its database and files, the forwarding switch, a DNS update through the registrar's API, and checks. Sites moved one by one through the day. The busiest store, with its admin panel and marketplace integrations, went last, after every other site had proven the process.

Phase 2, day 2 morning: back up everything, then wipe the original

With every site served from the temporary machine, the original only forwarded leftover traffic. Before anyone touched its disk, we made a final backup of everything a reinstall would erase: every database, plus archives of the system configuration, home directories, scripts, logs and application data. A checksum file was written next to them, and the copy on our local network storage was verified against those checksums.

Then came the reinstall itself, with the provider's "reinstall OS" option and Debian 13. The fresh system needed some cleanup before it was usable:

• The provider's first-boot setup wrote routes for the private network that sent traffic to a gateway that did not exist. We removed them and stopped that tool from managing the network again.
• SSH went to keys only. Root can log in only with a key, and an upload account is limited to SFTP with no shell.
• The firewall allowlist comes from a single file of trusted addresses. One command turns it into the web server's allow rules, the brute-force blocker's ignore list and the firewall rules, so the three can never disagree.
• The mail relay was listening publicly by default, so we bound it to localhost.

Then the same stack, at the same versions, went on as on the temporary server, followed by our security tooling: malware and webshell scanners, file-integrity monitoring, audit rules on what the web user executes, and log watching.

Phase 3, day 2 evening: rehearse the move back, then do it

The second move was the riskier one, because this time every site moves at once. So we rehearsed it while the sites stayed live on the temporary server:

1. A full copy of all site files (about 200 GB) to the rebuilt server.
2. A rehearsal copy of all 31 databases with their users and grants.
3. A search snapshot, restored on the rebuilt server (20 of 20 shards).
4. A test pass: four AI test agents ran in parallel, each covering one area (storefronts, non-shop sites, back-office tools, database users). Together they ran 114 checks comparing the two servers, and all of them passed.

Only after that did the cutover start. It followed a written runbook:

The checks afterwards: every site returned 200 from the rebuilt server, payment webhooks answered 200, and the marketplace listing sync ran with zero failures. Total maintenance time was about 26 minutes.

The temporary server becomes the standby

We kept the temporary server as a warm standby. It forwards any visitor whose DNS is still out of date. Every hour it receives a copy of the site files, and every night a copy of every database with its users. Its cron jobs, certificate renewal and file monitoring are switched off, so it never acts like production by accident. One command on either machine flips production back the other way.

What the checklist missed

A move like this always leaves something behind. Ours turned up in the first hours after cutover, mostly because cron emailed us the errors.

• Cron jobs that never start, and tell no one. Several jobs wrote their output to files in the system log folder, and those files were not part of any sync. When cron cannot open the log file in a >> file redirect, the job never runs. Two of them were order-sync jobs. We now check that every cron job's log file exists and that the job's user can write to it.
• Helper scripts outside the site folders. A cache guard and a couple of tools in the system's local program folders were missing on the rebuilt box. A plain file-list diff of those folders between the two machines found them in seconds.
• Permissions beyond owner and mode. One service reads a secret file through a filesystem ACL. The first sync did not copy ACLs, so the service could not read it. All syncs now use rsync -aHAX.
• Kernel tuning. The standby kept memory swapping tuned for databases (vm.swappiness=10), and the reinstall reset it to the default. Diff the kernel settings files as well.
• People's SSH keys. We rebuilt the server's own keys carefully but missed an administrator's personal keys, including the one their SSH app had generated for itself. Back up and restore every account's authorized_keys, not just the ones you know about.
• A wildcard in a shell loop. Our DNS loop over host names turned * into a list of local file names. The registrar API refused the bad names, so nothing broke. Turn globbing off (set -f) in any loop that handles DNS names.
• Old gaps, found while checking. Four www. addresses had never had an HTTPS certificate, and two certificates were for names nothing used any more. Neither had anything to do with the move. A full sweep turns up this kind of old problem too.

Monitoring that stays

The move left the server better watched than before:

• A log watcher runs every 5 minutes and emails only for real errors, with the same alert sent at most once an hour.
• A daily digest summarizes backups, scans and alerts in one email.
• File-integrity monitoring watches every site's code. Scanners check for webshells, malware and payment skimmers every few minutes.
• Audit rules record any command the web user runs.
• An hourly AI security review collects the hour's security log lines into one summary. An AI model with no tools and no access to the server reads it and returns OK or ALERT. We get an email only for an ALERT.

The result: the same hardware, now on a supported OS with a patched kernel, a warm standby, and better monitoring. The cutover took about 26 minutes of maintenance, and no orders were lost.

If you are planning one

1. Match user and group IDs before copying a single file.
2. Copy database users from their password hashes; do not reset passwords.
3. Have the old server forward traffic over a private network while DNS propagates.
4. Move sites one at a time on the way out. Rehearse everything on the way back.
5. Back up, verify the checksums, and only then wipe.
6. After cutover, diff the two machines: local programs, system configuration, enabled services, cron log targets, packages, kernel settings and SSH keys.
7. Keep the other machine as a standby, with one command to switch back.

Server names, addresses, domains and the provider are left out on purpose. All figures are from the actual project. Sources for the bug: the Debian security tracker and Red Hat bulletin RHSB-2026-016.

This is how we run the servers behind our managed hosting plans: patched, watched and backed up, with a way back at every step. See all our services, or tell us about your site.