A Gitea compromise hidden behind a failed Git sync

A notes repository stopped synchronizing. The cause was a cryptocurrency miner running under Gitea’s service account. Killing it didn’t help for long: the attacker had changed global Git configuration so that an ordinary fetch could run a script and start it again.

Gitea and CI went offline for a rebuild. Over September 11-12, recovery restored 301 repositories, object-backed data, SSO, and representative CI workflows. The response also deleted a Coder workspace’s home volume. That data was not recovered. Services returned in stages; there isn’t a reliable measurement of total downtime.

From public markup to code execution Link to heading

The server ran Gitea 1.26.1, affected by CVE-2026-59774. The vulnerability was patched in 1.27.1.

The exploit starts at a public repository’s markup endpoint. The vulnerable Org-mode renderer accepts file inclusion outside the repository. Reading app.ini can expose INTERNAL_TOKEN, which authorizes internal logger operations. Those operations can then be abused to inject Git configuration and execute code as the Gitea service account. No repository write permission is needed. Gitea’s advisory describes the chain.

The archived application log records these requests against eric/docker-getting-started on September 10:

That’s 12:13:53-12:13:58 PM PDT. Similar sequences appear on September 11.

The files supplied the other half of the evidence. Global Git configuration contained uploadpack.packObjectsHook entries pointing to attacker-controlled scripts. This setting normally lets an administrator customize pack generation; here it made a fetch an execution trigger. Router-log text embedded in the altered files matched the logger-injection mechanism. The September scripts downloaded and supervised a binary identical to the official XMRig miner.

The request sequence and file contents strongly support this CVE as the entry route. The initial disclosure response wasn’t saved, so token theft through that response remains inferred. Mining was confirmed, but the available evidence cannot establish that it was the attacker’s only activity.

A new container image alone would have left the hook on disk. The rebuild had to replace the Git configuration too.

The snapshots showed an earlier repair Link to heading

September 10 was the earliest matching request sequence in the retained logs. It wasn’t the beginning of the disk evidence.

August 21-23 snapshots contained a saved .gitconfig.corrupt-20260812 with 46 execution-hook settings and an altered repository hook invoking an unknown temporary payload. Those files existed by August 21, regardless of the date in the backup filename.

The active configuration told a different story. In every snapshot, it matched the saved copy exactly except that all 46 hook settings had been removed. The altered hook file was still there.

Someone appears to have repaired the configuration earlier. File metadata points to August 12 Pacific time, but we haven’t identified who performed the repair. An agent or administrator could have created the backup while cleaning up the file. Calling that backup malicious based on its name would have missed the distinction between preserving injected content and keeping it active.

Earlier snapshots didn’t provide a readable clean baseline. The collected Gitea logs covered September 3-11; the saved ingress capture covered September 11. We can place the injected files before the September response, but cannot date the first compromise. The older payload was not recovered.

Recovery deleted a workspace Link to heading

During the Docker TLS migration, an agent updated a Coder template. The repository version used a different storage class from the running workspace. Kubernetes cannot change that field on an existing PVC, so the update replaced the claim. The old volume’s reclaim policy then deleted its Ceph image.

There was no verified recovery point. The workspace’s home directory remains lost.

This was response-caused damage. A transport change picked up an unrelated storage change, and the execution path had no gate to stop it. Before a stateful template update, the plan needs to identify volume replacement explicitly and require approval backed by a tested recovery point.

The response also created an extra runner VM without approval. It was later removed. CI ultimately used a Kubernetes runner and the existing Docker host over mTLS. That authenticates access to the daemon, but a trusted runner still has broad authority over it. The incident did not establish Kubernetes as the entry point or justify another VM as a prerequisite for recovery.

A valid archive wasn’t enough Link to heading

Hermes, the personal-assistant environment, recovered from an August 17 backup, nearly four weeks old. Its archive checksum passed. A SQLite search index inside it was malformed.

Database checks caught the defect, and rebuilding the index preserved the messages. A fresh backup later passed full S3 readback verification. The first attempt exceeded the pod’s temporary-storage limit and evicted Hermes before scratch storage was corrected.

The backup checks now cover database consistency and downloaded archive integrity. Freshness needs attention too: a configured nightly job had not produced the recent backup we expected.

Service acceptance had the same problem. Git, SSO, object storage, and CI each needed functional tests. Restoring them piecemeal prolonged disruption after the web application was already reachable.

What changes after this incident Link to heading

The completed work includes a clean Gitea deployment, regenerated Git configuration, Docker mTLS, and a verified fresh Hermes backup. The next prevention work is specific:

• Test that internal endpoints remain inaccessible publicly after ingress changes, and track critical Gitea advisories.
• Alert on unexpected Git execution settings and retain security logs outside the application’s write access.
• Block unapproved volume replacement and infrastructure creation during recovery.
• Alert on stale backups and rehearse restoration through database checks and working application requests.

These are follow-up priorities, not claims that all four controls are deployed.

The incident closed on September 12 at 19:31 PDT. The owner accepted the lost workspace and remaining credential risks. The recovery record also documents the work explicitly waived at closure.

The most expensive mistake was in the response: a Docker transport update deleted unrelated persistent data. The next recovery needs a hard stop at that boundary.

This post follows Google’s example postmortem and blameless postmortem guidance.