SRE on Eric X. Liu's Personal Pagehttps://ericxliu.me/tags/sre/Recent content in SRE on Eric X. Liu's Personal PageHugoenSat, 12 Sep 2026 21:13:37 -0700A Gitea compromise hidden behind a failed Git synchttps://ericxliu.me/posts/gitea-compromise-postmortem/Sat, 12 Sep 2026 19:31:00 -0700https://ericxliu.me/posts/gitea-compromise-postmortem/<p>A notes repository stopped synchronizing. The cause was a cryptocurrency miner running under Gitea&rsquo;s service account. Killing it didn&rsquo;t help for long: the attacker had changed global Git configuration so that an ordinary fetch could run a script and start it again.</p> <p>Gitea and CI went offline for a rebuild. Over September 11-12, recovery restored 301 repositories, object-backed data, SSO, and representative CI workflows. The response also deleted a Coder workspace&rsquo;s home volume. That data was not recovered. Services returned in stages; there isn&rsquo;t a reliable measurement of total downtime.</p>