The Cybersecurity and Infrastructure Security Agency (CISA)
published a blog post sharing lessons from an
incident in May when code and credentials belonging to the agency were leaked in a contractor’s public GitHub repository.
The GitHub repository, discovered by GitGuardian, was titled “Private-CISA” and was owned by an employee at Nightwing, a company that provides security and intelligence services to government agencies.
The incident was first
publicly reported by Brian Krebs of KrebsonSecurity, who was contacted by GitGuardian Staff Cybersecurity Researcher Guillaume Valadon after a lack of response from the repository’s owner. Krebs then contacted CISA, leading to the repository’s removal.
CISA’s after-action blog post, published Thursday, confirmed that a public repository containing CISA’s infrastructure as code, build code, AWS GovCloud keys and additional secrets was published to the contractor’s personal GitHub account rather than CISA’s official GitHub.
According to Krebs, the repo reportedly contained plaintext usernames and passwords for “dozens of internal CISA systems,” administrator credentials for three AWS GovCloud servers, internal CISA and DHS files and more. Additionally, logs showed the repo owner had disabled GitHub’s default secret scanning feature for public repos.
“Secrets in public repositories is one of the oldest mistakes in the industry, and the fact that it reached CISA’s contractor is a reminder that process failures don’t respect reputation. The fix is mechanical and well-understood. Pre-commit hooks, automated scanning at the pipeline level, and credential rotation as a drill rather than an emergency,” Seemant Sehgal, founder & CEO at Breachlock, said in comments to SC Media.
CISA said the contracted employee who maintained the repo “uploaded copies of a CISA build and deployment repository to their personal GitHub account for the purpose of creating cloud infrastructure autonomously” and also copied admin and build credentials to the repo.
Once notified of the leak by Krebs, CISA immediately removed the repository while saving a copy for analysis, took its development environment offline until all associated credentials could be reset, and removed system access to the individual behind the leak, according to Thursday’s blog post.
To remediate the incident, CISA reset all exposed credentials as well as all credentials across all environments where the repo owner had been an administrator. No evidence was found of the leaked credentials being used outside of CISA’s environments, and no customer or mission data was compromised, the agency said.
“Teams must deploy continuous, automated secret scanning across all corporate and third-party adjacent repositories, implement short-lived token architectures to drastically limit blast radiuses, and mandate rigorous identity monitoring for any administrative access keys,” said Noelle Murata, senior security engineer at Xcape, in an email to SC Media.
“Trusting a contractor with your production infrastructure keys without automated oversight is just an incident report waiting for a date,” Murata added.
CISA improves secrets monitoring, incident reporting, response playbook after leak
The postmortem acknowledged weaknesses in CISA’s systems and processes revealed by the incident, including inadequate controls on public repo uploads and private repo secrets, the lack of a clear incident reporting mechanism for the researcher to use, and the absence of an incident response playbook for GitHub and cloud-related incidents, forcing the agency to build one during the response itself.
As a result, the agency made several changes in the wake of the GitHub leak. The allow and deny lists for its repositories were retuned, users’ ability to upload to public repositories was limited and CISA’s endpoint detection and response (EDR) solution was engaged to monitor and manage future uploads, allowing code to be pulled from public repos while preventing the public upload of sensitive content.
CISA’s private repositories were also found to contain secrets, which were removed and rotated, and an action plan was created to monitor secret exposure across all repos. Additionally, CISA had already begun consolidating developer environments before the incident to ensure better oversight and consistent security controls and has continued to advance such efforts after the leak.
The need for GitGuardian to contact KrebsonSecurity in order to report the incident was also acknowledged, with the agency revealing that the researcher both attempted to email the contractor and submit the incident through the vulnerability disclosure program due to a lack of a well-defined communication channel for such incidents. As a result, CISA has committed to refining and clarifying reporting avenues for researchers to report future incidents involving CISA assets.
Building comprehensive incident response playbooks that cover the gamut of possible scenarios, including GitHub-related incidents and cloud key exposures, was noted as critical to support a rapid response, and CISA also encouraged organizations to fine-tune playbooks using lessons for every response.
Additionally, CISA noted that its key rotation process during the incident was slowed down due to the complexity and interconnectedness of its systems with both government and industry partners, emphasizing the importance of cryptographic key agility to remediate such exposures.
“CISA’s postmortem recommends secret scanning, credential rotation, and better playbooks. Missing from that list: mandatory third-party risk assessments and ongoing verification that contractors actually follow security requirements,” noted Jacob Krell, senior director of secure AI solutions & cybersecurity at Suzu Labs, in comments to SC Media. “The contractor who caused this incident disabled the exact scanning CISA now recommends, and proper vetting would have caught it.”
CISA’s postmortem also highlighted “what worked well,” including its quick response once informed of the leak and productive collaboration with the researcher, its application of granular Zero Trust principles allowing for stronger visibility and ability to contain the incident, and its strong logging capabilities enabling a thorough investigation, with additional logging opportunities identified and implemented after the incident.
The blog concluded by highlighting the importance of transparency and lesson sharing after incidents, strengthening both trust and knowledge across the cybersecurity community.
“It is not a matter of ‘if,’ but ‘when’ a cybersecurity incident will happen to your organization. It is important to the broader cybersecurity community that we address these matters openly to strengthen trust and foster transparency. Such transparency unlocks opportunities for learning that will enhance not only CISA’s security posture but that of other organizations as well,” agency officials wrote.
John Strand, owner of Black Hills Information Security, told SC Media the biggest lesson from CISA’s GitHub leak is the importance of identity security.
“Whatever happened to CISA can happen to any organization. We need to start mapping credentials like we map IP addresses and software. Identity is the new perimeter,” Strand wrote in an email.
