Newsletter
TechAnV Blog
Get updates on security engineering, Rust, eBPF, and DevSecOps. No spam, unsubscribe anytime.
Check your inbox and click the confirmation link to complete your subscription.
Rewriting a Git repo to remove secrets from the history#
I decided to make a GitHub repository public today that had previously been private. Unfortunately the revision history of that repository included some secret values, one of which I could not figure out a way to revoke.
I found a way to rewrite the entire repository from scratch to omit those values, using BFG Repo-Cleaner by Roberto Tyley. It’s a tool that is recommended on the GitHub help page.
Installing the tool#
BFG is a Java app. My Mac had Java installed, so installing it was just a case of downloading the latest .jar file from the download link on the website.
Running java -jar bfg-1.14.0.jar confirmed that this worked:
1% java -jar bfg-1.14.0.jar2bfg 1.14.03Usage: bfg [options] [<repo>]4...Removing the secrets#
I made a list of secret strings that I wanted to remove and saved them in a file called secrets.txt.
Then I followed the BFG instructions to create a fresh clone of my repository:
1cd /tmp2git clone ~/Dropbox/Development/my-repo --mirrorThis gave me a /tmp/my-repo.git folder containing the git history but not any actually checked out files.
Then I ran this:
1cd /tmp/my-repo.git2java -jar /tmp/bfg-1.14.0.jar --replace-text /tmp/secrets.txtI got the following output:
1Using repo : /private/tmp/my-repo.git2
3Found 11 objects to protect4Found 3 commit-pointing refs : HEAD, refs/heads/main, refs/remotes/origin/main5
6Protected commits7-----------------8
9These are your protected commits, and so their contents will NOT be altered:10
11 * commit 6badd000 (protected by 'HEAD')12
13Cleaning14--------15
16Found 11 commits17Cleaning commits: 100% (11/11)18Cleaning commits completed in 98 ms.19
20Updating 2 Refs21---------------22
23 Ref Before After24 ----------------------------------------------25 refs/heads/main | 6badd000 | 15881e3b26 refs/remotes/origin/main | 4af64070 | d404ebd027
28Updating references: 100% (2/2)29...Ref update completed in 35 ms.30
31Commit Tree-Dirt History32------------------------33
34 Earliest Latest35 | |36 D D D D DD D D D D m37
38 D = dirty commits (file tree fixed)39 m = modified commits (commit message or parents changed)40 . = clean commits (no changes to file tree)41
42 Before After43 -------------------------------------------44 First modified commit | ff14c49c | be20428445 Last dirty commit | 4af64070 | d404ebd046
47Changed files48-------------49
50 Filename Before & After51 -----------------------------------------------------------52 __init__.py | d5936659 ⇒ 121d236c, d96951be ⇒ ac125def, ...53
54
55In total, 30 object ids were changed. Full details are logged here:56
57 /private/tmp/my-repo.git.bfg-report/2023-01-24/17-11-5558
59BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressiveAnd this had done the trick!
All of my commit hashes had changed, which was expected for a rewrite of the repository.
I knew that my first commit to the repo had a secret in it - so I ran git log --pretty=oneline | tail -n 1 to find that first commit hash, then ran git show be20428438bf42452f1a783d7b00dad84effc002 to see the commit… and sure enough it had this in it:
1<p><label>JWT: <input name="jwt" value="***REMOVED***"></label></p>That ***REMOVED*** used to be one of the secrets that I had wanted to removed.
Final steps#
As suggested in the output from the command, I ran this:
1git reflog expire --expire=now --all && git gc --prune=nowI didn’t want to risk anything persisting on GitHub, so I deleted the entire repository and then created a new one with the same name. Then I pushed my repo to that:
1git remote remove origin2git remote add origin git@github.com:simonw/my-repo.git3git push -u origin mainThis was likely unnecessary: the GitHub documentation says that a force push should remove all traces of the old commits.