this post was submitted on 01 Mar 2024
14 points (100.0% liked)

commandline

1761 readers
1 users here now

founded 1 year ago
MODERATORS
 

After an unexpected need to reset my work machine ๐Ÿ˜“, and needing to set up my development environment again by hand ๐Ÿ› ๏ธ, I decided to create a solution to quickly restore my local git repositories (and associated folder structure) ๐Ÿ”„.

I took this opportunity to write two bash scripts that clone and update all repositories on GitHub belonging to either a user or an organization ๐Ÿ“ฆ.

This means that, for example, with a single command โŒจ๏ธ, you can clone hundreds or thousands of repositories, with high levels of concurrency (50 clones in parallel is doable ๐Ÿ’จ).

The scripts allow for a configurable clone depth, a limit for the number of repositories cloned, and a level of concurrency that decides how many clones are run in parallel ๐Ÿ“ˆ.

By running the following command:

git-clone-all --owner f3rno64 --limit 200 --jobs 40 --dir ./f3rno64

I was able to clone all 174 personal repositories ๐Ÿ“š, with full commit histories and all tags & branches, in 58 seconds โฑ๏ธ.

I wrote a blog post describing this in more detail here ๐Ÿ“, check it out for a breakdown of the arguments and examples of usage.

The GitHub repository is f3rno64/mass-git-scripts and the README also includes examples and general usage instructions ๐Ÿ—‚๏ธ.

Please check it out and let me know what you think! ๐Ÿ’ฌ

I hope you find it useful, and any feedback or suggestions for improvement would be greatly appreciated! ๐Ÿ™

you are viewing a single comment's thread
view the rest of the comments
[โ€“] [email protected] 2 points 8 months ago (1 children)

Bit of a protip if you're interested. If you wrap the scripts in a subshell (that is, ( and ) at the beginning and end) you don't need to unset the variables you exported.

[โ€“] f3rno64 2 points 8 months ago

Thanks for the tip! I will do so ๐Ÿ™‡โ€โ™‚๏ธ