this post was submitted on 02 Jul 2023
128 points (99.2% liked)

Git

2920 readers
1 users here now

Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency.

Resources

Rules

  1. Follow programming.dev rules
  2. Be excellent to each other, no hostility towards users for any reason
  3. No spam of tools/companies/advertisements. It’s OK to post your own stuff part of the time, but the primary use of the community should not be self-promotion.

Git Logo by Jason Long is licensed under the Creative Commons Attribution 3.0 Unported License.

founded 2 years ago
MODERATORS
 

Hey folks,

This is an update to: https://sh.itjust.works/post/580838

On https://github.com/csm10495/commit-ment, I made it to somewhere around 22 million commits. I can't imagine this is not a world record for commits to a single branch. In the middle of the night, I got an email from GitHub support saying:

A few minutes later, I got another email like so:

I've asked them if the two emails are related (and I guess if the first one is some sort of error since there was no personal info in that repo). I've also asked if they can give any information about what triggered the email and if they can give me more info about what it looks look on their side.

I've also asked if they can re-enable it so I can give one more commit to say the final results on the readme then (public) archive it.

We'll see what they say.

Doing a pull is interesting at the moment, it shows:

git pull origin master --no-rebase -vvv
ERROR: Access to this repository has been disabled by GitHub staff due to
excessive resource use. Please contact support via
https://support.github.com/contact to restore access to this repository.
Read about how to decrease the size of your repository:
  https://docs.github.com/articles/what-is-my-disk-quota

fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Similar thing happens if you try to clone: [email protected]:csm10495/commit-ment.git

So yeah, I figured this would happen sooner or later. I just hope they can tell me a bit more about what it looks like on their side since managing this repo on my box is a pain, I can't imagine what it could look like on theirs. I'm also curious how pull requests could merge at such a rate given that just doing a pull on my end could take minutes. So many questions!

This whole project was really just for curiosity on my end, so anything I can learn/find out is much appreciated on all ends.

Anyways, just figured I'd update y'all.

top 22 comments
sorted by: hot top controversial new old
[–] [email protected] 45 points 1 year ago (1 children)

Congrats on becoming the reason a non trivial rate like is going to be imposed πŸ˜†

[–] [email protected] 11 points 1 year ago

Seriously, I would be upset but I'm also kind of impressed. Bravo.

[–] [email protected] 17 points 1 year ago (2 children)

Hello HN! OP, you've reached the front page of Hacker News:

[–] csm10495 2 points 1 year ago

I'm honored.

[–] [email protected] 1 points 1 year ago

I wonder if this is a sign that Lemmy has achieved real Reddit replacement status. HN occasionally links to some interesting Reddit post like it did here for Lemmy

[–] [email protected] 14 points 1 year ago (1 children)

excessive resource use

What's the final size of the git folder?
What's the final time and CPU usage to process the different git commands?

[–] csm10495 14 points 1 year ago* (last edited 1 year ago) (1 children)

Last check was over 8GB

A full (or full master branch) clone was well over 4 hours. A shallow (--depth 1) clone would take like 5 seconds. A pull origin master on a local copy that was less than 2 days old was well over an hour.

I was using GitHub Actions to get the latest 'total commit count'. I was actually using https://github.com/actions/cache to cache a full repo clone, pull origin master then update that cached copy after to not have to do a full repo clone each time. At that point run a rev-list command to get the count.

Doing that process would take a few minutes.

I also recently started having GitHub Actions make more branches to be auto merged into master. They allow actions to be run every 10 minutes, so I was having it make commits for 10 minutes then it would push temp branches that would be PR'd and merged via the gh cli by my locally running instance.

Just that 10 minutes of commits would be over 4500 commits per branch. Doing the push from that point would take about 2 minutes.

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago)

Did you try any blobless or treeless clones? They're ways to more easily clone large repositories you don't need the full history of.

[–] [email protected] 9 points 1 year ago

That's hilarious! I'm actually impressed it lasted for 22 million commits, I would have thought the breaking point would be earlier.

[–] [email protected] 8 points 1 year ago (1 children)

wow, that's pretty intense.i hadn't heard of your project before. was the intent just to see how many commits github could handle?

[–] csm10495 5 points 1 year ago (1 children)

More/less yep. I also wanted to see the limits of such a high amount of individual commits.

[–] gravitas_deficiency 1 points 1 year ago

Well congrats, because I think you just achieved engineering fame lol

[–] [email protected] 8 points 1 year ago

Demand a special profile achievement badge from them as a consolation!

[–] [email protected] 7 points 1 year ago

Congratulation on breaking Github...

It's interesting to see that your personal repo can affect another repository.

[–] [email protected] 6 points 1 year ago

well, time to squash those commits and begin again

[–] csm10495 5 points 1 year ago (1 children)

Tiny mini update: GitHub support has been really nice and all is fine. They told me that this repo was taking up 5.5TB of resources doing certain operations. Also pointed me to https://github.com/github/git-sizer and https://github.blog/2018-03-05-measuring-the-many-sizes-of-a-git-repository/

They are happy to have the 5.5TB back... and I'm happy to give them back. I'll probably upload/archive the code and the final notes in a new repo, and then close this chapter.

If I ever really want to play with this again, I'd get a GitHub Enterprise license and let it go wild on a local box.

[–] [email protected] 1 points 1 year ago

I think it's a interesting case that they didn't expect somehow, I have a real repo that has around 1.5M commits on GHEC and surprisingly it works rather well so far, although some functionality (that I don't really care about) is slower than usual.

[–] [email protected] 4 points 1 year ago

I think this means you win

[–] [email protected] 3 points 1 year ago

Lol, how and why did you even come up with this

[–] [email protected] 3 points 1 year ago (1 children)

How did it go (or how is it going) with the local Git repository? I suspect it's still working as expected?

[–] csm10495 4 points 1 year ago

The local repo is still working, but I've kind of stopped the project at this point.

[–] [email protected] 2 points 1 year ago

All i have to say is GG !

load more comments
view more: next β€Ί