Device is temporarily blocked from codeberg after pushing/pulling lfs objects #786

Open
opened 3 weeks ago by FinnPerry · 1 comments

I created this repo to store a vendored package in. It contains some binary files which I've tracked as lfs.

Whenever I push or pull lfs data from the repo, it seems like I get timed out from codeberg and am temporarily unable to pull/push to any of my repos or access the web ui from that device.

Once I regained access (the next day) I tried splitting the repo into several commits and pushing them one by one, then squishing back to 1 commit. That worked but then I tried cloning and got timed out again, so I'm guessing it's to do with the amount of data being transfered at once.

Is this intentional and I just shouldn't store this kind of repo on codeberg or is this a bug?

I created [this repo](https://codeberg.org/FinnPerry/oculus-integration-sdk) to store a vendored package in. It contains some binary files which I've tracked as lfs. Whenever I push or pull lfs data from the repo, it seems like I get timed out from codeberg and am temporarily unable to pull/push to any of my repos or access the web ui from that device. Once I regained access (the next day) I tried splitting the repo into several commits and pushing them one by one, then squishing back to 1 commit. That worked but then I tried cloning and got timed out again, so I'm guessing it's to do with the amount of data being transfered at once. Is this intentional and I just shouldn't store this kind of repo on codeberg or is this a bug?
Owner

Hmm, the repository is indeed very large, but it should not be completely impossible to push such large projects.

We have recently deployed an adaptive rate limiting which looks at the share of requests a single IP makes. If our load is high and a single IP is doing more than 4% of all requests of that instance, it will get blocked (we are starting with the most requests based on load).

It seemed to us very unlikely that legitimate users get blocked by this. We investigated many cases, and it was mostly bots and crawlers.

However, I can well imagine, that accessing many files with LFS is indeed a problem, because it spawns many connections.

We'll think about it. Maybe we increase the detection window for the share calculation, or we have a better idea ...

Thank you for the report, and sorry for the inconvenience!

Hmm, the repository is indeed very large, but it should not be completely impossible to push such large projects. We have recently deployed an adaptive rate limiting which looks at the share of requests a single IP makes. If our load is high and a single IP is doing more than 4% of all requests of that instance, it will get blocked (we are starting with the most requests based on load). It seemed to us very unlikely that legitimate users get blocked by this. We investigated many cases, and it was mostly bots and crawlers. However, I can well imagine, that accessing many files with LFS is indeed a problem, because it spawns many connections. We'll think about it. Maybe we increase the detection window for the share calculation, or we have a better idea ... Thank you for the report, and sorry for the inconvenience!
fnetX added the
bug
infrastructure
Codeberg
labels 3 weeks ago
Sign in to join this conversation.
No Milestone
No Assignees
2 Participants
Notifications
Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Codeberg/Community#786
Loading…
There is no content yet.