Performance issues #453

Closed
opened 4 months ago by ddevault · 8 comments

Hello! When I am not logged in to Codeberg, each page load on Codeberg takes anywhere from 60 seconds to 3 or 4 minutes. I initially suspected a routing issue, but given that the problem went away after I managed to log in, I'm not so sure.

Hello! When I am not logged in to Codeberg, each page load on Codeberg takes anywhere from 60 seconds to 3 or 4 minutes. I initially suspected a routing issue, but given that the problem went away after I managed to log in, I'm not so sure.
Collaborator

I can' reproduce this on my machine. Do you eventually have a browser log that show which requests take this long?

I can' reproduce this on my machine. Do you eventually have a browser log that show which requests take this long?
Poster

Doesn't seem to be an issue today. I'll update this the next time I can reproduce it.

Doesn't seem to be an issue today. I'll update this the next time I can reproduce it.
fnetX added the
infrastructure
question
labels 4 months ago

I encountered the same issue. Codeberg had really poor performance yesterday when I tried to sign up. The landing page would load but anything other than that would take several minutes to load. However, that is not the case today and every page seems to be loading fine.

I encountered the same issue. Codeberg had really poor performance yesterday when I tried to sign up. The landing page would load but anything other than that would take several minutes to load. However, that is not the case today and every page seems to be loading fine.
Collaborator

We are investigating these reports. It's somewhat unlikely to be related to the logged-in state. Can you tell us

  • the exact time when you run into those issues
  • the page you were browsing and maybe if you did anything special before (interesting especially if you are browsing big repos, reloading the page, encounter timeouts etc)
  • if you can open any other page during that time
  • how long it takes until you can continue browsing Codeberg again

System parameters look fine during the last days, the last warning was issued at the beginning of May. We found some weird request behaviour in the logs, but we cannot really tell how this relates yet.

Also, we reproduced some scenarios where we can temporarily render the whole instance unresponsive on codeberg-test.org (single-core VPS, much easier to trap). We'd be glad to receive some more hints when looking for indices that this happens on production, too. We also suspect a conneciton to some backend changes (go-git vs direct git usage).

We are investigating these reports. It's somewhat unlikely to be related to the logged-in state. Can you tell us - the exact time when you run into those issues - the page you were browsing and maybe if you did anything special before (interesting especially if you are browsing big repos, reloading the page, encounter timeouts etc) - if you can open any other page during that time - how long it takes until you can continue browsing Codeberg again System parameters look fine during the last days, the last warning was issued at the beginning of May. We found some weird request behaviour in the logs, but we cannot really tell how this relates yet. Also, we reproduced some scenarios where we can temporarily render the whole instance unresponsive on codeberg-test.org (single-core VPS, much easier to trap). We'd be glad to receive some more hints when looking for indices that this happens on production, too. We also suspect a conneciton to some backend changes (go-git vs direct git usage).
Poster

I can reproduce this now while logged in, actually. All pages are pretty slow. I've attached a HAR file from loading this page which should provide all the details you need (though I have removed cookies).

I can reproduce this now while logged in, actually. All pages are pretty slow. I've attached a HAR file from loading this page which should provide all the details you need (though I have removed cookies).
Collaborator

If you are interested in reading on, please also follow the discussion upstream: https://github.com/go-gitea/gitea/issues/15707#issuecomment-850399192

If you are interested in reading on, please also follow the discussion upstream: https://github.com/go-gitea/gitea/issues/15707#issuecomment-850399192
fnetX changed title from Extremely poor performance while logged out to Performance issues 4 months ago
Collaborator

Yesterday, we migrated the backend back to go-git for now and the performance was improved again. We hope to help optimize the backend performance again, so that it is fast and uses less RAM.

Also, we are trying to tune caching with Redis.

Yesterday, we migrated the backend back to go-git for now and the performance was improved again. We hope to help optimize the backend performance again, so that it is fast **and** uses less RAM. Also, we are trying to tune caching with Redis.
fnetX added
upstream
and removed
question
labels 4 months ago
rwa added the
gitea-related
label 3 months ago
Collaborator

The issue was addressed by changing the backend implementation and adding Redis as a commit info cache.

The site's performance isn't heavily degraded now as it was at the time of reporting, but further improving it is of course very welcome (will close this issue nevertheless).

I wasn't following the exact upstream discussions recently (I don't think I can help there at the moment), but some plans include deferring the last commit information into an async function.

The issue was addressed by changing the backend implementation and adding Redis as a commit info cache. The site's performance isn't heavily degraded now as it was at the time of reporting, but further improving it is of course very welcome (will close this issue nevertheless). I wasn't following the exact upstream discussions recently (I don't think I can help there at the moment), but some plans include deferring the last commit information into an async function.
fnetX closed this issue 2 months ago
Sign in to join this conversation.
No Milestone
No Assignees
4 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
There is no content yet.