[FEAT] Improving migrations #398

New Issue

Open

opened 2023-02-20 14:18:20 +00:00 by fnetX · 7 comments

fnetX commented

2023-02-20 14:18:20 +00:00

Owner

Needs and benefits

Migrating back and forth between Forgejo instances is among the most important features the software can deliver, breaking vendor lock-in and encouraging users to spread over available instances once federation matures.

However, the migration feature is among the part where a lot of user support requests to Codeberg originate from, because the user experience is not always smooth.

Feature Description

I think, the following domains need improvement, and should be tracked in separate issues by time (feel free to update this comment).

Performance:

Forgejo→Forgejo seems to take most time fetching comment reaction for each individual comment, which is rather slow (API update)

UX:

allow to auto-migrate a list of migrations from the user account
give more helpful error messages and / or encourage users to seek help with their local admins or the Forgejo team (the "Migration failed" screen is rather unhelpful and not motivating)

Other:

allow resuming (failed?) migrations (allow resuming failed migrations or updating existing ones to the latest state would make it much smoother for projects to keep a full mirror and switch when the time comes, instead of re-migrating everything)

## Needs and benefits Migrating back and forth between Forgejo instances is among the most important features the software can deliver, breaking vendor lock-in and encouraging users to spread over available instances once federation matures. However, the migration feature is among the part where a lot of user support requests to Codeberg originate from, because the user experience is not always smooth. ## Feature Description I think, the following domains need improvement, and should be tracked in separate issues by time (feel free to update this comment). Performance: - Forgejo→Forgejo seems to take most time fetching comment reaction for each individual comment, which is rather slow (API update) UX: - allow to auto-migrate a list of migrations from the user account - give more helpful error messages and / or encourage users to seek help with their local admins or the Forgejo team (the "Migration failed" screen is rather unhelpful and not motivating) Other: - allow resuming (failed?) migrations (allow resuming failed migrations or updating existing ones to the latest state would make it much smoother for projects to keep a full mirror and switch when the time comes, instead of re-migrating everything)

👍 1 ❤️ 1

fnetX added the

enhancement/feature

Discussion

labels 2023-02-20 14:18:21 +00:00

fnetX commented

2023-02-20 17:30:15 +00:00

Author

Owner

Just to give an idea what a single (still ongoing BTW) migration causes:

17:26:34 root@gitea-production / # journalctl --grep "/api/v1/repos/calckey" | wc -l
1354173
17:28:23 root@gitea-production / # journalctl --grep "/api/v1/repos/calckey/calckey/issues/comments/(.*)/reactions" | wc -l
1321053
17:28:59 root@gitea-production / #

No wonder that migrating active repositories literally takes forever when you look at the number of necessary API calls.

Just to give an idea what a single (still ongoing BTW) migration causes: ~~~ 17:26:34 root@gitea-production / # journalctl --grep "/api/v1/repos/calckey" | wc -l 1354173 17:28:23 root@gitea-production / # journalctl --grep "/api/v1/repos/calckey/calckey/issues/comments/(.*)/reactions" | wc -l 1321053 17:28:59 root@gitea-production / # ~~~ No wonder that migrating active repositories literally takes forever when you look at the number of necessary API calls.

fnetX referenced this issue from Codeberg/Community

2023-02-20 17:33:12 +00:00

Remove ratelimits for migrating repositories #896

Gusted commented

2023-02-20 20:51:50 +00:00

Owner

Mentioned this elsewhere, I'm not so sure about the states of affairs w.r.t. F3. But it should be possible to package a repository with its metadata as an F3 package and teach Forgejo to import and export that. @dachary @earl-warren