Custom domain support for pages.codeberg.org #115

Open
by Ghost opened 2 years ago · 29 comments
Ghost commented 2 years ago
  • Github and Gitlab already has the feature of Adding Custom Domains for their "Website Hosting for Projects".

  • I think that Codeberg Pages must also support custom domains ( providing subdomains like projectname.codeberg.io would be cool too).

  • Even though we can automate the process,i propose this:

    • There must be repo like Codeberg Issues where project owner requests custom domain/subdomain of codeberg.org / codeberg.io for his project page)
    • If the admin feels that it's acceptable,he adds custom domain for his project page(also enables let's encrypt HTTPS for the site).
- Github and Gitlab already has the feature of Adding Custom Domains for their "Website Hosting for Projects". - I think that [Codeberg Pages](https://pages.codeberg.org/) must also support custom domains ( providing subdomains like projectname.codeberg.io would be cool too). - Even though we can automate the process,i propose this: - There must be repo like [Codeberg Issues](https://codeberg.org/Codeberg/Community) where project owner requests custom domain/subdomain of codeberg.org / codeberg.io for his project page) - If the admin feels that it's acceptable,he adds custom domain for his project page(also enables let's encrypt HTTPS for the site).
Ghost closed this issue 2 years ago
af commented 2 years ago

Any updates on this? Why this was closed?

Any updates on this? Why this was closed?
hw reopened this issue 2 years ago
af commented 2 years ago

In https://mastodon.technology/@codeberg/103947132581783430 it was suggested that it is implemented. Maybe one can provide a short tutorial on how to do it with one of the domain providers, Gandi or Namecheap or whatever. What kind of the DNS record I should set up?

In https://mastodon.technology/@codeberg/103947132581783430 it was suggested that it is implemented. Maybe one can provide a short tutorial on how to do it with one of the domain providers, Gandi or Namecheap or whatever. What kind of the DNS record I should set up?
hw commented 2 years ago
Owner

No idea why this was closed ... some comments:

  • Allowing arbitrary subdomains to codeberg.org would open security vulnerability and CSRF attack surface. For details see this blog.
  • We could indeed reserve a custom domain for hosting subdomains, as other services do. This would however require wildcard SSL and some custom server setup (not impossible but not a priority either, contributions for such a setup surely welcome!).
  • There are other security issues that have to be thought through, a systematic review is very welcome.

The simple goal of pointing a custom domain to a custom codeberg page can get achieved without the above, using the URI record (see https://en.wikipedia.org/wiki/URI_record for background and example). Also redirect services.

No idea why this was closed ... some comments: - Allowing arbitrary subdomains to codeberg.org would open security vulnerability and CSRF attack surface. For details see [this blog](https://github.blog/2013-04-05-new-github-pages-domain-github-io/). - We could indeed reserve a custom domain for hosting subdomains, as other services do. This would however require wildcard SSL and some custom server setup (not impossible but not a priority either, contributions for such a setup surely welcome!). - There are other security issues that have to be thought through, a systematic review is very welcome. The simple goal of pointing a custom domain to a custom codeberg page can get achieved without the above, using the URI record (see https://en.wikipedia.org/wiki/URI_record for background and example). Also redirect services.
af commented 2 years ago

Thanks for the reply.

Adding subdomains to codeberg.org would open security vulnerability and CSRF attack surface. For details see this blog.

I think this is covered in gitlab with domain verification: https://about.gitlab.com/handbook/support/workflows/verify_pages_domain.html

This would however require wildcard SSL and some custom server setup (not impossible but not a priority either, contributions for such a setup surely welcome!).

Let's encrypt allows Wildcard Certificates (for example: https://dev.to/nabbisen/let-s-encrypt-wildcard-certificate-with-certbot-plo)

The simple goal of pointing a custom domain to a custom codeberg page can get achieved without the above, using the URI record (see https://en.wikipedia.org/wiki/URI_record for background and example). Also redirect services.

A link to a service or a tutorial would be great. Since I couldn't find anything (probably looking in a wrong way): https://www.digitalocean.com/community/questions/digitalocean-dns-where-is-the-redirect-alias-dns-record

Thank you.

Thanks for the reply. > Adding subdomains to codeberg.org would open security vulnerability and CSRF attack surface. For details see this blog. I think this is covered in gitlab with domain verification: https://about.gitlab.com/handbook/support/workflows/verify_pages_domain.html > This would however require wildcard SSL and some custom server setup (not impossible but not a priority either, contributions for such a setup surely welcome!). Let's encrypt allows Wildcard Certificates (for example: https://dev.to/nabbisen/let-s-encrypt-wildcard-certificate-with-certbot-plo) > The simple goal of pointing a custom domain to a custom codeberg page can get achieved without the above, using the URI record (see https://en.wikipedia.org/wiki/URI_record for background and example). Also redirect services. A link to a service or a tutorial would be great. Since I couldn't find anything (probably looking in a wrong way): https://www.digitalocean.com/community/questions/digitalocean-dns-where-is-the-redirect-alias-dns-record Thank you.
hw commented 2 years ago
Owner

Thanks for the reply.

Adding subdomains to codeberg.org would open security vulnerability and CSRF attack surface. For details see this blog.

I think this is covered in gitlab with domain verification: https://about.gitlab.com/handbook/support/workflows/verify_pages_domain.html

This is another, unrelated security issue (one for point 3 in the bullet list ;) ... ). Would need some extra work in the UI. Contributions welcome!

Let's encrypt allows Wildcard Certificates (for example: https://dev.to/nabbisen/let-s-encrypt-wildcard-certificate-with-certbot-plo)

Yes, still some effort to set up. Contributions for setup welcome!

link to a service [...]

Cannot do advertising or support for particular providers here, please contact your registrar how to set up RFC7553 records for this particular service, or maybe try entering terms like "redirect service", "tutorial URI record", "how to set up URI record" in your favorite search engine. The top results here were mostly relevant.

> Thanks for the reply. > > > Adding subdomains to codeberg.org would open security vulnerability and CSRF attack surface. For details see this blog. > > I think this is covered in gitlab with domain verification: https://about.gitlab.com/handbook/support/workflows/verify_pages_domain.html This is another, unrelated security issue (one for point 3 in the bullet list ;) ... ). Would need some extra work in the UI. Contributions welcome! > Let's encrypt allows Wildcard Certificates (for example: https://dev.to/nabbisen/let-s-encrypt-wildcard-certificate-with-certbot-plo) Yes, still some effort to set up. Contributions for setup welcome! > link to a service [...] Cannot do advertising or support for particular providers here, please contact your registrar how to set up RFC7553 records for this particular service, or maybe try entering terms like "redirect service", "tutorial URI record", "how to set up URI record" in your favorite search engine. The top results here were mostly relevant.
Collaborator

Hey there, I'm hosting many (static) pages with lighttpd. It's very well possible to automate the process of creating a folder for a domain, asking LetsEncrypt to verify this specific domain (it would verify that someone pointed it at the codeberg server, manual checks beforehand are necessary in order to not hit any LetsEncrypt limits) and automate deployment from codeberg ...

I guess I could assist in spinning something like this up, I'm also familiar with other web servers.

I'd appreciate assistance from someone who is currently operating codeberg infrastructure.

Hey there, I'm hosting many (static) pages with lighttpd. It's very well possible to automate the process of creating a folder for a domain, asking LetsEncrypt to verify this specific domain (it would verify that someone pointed it at the codeberg server, manual checks beforehand are necessary in order to not hit any LetsEncrypt limits) and automate deployment from codeberg ... I guess I could assist in spinning something like this up, I'm also familiar with other web servers. I'd appreciate assistance from someone who is currently operating codeberg infrastructure.
hw commented 2 years ago
Owner

asking LetsEncrypt to verify this specific domain

Doing it one-by-one per subdomain will likely run into scaling issues, if we do this, wildcard certificates seem mandatory.

I guess I could assist in spinning something like this up [...]

A test setup and example config/setup would be nice! There are probably also some changes required to https://codeberg.org/Codeberg/build-deploy-gitea/src/branch/master/var/www/pages/index.php.

If you like to experiment, please consider a nginx config, as this is already running the reg-server backend.

> asking LetsEncrypt to verify this specific domain Doing it one-by-one per subdomain will likely run into scaling issues, if we do this, wildcard certificates seem mandatory. > I guess I could assist in spinning something like this up [...] A test setup and example config/setup would be nice! There are probably also some changes required to https://codeberg.org/Codeberg/build-deploy-gitea/src/branch/master/var/www/pages/index.php. If you like to experiment, please consider a nginx config, as this is already running the reg-server backend.
Collaborator

Doing it one-by-one per subdomain will likely run into scaling issues, if we do this, wildcard certificates seem mandatory.

I thought we are talking about custom domains?
If we used something like user.codeberg.io we should indeed use a wildcard cert ...

Well, I never used nginx but I could have a look at it.
I also had a further look at the current tech stack around Codeberg and I fear that I can't promise a nice integration.
I'd like to play around with Caddy or something like this as it offers on-the-fly API configuration which would be a nice tool for not needing to modify system configuration files from within a custom application. It's also modern and written in Golang and should offer all the features that are needed here ...

Otherwise we could try and have a static file linking or proxying from ^https://(.*)\.codeberg\.io/(.*)$ to https://codeberg.org/pages/$0/$1 or something like that if this is possible with nginx or another web stack ...

> Doing it one-by-one per subdomain will likely run into scaling issues, if we do this, wildcard certificates seem mandatory. I thought we are talking about custom domains? If we used something like user.codeberg.io we should indeed use a wildcard cert ... Well, I never used nginx but I could have a look at it. I also had a further look at the current tech stack around Codeberg and I fear that I can't promise a nice integration. I'd like to play around with Caddy or something like this as it offers on-the-fly API configuration which would be a nice tool for not needing to modify system configuration files from within a custom application. It's also modern and written in Golang and should offer all the features that are needed here ... Otherwise we could try and have a static file linking or proxying from `^https://(.*)\.codeberg\.io/(.*)$` to `https://codeberg.org/pages/$0/$1` or something like that if this is possible with nginx or another web stack ...
Collaborator

I understood custom domain in the sense of FQDN ...

I understood custom domain in the sense of FQDN ...
hw commented 1 year ago
Owner

I understood custom domain in the sense of FQDN ...

Mh, not sure how this should work technically without being a domain registrar?

Could you outline the approach you have in mind here?

> I understood custom domain in the sense of FQDN ... Mh, not sure how this should work technically without being a domain registrar? Could you outline the approach you have in mind here?
Collaborator

Custom (FQDN) support for i. e. GitHub pages is AFAIK that a user registers a domain and points it to the IP address of a codeberg server.

The benefit over custom hosting is that it's nicely integrated to the git / pages feature without the need to worry about automated deployment to your own web space if you rent it somewhere else.

Many users like the pages feature but still want the "pro feeling" of a custom domain.

I still think that this feature is low-priority as I expect few people to worry about setting up a custom domain. But the codeberg.io (or another) subdomain approach would be useful and it's worth to design this with a custom domain support in mind which shouldn't be hard to add if the system is up and running ...

Custom (FQDN) support for i. e. GitHub pages is AFAIK that a user registers a domain and points it to the IP address of a codeberg server. The benefit over custom hosting is that it's nicely integrated to the git / pages feature without the need to worry about automated deployment to your own web space if you rent it somewhere else. Many users like the pages feature but still want the "pro feeling" of a custom domain. I still think that this feature is low-priority as I expect few people to worry about setting up a custom domain. But the codeberg.io (or another) subdomain approach would be useful and it's worth to design this with a custom domain support in mind which shouldn't be hard to add if the system is up and running ...
hw commented 1 year ago
Owner

a user registers a domain and points it to the IP address of a codeberg server.

Thats's what I had in mind too. This can happen via RFC7553 URI record or CNAME/ALIAS record.

URI records point to URLs (this is possible right now), CNAME/ALIAS records point to (sub-)domains, here we would need a dedicated TLD with user-specific subdomains to avoid the CSRF security issues above.

A/AAAA records point to IP addresses, which seem impractible in this case?

What other options are possible?

> a user registers a domain and points it to the IP address of a codeberg server. Thats's what I had in mind too. This can happen via [RFC7553 URI record](https://en.wikipedia.org/wiki/URI_record) or [CNAME/ALIAS record](https://en.wikipedia.org/wiki/CNAME_record). URI records point to URLs (this is possible right now), CNAME/ALIAS records point to (sub-)domains, here we would need a dedicated TLD with user-specific subdomains to avoid the CSRF security issues above. A/AAAA records point to IP addresses, which seem impractible in this case? What other options are possible?
Collaborator

Well, the CNAME approach is common for this purpose. The domain name takes longer to resolve when using CNAME since there is need of one additional lookup, but this can be addressed by choosing appropriate TTL (time to live) values so it's at least cached for a while.

I never tried the URI record, is this visible to the user or does the user FQDN stay in the address bar while the data is retrieved by a longer codeberg.org/ url ?

A/AAAA records are technically okay as well, the user could add the IP address(es) of the Codeberg pages server(s), but with the caveeat that changes to the setup (e. g. a changing or addition or removal of an IP address) does not automatically update the user's DNS zone (he has to update this manually) while a CNAME to i. e. pages.codeberg.org will always be valid as the Codeberg team would update this record with the actual technical values.

If we added sub domain hosting with e. g. codeberg.io, I'd use a wildcard with A/AAAA records and make sure they can be easily updated via an API to the DNS server / registrar. FQDN support could be officially supported via a CNAME to codeberg.io or even the user's page and allow him to update this via frontend.

If we are really geeky we could do a complete automatic setup: When entering fnetx.codeberg.io it would automatically detect that it's my page hosting requested and if a user accesses my new softwarexy.example.com which CNAMEs to fnetx.codeberg.io it could automatically obtain a LetsEncrypt cert and set up the correct hosting so we could have a webserver that automatically checks which page is requested and set it up in under a minute after the first page access 😂

Well, the CNAME approach is common for this purpose. The domain name takes longer to resolve when using CNAME since there is need of one additional lookup, but this can be addressed by choosing appropriate TTL (time to live) values so it's at least cached for a while. I never tried the URI record, is this visible to the user or does the user FQDN stay in the address bar while the data is retrieved by a longer codeberg.org/ url ? A/AAAA records are technically okay as well, the user could add the IP address(es) of the Codeberg pages server(s), but with the caveeat that changes to the setup (e. g. a changing or addition or removal of an IP address) does not automatically update the user's DNS zone (he has to update this manually) while a CNAME to i. e. pages.codeberg.org will always be valid as the Codeberg team would update this record with the actual technical values. If we added sub domain hosting with e. g. codeberg.io, I'd use a wildcard with A/AAAA records and make sure they can be easily updated via an API to the DNS server / registrar. FQDN support could be officially supported via a CNAME to codeberg.io or even the user's page and allow him to update this via frontend. *If we are really geeky we could do a complete automatic setup: When entering fnetx.codeberg.io it would automatically detect that it's my page hosting requested and if a user accesses my new softwarexy.example.com which CNAMEs to fnetx.codeberg.io it could automatically obtain a LetsEncrypt cert and set up the correct hosting so we could have a webserver that automatically checks which page is requested and set it up in under a minute after the first page access* :joy:
Collaborator

To the last paragraph: This is not geeky at all, I just learned that the Caddy webserver kinda supports this and calls it On-Demand-TLS. See https://caddyserver.com/docs/automatic-https#on-demand-tls

To the last paragraph: This is not geeky at all, I just learned that the Caddy webserver kinda supports this and calls it On-Demand-TLS. See https://caddyserver.com/docs/automatic-https#on-demand-tls

Prettier URL with dedicated Codeberg subdomain, and a Let's Encrypt cert for that would be awesome.

Prettier URL with dedicated Codeberg subdomain, and a Let's Encrypt cert for that would be awesome.
Collaborator

So, I think having momar.codeberg.io/blubb/ (or codeberg.dev or something like that) point to momar/blubb just like on GitHub is a good idea. This is if we want to allow per-repo pages.

I don't think this creates any noteworthy security issues, as everything hosted on *.codeberg.io would just be static HTML/JavaScript and is public anyways.
localStorage and sessionStorage would also be limited to the specific user, which would then even make it possible to use Codeberg Pages to host JAMStack applications, as opposed to right now.

For DNS records, if we only want to allow per-user pages, a simple CNAME momar.codeberg.io makes sense - it can be parsed by the server and resolved to the correct repository. If we want to allow one page per repo, it could make sense to use a CNAME codeberg.io and then a TXT momar.codeberg.io/blubb for the target repository, at least that's the easiest way I can think of right now.

What I'm not that happy about with this approach is that anyone could just add any TXT record, so anyone could create a domain for any repository - I think this is something the owners want to have as much control over as possible, even if it's relatively easy to circumvent with a reverse proxy. So, I'd suggest to require email confirmation for new or changed domains for a repo, to ensure that the owner actually meant this domain to point to his repo.

So, I think having `momar.codeberg.io/blubb/` (or codeberg.dev or something like that) point to momar/blubb just like on GitHub is a good idea. This is if we want to allow per-repo pages. I don't think this creates any noteworthy security issues, as everything hosted on \*.codeberg.io would just be static HTML/JavaScript and is public anyways. localStorage and sessionStorage would also be limited to the specific user, which would then even make it possible to use Codeberg Pages to host JAMStack applications, as opposed to right now. For DNS records, if we only want to allow per-user pages, a simple `CNAME momar.codeberg.io` makes sense - it can be parsed by the server and resolved to the correct repository. If we want to allow one page per repo, it could make sense to use a `CNAME codeberg.io` and then a `TXT momar.codeberg.io/blubb` for the target repository, at least that's the easiest way I can think of right now. What I'm not that happy about with this approach is that anyone could just add any TXT record, so anyone could create a domain for any repository - I think this is something the owners want to have as much control over as possible, even if it's relatively easy to circumvent with a reverse proxy. So, I'd suggest to require email confirmation for new or changed domains for a repo, to ensure that the owner actually meant this domain to point to his repo.
Collaborator

Oh, and adding Let's Encrypt on demand shouldn't be an issue I guess.

Oh, and adding Let's Encrypt on demand shouldn't be an issue I guess.

I would love ❤️ to see custom domain support, and preferably site per repository feature similar to github. Right now I have humanetech.community, innercircles.community, and some others entirely on github and will have more also on github just for the domain name support. I'd much prefer to remove all from github, though.

I would love :heart: to see custom domain support, and preferably site per repository feature similar to github. Right now I have [humanetech.community](https://humanetech.community), [innercircles.community](https://innercircles.community), and some others entirely on github and will have more also on github just for the domain name support. I'd much prefer to remove all from github, though.
fnetX added the
contribution welcome
pages
enhancement
labels 7 months ago
Collaborator

Quick update: This will be implemented in the new pages version (https://codeberg.org/momar/codeberg-pages) at some point, using CNAME [[branch.]repo.]user.codeberg.page, or TXT instead of CNAME with an ALIAS/A/AAAA record.

It will also be required to add a domains.txt file to the repo/branch, with all allowed domains (and a maximum of 5), to mitigate DoS by requesting certificates from domains not set up for Codeberg pages. The first line in that file will be the canonical domain, that will be redirected to from all other domains and the example.codeberg.page format.

For certificates to be working, we will need a second IP address, as the Pages server can't really stay behind a reverse proxy - it has to detect the SNI of a new connection, check if there's a pages repo set in DNS, if the domains.txt matches, and only then request a certificate, which is then being served - this would get really complicated with a reverse proxy.
A second server for this doesn't really make sense as it has to access the Gitea API (and that would cause a lot of network overhead), but HAProxy could listen on port :80/:443 on one IP, and Codeberg Pages could listen on port :80/:443 on another IP.

Quick info about Let's Encrypt rate limits: 300 new orders per account per 3 hours, and we can theoretically use multiple accounts, although I think if we need that, it makes more sense to request a limit increase directly from them; renewal isn't rate-limited. I would suggest to add an additional limit of 5 domains per Codeberg user per 3 hours, to avoid DoS.

Quick update: This will be implemented in the new pages version (https://codeberg.org/momar/codeberg-pages) at some point, using `CNAME [[branch.]repo.]user.codeberg.page`, or TXT instead of CNAME with an ALIAS/A/AAAA record. It will also be required to add a `domains.txt` file to the repo/branch, with all allowed domains (and a maximum of 5), to mitigate DoS by requesting certificates from domains not set up for Codeberg pages. The first line in that file will be the canonical domain, that will be redirected to from all other domains and the example.codeberg.page format. For certificates to be working, we will need a second IP address, as the Pages server can't really stay behind a reverse proxy - it has to detect the SNI of a new connection, check if there's a pages repo set in DNS, if the domains.txt matches, and only then request a certificate, which is then being served - this would get really complicated with a reverse proxy. A second server for this doesn't really make sense as it has to access the Gitea API (and that would cause a lot of network overhead), but HAProxy could listen on port :80/:443 on one IP, and Codeberg Pages could listen on port :80/:443 on another IP. Quick info about Let's Encrypt rate limits: 300 new orders per account per 3 hours, and we can theoretically use multiple accounts, although I think if we need that, it makes more sense to [request a limit increase](https://letsencrypt.org/docs/rate-limits/#a-id-overrides-a-overrides) directly from them; renewal isn't rate-limited. I would suggest to add an additional limit of 5 domains per Codeberg user per 3 hours, to avoid DoS.

This is wonderful, wonderful news @momar .. thanks heaps for all the efforts that you and others have put into setting this up! 🥇

This is wonderful, wonderful news @momar .. thanks heaps for all the efforts that you and others have put into setting this up! 🥇
momar self-assigned this 5 months ago
momar removed the
contribution welcome
label 5 months ago
Tealk referenced this issue from a commit 4 months ago

It will also be required to add a domains.txt file to the repo/branch, with all allowed domains (and a maximum of 5), to mitigate DoS by requesting certificates from domains not set up for Codeberg pages. The first line in that file will be the canonical domain, that will be redirected to from all other domains and the example.codeberg.page format.

do i understand correctly that if i create this text file and i call user.codeberg.page it should redirect to the domain which is in the first line of the text file

> It will also be required to add a `domains.txt` file to the repo/branch, with all allowed domains (and a maximum of 5), to mitigate DoS by requesting certificates from domains not set up for Codeberg pages. The first line in that file will be the canonical domain, that will be redirected to from all other domains and the example.codeberg.page format. do i understand correctly that if i create this text file and i call user.codeberg.page it should redirect to the domain which is in the first line of the text file
Collaborator

In the future, yes. It's still an idea at the moment.

In the future, yes. It's still an idea at the moment.

oh ok I had read that it already works, too bad :'(

oh ok I had read that it already works, too bad :'(

Thank you @momar for your work on this, and delightful to see the progress. I posted on fediverse to credit you.

Thank you @momar for your work on this, and delightful to see the progress. I [posted on fediverse](https://mastodon.social/@humanetech/106549179969950959) to credit you.

Thankyou @momar. This would help many people shift away from gh/netlify towards freer option. Very excited for a new release.

Thankyou @momar. This would help many people shift away from gh/netlify towards freer option. Very excited for a new release.
Collaborator

Thanks! Just to give you a quick update, what's missing right now is basically rate limiting for certificates (so that one user can not use up all our Let's Encrypt contingent), the deployment to the new server and lots and lots of testing. I will try to get this ready by next weekend.

I assume that @circlebuilder, @SidhantGoyal & @Tealk would probably want to participate in a closed beta? For anyone who wants to participate, vote 🚀 on this comment.

Thanks! Just to give you a quick update, what's missing right now is basically rate limiting for certificates (so that one user can not use up all our Let's Encrypt contingent), the deployment to the new server and lots and lots of testing. I will try to get this ready by next weekend. I assume that @circlebuilder, @SidhantGoyal & @Tealk would probably want to participate in a closed beta? For anyone who wants to participate, vote 🚀 on this comment.

Definitely would. I have this site for it now.

Definitely would. I have [this site](https://fediverse.codeberg.page) for it now.

I assume that @circlebuilder, @SidhantGoyal & @Tealk would probably want to participate in a closed beta? For anyone who wants to participate, vote 🚀 on this comment.

Yes very very much

I have two pages created with hugo that I would then display over it.
https://butterflyaspect.codeberg.page
https://rollenspielmonster.codeberg.page

> I assume that @circlebuilder, @SidhantGoyal & @Tealk would probably want to participate in a closed beta? For anyone who wants to participate, vote 🚀 on this comment. Yes very very much I have two pages created with hugo that I would then display over it. https://butterflyaspect.codeberg.page https://rollenspielmonster.codeberg.page

That would be great!!

That would be great!!
Sign in to join this conversation.
No Milestone
No Assignees
8 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
There is no content yet.