#1 Migration of the website

Open
opened 2 months ago by brickup · 16 comments
brickup commented 2 months ago

Hi there,

we’ve made it, the website is back up and running! Right now, switching.software is a copy of every cached page of the switching.social website, pulled from different sources and points in time.

But there is a problem ahead: We only have plain HTML pages with all the content, headers, footers and technical stuff intertwined. See for example the Facebook site in the archive repository

This works, but the design and content is nearly unmaintainable. So, I think we need to extract all the articles and separate them from the technical stuff.

Here is my proposal on how to move on from here:

  1. We move to Grav CMS, as it is also currently used by my German fork of the site and working perfectly fine. It’s a simple flat-file content management system with everything we need baked-in (website).

  2. We create seperate markdown files for every entry, meaning every proposed software alternative. So, every entry (say “Matrix”) can be easily reused in multiple lists (say for “Whatsapp”, “Slack”, …).

  3. We do all of this via codeberg.org - meaning this git repository and issues like this one here. Codeberg is a non-profit non-tracking git hosting provider for Open Source projects like ours.

Here is what I would like this project to look like in the end, showcased via the German version:

  • Have a look at this page concerning Twitter.

  • The page itself is a folder with a simple file inside, that pulls all entries replacing twitter.

  • Every list entry is also a folder with a markdown+YAML file, like this file for Mastodon in this folder

  • That’s why it’s easy to create another list with the same entries but in a different order, see this Facebook page created by this file

  • An English example of all this can be found in the “use” and “replace” subfolders here.

What do you think of the three proposals? And who would like to help me pull this off? You can comment here (after creating an account) or text me on Mastodon.

Hi there, we've made it, **the website is back up and running!** Right now, [switching.software](https://switching.software) is a copy of every cached page of the switching.social website, pulled from different sources and points in time. But there is a problem ahead: We only have plain HTML pages with all the content, headers, footers and technical stuff intertwined. See for example [the Facebook site](https://codeberg.org/swiso-en/archive/src/branch/master/ethical-alternatives-to-facebook/index.html) in the [archive repository](https://codeberg.org/swiso-en/archive) This works, but the design and content is nearly unmaintainable. So, I think we need to extract all the articles and separate them from the technical stuff. Here is **my proposal** on how to move on from here: 1. **We move to Grav CMS**, as it is also currently used by [my German fork of the site](https://swiso.de) and working perfectly fine. It's a simple flat-file content management system with everything we need baked-in ([website](getgrav.org/)). 2. **We create seperate markdown files for every entry**, meaning every proposed software alternative. So, every entry (say "Matrix") can be easily reused in multiple lists (say for "Whatsapp", "Slack", ...). 3. **We do all of this via codeberg.org** - meaning this git repository and issues like this one here. Codeberg is a non-profit non-tracking git hosting provider for Open Source projects like ours. Here is what I would like this project to look like in the end, showcased via the German version: - Have a look at [this page](https://switchingsocial.de/ersetze/twitter) concerning Twitter. - The page itself is a [folder](https://gitlab.com/swisode/website/blob/primary/pages/ersetze/twitter) with [a simple file](https://gitlab.com/swisode/website/blob/primary/pages/ersetze/twitter/list.md#L7) inside, that pulls all entries replacing twitter. - Every list entry is also a folder with a markdown+YAML file, like [this file for Mastodon](https://gitlab.com/swisode/website/blob/primary/pages/nutze/mastodon/entry.md#L9) in [this folder](https://gitlab.com/swisode/website/blob/primary/pages/nutze/mastodon) - That's why it's easy to create another list with the same entries but in a different order, see [this Facebook page](https://switchingsocial.de/ersetze/facebook) created by [this file](https://gitlab.com/swisode/website/blob/primary/pages/ersetze/facebook/list.md#L7) - An English example of all this can be found in the "use" and "replace" subfolders [here](https://codeberg.org/swiso-en/website/src/branch/develop/pages). **What do you think of the three proposals?** And who would like to help me pull this off? You can comment here (after creating an account) or text me on [Mastodon](https://mstdn.swiso.org/@switchingsoftware).
Ghost commented 1 month ago

Hi, I guess the issues would be : a) cost, and b) look.

The cheaper the better obviously, but then if having the website hosted on Codeberg.org would be free but would also mean having the Codeberg.org header, the website would be less consistent.

I don’t know what are the costs nor look of any of those solutions, so I can’t give my opinion right now, but if your GitLab-based solution for the German website works like a charm and is easy to set up with yaml files, why not just applying it to the English version ?

Hi, I guess the issues would be : a) cost, and b) look. The cheaper the better obviously, but then if having the website hosted on Codeberg.org would be free but would also mean having the Codeberg.org header, the website would be less consistent. I don't know what are the costs nor look of any of those solutions, so I can't give my opinion right now, but if your GitLab-based solution for the German website works like a charm and is easy to set up with yaml files, why not just applying it to the English version ?
Jonah commented 1 month ago

The cheaper the better obviously, but then if having the website hosted on Codeberg.org would be free but would also mean having the Codeberg.org header, the website would be less consistent.

Pretty sure he’s saying the code will be hosted here on Codeberg, in this repository. The website will likely be hosted elsewhere, so there wouldn’t be Codeberg branding on the user-facing site.

I assume you have hosting lined up @brickup, but just in case: We (privacytools.io, or technically my company nabla.host) offer free web hosting (and Git hosting for that matter at git.privacytools.io) on our servers to privacy-oriented websites like this. Just wanted to extend that offer to you.

I think this sounds like a great idea overall. When I get a free moment I’ll try and move some of those entries in a Pull Request and/or see if I can get other PTIO team/community members to chip in :)

> The cheaper the better obviously, but then if having the website hosted on Codeberg.org would be free but would also mean having the Codeberg.org header, the website would be less consistent. Pretty sure he's saying the code will be hosted here on Codeberg, in this repository. The website will likely be hosted elsewhere, so there wouldn't be Codeberg branding on the user-facing site. I assume you have hosting lined up @brickup, but just in case: We (privacytools.io, or technically my company nabla.host) offer free web hosting (and Git hosting for that matter at git.privacytools.io) on our servers to privacy-oriented websites like this. Just wanted to extend that offer to you. I think this sounds like a great idea overall. When I get a free moment I'll try and move some of those entries in a Pull Request and/or see if I can get other PTIO team/community members to chip in :)
brickup commented 1 month ago
Owner

Yes, that’s what I’m saying. We are only hosting the git repository here, not the website :-)

And thanks a lot for the website hosting offer. Right now, I just quickly set up a Digital Ocean Droplet, as I wanted the site to be back online fast. But I would really appreciate better alternatives. Also maybe in a way, that I’m not the only person who has access to the server - single point of failure, you know :-)

I will contact you about this when we’ve made some progress with the codebase.

Yes, that's what I'm saying. We are only hosting the git repository here, not the website :-) And thanks a lot for the website hosting offer. Right now, I just quickly set up a Digital Ocean Droplet, as I wanted the site to be back online fast. But I would really appreciate better alternatives. Also maybe in a way, that I'm not the only person who has access to the server - single point of failure, you know :-) I will contact you about this when we've made some progress with the codebase.
jaredmoody commented 1 month ago
Collaborator

Have you considered using a static site generator instead of a CMS?

One potential downside I see of using a CMS is that it might be a little harder for people to contribute; i.e. the only people that can use the CMS are those that are granted access. With a static site, you have no user accounts or CMS backend to manage - contribution would just be via pull request to the source files.

Static HTML is also very easy to deploy and will use very few resources on the server.

Have you considered using a static site generator instead of a CMS? One potential downside I see of using a CMS is that it might be a little harder for people to contribute; i.e. the only people that can _use_ the CMS are those that are granted access. With a static site, you have no user accounts or CMS backend to manage - contribution would just be via pull request to the source files. Static HTML is also very easy to deploy and will use very few resources on the server.
supercell commented 1 month ago
Collaborator

I’m happy to help port the content over to Grav CMS (opened pull request #3), though I want to check, do we keep the ‘Ciao’ in the title or write something else?

I'm happy to help port the content over to Grav CMS (opened pull request [#3](https://codeberg.org/swiso-en/website/pulls/3)), though I want to check, do we keep the 'Ciao' in the title or write something else?
davidak commented 1 month ago

I like the structured content.

I second @jaredmoody and suggest to check out hugo. You can probably have similar structured content. Generation of the static html is just one command.

For hosting a static site, neocities might be a good option.

If the content is easily translatable would be also good, so any language could be maintained in one repository.

(i don’t have time to contribute in a significant way)

I like the structured content. I second @jaredmoody and suggest to check out [hugo](https://gohugo.io/). You can probably have similar structured content. Generation of the static html is just one command. For hosting a static site, [neocities](https://neocities.org/) might be a good option. If the content is easily translatable would be also good, so any language could be maintained in one repository. (i don't have time to contribute in a significant way)
brickup commented 1 month ago
Owner

Hi there,

thanks for all your comments! Yes, I guess a static site generator would be more appropriate. I just haven’t got any experiences using them, that’s why I started with Grav. I already had a quick look at Jekyll and the sites themselves look alike. So, I guess we can migrate the content quite easily from Grav to something else. Maybe I will give it a try after my vacation :smile:

I also thought about the translations. I think for now, we should focus on the English version only, as this is complex enough. But I guess, this could be some kind of vision for this project: To manage all languages together as a multi-language team in one repository (or with a common code base). That would be amazing. :+1:

PS: Also thanks for the first merge requests! :grin:

Hi there, thanks for all your comments! Yes, I guess a static site generator would be more appropriate. I just haven't got any experiences using them, that's why I started with Grav. I already had a quick look at Jekyll and the sites themselves look alike. So, I guess we can migrate the content quite easily from Grav to something else. Maybe I will give it a try after my vacation :smile: I also thought about the translations. I think for now, we should focus on the English version only, as this is complex enough. But I guess, this could be some kind of vision for this project: To manage all languages together as a multi-language team in one repository (or with a common code base). That would be amazing. :+1: PS: Also thanks for the first merge requests! :grin:
wtee commented 1 month ago

Thanks for getting the site back up! I’d be happy to help get it to a more maintainable state. Your proposal sounds good. I’m fine with using Grav or a static site generator as others have suggested.

Thanks for getting the site back up! I'd be happy to help get it to a more maintainable state. Your proposal sounds good. I'm fine with using Grav or a static site generator as others have suggested.
Jonah commented 1 month ago

But I guess, this could be some kind of vision for this project: To manage all languages together as a multi-language team in one repository (or with a common code base).

I love the Jekyll suggestion, but if this is a goal for the project you definitely should not go with Jekyll. That’s the number one reason we don’t have translations on the main privacytools.io repo, Jekyll has poor i18n support.

> But I guess, this could be some kind of vision for this project: To manage all languages together as a multi-language team in one repository (or with a common code base). I love the Jekyll suggestion, but if this is a goal for the project you definitely should not go with Jekyll. That's *the* number one reason we don't have translations on the main privacytools.io repo, Jekyll has poor i18n support.
juristi commented 1 month ago

Another vote for having the translations in the same repository. Feels a bit silly to spread the code base and effort into language specific projects. Even if in some cases the more heavily localized content could be appropriate, it probably could be handled within one project. Also it would get the latest updates included without having to separately merge them to a bunch of language specific repositories, they would be visible right away with English as fallback language instead.

It would be good to have some kind of decision on the technology before all the requests start rolling in.

(Probably won’t have any significant amount of time to contribute either, sorry :/ )

Another vote for having the translations in the same repository. Feels a bit silly to spread the code base and effort into language specific projects. Even if in some cases the more heavily localized content could be appropriate, it probably could be handled within one project. Also it would get the latest updates included without having to separately merge them to a bunch of language specific repositories, they would be visible right away with English as fallback language instead. It would be good to have some kind of decision on the technology before all the requests start rolling in. (Probably won't have any significant amount of time to contribute either, sorry :/ )
brickup commented 1 month ago
Owner

Hi there,

thanks for the info on Jekyll and multi-languages. On Grav, this works really good, I already use it (on https://brick.camp - bilingual).

I had a quick look at Hugo which seems to do it similar to Grav: Multiple markdown files with a language code in front of the extension.

I just returned from vacation and have quite a stressful week at work in front of me. But I will try to have a deeper look into Hugo and other options soon, so we come to a conclusion here. I can not wait to get this project rolling! :-)

In case this takes too long, we could keep using Grav, as I know for sure that it will get the job done. Even though it’s a bit more complex than needed. But I also prefer the option of a static site generator - even with my lack of experience in this domain.

Either way, I think migration of pages from html to markdown can continue even before a decision, as migrating from Grav to other makdown-based software will be straight-forward in my opinion.

Greetings Tobias

Hi there, thanks for the info on Jekyll and multi-languages. On Grav, this works really good, I already use it (on https://brick.camp - bilingual). I had a quick look at Hugo which seems to do it similar to Grav: Multiple markdown files with a language code in front of the extension. I just returned from vacation and have quite a stressful week at work in front of me. But I will try to have a deeper look into Hugo and other options soon, so we come to a conclusion here. I can not wait to get this project rolling! :-) In case this takes too long, we could keep using Grav, as I know for sure that it will get the job done. Even though it's a bit more complex than needed. But I also prefer the option of a static site generator - even with my lack of experience in this domain. Either way, I think migration of pages from html to markdown can continue even before a decision, as migrating from Grav to other makdown-based software will be straight-forward in my opinion. Greetings Tobias
brickup commented 1 month ago
Owner

Just a short update: I started messing around with Hugo today and think it can do the job. My goal now is to build a working prototype so there aren’t any nasty surprises later on.

Because right now, I’m not quite sure how to define the order of the list items properly (and language-dependent in the long run). Based on a “replace” taxonomy, one can use a weight per entry, but in case you want to have an item on two lists in different positions … difficult. I have some ideas I want to try there.

So, still some time needed but we’re slowly getting ahead :sweat_smile:

Just a short update: I started messing around with Hugo today and think it can do the job. My goal now is to build a working prototype so there aren't any nasty surprises later on. Because right now, I'm not quite sure how to define the order of the list items properly (and language-dependent in the long run). Based on a "replace" taxonomy, one can use [a weight per entry](https://gohugo.io/content-management/taxonomies/#order-taxonomies), but in case you want to have an item on two lists in different positions ... difficult. I have some ideas I want to try there. So, still some time needed but we're slowly getting ahead :sweat_smile:
jaredmoody commented 1 month ago
Collaborator

I’m happy to help with hugo stuff - do you want to start a hugo branch?

I'm happy to help with hugo stuff - do you want to start a hugo branch?
brickup commented 1 month ago
Owner

Done, see “hugo” :smile: Nothing fancy - maybe you have an idea for the ordering of entries in a list. I just created an “order” property in the yaml header of each list entry, but without any function yet.

EDIT: Got custom ordering of list entries working, see here.

Done, see "hugo" :smile: Nothing fancy - maybe you have an idea for the ordering of entries in a list. I just created an "order" property in the yaml header of each list entry, but without any function yet. EDIT: Got custom ordering of list entries working, see [here](https://codeberg.org/swiso-en/website/commit/c8dcd50e14e197be3a723a5d023c07fb191a3bd4).
brickup commented 3 weeks ago
Owner

Short update: I published the current state via codeberg’s static page feature:

https://pages.codeberg.org/swiso-en

Short update: I published the current state via codeberg's static page feature: [https://pages.codeberg.org/swiso-en](https://pages.codeberg.org/swiso-en)
brickup commented 3 weeks ago
Owner

I merged #5 today, so the migration from Grav to Hugo is completed. Also, all Google-related entries are finished thanks to @supercell.

Next, I’m gonna create a new issue concerning the remaining entries of switching.social. Then, we can close this issue, as we have a consensus on the technical foundation of this project.

Also, I’m gonna do a little mastodon post tomorrow concerning the progress. But for now, I need some sleep :grin:

I merged #5 today, so the migration from Grav to Hugo is completed. Also, all Google-related entries are finished thanks to @supercell. Next, I'm gonna create a new issue concerning the remaining entries of switching.social. Then, we can close this issue, as we have a consensus on the technical foundation of this project. Also, I'm gonna do a little mastodon post tomorrow concerning the progress. But for now, I need some sleep :grin:
brickup added the
in progress
label 3 weeks ago
Sign in to join this conversation.
No Milestone
No Assignees
8 Participants
Due Date

No due date set.

Loading…
Cancel
Save
There is no content yet.