calcardbackup

🇩🇪 auf deutsch lesen...

This Bash script exports calendars and addressbooks from ownCloud/Nextcloud to .ics and .vcf files and saves them to a compressed file. Additional options are available.

Contents

• Requirements
• Quick Installation Guide
• Upgrading calcardbackup
• Migrating from GitHub to Codeberg
• Options
• File Naming Convention
• Usage Examples
• Nextcloud-Snap Users
• Syncloud Users
• Synology Users
• Does this also work with a broken ownCloud/Nextcloud instance?
• Can I use this without shell access to the server?
• Considerations about Encryption
• Links
• Get in touch
• Donations
• License

Requirements

• local installation of ownCloud/Nextcloud >= 5.0 with MySQL/MariaDB, PostgreSQL or SQLite3
• command line client appropriate for database type
• the user running the script needs to be able to read the full path to ownClouds/Nextclouds config.php, to the script itself and all used configuration files
• GNU Bash >= 4.2 (check with bash --version)
• optional: package gnupg to encrypt backup
• optional: package zip to compress backup as zip-file (instead of tar.gz)
• optional: package curl to let the script perform some additional checks

Quick Installation Guide

1. Clone the repository to your server (outside of webroot!) and enter the repo:
   git clone https://codeberg.org/BernieO/calcardbackup.git
cd calcardbackup

2. Change the ownership of repo to your webserver's user (here www-data):
   sudo chown -R www-data:www-data .

3. Run the script as your webserver's user (here www-data) and give as first argument the path to your ownCloud/Nextcloud instance (here /var/www/nextcloud):
   sudo -u www-data ./calcardbackup "/var/www/nextcloud"

4. Check output of script - it will tell, if it needs any other options.

5. Find your backup in directory backups/.

There are many more options available: have a look at sections Options and Usage examples.

optional: setup a cronjob for an automatic daily run

Once calcardbackup runs without errors, a cronjob can be setup to run it automatically each day as follows:

1. create a logfile and transfer its ownership to the webserver's user (here www-data):
   sudo touch /var/log/calcardbackup.log
sudo chown www-data:www-data /var/log/calcardbackup.log

2. edit the cron table of the webserver's user (here www-data) with:
   sudo crontab -u www-data -e
   and add the following line to the end of the file (change paths to fit your setup!):

   0 2 * * * /path/to/calcardbackup/calcardbackup "/var/www/nextcloud" > /var/log/calcardbackup.log 2>&1
   

Cron will now execute calcardbackup each day at 02:00am.
The output of the last run is written to /var/log/calcardbackup.log.

Take into account that percent signs (%) need to be escaped to prevent cron from changing them into newline characters.

Upgrading calcardbackup

If you followed the instructions in the Quick Installation Guide, you just need to pull the new version to upgrade to the latest version:

cd /path/to/calcardbackup
sudo -u www-data git pull

Migrating from GitHub to Codeberg

To migrate from GitHub to Codeberg, it is easiest to rename the folder containing the GitHub repository and then follow the Quick Installation Guide again. If configuration files and/or the backup folder reside in the folder with the GitHub repository, move them to the new location.

Options

All options can be specified in a configuration file or as command line arguments. If started with no options at all or only -b|--batch, the script attempts to use file calcardbackup.conf in the script's directory as configuration file.
If no configuration file via option -c|--configfile is given, the path to your ownCloud/Nextcloud instance must be the very first argument.
Find detailed description of all available options below.

Usage: ./calcardbackup [DIRECTORY] [option [argument]] [option [argument]] [option [argument]] ...

Arguments in capital letters to options are mandatory.
Paths (FILE / DIRECTORY) are absolute paths or relative paths to working directory.
If a path begins with a tilde-slash combinantion ~/, the tilde is replaced with ${HOME}

-a | --address URL
       Optional: pass URL of ownCloud/Nextcloud Installation to script.
-b | --batch
       Batch mode: print nothing to stdout, except for path to backup.
       Depending on configuration this will be:
         - absolute path of compressed backup file
       or, if run with option '-x|--uncompressed' (see below),
         - absolute path of directory containing uncompressed files
-c | --configfile FILE
       Read configuration from FILE. See 'examples/calcardbackup.conf.example'
       All other options except for '-b|--batch' will be ignored!
-d | --date FORMAT
       Use FORMAT as file name extension for backup directory or compressed backup file.
       FORMAT needs to be a format descriptor of the command date().
       The default is -%Y-%m-%d and will result in a directory or file
       named: 'calcardbackup-2017-03-23' or 'calcardbackup-2017-03-23.tar.gz'
       Check 'man date' for more info about different formats and syntax.
-e | --encrypt FILE
       Encrypt backup file with AES256 (gnupg). First line of FILE will be used as passphrase
-h | --help
       Print version number and a short help text 
-i | --include-shares
       Backup shared addressbooks/calendars, too, but only once: e.g. a shared calendar
       won't be backed up if the same calendar was already backed up for another user.
       NOTE: this option will be ignored if not used together with option '-u|--usersfile'.
-ltm | --like-time-machine N
       Keep all backups for the last N days, keep only backups created on mondays for the time before.
-na | --no-addressbooks
       Do not backup addressbooks
-nc | --no-calendars
       Do not backup calendars
-o | --output DIRECTORY
       Use directory DIRECTORY to store backups.
       If this option is not given, folder 'backups/' in script's directory is created and used.
-one | --one-file-per-component
       Save each calendar component (e.g. event) and each addressbook component to a separate file
       named: USERNAME-(CALENDARNAME|ADDRESSBOOKNAME)_UID.(ics|vcf)
       In this mode, calcardbackup does not modify the data read from the database except for
       adding CR+LF at the end of the lines according to RFC5545/RFC6350.
       Use this option to investigate faulty database entries or to migrate calendars/addressbooks
       to a Radicale caldav/carddav Server or to vdirsyncer.
-p | --snap
       This option is mandatory if you are running nextcloud-snap
       (https://github.com/nextcloud/nextcloud-snap). With this option, calcardbackup has to be
       run with sudo (even running as root without sudo will fail!).
-r | --remove N
       Remove backups older than N days from backup folder (N needs to be a positive integer).
-rmo | --read-mysql-optionfiles
       Read all MySQL/MariaDB option files (when connecting to a MySQL/MariaDB database).
       NOTE: if calcardbackup can't use filedescriptors, all MySQL/MariaDB option files are being
             read anyway.
-s | --selfsigned
       Let cURL ignore an untrustful (e.g. selfsigned) certificate.
       cURL is used to retrieve status.php of the ownCloud/Nextcloud installation to perform
       some additional checks. If cURL can't access the URL due to an untrustful certificate,
       calcardbackup will run without executing these checks. The same applies, if cURL is
       not installed.
-u | --usersfile FILE
       Give location of FILE, which contains users to be backed up. One user per line.
       See 'examples/users.txt.example'
-x | --uncompressed
       Do not compress backup folder
-z | --zip
       Use zip to compress backup folder instead of creating a gzipped tarball (tar.gz)

NOTE:  - Option '-f|--fetch-from-database' (introduced with calcardbackup 0.6.0) is set as
         default for calcardbackup >= 0.8.0, thus it has no function anymore.
       - Option '-g|--get-via-http' is removed with v2.0.0 (deprecated since v0.8.0, 30.10.2018).

File Naming Convention

The filenames of exported calendars and addressbooks are set together from the username, the displayname of the calendar/addressbook and a file extension (.ics for calendars, .webcal for calendarsubscriptions, .vcf for addressbooks):

Tom_Meetings.ics
Tom_Bundesliga.webcal
Tom_Contacts.vcf

In following cases an additional string is added to the filename (combinations are possible):

• existing files will never be overwritten. Instead, a number is added to the filename right before the item extension:

  Tom_Meetings_01.ics
  

• when using option -i|--include-shares the string _shared-by-USERNAME is added to the filename of shared calendars/addressbooks (Tom shares his calendar Meetings with Emma):

  Emma_Meetings_shared-by-Tom.ics
  

• when using option --one|--one-file-per-component, the UID of the according item is added to the filename:

  Tom_Meetings_1234-5678.ics
  

• only Nextcloud >= 22: the string _DEL-CLNDR is added to the filename of a deleted calendar:

  Tom_Meetings_DEL-CLNDR.ics
  

• only Nextcloud >= 22: if a calendar holds deleted components, these deleted components are exported to a seperate calendarfile. The string _DEL-CMPNTS is added to the filename:

  Tom_Meetings_DEL-CMPNTS.ics
  

Usage Examples

1. ./calcardbackup /var/www/nextcloud -nc -x
   Do not backup calendars (-nc) and store backed up files uncompressed (-x) in folder named calcardbackup-YYYY-MM-DD (default) under ./backups/ (default).

2. ./calcardbackup /var/www/nextcloud --no-calendars --uncompressed
   This is exactly the same command as above but with long options instead of short options.

3. ./calcardbackup -c /etc/calcardbackup.conf
   Use configuration file /etc/calcardbackup.conf (-c /etc/calcardbackup.conf). Parameters for desired behaviour have to be given in that file (see examples/calcardbackup.conf.example).
   Don't give any other command line options in this case, because they will be ignored (except for -b|--batch).

4. ./calcardbackup
   Use file calcardbackup.conf in the script's directory as configuration file.
   This is basically the same as example no.3, but with the default location of the configuration file.

5. ./calcardbackup /var/www/nextcloud -b -d .%d.%H -z -e /home/tom/key -o /media/data/backupfolder/ -u /etc/calcardbackupusers -i -r 15
   Suppress output except for path to the backup (-b), use file name extension .DD.HH (-d .%d.%H), zip backup (-z), encrypt the zipped backup with using the first line in file /home/tom/key as encryption-key (-e /home/tom/key), save backup in folder /media/data/backupfolder/ (-o /media/data/backupfolder/), only back up items of usernames given in file /etc/calcardbackupusers (-u /etc/calcardbackupusers), include users' shared address books/calendars (-i) and delete all backups older than 15 days (-r 15).

6. sudo ./calcardbackup /var/snap/nextcloud/current/nextcloud -p
   This example is for nextcloud-snap users. calcardbackup will use the cli utility from nextcloud-snap to access the database (-p) and backup all calendars/addressbooks found in the database.

7. ./calcardbackup /var/www/nextcloud -ltm 30 -r 180
   Keep all backups for the last 30 days, but keep only backups created on mondays for the time before (-ltm 30) and remove all backups older than 180 days (-r 180).
   ⚠️ Make sure backups are also created on mondays when using option -ltm

Nextcloud-Snap Users

If you are running Nextcloud-Snap, you have to use option -p|--snap to tell calcardbackup to use the cli utility nextcloud.mysql-client from the snap package.
In order for this to work, calcardbackup has to be run with sudo (even running as root without sudo will fail).
As path to Nextcloud use the path to the configuration files of nextcloud. In a standard installation this would be /var/snap/nextcloud/current/nextcloud. See usage example no.6.

Syncloud Users

When using syncloud you must run the script as user nextcloud, add the path to the snap binaries to the PATH variable and use option -p when running calcardbackup:

sudo -u nextcloud PATH="${PATH}:/snap/bin" ./calcardbackup "/var/snap/nextcloud/current/nextcloud" -p

There is also a small code example on how to use calcardbackup in the wiki of the syncloud platform.

Synology Users

In Synology DiskStation Manager (DSM) the path to mysql needs to be added to the PATH variable before running calcardbackup. Example:

sudo -u http PATH="${PATH}:/usr/local/mariadb10/bin" ./calcardbackup "/volume1/web/nextcloud"

Does this also work with a broken ownCloud/Nextcloud instance?

Yes, it does! 😃

calcardbackup only needs the database (and access to it) from an ownCloud/Nextcloud installation to be able to extract calendars/addressbooks from the database and save them as .ics and .vcf files.
Here is how this can be accomplished:

1. create a dummy Nextcloud directory including subdirectory config:
   mkdir -p /usr/local/bin/nextcloud_dummy/config

2. create and edit file config.php to fit your needs as follows:
   nano /usr/local/bin/nextcloud_dummy/config/config.php

   • add database type according to config.sample.php

   • for MySQL/MariaDB/PostgreSQL:

     • add according database values according to config.sample.php

   • for SQLite3:

     • add path to the nextcloud_dummy folder as 'datadirectory' according to config.sample.php
     • copy the SQLite3 database to the nexcloud_dummy directory (filename of the SQLite3 database must be owncloud.db):
       cp /path/to/owncloud.db /usr/local/bin/nextcloud_dummy/owncloud.db

   • if the database belongs to an installation of ownCloud <= 8.2, the following line needs to be added:
     'version' => '8.0.0',

3. run calcardbackup and give as first argument the path to dummy Nextcloud directory created in step 1:
   ./calcardbackup /usr/local/bin/nextcloud_dummy

Can I use this without shell access to the server?

It depends on the settings of the hoster if it is possible to use calcardbackup without shell access: additionally to the script's requirements, a PHP function to run shell commands needs to be enabled: either passthru(), system(), exec(), or shell_exec(). If none of those functions is enabled, there is no way to run calcardbackup.
calcardbackup_wrapper.php is included in the repo. It invokes calcardbackup, if one of these functions is found to be usable. Use it as follows:

1. upload calcardbackup and calcardbackup_wrapper.php into a folder of your webserver

2. modify the command in the configuration section of calcardbackup_wrapper.php to reflect your paths and add options to fit your needs.

3. navigate your browser to the uploaded calcardbackup_wrapper.php and check the output.

Due to a lot of different configurations at the webhosters, it might happen that calcardbackup doesn't work or not all options are available (some hosters disable the required PHP functions for security reasons, others might have them enabled, but do not allow to run external commands). So, depending on the output, you might need to tweak the options accordingly.

If the amount of calender/addressbook data is big, php timeouts might occur. These can be mitigated by running the script twice: once with option -na to only export calendars and the second time with option -nc to export only addressbooks. You could also backup only certain users with each run (with option -u).

⚠️ Prevent unauthorized downloads of the backed up files by password protecting the backup folder (e.g. with a .htaccess file).

I'd be happy if you could provide feedback whether or not this works with your hoster.

Considerations about Encryption

If you want to use the included encryption possibility (option -e), be aware that:

• the files are encrypted by GnuPG, AES256 with the passphrase given in a separate file
• the passphrase is stored in a file. Other users with access to the server might be able to see the passphrase.
• calcardbackup is designed to run without user interaction, so there can't be a rock solid encryption. I consider the offered one as sufficient in most cases though.
• if you need rock solid encryption, don't let calcardbackup encrypt the backup. Instead, encrypt it yourself.
• command to decrypt (you will be prompted to enter the passphrase):
  gpg -o OUTPUT_FILE -d FILE_TO_DECRYPT.GPG

Links

Related Forum Threads

• help.nextcloud.com - Nextcloud
• central.owncloud.org - ownCloud
• cardbook.icu - Cardbook (Thunderbird)

Blog Articles about calcardbackup

• tim-peters.org - August 2022 (german)
• tuxwise.net - February 2022 (english)
• lumanet.de - February 2021 (english)
• strobelstefan.org - December 2020 (german)
• www.patrickweber.info - August 2020 (german)
• Neil's Tech Docs - April 2020 (english)
• newtoypia.blogspot.com - April 2019 (taiwanese)
• bob.gatsmas.de - October 2018 (german)

Media Coverage

• Linux Magazine (235) - June 2020 (english)

Linux Packages

• aur.archlinux.org - calcardbackup in the Arch User Repository (wiki)

Docker Image for calcardbackup

• hub.docker.com - Docker Image by waja

ICS and VCF Standard

• RFC 5545 - Internet Calendaring and Scheduling Core Object Specification (iCalendar)
• RFC 6350 - vCard Format Specification

Exporter Plugins from SabreDAV used by ownCloud/Nextcloud

• ICSExportPlugin.php - ICS Exporter public function mergeObjects
• VCFExportPlugin.php - VCF Exporter public function generateVCF

Get in Touch

Create an issue, drop a comment at my blogpost about calcardbackup or send an email.

Donations

calcardbackup is and will remain free open source software.
If you find calcardbackup useful, you could support the developer by making a donation. Any amount is appreciated!

• IBAN: DE58 1203 0000 1032 8146 40 (BIC: BYLADEM1001)
• PayPal (Friends): bernieo.code@gmx.de

License

Copyright (C) 2017 Bernhard Ostertag bernieo.code@gmx.de

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.