Signed-off-by: Bernhard Ostertag <bernieo.code@gmx.de> |
||
---|---|---|
examples | ||
.gitignore | ||
LICENSE.txt | ||
README.md | ||
README_GER.md | ||
calcardbackup | ||
calcardbackup_wrapper.php | ||
changelog.md |
README.md
calcardbackup
This Bash script exports calendars and addressbooks from ownCloud/Nextcloud to .ics and .vcf files and saves them to a compressed file. Additional options are available.
Contents
- Requirements
- Quick Installation Guide
- Upgrading calcardbackup
- Migrating from GitHub to Codeberg
- Options
- File Naming Convention
- Usage Examples
- Nextcloud-Snap Users
- Syncloud Users
- Synology Users
- Does this also work with a broken ownCloud/Nextcloud instance?
- Can I use this without shell access to the server?
- Considerations about Encryption
- Links
- Get in touch
- Donations
- License
Requirements
- local installation of ownCloud/Nextcloud >= 5.0 with MySQL/MariaDB, PostgreSQL or SQLite3
- command line client appropriate for database type
- the user running the script needs to be able to read the full path to ownClouds/Nextclouds
config.php
, to the script itself and all used configuration files - GNU Bash >= 4.2 (check with
bash --version
) - optional: package
gnupg
to encrypt backup - optional: package
zip
to compress backup as zip-file (instead of tar.gz) - optional: package
curl
to let the script perform some additional checks
Quick Installation Guide
-
Clone the repository to your server (outside of webroot!) and enter the repo:
git clone https://codeberg.org/BernieO/calcardbackup.git
cd calcardbackup
-
Change the ownership of repo to your webserver's user (here
www-data
):
sudo chown -R www-data:www-data .
-
Run the script as your webserver's user (here
www-data
) and give as first argument the path to your ownCloud/Nextcloud instance (here/var/www/nextcloud
):
sudo -u www-data ./calcardbackup "/var/www/nextcloud"
-
Check output of script - it will tell, if it needs any other options.
-
Find your backup in directory
backups/
.
There are many more options available: have a look at sections Options and Usage examples.
optional: setup a cronjob for an automatic daily run
Once calcardbackup runs without errors, a cronjob can be setup to run it automatically each day as follows:
-
create a logfile and transfer its ownership to the webserver's user (here
www-data
):
sudo touch /var/log/calcardbackup.log
sudo chown www-data:www-data /var/log/calcardbackup.log
-
edit the cron table of the webserver's user (here
www-data
) with:
sudo crontab -u www-data -e
and add the following line to the end of the file (change paths to fit your setup!):0 2 * * * /path/to/calcardbackup/calcardbackup "/var/www/nextcloud" > /var/log/calcardbackup.log 2>&1
Cron will now execute calcardbackup each day at 02:00am.
The output of the last run is written to /var/log/calcardbackup.log
.
Take into account that percent signs (%
) need to be escaped to prevent cron from changing them into newline characters.
Upgrading calcardbackup
If you followed the instructions in the Quick Installation Guide, you just need to pull the new version to upgrade to the latest version:
cd /path/to/calcardbackup
sudo -u www-data git pull
Migrating from GitHub to Codeberg
To migrate from GitHub to Codeberg, it is easiest to rename the folder containing the GitHub repository and then follow the Quick Installation Guide again. If configuration files and/or the backup folder reside in the folder with the GitHub repository, move them to the new location.
Options
All options can be specified in a configuration file or as command line arguments. If started with no options at all or only -b|--batch
, the script attempts to use file calcardbackup.conf
in the script's directory as configuration file.
If no configuration file via option -c|--configfile
is given, the path to your ownCloud/Nextcloud instance must be the very first argument.
Find detailed description of all available options below.
Usage: ./calcardbackup [DIRECTORY] [option [argument]] [option [argument]] [option [argument]] ...
Arguments in capital letters to options are mandatory.
Paths (FILE / DIRECTORY) are absolute paths or relative paths to working directory.
If a path begins with a tilde-slash combinantion ~/, the tilde is replaced with ${HOME}
-a | --address URL
Optional: pass URL of ownCloud/Nextcloud Installation to script.
-b | --batch
Batch mode: print nothing to stdout, except for path to backup.
Depending on configuration this will be:
- absolute path of compressed backup file
or, if run with option '-x|--uncompressed' (see below),
- absolute path of directory containing uncompressed files
-c | --configfile FILE
Read configuration from FILE. See 'examples/calcardbackup.conf.example'
All other options except for '-b|--batch' will be ignored!
-d | --date FORMAT
Use FORMAT as file name extension for backup directory or compressed backup file.
FORMAT needs to be a format descriptor of the command date().
The default is -%Y-%m-%d and will result in a directory or file
named: 'calcardbackup-2017-03-23' or 'calcardbackup-2017-03-23.tar.gz'
Check 'man date' for more info about different formats and syntax.
-e | --encrypt FILE
Encrypt backup file with AES256 (gnupg). First line of FILE will be used as passphrase
-h | --help
Print version number and a short help text
-i | --include-shares
Backup shared addressbooks/calendars, too, but only once: e.g. a shared calendar
won't be backed up if the same calendar was already backed up for another user.
NOTE: this option will be ignored if not used together with option '-u|--usersfile'.
-ltm | --like-time-machine N
Keep all backups for the last N days, keep only backups created on mondays for the time before.
-na | --no-addressbooks
Do not backup addressbooks
-nc | --no-calendars
Do not backup calendars
-o | --output DIRECTORY
Use directory DIRECTORY to store backups.
If this option is not given, folder 'backups/' in script's directory is created and used.
-one | --one-file-per-component
Save each calendar component (e.g. event) and each addressbook component to a separate file
named: USERNAME-(CALENDARNAME|ADDRESSBOOKNAME)_UID.(ics|vcf)
In this mode, calcardbackup does not modify the data read from the database except for
adding CR+LF at the end of the lines according to RFC5545/RFC6350.
Use this option to investigate faulty database entries or to migrate calendars/addressbooks
to a Radicale caldav/carddav Server or to vdirsyncer.
-p | --snap
This option is mandatory if you are running nextcloud-snap
(https://github.com/nextcloud/nextcloud-snap). With this option, calcardbackup has to be
run with sudo (even running as root without sudo will fail!).
-r | --remove N
Remove backups older than N days from backup folder (N needs to be a positive integer).
-rmo | --read-mysql-optionfiles
Read all MySQL/MariaDB option files (when connecting to a MySQL/MariaDB database).
NOTE: if calcardbackup can't use filedescriptors, all MySQL/MariaDB option files are being
read anyway.
-s | --selfsigned
Let cURL ignore an untrustful (e.g. selfsigned) certificate.
cURL is used to retrieve status.php of the ownCloud/Nextcloud installation to perform
some additional checks. If cURL can't access the URL due to an untrustful certificate,
calcardbackup will run without executing these checks. The same applies, if cURL is
not installed.
-u | --usersfile FILE
Give location of FILE, which contains users to be backed up. One user per line.
See 'examples/users.txt.example'
-x | --uncompressed
Do not compress backup folder
-z | --zip
Use zip to compress backup folder instead of creating a gzipped tarball (tar.gz)
NOTE: - Option '-f|--fetch-from-database' (introduced with calcardbackup 0.6.0) is set as
default for calcardbackup >= 0.8.0, thus it has no function anymore.
- Option '-g|--get-via-http' is removed with v2.0.0 (deprecated since v0.8.0, 30.10.2018).
File Naming Convention
The filenames of exported calendars and addressbooks are set together from the username, the displayname of the calendar/addressbook and a file extension (.ics
for calendars, .webcal
for calendarsubscriptions, .vcf
for addressbooks):
Tom_Meetings.ics
Tom_Bundesliga.webcal
Tom_Contacts.vcf
In following cases an additional string is added to the filename (combinations are possible):
- existing files will never be overwritten. Instead, a number is added to the filename right before the item extension:
Tom_Meetings_01.ics
- when using option
-i|--include-shares
the string_shared-by-USERNAME
is added to the filename of shared calendars/addressbooks (Tom shares his calendar Meetings with Emma):Emma_Meetings_shared-by-Tom.ics
- when using option
--one|--one-file-per-component
, the UID of the according item is added to the filename:Tom_Meetings_1234-5678.ics
- only Nextcloud >= 22: the string
_DEL-CLNDR
is added to the filename of a deleted calendar:Tom_Meetings_DEL-CLNDR.ics
- only Nextcloud >= 22: if a calendar holds deleted components, these deleted components are exported to a seperate calendarfile. The string
_DEL-CMPNTS
is added to the filename:Tom_Meetings_DEL-CMPNTS.ics
Usage Examples
-
./calcardbackup /var/www/nextcloud -nc -x
Do not backup calendars (-nc
) and store backed up files uncompressed (-x
) in folder namedcalcardbackup-YYYY-MM-DD
(default) under ./backups/ (default). -
./calcardbackup /var/www/nextcloud --no-calendars --uncompressed
This is exactly the same command as above but with long options instead of short options. -
./calcardbackup -c /etc/calcardbackup.conf
Use configuration file /etc/calcardbackup.conf (-c /etc/calcardbackup.conf
). Parameters for desired behaviour have to be given in that file (see examples/calcardbackup.conf.example).
Don't give any other command line options in this case, because they will be ignored (except for-b|--batch
). -
./calcardbackup
Use file calcardbackup.conf in the script's directory as configuration file.
This is basically the same as example no.3, but with the default location of the configuration file. -
./calcardbackup /var/www/nextcloud -b -d .%d.%H -z -e /home/tom/key -o /media/data/backupfolder/ -u /etc/calcardbackupusers -i -r 15
Suppress output except for path to the backup (-b
), use file name extension .DD.HH (-d .%d.%H
), zip backup (-z
), encrypt the zipped backup with using the first line in file /home/tom/key as encryption-key (-e /home/tom/key
), save backup in folder /media/data/backupfolder/ (-o /media/data/backupfolder/
), only back up items of usernames given in file /etc/calcardbackupusers (-u /etc/calcardbackupusers
), include users' shared address books/calendars (-i
) and delete all backups older than 15 days (-r 15
). -
sudo ./calcardbackup /var/snap/nextcloud/current/nextcloud -p
This example is for nextcloud-snap users. calcardbackup will use the cli utility from nextcloud-snap to access the database (-p
) and backup all calendars/addressbooks found in the database. -
./calcardbackup /var/www/nextcloud -ltm 30 -r 180
Keep all backups for the last 30 days, but keep only backups created on mondays for the time before (-ltm 30
) and remove all backups older than 180 days (-r 180
).
⚠️ Make sure backups are also created on mondays when using option-ltm
Nextcloud-Snap Users
If you are running Nextcloud-Snap, you have to use option -p|--snap
to tell calcardbackup to use the cli utility nextcloud.mysql-client
from the snap package.
In order for this to work, calcardbackup has to be run with sudo
(even running as root without sudo
will fail).
As path to Nextcloud use the path to the configuration files of nextcloud. In a standard installation this would be /var/snap/nextcloud/current/nextcloud
. See usage example no.6.
Syncloud Users
When using syncloud you must run the script as user nextcloud
, add the path to the snap binaries to the PATH
variable and use option -p
when running calcardbackup:
sudo -u nextcloud PATH="${PATH}:/snap/bin" ./calcardbackup "/var/snap/nextcloud/current/nextcloud" -p
There is also a small code example on how to use calcardbackup in the wiki of the syncloud platform.
Synology Users
In Synology DiskStation Manager (DSM) the path to mysql
needs to be added to the PATH
variable before running calcardbackup. Example:
sudo -u http PATH="${PATH}:/usr/local/mariadb10/bin" ./calcardbackup "/volume1/web/nextcloud"
Does this also work with a broken ownCloud/Nextcloud instance?
Yes, it does! 😃
calcardbackup only needs the database (and access to it) from an ownCloud/Nextcloud installation to be able to extract calendars/addressbooks from the database and save them as .ics and .vcf files.
Here is how this can be accomplished:
-
create a dummy Nextcloud directory including subdirectory
config
:
mkdir -p /usr/local/bin/nextcloud_dummy/config
-
create and edit file
config.php
to fit your needs as follows:
nano /usr/local/bin/nextcloud_dummy/config/config.php
-
add database type according to config.sample.php
-
for MySQL/MariaDB/PostgreSQL:
- add according database values according to config.sample.php
-
for SQLite3:
- add path to the nextcloud_dummy folder as 'datadirectory' according to config.sample.php
- copy the SQLite3 database to the nexcloud_dummy directory (filename of the SQLite3 database must be
owncloud.db
):
cp /path/to/owncloud.db /usr/local/bin/nextcloud_dummy/owncloud.db
-
if the database belongs to an installation of ownCloud <= 8.2, the following line needs to be added:
'version' => '8.0.0',
-
-
run calcardbackup and give as first argument the path to dummy Nextcloud directory created in step 1:
./calcardbackup /usr/local/bin/nextcloud_dummy
Can I use this without shell access to the server?
It depends on the settings of the hoster if it is possible to use calcardbackup without shell access: additionally to the script's requirements, a PHP function to run shell commands needs to be enabled: either passthru()
, system()
, exec()
, or shell_exec()
. If none of those functions is enabled, there is no way to run calcardbackup.
calcardbackup_wrapper.php
is included in the repo. It invokes calcardbackup, if one of these functions is found to be usable. Use it as follows:
-
upload
calcardbackup
andcalcardbackup_wrapper.php
into a folder of your webserver -
modify the command in the configuration section of
calcardbackup_wrapper.php
to reflect your paths and add options to fit your needs. -
navigate your browser to the uploaded
calcardbackup_wrapper.php
and check the output.
Due to a lot of different configurations at the webhosters, it might happen that calcardbackup doesn't work or not all options are available (some hosters disable the required PHP functions for security reasons, others might have them enabled, but do not allow to run external commands). So, depending on the output, you might need to tweak the options accordingly.
If the amount of calender/addressbook data is big, php timeouts might occur. These can be mitigated by running the script twice: once with option -na
to only export calendars and the second time with option -nc
to export only addressbooks. You could also backup only certain users with each run (with option -u
).
⚠️ Prevent unauthorized downloads of the backed up files by password protecting the backup folder (e.g. with a .htaccess
file).
I'd be happy if you could provide feedback whether or not this works with your hoster.
Considerations about Encryption
If you want to use the included encryption possibility (option -e
), be aware that:
- the files are encrypted by GnuPG, AES256 with the passphrase given in a separate file
- the passphrase is stored in a file. Other users with access to the server might be able to see the passphrase.
- calcardbackup is designed to run without user interaction, so there can't be a rock solid encryption. I consider the offered one as sufficient in most cases though.
- if you need rock solid encryption, don't let calcardbackup encrypt the backup. Instead, encrypt it yourself.
- command to decrypt (you will be prompted to enter the passphrase):
gpg -o OUTPUT_FILE -d FILE_TO_DECRYPT.GPG
Links
Related Forum Threads
- help.nextcloud.com - Nextcloud
- central.owncloud.org - ownCloud
- cardbook.icu - Cardbook (Thunderbird)
Blog Articles about calcardbackup
- tim-peters.org - August 2022 (german)
- tuxwise.net - February 2022 (english)
- strobelstefan.org - December 2020 (german)
- www.patrickweber.info - August 2020 (german)
- Neil's Tech Docs - April 2020 (english)
- newtoypia.blogspot.com - April 2019 (taiwanese)
- bob.gatsmas.de - October 2018 (german)
Media Coverage
- Linux Magazine (235) - June 2020 (english)
Linux Packages
- aur.archlinux.org - calcardbackup in the Arch User Repository (wiki)
Docker Image for calcardbackup
- hub.docker.com - Docker Image by waja
ICS and VCF Standard
- RFC 5545 - Internet Calendaring and Scheduling Core Object Specification (iCalendar)
- RFC 6350 - vCard Format Specification
Exporter Plugins from SabreDAV used by ownCloud/Nextcloud
- ICSExportPlugin.php - ICS Exporter
public function mergeObjects
- VCFExportPlugin.php - VCF Exporter
public function generateVCF
Get in Touch
Create an issue, drop a comment at my blogpost about calcardbackup or send an email.
Donations
calcardbackup is and will remain free open source software.
If you find calcardbackup useful, you could support the developer by making a donation. Any amount is appreciated!
- IBAN:
DE58 1203 0000 1032 8146 40
(BIC:BYLADEM1001
) - PayPal (Friends):
bernieo.code@gmx.de
License
Copyright (C) 2017 Bernhard Ostertag bernieo.code@gmx.de
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.