WIP: backups with borg and borgmatic #39

Manually merged

jeltz merged 80 commits from backups into master 2021-03-30 07:52:12 +02:00

Conversation 0 Commits 80 Files Changed 52 +1123 -207

80 Commits

Author	SHA1	Message	Date
jeltz	3f3f688da4	Use 'present' instead of 'latest' (ansible-lint)	2021-03-15 07:51:48 +01:00
jeltz	6713b550b6	Merge branch 'master' into backups	2021-03-15 07:50:11 +01:00
jeltz	cb3ec07121	Use 'inventory_hostname' instead of 'ansible_fqdn' ... While 'ansible_fdqn' can be changed by a compromised host, 'inventory_hostname' can't (hopefully). It should therefore no longer be possible for the said host to access the backups of another host.	2021-03-15 07:25:09 +01:00
otthorn	243ec1fe9d	[borgbackup_client] VaRi0u5 f1X3s	2021-03-15 01:04:42 +01:00
otthorn	e12f67c920	[borgbackup_client] ignore some Strech machines	2021-03-15 01:03:34 +01:00
otthorn	f8e5f0cc76	Manually cherrypicked from 65c94d8e84	2021-03-14 23:56:07 +01:00
otthorn	a54006c9d4	Various fixes	2021-03-14 23:39:12 +01:00
otthorn	6f36506a98	rm a _bak file	2021-03-14 22:09:27 +01:00
otthorn	41eb446114	small fix	2021-03-14 22:08:29 +01:00
otthorn	7480a7c565	[borgbackup_client] precedence rules and sain defaults for borg config	2021-03-14 22:02:34 +01:00
otthorn	59f2c94a61	Custom borgbackup configuration for the logs	2021-03-14 21:14:37 +01:00
otthorn	e570ce67b3	[borgbackup_client] do not backup /var/log/	2021-03-14 19:23:04 +01:00
otthorn	b14b359027	[borgbackup_client] add exlude path to conf	2021-03-14 19:21:15 +01:00
otthorn	33a1ec02f3	[borgbackup_client] update config directory to be homogeneous	2021-03-14 19:07:02 +01:00
otthorn	ebfc4f2a26	[borgbackup_client] do update cache	2021-03-14 19:03:44 +01:00
otthorn	0b5562f3f4	Add litl.adm.auro.re	2021-03-14 18:57:44 +01:00
jeltz	86f8b31159	Delegate facts for borgbackup_client	2021-03-14 18:44:13 +01:00
jeltz	d9f1104309	Move id_remote to /etc/borgmatic	2021-03-14 18:42:26 +01:00
otthorn	c6cae75031	[borgbackup_server] fix /borg permissions	2021-03-14 18:29:33 +01:00
otthorn	46d10022ea	[borgbackup_client] fix rentention date to int and list correctly source directories	2021-03-14 18:24:36 +01:00
otthorn	ff750c5b63	[borgbackup_client] remove 1 minute sleep and fix verbosity	2021-03-14 18:23:44 +01:00
otthorn	2651432582	[WIP] various fixes	2021-03-14 18:22:52 +01:00
otthorn	c5afbdbde4	remove borg_host_dir	2021-03-14 18:22:17 +01:00
otthorn	d928c7f7f0	[borgbackup_client] rename variable correclty	2021-03-14 16:11:40 +01:00
otthorn	021a5ef1e8	[borgbackup_client] various fixes for ssh keys	2021-03-14 16:11:18 +01:00
jeltz	c99b611b8f	Various fixes	2021-03-14 14:17:36 +01:00
jeltz	8112788396	[borgbackup_client] Add 'user:' in authorized_key	2021-03-14 13:18:30 +01:00
jeltz	2f2f71422f	[borgbackup_client] Move some handlers to tasks	2021-03-14 13:16:08 +01:00
jeltz	25e05069de	Merge pull request 'Monitor more switchs using Prometheus' (#42) from fix_exported_prometheus into master ... Reviewed-on: Aurore/ansible#42	2021-03-14 11:53:45 +01:00
jeltz	ac42401d6d	Merge branch 'master' into fix_exported_prometheus	2021-03-14 10:20:05 +01:00
jeltz	a43a9839f8	Monitor more switchs	2021-03-14 10:19:55 +01:00
jeltz	11578494ec	Merge pull request 'Centralisation des journaux (pas encore Elastic)' (#40) from logs-first-phase into master ... Reviewed-on: Aurore/ansible#40	2021-03-13 05:06:33 +01:00
jeltz	637b74a2ad	Fix some linter issues	2021-03-13 05:05:30 +01:00
jeltz	f45cd77510	Merge branch 'master' into logs-first-phase	2021-03-13 05:02:30 +01:00
jeltz	715d332d25	Merge pull request 'Migration des bases de données vers bdd.adm.auro.re' (#41) from bdd-saclay into master ... Reviewed-on: Aurore/ansible#41	2021-03-13 04:54:59 +01:00
otthorn	65c94d8e84	Fix trailing whitespace in the linter	2021-03-13 03:13:43 +01:00
otthorn	4150a77649	Remove vote from bdd.adm.auro.re	2021-03-13 03:09:21 +01:00
otthorn	a01a2095d6	Add passwords in all and vault for postgres db for wikijs, gitea, nextcloud, drone	2021-03-13 03:07:35 +01:00
otthorn	98171e449d	Update postgresql variable to match the new scheme in bdd.adm.auro.re	2021-03-13 02:48:16 +01:00
otthorn	8ec838962d	Postgresql is version 13 on bullseye	2021-03-13 02:35:39 +01:00
otthorn	bdc59049ae	Rename file for consistency	2021-03-13 02:35:03 +01:00
otthorn	1611f4a93c	[utils] move the sudo_upgrade playbook with other utils	2021-03-12 01:59:03 +01:00
otthorn	904678d1ac	[borgbackup_client] Add borg encryption passphrase to vault	2021-03-12 01:49:04 +01:00
otthorn	965bbe62a4	[borgbackup_client] configure encryption passphrase and storage	2021-03-12 01:46:35 +01:00
otthorn	d7a4995496	[borgbackup_client] backup the ZFS datasets	2021-03-12 01:06:48 +01:00
otthorn	d16676bfb6	[borgbackup_client] hostname should also be unique, but more legible than fqdn	2021-03-12 01:05:58 +01:00
otthorn	cdcfad7ac2	[borgbackup_client] backup /var/ also (for now)	2021-03-12 01:05:19 +01:00
otthorn	29f2823960	[borgbackup_client] tell borg that a postgresql database lies on re2o-bdd	2021-03-12 00:32:08 +01:00
otthorn	454bc66ae0	[borgbackup_client] backup /etc/ everywhere	2021-03-12 00:30:00 +01:00
otthorn	3f8ffbe164	[borgbackup_client] Add borg username and group defaults	2021-03-12 00:01:11 +01:00
otthorn	4123af6c01	[borgbackup_client] Install client on all machines (servers, vms and lxc containers)	2021-03-11 23:56:07 +01:00
otthorn	531f7593d2	[borgbackup_client] fix identation	2021-03-11 23:37:55 +01:00
otthorn	313314a674	[borgbackup_client] fix risky file permission on apt config for pinning version	2021-03-11 23:36:27 +01:00
otthorn	4642395330	[borgbackup_client] Add initial role defintion	2021-03-11 23:29:57 +01:00
jeltz	f0f56ecd3f	Fix linter-related issues	2021-03-11 20:15:35 +01:00
jeltz	7cf616f6c7	Add playbook for backups	2021-03-11 20:09:26 +01:00
jeltz	2ea7f6f9f7	Add host_vars for perceval	2021-03-11 20:09:09 +01:00
jeltz	db8dbb6c7a	Add borgbackup_server role	2021-03-11 20:08:41 +01:00
jeltz	6525508401	Forward journald logs to rsyslog	2021-03-02 01:24:53 +01:00
jeltz	77a5fdac6f	Remove some duplicate logs from syslog.log	2021-03-02 00:56:28 +01:00
jeltz	5d319cf167	Define rsyslog_{inputs,outputs} for all hosts	2021-03-02 00:52:38 +01:00
jeltz	529550f594	Don't use 'imjournal' ('imuxsock' is already used) ... I still don't understand why it increased the size of the firewall logs by a factor of 5 to 10, but we don't really need structured logs from systemd-journald and the author seems to discourage it's use, so I will not investigate further.	2021-03-02 00:46:16 +01:00
jeltz	ee041b9ead	Use 'simple' instead of 'oneshot' (rotate service)	2021-03-02 00:14:25 +01:00
jeltz	1f6bfeee23	Fix broadcast address on routeur-aurore	2021-03-01 20:04:38 +01:00
jeltz	0f55b90de9	Remove 10.129.0.1 gateway on routeur-aurore-*	2021-03-01 20:04:02 +01:00
jeltz	b13b22da05	Add ignored destinations for firewall logs	2021-03-01 19:39:11 +01:00
jeltz	8f815a30c5	Remove useless date (already added by journald)	2021-03-01 17:47:12 +01:00
jeltz	acd5721a5b	Fix typos in rotate-remote-logs.service.j2	2021-03-01 17:45:17 +01:00
jeltz	9547868c7d	Send nginx logs to local syslog	2021-03-01 17:40:05 +01:00
jeltz	cdb9f88614	Do not rate limit collection of journald logs	2021-03-01 16:31:52 +01:00
jeltz	9252249d18	Use 'true' instead of 'yes'	2021-03-01 04:15:54 +01:00
jeltz	e4b58c0bf4	Fix typo in 20-collector.conf.j2	2021-03-01 04:07:17 +01:00
jeltz	c65b3f090b	Compress and delete old remote logs ... Logrotate is not used because I didn't found an easy way to configure it to handle the compression/deletion of log files already rotated by rsyslog (it is probably possible, but I found the script to be easier).	2021-03-01 03:58:58 +01:00
jeltz	f7183095c1	Add explicit permissions for directories	2021-03-01 02:26:22 +01:00
jeltz	ba8b4e8c29	Fix the ordering of rsyslog.d files ... A call to sendLogsToRemote for logs received through RELP/UDP has been added (to send them to Logstash/Redis/…), so common.conf's prefix must be lower than collector.conf's. Note: future "third-party" config files will also call sendLogsToRemote and thus will also have to use a prefix higher than 10.	2021-03-01 02:15:28 +01:00
jeltz	02a8cb84df	Add log.yml playbook	2021-03-01 01:29:16 +01:00
jeltz	4a43bf8a16	Add logging configuration for log.adm.auro.re	2021-03-01 01:28:30 +01:00
jeltz	7fd1b5ff5d	Add rsyslog_collector role	2021-03-01 01:27:56 +01:00
jeltz	6263c31785	Add rsyslog_common role	2021-03-01 01:27:30 +01:00
jeltz	89181c6cd6	Add log.adm.auro.re to inventory	2021-02-28 22:59:36 +01:00