Centralisation des journaux (pas encore Elastic) #40

Manually merged

jeltz merged 112 commits from logs-first-phase into master 2021-03-30 07:52:13 +02:00

Conversation 0 Commits 112 Files Changed 61 +1894 -934

112 Commits

Author	SHA1	Message	Date
jeltz	637b74a2ad	Fix some linter issues	2021-03-13 05:05:30 +01:00
jeltz	f45cd77510	Merge branch 'master' into logs-first-phase	2021-03-13 05:02:30 +01:00
jeltz	715d332d25	Merge pull request 'Migration des bases de données vers bdd.adm.auro.re' (#41) from bdd-saclay into master ... Reviewed-on: Aurore/ansible#41	2021-03-13 04:54:59 +01:00
otthorn	65c94d8e84	Fix trailing whitespace in the linter	2021-03-13 03:13:43 +01:00
otthorn	4150a77649	Remove vote from bdd.adm.auro.re	2021-03-13 03:09:21 +01:00
otthorn	a01a2095d6	Add passwords in all and vault for postgres db for wikijs, gitea, nextcloud, drone	2021-03-13 03:07:35 +01:00
otthorn	98171e449d	Update postgresql variable to match the new scheme in bdd.adm.auro.re	2021-03-13 02:48:16 +01:00
otthorn	8ec838962d	Postgresql is version 13 on bullseye	2021-03-13 02:35:39 +01:00
otthorn	bdc59049ae	Rename file for consistency	2021-03-13 02:35:03 +01:00
jeltz	2a6c2b30de	Merge pull request 'Rôle pour motd' (#38) from update_motd into master ... Reviewed-on: Aurore/ansible#38	2021-03-11 19:34:41 +01:00
jeltz	7a691882f3	Merge pull request 'Add databases for Grafana and CAS' (#37) from add_bdd_services into master ... Reviewed-on: Aurore/ansible#37	2021-03-11 19:33:24 +01:00
jeltz	69516012a2	Add databases for Grafana and CAS	2021-03-11 19:28:14 +01:00
pz2891	e5299d8087	Merge pull request 'Merge monitoring of new ups, and adapt threesold' (#33) from monitoring_ups into master ... Reviewed-on: Aurore/ansible#33 Reviewed-by: otthorn <otthorn@noreply.localhost> Reviewed-by: jeltz <jeltz@noreply.localhost>	2021-03-11 15:09:43 +01:00
pz2891	f6dfd792d3	Merge branch 'master' into monitoring_ups	2021-03-11 15:06:33 +01:00
jeltz	40cd5c71ad	Merge pull request 'Fix: keep the logs for 90 days' (#36) from logrotate-fix-90-jours into master ... Reviewed-on: Aurore/ansible#36	2021-03-11 14:38:17 +01:00
pz2891	472feb2dc0	Merge branch 'master' into monitoring_ups	2021-03-11 14:32:11 +01:00
jeltz	077858724a	Merge branch 'master' into logrotate-fix-90-jours	2021-03-11 14:30:48 +01:00
jeltz	38386fa1a0	Merge branch 'master' into logrotate-fix-90-jours	2021-03-11 14:28:45 +01:00
jeltz	7c7de51903	Merge pull request '[Docker] do not cache pip' (#35) from docker_best_pratices into master ... Reviewed-on: Aurore/ansible#35	2021-03-11 14:25:08 +01:00
otthorn	a31f57e844	Merge branch 'master' into monitoring_ups	2021-03-11 14:23:39 +01:00
jeltz	974fcff1d3	Merge pull request 'Add playbook to deploy sudo update on all machines' (#34) from sudo_update into master ... Reviewed-on: Aurore/ansible#34	2021-03-11 14:22:20 +01:00
pz2891	6125856c60	Merge branch 'monitoring_ups'	2021-03-11 14:06:38 +01:00
pz2891	a64864150f	Remove services-web from hosts	2021-03-11 13:33:03 +01:00
pz2891	d233fc2759	Update of threesold for warning battery	2021-03-11 13:23:15 +01:00
jeltz	ebb3c894da	Merge pull request 'Ansiblesation des VMs de BDD' (#32) from service-bdd-ovh into master ... Reviewed-on: Aurore/ansible#32	2021-03-10 20:18:15 +01:00
jeltz	df4bee2980	Add kanboard database to bdd-ovh	2021-03-10 20:14:02 +01:00
jeltz	6095d9cef9	Add 'no_log' for postgres passwords	2021-03-10 18:18:08 +01:00
jeltz	630377edad	Create users and databases on bdd-ovh	2021-03-10 17:21:58 +01:00
otthorn	b3fa8a455d	Add/Update password for postgres db codimd, etherpad and synapse	2021-03-10 17:13:56 +01:00
otthorn	5871e1cfb8	Add/Update password for postgres db codimd, etherpad and synapse	2021-03-10 17:13:13 +01:00
jeltz	d16f444130	Use a dict for HBA hosts	2021-03-10 15:59:21 +01:00
jeltz	4f6eda8329	Use /run instead of /var/run to please systemd	2021-03-10 15:57:19 +01:00
jeltz	628e11488d	Switch postgresql to english	2021-03-10 15:22:01 +01:00
jeltz	bd05b702bb	Use '::' in place of '[::]'	2021-03-10 15:19:39 +01:00
jeltz	06b54d5f89	Use postgresql_privs	2021-03-10 14:27:14 +01:00
jeltz	40eadf802c	Add template and no_log for postgresql_user	2021-03-10 13:58:40 +01:00
jeltz	8e855d7009	Listen addresses must be quoted	2021-03-10 13:36:10 +01:00
jeltz	7a07155237	Install python3-psycopg2 (required by Ansible)	2021-03-10 13:35:18 +01:00
jeltz	36b04239fd	Rename 'postgresql_db' to 'postgresql_databases'	2021-03-10 13:34:58 +01:00
jeltz	f919ec689a	Fix 'ansible_header' → 'ansible_managed'	2021-03-10 13:25:36 +01:00
jeltz	9ef6202fdf	Add configuration for users and databases	2021-03-10 13:23:52 +01:00
jeltz	879e033857	Fix malformed role definition	2021-03-10 13:21:19 +01:00
pz2891	ba4db4a835	Fix undefined variable	2021-03-10 12:58:51 +01:00
pz2891	bbf4ac323c	Moniroting of ups environmental temperature	2021-03-10 12:55:11 +01:00
pz2891	d815434360	Add new ups monitored	2021-03-10 12:53:28 +01:00
otthorn	76361de3f1	Add playbook for DBs	2021-03-10 12:52:15 +01:00
otthorn	69c6d5b55d	add and clean bdd hosts	2021-03-10 12:51:07 +01:00
otthorn	0656dacbe8	Add config for bdd local	2021-03-10 12:47:15 +01:00
otthorn	6951e017b7	bdd config for synapse	2021-03-10 12:43:08 +01:00
otthorn	1105ea88c1	rename VM to a simpler name (services-bdd-ovh -> bdd-ovh)	2021-03-10 12:40:17 +01:00
otthorn	8b9bef865e	postgresql listen on pseudo-address	2021-03-10 12:26:18 +01:00
otthorn	dbbaf0d26d	remove tailling whitespaces	2021-03-10 12:11:02 +01:00
otthorn	a4c393d3fb	fix yaml ci truthy value	2021-03-10 12:10:06 +01:00
otthorn	d14306a86c	fix syntax for CI	2021-03-10 12:08:05 +01:00
otthorn	a625a58ddd	create role postgresql_server	2021-03-10 12:01:32 +01:00
otthorn	64ae2a8521	configure postgres for services-bdd-ovh	2021-03-10 12:01:16 +01:00
jeltz	452b605fc2	Merge pull request 'Update the list of packages installed via baseconfig' (#30) from add_dnsutils into master ... Reviewed-on: Aurore/ansible#30	2021-03-07 23:39:55 +01:00
jeltz	2c0727a419	Update the list of packages installed via baseconfig	2021-03-07 21:41:42 +01:00
jeltz	41779fb172	Merge pull request 'Add backup root SSH keys' (#27) from add-ssh-keys into master ... Reviewed-on: Aurore/ansible#27	2021-03-07 21:30:38 +01:00
jeltz	deb4372588	Merge branch 'master' into add-ssh-keys	2021-03-07 21:29:57 +01:00
jeltz	2e912fc47a	Add recovery SSH keys for ynerant and otthorn	2021-03-07 21:23:09 +01:00
jeltz	929baa300f	Use 'update_motd' in 'prometheus_federate' (again)	2021-03-06 04:48:39 +01:00
jeltz	71ee06c9c0	Fix typo	2021-03-06 04:45:00 +01:00
jeltz	bc2701d8ba	Use 'update_motd' in 'prometheus_federate'	2021-03-06 04:43:09 +01:00
jeltz	2353589da6	Ensures /etc/update-motd.d exists	2021-03-06 04:42:21 +01:00
jeltz	1d0200a1f0	Use 'update_motd' in 'prometheus'	2021-03-06 04:32:06 +01:00
jeltz	b81600aef8	Use 'update_motd' in 'baseconfig'	2021-03-06 04:31:20 +01:00
jeltz	7e92fdfab7	Create an 'update_motd' role	2021-03-06 04:30:32 +01:00
jeltz	e6a86640de	Merge pull request 'Fetch switch_snmp jobs' (#29) from fix_exported_prometheus into master ... Reviewed-on: Aurore/ansible#29	2021-03-06 02:44:40 +01:00
jeltz	cf07de4ec4	Fetch switch_snmp jobs	2021-03-06 02:41:58 +01:00
jeltz	7cc478b1ad	Merge pull request 'Use label federated_instance instead of instance' (#28) from fix_exported_prometheus into master ... Reviewed-on: Aurore/ansible#28	2021-03-06 02:09:09 +01:00
jeltz	b9269f3967	Fix monitoring.yml indentation (yamllint warning)	2021-03-06 02:00:56 +01:00
jeltz	e5be09656b	Monitor yggdrasil from prometheus-aurore	2021-03-06 01:58:02 +01:00
jeltz	8abca7916f	Add switch_snmp job for prometheus	2021-03-06 01:57:32 +01:00
jeltz	763cc2eb51	Generate targets_switch_snmp.json	2021-03-06 01:57:08 +01:00
jeltz	eaa0d2e0fc	Fix bad indent in snmp.yml.j2	2021-03-06 01:56:18 +01:00
jeltz	8ae94fa8f8	Rename vault_snmp_switch{s,}_community	2021-03-06 01:08:51 +01:00
jeltz	21fed6ae3f	Add useful lookups for switchs interfaces	2021-03-06 00:58:46 +01:00
jeltz	52124d2cad	Cleanup prometheus_federate's prometheus.yml.j2	2021-03-06 00:46:13 +01:00
jeltz	7d527be1c0	Remove duplicate alerts from 'prometheus-federate'	2021-03-06 00:45:43 +01:00
jeltz	32669e1fb1	Don't load Django rules prometheus-federate	2021-03-06 00:44:22 +01:00
jeltz	4ca7ebd144	Add a unique exported label (useful for federation)	2021-03-06 00:40:44 +01:00
jeltz	802bfcc698	'prometheus-federate' must not retrieve its own federated metrics	2021-03-06 00:38:36 +01:00
jeltz	958eaa1bcb	Use label federated_instance instead of instance	2021-03-05 00:54:44 +01:00
jeltz	6525508401	Forward journald logs to rsyslog	2021-03-02 01:24:53 +01:00
jeltz	77a5fdac6f	Remove some duplicate logs from syslog.log	2021-03-02 00:56:28 +01:00
jeltz	5d319cf167	Define rsyslog_{inputs,outputs} for all hosts	2021-03-02 00:52:38 +01:00
jeltz	529550f594	Don't use 'imjournal' ('imuxsock' is already used) ... I still don't understand why it increased the size of the firewall logs by a factor of 5 to 10, but we don't really need structured logs from systemd-journald and the author seems to discourage it's use, so I will not investigate further.	2021-03-02 00:46:16 +01:00
jeltz	ee041b9ead	Use 'simple' instead of 'oneshot' (rotate service)	2021-03-02 00:14:25 +01:00
jeltz	1f6bfeee23	Fix broadcast address on routeur-aurore	2021-03-01 20:04:38 +01:00
jeltz	0f55b90de9	Remove 10.129.0.1 gateway on routeur-aurore-*	2021-03-01 20:04:02 +01:00
jeltz	b13b22da05	Add ignored destinations for firewall logs	2021-03-01 19:39:11 +01:00
jeltz	8f815a30c5	Remove useless date (already added by journald)	2021-03-01 17:47:12 +01:00
jeltz	acd5721a5b	Fix typos in rotate-remote-logs.service.j2	2021-03-01 17:45:17 +01:00
jeltz	9547868c7d	Send nginx logs to local syslog	2021-03-01 17:40:05 +01:00
jeltz	cdb9f88614	Do not rate limit collection of journald logs	2021-03-01 16:31:52 +01:00
jeltz	9eeb8ccd73	Remove non-Ansible SSH root keys	2021-03-01 16:08:08 +01:00
jeltz	1fe8d1d28b	Remove "Root Aurore" SSH key + add histausse key	2021-03-01 13:28:49 +01:00
jeltz	9252249d18	Use 'true' instead of 'yes'	2021-03-01 04:15:54 +01:00
jeltz	e4b58c0bf4	Fix typo in 20-collector.conf.j2	2021-03-01 04:07:17 +01:00
jeltz	c65b3f090b	Compress and delete old remote logs ... Logrotate is not used because I didn't found an easy way to configure it to handle the compression/deletion of log files already rotated by rsyslog (it is probably possible, but I found the script to be easier).	2021-03-01 03:58:58 +01:00
jeltz	f7183095c1	Add explicit permissions for directories	2021-03-01 02:26:22 +01:00
jeltz	ba8b4e8c29	Fix the ordering of rsyslog.d files ... A call to sendLogsToRemote for logs received through RELP/UDP has been added (to send them to Logstash/Redis/…), so common.conf's prefix must be lower than collector.conf's. Note: future "third-party" config files will also call sendLogsToRemote and thus will also have to use a prefix higher than 10.	2021-03-01 02:15:28 +01:00
jeltz	02a8cb84df	Add log.yml playbook	2021-03-01 01:29:16 +01:00
jeltz	4a43bf8a16	Add logging configuration for log.adm.auro.re	2021-03-01 01:28:30 +01:00
jeltz	7fd1b5ff5d	Add rsyslog_collector role	2021-03-01 01:27:56 +01:00
jeltz	6263c31785	Add rsyslog_common role	2021-03-01 01:27:30 +01:00
jeltz	89181c6cd6	Add log.adm.auro.re to inventory	2021-02-28 22:59:36 +01:00
jeltz	c3d24c1cd0	Add SSH key for Jeltz	2021-02-28 21:47:42 +01:00
otthorn	a35488efdd	[Docker] do not cache pip	2021-02-18 00:14:12 +01:00
otthorn	3050a95699	Add playbook to deploy sudo update on all machines	2021-01-27 14:36:14 +01:00
jeltz	85d0dc9621	Fix: keep the logs for 90 days	2021-01-23 17:19:50 +01:00