Back up and restore with playbooks#
Danger
These playbooks are provided mainly to aid in the upgrade process. This is not a long-term backup solution.
Backup & restore is a destructive operation.
The provided playbooks are destructive.
Backup and restore operations will remove the existing contents of EIQ_IC_BACKUP_ROOT
before beginning.
Ensure that the EIQ_IC_BACKUP_ROOT
environment variable is set to:
a path on a volume with sufficient free space
a unique path on corresponding this backup operation
Limitations:
File based backups are not suitable for long-term archival use-cases, due to incompatible files formats between major software updates.
The playbooks must be run on control node with a large amount of storage.
A stable and fast network connection is required.
You can back up and restore your EclecticIQ Intelligence Center installation with instructions provided here.
The playbooks here provide a
basic file-based backup and restore by
copying data and configuration files to ${EIQ_IC_BACKUP_ROOT}/eiq.backup
on your control node
filesystem.
Requirements#
Manage backup destination:
You must manually manage your backup destination. By default, this is set to
${EIQ_IC_BACKUP_ROOT:-.}/eiq.backup
, or./eiq.backup
when you run the playbooks.Running the
pb-backup*.yml
playbooks more than once will overwrite the contents of${EIQ_IC_BACKUP_ROOT:-.}/eiq.backup
.Use version control to manage your backup destination, or change
EIQ_IC_BACKUP_ROOT
each time you make a backup of your system(s).
Tip
To change your backup destination, see Prepare nodes.
Playbook requirements:
Caution
You may have used an .envrc
file in earlier versions of the installation playbooks.
Rename it to .env
,
or create an .env
file using instructions in Set up .env.
You must have an existing
.env
file, or reconstruct an.env
file from the/etc/eclecticiq/platform_settings.py
of an existing EclecticIQ Intelligence Center installation. See Prepare control node.You must have an existing inventory file. If you do not have an existing inventory file, create a new file from examples available in
inventories/
that corresponds to your deployment configuration. See Prepare control node.
Control node requirements:
Control node must have sufficient disk space at
EIQ_IC_BACKUP_ROOT
. If theEIQ_IC_BACKUP_ROOT
environment variable is not set when you run the playbooks, it defaults to your working directory when running the playbooks (./
).
Managed node(s) requirements:
EclecticIQ Intelligence Center services and workers must not be running.
You must stop all database services (e.g. PostgreSQL, Elasticsearch, and Redis processes).
When performing a backup, your resulting backup must form a contemporaneous snapshot of the state of your system(s). This means that you must run the intended
pb-backup*.yml
playbooks and allow them to complete before attempting to restart any of these services.Conversely, when restoring from a given backup, you must restore all the backups created for a given snapshot before attempting to restart any of these services.
List of playbooks#
Playbook file |
Description |
---|---|
|
Back up EclecticIQ Intelligence Center configuration files. Files backed up with this playbook must be restored manually. |
|
Back up Elasticsearch configuration and data directory. |
|
Back up PostgreSQL configuration and data directory. |
|
Back up Redis configuration and data directory. |
|
Back up Kibana configuration and custom dashboards. Kibana has a separate set of backup instructions. See Run Kibana backup playbook. |
|
Restore Elasticsearch configuration and data directory. |
|
Restore PostgreSQL configuration and data directory. |
|
Restore Redis configuration and data directory. |
|
Restore Kibana configuration and custom dashboards. |
To have a working backup, you must at least back up your:
EclecticIQ Intelligence Center configuration (
pb-backup-config.yml
)PostgreSQL data and configuration (
pb-backup-pg.yml
)
Run Kibana backup playbook#
You must run the Kibana backup playbook (pb-backup_kibana.yml
)
before attempting an Upgrade with playbooks to preserve custom configuration and dashboards.
Before following instructions here, you must make sure that:
Kibana and Elasticsearch are running
Control node must be able to connect to Kibana and Elasticsearch
To back up Kibana configuration and dashboards,
Make sure your
.env
file is loaded:source .env
Run the
pb-backup_kibana.yml
playbook. Kibana and Elasticsearch must be running.ansible-playbook -i inventories/<inventory_file.yml> pb-backup_kibana.yml
When the backup completes, make sure that ic-kibana-objects.ndjson
:
Is successfully created in your backup directory on the control node.
Contains expected contents (e.g. named custom dashboards).
Run backup playbooks#
Tip
Run commands with --tags=all,logstash
if your deployment includes Logstash.
E.g. ansible-playbook --tags=all,logstash -i inventories/ic-prod-large.yml ...
Make sure your
.env
file is loaded:source .env
Stop all services and workers. Run
pb-stop_services.yml
:ansible-playbook -i inventories/<inventory_file.yml> pb-stop_services.yml
Run the backup playbooks:
ansible-playbook -i inventories/<inventory_file.yml> pb-backup_config.yml ansible-playbook -i inventories/<inventory_file.yml> pb-backup-pg.yml ansible-playbook -i inventories/<inventory_file.yml> pb-backup-es.yml ansible-playbook -i inventories/<inventory_file.yml> pb-backup-redis.yml
Wait for the playbooks to complete.
Once playbooks have completed, start all services:
ansible-playbook -i inventories/<inventory_file.yml> pb-start_services.yml
Run restore playbooks#
Caution
EclecticIQ Intelligence Center configuration backups (pb-backup_config.yml
) must be manually restored.
EIQ_IC_BACKUP_ROOT
must point to an existing backup location.
Tip
Run commands with --tags=all,logstash
if your deployment includes Logstash.
E.g. ansible-playbook --tags=all,logstash -i inventories/ic-prod-large.yml ...
Make sure your
.env
file is loaded:source .env
Stop all services and workers. Run
pb-stop_services.yml
:ansible-playbook -i inventories/<inventory_file.yml> pb-stop_services.yml
Run the restore playbooks:
ansible-playbook -i inventories/<inventory_file.yml> pb-restore-pg.yml ansible-playbook -i inventories/<inventory_file.yml> pb-restore-es.yml ansible-playbook -i inventories/<inventory_file.yml> pb-restore-redis.yml
Wait for the playbooks to complete.
Once playbooks have completed, start all services:
ansible-playbook -i inventories/<inventory_file.yml> pb-start_services.yml
Run Kibana restore playbook#
Run the Kibana restore playbook to restore Kibana configuration and dashboards. If you have any custom Kibana configuration or dashboards, you must run this to make them available.
Before following instructions here, you must make sure that:
Kibana and Elasticsearch are running
Control node must be able to connect to Kibana and Elasticsearch
ic-kibana-objects.ndjson
must be present in the backup destination directory on your control node.
To restore up Kibana configuration and dashboards,
Make sure your
.env
file is loaded:source .env
Run Kibana restore playbook:
pb-restore-kibana.yml
The playbook expects
ic-kibana-objects.ndjson
, to be in the backup destination directory on the control node.ansible-playbook -i inventories/ic-test-small.yml pb-restore-kibana.yml