Prepare nodes#
This section contains instructions on how to prepare your systems for an IC installation with EclecticIQ Intelligence Center playbooks.
Requirements#
You must have the following before running the playbooks:
Credentials for https://downloads.eclecticiq.com.
One host to act as an Ansible control node.
One or more hosts to act as Ansible managed nodes.
Prepare control and managed nodes#
Before you run the playbooks, follow the instructions in this section to prepare your control and managed nodes.
Control node v.s. Managed nodes
Your control node must be separate from managed nodes.
Your control node is where the ansible-playbook
is run, and managed nodes are what ansible-playbook
acts upon.
Running ansible-playbook
directly from managed nodes (instead of using a separate control node) can lead to unexpected behavior during playbook runs.
Prepare control node#
Control node v.s. Managed nodes
Your control node must be separate from managed nodes. See Prepare control and managed nodes.
You need at least one host to act as a control node.
This is the host where you run the ansible-playbook
command.
Your control node must have:
Python 3.8 and newer.
SSH access to managed nodes, as a user with sudo privileges.
Download and unarchive playbook package#
Tip
You may need to navigate to https://downloads.eclecticiq.com/ on your browser and log in before attempting to open the URLs below.
Download the latest EclecticIQ Intelligence Center installation playbooks
from https://downloads.eclecticiq.com/platform/ansible/2.x
and unarchive it.
You must
Use your https://downloads.eclecticiq.com credentials (
<EIQ_USER>:<EIQ_PASSWORD>
)
export PLAYBOOK_PACKAGE="eiq-ic-playbooks-2x-<timestamp>.zip"
curl -O -u '<EIQ_USER>:<EIQ_PASSWORD>' https://downloads.eclecticiq.com/platform/ansible/2.x/$PLAYBOOK_PACKAGE
# Example: curl -O -u user%40eclecticiq.com:sUp3rs3cr3tP4ssw0rd https://...
unzip $PLAYBOOK_PACKAGE
Tip
Save and version control your playbook configuration. Playbook configurations document your deployment, and can be used to administer and upgrade your EclecticIQ Intelligence Center instance.
Do not store credentials in version control.
Prepare Python environment#
Install Python 3.8 or newer.
Navigate to the EclecticIQ Intelligence Center installation playbooks directory.
cd ~/path/to/ic-playbooks
(Recommended) Version control your playbook configuration
Save and version control your playbook configuration. Playbook configurations document your deployment, and can be used to administer and upgrade your EclecticIQ Intelligence Center instance.
Do not store credentials in version control.
(Optional) Create and activate a Python virtual environment:
python3 -m venv .venv source .venv/bin/activate
Install Python dependencies:
python3 -m pip install -r python-requirements.txt
Set up secrets#
Create an .envrc
file and set the following environment variables.
You can use .envrc-example
as a template.
For upgrades
If you are setting up the .envrc
file to upgrade
an existing EclecticIQ Intelligence Center instance,
retrieve these values from the /etc/eclecticiq/platform_settings.py
file on your existing instance.
# Credentials for https://downloads.eclecticiq.com
export EIQ_REPO_USERNAME='user@example'
export EIQ_REPO_PASSWORD='super_secret_example_password'
# Set passwords for services
export EIQ_DB_PASSWORD='super_secret_example_password'
export EIQ_NEO4J_PASSWORD='super_secret_example_password'
export EIQ_NEO4JBATCHER_PASSWORD='super_secret_example_password'
export EIQ_REDIS_PASSWORD='super_secret_example_password'
# This variable must be defined if PG replication is being configured for 11host only
export PG_REPLICA_USER_PASSWORD="${EIQ_DB_PASSWORD}"
# IC secret. Set this to a random string.
# You can generate this with $(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1)
export EIQ_SECRET_KEY='2ziYbm7ZsbwzXtwQt4AfeEfY3hvCrHzv'
# Set credentials for initial user on EclecticIQ Intelligence Center
export EIQ_ADMIN_USERNAME='admin'
export EIQ_ADMIN_EMAIL='[email protected]'
# Password requirements: at least 10 char with on uppercase, one lowercase, one number and 1 symbol
# We use some sort of password strength validation
export EIQ_ADMIN_PASSWORD='super_secret_example_passw0rd#'
Once done, load these values from the .envrc
file by running:
source .envrc
Tip
For information on how to use Ansible with remote users that require passwords for
sudo
and SSH keys, see Appendix.
Prepare inventory file#
Select and modify the inventory file that suits your deployment architecture.
This package provides pre-configured inventory files for the following deployments:
Deployment |
Inventory file |
1 node |
|
2 node |
|
4 node |
|
6 node |
|
11 node |
|
Tip
Use at least 6 nodes when deploying for production.
Edit that inventory file,
and change only the ansible_host
values so that they point to the
correct target managed node.
This is the inventory file that you must
invoke when running the ansible-playbook
command.
For example, for a 6 node deployment,
modify the inventories/ic-6host-prod.yml
file:
---
ic6host: &6host
hosts:
application-node:
ansible_host: 10.10.2.11
pg-node-01:
ansible_host: 10.10.2.12
elastic-node-01:
eiq_ic_elasticsearch__node_name: elastic01
ansible_host: 10.10.2.13
elastic-node-02:
eiq_ic_elasticsearch__node_name: elastic02
ansible_host: 10.10.2.14
elastic-node-03:
eiq_ic_elasticsearch__node_name: elastic03
ansible_host: 10.10.2.15
neo4j-node-01:
ansible_host: 10.10.2.16
// ...
When you run the ansible-playbook
command,
you must include this inventory file:
ansible-playbook -i inventories/ic-6host-prod.yml <playbook_name.yml>
Prepare ansible.cfg file#
Create an ansible.cfg
file at the project root
and set the following values.
Use ansible-example.cfg
as a template.
[defaults]
interpreter_python=/usr/bin/python3
roles_path=roles
collections_path=collections
remote_user=ec2-user
private_key_file=~/.ssh/path_to_your_key
Tip
For more configuration options, see Ansible: Ansible Configuration Settings
Parameter |
Example |
Description |
---|---|---|
|
|
(Optional) Path to Python interpreter on control node. |
|
|
Path to Ansible roles. This package provides required roles in the |
|
|
Path to Ansible collections. Thsi package provides required collections in the |
|
|
User name on managed nodes. This user must have sudo privileges. Control node must have SSH access to this user account on managed nodes. |
|
|
SSH private key file on control node. This should provide SSH access to a user account with sudo privileges on managed nodes. |
Set IC version to install#
Set EclecticIQ Intelligence Center version to install by modifying group_vars/all/config-all.yml
.
In that file, change only these values:
Parameter |
Description |
---|---|
|
Specify an IC version to install.
You must write this as Default: Tip The playbooks will always install the latest version of EclecticIQ Intelligence Center within the specified minor version. |
|
Specify a version of EclecticIQ Intelligence Center public API package to install. This must match the EclecticIQ Intelligence Center being installed/upgraded to. The following is a list of IC versions and
their compatible
|
The following table lists other
parameters in the config-all.yml
file.
Do not change these values unless otherwise instructed.
Parameter |
Description |
---|---|
|
Default: |
|
Data directory on managed nodes.
Related to filesystem mounts.
Default: |
|
Default: |
|
Default: |
|
Do not change. List of values. Default: - eclecticiq-extension-all~={{ eiq_ic_version }}.0
- "eclecticiq-extension-api=={{ eiq_ic_api_version }}"
|
|
Default: |
|
Default: |
|
Default: |
|
Default: |
|
Do not change. |
|
Default: |
|
Default: |
Prepare managed nodes#
Recommended
Managed nodes should be separate from the control node.
Each managed node must have:
One of the operating systems supported by EclecticIQ Intelligence Center:
CentOS 7.9
RHEL 7.9
System Python 3.6.8 or newer.
Must be the same Python interpreter used by
libselinux-python3
.libselinux-python3
Filesystem mounts where required.
Install libselinux-python3#
On RedHat-based systems (RHEL, CentOS, etc.),
you must install libselinux-python3
on all managed nodes
to allow Ansible to execute playbook tasks on them.
Run as root:
yum install -y libselinux-python3
Note
Installing libselinux-python3
on CentOS 7 and RHEL 7
should also automatically ensure that a compatible
version of Python 3 is installed.
Allow incoming requests for services#
When performing a multi-node install (2 nodes and above), you may need to configure managed nodes to allow incoming connections from EclecticIQ Intelligence Center application node for certain services to work.
The table below lists EclecticIQ Intelligence Center services and the ports they listen on:
Service |
Port(s) |
---|---|
Ingestion, Celery |
|
PostgreSQL |
|
Redis |
|
Elasticsearch |
|
Neo4j |
|
Filesystem mounts#
(Optional) We recommend provisioning your managed nodes with volume mounts at the following locations:
On all managed nodes
On all managed nodes, we recommend mounting
/var/log
on its own volume:
Component |
Mount point |
Min. size (GB) |
Recommended size (GB) |
---|---|---|---|
Logs |
|
10 |
20 |
Mount volumes for data stores
The following describes volume mount and sizing recommendations for individual services/node hosting those services:
Component |
Mount point |
Min. size (GB) |
Recommended size (GB) |
---|---|---|---|
Elasticsearch |
|
100 |
150 |
Neo4j |
|
25 |
50 |
PostgreSQL |
|
200 |
300 |
Redis |
|
10 |
20 |