Prepare nodes#
This section contains instructions on how to prepare your systems for an EclecticIQ Intelligence Center installation with Ansible playbooks.
Requirements#
You must have the following before running the playbooks:
Credentials for https://downloads.eclecticiq.com.
One host to act as an Ansible control node.
One or more hosts to act as Ansible managed nodes.
Upgrading from CentOS 7/RHEL 7
If you are installing or upgrading EcelcticIQ Intelligence Center on managed on older operating systems (e.g. CentOS 7, RHEL 7), you must upgrade the operating system to Rocky Linux 8 or RHEL 8 before following the instructions here.
See:
Deployment guidance#
Managed nodes should follow guidance here: Deployment architecture guidance
Prepare control and managed nodes#
Before you run the playbooks, follow the instructions in this section to prepare your control and managed nodes.
Control node v.s. Managed nodes
Your control node must be separate from managed nodes.
Your control node is where the ansible-playbook
is run.
Managed nodes are what ansible-playbook
acts upon.
Avoid running ansible-playbook
directly from managed nodes
(instead of using a separate control node) –
this can lead to unexpected behavior.
Prepare control node#
Control node v.s. Managed nodes
Your control node must be separate from managed nodes. See Prepare control and managed nodes.
You need at least one host to act as a control node.
This is the host where you run the ansible-playbook
command.
Your control node must have:
Python 3.8 and newer.
SSH access to managed nodes, as a user with sudo privileges.
Download and unarchive playbook package#
Tip
You may need to navigate to https://downloads.eclecticiq.com/ on your browser and log in before attempting to open the URLs below.
Download the latest EclecticIQ Intelligence Center installation playbooks from https://downloads.eclecticiq.com/platform/ansible/3.x and unarchive it.
Tip
Save and version control your playbook configuration. Playbook configurations document your deployment, and can be used to administer and upgrade your EclecticIQ Intelligence Center instance.
Do not store credentials in version control.
Prepare Python environment#
Install Python 3.8 or newer.
Navigate to the EclecticIQ Intelligence Center installation playbooks directory.
cd ~/path/to/ic-playbooks
(Recommended) Version control your playbook configuration
Save and version control your playbook configuration. Playbook configurations document your deployment, and can be used to administer and upgrade your EclecticIQ Intelligence Center instance.
Do not store credentials in version control.
(Optional) Create and activate a Python virtual environment:
python3 -m venv .venv source .venv/bin/activate
Install Python dependencies:
python3 -m pip install -r python-requirements.txt
Set up .env#
Caution
You may have used an .envrc
file in earlier versions of the installation playbooks.
Rename it to .env
to follow the instructions here.
Create an .env
file.
You can use env-example
as a template.
Tip
Instead of using environment variables,
you can directly modify group_vars/all/config.yml
instead with these values.
You must set values for all these environment variables:
# Recommend that you generate passwords with the below command:
# printf '~^'; cat /dev/urandom | LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1
# Password requirements: at least 10 char with on uppercase, one lowercase, one number and 1 symbol
# https://docs.eclecticiq.com/ic/current/install-configure-upgrade/rocky/install-the-ic/before-you-start/#user-name-and-password
export EIQ_IC_VERSION="3.3"
export EIQ_REPO_USERNAME="user@example"
export EIQ_REPO_PASSWORD=<example_password>
export EIQ_DB_PASSWORD=<example_password>
# This variable should be defined if PG replication is being configured
export PG_REPLICA_USER_PASSWORD="${EIQ_DB_PASSWORD}"
export EIQ_REDIS_PASSWORD=<example_password>
export EIQ_ES_ELASTIC_PASSWORD=<example_password>
export EIQ_ES_KIBANA_SYSTEM_PASSWORD=<example_password>
export EIQ_ES_LOGSTASH_SYSTEM_PASSWORD=<example_password>
export EIQ_PLATFORM_SECRET_KEY=<example_password>
export EIQ_ADMIN_USERNAME="admin"
export EIQ_ADMIN_PASSWORD=<example_password>
export EIQ_ADMIN_EMAIL="[email protected]"
# Pre-configure Intelligence Center. If all thee following variables are
# set, automatic diagnostics suite will be run after deployment.
# Hostname or IP at which TIP will be accessed
export EIQ_CONFIGURATIONS_SERVER_SERVER_NAME="127.0.0.1"
# From address for TIP emails
export EIQ_CONFIGURATIONS_EMAIL_FROM_EMAIL="[email protected]"
# Reply-to address for TIP emails
export EIQ_CONFIGURATIONS_EMAIL_REPLY_TO="${EIQ_ADMIN_EMAIL}"
export IC_ADMIN_USERNAME="${EIQ_ADMIN_USERNAME}"
export IC_ADMIN_PASSWORD="${EIQ_ADMIN_PASSWORD}"
# Destination for (possibly large) backups on the ansible controller
export EIQ_IC_BACKUP_ROOT="."
# these should be set for automated configuration of RHEL optional repos (for perl dependencies of PostgreSQL)
# Customer who don't wish to supply credentials, should manually enable the repos needed to install perl and perl libraries from RHEL
export RHEL_USERNAME='rh_user'
export RHEL_PASSWORD=<example password>
# Define hosts/IPs, based on the chosen inventory
export EIQ_HOST_IC_APP_01="127.0.0.1" # This is the only manadatory variable, present in all inventories
export EIQ_HOST_IC_APP_02="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_01="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_02="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_03="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_04="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_05="127.0.0.1"
export EIQ_HOST_IC_CELERY="127.0.0.1"
export EIQ_HOST_IC_INGESTION="127.0.0.1"
export EIQ_HOST_IC_WORKER_01="127.0.0.1"
export EIQ_HOST_IC_WORKER_02="127.0.0.1"
export EIQ_HOST_PG_NODE_01="127.0.0.1"
export EIQ_HOST_PG_NODE_02="127.0.0.1"
export EIQ_HOST_REDIS_NODE_01="127.0.0.1"
export EIQ_HOST_REDIS_NODE_02="127.0.0.1"
export EIQ_HOST_NEO4J_NODE_01="127.0.0.1" # only required for Neo4j removal
The following sections describes environment variables used and their possible values.
Target software versions#
Set versions of software to install or upgrade to.
# Target IC version to install
export EIQ_IC_VERSION="3.1"
# Do not change
export EIQ_PYTHON_VERSION="3.8"
Credentials#
Set credentials for IC and its components.
If you are upgrading from an existing installation,
you must copy existing credential entries from that instance. They can be found in /etc/eclecticiq/platform_settings.py
.
# Credentials for https://downloads.eclecticiq.com
export EIQ_REPO_USERNAME="user@example"
export EIQ_REPO_PASSWORD="<example_password>"
# Password for PostgreSQL instance
export EIQ_DB_PASSWORD="<example_password>"
# This variable should be defined if PG replication is being configured
export PG_REPLICA_USER_PASSWORD="${EIQ_DB_PASSWORD}"
# Password for Redis
export EIQ_REDIS_PASSWORD="<example_password>"
# Secret token for IC. Set this to a long string.
export EIQ_PLATFORM_SECRET_KEY="<example_password>"
# Admin credentials for IC
export EIQ_ADMIN_USERNAME="admin"
export EIQ_ADMIN_PASSWORD="<example_password>"
export EIQ_ADMIN_EMAIL="[email protected]"
export IC_ADMIN_USERNAME="${EIQ_ADMIN_USERNAME}"
export IC_ADMIN_PASSWORD="${EIQ_ADMIN_PASSWORD}"
System settings#
These environment variables set system settings for certain tasks to run.
# Destination for (possibly large) backups on the ansible controller
export EIQ_IC_BACKUP_ROOT="."
## Required for RHEL only.
# If not provided here, must be manually provisioned on RHEL nodes.
# Set your Red Hat Enterprise Linux subscription credentials here
# For pulling required PostgreSQL Perl dependencies.
export RHEL_USERNAME='rh_user'
export RHEL_PASSWORD=<example password>
Intelligence Center settings#
These environment variables set Intelligence Center system settings:
# Hostname or IP at which TIP will be accessed
export EIQ_CONFIGURATIONS_SERVER_SERVER_NAME="127.0.0.1"
# From address for TIP emails
export EIQ_CONFIGURATIONS_EMAIL_FROM_EMAIL="[email protected]"
# Reply-to address for TIP emails
export EIQ_CONFIGURATIONS_EMAIL_REPLY_TO="${EIQ_ADMIN_EMAIL}"
Swap file#
You can set the playbooks to configure the swap file on managed nodes by setting these environment variables.
# Set the swap file size you aim for:
# By default, set to "24G"
export EIQ_SWAP_FILE_SIZE="24G"
# Enable using SWAP space
# By default, set to "false"
export EIQ_ENABLE_SWAP_CONFIG="true"
Inventory configuration#
These playbooks allow you to configure
network addresses of managed nodes
without manually changing inventory files in
the inventories/
folder.
These must still be used in conjunction with inventory
files. For example, setting EIQ_HOST_IC_APP_01=<target_ip>
here
allows you to run ansible-playbook -i inventories/ic-test-small.yml ...
without modifying the ic-test-small.yml
inventory file.
Tip
Instead of using environment variables,
you can directly modify group_vars/all/config.yml
instead with these values.
# Define hosts/IPs, based on the chosen inventory
export EIQ_HOST_IC_APP_01="127.0.0.1" # This is the only manadatory variable, present in all inventories
export EIQ_HOST_IC_APP_02="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_01="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_02="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_03="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_04="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_05="127.0.0.1"
export EIQ_HOST_IC_CELERY="127.0.0.1"
export EIQ_HOST_IC_INGESTION="127.0.0.1"
export EIQ_HOST_IC_WORKER_01="127.0.0.1"
export EIQ_HOST_IC_WORKER_02="127.0.0.1"
export EIQ_HOST_PG_NODE_01="127.0.0.1"
export EIQ_HOST_PG_NODE_02="127.0.0.1"
export EIQ_HOST_REDIS_NODE_01="127.0.0.1"
export EIQ_HOST_REDIS_NODE_02="127.0.0.1"
export EIQ_HOST_NEO4J_NODE_01="127.0.0.1" # only required for Neo4j removal
Each deployment type/inventory file requires you to set different environment variables.
The following list shows which environment variables are required per deployment type:
inventories/ic-test-small.yml
EIQ_HOST_IC_APP_01
inventories/ic-test-two.yml
EIQ_HOST_IC_APP_01
EIQ_HOST_PG_NODE_01
inventories/ic-test-medium.yml
EIQ_HOST_IC_APP_01
EIQ_HOST_ELASTIC_NODE_01
EIQ_HOST_ELASTIC_NODE_02
EIQ_HOST_ELASTIC_NODE_03
inventories/ic-test-large.yml
EIQ_HOST_IC_APP_01
EIQ_HOST_IC_WORKER_01
EIQ_HOST_PG_NODE_01
EIQ_HOST_REDIS_NODE_01
EIQ_HOST_ELASTIC_NODE_01
EIQ_HOST_ELASTIC_NODE_02
inventories/ic-prod-medium-pg1.yml
EIQ_HOST_IC_APP_01
EIQ_HOST_PG_NODE_01
EIQ_HOST_ELASTIC_NODE_01
EIQ_HOST_ELASTIC_NODE_02
EIQ_HOST_ELASTIC_NODE_03
inventories/ic-prod-medium.yml
EIQ_HOST_IC_APP_01
EIQ_HOST_PG_NODE_01
EIQ_HOST_PG_NODE_02
EIQ_HOST_ELASTIC_NODE_01
EIQ_HOST_ELASTIC_NODE_02
EIQ_HOST_ELASTIC_NODE_03
inventories/ic-prod-large.yml
EIQ_HOST_IC_INGESTION
EIQ_HOST_IC_APP_01
EIQ_HOST_IC_CELERY
EIQ_HOST_PG_NODE_01
EIQ_HOST_PG_NODE_02
EIQ_HOST_REDIS_NODE_01
EIQ_HOST_REDIS_NODE_02
EIQ_HOST_ELASTIC_NODE_01
EIQ_HOST_ELASTIC_NODE_02
EIQ_HOST_ELASTIC_NODE_03
Prepare ansible.cfg file#
Create an ansible.cfg
file at the project root
and set the following values.
Use ansible-example.cfg
as a template.
[defaults]
interpreter_python=auto_silent
roles_path=roles
collections_path=collections
remote_user=example-user
private_key_file=~/.ssh/path_to_your_key_id_rsa
[ssh_connection]
ssh_args = -o ServerAliveInterval=30
pipelining = True
Tip
For more configuration options, see Ansible: Ansible Configuration Settings
Parameter |
Example |
Description |
---|---|---|
|
|
(Optional) Path to Python interpreter on control node. |
|
|
Path to Ansible roles. This package provides required roles in the |
|
|
Path to Ansible collections. This package provides required collections in the |
|
|
User name on managed nodes. This user must have sudo privileges. Control node must have SSH access to this user account on managed nodes. |
|
|
SSH private key file on control node. This should provide SSH access to a user account with sudo privileges on managed nodes. |
Prepare managed nodes#
Control node v.s. Managed nodes
Your control node must be separate from managed nodes. See Prepare control and managed nodes.
Each managed node must have:
One of the operating systems supported by EclecticIQ Intelligence Center:
Rocky Linux 8
RHEL 8
Upgrading from CentOS 7/RHEL 7
If you are installing or upgrading EcelcticIQ Intelligence Center on managed on older operating systems (e.g. CentOS 7, RHEL 7), you must upgrade the operating system to Rocky Linux 8 or RHEL 8 before following the instructions here.
See:
Filesystem mounts where required. See Deployment architecture guidance.
Install Python 3.6 and python3-libselinux#
Note
Not required when using
eiq-ic-playbooks-3x-1688550826
(5 July 2023) package or newer.
Use the latest playbook package.
Install Python 3.6
Warning
Do not do this for upgrades, or if Python 3.6 is already present on managed nodes.
Caution
This is required for these playbooks to be able to manage SELinux configuration on managed nodes.
Default BaseOS yum
repositories in
Rocky Linux 8 and RHEL 8 only have
a version of python3-libselinux
that is compiled for Python 3.6.
If Python 3.6 is not installed,
the tasks that manage
SELinux configuration
will fail.
If a managed node uses a minimal distribution of Rocky Linux 8 or RHEL 8, Python 3.6 may not be present on the host.
If Python 3.6 is not present on the managed node, install it. Run as root:
yum install -y python36
Install python3-libselinux
On Red Hat-based systems (RHEL, CentOS, etc.),
you must install python3-libselinux
on all managed nodes
to allow Ansible to execute playbook tasks on them.
Run as root:
yum install -y python3-libselinux
Allow incoming requests for services#
When performing a multi-node install (2 nodes and above), you may need to configure managed nodes to allow incoming connections from EclecticIQ Intelligence Center application node for certain services to work.
The table below lists EclecticIQ Intelligence Center services and the ports they listen on:
Service |
Port(s) |
---|---|
Ingestion, Celery |
|
PostgreSQL |
|
Redis |
|
Elasticsearch |
|