Prepare nodes#

This section contains instructions on how to prepare your systems for an EclecticIQ Intelligence Center installation with Ansible playbooks.

Requirements#

You must have the following before running the playbooks:

Upgrading from CentOS 7/RHEL 7

If you are installing or upgrading EcelcticIQ Intelligence Center on managed on older operating systems (e.g. CentOS 7, RHEL 7), you must upgrade the operating system to Rocky Linux 8 or RHEL 8 before following the instructions here.

See:

Deployment guidance#

Managed nodes should follow guidance here: Deployment architecture guidance

Prepare control and managed nodes#

Before you run the playbooks, follow the instructions in this section to prepare your control and managed nodes.

Control node v.s. Managed nodes

Your control node must be separate from managed nodes.

Your control node is where the ansible-playbook is run. Managed nodes are what ansible-playbook acts upon. Avoid running ansible-playbook directly from managed nodes (instead of using a separate control node) – this can lead to unexpected behavior.

Prepare control node#

Control node v.s. Managed nodes

Your control node must be separate from managed nodes. See Prepare control and managed nodes.

You need at least one host to act as a control node. This is the host where you run the ansible-playbook command.

Your control node must have:

  • Python 3.8 and newer.

  • SSH access to managed nodes, as a user with sudo privileges.

Download and unarchive playbook package#

Tip

You may need to navigate to https://downloads.eclecticiq.com/ on your browser and log in before attempting to open the URLs below.

Download the latest EclecticIQ Intelligence Center installation playbooks from https://downloads.eclecticiq.com/platform/ansible/3.x and unarchive it.

Tip

Save and version control your playbook configuration. Playbook configurations document your deployment, and can be used to administer and upgrade your EclecticIQ Intelligence Center instance.

Do not store credentials in version control.

Prepare Python environment#

  1. Install Python 3.8 or newer.

  2. Navigate to the EclecticIQ Intelligence Center installation playbooks directory.

    cd ~/path/to/ic-playbooks

    (Recommended) Version control your playbook configuration

    Save and version control your playbook configuration. Playbook configurations document your deployment, and can be used to administer and upgrade your EclecticIQ Intelligence Center instance.

    Do not store credentials in version control.

  3. (Optional) Create and activate a Python virtual environment:

    python3 -m venv .venv
source .venv/bin/activate

  4. Install Python dependencies:

    python3 -m pip install -r python-requirements.txt

Set up .env#

Caution

You may have used an .envrc file in earlier versions of the installation playbooks. Rename it to .env to follow the instructions here.

Create an .env file. You can use env-example as a template.

Tip

Instead of using environment variables, you can directly modify group_vars/all/config.yml instead with these values.

You must set values for all these environment variables:

# Recommend that you generate passwords with the below command:
# printf '~^'; cat /dev/urandom | LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1

# Password requirements: at least 10 char with on uppercase, one lowercase, one number and 1 symbol
# https://docs.eclecticiq.com/ic/current/install-configure-upgrade/rocky/install-the-ic/before-you-start/#user-name-and-password

export EIQ_IC_VERSION="3.2"

export EIQ_PYTHON_VERSION="3.8"


export EIQ_REPO_USERNAME="user@example"
export EIQ_REPO_PASSWORD=<example_password>

export EIQ_DB_PASSWORD=<example_password>

# This variable should be defined if PG replication is being configured
export PG_REPLICA_USER_PASSWORD="${EIQ_DB_PASSWORD}"

export EIQ_REDIS_PASSWORD=<example_password>

export EIQ_ES_ELASTIC_PASSWORD=<example_password>
export EIQ_ES_KIBANA_SYSTEM_PASSWORD=<example_password>
export EIQ_ES_LOGSTASH_SYSTEM_PASSWORD=<example_password>

export EIQ_PLATFORM_SECRET_KEY=<example_password>

export EIQ_ADMIN_USERNAME="admin"
export EIQ_ADMIN_PASSWORD=<example_password>
export EIQ_ADMIN_EMAIL="[email protected]"

export IC_ADMIN_USERNAME="${EIQ_ADMIN_USERNAME}"
export IC_ADMIN_PASSWORD="${EIQ_ADMIN_PASSWORD}"

# Destination for (possibly large) backups on the ansible controller
export EIQ_IC_BACKUP_ROOT="."

# these should be set for automated configuration of RHEL optional repos (for perl dependencies of PostgreSQL)
# Customer who don't wish to supply credentials, should manually enable the repos needed to install perl and perl libraries from RHEL
export RHEL_USERNAME='rh_user'
export RHEL_PASSWORD=<example password>


# Define hosts/IPs, based on the chosen inventory
export EIQ_HOST_IC_APP_01="127.0.0.1"   # This is the only manadatory variable, present in all inventories
export EIQ_HOST_IC_APP_02="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_01="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_02="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_03="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_04="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_05="127.0.0.1"
export EIQ_HOST_IC_CELERY="127.0.0.1"
export EIQ_HOST_IC_INGESTION="127.0.0.1"
export EIQ_HOST_IC_WORKER_01="127.0.0.1"
export EIQ_HOST_IC_WORKER_02="127.0.0.1"
export EIQ_HOST_PG_NODE_01="127.0.0.1"
export EIQ_HOST_PG_NODE_02="127.0.0.1"
export EIQ_HOST_REDIS_NODE_01="127.0.0.1"
export EIQ_HOST_REDIS_NODE_02="127.0.0.1"

export EIQ_HOST_NEO4J_NODE_01="127.0.0.1" # only required for Neo4j removal

The following sections describes environment variables used and their possible values.

Target software versions#

Set versions of software to install or upgrade to.

# Target IC version to install
export EIQ_IC_VERSION="3.1"

# Do not change
export EIQ_PYTHON_VERSION="3.8"
Credentials#

Set credentials for IC and its components.

If you are upgrading from an existing installation, you must copy existing credential entries from that instance. They can be found in /etc/eclecticiq/platform_settings.py.

# Credentials for https://downloads.eclecticiq.com
export EIQ_REPO_USERNAME="user@example"
export EIQ_REPO_PASSWORD="<example_password>"

# Password for PostgreSQL instance
export EIQ_DB_PASSWORD="<example_password>"

# This variable should be defined if PG replication is being configured
export PG_REPLICA_USER_PASSWORD="${EIQ_DB_PASSWORD}"

# Password for Redis
export EIQ_REDIS_PASSWORD="<example_password>"

# Secret token for IC. Set this to a long string.
export EIQ_PLATFORM_SECRET_KEY="<example_password>"

# Admin credentials for IC
export EIQ_ADMIN_USERNAME="admin"
export EIQ_ADMIN_PASSWORD="<example_password>"
export EIQ_ADMIN_EMAIL="[email protected]"

export IC_ADMIN_USERNAME="${EIQ_ADMIN_USERNAME}"
export IC_ADMIN_PASSWORD="${EIQ_ADMIN_PASSWORD}"
System settings#

These environment variables set system settings for certain tasks to run.

# Destination for (possibly large) backups on the ansible controller
export EIQ_IC_BACKUP_ROOT="."

## Required for RHEL only.
# If not provided here, must be manually provisioned on RHEL nodes.
# Set your Red Hat Enterprise Linux subscription credentials here
# For pulling required PostgreSQL Perl dependencies.
export RHEL_USERNAME='rh_user'
export RHEL_PASSWORD=<example password>
Inventory configuration#

These playbooks allow you to configure network addresses of managed nodes without manually changing inventory files in the inventories/ folder.

These must still be used in conjunction with inventory files. For example, setting EIQ_HOST_IC_APP_01=<target_ip> here allows you to run ansible-playbook -i inventories/ic-test-small.yml ... without modifying the ic-test-small.yml inventory file.

Tip

Instead of using environment variables, you can directly modify group_vars/all/config.yml instead with these values.

# Define hosts/IPs, based on the chosen inventory
export EIQ_HOST_IC_APP_01="127.0.0.1"   # This is the only manadatory variable, present in all inventories
export EIQ_HOST_IC_APP_02="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_01="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_02="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_03="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_04="127.0.0.1"
export EIQ_HOST_ELASTIC_NODE_05="127.0.0.1"
export EIQ_HOST_IC_CELERY="127.0.0.1"
export EIQ_HOST_IC_INGESTION="127.0.0.1"
export EIQ_HOST_IC_WORKER_01="127.0.0.1"
export EIQ_HOST_IC_WORKER_02="127.0.0.1"
export EIQ_HOST_PG_NODE_01="127.0.0.1"
export EIQ_HOST_PG_NODE_02="127.0.0.1"
export EIQ_HOST_REDIS_NODE_01="127.0.0.1"
export EIQ_HOST_REDIS_NODE_02="127.0.0.1"

export EIQ_HOST_NEO4J_NODE_01="127.0.0.1" # only required for Neo4j removal

Each deployment type/inventory file requires you to set different environment variables.

The following list shows which environment variables are required per deployment type:

  • inventories/ic-test-small.yml

    • EIQ_HOST_IC_APP_01

  • inventories/ic-test-two.yml

    • EIQ_HOST_IC_APP_01

    • EIQ_HOST_PG_NODE_01

  • inventories/ic-test-medium.yml

    • EIQ_HOST_IC_APP_01

    • EIQ_HOST_ELASTIC_NODE_01

    • EIQ_HOST_ELASTIC_NODE_02

    • EIQ_HOST_ELASTIC_NODE_03

  • inventories/ic-test-large.yml

    • EIQ_HOST_IC_APP_01

    • EIQ_HOST_IC_WORKER_01

    • EIQ_HOST_PG_NODE_01

    • EIQ_HOST_REDIS_NODE_01

    • EIQ_HOST_ELASTIC_NODE_01

    • EIQ_HOST_ELASTIC_NODE_02

  • inventories/ic-prod-medium-pg1.yml

    • EIQ_HOST_IC_APP_01

    • EIQ_HOST_PG_NODE_01

    • EIQ_HOST_ELASTIC_NODE_01

    • EIQ_HOST_ELASTIC_NODE_02

    • EIQ_HOST_ELASTIC_NODE_03

  • inventories/ic-prod-medium.yml

    • EIQ_HOST_IC_APP_01

    • EIQ_HOST_PG_NODE_01

    • EIQ_HOST_PG_NODE_02

    • EIQ_HOST_ELASTIC_NODE_01

    • EIQ_HOST_ELASTIC_NODE_02

    • EIQ_HOST_ELASTIC_NODE_03

  • inventories/ic-prod-large.yml

    • EIQ_HOST_IC_INGESTION

    • EIQ_HOST_IC_APP_01

    • EIQ_HOST_IC_CELERY

    • EIQ_HOST_PG_NODE_01

    • EIQ_HOST_PG_NODE_02

    • EIQ_HOST_REDIS_NODE_01

    • EIQ_HOST_REDIS_NODE_02

    • EIQ_HOST_ELASTIC_NODE_01

    • EIQ_HOST_ELASTIC_NODE_02

    • EIQ_HOST_ELASTIC_NODE_03

Prepare ansible.cfg file#

Create an ansible.cfg file at the project root and set the following values.

Use ansible-example.cfg as a template.

[defaults]
interpreter_python=auto_silent
roles_path=roles
collections_path=collections
remote_user=example-user
private_key_file=~/.ssh/path_to_your_key_id_rsa

[ssh_connection]
ssh_args = -o ServerAliveInterval=30
pipelining = True

Tip

For more configuration options, see Ansible: Ansible Configuration Settings

Parameter

Example

Description

interpreter_python

auto_silent

(Optional) Path to Python interpreter on control node.

roles_path

roles

Path to Ansible roles. This package provides required roles in the roles/ directory. Avoid changing.

collections_path

collections

Path to Ansible collections. This package provides required collections in the collections/ directory. Avoid changing.

remote_user

ec2-user

User name on managed nodes. This user must have sudo privileges. Control node must have SSH access to this user account on managed nodes.

private_key_file

~/.ssh/path_to_your_key

SSH private key file on control node. This should provide SSH access to a user account with sudo privileges on managed nodes.

Prepare managed nodes#

Control node v.s. Managed nodes

Your control node must be separate from managed nodes. See Prepare control and managed nodes.

Each managed node must have:

Install Python 3.6 and python3-libselinux#

Note

Not required when using eiq-ic-playbooks-3x-1688550826 (5 July 2023) package or newer. Use the latest playbook package.

Install Python 3.6

Warning

Do not do this for upgrades, or if Python 3.6 is already present on managed nodes.

Caution

This is required for these playbooks to be able to manage SELinux configuration on managed nodes.

Default BaseOS yum repositories in Rocky Linux 8 and RHEL 8 only have a version of python3-libselinux that is compiled for Python 3.6. If Python 3.6 is not installed, the tasks that manage SELinux configuration will fail.

If a managed node uses a minimal distribution of Rocky Linux 8 or RHEL 8, Python 3.6 may not be present on the host.

If Python 3.6 is not present on the managed node, install it. Run as root:

yum install -y python36

Install python3-libselinux

On Red Hat-based systems (RHEL, CentOS, etc.), you must install python3-libselinux on all managed nodes to allow Ansible to execute playbook tasks on them. Run as root:

yum install -y python3-libselinux

Allow incoming requests for services#

When performing a multi-node install (2 nodes and above), you may need to configure managed nodes to allow incoming connections from EclecticIQ Intelligence Center application node for certain services to work.

The table below lists EclecticIQ Intelligence Center services and the ports they listen on:

Service

Port(s)

Ingestion, Celery

  • 8000

PostgreSQL

  • 5432

Redis

  • 6379

Elasticsearch

  • 9200

  • 9300 (between Elasticsearch nodes only)