Prepare nodes

This section contains instructions on how to prepare your systems for an EclecticIQ Intelligence Center installation with Ansible playbooks.

Requirements

You must have the following before running the playbooks:

• Credentials for https://downloads.eclecticiq.com.

• One host to act as an Ansible control node.

• One or more hosts to act as Ansible managed nodes.

Deployment guidance

We recommend preparing managed nodes according to the guidance provided here: Deployment architecture guidance

Prepare control and managed nodes

Before you run the playbooks, follow the instructions in this section to prepare your control and managed nodes.

Control node v.s. Managed nodes

Your control node must be separate from managed nodes.

Your control node is where the ansible-playbook is run, and managed nodes are what ansible-playbook acts upon. Running ansible-playbook directly from managed nodes (instead of using a separate control node) can lead to unexpected behavior during playbook runs.

Prepare control node

Control node v.s. Managed nodes

Your control node must be separate from managed nodes. See Prepare control and managed nodes.

You need at least one host to act as a control node. This is the host where you run the ansible-playbook command.

Your control node must have:

• Python 3.8 and newer.

• SSH access to managed nodes, as a user with sudo privileges.

Download and unarchive playbook package

Tip

You may need to navigate to https://downloads.eclecticiq.com/ on your browser and log in before attempting to open the URLs below.

Download the latest EclecticIQ Intelligence Center installation playbooks from https://downloads.eclecticiq.com/platform/ansible/3.x and unarchive it.

You must

• Use your https://downloads.eclecticiq.com credentials (<EIQ_USER>:<EIQ_PASSWORD>)

export PLAYBOOK_PACKAGE="eiq-ic-playbooks-3x-<timestamp>.zip"

curl -O -u '<EIQ_USER>:<EIQ_PASSWORD>' https://downloads.eclecticiq.com/platform/ansible/3.x/$PLAYBOOK_PACKAGE

# Example: curl -O -u user%40eclecticiq.com:sUp3rs3cr3tP4ssw0rd https://...

unzip $PLAYBOOK_PACKAGE

Tip

Save and version control your playbook configuration.

These playbook configurations can be useful for documenting your deployments, and are used for future EclecticIQ Intelligence Center upgrades.

Do not store credentials in version control.

Prepare Python environment

1. Install Python 3.8 or newer.

2. Navigate to the EclecticIQ Intelligence Center installation playbooks directory.

   cd ~/path/to/ic-playbooks
   

   (Recommended) Version control your playbook configuration

   Save and version control your playbook configuration.

   These playbook configurations can be useful for documenting your deployments, and are used for future EclecticIQ Intelligence Center upgrades.

   Do not store credentials in version control.

3. (Optional) Create and activate a Python virtual environment:

   python3 -m venv .venv
   source .venv/bin/activate
   

4. Install Python dependencies:

   python3 -m pip install -r python-requirements.txt
   

Set up secrets

Caution

You may have used an .envrc file in earlier versions of the installation playbooks. Rename it to .env to follow the instructions here.

Create an .env file and set the following environment variables. You can use .env-example as a template.

For upgrades

If you are setting up the .env file to upgrade an existing EclecticIQ Intelligence Center instance, retrieve these values from the /etc/eclecticiq/platform_settings.py file on your existing instance.

# Recommend that you generate passwords with the below command:
# printf '~^'; cat /dev/urandom | LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1

# Password requirements: at least 10 char with on uppercase, one lowercase, one number and 1 symbol
# https://docs.eclecticiq.com/ic/current/install-configure-upgrade/rocky/install-the-ic/before-you-start/#user-name-and-password

export EIQ_IC_VERSION="3.1"
export EIQ_IC_API_VERSION="3.1.*"
export EIQ_PYTHON_VERSION="3.8"

export EIQ_REPO_USERNAME="user@example"
export EIQ_REPO_PASSWORD="<example_password>"

export EIQ_DB_PASSWORD="<example_password>"

# This variable should be defined if PG replication is being configured
export PG_REPLICA_USER_PASSWORD="${EIQ_DB_PASSWORD}"

export EIQ_REDIS_PASSWORD="<example_password>"

export EIQ_SECRET_KEY="<example_password>"

export EIQ_ADMIN_USERNAME="admin"
export EIQ_ADMIN_PASSWORD="<example_password>"
export EIQ_ADMIN_EMAIL="[email protected]"

export IC_ADMIN_USERNAME="${EIQ_ADMIN_USERNAME}"
export IC_ADMIN_PASSWORD="${EIQ_ADMIN_PASSWORD}"

# Case sensitive, should be set to True only if we are prepared to destroy the elastic cluster without warning or prompt
export EIQ_ELASTIC_SEARCH_SILENT_DELETE="False"

####
# This should be set to False for fresh install, True for an upgrade
####
export EIQ_IS_UPGRADE="True"

# Destination for (possibly large) backups on the ansible controller
export EIQ_IC_BACKUP_ROOT="."

Once done, load these values from the .env file by running:

source .env

Tip

For information on how to use Ansible with remote users that require passwords for sudo and SSH keys, see Appendix.

Prepare inventory file

Select and modify the inventory file that suits your deployment architecture.

This package provides pre-configured inventory files for the following deployments:

Tip

Inventory files here correspond to configurations listed in Deployment architecture guidance.

ic-test-small.yml	1 node test deployment
ic-test-two.yml	2 node test deployment
ic-test-medium.yml	4 node test deployment
ic-prod-medium-pg1.yml	5 node production deployment
ic-prod-medium.yml	6 node production deployment
ic-prod-large.yml	10 node production deployment
ic-remove-neo4j.yml	Only for removing Neo4j. Not needed for fresh installation.

Tip

Use at least the inventories/ic-prod-medium.yml inventory file when deploying for production.

Edit that inventory file, and change only the ansible_host values so that they point to the correct target managed node. This is the inventory file that you must invoke when running the ansible-playbook command.

For example, for a prod-medium (6/7 node) deployment, modify the inventories/ic-prod-medium.yml file:

---
icprodmedium: &prodmedium
  hosts:
    ic-application:
      ansible_host: 10.10.1.331 # Host for IC application

    pg-node-01:
      ansible_host: 10.10.1.332 # PostgreSQL host

    pg-node-02:
      ansible_host: 10.10.1.333 # PostgreSQL host

    elastic-node-01:
      es_node_name: elastic-node-01
      ansible_host: 10.10.1.334 # Elasticsearch node 1

    elastic-node-02:
      es_node_name: elastic-node-02
      ansible_host: 10.10.1.335 # Elasticsearch node 2

    elastic-node-03:
      es_node_name: elastic-node-03
      ansible_host: 10.10.1.336 # Elasticsearch node 3

// ...
---

When you run the ansible-playbook command, you must include this inventory file:

ansible-playbook -i inventories/ic-prod-medium.yml <playbook_name.yml>

Prepare ansible.cfg file

Create an ansible.cfg file at the project root and set the following values.

Use ansible-example.cfg as a template.

[defaults]
interpreter_python=auto_silent
roles_path=roles
collections_path=collections
remote_user=example-user
private_key_file=~/.ssh/path_to_your_key_id_rsa

Tip

For more configuration options, see Ansible: Ansible Configuration Settings

Parameter	Example	Description
interpreter_python	auto_silent	(Optional) Path to Python interpreter on control node.
roles_path	roles	Path to Ansible roles. This package provides required roles in the roles/ directory. Avoid changing.
collections_path	collections	Path to Ansible collections. Thsi package provides required collections in the collections/ directory. Avoid changing.
remote_user	ec2-user	User name on managed nodes. This user must have sudo privileges. Control node must have SSH access to this user account on managed nodes.
private_key_file	~/.ssh/path_to_your_key	SSH private key file on control node. This should provide SSH access to a user account with sudo privileges on managed nodes.

Set IC version to install

Set EclecticIQ Intelligence Center version to install by setting the following environment variables in .env:

####
# Set EclecticIQ Intelligence Center package versions to install.
# Use these values to install EclecticIQ Intelligence Center 3.0.
####
export EIQ_IC_VERSION='3.0'
export EIQ_IC_API_VERSION='3.0.*'
export EIQ_PYTHON_VERSION='3.8'

After setting these values, make sure to reload your .env file:

source .env

Prepare managed nodes

Control node v.s. Managed nodes

Your control node must be separate from managed nodes. See Prepare control and managed nodes.

Each managed node must have:

• One of the operating systems supported by EclecticIQ Intelligence Center:

  • Rocky Linux 8

  • RHEL 8

• System Python 3.6.8. (See Install Python 3.6)

• python3-libselinux

• Allow incoming requests for services

• Filesystem mounts where required. See Deployment architecture guidance.

Install Python 3.6

Warning

Do not do this for upgrades, or if Python 3.6 is already present on managed nodes.

Caution

This is required for these playbooks to be able to manage SELinux configuration on managed nodes.

Default BaseOS yum repositories in Rocky Linux 8 and RHEL 8 only have a version of python3-libselinux that is compiled for Python 3.6. If Python 3.6 is not installed, the tasks that manage SELinux configuration will fail.

If a managed node uses a minimal distribution of Rocky Linux 8 or RHEL 8, Python 3.6 may not be present on the host.

If Python 3.6 is not present on the managed node, install it. Run as root:

yum install -y python36

Install python3-libselinux

On RedHat-based systems (RHEL, CentOS, etc.), you must install python3-libselinux on all managed nodes to allow Ansible to execute playbook tasks on them. Run as root:

yum install -y python3-libselinux

Allow incoming requests for services

When performing a multi-node install (2 nodes and above), you may need to configure managed nodes to allow incoming connections from EclecticIQ Intelligence Center application node for certain services to work.

The table below lists EclecticIQ Intelligence Center services and the ports they listen on:

Service	Port(s)
Ingestion, Celery	8000
PostgreSQL	5432
Redis	6379
Elasticsearch	9200 9300 (between Elasticsearch nodes only)