Perform a distributed installation CentOS#

Tip

Make sure you’ve set up the repositories before attempting these instructions.

This procedure guides you through a distributed installation across multiple target hosts:

  • Database servers hosting the databases:

    • Elasticsearch and Logstash

    • Neo4j and eclecticiq-neo4jbatcher

    • PostgreSQL

    • Redis

  • Intelligence Center (IC) servers hosting:

    • The IC backend components.

    • The IC frontend components.

Install first the databases on their dedicated servers, and then the IC on its other dedicated servers.

⚠️ Permissive defaults

Instructions here set permissive default network bindings.

You should change these to a more restrictive configuration.

Elasticsearch and Logstash#

Elasticsearch and Logstash should be installed on the same host.

Logstash is an optional service.

This procedure consists of the following steps:

  • Install Java.

  • Install and configure Elasticsearch.

  • Enable and start the Elasticsearch service.

  • Install and configure Logstash.

  • Enable and start the Logstash service.

Install Java#

Install Java 11 (OpenJDK).

yum -y install java-11-openjdk-headless tzdata-java
update-alternatives --set java $(readlink -f /usr/lib/jvm/jre-11/bin/java)

Install and start Elasticsearch#

  1. Install the Elasticsearch package:

    # Install Elasticsearch
yum -y install eclecticiq-elasticsearch

  2. Configure the Elasticsearch service to listen on for incoming connections on 0.0.0.0:

    # Elasticsearch bind address
echo -e "[Service]\nEnvironment=BINDING_ADDRESS=0.0.0.0" > /etc/systemd/system/elasticsearch.service.d/20-eclecticiq_binding_address.conf

  3. Elasticsearch cluster with multiple master-eligible nodes: If you are forming a cluster with multiple master-eligible nodes, you must add cluster.initial_master_nodes to /etc/eclecticiq-elasticsearch/elasticsearch.yml on those nodes before starting the cluster for the first time.

    For more information, see Elasticsearch [7.16]: Bootstrapping a cluster.

  4. Start and enable the service:

    # Start and enable the Elasticsearch service
systemctl start elasticsearch
systemctl enable elasticsearch

Export the Logstash binding address#

On the Elasticsearch host, set the syslog_server environment variable to the network address of the Logstash service:

syslog_server=

If you’ve installed Elasticsearch and Logstash on the same host, this can be set to 127.0.0.1.

Install and start Logstash#

  1. Install Logstash:

    # Install Logstash
yum -y install eclecticiq-logstash

  2. Start and enable the service:

    # Start and enable the Logstash service
systemctl start logstash
systemctl enable logstash

Neo4j#

Neo4j and eclecticiq-neo4jbatcher should be installed on the same host.

This procedure consists of the following steps:

  • Set and export the Neo4j and eclecticiq-neo4jbatcher credentials.

  • Install and configure Neo4j.

  • Enable and start the Neo4j service.

  • Install and configure eclecticiq-neo4jbatcher.

  • Enable and start the Nginx web server (it is a dependency of eclecticiq-neo4jbatcher) and the eclecticiq-neo4jbatcher services.

Set environment variables#

Set the following environment variables:

neo4j_pass=
neo4jbatcher_user=
neo4jbatcher_pass=

Env. variable

Description

neo4j_pass

Password for neo4j user.

neo4jbatcher_user

User for Neo4j ingestion preprocessor.

neo4jbatcher_pass

User password for Neo4j ingestion preprocessor.

Install and start Neo4j#

  1. Install Neo4j:

    # Install Neo4j
yum -y install eclecticiq-neo4j

  2. Configure the service to listen for incoming connections on 0.0.0.0:

    # Neo4j bind address
echo "dbms.connectors.default_listen_address=0.0.0.0" >> /etc/eclecticiq-neo4j/neo4j.conf

  3. Start and enable the service:

    # Start and enable the Neo4j service
systemctl start neo4j
systemctl enable neo4j

Install and start eclecticiq-neo4jbatcher#

eclecticiq-neo4jbatcher is an ingestion preprocessing module for Neo4j.

It temporarily stores incoming data, queues it, prepares, and then sends the data batches on to the graph database.

  1. Install eclecticiq-neo4jbatcher:

    # Install Neo4jbatcher
yum -y install eclecticiq-neo4jbatcher

  2. Configure the service to listen for incoming connections on 0.0.0.0:

    # eclecticiq-neo4jbatcher bind address
echo -e "BIND_HOST=0.0.0.0" >> /etc/eclecticiq-neo4jbatcher/neo4jbatcher.conf

  3. Start and enable the Nginx web server (eclecticiq-neo4jbatcher depends on it) and the eclecticiq-neo4jbatcher services:

    # Start and enable the Nginx web service
systemctl start nginx
systemctl enable nginx

# Start and enable the eclecticiq-neo4jbatcher service
systemctl start eclecticiq-neo4jbatcher
systemctl enable eclecticiq-neo4jbatcher

PostgreSQL#

This procedure consists of the following steps:

  • Set and export the PostgreSQL credentials.

  • Install PostgreSQL.

  • Enable and start the PostgreSQL service.

Set environment variables#

Set the following environment variables:

db_user=
db_name=
db_pass=

Env. variable

Description

db_user

PostgreSQL user name.

db_pass

PostgreSQL user password.

db_name

Name of PostgreSQL database used by the IC.

Install and start PostgreSQL#

  1. Install PostgreSQL:

    # Install PostgreSQL
yum -y install eclecticiq-postgres

  2. Start and enable the service:

    # Start/Enable PostgreSQL service
systemctl start postgresql-11
systemctl enable postgresql-11

Redis#

This procedure consists of the following steps:

  • Set and export the Redis credentials.

  • Install and configure Redis.

  • Enable and start the Redis service.

Set environment variables#

Set the following environment variables:

redis_pass=

Env. variable

Description

redis_pass

Password for redis user.

Install and start Redis#

  1. Install Redis:

    # Install Redis
yum -y install eclecticiq-redis

  2. Configure the service to listen for incoming connections on 0.0.0.0:

    # Redis bind address
echo "bind 0.0.0.0" >> /etc/eclecticiq-redis/local.conf

  3. Start and enable the service:

    # Start/Enable Redis service
systemctl start redis
systemctl enable redis

EclecticIQ Intelligence Center#

Install the IC after you’ve installed and started the above services.

The -backend and -frontend packages can be installed on separate hosts.

This procedure consists of the following steps:

  • Set a basic configuration by defining the hostnames and the credentials to use during the IC installation.

  • Install the IC backend components on their dedicated server.

  • Enable and start the IC backend services on the local host.

  • Install the IC frontend components on their dedicated server.

  • Enable and start the IC web service on the local host.

Set service locations in environment variables#

Set the following environment variables to each services’ location on the network:

db_host=
redis_host=
es_host=
neo4j_host=
neo4jbatcher_host=
syslog_server=

Env. variable

Description

db_host

Network address of PostgreSQL service.

redis_host

Network address of Redis service.

es_host

Network address of Elasticsearch service.

neo4j_host

Network address of Neo4j service.

neo4jbatcher_host

Network address of eclecticiq-neo4jbatcher service.

syslog_server

Network address of Logstash service.

Set credentials in environment variables#

Set as environment variables the credentials needed to connect to services:

db_user=
db_name=
db_pass=
neo4j_pass=
neo4jbatcher_user=
neo4jbatcher_pass=
redis_pass=

Env. variable

Description

db_user

PostgreSQL user name.

db_pass

PostgreSQL user password.

db_name

Name of PostgreSQL database used by the IC.

neo4j_pass

Password for neo4j user.

neo4jbatcher_user

User for Neo4j ingestion preprocessor.

neo4jbatcher_pass

User password for Neo4j ingestion preprocessor.

redis_pass

Password for redis user.

Set parameters in /etc/default/eclecticiq-platform#

Alternatively, set these parameters in the /etc/default/eclecticiq-platform file:

cat << EOF >> /etc/default/eclecticiq-platform
db_host=${db_host}
db_user=${db_user}
db_name=${db_name}
db_pass=${db_pass}
redis_host=${redis_host}
es_host=${es_host}
neo4j_host=${neo4j_host}
neo4j_pass=${neo4j_pass}
neo4jbatcher_host=${neo4jbatcher_host}
neo4jbatcher_user=${neo4jbatcher_user}
neo4jbatcher_pass=${neo4jbatcher_pass}
redis_pass=${redis_pass}
syslog_server=${syslog_server}
EOF

Install the IC backend#

Set a secret key in /etc/default/eclecticiq-platform:

EIQ_SECRET_KEY=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1)
echo "EIQ_SECRET_KEY=$EIQ_SECRET_KEY" >> /etc/default/eclecticiq-platform

Install the IC backend:

# Install the backend
yum -y install eclecticiq-platform-backend-all

Install the extensions#

On the IC backend host, install the extensions:

📔 You must set up the /opt/eclecticiq-platform-backend/pip.conf and ~/.netrc files in “Set up the repositories” for this command to work.

# Uninstall old extensions from venv
/opt/eclecticiq-platform-backend/bin/pip freeze | grep '^eclecticiq-extension-' | grep -v '^eclecticiq-extension-api=' | cut -d= -f1 | xargs -r /opt/eclecticiq-platform-backend/bin/pip uninstall -y

# Install specified extensions in venv
/opt/eclecticiq-platform-backend/bin/pip install $extensions_packages

Start the services#

Start and enable services:

# start/enable backend 3rd party services
systemctl restart nginx
systemctl enable nginx
systemctl start kibana
systemctl enable kibana
systemctl start statsite
systemctl enable statsite
systemctl restart rsyslog
systemctl enable rsyslog

Restrict access to configuration files:

chown root:eclecticiq /etc/default/eclecticiq-platform*
chmod 640 /etc/default/eclecticiq-platform*

Install the IC frontend#

Install the IC frontend and documentation:

# Install the frontend and documentation packages
yum -y install eclecticiq-platform-frontend eclecticiq-platform-docs

Start Nginx#

Restart the Nginx service:

# Start/Enable web service
systemctl restart nginx
systemctl enable nginx

Database migrations#

Once done with the instructions here, you must perform database migrations to finish installing the IC.

Important

To complete the installation procedure, make sure you run the database migrations before continuing with the other configuration steps.