Upgrade EclecticIQ Intelligence Center VM#

This procedure guides you through the platform upgrade process from release 2.9.x to release 2.14.0:

Prepare systems for upgrade#

Follow the instructions in this section to prepare your systems to upgrade the Intelligence Center.

Update repository config#

Edit /etc/yum.repos.d/platform-packages-centos.repo to update the version number in baseurl:

# Update this line
baseurl=https://downloads.eclecticiq.com/platform-packages-centos/<target_minor_version>

⚠️ Do not update the version specified in /etc/yum.repos.d/platform-dependencies-centos.repo

If these files do not exist on your system, see the “Set up the repositories” page for your OS for instructions to create them.

Install Python 3.8#

Release 2.12.0 and newer use Python 3.8.

When upgrading from 2.11 and older to 2.12.0 and newer, you must also install Python 3.8 using EclecticIQ-provided packages.

✅ Installing EclecticIQ-provided Python 3.8 packages installs Python 3.8 alongside the system Python packages.

To do this, follow the instructions on the “Set up the repositories” page for your OS to create a platform-dependencies-centos-python38.repo file on your IC host before following the rest of the upgrade instructions.

Set environment variables#

Set the following environment variables on your system:

extensions_repo_protocol=https
extensions_repo_domain=downloads.eclecticiq.com
extensions_repo_path=Extensions
extensions_packages=eclecticiq-extension-all~=2.14.0

These are used later when installing extensions.

📔 The version set in extensions_packages must only match the minor version of the IC. E.g., for IC 2.9.3, extensions_packages must be set to eclecticiq-extension-all~=2.9.0.

Stop the services#

Intelligence Center host#

If the IC instance is distributed across more than one host, carry out these actions only in the machine hosting the eclecticiq-platform-backend-* components.

  • Stop all platform services, Kibana, and Statsite:

# Stop platform services, kibana and statsite
systemctl stop eclecticiq-platform-backend-services
systemctl stop eclecticiq-platform-backend-workers
systemctl stop kibana
systemctl stop statsite

  • Enforce stricter access rights for selected platform files:

chown root:eclecticiq /etc/default/eclecticiq-platform*
chmod 640 /etc/default/eclecticiq-platform*

  • Back up the IC settings file to restore custom settings, in case they are overwritten during the upgrade:

# Back up platform settings
cp /etc/eclecticiq/platform_settings.py{,.$(date +%s).bak}

Database machines#

  • Stop all databases and log aggregator components; that is, Elasticsearch, Logstash, PostgreSQL, Neo4j, eclecticiq-neo4jbatcher, and Redis:

systemctl stop elasticsearch
systemctl stop logstash
systemctl stop postgresql-11
systemctl stop neo4j
systemctl stop eclecticiq-neo4jbatcher
systemctl stop redis

Perform the upgrade#

Intelligence Center host#

In the /etc/eclecticiq/platform_settings.py platform settings configuration file you need to specify:

  • The address of the machine hosting eclecticiq-neo4jbatcher:

    • In platform_settings.py browse to the NEO4J_BATCHING_URL parameter.

    • Assign to NEO4J_BATCHING_URL the URL pointing to the eclecticiq-neo4jbatcher instance.

  • The temporary location of the data that eclecticiq-neo4jbatcher prepares and queues up for ingestion in the graph database:

    • Export the environment variable pointing to the directory where eclecticiq-neo4jbatcher stores preprocessed data.

    • In platform_settings.py assign the exported value to the GRAPH_INGESTION_SPOOL_DIRECTORY parameter.

To export the environment variable pointing to the directory where eclecticiq-neo4jbatcher stores preprocessed data:

NEO4JB_STORAGE_DIR=/var/lib/neo4j/import/tmp/neo4j-batcher

To assign the exported value to the GRAPH_INGESTION_SPOOL_DIRECTORY parameter in platform_settings.py:

# Set the location of the preprocessed data in the IC settings
sed -ri 's|^(GRAPH_INGESTION_SPOOL_DIRECTORY\s*=\s*).*$|\1'"'${NEO4JB_STORAGE_DIR}'"'|g' /etc/eclecticiq/platform_settings.py

All machines#

  • Update all yum packages in all machines hosting the IC instance:

# Update all packages
ES_PATH_CONF=/etc/eclecticiq-elasticsearch yum -y upgrade

  • Ensure Java 11 is installed and used

yum -y install java-11-openjdk-headless tzdata-java
update-alternatives --set java $(readlink -f /usr/lib/jvm/jre-11/bin/java)

  • Ensure that Elasticsearch, Kibana and Logstash are no longer using the -oss packages.

yum reinstall -y elasticsearch kibana logstash eclecticiq-elasticsearch eclecticiq-kibana eclecticiq-logstash

The Elasticsearch configuration path environment variable must point to the location where the Elasticsearch configuration files are stored, in the machine where Elasticsearch is installed: ES_PATH_CONF=/etc/eclecticiq-elasticsearch.

After completing the update, yum -upgrade automatically (re)starts the services to load any configuration changes.
The Elasticsearch service may fail to start automatically. This is expected; the upgrade handles it by starting all database services later on in the process (see Start the services — Database machines below).
If you want to manually start it at this point:

systemctl start elasticsearch

Neo4j machine#

If you are upgrading an IC instance installed on a single machine, skip this step.
This step applies only to upgrades of IC instances distributed across two or more machines.

The upgrade package may overwrite the existing neo4j.conf configuration file for Neo4j.
To restore the custom setting that instructs Neo4j to listen to public interfaces, manually run the command below.

  • In a two-machines or in a distributed setup, run the following command in the machine where the eclecticiq-neo4j package is installed:

echo "dbms.connectors.default_listen_address=0.0.0.0" >> /etc/eclecticiq-neo4j/neo4j.conf

Reinstall the extensions#

By default, the upgrade reinstalls all extensions:

# Uninstall old extensions from venv
/opt/eclecticiq-platform-backend/bin/pip freeze | grep '^eclecticiq-extension-' | grep -v '^eclecticiq-extension-api=' | cut -d= -f1 | xargs -r /opt/eclecticiq-platform-backend/bin/pip uninstall -y

# Install specified extensions in venv
/opt/eclecticiq-platform-backend/bin/pip install $extensions_packages

Start the services#

Database machines#

  • Start all databases and log aggregator components; that is, Elasticsearch, Logstash, PostgreSQL, Neo4j, and Redis:

systemctl start elasticsearch
systemctl start logstash
systemctl start postgresql-11
systemctl start neo4j
systemctl start eclecticiq-neo4jbatcher
systemctl start redis

Intelligence Center host#

  • Start Kibana and Statsite.
    Do not start the IC services, yet:

# Start kibana and statsite
systemctl start kibana
systemctl start statsite

  • Manually compare /etc/eclecticiq/platform_settings.py with the file with the same name, a timestamp, and a .bak extension; restore any overwitten custom settings, if necessary.

Database migrations#

As a last step in the IC installation process, you must run the database migration scripts.

  • In an installation on two machines, make sure you run the database migrations on the server hosting the IC.
  • In a distributed installation, make sure you run the database migrations on the server hosting the IC backend.

Important

To complete the installation procedure, make sure you run the database migrations before continuing with the other configuration steps.