Before you upgrade VM#

Note

The Rundoc-powered installation and upgrade script only supports:

  • Single machine installs.

  • Installations performed using EclecticIQ Intelligence Center (IC) install script.

If you are upgrading a distributed installation, you must perform the operation manually.

Before upgrading EclecticIQ Intelligence Center, we recommend that follow the instructions in this guide.

Disable rules#

Disable all Intelligence Center rules:

  • Entity rules

  • Observable rules

  • Enrichment rules

  • Discovery rules

To disable rules:

  1. Go to Data configuration (Data configuration icon) > Rules.

  2. For each of the rule types (Entity, Observable, Enrichment, Discovery), select its corresponding tab to open a list of those rules.

  3. Select the checkbox to the left of the Rule name column to select all visible rules.

    Tip

    If you have more items than are visible on the screen, you must either:

    • Increase the number of visible items per page and then select them.

    • Select Next page (>) and then select the newly selected items to add them to the list of currently selected items.

  4. Select More (More) > Disable from the list’s top-right corner to disable all selected rules.

Tip

To re-enable rules after finishing the upgrade:

  1. Follow the steps above.

  2. Instead of selecting More (More) > Disable, select More (More) > Enable.

Back up your data#

Before proceeding to upgrade the platform or any of its third-party components, always back up your data.

Stop EclecticIQ Intelligence Center#

Stop all backend services:

systemctl stop eclecticiq-platform-backend-services

Clear Celery queues#

  1. Use the redis-cli command to check that Celery queues are empty:

    # Start redis-cli in interactive mode
redis-cli

# Run these commands in the redis-cli shell
llen enrichers
llen integrations
llen priority_enrichers
llen priority_providers
llen priority_utilities
llen providers
llen reindexing
llen utilities

  2. If any of the queues are not empty, run the following commands to delete that queue:

    # Launch redis-cli
$ redis-cli

# Delete the entity ingestion queue
$ > del "queue:ingestion:inbound"

# Delete the graph ingestion queue
$ > del "queue:graph:inbound"

# Delete the search indexing queue
$ > del "queue:search:inbound"

  3. Stop the remaining Celery workers:

    systemctl stop eclecticiq-platform-backend-worker*.service

Clean up PID files#

Check that there are no leftover PID files

  1. Check for running IC processes:

    ps auxf | grep beat

  2. Run kill to stop any remaining IC processes.

  3. Manually remove any leftover PID files with the rm command.

    Usually, PID files are stored in /var/run.

Review configuration files#

IC configuration files#

EclecticIQ Intelligence Center stores configuration files in /etc/eclecticiq/. Back up these files before performing an upgrade.

Note

Release notes may instruct you to update these files for an upgrade.

Config file

Description

platform_settings.py

Contains core platform settings such as:

  • Security keys

  • Authentication bearer token expiration time

  • URLs pointing to external components

  • Celery-managed tasks

  • LDAP or SAML configuration.

opentaxii.yml

Contains OpenTAXII configuration parameters such as:

  • URL and port of the TAXII server

  • Inbound queue

  • Message broker.

Full list of configuration files to back up#

The following is a full list of configuration file locations. Back up these files before performing an upgrade:

  # General
  - /etc/environment
  - /etc/yum.repos.d/eclecticiq-ic.repo
  # Platform
  - /etc/eclecticiq/platform_settings.py
  - /etc/eclecticiq/opentaxii.yml
  - /etc/eclecticiq/proxy_url

  - /etc/default/eclecticiq-platform
  - /etc/default/eclecticiq-platform-backend-worker-outgoing-transports
  - /etc/default/eclecticiq-platform-backend-worker-common
  - /etc/default/eclecticiq-platform-backend-worker-outgoing-transports-priority
  - /etc/default/eclecticiq-platform-backend-worker-discovery
  - /etc/default/eclecticiq-platform-backend-worker-reindexing
  - /etc/default/eclecticiq-platform-backend-worker-discovery-priority
  - /etc/default/eclecticiq-platform-backend-worker-retention-policies
  - /etc/default/eclecticiq-platform-backend-worker-entity-rules-priority
  - /etc/default/eclecticiq-platform-backend-worker-retention-policies-priority
  - /etc/default/eclecticiq-platform-backend-worker-extract-rules-priority
  - /etc/default/eclecticiq-platform-backend-worker-utilities
  - /etc/default/eclecticiq-platform-backend-worker-incoming-transports
  - /etc/default/eclecticiq-platform-backend-worker-utilities-priority
  - /etc/default/eclecticiq-platform-backend-worker-incoming-transports-priority

  - /lib/systemd/system/eclecticiq-platform-backend-graphindex.service
  - /lib/systemd/system/eclecticiq-platform-backend-ingestion.service
  - /lib/systemd/system/eclecticiq-platform-backend-ingestion@.service
  - /lib/systemd/system/eclecticiq-platform-backend-opentaxii.service
  - /lib/systemd/system/eclecticiq-platform-backend-scheduler.service
  - /lib/systemd/system/eclecticiq-platform-backend-searchindex.service
  - /lib/systemd/system/eclecticiq-platform-backend-services.service
  - /lib/systemd/system/eclecticiq-platform-backend-web.service
  - /lib/systemd/system/eclecticiq-platform-backend-worker@.service
  - /lib/systemd/system/eclecticiq-platform-backend-workers.service
  - /lib/systemd/system/eclecticiq-secrets-setter.service

  - /opt/eclecticiq-platform-backend/alembic.ini

  # ElasticSearch
  - /etc/eclecticiq-elasticsearch/elasticsearch.yml
  - /etc/eclecticiq-elasticsearch/jvm.options
  - /etc/eclecticiq-elasticsearch/log4j2.properties
  - /etc/elasticsearch/elasticsearch-plugins.example.yml
  - /etc/elasticsearch/elasticsearch.keystore
  - /etc/elasticsearch/elasticsearch.yml
  - /etc/elasticsearch/jvm.options
  - /etc/elasticsearch/log4j2.properties
  - /etc/elasticsearch/role_mapping.yml
  - /etc/elasticsearch/roles.yml
  - /etc/elasticsearch/users
  - /etc/elasticsearch/users_roles
  - /etc/systemd/system/elasticsearch.service.d/20-eclecticiq.conf
  - /etc/sysconfig/elasticsearch

  - /media/elasticsearch/nodes
  - /media/elasticsearch/tmp

  # Kibana
  - /etc/eclecticiq-kibana/kibana.yml
  - /etc/kibana/kibana.yml
  - /etc/kibana/node.options
  - /etc/systemd/system/kibana.service.d/20-eclecticiq_es_hosts.conf

  # Logstash
  - /etc/logstash/logstash.yml
  - /etc/logstash/jvm.options
  - /etc/logstash/log4j2.properties
  - /etc/logstash/logstash-sample.conf
  - /etc/logstash/pipelines.yml
  - /etc/logstash/startup.options
  - /etc/logstash/conf.d/eclecticiq.conf
  - /etc/default/logstash
  - /etc/systemd/system/logstash.service.d/20-eclecticiq-env-vars.conf

  # Neo4j
  - /etc/eclecticiq-neo4j/neo4j.conf
  - /etc/eclecticiq-neo4j/template-neo4j.conf

  # Neo4jbatcher, together with platform conf.
  - /etc/eclecticiq-neo4jbatcher/neo4jbatcher.conf
  - /lib/systemd/system/eclecticiq-neo4jbatcher.service
  - /etc/systemd/system/eclecticiq-neo4jbatcher.service.d/20-eclecticiq.conf

  # statsite
  - /opt/statsite/etc/statsite.conf
  - /opt/statsite/etc/elasticsearch_template.json
  - /opt/statsite/etc/statsite.service
  - /etc/systemd/system/statsite.service.d/override.conf

  # Redis
  - /etc/eclecticiq-redis/redis.conf
  - /etc/eclecticiq-redis/local.conf
  # - /etc/redis/redis.conf
  - /etc/systemd/system/redis.service.d/20-eclecticiq_data_dir.conf
  - /etc/sysctl.d/10-eclecticiq_overcommit_memory.conf

  # Nginx
  - /etc/eclecticiq-nginx/locations.conf.d/neo4jbatcher.conf
  - /etc/eclecticiq-nginx/locations.conf.d/platform-frontend.conf
  - /etc/eclecticiq-nginx/locations.conf.d/tip-backend.conf
  - /etc/eclecticiq-nginx/nginx.centos.conf
  - /etc/eclecticiq-nginx/nginx.common.conf
  - /etc/eclecticiq-nginx/nginx.conf
  - /etc/eclecticiq-nginx/nginx.rhel.conf
  - /etc/eclecticiq-nginx/nginx.ubuntu.conf
  - /etc/eclecticiq-nginx/proxy_params.conf
  - /etc/eclecticiq-nginx/sites.conf.d/eclecticiq-default.conf
  - /etc/systemd/system/nginx.service.d/20-eclecticiq.conf

  # Postgres
  - /etc/eclecticiq-postgres/eclecticiq-postgres.conf
  - /etc/eclecticiq-postgres/listen-addresses.conf
  - /etc/eclecticiq-postgres/pg_hba.conf
  - /etc/systemd/system/postgresql-11.service.d/eclecticiq-postgres.conf
  - /media/pgsql/11/data/postgresql.conf

  # Postfix
  - /etc/postfix/main.cf

  # Logrotate
  - /etc/logrotate.d/eclecticiq

  # Rsyslog
  - /opt/eclecticiq-rsyslog-forwarder
  - /etc/rsyslog.d/eclecticiq.conf

Elasticsearch#

See Before you upgrade: Elasticsearch 7 VM.

About databases and network bindings#

On a single machine installation, network interface bindings for services are set to 127.0.0.1 by default, except for PostgreSQL which has a different configuration.

Instructions may have asked you to change this to a more permissive binding in multi-machine installations, or you may be using an older installation where defaults were set to 0.0.0.0.

The table below shows a list of configuration files where network interface bindings are set for each service.

You may want to change these bindings to suit your environment.

Service name

File path(s)

Parameters

Notes

Elasticsearch

 
/etc/systemd/system/elasticsearch.service.d/20-eclecticiq.conf

[Service]
Environment=BINDING_ADDRESS=127.0.0.1

For more information, see Elasticsearch’s documentation.

Neo4j

 
/etc/eclecticiq-neo4j/neo4j.conf

dbms.connector.bolt.listen_address=:7687
dbms.connector.http.listen_address=:7474
dbms.connector.https.listen_address=:7473

dbms.connectors.default_listen_address is left unset, and defaults to 127.0.0.1.

For more information, see Configure connectors and dbms.connectors.default_listen_address.

PostgreSQL

 
/etc/eclecticiq-postgres/pg_hba.conf

TYPE    DATABASE        USER            ADDRESS                 METHOD
local   all             postgres                                trust
host    all             all             samenet                 md5
host    all             all             0.0.0.0/0               password

For more information, see The pg_hba.conf File.

Redis

 
/etc/eclecticiq-redis/redis.conf

bind 127.0.0.1

For more information, see Redis security and redis.conf.