After upgrading VM

Check the configuration

After installing the platform, browse to /etc/eclecticiq/.
Configuration files are stored here. You can find both the new/latest configuration files, as well as the ones belonging to the previous version of the platform you upgraded from.


Core platform configuration files

platform_settings.py

Contains core platform settings such as:

  • Security keys

  • Authentication bearer token expiration time

  • URLs pointing to external components

  • Celery-managed tasks

  • LDAP or SAML configuration.

opentaxii.yml

Contains OpenTAXII configuration parameters such as:

  • URL and port of the TAXII server

  • Inbound queue

  • Message broker.

Verify that the platform configuration files reflect the new, upgraded environment.
You may need to carry out this task manually. In this case, the files with a tool like Meld.

Check third-party configurations

After checking the platform configuration to make sure it correctly describes the upgraded environment, do the same with the configurations of third-party components and dependencies.
You may have to carry out this task manually. In this case, the files with a tool like Meld.

About proxy settings

If the platform is configured to access the Internet through a proxy, make sure that the platform proxy configuration always bypasses local hosts localhost and 127.0.0.1.

If you cannot access the platform proxy settings, and if terminal commands acting on platform resources fail to execute correctly, bypass the proxy server on the fly.
To do so, prepend NO_PROXY='127.0.0.1,localhost' to the platform commands you execute in the terminal.

Example

NO_PROXY='127.0.0.1,localhost' /opt/eclecticiq-platform-backend/bin/eiq-platform graph upgrade

About Elasticsearch indices

If you need to prioritize migrating Elasticsearch indices, process at least the following ones:

  • stix: indexes entities

  • extracts: indexes observables


Elasticsearch index name

Description

audit*

Records audit trail events related to entities, datasets, enrichers, incoming and outgoing feeds, rules, tasks, and user account authentication attempts.

documents

Records log information related to ingestion, tasks, and task scheduling.

draft-entities*

Indexes draft entity data, that is, entities that are currently saved as drafts, and that have not yet been published to the platform.

These entities are not searchable in the platform.

extracts*

Indexes all observable data.

logstash*

Indexes log aggregation and logging information such as host, HTTP request types, HTTP response status codes, platform component, and path to the log directories where log entries are saved to.

statsite*

Collects metrics about received packets and detected invalid or not well-formed lines in ingested packets.

This index works with both StatsD and Statsite.

stix*

Indexes published entity data, that is, entities that are published to the platform.

These entities are searchable in the platform.

Run a final check

As a last step before launching the platform, it is good practice to check the following points:

  • Core processes and services

  • Search, indexing and graph

  • Availability

Check core processes and services

  • To check if a core service is enabled to start at system bootup:

    systemctl is-enabled ${service_name}

  • To check if a core service is running:

    systemctl status ${service_name}

  • To start a core service:

    systemctl start ${service_name}

Nginx

Verify that Nginx is up and running by checking the web server status:

systemctl status nginx

PostgreSQL

Verify that PostgreSQL is up and running by checking its status:

systemctl status postgresql-11
 
# Or:
systemctl list-units | grep -i postgre

Check search indexing and graph

Elasticsearch

Verify that Elasticsearch is up and running by checking its status:

systemctl status elasticsearch

Neo4j

Verify that Neo4j is up and running by checking its status:

systemctl status neo4j

Check search indexing and graph availability

Make sure that Elasticsearch and Neo4j are available by sending cURL requests to the corresponding endpoints:


  • # Check Elasticsearch availability
    curl localhost:9200

  • # Check Neo4j availability
    # HTTP port: 7474; HTTPS port: 7473
    curl localhost:7474

Re-enable and run the rules

Before starting the ingestion processes, enable again the rules you previously disabled.
Run the re-enabled rules after completing the data migration, so that they can filter out any observables marked to be ignored.

Enable all existing platform rules: entity, observable, enrichment, and discovery rules.

You can enable rules in one of the following ways:

In the rule detail pane

  1. Click Data configuration > Rules > Observable ; or: Data configuration > Rules > Entity; or: Data configuration > Rules > Enrichment; or: Data configuration > Rules > Discovery to display the observable, entity, enrichment, or discovery rule overview.

  2. In the rule overview click anywhere in the row corresponding to the rule you want to enable.

  3. In the rule detail panel:

    1. Click > Enable to enable the rule.
      Alternatively:

    2. In the Details tab click Enable.

A notification message is displayed to confirm the change.

In the rule overview

  1. Click Data configuration > Rules > Observable ; or: Data configuration > Rules > Entity; or: Data configuration > Rules > Enrichment; or: Data configuration > Rules > Discovery to display the observable, entity, enrichment, or discovery rule overview.

  2. In the row corresponding to the rule you want to enable, click > Enable.

A notification message is displayed to confirm the change.

Bulk enable

  1. Click Data configuration > Rules > Observable ; or: Data configuration > Rules > Entity; or: Data configuration > Rules > Enrichment; or: Data configuration > Rules > Discovery to display the observable, entity, enrichment, or discovery rule overview.

  2. In the top-left corner click the quick filter icon images/download/attachments/3604538/filter.PNG to display the available rule quick filters.

  3. Click Show, select Disabled, and then click OK to display only disabled rules.

  4. To select all the rules on the view, click the checkbox in the top-left corner of the table.

  5. To enable all the selected rules in bulk, in the quick filter horizontal bar click > Enable.

A notification message is displayed to confirm the change.

Restart platform services

After editing or updating systemd-managed unit configuration files, you must restart all systemd-managed platform services.
It enables systemd to reload all configurations, and to apply any changes to make them effective.

To restart systemd-managed platform services through the command line:

systemctl restart eclecticiq-platform-backend-services

Install extensions

After successfully completing the platform upgrade, you can proceed to install extensions as necessary to expand platform functionality, and to add support for a broad range of transport types and content types for incoming and outgoing feeds, as well as many enrichers.

To avoid compatibility issues while doing a major library upgrade or change, rebuild the Python virtual environment.
We recommend relying on automatic OS upgrades, instead of manually rebuilding packages.