Packaging system CentOS#

EclecticIQ Platform packaging system#

This article provides a high-level description of the platform installation packaging system organization and functionality.

About the EclecticIQ Platform packaging system#

EclecticIQ Platform release 2.5.0 included a thorough overhaul of the platform installation packaging system. We rebuilt the packaging system to introduce two improvements:

  • Simplify installing the platform in a distributed environment.

  • Reduce user intervention during installation and upgrade.
    Ideally, upgrading should only involve running apt-get upgrade or yum upgrade.
    Although we are not there yet, we are making progress towards this goal.

About the package hierarchy#

The platform installation packages include:

  • Packages with platform components.

  • Packages with third-party dependencies.

  • Metapackages, for orchestration.

Platform packages:

  • Backend:

    • Web server / API

    • Ingestion

    • Graph indexing

    • Search indexing

    • Workers

  • Frontend

  • Documentation

Packages with third-party dependencies:

  • Databases:

    • Elasticsearch

    • Neo4j

    • PostgreSQL

    • Redis

  • Kibana

  • Logstash

  • Nginx

  • Statsite

Metapackages act like package herders: they group together a set of packages to make it easier to install it.
Metapackages group packages based on functional area.

Example:

  • The eclecticiq-platform-backend-all package fetches and installs all the necessary resources to deploy and to configure the platform backend on a single target machine.

  • The eclecticiq-platform-databases package fetches and installs all the necessary resources to deploy the databases the platform uses.

Go to the packaging system dependency structure section to see the dependency tree.

About setting default values#

During installations or upgrades, it is possible to set default values to apply every time you install or upgrade the platform. This minimizes manually editing configuration files as a post-installation task.

Although it is currently not possible to enable granular control in the preconfiguration options, the feature enables setting default values for most commonly edited configuration values. Again, we are making progress towards this goal.

  • On CentOS and RHEL, the platform enables preconfiguration through the /etc/default/eclecticiq-platform file.

The following examples demonstrate how to to preconfigure database access credentials through the /etc/default/eclecticiq-platform file on CentOS and RHEL.

  • On CentOS and RHEL, add the following lines to /etc/default/eclecticiq-platform:

db_name=eclecticiq_platform
db_user=custom_username
db_pass=VerySecurePassword

Preconfiguration variables#

On CentOS and RHEL, you can create a preconfiguration by setting key/value pairs, and by saving them to the /etc/default/eclecticiq-platform file.
These are the supported variables you can assign preconfiguration values to:

# PostegreSQL, Elasticsearch, Neo4j
data_dir          | Defines the default mount point directory the database is stored to.

# PostegreSQL
db_host           | Defines the hostname identifying the location in the network of the PostgreSQL database service.
db_name           | Defines the name of the main PostgreSQL database.
db_pass           | Defines the password associated with the user that is granted access to the PostgreSQL database service.
db_user           | Defines the user name of the user that is granted access to the PostgreSQL database service.

# Elasticsearch
es_host           | Defines the hostname identifying the location in the network of the Elasticsearch database service.
es_port           | Defines the (open) port the Elasticsearch database service listens on.
es_protocol       | Defines the transport protocol used to communicate with the Elasticsearch database service.

# Kibana
kibana_host       | Defines the hostname identifying the location in the network of the Kibana service.

# eclecticiq-neo4jbatcher
neo4jbatcher_host | Defines the hostname identifying the location in the network of the eclecticiq-neo4jbatcher preprocessing service.
neo4jbatcher_pass | Defines the password associated with the user that is granted access to the eclecticiq-neo4jbatcher preprocessing service.
neo4jbatcher_user | Defines the user name of the user that is granted access to the eclecticiq-neo4jbatcher preprocessing service.

# Neo4j
neo4j_host        | Defines the hostname identifying the location in the network of the Neo4j graph database service.
neo4j_pass        | Defines the password associated with the user that is granted access to the Neo4j graph database service.

# Redis
redis_host        | Defines the hostname identifying the location in the network of the Redis database service.
redis_pass        | Defines the password associated with the user that is granted access to the Redis database service.

# Platform
secret_key        | Defines the generated platform secret key associated with the `EIQ_SECRET_KEY` platform setting parameter.
syslog_server     | Defines the hostname identifying the location in the network of the Logstash service.

About storing configurations#

All EclecticIQ Platform packages store their configuration files to /etc.
/etc subdirectories holding platform configuration files have names that always start with eclecticiq-.
If you need to edit or to update platform configuration files, browse first to these locations.
For example, the /etc/eclecticiq-redis directory contains the platform custom configuration for Redis.

There may be exceptions to this rule. For example, when the configured service is set up to include the platform custom configuration, or when the platform custom configuration file contains default/boilerplate settings.

About systemd#

From EclecticIQ Platform release 2.5.0, systemd runs and manages all platform-related services.
To control these services, you use standard systemd commands.

systemd command examples#

The following examples demonstrate typical systemd commands in the platform context.

  • To restart the platform API / the web server:

systemctl restart eclecticiq-platform-backend-web

  • To obtain a list of all available platform services:

systemctl list-unit-files 'eclecticiq*'

  • To obtain a list of all running platform services:

systemctl list-units 'eclecticiq*'

  • To restart all running platform services:

systemctl restart $(systemctl list-units 'eclecticiq*' | awk '{print $1}')

About logging#

systemd sends platform process logs to rsyslog to achieve the following goals:

  • Write platform process logs to files, and store them to /var/log/eclecticiq.

  • Send platform process logs to Logstash, so that log data is:

    • Indexed in Elasticsearch.

    • Viewable and browsable in Kibana.

All platform processes also output their logs in JSON format to stderr.
To browse logs in the terminal, use journalctl.
For example, to view the platform API process logs:

journalctl -u eclecticiq-platform-backend-web

The recommended way to review log file information is through Kibana, or by opening log files directly. Use journalctl as a fallback option.

Packaging system dependency structure#