Check sequence generation capabilities#
EclecticIQ Intelligence Center (EIQ IC) depends on sequence generators to provide IDs for the records in its databases. Although this generator can provide a large number of IDs (about 2 billion), we’ve recently seen the first instances of the sequence exceeding the generators’ ability to provde IDs.
When this happens, the generators and databases need to be migrated to an architecture that allowsfor the generation of a significantly larger number of IDs. Because the part of the records will have been replicated in the ElasticSearch (ES) database a migration of the ES database is also necessary, in order for it to accept the new ID format.
Mitigation strategy
Read the instructions for the scenario matching your situation below.
The best mitigation strategy is to upgrade to v3.4.4 and then v.3.5.0 as soon as they release, because they come with built-in sequence generator health check and database migration, respectively.
Upgrading to v3.4.4#
EIQ IC v3.4.4 comes packaged with a health check that will warn you when the sequence generator is approaching its limit. If it does, follow the steps below.
Manually migrate your PG and ES databases:
This procedure consists of the following steps:
Export the platform admin user credentials
Set the platform CLI
Stop services
Set secret key
Run data migrations
Start and enable services
Export the platform admin user credentials
Set and export the platform admin user credentials:
EIQ_ADMIN_USERNAME=admin
EIQ_ADMIN_PASSWORD=
EIQ_ADMIN_EMAIL=
EIQ_ADMIN_USERNAME
: defines the user name of the user that is granted admin-level access rights to the platform.
There must always be at least one platform admin user.
You can change the default platform admin user name value.
Default value:admin
EIQ_ADMIN_PASSWORD
: defines the password associated with the platform admin user.
For more information, see Password guidelines and limitationsEIQ_ADMIN_EMAIL
: defines a valid email address associated with the platform admin user.
The platform sends automated messages and notifications to this email address.
Set the platform CLI
The platform command line interface includes a set of tools and utilities to support system administrators with platform setup, configuration, and management tasks.
Assign the path of the platform command line interface to the variable.
This makes it easier to invoke the command line tool in the bash session:
eiq_platform=/opt/eclecticiq-platform-backend/bin/eiq-platform
eiq_platform
: leave as is/do not change it.
Defines the predefined path value to the platform command line interface tool.
Stop services
Stop all platform services:
# Stop all platform services
systemctl stop eclecticiq-platform-backend-services
systemctl stop eclecticiq-platform-backend-workers
Set secret key
Start and enable the secret setter service to set the platform secret key:
# Start and enable the secret setter service
systemctl start eclecticiq-secrets-setter
systemctl enable eclecticiq-secrets-setter
Run data migrations
Migrate the PostgreSQL database:
# Migrate the PostgreSQL database
$eiq_platform database upgrade
After migrating the PostgreSQL database, proceed to load the database fixtures:
# Load the fixtures
$eiq_platform database load-fixtures
Migrate the Elasticsearch database:
# Migrate the Elasticsearch database
yes | $eiq_platform search upgrade
Start and enable the services
As a last step after completing the database upgrade process, start the platform services.
Run this command to start the platform services, and to enable them to automatically start at system boot:
# start/enable platform services
systemctl start eclecticiq-platform-backend-services
systemctl enable eclecticiq-platform-backend-services
The database migration process is completed.
Not upgrading to 3.4.4#
To see if your EIQ IC instance is close to running out of IDs, please run the script provided below:
SELECT
sequencename,
(
(last_value / max_value::REAL) * 100
)::NUMERIC(10, 2) AS percent_usage
FROM
pg_sequences
WHERE
max_value > 0
AND (last_value / max_value::REAL) > 0.7
ORDER BY
percent_usage DESC
LIMIT
50
;
If the script returns one or more rows with more than 70% sequence usage
It means your database is at risk of running out of available IDs. You can carry out a migration manually.
>90% usage
If the script returns one or more rows with more than 90% sequence usage, immediate migration is recommended.
Manually migrate your PG and ES databases:
This procedure consists of the following steps:
Export the platform admin user credentials
Set the platform CLI
Stop services
Set secret key
Run data migrations
Start and enable services
Export the platform admin user credentials
Set and export the platform admin user credentials:
EIQ_ADMIN_USERNAME=admin
EIQ_ADMIN_PASSWORD=
EIQ_ADMIN_EMAIL=
EIQ_ADMIN_USERNAME
: defines the user name of the user that is granted admin-level access rights to the platform.
There must always be at least one platform admin user.
You can change the default platform admin user name value.
Default value:admin
EIQ_ADMIN_PASSWORD
: defines the password associated with the platform admin user.
For more information, see Password guidelines and limitationsEIQ_ADMIN_EMAIL
: defines a valid email address associated with the platform admin user.
The platform sends automated messages and notifications to this email address.
Set the platform CLI
The platform command line interface includes a set of tools and utilities to support system administrators with platform setup, configuration, and management tasks.
Assign the path of the platform command line interface to the variable.
This makes it easier to invoke the command line tool in the bash session:
eiq_platform=/opt/eclecticiq-platform-backend/bin/eiq-platform
eiq_platform
: leave as is/do not change it.
Defines the predefined path value to the platform command line interface tool.
Stop services
Stop all platform services:
# Stop all platform services
systemctl stop eclecticiq-platform-backend-services
systemctl stop eclecticiq-platform-backend-workers
Set secret key
Start and enable the secret setter service to set the platform secret key:
# Start and enable the secret setter service
systemctl start eclecticiq-secrets-setter
systemctl enable eclecticiq-secrets-setter
Run data migrations
Migrate the PostgreSQL database:
# Migrate the PostgreSQL database
$eiq_platform database upgrade
After migrating the PostgreSQL database, proceed to load the database fixtures:
# Load the fixtures
$eiq_platform database load-fixtures
Migrate the Elasticsearch database:
# Migrate the Elasticsearch database
yes | $eiq_platform search upgrade
Start and enable the services
As a last step after completing the database upgrade process, start the platform services.
Run this command to start the platform services, and to enable them to automatically start at system boot:
# start/enable platform services
systemctl start eclecticiq-platform-backend-services
systemctl enable eclecticiq-platform-backend-services
The database migration process is completed.
If the script returns 0 rows
It means your database isn’t close to running out of available IDs. The mitigation packaged with v3.5.0 and later releases will prevent it from becoming a problem.
Mitigations if not upgrading
If you are not planning an upgrade to v3.5.0 or a later version in the foreseeable future make sure you manually migrate your databases as the ID limit might still cause issues.
If you’re not planning an upgrade to v3.4.4, make sure to intermittenly