Release notes 3.0.0#

Product	EclecticIQ Intelligence Center
Release version	3.0.0
Release date	8 May 2023
Time to upgrade	~40 minutes to upgrade an instance with 2.67 million entities, 1.85 million observables. From the previous release Using the installation script For an instance running on one machine
Time to migrate	For an instance with 2.67 million entities, 1.85 million observables: PostgreSQL migration: 13m30s Elasticsearch migration: 18m40s

Highlights #

EclecticIQ Intelligence Center 3.0.0 is a major release that major step forward in the development of our Threat Intelligence Platform and strengthens its foundation for your future CTI use cases.

This release introduces significant improvements to the underlying data model, which enables users to exchange and create more granular and accurate threat intelligence using eight additional STIX 2.1 compatible objects. As a result, the traditional TTP object used for all types of TTPs will be phased out at the end of the year. However, with a self-defined rule, users can easily convert these entities to their new corresponding STIX 2.1 entities. With the ability to freely create and modify relationships with all entity types and data fields, users can take advantage of the STIX 2.1 predefined labels or create their own. Additionally, an updated version (v2) of the Public REST API is available, allowing users to create even more tailored automated workflows than before.

Furthermore, Intelligence Center 3.0 includes improvements to the rules feature, such as path auto-completion for creating detailed entity rules more easily and the ability to apply a specific action to multiple sources using a single entity rule, reducing the number of rules users need to create and maintain. Users can also use auto-updating rules for whitelisting observables by installing a newly available Knowledge Pack.

Users will appreciate the updated built-in MITRE ATT&CK Enterprise framework to the latest version (v12.1). This means they can characterize techniques and tactics with the highest precision. They can also create and share new observable types to increase the visibility of new kinds of threats and assign confidence scores to all entity types present in the platform.

We are excited to present the addition of a dark mode. Users can now choose a color scheme that uses light-colored text, icons, and graphical user interface elements on a dark background to reduce eye strain or for personal preference. Users can also configure Intelligence Center to switch modes automatically according to their system settings.

Lastly, users can rest assured that they will not lose any of their workflow configurations when upgrading from the previous release. Intelligence Center 2.14 is fully forward compatible with this version, so you can start using it immediately without needing to change the way you operate.

We hope you find this information useful and enjoy the new features and improvements offered in this release.

Important #

Request EclecticIQ Intelligence Center 3.0 license key #

If you are upgrading from EclecticIQ Intelligence Center 2.x, please request a new license key for your EclecticIQ Intelligence Center 3.x instance. Existing EclecticIQ Intelligence Center 2.x license keys are not compatible with EclecticIQ Intelligence Center 3.0 and newer.

To get a license key for EclecticIQ Intelligence Center 3.0 and newer, please email EclecticIQ’s Customer Success team at csm@eclecticiq.com.

Upgrade operating system #

Important

EclecticIQ Intelligence Center 3.0.0 and newer requires one of these supported operating systems:

Red Hat Enterprise Linux 8
Rocky Linux 8

If you are using an older operating system such as CentOS 7 or RHEL 7, you must upgrade your operating system to one of the supported operating systems before attempting to install EclecticIQ Intelligence Center 3.0.

See:

What’s new #

New and updated entity types #

This release:

Adds 8 new STIX 2.1-compatible entity types: Infrastructure, Malware, Malware Analysis, Intrustion set, Attack patterns, Tool, Identity, and Location.
Deprecates the TTP entity. The entity is still available for use, with plans for removal in future EclecticIQ Intelligence Center versions.
Changes the Threat Actor entity: fields have been changed to support STIX 2.1 by default. This updated Threat Actor entity is still compatible with older versions of EclecticIQ Intelligence Center, and can be exported to STIX 1.2.
You can convert TTP and Threat Actor entities to STIX 2.1-compatible entity types with the new Convert entity entity rule action. To create entity rules, go to Data configuration > Rules > Entity rules.

These entities are fully supported for incoming feeds, manual uploads, manual exports, and outgoing feeds that use the EclecticIQ JSON and STIX 2.1 content types.

Upgraded relationships for entities #

Relationships between entities are now more flexible, and allow you to add metadata directly to them. You can now create relationships between any two entities, add properties such as TLP and a start/stop time, and set a STIX 2.1-compliant type or a custom type/label for it.

Dark and light mode #

Users can select the default appearance by going to Dark and light mode in their user account settings.

Public API v2 #

Public API v2 builds upon Public API v1, and allows you to programmatically access Intelligence Center features and work with the data stored in your instance.

MITRE ATT&CK Enterprise v12.1 #

MITRE ATT&CK Enterprise v12.1 classifications are now available. v12.1 and v9 classifications will co-exist on EclecticIQ Intelligence Center, but will prefer v12.1 where a particular classification is modified.

You will continue to be able to use and search for entities by their v9 and v12.1 classifications.

Known issue

MITRE ATT&CK classifications that have been renamed or relocated (ATT&CK ID has changed) in ATT&CK v12 will no longer be searchable by their older names or ID.

Queries (e.g. used in Dynamic Datasets) that depend on an ATT&CK ID or name that has changed in v12 may fail because of this.

Set Confidence level for all entities #

You can now set a confidence level for all entities.

Path selector now has autocomplete #

Some features such as the complex content criteria tool in entity rules allow you to build rules using a path selector to target specific fields in an entity. This path selector is now improved, allowing you to write entity paths with the help of autocomplete.

New observable types #

Added support for new observable types:

certificate-serial-number
cpu-architecture
crypto-address
file-size
hash-sha224
hash-sha384
malware-key
process-name
region
user-agent

Introduce support for Extensions Developer Kit (Preview)#

This release adds the ability to connect to the upcoming Extensions Developer Kit (EDK). The EDK is a suite of tools that allow developers to build and run new EDK-based extensions alongside existing extensions on EclecticIQ Intelligence Center. The EDK will be announced and released separately later. But you will soon be able to start using a preview of the EDK as an EclecticIQ Labs feature. See EclecticIQ Labs For more information on this preview access, please contact your customer success manager.

Convert entities to STIX 2.1-compatible entities #

Entity rules have a new Convert entity for converting TTPs and threat actor entities to STIX 2.1-compatible entities:

TTP entities can be converted to: Attack pattern, Malware, Malware (family), Infrastructure, Tool, Identity entities.
Threat actor entities can be converted to: Intrusion set entity.

Neo4j has been removed #

This release no longer installs Neo4j and its related services, significantly reducing the resources that EclecticIQ Intelligence Center requires to run.

If you are upgrading from an earlier version of EclecticIQ Intelligence Center, remove Neo4j and its related components by running on EclecticIQ Intelligence Center instance (for single node setups):

# Stop EclecticIQ Intelligence Center services
systemctl stop eclecticiq-platform-backend-services
systemctl stop eclecticiq-platform-backend-workers

# Remove Neo4j and components
rpm -e --nodeps eclecticiq-neo4jbatcher eclecticiq-platform-backend-graphindex eclecticiq-neo4j neo4j || true

# Start EclecticIQ Intelligence Center services
systemctl start eclecticiq-platform-backend-services
systemctl start eclecticiq-platform-backend-workers

Improvements #

Updates to rules #

Entity rules now allow you to select multiple sources when setting Criteria selection > Sources.
Default observables rules are now provided by the Best Practice - Observable Rules knowledge pack.

General improvements to feeds #

Incoming feeds: Port observables are now automatically extracted from URIs detected in unstructured text when Skip extraction of observables from unstructured text is left disabled.
Outgoing feeds: HTML reports and PDF content types now display a TLP field.

Search improvements #

When searching for entities by the contents of their data.description field, you can now search for a value that is more than 256 characters long.

Fixes #

Entity rules: predefined Path items for content criteria may not work

Content criteria in entity rules now allow you to set and select paths with autocompletion. Start typing in the Path field to see possible paths to set for a condition in your content criteria.

Observable IDs in STIX 1.2 exports do not use UUID, causing certain feeds to fail

When exporting entities as STIX 1.2 XML, observables linked to that entity are exported as cybox:Observable objects that are assigned a QName like <stix12_namespace_alias>:<observable_kind>-<id>. In earlier versions of EclecticIQ Intelligence Center, <id> is a number, because STIX 1.2 does not require embedded objects to use a UUID. This release onward, observables in STIX 1.2 exports are assigned a QName with a UUID. E.g.: cti_producer:Observable-851974e5-4ded-545f-b9c1-24bea928428d.

Tasks/tickets created through the REST API results in error when edited in the UI

Fixes an issue where tasks/tickets created through the REST API causes an error when subsequently edited in the UI.
Report attachments with malformed filenames cannot be downloaded through Public API v1

Fixes an issue where an attachment containing a carriage return in its filename could be ingested from an external source and subsequently cannot be downloaded using the Public API v1.
Observables have an incorrect count of ‘Sightings’ because of duplicate counts

Fixes issue where observables directly linked to a sighting entity would have their “sighted” count incorrectly increased.
SHA-256 hashes that are split by newlines are now correctly extracted

Fixes issue where SHA-256 hashes would not be extracted on ingestion if the hash is split by a newline or carriage return.
Add validation for API key fields in incoming feeds

Fixes an issue where certain incoming feeds would fail silently if a non-ASCII character is entered in the API key field of certain feeds.
Non-admin users were not receiving discovery task notifications

Fixes issue where users without administrator privileges would not be able to receive discovery task UI and email notifications.

Prevent false positive “Unauthorized” entries in audit log

Fixes an issue where if a user has EclecticIQ Intelligence Center open in multiple tabs, the audit log shows false positives.
“Reingest all failed” fails with timeout when there is a large number of blobs to reprocess

Fixes issue where selecting “Reingest all failed” to retry ingesting a large number ofo failed blobs in a given incoming feed fails with a HTTP timeout.
Outgoing feed schedules now uses localized time instead of only UTC

Security fixes #

Tip

To see a detailed list of security issues and their mitigations, go to All security issues and mitigations.

EIQ-2022-0004

Fixes issue where users with only read knowledge-packs permissions can delete knowledge packs from EclecticIQ Intelligence Center by sending a DELETE /knowledge-packs/{id} request.
EIQ-2023-0001

Fixes issue where users could create a report entity and reference an image in an <img> or <embed> tag, and cause that image to be embedded and displayed in a PDF exported from that entity.
EIQ-2023-0002

Fixes issue where EclecticIQ Intelligence Center is vulnerable to server-side request forgery (SSRF) and directory traversal attacks through PDF exports that allow users to load data from the local filesystem or from an external URI when the exported report entity contains anchor tags (<a>) in the Summary or Analysis fields.

Known issues #

Changes are lost if, while creating a new entity, the entity fails to publish

While creating a new entity, if the entity fails to save when selecting Publish, the work-in-progress entity can be lost. To avoid this, select Save draft to save a draft before selecting Publish.
Queries that depend on an ATT&CK ID or name that has changed in v12 may fail

MITRE ATT&CK classifications that have been renamed or relocated (ATT&CK ID has changed) in ATT&CK v12 will no longer be searchable by their older names or ID. Queries (e.g. used in Dynamic Datasets) that depend on an ATT&CK ID or name that has changed in v12 may fail because of this.
TLPs applied to relationship objects are not affected by TLP filters

You can now add TLP colors to relationship objects. However, you cannot use TLP colors with TLP filters yet.
Selecting TLP in entity view to override it does not apply to exports

Edit the entity to change its TLP, or override TLPs at feed level instead.
Certain entities added in 3.0 and newer will cause a STIX 1.2 outgoing feed to fail

Including certain entities in an outgoing feed using the STIX 1.2 content type will cause the feed to fail. Entities affected: Location, Identity, and Malware Analysis.
Certain entities added in 3.0 and newer display an option to export as STIX 1.2, but cannot

Nothing happens when Export > STIX 1.2 is selected for Location, Identity, and Malware Analysis entities. These entity types are not compatible with STIX 1.2 exports.
Exploit Target entities with references can create an invalid STIX 2.1 bundle on export

Exploit Target entities have an optional Vulnerability characteristic where you can set additional information. When an Exploit Target with References set in the Vulnerability characteristic, exporting to STIX 2.1 by default sets the type of these references to CVE, which causes an invalid STIX 2.1 bundle to be created if the set references are not valid CVE-IDs.
STIX 2.1 for outgoing feeds: TLP override and filtering has side-effects

See STIX 2.1 Known issues for a list of known issues.
When deleting content of an incoming feed, deleted observables are not included in the count of deleted objects.
Using STIX 2.1 content type to transmit data from one EclecticIQ Intelligence Center instance to another generates duplicates

When using the STIX 2.1 content type to send intelligence from one EclecticIQ Intelligence Center instance (Instance A) to another (Instance B), any updates to entities on Instance A that has already been sent to Instance B will result in duplicate entities being sent to Instance B instead of updating existing entities there.
When upgrading from 2.14 to 3.0, entities with certain fields that contain null values may cause database migrations to fail

In rare instances when upgrading from EclecticIQ Intelligence Center 2.14 to 3.0, older entities with null values in certain fields that don’t expect it may cause the database migration to fail, due to stricter validation of entity schemas. If this occurs, do not continue. Save the trace log and contact customer support for assistance to remediate.
Delete observable actions in policies may cause policies to run for excessively long periods of time.

As of 2.12.0, Delete observable actions are skipped by default to allow policies to run more reliably.
Elasticsearch 7 encounters “Data too large” errors: See Elasticsearch 7: “Data too large”.
Systemd splits log lines exceeding 2048 characters into 2 or more lines.

As a result, log lines exceeding 2048 characters become invalid JSON, causing Logstash to be unable to parse them correctly.
When more than 1000 entities are loaded on the graph, you cannot load related entities and observables by selecting Load entities, Load observables, or Load entities by observable from the context menu.
When creating groups in the graph, it is not possible to merge multiple groups into one.
If an ingestion process crashes while ingestion is still ongoing, data may not always sync to Elasticsearch.
Users can leverage rules to access groups that act as data sources, even if those users are not members of the groups they access through rules.
Running multiple outgoing feed tasks may cause the Intelligence Center to consume a large amount of memory over time, because certain outgoing feeds such as HTTP download must load the data into memory in order to make it available to feed consumers.

Public API compatibility #

From EclecticIQ Intelligence Center 2.12.0 onward, the public API is packaged together with EclecticIQ Intelligence Center.

The following reference table lists the versions of the public API package and EclecticIQ Intelligence Center versions they are compatible with:

Intelligence Center version(s)	Public API package version(s)	Public API version
2.11 - 2.12	`eclecticiq-extension-api==1.0.*`	v1
2.13.0	`eclecticiq-extension-api==1.*`	v1
2.14.0 and newer	Now follows EclecticIQ Intelligence Center versioning scheme. E.g., EclecticIQ Intelligence Center 2.14 is now compatible with `eclecticiq-extension-api==2.14.*`	v1
3.0.x	`eclecticiq-extension-api==3.0.*`	v2

Download #

For more information about setting up repositories, refer to the installation documentation for your target operating system.

EclecticIQ Intelligence Center and dependencies for Rocky Linux and RHEL	Platform packages: https://downloads.eclecticiq.com/platform-packages-centos/ Platform dependencies: https://downloads.eclecticiq.com/platform-dependencies-centos-2.9/ Note The Intelligence Center dependencies URL for versions 2.9 and later is `https://downloads.eclecticiq.com/platform-dependencies-centos-2.9/`. It contains packages that are incompatible with versions 2.8 and earlier.
EclecticIQ Intelligence Center extensions	Platform extensions: https://downloads.eclecticiq.com/Extensions/

Upgrade #

The diagram below describes upgrade paths for EcelcticIQ Intelligence Center. See the following for upgrade instructions:

In order to upgrade to EclecticIQ Intelligence Center 3.0, you must:

Be running one of the supported operating systems.

See Upgrade operating system.
Upgrade from EclecticIQ Intelligence Center 2.14.

If you are running an older version of EclecticIQ Intelligence Center, you must upgrade to 2.14 before attempting to upgrade to EclecticIQ Intelligence Center 3.0.

See Install Configure Upgrade.