Release notes 2.11.0

Product

EclecticIQ Intelligence Center

Release version

2.11.0

Release date

7 December 2021

Summary

Minor release

Upgrade impact

Medium

Time to upgrade

~18 minutes to upgrade an instance with 4 million entities.

  • From the previous release

  • Using the installation script

  • For an instance running on one machine

Additional ~6 minutes to run pre-upgrade scripts for upgrading from 2.8.x and earlier.

Time to migrate

  • PostgreSQL database: ~6 minutes per 4 million entities

  • Elasticsearch database: ~1 minute per 4 million entities

  • Neo4j database: ~1 minute per 4 million entities.

Highlights

EclecticIQ Intelligence Center 2.11.0 is a minor release. It contains new features, improvements to existing functionality, as well as bug fixes.

With this release we pick up right where we left off by adding new features and improvements for initiatives that span multiple releases:

  1. MITRE ATT&CK support: Threat data from premier incoming feeds that has been mapped to enterprise ATT&CK techniques can now be easily searched for or filtered on by ATT&CK ID. All of these mappings can also be analyzed visually by adding them as a layer on top of the graph. This means that you can move up the Pyramid of Pain and investigate threat actors by their behavior instead of by indicators, resulting in a much better understanding of cyber threats and the actions they require.

  2. Expanded STIX 2.1 interoperability: Installation of the upcoming 2.11 extension release will allow you to ingest and share an important subset of objects in STIX 2.1 format over TAXII 2.1, including Campaigns, and Vulnerabilities and Relationship Objects. This means that Intelligence Center 2.11 will offer out-of-the-box interoperability with STIX 2.1 Preferred certified feed vendors and security controls. This marks an important milestone in our journey towards adopting the STIX & TAXI 2.1 standards for intelligence exchange.

  3. Knowledge packs: You can now create your own custom Knowledge Packs and make them available for easy installation on other Intelligence Center instances. As a result, all analysts across your Intelligence Center community or customer base, can have access to identical workspaces to collectively start tracking threats, without spending any time on configuration. Available as a Beta feature.

  4. Report editor: We are implementing a brand-new rich-text editor under the hood that will allow you to further customize new or existing report entities. As a first step with this new text editor, you can now insert tables into reports to help your readers look up specific information. Like images they are a great way to capture your reader’s attention and help tell the story. Available as a Beta feature.

We hope you enjoy reading these release notes — once again accompanied by short feature videos for your convenience — and watching the quick tour video from the team.

View a quick tour images/download/attachments/86441645/release-2-11-quick-tour.jpg

Upcoming

  • 2.12 uses Python 3.8

    Python 3.6 is scheduled for End-of-Life on 23rd December 2021. To address this, release 2.12 onwards will use Python 3.8.

  • Rebranding to EclecticIQ Intelligence Center in documentation

    Rebranding from EclecticIQ Platform to EclecticIQ Intelligence Center in documentation is in progress. You may still see instances of “EclecticIQ Platform” remaining — please bear with us while we update the documentation.

What’s new

  • MITRE ATT&CK in graphs

    You can now display MITRE ATT&CK classifications for entities that you’re working with in your graphs. In a graph:

    • Select ATT&CK > Show as Objects to display MITRE ATT&CK classifications as nodes on the graph.

    • Select ATT&CK > Show Name to display the full name of the MITRE ATT&CK classification on the graph.

    For more information, see the documentation.

    Enhanced MITRE ATT&CK support images/download/attachments/86441645/release-2-11-enhanced-mitre-attack-support.jpg

  • MITRE ATT&CK support in integrations

    A select number of incoming feed transport types now have MITRE ATT&CK support. This means that when vendors include MITRE ATT&CK classifications for the data ingested through these transports, these ATT&CK classifications are processed and set for their resulting entities.

    The following transports have MITRE ATT&CK support as of release 2.11:

    • AlienVault OTX Pulses Feed

    • Group-IB Attack Phishing Kit

    • Group-IB Attacks Phishing

    • Group-IB Human Intelligence Threat

    • Recorded Future Analyst Note Feed

    • Flashpoint Indicator Intelligence feed

    • Flashpoint Intelligence Reports Provider

  • STIX 2.1: Campaign and Vulnerability

    This release moves the Intelligence Center closer to STIX 2 Preferred status by adding support for the following STIX 2.1 objects:

    • Campaign SDO (Ingestion and export)

    • Vulnerability SDO (Ingestion and export)

  • Updated knowledge packs interface

    The user interface for knowledge packs has been revamped for a better user experience. In this release:

    • Knowledge packs are displayed as a list

    • Enable a knowledge pack to add its contents to your Intelligence Center instance.

    • Disable a knowledge pack to remove its contents from your instance.

    For more information, see the documentation.

Knowledge pack-related permissions have changed.

For more information, see New knowledge pack permissions.

  • (Beta) Create, publish, and remove knowledge packs

    You can now create and publish knowledge packs, allowing you to share incoming feed configurations, datasets, and workspaces with other Intelligence Center instances.

    To set up your Intelligence Center instance as a knowledge pack producer, enable (Beta) Knowledge packs creation in System settings.

    You can also remove knowledge packs by disabling them.

    For more information, see the documentation.

    Custom knowledge pack sharing (beta) images/download/attachments/86441645/release-2-11-custom-knowledge-pack-sharing.jpg

  • (Beta) Create tables in new text editor for Reports

    This release includes an improved text editor for the entity builder. The new editor provides an improved user experience, and allows you to add simple tables to report entities.

    To enable the new text editor, enable (Beta) Enable Modern text editor in System settings.

    For more information, see Beta features.

    Known issue

    Linking to entities and observables using the improved text editor can crash the UI if the title of the linked object contains a non-ASCII character.

    Insert tables into reports (beta) images/download/attachments/86441645/release-2-11-insert-tables-into-reports.jpg

Important bug fixes

  • Incorrect timestamps in UI

    UI would display and use UTC timestamps instead of timestamps using the configured timezone in System settings. All timestamps in the UI now show timestamps using the configured timezone offset.

  • Viewing list of incoming feeds would cause slow performance

    Viewing the list of incoming feeds in Data configuration would lead to slow performance. This is because an upgraded version of Marshmallow would attempt to validate URLs by making DNS queries, which then causes inadvertent attempts to validate the URLs for all displayed feeds in the list.

    This release changes the way feeds are displayed in Data configuration to avoid this issue.

  • TAXII inbox incoming feeds had ‘Reingest failed packages’ incorrectly disabled

    TAXII inbox incoming feeds may have the Reingest failed packages button incorrectly disabled. This has been fixed.

  • Sending test emails would fail for ‘Send email’ outgoing feeds

    Send email outgoing feeds would fail to send test emails with the Send test email… button. This has been fixed.

Security fixes

For a summary of recent security issues, see Security issues and mitigations.

  • Users with only modify workspaces permissions can add or remove collaborators on a workspace they have access to

    This is addressed in EIQ-2021-0014.

  • Users with only ‘modify entities’ and ‘read files’ permissions can access and export attachments from report entities they do not have access to.

    This is addressed in EIQ-2021-0013.

  • Users with only ‘modify tickets’ and ‘read ticket-comments’ permissions can modify properties of a task object they can access to move and see task comments from tasks they should not have access to.

    This is addressed in EIQ-2021-0012.

  • Users without direct assignment to a listed workspace can view details they should not see.

    This is addressed in EIQ-2021-0011.

  • Users with only ‘modify files’ permissions can move files from their workspace to other workspaces they don’t have access to.

    This is addressed in EIQ-2021-0010.

  • Users with only ‘modify ticket-comments’ and ‘read tickets’ permissions can edit and delete comments on a Task they are at least a stakeholder on.

    This is addressed in EIQ-2021-0009.

Known issues

  • (Beta) Modern editor can cause UI crash when linking to objects with non-ASCII characters in its title

    You can link to entities or observables in your Reports using both the (Beta) Modern editor and the older editor. However, if the title of that entity or observable contains non-ASCII characters, the UI crashes.

  • Elasticsearch 7 encounters “Data too large” errors: See Known issue with Elasticsearch 7: “Data too large”.

  • Entity incorrectly warns it is outdated: When viewing an entity, the entity may warn that it is not the latest version when it actually is. This is related to an issue where with attachments that have been depulicated multiple times, causing issues in the final state of the entity.

  • When you configure the Intelligence Center databases during a Intelligence Center installation or upgrade, you must specify passwords for the databases.

  • Systemd splits log lines exceeding 2048 characters into 2 or more lines.

    As a result, log lines exceeding 2048 characters become invalid JSON, causing Logstash to be unable to parse them correctly.

  • When more than 1000 entities are loaded on the graph, you cannot load related entities and observables by selecting Load entities, Load observables, or Load entities by observable from the context menu.

  • When creating groups in the graph, it is not possible to merge multiple groups into one.

  • If an ingestion process crashes while ingestion is still ongoing, data may not always sync to Elasticsearch.

  • Users can leverage rules to access groups that act as data sources, even if those users are not members of the groups they access through rules.

  • Running multiple outgoing feed tasks may cause the Intelligence Center to consume a large amount of memory over time, because certain outgoing feeds such as HTTP download must load the data into memory in order to make it available to feed consumers.

New knowledge pack permissions

In 2.11, permissions to install and access knowledge packs have changed from ... configuration-bundles to ... knowledge-packs.

This means that if you’ve previously assigned ... configuration-bundles permissions to users or roles, you have to re-assign the corresponding ... knowledge-packs permissions to allow those users or roles to retain access to knowledge pack features.

Users assigned permissions through the System Admin role are not affected.

The table below lists the changed permissions:

Before

2.11.0 and after

install configuration-bundles

install knowledge-packs

modify configuration-bundles

modify knowledge-packs

read configuration-bundles

read knowledge-packs

For more specific information on knowledge packs, see the documentation.

Known issue with Elasticsearch 7: “Data too large”

Since release 2.9.0, the Intelligence Center comes bundled with Elasticsearch (ES) 7.9.1. ES 7 adds a new real memory circuit breaker that causes ES nodes to respond with a circuit_breaking_exception error when it detects that memory use has reached 95% of the totally available JVM heap.

Because of this change, you may encounter issues related to available memory where previously at the same workloads, ES would appear to run smoothly.

If your plaform is encountering issues related to Elasticsearch responding with a circuit_breaking_exception error, you can do the following to mitigate:

Increase available memory for ES

The circuit_breaking_exception error occurs only when ES detects that you are about to go over a memory use threshold that would cause it to fail.

Increase the amount of memory available to ES, or move it to its own host where it does not compete with the Intelligence Center for resources to keep your ES nodes running.

(Not recommended) Disable the “real memory circuit breaker”

This may allow ES to reach an out of memory state and fail.

(Not recommended) To disable the “real memory circuit breaker”, set the indices.breaker.total.use_real_memory parameter in your ES configuration to false.

This allows ES to use the ES 6 parent circuit breaker instead, but disables the safety guarantees that the real memory circuit breaker provides.

Security issues and mitigations

To see a detailed list of security issues and their mitigations, go to All security issues and mitigations.

ID

CVE

Description

Severity

Status

Affected versions

EIQ-2021-0015

-

Users with only modify workspace-comments and read workspace permissions can edit and delete comments in workspaces where they are set as a collaborator.

2 - MEDIUM

images/download/attachments/86441645/clock.svg-x24.png Planned

2.10.x and earlier.

EIQ-2021-0014

-

Users with only modify workspaces permissions can add or remove collaborators on a workspace they have access to

1 - LOW

images/download/attachments/86441645/check.svg 2.11.0

2.10.x and earlier.

EIQ-2021-0013

-

Users with only modify entities and read files permissions can access and export attachments from report entities they do not have access to.

2 - MEDIUM

images/download/attachments/86441645/check.svg 2.11.0

2.10.x and earlier.

EIQ-2021-0012

-

Users with only modify tickets and read ticket-comments permissions can modify properties of a task object they can access to move and see task comments from tasks they should not have access to.

2 - MEDIUM

images/download/attachments/86441645/check.svg 2.11.0

2.10.x and earlier.

EIQ-2021-0011

-

Users without direct assignment to a listed workspace can view details they should not see.

1 - LOW

images/download/attachments/86441645/check.svg 2.11.0

2.10.x and earlier.

EIQ-2021-0010

-

Users with only modify files permissions can move files from their workspace to other workspaces they don’t have access to.

2 - MEDIUM

images/download/attachments/86441645/check.svg 2.11.0

2.10.x and earlier.

EIQ-2021-0009

-

Users with only modify ticket-comments and read tickets permissions can edit and delete comments on a Task they are at least a stakeholder on.

2 - MEDIUM

images/download/attachments/86441645/check.svg 2.11.0

2.10.x and earlier.

EIQ-2021-0007

-

Users could create entities in Source Groups indirectly assigned through Groups, instead of only being able to create entities in Groups they are directly assigned to.

1 - LOW

images/download/attachments/86441645/check.svg 2.10.1

2.10.0 and earlier.

EIQ-2021-0007

-

Users could create entities in Source Groups indirectly assigned through Groups, instead of only being able to create entities in Groups they are directly assigned to.

1 - LOW

images/download/attachments/86441645/check.svg 2.9.2

2.9.1 and earlier.

EIQ-2021-0006

-

SVG file upload could allow cross-site scripting (XSS)

2 - MEDIUM

images/download/attachments/86441645/check.svg 2.9.2

2.9.1 and earlier.

EIQ-2021-0005

-

HTML injection through the GUI

2 - MEDIUM

images/download/attachments/86441645/check.svg 2.9.2

2.9.1 and earlier.

EIQ-2021-0004

CairoSVG is vulnerable to regular expression denial of service

2 - MEDIUM

images/download/attachments/86441645/check.svg 2.10.0

2.9.1 and earlier.

EIQ-2021-0003

PySAML2 improper verification of cryptographic signature

2 - MEDIUM

images/download/attachments/86441645/check.svg 2.10.0

2.9.1 and earlier.

EIQ-2021-0002

Pillow is vulnerable to buffer overflows

2 - MEDIUM

images/download/attachments/86441645/check.svg 2.10.0

2.9.1 and earlier.

Download

For more information about setting up repositories, refer to the installation documentation for your target operating system.

EclecticIQ Intelligence Center and dependencies for CentOS and RHEL

The Intelligence Center dependencies URL for versions 2.9 and later is https://downloads.eclecticiq.com/platform-dependencies-centos-2.9/ . It contains packages that are incompatible with versions 2.8 and earlier.

EclecticIQ Intelligence Center extensions

Upgrade

The following diagram describes the upgrade path you should take depending on the Intelligence Center version you are upgrading from.

For example:

  • You can upgrade from version 2.9.1 of the Intelligence Center to 2.10.0 directly,

  • To upgrade from 2.4.0 to 2.10.0, you must first upgrade to 2.5.0, then upgrade from 2.5.0 to 2.10.0.

When upgrading from 2.8.x and earlier to 2.9.x and later:

  • You must run the pre-upgrade script to allow it to work with Elasticsearch 7.9.1.

  • You must run the pre-upgrade script on the Intelligence Center version you are upgrading from.

    For example, when upgrading from 2.8.0 to 2.10.1, you must run the pre-upgrade script on the Intelligence Center while it is running version 2.8.0.

images/download/attachments/86441645/graphviz-05ba5490d30e39f8ce5cbe6168460d3f01da3e25.svg

Upgrade diagram

From 2.5.0, the upgrades paths have been tested using the EclecticIQ Intelligence Center install script compiled by Rundoc.

The script only supports:

  • Single machine installs.

  • Instances installed using the Intelligence Center install script.

and does not support Intelligence Center instances installed in distributed environments.