Exchanging data between EclecticIQ Intelligence Center instances#
About data exchange between EclecticIQ Intelligence Center instances#
You can exchange data between two platform instances using TAXII feeds. This is what you need to implement this platform-to-platform integration:
A group and a user with access rights to both platform instances.
Basic authentication (optional)
A TAXII inbox or a TAXII poll outgoing feed
A TAXII inbox or a TAXII poll incoming feed
An EclecticIQ JSON content type for both outgoing and incoming feeds
Two platform instances acting as data source/publisher and data destination/recipient, respectively.
For the sake of keeping things explicit and unambiguous, let’s give names to our communicating platform instances:
Let’s call the publishing platform instance Alice. This is the data source of the feed content.
And let’s call the recipient platform instance Barbara. This is the data destination and the consumer of the feed content.
Assign unique names to TAXII feeds: TAXII inbox and TAXII poll feeds in the platform, both incoming and outgoing, must have unique names.
Both platform instances must exchange data through the same TAXII transport protocol service: either TAXII inbox or TAXII poll.
Although TAXII feed configurations make available more than one content type for the incoming and the outgoing feeds, platform to platform data exchange officially supports only EclecticIQ JSON as a valid data exchange content type.
Regardless of Alice’s outgoing and Barbara’s incoming feeds using either the TAXII inbox or the TAXII poll transport types, they both need to publish data to and retrieve data from the same TAXII collection:
Alice (publisher) should publish content through a TAXII outgoing feed and a specified Collection name – for a TAXII poll transport type – or a Destination collection name – for a TAXII inbox transport type.
Barbara (recipient) should ingest content through a TAXII incoming feed, and it should ingest from or poll the same Collection name specified under Collection name or Destination collection name in Alice’s TAXII outgoing feed configuration.
About user access control and permissions
Specify the automation user’s user name and password in the corresponding feed configuration fields.
If you enable basic authentication, make sure the automation role has the required additional permissions.
Check package size limits#
The /etc/eclecticiq/platform_settings.py configuration file includes settings that control and limit package file size for ingestion – incoming feeds and manual file upload – and for dissemination – outgoing feeds.
# Package size limits (in bytes) MAX_BLOB_SIZE = 20 * 1024 * 1024 MAX_UPLOADED_BLOB_SIZE = MAX_BLOB_SIZE / 2
MAX_BLOB_SIZEdefines the max allowed file size for ingested packages through incoming feeds and for published packages through outgoing feeds.
The size is expressed in bytes.
20 * 1024 * 1024(20 MB)
MAX_UPLOADED_BLOB_SIZEdefines the max allowed file size for ingested packages through manual file upload.
The size is expressed in bytes.
MAX_BLOB_SIZE / 2(10 MB)
You can set custom values for
For example, you can set
MAX_UPLOADED_BLOB_SIZE = MAX_BLOB_SIZE, or
MAX_UPLOADED_BLOB_SIZE = 30 * 1024 * 1024 to enable manually uploading
large PDF files to the platform.
The maximum limit you can set for file uploads is
100 * 1024 * 1024
Changes to these fields may affect ingestion, packaging, and dissemination behavior, as well overall platform performance. For example, setting too large a size may result in ingestion bottlenecks or upload timeouts.
Whitelist mount points#
Explicitly whitelist mount point paths to make them accessible to incoming and to outgoing feeds. If you do not whitelist the mount point path an incoming or an outgoing feed should access to retrieve data for ingestion or for publication, the feed will not be able to fetch or to publish any content.
/etc/eclecticiq/platform_settings.py configuration file includes
dedicated mount point whitelists for ingestion – incoming feeds – and
for dissemination – outgoing feeds.
# Directories that can be accessed from mount point feeds. POLL is for incoming # feeds, PUSH is for outgoing feeds. Example: ["/mnt/", "/media/"] MOUNT_POINT_POLL_ALLOWED_DIRECTORIES: Sequence[str] =  MOUNT_POINT_PUSH_ALLOWED_DIRECTORIES: Sequence[str] = 
MOUNT_POINT_POLL_ALLOWED_DIRECTORIESis a list of allowed mount point paths that incoming feeds can access to fetch data from.
MOUNT_POINT_PUSH_ALLOWED_DIRECTORIESis a list of allowed mount point paths that outgoing feeds can access to publish data to.
To whitelist a mount point path:
Open the configuration file:
sudo vi /etc/eclecticiq/platform_settings.py
MOUNT_POINT_POLL_ALLOWED_DIRECTORIESto make network locations accessible to incoming feeds, or for for
MOUNT_POINT_PUSH_ALLOWED_DIRECTORIESto make network locations accessible to outgoing feeds.
Both parameters are lists that take valid directory paths as list elements.
Each path in the list points to a location that incoming feeds can access to fetch the data to be ingested, or that outgoing feeds can access to publish the content of a feed run.
Incoming and outgoing feeds can access files and directories inside the specified locations, based on the configured access rights of the available assets and resources.
Add as many paths to each list as necessary, then save the file and exit.
# Whitelist specific dirs; specific file types; everything inside subdirs of a dir MOUNT_POINT_PUSH_ALLOWED_DIRECTORIES = [ "/mnt/", "/media/", "/media/data/" ]
About TAXII services#
After configuring a TAXII server, you can set up TAXII services. A TAXII service is a specialized data handler that implements a specific TAXII capability.
The platform supports the following TAXII services:
Collection management service
You can use a TAXII collection management service to:
You can poll a TAXII collection to:
TAXII data collections — structured TAXII data feeds, and unstructured TAXII datasets — are examples of TAXII inbox and TAXII poll service content.
You can use a TAXII discovery service to obtain information about the availability and the use of TAXII services such as collection management, inbox, and polling.
The TAXII inbox service enables data producers to initiate push messages to service consumers. This service can be based on a subscription model, or it can be an unsolicited payload a producer pushes to a consumer.
The TAXII poll service enables consumers to request TAXII data collection content from a TAXII producer, usually through TAXII outgoing feeds.
View TAXII services#
To access an overview of the existing and configured TAXII services available in the platform :
To sort items by column header:
From the drop-down menu select View.
In the TAXII view click anywhere in the row corresponding to the TAXII service whose configuration you want to inspect.
The TAXII settings view displays the current configuration of the specified service.