Exchanging data between platforms

About data exchange between platforms

You can exchange data between two platform instances using TAXII feeds.
This is what you need to implement this platform-to-platform integration:

• A group and a user with access rights to both platform instances.

• Basic authentication (optional)

• A TAXII inbox or a TAXII poll outgoing feed

• A TAXII inbox or a TAXII poll incoming feed

• An EclecticIQ JSON content type for both outgoing and incoming feeds

• Two platform instances acting as data source/publisher and data destination/recipient, respectively.

For the sake of keeping things explicit and unambiguous, let's give names to our communicating platform instances:

• Let’s call the publishing platform instance Alice.
  This is the data source of the feed content.

• And let’s call the recipient platform instance Barbara.
  This is the data destination and the consumer of the feed content.

Check package size limits

The /etc/eclecticiq/platform_settings.py configuration file includes settings that control and limit package file size for ingestion – incoming feeds and manual file upload – and for dissemination – outgoing feeds.

settings.py (sourced from EIQ platform-backend)

• MAX_BLOB_SIZE defines the max allowed file size for ingested packages through incoming feeds and for published packages through outgoing feeds.
  The size is expressed in bytes.
  Default value: 20 * 1024 * 1024 (20 MB)

• MAX_UPLOADED_BLOB_SIZE defines the max allowed file size for ingested packages through manual file upload.
  The size is expressed in bytes.
  Default value: MAX_BLOB_SIZE / 2 (10 MB)

You can set custom values for MAX_BLOB_SIZE and MAX_UPLOADED_BLOB_SIZE.
For example, you can set MAX_UPLOADED_BLOB_SIZE = MAX_BLOB_SIZE, or MAX_UPLOADED_BLOB_SIZE = 30 * 1024 * 1024 to enable manually uploading large PDF files to the platform.

The maximum limit you can set for file uploads is 100 * 1024 * 1024 (100 MB).

Whitelist mount points

Explicitly whitelist mount point paths to make them accessible to incoming and to outgoing feeds.
If you do not whitelist the mount point path an incoming or an outgoing feed should access to retrieve data for ingestion or for publication, the feed will not be able to fetch or to publish any content.

The /etc/eclecticiq/platform_settings.py configuration file includes dedicated mount point whitelists for ingestion – incoming feeds – and for dissemination – outgoing feeds.

settings.py (sourced from EIQ platform-backend)

• MOUNT_POINT_POLL_ALLOWED_DIRECTORIES is a list of allowed mount point paths that incoming feeds can access to fetch data from.

• MOUNT_POINT_PUSH_ALLOWED_DIRECTORIES is a list of allowed mount point paths that outgoing feeds can access to publish data to.

To whitelist a mount point path:

1. Open the configuration file:

   sudo vi /etc/eclecticiq/platform_settings.py

2. Look for MOUNT_POINT_POLL_ALLOWED_DIRECTORIES to make network locations accessible to incoming feeds, or for for MOUNT_POINT_PUSH_ALLOWED_DIRECTORIES to make network locations accessible to outgoing feeds.
   Both parameters are lists that take valid directory paths as list elements.
   Each path in the list points to a location that incoming feeds can access to fetch the data to be ingested, or that outgoing feeds can access to publish the content of a feed run.
   Incoming and outgoing feeds can access files and directories inside the specified locations, based on the configured access rights of the available assets and resources.

3. Add as many paths to each list as necessary, then save the file and exit.
   Example:

   # Whitelist specific dirs; specific file types; everything inside subdirs of a dir

   MOUNT_POINT_PUSH_ALLOWED_DIRECTORIES = [ "/mnt/", "/media/", "/media/data/" ]

About TAXII services

After configuring a TAXII server, you can set up TAXII services.
A TAXII service is a specialized data handler that implements a specific TAXII capability.

The platform supports the following TAXII services:

View TAXII services

To access an overview of the existing and configured TAXII services available in the platform :

1. In the side navigation bar click > STIX and TAXII > TAXII.
   The TAXII view displays the currently configured TAXII services for the platform.
   
   To sort items by column header:

   1. Click the header of the column whose content you want to sort.

   2. Click or to sort the content in either ascending or descending order, respectively.

2. Click the menu icon in the row corresponding to the TAXII service you want to inspect.

3. From the drop-down menu select View.
   
   Alternatively:
   
   In the TAXII view click anywhere in the row corresponding to the TAXII service whose configuration you want to inspect.
   The TAXII settings view displays the current configuration of the specified service.