Outgoing EclecticIQ Platform MISP feed

By configuring an outgoing feed you can push data from EclecticIQ Platform to the MISP Platform.

There are some differences between the EclecticIQ and the MISP data models. Therefore, certain constraints are applied to the outgoing feed.

The MISP model is treelike in its core and only supports MISP attributes that are directly linked to, or are part of, an entity.
This means that the outgoing data feed, that is being pushed to the MISP Platform, can only apply a principle to an entity that is linked to an indicator through relationships that can be mapped to a MISP structure.

In the EclecticIQ Platform, entities can be connected in ways that are not supported by the MISP Platform. To provide appropriate mappings, the outgoing feed works by fetching all incidents from the dataset, then fetching all other entities that are part of the dataset, that are linked to the specific incident, and supported by MISP to EclecticIQ Platform mappings.

The outgoing feed will build a MISP-like data model and push that data to the MISP Platform. It will not include any entities that do not meet above conditions.
For example, an indicator that is not linked to any incident but is a part of a MISP outgoing feed dataset will not be pushed. Similarly, an indicator that is a part of an MISP outgoing feed dataset and is linked to an incident that is not a part of a MISP dataset, will not be pushed to MISP.

Prerequisites

  • A configured MISP Platform instance

  • A configured EclecticIQ Platform instance

Limitations

  • In this integration, outgoing feeds can generate and update MISP events with simple attributes only.
    It is not possible to generate or update MISP events that contain objects, galaxies, or composite attributes.

  • Threat level metadata from a MISP event is not preserved in the EclecticIQ Platform during a in MISP to EclecticIQ Platform communication.

  • Attribute update or deletion is not supported in a EclecticIQ Platform to MISP communication.
    If an attribute is updated in the EclecticIQ Platform, a new attribute will be created with the updated value but the old value will not be updated or deleted on the MISP Platform.

Configuration

Get your MISP key

  1. In the MISP Platform, click My profile.

  2. Copy the Authkey , and store it for later.

Configure and create a dataset

Dataset guidelines

When configuring a dataset, to allow granular control over the data that is synced to the MISP Platform, make sure of the following:

  • All incidents that represent events that should be pushed to MISP must be made part of the MISP outgoing feed dataset.

  • All related entities that represent data that will be pushed to MISP, in form of attributes and that are connected to appropriate incident, must be made part of MISP outgoing feed dataset.

Create a dataset

For more information on how to create a dataset, and how to create a dynamic dataset, see Datasets.

  1. In EclecticIQ Platform, create a dataset.

  2. In the Dataset name field, enter a name for the dataset.

  3. In the Workspaces field, click to add applicable workspaces.

  4. Click Save.

  5. Open your newly created dataset.

  6. To add entities, click .

  7. To select entities, click the checkbox on the left of the entities that you would like to add. Ensure that you select all incidents and indicators that must be pushed to the MISP Platform.

  8. Click Select.

Configure an outgoing feed in EclecticIQ Platform

  1. In EclecticIQ Platform, create an outgoing feed.

  2. In the Feed name field, enter a name for the feed.

  3. In the Transport type field, select MISP upload.

  4. In the Content type field, select MISP JSON model.

  5. In the Datasets field, select the dataset that you created earlier.

  6. In the Update strategy field, select Replace.

  7. In the MISP URL field, enter the URL or your MISP instance.

  8. In the MISP Key field, enter your MISP key.

  9. Select the Include tags, SSL validation, and Use cert keys check boxes.

    We recommend that the execution schedule is set to the length of time taken to update and ingest the feed plus 2 hours.

    To check how long a feed takes to update and process: open EclecticIQ Platform UI, click on the feed, click Download now, and monitor the Last Download Status feed until a flag appears under the Ingested entities column.

  10. In the Execution schedule field, select a schedule that provides the feed with enough time to update and be processed.

  11. In the Processing section, select values based on your preferences. These values will not be linked to MISP.

  12. On the right of the Save button, click > Save and run.

Check that the feed is working

  1. In the MISP Platform, refresh the view. The new or updated event will be listed in the event list.

  2. Click View event.

Test the integration

This procedure is relevant only in testing and debugging scenarios.

To test the MISP connection and to test pushing an event programmatically:

  1. Create a python script:

    import pymisp
     
    # Replace '${misp_instance_url}' with the URL of the MISP instance connected to the platform.
    # Replace '${misp_instance_authkey}' with the Authkey of the MISP instance connected to the platform.
    MISP_URL = 'https://${misp_instance_url}'
    MISP_KEY = '${misp_instance_authkey}'
     
    # If necessary, you can pass here also server and client certificates
    misp = pymisp.PyMISP(MISP_URL, MISP_KEY)
     
    misp.test_connection()
    misp.new_event(info='Test event', analysis=0, distribution=0, threat_level_id=3)

  2. Assign it a name – for example, misp_integration_test – and then save it.

  3. Switch to the eclecticiq user

    sudo -u eclecticiq

  4. Activate a Python virtual environment for the platform:

    source /opt/eclecticiq-platform-backend/bin/activate

  5. Export the platform settings configuration to create the necessary environment variables

    export EIQ_PLATFORM_SETTINGS="/etc/eclecticiq/platform_settings.py"

  6. In the active Python virtual environment, run the script:

    python misp_integration_test