Incoming feed - MISP#

Note

This article describes how to configure incoming feeds for a particular feed source. To see how to configure incoming feeds in general, see Create and configure incoming feeds.

Specifications

Transport type

MISP API

Content type

MISP JSON

Ingested data

Ingest MISP events with attributes and MISP objects.

Processed data

Creates entities and related observables from ingested data.

Requirements#

  • URL used to access your MISP instance.

  • MISP Authkey.

To get a MISP Authkey:

  1. In your MISP instance, click My profile.

  2. Copy the Authkey, and store it for later.

Configure the incoming feed#

  1. Create or edit an incoming feed.

  2. Under Transport and content, fill out these fields:

    Note

    Required fields are marked with an asterisk (*).

    Field

    Description

    Transport type*

    Select MISP API from the drop-down menu.

    Content type*

    Select MISP JSON from the drop-down menu.

    MISP URL*

    Set this to the URL you access your MISP instance at.

    MISP key*

    Set this to your MISP Authkey.

    Include tags

    Select to include tags when ingesting data from the target MISP instance.

    Prioritize TLP tag

    Select to have ingested entities inherit TLP (Traffic Light Protocol) values from TLP tags set by the feed source instead of inheriting Distribution setting.

    Setting a value for the Override TLP field sets all ingested entities to that TLP value instead of inheriting the TLP value from the feed source.

    Include IDS flag as tag

    Selected by default. When selected, the platform checks if the ‘to_ids’ flag is set to ‘true’ for for incoming MISP attributes, and adds a tag named ‘IDS’ to the resulting entities.

    Reduce lock contention

    Select Reduce lock contention to speed up ingestion.

    Caution

    Entities will update at random. For more information, see Reduce lock contention below.

    SSL verification

    Selected by default. Select this option to enable SSL for this feed.

    Start ingesting from*

    Ingest data from the feed source starting from this date and time. Use the drop-down calendar to select the date and time you want to start ingesting feed data from.

    End ingestion

    By default, this is set to the date and time the incoming feed is created.

    Set this to the latest date and time for the latest incident the platform should ingest from the feed source.

    SSL cert location

    Used when connecting to a feed source that uses a custom CA. Set this as the path to the SSL certificate to use when authenticating the feed source.

    For more information, see SSL certificates.

    Client cert location

    Used when connecting to a MISP instance that requires a client certificate to authenticate.

    Set the absolute path to the certificate file here. For example:

    /path/to/cert.crt

    For more information, see SSL certificates.

    Client cert location

    Used when connecting to a MISP instance that requires a client certificate to authenticate. The certificate used to authenticate may require a certificate key file.

    Set the absolute path to the key file here. For example:

    /path/to/cert.key

    For more information, see SSL certificates.

    Default request interval

    By default, this is set to 240 hours.

    The Default request interval allows you to control the amount of data each request made to the MISP instance retrieves by limiting the events retrieved to the specified period.

    For example, setting this to 240 hours allows each request to retrieve a maximum of 10 days worth of events from the MISP instance.

    Reducing this value may help with performance issues related to receiving too many feed packages from this feed.

    See Tuning request intervals.

    Filter by MISP event info

    Enter a regular expression to exclude MISP events from ingestion if the expression matches the content of their “Event Info” field (info).

    For example, enter:

    ^(COVID19\-)

    to exclude all events with “Event Info” fields starting with “COVID19-“.

    Filter by the creating organization’s name

    Enter a regular expression to exclude MISP events that have a creating organization (orgc) that matches this expression.

    For example, enter:

    Iglocska

    to exclude all events that have creating organizations with names that contain the word “Iglocska”.

  3. Store your changes by selecting Save.

Distribution settings#

MISP labels events and attributes using distribution settings, which tells us who these events and attributes can be shared with.

The platform maps these distribution settings to TLP values:

TLP value

MISP distribution setting

Not Set

No distribution setting

WHITE

All communities

GREEN

Connected communities

AMBER

This community only

RED

Your organization only

Reduce lock contention#

Enable the Reduce lock contention option to speed up ingestion for the MISP feed. This is done by splitting incoming data into smaller packages that are then randomly redistributed among ingestion workers, reducing the likelihood that ingestion is stalled by having several workers attempting to update the same record.

This means that if you start working with your data before ingestion completes, you may be working with incomplete entities. If ingestion fails, run the incoming feed again and let it finish to make sure that you have a complete set of data from the source MISP feed.

SSL certificates#

To use an SSL certificate, it must be:

  • Accessible on the EclecticIQ Intelligence Center host.

  • Placed in a location that can be accessed by the eclecticiq user.

  • Owned by eclecticiq:eclecticiq.

To make sure that EclecticIQ Intelligence Center can access the SSL certificate:

  1. Upload the SSL certificate to a location on the EclecticIQ Intelligence Center host.

  2. On the EclecticIQ Intelligence Center host, open the terminal.

  3. Change ownership of the SSL certificate by running as root in the terminal:

    chown eclecticiq:eclecticiq /path/to/cert.pem

    Where /path/to/cert.pem is the location of the SSL certificate EclecticIQ Intelligence Center needs to access.

Supported attributes and mapping#

The following table displays all attributes that are supported by this integration and also describes how attributes are mapped in the EclecticIQ Platform during an incoming or outgoing feed.

MISP

EIQ Observable

EIQ Entity

ip-src

  • ipv4 / ipv6

  • source

Multiple

ip-dst

  • ipv4 / ipv6

  • destination

Multiple

domain

domain

Multiple

hostname

host

Multiple

url / uri

uri

Multiple

md5 / sha1 / sha256 / sha512

hash-md5 / hash-sha1 / hash-sha256 / hash-sha512

Multiple

filename

file

Multiple

threat-actor

(multiple within ThreatActor entity)

ThreatActor

campaign-name

(name within Campaign entity)

Campaign

link

uri

Multiple

email-src

email

Multiple

email-dst / target-email

email

Multiple

email-subject

Indicator title

Indicator title

email-attachment

file

Multiple

attachment

file

Multiple

mutex

mutex

Multiple

vulnerability

cve

ExploitTarget

snort

snort

Test Mechanism on Indicator

yara

yara

Test Mechanism on Indicator

Tuning request intervals#

Adjusting the request interval determines the batch size of data that is fetched from MISP.

A lower interval value will result in less data being pulled in one API query. This is useful if the target MISP system has large amounts of data. Note that data is pulled by the date or time that it was published on MISP.

For example, if you set the interval to 24, each API call issued to MISP will be filtered with a time span of 24 hours. A lower interval will result in more calls being made to the MISP API, the MISP responses will contain less data, and each individual call will place less stress on the MISP API.