Incoming feed - Recorded Future URL Feed

This article describes how to configure incoming feeds for a particular feed source. To see how to configure incoming feeds in general, see Configure incoming feeds general options.

Specifications

Transport types

Recorded Future URL Feed

Content type

Recorded Future CSV

Ingested data

Ingests URL risk lists from Recorded Future. URL found in these lists have matched one or more risk rules.

Endpoint(s)

https://api.recordedfuture.com/v2/fusion/files/?path=/public/risklists/default_url_risklist.csv

Processed data

See Data mapping.

Requirements

  • Email address registered with Recorded Future.

  • Recorded Future API key.

Configure the incoming feed

  1. Create or edit an incoming feed.

  2. Under Transport and content, fill out these fields:

    Required fields are marked with an asterisk (*).

    Field

    Description

    Transport type*

    Select Recorded Future URL Feed from the drop-down menu.

    Content type*

    Select Recorded Future CSV from the drop-down menu.

    API URL*

    By default, this is set to https://api.recordedfuture.com/v2/fusion/files/?path=/public/risklists/default_url_risklist.csv.

    API key*

    Set this to your Recorded Future API key.

    SSL verification

    Selected by default. Select this option to enable SSL for this feed.

    Path to SSL certificate file.

    Used when connecting to a feed source that uses a custom CA. Set this as the path to the SSL certificate to use when authenticating the feed source.

    For more information, see SSL certificates.

    Stop feed after x minutes*

    Set to 450 minutes by default.

    Once the feed reaches this time limit, no more packages are requested from the feed source. In-progress downloads and ingestion will continue.

  3. Store your changes by selecting Save.

SSL certificates

To use an SSL certificate with the platform, it must be:

  • Accessible on the EclecticIQ Platform host.

  • Placed in a location that can be accessed by the eclecticiq user.

  • Owned by eclecticiq:eclecticiq.

To make sure that the platform can access the SSL certificate:

  1. Upload the SSL certificate to a location on the platform host.

  2. On the platform host, open the terminal.

  3. Change ownership of the SSL certificate by running as root in the terminal:

    chown eclecticiq:eclecticiq /path/to/cert.pem

    Where /path/to/cert.pem is the location of the SSL certificate the platform needs to access.

Data mapping

The sections below contain non-exhaustive information about how Recorded Future URL Feed data is mapped to EclecticIQ Platform entities and observables.

The Recorded Future URL Feed downloads risk lists from Recorded Future in csv format. Each record in these lists have matched one or more risk rules on the Recorded Future platform.

Records are ingested by EclecticIQ Platform as URL Indicators.

Map risk list to Indicator

This table shows how each record from a Recorded Future URL Feed feed is mapped to Indicators on the platform:

Indicator field name

Mapped from feed source

Example value

Description

Title

Name

https://example.com/wp-content/sites/eb245443b7

URL that matches one or more risk rules.

Confidence

Risk

Medium

See Confidence mapping.

Analysis

EvidenceDetails

Risk Rule: Phishing

X sightings on Y source: Bad Actor Name. More information on sightings here.

Criticality: 3

Risk rules that match this URL are listed in this field.

Risk Rule: <Rule name>
<Evidence string>
Criticality: <Criticality value>

Timestamps for matched risk rules are set as the Indicator’s Estimated time -> Observed field.

Tags

Name and CriticalityLabel

Positive Malware Verdict - Malicious

Risk rules that match the URL are also added as tags to the Indicator, allowing you to search and filter Indicators using a combination of <Rule name> - <CriticalityLabel>.

Confidence mapping

This table shows how Recorded Future risk scores are mapped to Confidence values in ingested Indicators.

Recorded Future risk score

Confidence value

1 – 33

Low

34 – 66

Medium

67 – 100

High