Incoming feed - TAXII poll#

Note

This article describes how to configure incoming feeds for a particular feed source. To see how to configure incoming feeds in general, see Create and configure incoming feeds.

Note

Assign unique names to TAXII feeds: TAXII inbox and TAXII poll feeds in the platform, both incoming and outgoing, must have unique names.

Specifications

Transport type

TAXII poll

Content type

  • Eclectic JSON

  • Email message

  • MISP JSON

  • PDF

  • SpyCloud Breach Data JSON

  • STIX 1.0

  • STIX 1.1

  • STIX 1.1.1

  • STIX 1.2

  • STIX 2.1

  • Text

Ingested data

Structured STIX packages.

Processed data

Structured, STIX-compliant entities.

Description

Retrieve and process information from specific data sources supporting the FEED-TYPE transport type.

Note

Before configuring a TAXII transport type for an incoming or an outgoing feed, make sure that the appropriate TAXII service is correctly configured in the platform system settings.

TAXII inbox and TAXII poll transport types require Cabby.
For more information, see official Cabby documentation, the Cabby public repo on GitHub, and the Cabby download page

Note

Assign unique names to TAXII feeds: TAXII inbox and TAXII poll feeds in the platform, both incoming and outgoing, must have unique names.

Configure the incoming feed#

  1. Create or edit an incoming feed.

  2. From the Transport type drop-down menu, select TAXII poll.

  3. From the Content type drop-down menu, select the appropriate content type for the data you want to ingest through the incoming feed.
    The selected content type for the feed should match the actual format of the source data.
    This can vary, depending on the intelligence sources you retrieve the data from.

  4. Select the Accept password protected archives checkbox to specify a global password to open any archives retrieved through the incoming feed.
    If the archives are password-protected, enter it in the Archive password input field.
    The specified password acts as a master password, and it is used to try to unlock and access any archives retrieved with the feed.

    Supported archive formats:

    • .rar

    • .tar

    • .tar.bz2

    • .tar.gz

    • .tar.z

    • .zip

  5. In the Auto discovery field, enter the URL pointing to a TAXII discovery service.
    Feed consumers can send a request to the discovery service to obtain a list of the available TAXII services they can access and poll for content updates.
    Example: http://hailataxii.com/taxii-discovery-service
    The URL you enter here must match the platform instance base URL plus the TAXII TAXII discovery service URL endpoint configured for the platform.
    Example:

    Platform base URL

    TAXII discovery URL

    Auto Discovery URL

    eclecticiq.platform.org

    /taxii/discovery

    https://eclecticiq.platform.org/taxii/discovery

  6. In the Polling service URL field, enter the URL pointing to a TAXII poll service.
    Feed consumers can send a request to the TAXII poll service to pull data from a configured TAXII data collection, and to obtain information on available and/or updated content.
    Example: http://hailataxii.com/taxii-poll-service

    The URL you enter here must match the platform instance base URL plus the TAXII TAXII poll service URL endpoint configured for the platform.
    Example:

    Platform base URL

    TAXII poll URL

    Polling service URL URL

    eclecticiq.platform.org

    /taxii/poll

    https://eclecticiq.platform.org/taxii/poll

  7. The Collection name field is automatically populated when you select a TAXII collection by clicking in an Auto Discovery input field populated with a valid URL to an existing TAXII discovery service.
    Example: guest.Abuse_ch.

  8. From the TAXII version drop-down menu, select the TAXII version your system and the data source TAXII server support:

    If the data source TAXII server requires passing additional HTTP headers in the request, you can specify them under Extra headers.

  9. Click Add or More to insert new rows or input fields, as necessary, where you can enter additional HTTP header and corresponding value pairs.

    • In the left input field, enter the HTTP header type.
      Example: X-TAXII-Protocol

    • In the right input field, enter the HTTP header value.
      Example: urn:taxii.mitre.org:protocol:https:1.1

  10. To remove an entry from this section, click corresponding to the item(s) you want to remove.

  11. In the Subscription ID field, enter the name, label or ID identifying the subscription session. Usually, the data source TAXII server assigns such an ID, and it returns it in the response to a successful request.
    The subscription ID is used in subsequent requests to poll the service to receive content, and to manage available content through the feed.

  12. Click the Start ingesting from field, use the drop-down menu calendar to select an initial date and, where available, an initial time to fetch content from the intelligence provider/data source starting from a specific date in the past.
    The ingestion date you specify here refers to package timestamps. It does not refer to entity timestamps. Entities in a package can have different, older, timestamps.
    The first time you run the feed, it ingests data starting from the specified date in the past.
    Subsequent runs start incrementally from the time of the previous feed run.
    If you do not specify any start date, the feed defaults to ingesting data from January 1st, 1970.

  13. In the Days per poll field, enter an integer to specify the maximum number of days to poll at a time.
    If you select a start date to poll data from, you can enter an integer to specify the maximum number of days to poll at a time.
    This enables polling in multiple smaller batches, instead of a single batch, starting from the selected initial date.
    Each time the feed runs, it sends multiple poll requests to progressively download in batches all the relevant content from the specified start date until the present moment.

    Note

    This option works only if you select an ingestion start date in Start ingesting from.

  14. Select the Basic authentication checkbox to fill out the required information, if the data source TAXII server requires basic authentication to access the corresponding TAXII services.

  15. In the Username field, enter a valid user name to authenticate and be granted the necessary authorization to access the location of the outgoing feed content.

  16. In the Password field, enter a valid password to authenticate and be granted the necessary authorization to access the location of the outgoing feed content.

  17. In the EclecticIQ authentication URL field, enter the URL pointing to the EclecticIQ Platform instance, including the endpoint that takes the user name and password inputs to send them to the authentication mechanism.
    Example: https://${platform_host_name}/api/auth
    If the TAXII server requires an SSL certificate to authenticate and to authorize access to the corresponding TAXII services, select this checkbox to fill out the required information.

  18. In the SSL certificate field, copy-paste the content of a valid SSL certificate to authenticate.

    • SSL certificate file format:.pem Example:

      -----BEGIN CERTIFICATE REQUEST-----
MIICvDCCAaQCAQAwdzELMAkGA1UEBhMCVVMxDTALBgNVBAgMBFV0YWgxDzANBgNV
BAcMBkxpbmRvbjEWMBQGA1UECgwNRGlnaUNlcnQgSW5jLjERMA8GA1UECwwIRGln
aUNlcnQxHTAbBgNVBAMMFGV4YW1wbGUuZGlnaWNlcnQuY29tMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEA8+To7d+2kPWeBv/orU3LVbJwDrSQbeKamCmo
wp5bqDxIwV20zqRb7APUOKYoVEFFOEQs6T6gImnIolhbiH6m4zgZ/CPvWBOkZc+c
1Po2EmvBz+AD5sBdT5kzGQA6NbWyZGldxRthNLOs1efOhdnWFuhI162qmcflgpiI
WDuwq4C9f+YkeJhNn9dF5+owm8cOQmDrV8NNdiTqin8q3qYAHHJRW28glJUCZkTZ
wIaSR6crBQ8TbYNE0dc+Caa3DOIkz1EOsHWzTx+n0zKfqcbgXi4DJx+C1bjptYPR
BPZL8DAeWuA8ebudVT44yEp82G96/Ggcf7F33xMxe0yc+Xa6owIDAQABoAAwDQYJ
KoZIhvcNAQEFBQADggEBAB0kcrFccSmFDmxox0Ne01UIqSsDqHgL+XmHTXJwre6D
hJSZwbvEtOK0G3+dr4Fs11WuUNt5qcLsx5a8uk4G6AKHMzuhLsJ7XZjgmQXGECpY
Q4mC3yT3ZoCGpIXbw+iP3lmEEXgaQL0Tx5LFl/okKbKYwIqNiyKWOMj7ZR/wxWg/
ZDGRs55xuoeLDJ/ZRFf9bI+IaCUd1YrfYcHIl3G87Av+r49YVwqRDT0VDV7uLgqn
29XI1PpVUNCPQGn9p/eX6Qo7vpDaPybRtA2R7XLKjQaF9oXWeCUqy1hvJac9QFO2
97Ob1alpHPoZ7mWiEuJwjBPii6a9M9G30nUo39lBi1w=
-----END CERTIFICATE REQUEST-----

    • SSL key:
      Copy-paste the content of a valid SSL key to authenticate.
      SSL key file format: .pem
      Example:

      -----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEA3Tz2mr7SZiAMfQyuvBjM9Oi..Z1BjP5CE/Wm/Rr500P
RK+Lh9x5eJPo5CAZ3/ANBE0sTK0ZsDGMak2m1g7..3VHqIxFTz0Ta1d+NAj
wnLe4nOb7/eEJbDPkk05ShhBrJGBKKxb8n104o/..PdzbFMIyNjJzBM2o5y
5A13wiLitEO7nco2WfyYkQzaxCw0AwzlkVHiIyC..71pSzkv6sv+4IDMbT/
XpCo8L6wTarzrywnQsh+etLD6FtTjYbbrvZ8RQM..Hg2qxraAV++HNBYmNW
kbJ+q+rsJxQlaipn2M4lGuQJEfIxELFDyd3XpxP..Un/82NZNXlPmRIopXs
2T91jiLZEUKQw+n73j26adTbteuEaPGSrTZxBLR..yssO0wWomUyILqVeti
+PK+aXKwguI6bxLGZ3of0UH+mGsSl0mkp7kYZCm..OTQtfeRqP8rDSC7DgA
kHc5ajYqh04AzNFaxjRo+M3IGICUaOdKnXd0Fda..QwfoaX4QlRTgLqb7AN
ZTzM9WbmnYoXrx17kZlT3lsCgYEAm757XI3WJVj..WoLj1+v48WyoxZpcai
uv9bT4Cj+lXRS+gdKHK+SH7J3x2CRHVS+WH/SVC..DxuybvebDoT0TkKiCj
BWQaGzCaJqZa+POHK0klvS+9ln0/6k539p95tfX..X4TCzbVG6+gJiX0ysz
Yfehn5MCgYEAkMiKuWHCsVyCab3RUf6XA9gd3qY..fCTIGtS1tR5PgFIV+G
engiVoWc/hkj8SBHZz1n1xLN7KDf8ySU06MDggB..hJ+gXJKy+gf3mF5Kmj
DtkpjGHQzPF6vOe907y5NQLvVFGXUq/FIJZxB8k..fJdHEm2M4=
-----END RSA PRIVATE KEY-----

    • SSL key password:
      Enter the SSL password or passphrase for the SSL key.
      This field is masked.

  19. Select the SSL verification checkbox to test the SSL connection and to verify that it works as expected, if the TAXII server requires an SSL certificate to authenticate and to access its TAXII services.

  20. In the Path to SSL CA bundle file field, enter the path to the CA bundle file containing the public, intermediate, and root certificates for SSL authentication. The SSL CA bundle specified here is part of the server certificate validation chain.
    SSL CA bundle file format: .ca-bundle.

  21. To store your changes, click Save; to discard them, click Cancel.

Assign permissions to the user role#

The designated platform user role to manage TAXII feeds requires read access to specific platform resources:

Resource

Access level

Data sources:

  • Incoming feeds

  • Groups

Read

Feeds:

  • Incoming feeds

  • Outgoing feeds

Read

TAXII services:

  • Discovery

  • Collection

  • Inbox

  • Poll

Read

To manage data exchange through a TAXII feed, a platform user needs at least a basic set of permissions.
If the user also interacts with other platform features, such as datasets and workspaces, you can integrate this basic permission set with the default permissions granted to the default Threat Analyst role.

These are non-mandatory guidelines. You may need to fine-tune user permissions based on trial and error, practical experience to best suit your environment and your needs.

To view permissions for the the default Threat Analyst role:

  1. In the side navigation bar click > User management > Roles.
    To sort items by column header:

    1. Click the header of the column whose content you want to sort.

    2. Click or to sort the content in either ascending or descending order, respectively.

  2. Under Role name, select Threat Analyst.

  3. In the Threat Analyst detail pane, in the Overview tab, you can view a list of permissions granted to the role.

Basic permission set for the user role#

Sender automation role

Receiver automation role

Required

Notes

  • read configurations

  • read content-blocks

  • read content-types

  • read destinations

  • read entities

  • read extracts

  • read intel-sets

  • read outgoing-feeds

  • read sources

  • read taxii-services

  • read transports

  • read configurations

  • read content-blocks

  • read content-types

  • read destinations

  • read entities

  • read extracts

  • read incoming-feeds

  • read intel-sets

  • read sources

  • read taxii-services

  • read transports

Yes

Different permissions between sender and receiver automation roles are highlighted in bold.

  • modify incoming-feeds

  • modify taxii-services

See notes

The sender automation user role must have also these permissions if:

  • A platform-to-platform data exchange implementation uses a TAXII inbox outgoing feed TAXII inbox incoming feed setup.

  • A TAXII inbox outgoing feed uses Basic authentication.

modify outgoing-feeds

See notes

The receiver automation user role must have also this permission if:

  • A platform-to-platform data exchange implementation uses a TAXII inbox outgoing feed TAXII inbox incoming feed setup.

  • A TAXII inbox incoming feed uses Basic authentication.