Create a TAXII incoming feed

To exchange data between two platform instances configure:

  • A TAXII inbox or a TAXII poll feed (outgoing incoming).

  • A TAXII data feed collection both platform instances can access to:

    • Make data available (Alice, the publishing platform instance).

    • Retrieve data (Barbara, the recipient platform instance).

  • An EclecticIQ JSON content type for both the outgoing and the incoming feeds.

On the receiving platform instance (Barbara) configure either a TAXII inbox incoming feed or a TAXII poll incoming feed.


About transport

  • Both platform instances must exchange data through the same TAXII transport protocol service: either TAXII inbox or TAXII poll.

About content

  • Although TAXII feed configurations make available more than one content type for the incoming and the outgoing feeds, platform to platform data exchange officially supports only EclecticIQ JSON as a valid data exchange content type.

  • Regardless of Alice's outgoing and Barbara's incoming feeds using either the TAXII inbox or the TAXII poll transport types, they both need to publish data to and retrieve data from the same TAXII collection:

    • Alice (publisher) should publish content through a TAXII outgoing feed and a specified Collection name – for a TAXII poll transport type – or a Destination collection name – for a TAXII inbox transport type.

    • Barbara (recipient) should ingest content through a TAXII incoming feed, and it should ingest from or poll the same Collection name specified under Collection name or Destination collection name in Alice's TAXII outgoing feed configuration.

About user access control and permissions

  • Set the automation user and, where applicable, group you created as the user and the authorized group that are granted access to the feed.

  • Specify the automation user's user name and password in the corresponding feed configuration fields.

  • If you enable basic authentication, make sure the automation role has the required additional permissions.

Create a TAXII inbox incoming feed

Set up and configure transport and content types for TAXII inbox incoming feeds to retrieve and process information from specific data sources supporting the TAXII inbox transport type.

To configure the general options for TAXII inbox incoming feeds, see Configure the general options and the other relevant child articles under Incoming feeds.

Assign unique names to TAXII feeds: TAXII inbox and TAXII poll incoming and outgoing feeds in the platform should all have unique names.

Configure the transport type


Before configuring a TAXII transport type for an incoming or an outgoing feed, make sure that the appropriate TAXII service is correctly configured in the platform system settings.

TAXII inbox and TAXII poll transport types require Cabby.
For more information, see official Cabby documentation, the Cabby public repo on GitHub, and the Cabby download page.


  1. Create or edit an incoming feed.

  2. From the Transport type drop-down menu, select TAXII inbox.

  3. From the Content type drop-down menu, select the appropriate content type for the data you want to ingest through the incoming feed.
    The selected content type for the feed should match the actual format of the source data.
    This can vary, depending on the intelligence sources you retrieve the data from.

  4. Select the Accept password protected archives checkbox to specify a global password to open any archives retrieved through the incoming feed.
    If the archives are password-protected, enter it in the Archive password input field.
    The specified password acts as a master password, and it is used to try to unlock and access any archives retrieved with the feed.
    Supported archive formats:

    • .rar

    • .tar

    • .tar.bz2

    • .tar.gz

    • .tar.z

    • .zip

  5. Select the Public checkbox to make the incoming feed available to all platform groups and to all platform users.
    Leave it deselected to make the incoming feed available only to specific groups.

  6. From the the Authorized groups drop-down menu, select one or more groups to grant them access to the feed.
    This option restricts access to the incoming feed only to the selected user groups and to their members.
    Authorized groups is only available when the Public checkbox is deselected (default setting).

  7. In the Collection name field, enter the name of the TAXII data collection you want to use to consolidate the incoming feed content.
    The data collection name can be max. 1024 characters long, and its XML schema must comply with the xsd:anyURI data type.
    Example: MalwareDomainList_Hostlist.

  8. To store your changes, click Save; to discard them, click Cancel.


Before deleting a group, check that is not an authorized group in an incoming or an outgoing feed configuration.
Deleting a group that is currently selected as an authorized group to access an incoming or an outgoing feed content breaks feed functionality.

If remove such a group:

  1. Remove it from the Authorized groups selection in the relevant incoming and/or outgoing feed(s).

  2. Proceed to delete the group.

Create a TAXII poll incoming feed

Set up and configure transport and content types for TAXII poll incoming feeds to retrieve and process information from specific data sources supporting the TAXII poll transport type.

To configure the general options for TAXII inbox incoming feeds, see Configure the general options and the other relevant child articles under Incoming feeds.

Assign unique names to TAXII feeds: TAXII inbox and TAXII poll incoming and outgoing feeds in the platform should all have unique names.

Configure the transport type


Before configuring a TAXII transport type for an incoming or an outgoing feed, make sure that the appropriate TAXII service is correctly configured in the platform system settings.

TAXII inbox and TAXII poll transport types require Cabby.
For more information, see official Cabby documentation, the Cabby public repo on GitHub, and the Cabby download page.


  1. Create or edit an incoming feed.

  2. From the Transport type drop-down menu, select TAXII poll.

  3. From the Content type drop-down menu, select the appropriate content type for the data you want to ingest through the incoming feed.
    The selected content type for the feed should match the actual format of the source data.
    This can vary, depending on the intelligence sources you retrieve the data from.

  4. Select the Accept password protected archives checkbox to specify a global password to open any archives retrieved through the incoming feed.
    If the archives are password-protected, enter it in the Archive password input field.
    The specified password acts as a master password, and it is used to try to unlock and access any archives retrieved with the feed.
    Supported archive formats:

    • .rar

    • .tar

    • .tar.bz2

    • .tar.gz

    • .tar.z

    • .zip

  5. In the Auto discovery field, enter the URL pointing to a TAXII discovery service.
    Feed consumers can send a request to the discovery service to obtain a list of the available TAXII services they can access and poll for content updates.
    Example: http://hailataxii.com/taxii-discovery-service
    The URL you enter here must match the platform instance base URL plus the TAXII TAXII discovery service URL endpoint configured for the platform.
    Example:

    Platform base URL

    TAXII discovery URL

    Auto Discovery URL

    eclecticiq.platform.org

    /taxii/discovery

    https://eclecticiq.platform.org/taxii/discovery

  6. In the Polling service URL field, enter the URL pointing to a TAXII poll service.
    Feed consumers can send a request to the TAXII poll service to pull data from a configured TAXII data collection, and to obtain information on available and/or updated content.
    Example: http://hailataxii.com/taxii-poll-service
    The URL you enter here must match the platform instance base URL plus the TAXII TAXII poll service URL endpoint configured for the platform.
    Example:

    Platform base URL

    TAXII poll URL

    Polling service URL URL

    eclecticiq.platform.org

    /taxii/poll

    https://eclecticiq.platform.org/taxii/poll


  7. The Collection name field is automatically populated when you select a TAXII collection by clicking in an Auto Discovery input field populated with a valid URL to an existing TAXII discovery service.
    Example: guest.Abuse_ch.

  8. From the TAXII version drop-down menu, select the TAXII version your system and the data source TAXII server support:


    If the data source TAXII server requires passing additional HTTP headers in the request, you can specify them under Extra headers.

  9. Click Add or More to insert new rows or input fields, as necessary, where you can enter additional HTTP header and corresponding value pairs.

    • In the left input field, enter the HTTP header type.
      Example: X-TAXII-Protocol

    • In the right input field, enter the HTTP header value.
      Example: urn:taxii.mitre.org:protocol:https:1.1

  10. To remove an entry from this section, click corresponding to the item(s) you want to remove.

  11. In the Subscription ID field, enter the name, label or ID identifying the subscription session.
    Usually, the data source TAXII server assigns such an ID, and it returns it in the response to a successful request.
    The subscription ID is used in subsequent requests to poll the service to receive content, and to manage available content through the feed.

  12. Click the Start ingesting from field, use the drop-down menu calendar to select an initial date and, where available, an initial time to fetch content from the intelligence provider/data source starting from a specific date in the past.
    The ingestion date you specify here refers to package timestamps. It does not refer to entity timestamps. Entities in a package can have different, older, timestamps.
    The first time you run the feed, it ingests data starting from the specified date in the past.
    Subsequent runs start incrementally from the time of the previous feed run.
    If you do not specify any start date, the feed defaults to ingesting data from January 1st, 1970.

  13. In the Days per poll field, enter an integer to specify the maximum number of days to poll at a time.
    If you select a start date to poll data from, you can enter an integer to specify the maximum number of days to poll at a time.
    This enables polling in multiple smaller batches, instead of a single batch, starting from the selected initial date.
    Each time the feed runs, it sends multiple poll requests to progressively download in batches all the relevant content from the specified start date until the present moment.

    This option works only if you select an ingestion start date in Start ingesting from.

  14. Select the Basic authentication checkbox to fill out the required information, if the data source TAXII server requires basic authentication to access the corresponding TAXII services.

  15. In the Username field, enter a valid user name to authenticate and be granted the necessary authorization to access the location of the outgoing feed content.

  16. In the Password field, enter a valid password to authenticate and be granted the necessary authorization to access the location of the outgoing feed content.

  17. In the EclecticIQ authentication URL field, enter the URL pointing to the EclecticIQ Platform instance, including the endpoint that takes the user name and password inputs to send them to the authentication mechanism.
    Example: https://${platform_host_name}/api/auth
    If the TAXII server requires an SSL certificate to authenticate and to authorize access to the corresponding TAXII services, select this checkbox to fill out the required information.

  18. In the SSL certificate field, copy-paste the content of a valid SSL certificate to authenticate.

    • SSL certificate file format: .pem
      Example:

      -----BEGIN CERTIFICATE REQUEST-----
      MIICvDCCAaQCAQAwdzELMAkGA1UEBhMCVVMxDTALBgNVBAgMBFV0YWgxDzANBgNV
      BAcMBkxpbmRvbjEWMBQGA1UECgwNRGlnaUNlcnQgSW5jLjERMA8GA1UECwwIRGln
      aUNlcnQxHTAbBgNVBAMMFGV4YW1wbGUuZGlnaWNlcnQuY29tMIIBIjANBgkqhkiG
      9w0BAQEFAAOCAQ8AMIIBCgKCAQEA8+To7d+2kPWeBv/orU3LVbJwDrSQbeKamCmo
      wp5bqDxIwV20zqRb7APUOKYoVEFFOEQs6T6gImnIolhbiH6m4zgZ/CPvWBOkZc+c
      1Po2EmvBz+AD5sBdT5kzGQA6NbWyZGldxRthNLOs1efOhdnWFuhI162qmcflgpiI
      WDuwq4C9f+YkeJhNn9dF5+owm8cOQmDrV8NNdiTqin8q3qYAHHJRW28glJUCZkTZ
      wIaSR6crBQ8TbYNE0dc+Caa3DOIkz1EOsHWzTx+n0zKfqcbgXi4DJx+C1bjptYPR
      BPZL8DAeWuA8ebudVT44yEp82G96/Ggcf7F33xMxe0yc+Xa6owIDAQABoAAwDQYJ
      KoZIhvcNAQEFBQADggEBAB0kcrFccSmFDmxox0Ne01UIqSsDqHgL+XmHTXJwre6D
      hJSZwbvEtOK0G3+dr4Fs11WuUNt5qcLsx5a8uk4G6AKHMzuhLsJ7XZjgmQXGECpY
      Q4mC3yT3ZoCGpIXbw+iP3lmEEXgaQL0Tx5LFl/okKbKYwIqNiyKWOMj7ZR/wxWg/
      ZDGRs55xuoeLDJ/ZRFf9bI+IaCUd1YrfYcHIl3G87Av+r49YVwqRDT0VDV7uLgqn
      29XI1PpVUNCPQGn9p/eX6Qo7vpDaPybRtA2R7XLKjQaF9oXWeCUqy1hvJac9QFO2
      97Ob1alpHPoZ7mWiEuJwjBPii6a9M9G30nUo39lBi1w=
      -----END CERTIFICATE REQUEST-----

    • SSL key:
      Copy-paste the content of a valid SSL key to authenticate.
      SSL key file format: .pem
      Example:

      -----BEGIN RSA PRIVATE KEY-----
      MIIEpQIBAAKCAQEA3Tz2mr7SZiAMfQyuvBjM9Oi..Z1BjP5CE/Wm/Rr500P
      RK+Lh9x5eJPo5CAZ3/ANBE0sTK0ZsDGMak2m1g7..3VHqIxFTz0Ta1d+NAj
      wnLe4nOb7/eEJbDPkk05ShhBrJGBKKxb8n104o/..PdzbFMIyNjJzBM2o5y
      5A13wiLitEO7nco2WfyYkQzaxCw0AwzlkVHiIyC..71pSzkv6sv+4IDMbT/
      XpCo8L6wTarzrywnQsh+etLD6FtTjYbbrvZ8RQM..Hg2qxraAV++HNBYmNW
      kbJ+q+rsJxQlaipn2M4lGuQJEfIxELFDyd3XpxP..Un/82NZNXlPmRIopXs
      2T91jiLZEUKQw+n73j26adTbteuEaPGSrTZxBLR..yssO0wWomUyILqVeti
      +PK+aXKwguI6bxLGZ3of0UH+mGsSl0mkp7kYZCm..OTQtfeRqP8rDSC7DgA
      kHc5ajYqh04AzNFaxjRo+M3IGICUaOdKnXd0Fda..QwfoaX4QlRTgLqb7AN
      ZTzM9WbmnYoXrx17kZlT3lsCgYEAm757XI3WJVj..WoLj1+v48WyoxZpcai
      uv9bT4Cj+lXRS+gdKHK+SH7J3x2CRHVS+WH/SVC..DxuybvebDoT0TkKiCj
      BWQaGzCaJqZa+POHK0klvS+9ln0/6k539p95tfX..X4TCzbVG6+gJiX0ysz
      Yfehn5MCgYEAkMiKuWHCsVyCab3RUf6XA9gd3qY..fCTIGtS1tR5PgFIV+G
      engiVoWc/hkj8SBHZz1n1xLN7KDf8ySU06MDggB..hJ+gXJKy+gf3mF5Kmj
      DtkpjGHQzPF6vOe907y5NQLvVFGXUq/FIJZxB8k..fJdHEm2M4=
      -----END RSA PRIVATE KEY-----

    • SSL key password:
      Enter the SSL password or passphrase for the SSL key.
      This field is masked.

  19. Select the SSL verification checkbox to test the SSL connection and to verify that it works as expected, if the TAXII server requires an SSL certificate to authenticate and to access its TAXII services.

  20. In the Path to SSL CA bundle file field, enter the path to the CA bundle file containing the root, intermediate, and public certificates for SSL authentication.
    The SSL CA bundle specified here is part of the server certificate validation chain.
    SSL CA bundle file format: .ca-bundle.

  21. To store your changes, click Save; to discard them, click Cancel.

Configure the content type

  1. From the Content type drop-down menu, select EclecticIQ JSON as the appropriate content type for the data you want to exchange between the two platform instances.
    The selected content type needs to match the actual format of the source/input data.

  2. From the Datasets drop-down menu, select one or more existing datasets to use as sources to populate the outgoing feed content.
    For the feed not to be empty, at least one selected dataset should contain entities and observables in the same format as the configured content type for the feed.

  3. From the Update strategy drop-down menu, select the preferred method to populate the outgoing feed with data before publishing it:

    About update strategies

    Update strategies help define how content is aggregated and packaged for publication when an outgoing feed task runs:

    • Append: the published packages contain only new entities and observables ingested in the platform after the previous execution of the outgoing feed.
      Every time the outgoing feed task runs, it generates the content for publication by retrieving only new, unpublished entities and observables.

    • Replace: the published packages contain new entities and observables, as well as existing ones that were included also in the previous execution of the outgoing feed.
      Every time the outgoing feed task runs, it generates the content for publication by retrieving:

      • Existing entities and observables that were published in the previous execution of the outgoing feed.

      • New entities and observables ingested after the previous execution of the outgoing feed.

    • Diff: this option is available only for the EclecticIQ Entities CSV and EclecticIQ Observables CSV content types.
      Every time the outgoing feed task runs, new data is compared against existing data to identify any differences between the two datasets:

      • At entity level: any entities added to or removed from the set, if EclecticIQ Entities CSV is the designated content type for the feed.

      • At observable level: any observable added to or removed from the entities in the set, if EclecticIQ Observables CSV is the designated content type for the feed.

      Depending on the selected CSV content option, each row in the CSV output contains information about one entity being added or removed, or one observable being added or removed.
      An extra diff column is added to the output CSV to indicate if a row, and therefore either an entity or an observable, has been added to or removed from the set.

      This option enables identifying changes in a feed between two executions without downloading the whole feed every time.
      Update strategies rely on the last_updated_at database field to identify entities whose timestamp value was updated since the previous execution of the outgoing feed.
      Entities with a more recent timestamp value compared to the previous execution of the outgoing feed are packaged and included in the published content of the outgoing feed.

      • Changes to the data section of an entity create a new version of the entity.
        They also add a new log entry to the entity history to record the changes.

      • Changes to the meta section of an entity do not create a new version of the entity.
        However, they do update the timestamp value of the last_update_at database field.

The EclecticIQ JSON content type is suitable for machine consumption.
For example, you can use EclecticIQ JSON as a content type in a platform-to-platform data exchange setup.

Under Content configuration, set the EclecticIQ JSON content type option:

  1. Select the Override producer checkbox to replace the value defining the identity of the original producer of the data with the producer name defined for the platform.
    To find this value, click > STIX and TAXII > STIX > Add STIX settings > Producer.
    Leave it deselected to include the identity of the original producer of the information.

    This setting affects the data.producer.identity.name value in the entity JSON entity data structure:

    {
    "data": {
    "producer": {
    "type": "information-source",
    "identity": {
    "type": "identity"
    "name": "${producer_identity}", // ex.:'EclecticIQ'
     
    }
    }
    }
    }