Create a TAXII incoming feed#

To exchange data between two platform instances configure:

  • A TAXII inbox or a TAXII poll feed (outgoing -> incoming).

  • A TAXII data feed collection both platform instances can access to:

    • Make data available (Alice, the publishing platform instance).

    • Retrieve data (Barbara, the recipient platform instance).

  • An EclecticIQ JSON content type for both the outgoing and the incoming feeds.

On the receiving platform instance (Barbara) configure either a TAXII inbox incoming feed or a TAXII poll incoming feed.

Note

About transport

  • Both platform instances must exchange data through the same TAXII transport protocol service: either TAXII inbox or TAXII poll.

About content

  • Although TAXII feed configurations make available more than one content type for the incoming and the outgoing feeds, platform to platform data exchange officially supports only EclecticIQ JSON as a valid data exchange content type.

  • Regardless of Alice’s outgoing and Barbara’s incoming feeds using either the TAXII inbox or the TAXII poll transport types, they both need to publish data to and retrieve data from the same TAXII collection:

    • Alice (publisher) should publish content through a TAXII outgoing feed and a specified Collection name – for a TAXII poll transport type – or a Destination collection name – for a TAXII inbox transport type.

    • Barbara (recipient) should ingest content through a TAXII incoming feed, and it should ingest from or poll the same Collection name specified under Collection name or Destination collection name in Alice’s TAXII outgoing feed configuration.

About user access control and permissions

  • Set the automation user and, where applicable, group you created as the user and the authorized group that are granted access to the feed.

  • Specify the automation user’s user name and password in the corresponding feed configuration fields.

  • If you enable basic authentication, make sure the automation role has the required additional permissions.

Create a TAXII inbox incoming feed#

Set up and configure transport and content types for TAXII inbox incoming feeds to retrieve and process information from specific data sources supporting the TAXII inbox transport type.

To configure the general options for TAXII inbox incoming feeds, see Configure the general options and the other relevant child articles under Incoming feeds.

Note

Assign unique names to TAXII feeds: TAXII inbox and TAXII poll incoming and outgoing feeds in the platform should all have unique names.

Configure the transport type#

Note

Before configuring a TAXII transport type for an incoming or an outgoing feed, make sure that the appropriate TAXII service is correctly configured in the platform_settings.py.

TAXII inbox and TAXII poll transport types require Cabby.

For more information, see official Cabby documentation, the Cabby public repo on GitHub, and the Cabby download page.

  1. Create or edit an incoming feed.

  2. From the Transport type drop-down menu, select TAXII inbox.

  3. From the Content type drop-down menu, select the appropriate content type for the data you want to ingest through the incoming feed.

    The selected content type for the feed should match the actual format of the source data.

    This can vary, depending on the intelligence sources you retrieve the data from.

  4. Select the Accept password protected archives checkbox to specify a global password to open any archives retrieved through the incoming feed.

    If the archives are password-protected, enter it in the Archive password input field.

    The specified password acts as a master password, and it is used to try to unlock and access any archives retrieved with the feed.

    Supported archive formats:

    • .rar

    • .tar

    • .tar.bz2

    • .tar.gz

    • .tar.z

    • .zip

  5. Select the Public checkbox to make the incoming feed available to all platform groups and to all platform users.

    Leave it deselected to make the incoming feed available only to specific groups.

  6. From the the Authorized groups drop-down menu, select one or more groups to grant them access to the feed.

    This option restricts access to the incoming feed only to the selected user groups and to their members.

    Authorized groups is only available when the Public checkbox is deselected (default setting).

  7. In the Collection name field, enter the name of the TAXII data collection you want to use to consolidate the incoming feed content.

    The data collection name can be max. 1024 characters long, and its XML schema must comply with the xsd:anyURI data type.

    Example: MalwareDomainList_Hostlist.

  8. To store your changes, click Save; to discard them, click Cancel.

Note

Before deleting a group, check that is not an authorized group in an incoming or an outgoing feed configuration.

Deleting a group that is currently selected as an authorized group to access an incoming or an outgoing feed content breaks feed functionality.

If remove such a group:

  1. Remove it from the Authorized groups selection in the relevant incoming and/or outgoing feed(s).

  2. Proceed to delete the group.

Create a TAXII poll incoming feed#

Set up and configure transport and content types for TAXII inbox incoming feeds to retrieve and process information from specific data sources supporting the TAXII inbox transport type.

To configure the general options for TAXII inbox incoming feeds, see Configure the general options and the other relevant child articles under Incoming feeds.

Note

Assign unique names to TAXII feeds: TAXII inbox and TAXII poll incoming and outgoing feeds in the platform should all have unique names.

Configure the transport type#

Note

Before configuring a TAXII transport type for an incoming or an outgoing feed, make sure that the appropriate TAXII service is correctly configured in the platform_settings.py.

TAXII inbox and TAXII poll transport types require Cabby.

For more information, see official Cabby documentation, the Cabby public repo on GitHub, and the Cabby download page.

  1. Create or edit an incoming feed.

  2. From the Transport type drop-down menu, select TAXII poll.

  3. From the Content type drop-down menu, select the appropriate content type for the data you want to ingest through the incoming feed.

    The selected content type for the feed should match the actual format of the source data.

    This can vary, depending on the intelligence sources you retrieve the data from.

  4. Select the Accept password protected archives checkbox to specify a global password to open any archives retrieved through the incoming feed.

    If the archives are password-protected, enter it in the Archive password input field.

    The specified password acts as a master password, and it is used to try to unlock and access any archives retrieved with the feed.

    Supported archive formats:

    • .rar

    • .tar

    • .tar.bz2

    • .tar.gz

    • .tar.z

    • .zip

  5. In the Auto discovery field, enter the URL pointing to a TAXII discovery service.

    Feed consumers can send a request to the discovery service to obtain a list of the available TAXII services they can access and poll for content updates.

    Example: http://hailataxii.com/taxii-discovery-service

    The URL you enter here must match EclecticIQ Intelligence Center instance base URL plus the TAXII discovery service URL endpoint configured for EclecticIQ Intelligence Center.

    Example:

    Platform base URL

    TAXII discovery URL

    Auto Discovery URL

    eclecticiq.platform.org

    /taxii/discovery

    https://eclecticiq.platform.org/taxii/discovery

  6. In the Polling service URL field, enter the URL pointing to a TAXII poll service.

    Feed consumers can send a request to the TAXII poll service to pull data from a configured TAXII data collection, and to obtain information on available and/or updated content.

    Example: http://hailataxii.com/taxii-poll-service

    The URL you enter here must match EclecticIQ Intelligence Center instance base URL plus the TAXII discovery service URL endpoint configured for EclecticIQ Intelligence Center.

    Example:

    Platform base URL

    TAXII poll URL

    Polling service URL URL

    eclecticiq.platform.org

    /taxii/poll

    https://eclecticiq.platform.org/taxii/poll

  7. The Collection name field is automatically populated when you select a TAXII collection by clicking in an Auto Discovery input field populated with a valid URL to an existing TAXII discovery service.

    Example: guest.Abuse_ch.

  8. From the TAXII version drop-down menu, select the TAXII version your system and the data source TAXII server support:

    If the data source TAXII server requires passing additional HTTP headers in the request, you can specify them under Extra headers.

  9. Select + Add or + More to insert new rows or input fields, as necessary, where you can enter additional HTTP header and corresponding value pairs.

    • In the left input field, enter the HTTP header type. Example: X-TAXII-Protocol

    • In the right input field, enter the HTTP header value. Example: urn:taxii.mitre.org:protocol:https:1.1

  10. To remove an entry from this section, click corresponding to the item(s) you want to remove.

  11. In the Subscription ID field, enter the name, label or ID identifying the subscription session.

    Usually, the data source TAXII server assigns such an ID, and it returns it in the response to a successful request.

    The subscription ID is used in subsequent requests to poll the service to receive content, and to manage available content through the feed.

  12. Select the Start ingesting from field, use the drop-down menu calendar to select an initial date and, where available, an initial time to fetch content from the intelligence provider/data source starting from a specific date in the past.

    The ingestion date you specify here refers to package timestamps. It does not refer to entity timestamps. Entities in a package can have different, older, timestamps.

    The first time you run the feed, it ingests data starting from the specified date in the past.

    Subsequent runs start incrementally from the time of the previous feed run.

    If you do not specify any start date, the feed defaults to ingesting data from January 1st, 1970.

  13. In the Days per poll field, enter an integer to specify the maximum number of days to poll at a time.

    If you select a start date to poll data from, you can enter an integer to specify the maximum number of days to poll at a time.

    This enables polling in multiple smaller batches, instead of a single batch, starting from the selected initial date.

    Each time the feed runs, it sends multiple poll requests to progressively download in batches all the relevant content from the specified start date until the present moment.

    Note

    This option works only if you select an ingestion start date in Start ingesting from.

  14. Select the Basic authentication checkbox to fill out the required information, if the data source TAXII server requires basic authentication to access the corresponding TAXII services.

  15. In the Username field, enter a valid user name to authenticate and be granted the necessary authorization to access the location of the outgoing feed content.

  16. In the Password field, enter a valid password to authenticate and be granted the necessary authorization to access the location of the outgoing feed content.

  17. In the EclecticIQ authentication URL field, enter the URL pointing to the EclecticIQ Platform instance, including the endpoint that takes the user name and password inputs to send them to the authentication mechanism.

    Example: https://${platform_host_name}/api/auth

    If the TAXII server requires an SSL certificate to authenticate and to authorize access to the corresponding TAXII services, select this checkbox to fill out the required information.

  18. In the SSL certificate field, copy-paste the content of a valid SSL certificate to authenticate.

    • SSL certificate file format: .pem

      Example:

      -----BEGIN CERTIFICATE REQUEST-----
MIICvDCCAaQCAQAwdzELMAkGA1UEBhMCVVMxDTALBgNVBAgMBFV0YWgxDzANBgNV
BAcMBkxpbmRvbjEWMBQGA1UECgwNRGlnaUNlcnQgSW5jLjERMA8GA1UECwwIRGln
aUNlcnQxHTAbBgNVBAMMFGV4YW1wbGUuZGlnaWNlcnQuY29tMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEA8+To7d+2kPWeBv/orU3LVbJwDrSQbeKamCmo
wp5bqDxIwV20zqRb7APUOKYoVEFFOEQs6T6gImnIolhbiH6m4zgZ/CPvWBOkZc+c
1Po2EmvBz+AD5sBdT5kzGQA6NbWyZGldxRthNLOs1efOhdnWFuhI162qmcflgpiI
WDuwq4C9f+YkeJhNn9dF5+owm8cOQmDrV8NNdiTqin8q3qYAHHJRW28glJUCZkTZ
wIaSR6crBQ8TbYNE0dc+Caa3DOIkz1EOsHWzTx+n0zKfqcbgXi4DJx+C1bjptYPR
BPZL8DAeWuA8ebudVT44yEp82G96/Ggcf7F33xMxe0yc+Xa6owIDAQABoAAwDQYJ
KoZIhvcNAQEFBQADggEBAB0kcrFccSmFDmxox0Ne01UIqSsDqHgL+XmHTXJwre6D
hJSZwbvEtOK0G3+dr4Fs11WuUNt5qcLsx5a8uk4G6AKHMzuhLsJ7XZjgmQXGECpY
Q4mC3yT3ZoCGpIXbw+iP3lmEEXgaQL0Tx5LFl/okKbKYwIqNiyKWOMj7ZR/wxWg/
ZDGRs55xuoeLDJ/ZRFf9bI+IaCUd1YrfYcHIl3G87Av+r49YVwqRDT0VDV7uLgqn
29XI1PpVUNCPQGn9p/eX6Qo7vpDaPybRtA2R7XLKjQaF9oXWeCUqy1hvJac9QFO2
97Ob1alpHPoZ7mWiEuJwjBPii6a9M9G30nUo39lBi1w=
-----END CERTIFICATE REQUEST-----

    • SSL key: Copy-paste the content of a valid SSL key to authenticate. SSL key file format: .pem

      Example:

      -----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEA3Tz2mr7SZiAMfQyuvBjM9Oi..Z1BjP5CE/Wm/Rr500P
RK+Lh9x5eJPo5CAZ3/ANBE0sTK0ZsDGMak2m1g7..3VHqIxFTz0Ta1d+NAj
wnLe4nOb7/eEJbDPkk05ShhBrJGBKKxb8n104o/..PdzbFMIyNjJzBM2o5y
5A13wiLitEO7nco2WfyYkQzaxCw0AwzlkVHiIyC..71pSzkv6sv+4IDMbT/
XpCo8L6wTarzrywnQsh+etLD6FtTjYbbrvZ8RQM..Hg2qxraAV++HNBYmNW
kbJ+q+rsJxQlaipn2M4lGuQJEfIxELFDyd3XpxP..Un/82NZNXlPmRIopXs
2T91jiLZEUKQw+n73j26adTbteuEaPGSrTZxBLR..yssO0wWomUyILqVeti
+PK+aXKwguI6bxLGZ3of0UH+mGsSl0mkp7kYZCm..OTQtfeRqP8rDSC7DgA
kHc5ajYqh04AzNFaxjRo+M3IGICUaOdKnXd0Fda..QwfoaX4QlRTgLqb7AN
ZTzM9WbmnYoXrx17kZlT3lsCgYEAm757XI3WJVj..WoLj1+v48WyoxZpcai
uv9bT4Cj+lXRS+gdKHK+SH7J3x2CRHVS+WH/SVC..DxuybvebDoT0TkKiCj
BWQaGzCaJqZa+POHK0klvS+9ln0/6k539p95tfX..X4TCzbVG6+gJiX0ysz
Yfehn5MCgYEAkMiKuWHCsVyCab3RUf6XA9gd3qY..fCTIGtS1tR5PgFIV+G
engiVoWc/hkj8SBHZz1n1xLN7KDf8ySU06MDggB..hJ+gXJKy+gf3mF5Kmj
DtkpjGHQzPF6vOe907y5NQLvVFGXUq/FIJZxB8k..fJdHEm2M4=
-----END RSA PRIVATE KEY-----

    • SSL key password:

      Enter the SSL password or passphrase for the SSL key. This field is masked.

  19. Select the SSL verification checkbox to test the SSL connection and to verify that it works as expected, if the TAXII server requires an SSL certificate to authenticate and to access its TAXII services.

  20. In the Path to SSL CA bundle file field, enter the path to the CA bundle file containing the root, intermediate, and public certificates for SSL authentication.

    The SSL CA bundle specified here is part of the server certificate validation chain.

    SSL CA bundle file format: .ca-bundle.

  21. To store your changes, click Save; to discard them, click Cancel.

Configure the content type#

  1. From the Content type drop-down menu, select EclecticIQ JSON as the appropriate content type for the data you want to exchange between the two platform instances. The selected content type needs to match the actual format of the source/input data.