Configure content types

ArcSight CEF

The ArcSight CEF content type is suitable for machine consumption.

Typical use cases include feeding an ArcSight CEF outgoing feed to a SIEM system such as ArcSight ESM or to a Syslog server for further processing.

The ArcSight CEF content type does not have any specific configuration parameters.

EclecticIQ Entities CSV

The EclecticIQ Entities CSV content type outputs CSV files with column headers where each row holds data describing one entity.

The EclecticIQ Entities CSV content type enables comparing two different outputs from the same outgoing feed to diff them, and to examine any changes at entity-level.

This is a quick and inexpensive way to check for feed content changes and updates since the previous feed task run.

To do so, select Update strategy, and click Diff.

The EclecticIQ Entities CSV content type does not have any specific configuration parameters.

EclecticIQ Observables CSV

The EclecticIQ Observables CSV outgoing feed outputs CSV files with column headers where each row holds data describing one observable.

For example: an IP address, a hash, a geographic location name, and so on.

The EclecticIQ Observables CSV content type enables comparing two different outputs from the same outgoing feed to diff them, and to examine any changes at observable-level.

This is a quick and inexpensive way to check for feed content changes and updates since the previous feed task run.

To do so, select Update strategy, and click Diff.

The EclecticIQ Entities CSV content type does not have any specific configuration parameters.

If you choose EclecticIQ Observables CSV, select also at least one observable type from the Observable types drop-down menu, and at least one enrichment observable type from the Enrichment observable types drop-down menu.

EclecticIQ HTML Report

Creates a static HTML file for each Report entity in a dataset and makes them available through the outgoing feed.

Using the Send email transport type sends the HTML report as an email attachment.

For information on update strategies, see About outgoing feeds.

To configure the EclecticIQ HTML Report content type, fill out these fields under the Content configuration section when you set up your outgoing feed:

Required fields are marked with an asterisk (*).

Field

Description

Include following tags and taxonomy*

Tags or taxonomies added here are added as “Tags” to the HTML report.

Type tag names or select one or more tags from the drop-down menu.

Selecting a “parent” tag from the drop-down menu, such as Admiralty code, adds all its children.

Include terms of use

Select to add a “Terms of use” section to the report.

The “Terms of use” section is filled with the contents of the Default terms of use field in your Intel report settings. Set it by going to Settings (images/download/attachments/82475396/cog.svg ) > System settings > Intel report > Edit settings and adding your terms of use to the Default terms of use field.

Include logo

Select to add your organization’s logo to the generated report.

This uses the image specified in your Intel report settings to brand your reports. Set it by going to Settings (images/download/attachments/82475396/cog.svg ) > System settings > Intel report > Edit settings and adding a URL to your logo image in the Specify a URL for your company logo used in the email template field.

Your image must:

  • Be in .png or .jpg format

  • Have a maximum size of 200 x 200 px

  • Be less than 320 KB

  • Be accessible from the platform host

Include contact information

Select to add contact details to your report.

This uses the information specified in your Intel report settings to brand your reports. Set it by going to Settings (images/download/attachments/82475396/cog.svg ) > System settings > Intel report > Edit settings and adding contact details to the D**efault contact information** field.

Root URL of EclecticIQ platform installation

Set this to the URL at which you can access the platform at.

Defaults to the host name set in Settings (images/download/attachments/82475396/cog.svg ) > System settings > General > Hostname if left empty.

Additional information

Add information you want to include with your reports.

The contents of this field is included at the end of each generated report.

Image of sample HTML report

images/download/attachments/82475396/download-html-report-taxii-stand-c993eea5-1.png

EclecticIQ HTML Report Digest

Creates a HTML digest that summarizes the Report entities that are included in a dataset, and makes it available through the outgoing feed.

Using the Send email transport type sends the HTML report as an email attachment.

For information on update strategies, see About outgoing feeds.

To configure the EclecticIQ HTML Report Digest content type, fill out these fields under the Content configuration section when you set up your outgoing feed:

Required fields are marked with an asterisk (*).

Field

Description

Include following tags and taxonomy*

Tags or taxonomies added here are added as “Tags” to the HTML report.

Type tag names or select one or more tags from the drop-down menu.

Selecting a “parent” tag from the drop-down menu, such as Admiralty code, adds all its children.

Include terms of use

Select to add a “Terms of use” section to the report.

The “Terms of use” section is filled with the contents of the Default terms of use field in your Intel report settings. Set it by going to Settings (images/download/attachments/82475396/cog.svg ) > System settings > Intel report > Edit settings and adding your terms of use to the Default terms of use field.

Include logo

Select to add your organization’s logo to the generated report.

This uses the image specified in your Intel report settings to brand your reports. Set it by going to Settings (images/download/attachments/82475396/cog.svg ) > System settings > Intel report > Edit settings and adding a URL to your logo image in the Specify a URL for your company logo used in the email template field.

Your image must:

  • Be in .png or .jpg format

  • Have a maximum size of 200 x 200 px

  • Be less than 320 KB

  • Be accessible from the platform host

Include contact information

Select to add contact details to your report.

This uses the information specified in your Intel report settings to brand your reports. Set it by going to Settings (images/download/attachments/82475396/cog.svg ) > System settings > Intel report > Edit settings and adding contact details to the D**efault contact information** field.

Root URL of EclecticIQ platform installation

Set this to the URL at which you can access the platform at.

Defaults to the host name set in Settings (images/download/attachments/82475396/cog.svg ) > System settings > General > Hostname if left empty.

Additional information

Add information you want to include with your reports.

The contents of this field is included at the end of each generated report.

Image of sample HTML digest report

images/download/attachments/82475396/download-html-digest-taxii-stand-259007a0-1.png

EclecticIQ JSON

The EclecticIQ JSON content type is suitable for machine consumption.

For example, you can use EclecticIQ JSON as a content type in a platform-to-platform data exchange setup.

Under Content configuration, set the EclecticIQ JSON content type option:

  1. Select the Override producer checkbox to replace the value defining the identity of the original producer of the data with the producer name defined for the platform.

    To find this value, click Settings (images/download/attachments/82475396/cog.svg ) > STIX and TAXII > STIX > Add STIX settings > Producer.

    Leave it deselected to include the identity of the original producer of the information.

    This setting affects the data.producer.identity.name value in the entity JSON entity data structure:

    {
      "data": {
        "producer": {
          "type": "information-source",
          "identity": {
            "type": "identity"
            "name": "${producer_identity}", // ex.:'EclecticIQ'
    
          }
        }
      }
    }

PAN-OS external Dynamic List

About PAN-OS external Dynamic List

The PAN-OS External Dynamic List outgoing feed sends the Palo Alto firewall blocklists containing IP domain, and URL sightings.

In a typical scenario, you want the platform to send the firewall only malicious IPs, domains, and URLs that are flagged by reliable sources because the firewall will block them.

It is a good idea to create a dataset in the platform that acts as a data source for the outgoing feed:

  • If you create a static dataset, make sure you populate it only with entities having appropriate characteristics.

  • If you create a dynamic dataset, make sure the search query acting as the filtering rule for the dynamic dataset content is set up to populate it with the desired entities.

In the PAN-OS External Dynamic List outgoing feed configuration, make sure you set the appropriate filters for observable maliciousness under Allowed observable states, TLP code, and source reliability.

For example:

  • Set Allowed observable states to include only malicious ones.

    Do not include any observables marked as Safe or Ignore.

  • Set the source reliability to the desired level.

    For example, you may want to exclude data from unreliable sources, as the firewall may automatically apply a blocking policy to it.

  • Set the TLP code based on the applicable content sharing policies in your organization.

  • Set the outgoing feed Update strategy to Replace, so that each feed run replaces the existing extended dynamic list content with both existing and new data since the previous feed run.

    Extended dynamic lists are not based on incremental changes: each time the firewall reloads them, previous block rules based on the same list are dropped, and new ones are created.

    Therefore, you always want each feed output (that is, each published list) to include all the available content up to that moment.

Configure PAN-OS External Dynamic List

  1. Go to the Transport and content section, and from the Content type drop-down menu, select PAN-OS External Dynamic List.

  2. Go to the Content configuration section, and from the Palo Alto PAN-OS External Dynamic List drop-down menu, select type of list you want to publish through the outgoing feed:

    • PAN-OS IP External Dynamic List: the feed outputs an IP address list to feed the Palo Alto firewall with, so that the firewall blocks the included IP addresses.

    • PAN-OS Domain External Dynamic List: the feed outputs a domain name list to feed the Palo Alto firewall with, so that the firewall blocks the included domain names.

    • PAN-OS URL External Dynamic List: the feed outputs a URL list to feed the Palo Alto firewall with, so that the firewall blocks the included URLs.

  3. Go to the Transport configuration section, and set the appropriate transport type options for the selected transport type for the PAN-OS External Dynamic List content.

Supported External Dynamic List types

Palo Alto External Dynamic Lists (EDL) support IP addresses, domains, and URLs.

The external dynamic lists the platform published through the PAN-OS External Dynamic List outgoing feed support these list types:

List type

Description

Examples

IP addresses

  • The platform exports individual IP addresses to an EDL list.

  • The exported EDL list holds one record, that is, one IP address, per line.

  • Exported IP addresses support both ipv4 and ipv6.

192.168.20.10
2001:db8:123:1::1
8.8.8.8
192.168.20.10

Domain names

  • The platform exports domains to an EDL list.

  • The exported EDL list holds one record, that is, one domain, per line.

www.instagram.com
malicious.com
abc.baddomain.cc

URLs

  • The platform exports URLs to an EDL list.

  • The exported EDL list holds one record, that is, one URL, per line.

  • Exported URLs should not include the transport protocol prefix.

  • Examples of not allowed protocol prefixes:

    • http://

    • https://

    • ftp://

  • URLs record format is not case sensitive.

  • Wildcard characters are supported.

www.wellknownsite.com/bad-page
www.yahoo.com/search=*
www.facebook.com/messenger

You can use PAN-OS External Dynamic List outgoing feeds to feed back to a PAN-OS-controlled firewall sightings generated from incoming indicators of compromise whose source is the same or another PAN-OS-controlled firewall.

Plain text value

The Plain text value content type is suitable for machine consumption.

Typical use cases include feeding a plain text value outgoing feed to an external compatible device to instrument further processing or to trigger a response action.

The Plain text value content configuration options set up a rule to identify matching data to include in the outgoing feed content.

The rule works like this:

  1. “Check if this JSON field exists”: check if the JSON path defined in Field to check a conditional value in points to an existing entity field in the entity JSON structure.

  2. “If the JSON field exists, check if its value matches this input”: if the JSON path defined in Field to check a conditional value in points to an existing entity JSON field, check if its value matches the input defined in Only use entities that match this conditional value.

  3. “Check if this other JSON field exists. If it does, get its value and add it to the feed content”: if the JSON path defined in Field to check a conditional value in points to an existing entity JSON field, and if its value matches the input defined in Only use entities that match this conditional value, go to the JSON field defined in Field to take values from, and retrieve its value: this is what we want to include in the outgoing feed content.

Under Content configuration set the Plain text value content type options to define the rule behavior:

  1. Field to check a conditional value in: this condition works together with Only use entities that match this conditional value to define the key/value pair the rule uses to filter specific entities.

    Enter a JSON path pointing to the JSON field of the entity type whose content you want to include for publication in the outgoing feed.

    • JSON paths are relative to the top-level data field. The the top-level data field is the implicit root of a JSON path.

      For example, to point to the title or to the description fields of an entity, specify title or description, instead of data.title or data.description.

    • JSON paths are strings pointing to specific JSON fields in a JSON data structure.

      They define locations in the entity structure where the rule looks for data values matching specified data patterns.

    • The JSON path format is a string where dots (.) define parent-child relationships in the JSON data structure.

      Do not include square brackets ([ ]) in the path input: they are stripped during execution.

      ..? so how do we deal with arrays/lists?

      Therefore, you cannot use square brackets to point to array members through their array index values.

  2. Only use entities that match this conditional value: this condition works together with Field to check a conditional value in to define the key/value pair the rule uses to filter specific entities.

    Enter a string to define the value to match.

    Example: snort

    Allowed values:

    • Literal alphanumeric text input.

    • Regex data pattern.

      • The regex syntax this field accepts is the Elasticsearch regular expression syntax.

        The main peculiarities of the Elasticsearch DSL query syntax are:

        • Anchors – ^ and $ – are implied at the beginning and at the end of the regex.

          Do not include them in the regex pattern input.

        • If you insert explicit anchor characters in the value input field, they are interpreted as literal values.

        • Escape special characters:

          ( . ? + | { } [ ] ( ) " \ / )

          To escape a special characters, prepend a backslash \ to it.

          Example: \{ \}

          At the moment, Elasticsearch regular expression syntax optional operators are not supported.

          Although the regex syntax used in the UI of the platform is that of ElasticSearch, the regex syntax used by the platform’s ingestion engine is Python regex. In the vast majority of cases this does not lead to conflicts, but if your discovery rule is not working the way you want it to, this may be the reason.

          Here are two differences that commonly cause problems:

          • ElasticSearch searches for whole strings, while Python searches everywhere in a string. For example,

            Input: Cert-BUND

            Pattern: ert

            ES result: <none>

            Python: will match

          • ElasticSearch regex syntax requires you to leave out anchors, whereas Python regex syntax requires you to include them.

          If you are experiencing problems in this area, contact Customer Support for help in writing the discovery rule you want.

  3. Field to take values from: defines the entity JSON field whose value you want to to add to the outgoing feed.

    Enter a JSON path pointing to the entity JSON field whose value you want to fetch and include for publication in the outgoing feed.

    • JSON paths are relative to the top-level data field. The the top-level data field is the implicit root of a JSON path.

      For example, to point to the title or to the description fields of an entity, specify title or description, instead of data.title or data.description.

    • JSON paths are strings pointing to specific JSON fields in a JSON data structure.

      They define locations in the entity structure where the rule looks for data values matching specified data patterns.

    • The JSON path format is a string where dots (.) define parent-child relationships in the JSON data structure.

      Do not include square brackets ([ ]) in the path input: they are stripped during execution.

      ..? so how do we deal with arrays/lists?

      Therefore, you cannot use square brackets to point to array members through their array index values.

Example

You can configure this rule to send relevant data to an external Snort or Suricata instance, where they can be further processed or used to initiate a specific response action:

  • Field to check a conditional value in: data.test_mechanisms.test_mechanism_type

  • Only use entities that match this conditional value: snort

  • Field to take values from: data.test_mechanisms.rules.value

The rule uses these conditions to:

  1. Look for platform entities containing Snort rules: data.test_mechanisms.test_mechanism_type: snort

  2. If the previous condition yields matching entities, the rule looks in those entities to see if they contain this field: data.test_mechanisms.rules.value

  3. If they do, the rule fetches the value stored in data.test_mechanisms.rules.value to include it in the outgoing feed content.

Retrieved values are added to the outgoing feed, one value per line.

The values in question should be valid Snort rules for the resulting outgoing feed content to be meaningful.

Example:

alert tcp $HOME_NET any -> [72.20.35.70,72.20.35.120] 6661 (msg:\"ET CNC Shadowserver Reported CnC Server Port 6661 Group 1\"; flags:S; reference:url,doc.emergingthreats.net/bin/view/Main/BotCC; reference:url,www.shadowserver.org; threshold: type limit, track by_src, seconds 360, count 1; classtype:trojan-activity; flowbits:set,ET.Evil; flowbits:set,ET.BotccIP; sid:2405018; rev:3633;)

STIX 1.2

The STIX 1.2 content type is suitable for machine consumption.

Typical use cases include feeding a STIX 1.2-format outgoing feed to an external STIX-compatible device to instrument further processing or to trigger a response action.

Under Content configuration, do the following:

  1. Select the Override producer checkbox to replace the original producer identity with the Producer name defined for the platform.

    You find this original identity by going to the left navigation bar, click (images/download/attachments/82475396/cog.svg ) > STIX and TAXII > STIX.

    You will find the original identity in the Producer field.

    Leave it deselected to include the original producer of the information.

    This setting changes the following nested XML element in the entity STIX structure:

        EclecticIQ
  2. Select the Include EclecticIQ-specific STIX extensions checkbox to enable EclecticIQ STIX extensions for the entities and the observables included in the outgoing feed content.

    Leave the option deselected, unless the recipient of the feed content can correctly validate and parse STIX 1.2 content with EclecticIQ STIX extensions.

    Otherwise, integrations may break, and data exchange may fail.

STIX validation

The STIX validator can understand the extension schemas, provided they are stored in the same directory as they official STIX schemas.

When using the STIX validator, specify the location of the standard and the extension schemas by passing the --schema-dir argument:

stix_validator.py --schema-dir /path/to/stix/schemas input.xml