Outgoing feed - Palo Alto PAN-OS External Dynamic List

You can publish PAN-OS External Dynamic List using any of the supported transport types to make the outgoing feed content available to a Palo Alto Networks PAN-OS-controlled firewall to trigger rule-based follow-up actions.

The data exchange format is Palo Alto Networks External Dynamic List. This enables the firewall to dynamically import the lists and to enforce policy without making configuration changes or commits on the firewall.
For this data exchange setup to work, the platform instance publishing the lists needs to be able to store them to a location that the firewall can access to retrieve the lists.

Configure content types

1. Create or edit an outgoing feed.

2. From the Content type drop-down menu, select PAN-OS External Dynamic List.

3. From the Palo Alto PAN-OS External Dynamic List drop-down menu, select type of list you want to publish through the outgoing feed:

   • PAN-OS IP External Dynamic List: the feed outputs an IP address list to feed the Palo Alto Networks firewall with, so that the firewall blocks the included IP addresses.

   • PAN-OS Domain External Dynamic List: the feed outputs a domain name list to feed the Palo Alto Networks firewall with, so that the firewall blocks the included domain names.

   • PAN-OS URL External Dynamic List: the feed outputs a URL list to feed the Palo Alto Networks firewall with, so that the firewall blocks the included URLs.

4. Go to the Transport configuration section, and set the appropriate transport type options for the selected transport type for the PAN-OS External Dynamic List content.

Configure transport types

Amazon S3 push

The Amazon S3 push transport type for outgoing feeds publishes data in the supported content types to the specified location on the designated Amazon S3 bucket.

1. From the Transport type drop-down menu, select Amazon S3 push.

2. In the Secret key field, enter your Amazon S3 push secret key.
   Sign up to Amazon Web Services, and then create one or more accounts, as necessary, to use their S3 data storage service.
   The secret key is part of your authentication credentials to log in to and to access Amazon S3 services.

3. In the Access key field, enter your Amazon S3 push access key.
   Along with your secret key, the access key enables you to authenticate to access Amazon S3 services.

4. In the Bucket field, enter the name of the Amazon S3 bucket to use as a target location for the outgoing feed published content.
   Buckets are data containers in the S3 environment.
   Buckets are region-specific, and their names must comply with standard DNS naming conventions.
   The default format of the URL to access a bucket is https://${bucket_name}.s3-${aws-region}.amazonaws.com

5. In the Path field, enter the path to the target directory where the content published through the outgoing feed is stored, relative to the bucket root.
   Example: /intel/actors/hacktivis

6. Under Feed content, you can define the data source and the update strategy for the outgoing feed:

7. From the Datasets drop-down menu, select one or more existing datasets to use as sources to populate the outgoing feed content.
   For the feed not to be empty, at least one selected dataset should contain entities and observables in the same format as the configured content type for the feed.

8. From the Update strategy drop-down menu, select the preferred method to populate the outgoing feed with data before publishing it:

   • Replace: every time the outgoing feed runs, it fetches new and existing data — new and existing entities and observables included in the previous execution of the feed — to generate the content to publish through the feed.

FTP upload

The FTP upload transport type for outgoing feeds publishes data in the supported content types to the specified location on the designated FTP server.

1. From the Transport type drop-down menu, select FTP upload.

2. In the FTP server URL field, enter the target ftp:// location on the FTP server to upload the outgoing feed content to, so as to make it available for retrieval.
   Example: ftp://ftp.server.com/feeds/outgoing/folder

3. In the Username field, enter a valid user name to authenticate and be granted the necessary authorization to access the location of the outgoing feed content.

4. In the Password field, enter a valid password to authenticate and be granted the necessary authorization to access the location of the outgoing feed content.

5. Select the Include documents attached to entities checkbox to include in the outgoing feed content also any attachments to the published entities such as MS Word documents or PDF files.

6. Under Feed content, you can define the data source and the update strategyfor the outgoing feed:

7. From the Datasets drop-down menu, select one or more existing datasets to use as sources to populate the outgoing feed content.
   For the feed not to be empty, at least one selected dataset should contain entities and observables in the same format as the configured content type for the feed.

8. From the Update strategy drop-down menu, select the preferred method to populate the outgoing feed with data before publishing it:

   • Replace: every time the outgoing feed runs, it fetches new and existing data — new and existing entities and observables included in the previous execution of the feed — to generate the content to publish through the feed.

SFTP upload

The SFTP upload transport type for outgoing feeds publishes data in the supported content types to the specified location on the designated SFTP server.

1. From the Transport type drop-down menu, select SFTP upload.

2. In the SFTP server URL field, enter the target sftp:// location on the SFTP server to upload the outgoing feed content to, so as to make it available for retrieval.
   Format: sftp://${sftp_server}:${port}/${path_to_target_directory}
   Example: sftp://sftp.server.com:22/source-data/for-the-feed

3. In the Username field, enter a valid user name to authenticate and be granted the necessary authorization to access the location of the outgoing feed content.

4. In the Password field, enter a valid password to authenticate and be granted the necessary authorization to access the location of the outgoing feed content.

5. Select the Include documents attached to entities checkbox to include in the outgoing feed content also any attachments to the published entities such as MS Word documents or PDF files.

6. Select the Use SSH key checkbox to enable logging in through SSH to apply this security layer to the outgoing feed.

7. In the SSH private key field, paste the private SSH key you want to use to access the target location where the SFTP upload outgoing feed should publish content to.
   Example:

   -----BEGIN RSA PRIVATE KEY-----

   MIIEpQIBAAKCAQEA3Tz2mr7SZiAMfQyuvBjM9Oi..Z1BjP5CE/Wm/Rr500P

   RK+Lh9x5eJPo5CAZ3/ANBE0sTK0ZsDGMak2m1g7..3VHqIxFTz0Ta1d+NAj

   wnLe4nOb7/eEJbDPkk05ShhBrJGBKKxb8n104o/..PdzbFMIyNjJzBM2o5y

   5A13wiLitEO7nco2WfyYkQzaxCw0AwzlkVHiIyC..71pSzkv6sv+4IDMbT/

   XpCo8L6wTarzrywnQsh+etLD6FtTjYbbrvZ8RQM..Hg2qxraAV++HNBYmNW

   kbJ+q+rsJxQlaipn2M4lGuQJEfIxELFDyd3XpxP..Un/82NZNXlPmRIopXs

   2T91jiLZEUKQw+n73j26adTbteuEaPGSrTZxBLR..yssO0wWomUyILqVeti

   +PK+aXKwguI6bxLGZ3of0UH+mGsSl0mkp7kYZCm..OTQtfeRqP8rDSC7DgA

   kHc5ajYqh04AzNFaxjRo+M3IGICUaOdKnXd0Fda..QwfoaX4QlRTgLqb7AN

   ZTzM9WbmnYoXrx17kZlT3lsCgYEAm757XI3WJVj..WoLj1+v48WyoxZpcai

   uv9bT4Cj+lXRS+gdKHK+SH7J3x2CRHVS+WH/SVC..DxuybvebDoT0TkKiCj

   BWQaGzCaJqZa+POHK0klvS+9ln0/6k539p95tfX..X4TCzbVG6+gJiX0ysz

   Yfehn5MCgYEAkMiKuWHCsVyCab3RUf6XA9gd3qY..fCTIGtS1tR5PgFIV+G

   engiVoWc/hkj8SBHZz1n1xLN7KDf8ySU06MDggB..hJ+gXJKy+gf3mF5Kmj

   DtkpjGHQzPF6vOe907y5NQLvVFGXUq/FIJZxB8k..fJdHEm2M4=

   -----END RSA PRIVATE KEY-----

8. In the SSH key password field, enter your SSH key password, if your SSH key is password-protected.
   If your SSH key is not password-protected, you can leave this field empty.

9. Select the Host authentication mode checkbox to automatically add and save the new host name and the new host key to the local Paramiko HostKeys dictionary.

10. Under Feed content, you can define the data source and the update strategy for the outgoing feed:

11. From the Datasets drop-down menu, select one or more existing datasets to use as sources to populate the outgoing feed content.
    For the feed not to be empty, at least one selected dataset should contain entities and observables in the same format as the configured content type for the feed.

12. From the Update strategy drop-down menu, select the preferred method to populate the outgoing feed with data before publishing it:

    • Replace: every time the outgoing feed runs, it fetches new and existing data — new and existing entities and observables included in the previous execution of the feed — to generate the content to publish through the feed.

Mount point upload

The Mount point upload transport type for outgoing feeds publishes data in the supported content types to the specified location on a local or network unit.

1. From the Transport type drop-down menu, select Mount point upload.

2. In the Mount point path field, enter the path to the local or network unit to save the outgoing feed content to, so as to make it available for retrieval.
   Example: /media/feeds/outgoing/folder

3. Select the Include documents attached to entities checkbox to include in the outgoing feed content also any attachments to the published entities such as MS Word documents or PDF files.

   Explicitly whitelist mount point paths to make them accessible to incoming and to outgoing feeds.
   If you do not whitelist the mount point path an incoming or an outgoing feed should access to retrieve data for ingestion or for publication, the feed will not be able to fetch or to publish any content.

   The /etc/eclecticiq/platform_settings.py configuration file includes dedicated mount point whitelists for ingestion – incoming feeds – and for dissemination – outgoing feeds.

   settings.py (sourced from EIQ platform-backend)

   Author	Rutger Prins
   Commit	ab323b23ebb93fde6c62b124f6823579957bd1d5
   Timestamp	August, 27, 2021 08:57 AM
   Full path	eiq/platform/settings.py
   Title	Merge branch 'ext-commons-update-2.10.x' into 'release-2.10.x'
   Description	Extension Commons update 2.10.x See merge request engineering/platform-backend!6075

   # Directories that can be accessed from mount point feeds. POLL is for incoming

   # feeds, PUSH is for outgoing feeds. Example: ["/mnt/", "/media/"]

   MOUNT_POINT_POLL_ALLOWED_DIRECTORIES: Sequence[str] = []

   MOUNT_POINT_PUSH_ALLOWED_DIRECTORIES: Sequence[str] = []

   • MOUNT_POINT_POLL_ALLOWED_DIRECTORIES is a list of allowed mount point paths that incoming feeds can access to fetch data from.

   • MOUNT_POINT_PUSH_ALLOWED_DIRECTORIES is a list of allowed mount point paths that outgoing feeds can access to publish data to.

   To whitelist a mount point path:

   1. Open the configuration file:

      sudo vi /etc/eclecticiq/platform_settings.py

   2. Look for MOUNT_POINT_POLL_ALLOWED_DIRECTORIES to make network locations accessible to incoming feeds, or for for MOUNT_POINT_PUSH_ALLOWED_DIRECTORIES to make network locations accessible to outgoing feeds.
      Both parameters are lists that take valid directory paths as list elements.
      Each path in the list points to a location that incoming feeds can access to fetch the data to be ingested, or that outgoing feeds can access to publish the content of a feed run.
      Incoming and outgoing feeds can access files and directories inside the specified locations, based on the configured access rights of the available assets and resources.

   3. Add as many paths to each list as necessary, then save the file and exit.
      Example:

      # Whitelist specific dirs; specific file types; everything inside subdirs of a dir

      MOUNT_POINT_PUSH_ALLOWED_DIRECTORIES = [ "/mnt/", "/media/", "/media/data/" ]

4. Under Feed content, you can define the data source and the update strategy for the outgoing feed:

5. From the Datasets drop-down menu, select one or more existing datasets to use as sources to populate the outgoing feed content.
   For the feed not to be empty, at least one selected dataset should contain entities and observables in the same format as the configured content type for the feed.

6. From the Update strategy drop-down menu, select the preferred method to populate the outgoing feed with data before publishing it:

   • Replace: every time the outgoing feed runs, it fetches new and existing data — new and existing entities and observables included in the previous execution of the feed — to generate the content to publish through the feed.

TAXII inbox

The TAXII inbox transport type for outgoing feeds publishes data in the supported content types through the TAXII inbox service:

1. From the Transport type drop-down menu, select TAXII inbox.

2. In the Auto Discovery field, enter the URL pointing to a TAXII discovery service.
   Feed consumers can send a request to the discovery service to obtain a list of the available TAXII services they can access and poll for content updates.
   Example: http://hailataxii.com/taxii-discovery-service

3. In the Inbox service URL field, enter the URL pointing to the location of the TAXII data collections available through the TAXII inbox service.
   Example: https://example.com/taxii-inbox

4. In the Destination collection name field, enter an existing collection name as the target container for the outgoing feed data.
   Example: collection.Default

5. From the TAXII version drop-down menu, select the TAXII version your system supports:
   

   • 1.0

   • 1.1

6. In the EclecticIQ authentication URL field, enter the URL pointing to the EclecticIQ Platform instance, including the endpoint that takes the user name and password inputs to send them to the authentication mechanism.
   Example: https://${platform_host_name}/api/auth

7. Select the Basic authentication checkbox to fill out the required information, if the data source TAXII server requires basic authentication to access the corresponding TAXII services.
   Username: enter a valid user name to authenticate and be granted the necessary authorization to access the location of the outgoing feed content.
   Password: enter a valid password to authenticate and be granted the necessary authorization to access the location of the outgoing feed content.

8. Select the SSL certificate authentication checkbox to fill in the required information, if the TAXII server requires an SSL certificate to authenticate and to authorize access to the corresponding TAXII services.

9. In the SSL certificate field, copy-paste the content of a valid SSL certificate to authenticate.
   SSL certificate file format: .pem
   Example:
   
   

   -----BEGIN CERTIFICATE REQUEST-----

   MIICvDCCAaQCAQAwdzELMAkGA1UEBhMCVVMxDTALBgNVBAgMBFV0YWgxDzANBgNV

   BAcMBkxpbmRvbjEWMBQGA1UECgwNRGlnaUNlcnQgSW5jLjERMA8GA1UECwwIRGln

   aUNlcnQxHTAbBgNVBAMMFGV4YW1wbGUuZGlnaWNlcnQuY29tMIIBIjANBgkqhkiG

   9w0BAQEFAAOCAQ8AMIIBCgKCAQEA8+To7d+2kPWeBv/orU3LVbJwDrSQbeKamCmo

   wp5bqDxIwV20zqRb7APUOKYoVEFFOEQs6T6gImnIolhbiH6m4zgZ/CPvWBOkZc+c

   1Po2EmvBz+AD5sBdT5kzGQA6NbWyZGldxRthNLOs1efOhdnWFuhI162qmcflgpiI

   WDuwq4C9f+YkeJhNn9dF5+owm8cOQmDrV8NNdiTqin8q3qYAHHJRW28glJUCZkTZ

   wIaSR6crBQ8TbYNE0dc+Caa3DOIkz1EOsHWzTx+n0zKfqcbgXi4DJx+C1bjptYPR

   BPZL8DAeWuA8ebudVT44yEp82G96/Ggcf7F33xMxe0yc+Xa6owIDAQABoAAwDQYJ

   KoZIhvcNAQEFBQADggEBAB0kcrFccSmFDmxox0Ne01UIqSsDqHgL+XmHTXJwre6D

   hJSZwbvEtOK0G3+dr4Fs11WuUNt5qcLsx5a8uk4G6AKHMzuhLsJ7XZjgmQXGECpY

   Q4mC3yT3ZoCGpIXbw+iP3lmEEXgaQL0Tx5LFl/okKbKYwIqNiyKWOMj7ZR/wxWg/

   ZDGRs55xuoeLDJ/ZRFf9bI+IaCUd1YrfYcHIl3G87Av+r49YVwqRDT0VDV7uLgqn

   29XI1PpVUNCPQGn9p/eX6Qo7vpDaPybRtA2R7XLKjQaF9oXWeCUqy1hvJac9QFO2

   97Ob1alpHPoZ7mWiEuJwjBPii6a9M9G30nUo39lBi1w=

   -----END CERTIFICATE REQUEST-----

10. In the SSL key field, copy-paste the content of a valid SSL key to authenticate.
    SSL key file format: .pem
    Example:
    
    

    -----BEGIN RSA PRIVATE KEY-----

    MIIEpQIBAAKCAQEA3Tz2mr7SZiAMfQyuvBjM9Oi..Z1BjP5CE/Wm/Rr500P

    RK+Lh9x5eJPo5CAZ3/ANBE0sTK0ZsDGMak2m1g7..3VHqIxFTz0Ta1d+NAj

    wnLe4nOb7/eEJbDPkk05ShhBrJGBKKxb8n104o/..PdzbFMIyNjJzBM2o5y

    5A13wiLitEO7nco2WfyYkQzaxCw0AwzlkVHiIyC..71pSzkv6sv+4IDMbT/

    XpCo8L6wTarzrywnQsh+etLD6FtTjYbbrvZ8RQM..Hg2qxraAV++HNBYmNW

    kbJ+q+rsJxQlaipn2M4lGuQJEfIxELFDyd3XpxP..Un/82NZNXlPmRIopXs

    2T91jiLZEUKQw+n73j26adTbteuEaPGSrTZxBLR..yssO0wWomUyILqVeti

    +PK+aXKwguI6bxLGZ3of0UH+mGsSl0mkp7kYZCm..OTQtfeRqP8rDSC7DgA

    kHc5ajYqh04AzNFaxjRo+M3IGICUaOdKnXd0Fda..QwfoaX4QlRTgLqb7AN

    ZTzM9WbmnYoXrx17kZlT3lsCgYEAm757XI3WJVj..WoLj1+v48WyoxZpcai

    uv9bT4Cj+lXRS+gdKHK+SH7J3x2CRHVS+WH/SVC..DxuybvebDoT0TkKiCj

    BWQaGzCaJqZa+POHK0klvS+9ln0/6k539p95tfX..X4TCzbVG6+gJiX0ysz

    Yfehn5MCgYEAkMiKuWHCsVyCab3RUf6XA9gd3qY..fCTIGtS1tR5PgFIV+G

    engiVoWc/hkj8SBHZz1n1xLN7KDf8ySU06MDggB..hJ+gXJKy+gf3mF5Kmj

    DtkpjGHQzPF6vOe907y5NQLvVFGXUq/FIJZxB8k..fJdHEm2M4=

    -----END RSA PRIVATE KEY-----

11. In the SSL key password field, enter the SSL password or passphrase for the SSL key.
    This field is masked.

12. Select the SSL verification checkbox, if the TAXII server requires an SSL certificate to authenticate and to access its TAXII services, to verify that it works as expected.

13. In the SSL CA bundle file path field, enter the path to the CA bundle file containing the root, intermediate, and public certificates for SSL authentication.
    The SSL CA bundle specified here is part of the server certificate validation chain.
    SSL CA bundle file format: .ca-bundle

14. To store your changes, click Save; to discard them, click Cancel.

15. Under Feed content you can define the data source and the update strategy for the outgoing feed:

16. From the Datasets drop-down menu, select one or more existing datasets to use as sources to populate the outgoing feed content.
    For the feed not to be empty, at least one selected dataset should contain entities and observables in the same format as the configured content type for the feed.

17. From the Update strategy drop-down menu, select the preferred method to populate the outgoing feed with data before publishing it:

    • Replace: every time the outgoing feed runs, it fetches new and existing data — new and existing entities and observables included in the previous execution of the feed — to generate the content to publish through the feed.

TAXII poll

The TAXII poll transport type for outgoing feeds publishes data in the supported content types through the TAXII poll service:

1. From the Transport type drop-down menu, select TAXII poll.

2. Select the Public checkbox to make the outgoing feed available to all platform groups and to all platform users.
   Leave it deselected to make the outgoing feed available only to specific groups.
   You can select the intended recipient groups in Authorized groups.
   Default value: deselected

3. From the Authorized groups drop-down menu, select one or more groups to grant them access to the feed.
   This option restricts access to the outgoing feed only to the selected user groups and to their members.|
   Authorized groups is only available when Public is deselected (default setting).

4. In the Collection name field, enter the name of the TAXII data collection you want to use to consolidate the outgoing feed content.
   The data collection name can be max. 1024 characters long, and its XML schema should comply with the xsd:anyURI data type.
   Example: MalwareDomainList_Hostlist
   
   

   Before deleting a group, check that is not an authorized group in an incoming or an outgoing feed configuration.
   Deleting a group that is currently selected as an authorized group to access an incoming or an outgoing feed content breaks feed functionality.

   If remove such a group:

   1. Remove it from the Authorized groups selection in the relevant incoming and/or outgoing feed(s).

   2. Proceed to delete the group.

   
   

5. Under Feed content, you can define the data source and the update strategy for the outgoing feed:

6. From the Datasets drop-down menu, select one or more existing datasets to use as sources to populate the outgoing feed content.
   For the feed not to be empty, at least one selected dataset should contain entities and observables in the same format as the configured content type for the feed.

7. From the Update strategy drop-down menu, select the preferred method to populate the outgoing feed with data before publishing it:

   • Replace: every time the outgoing feed runs, it fetches new and existing data — new and existing entities and observables included in the previous execution of the feed — to generate the content to publish through the feed.

8. To store your changes, click Save; to discard them, click Cancel.

About consuming the feed content

The PAN-OS External Dynamic List outgoing feed sends the Palo Alto Networks firewall blocklists containing IP domain, and URL sightings.
In a typical scenario, you want the platform to send the firewall only malicious IPs, domains, and URLs that are flagged by reliable sources because the firewall will block them.

It is a good idea to create a dataset in the platform that acts as a data source for the outgoing fee:

• If you create a static dataset, make sure you populate it only with entities having appropriate characteristics.

• If you create a dynamic dataset, make sure the search query acting as the filtering rule for the dynamic dataset content is set up to populate it with the desired entities.

In the PAN-OS External Dynamic List outgoing feed configuration, make sure you set the appropriate filters for observable maliciousness under Allowed observable states, TLP code, and source reliability.
For example:

• Set Allowed observable states to include only malicious ones.
  Do not include any observables marked as Safe or Ignore.

• Set the source reliability to the desired level.
  For example, you may want to exclude data from unreliable sources, as the firewall may automatically apply a blocking policy to it.

• Set the TLP code based on the applicable content sharing policies in your organization.

• Set the outgoing feed Update strategy to Replace, so that each feed run replaces the existing extended dynamic list content with both existing and new data since the previous feed run.
  Extended dynamic lists are not based on incremental changes: each time the firewall reloads them, previous block rules based on the same list are dropped, and new ones are created.
  Therefore, you always want each feed output (that is, each published list) to include all the available content up to that moment.

About external dynamic lists

Palo Alto External Dynamic Lists support IP addresses, domains, and URLs.

The external dynamic lists the platform published through the PAN-OS External Dynamic List outgoing feed support these data types: