Update strategy#

Update strategies allow you to configure how an outgoing feed decides to package data each time it runs.

An outgoing feed packs data from the datasets it is configured to retrieve data from. Each time the outgoing feed runs, it checks if records have been added or removed from those datasets. An update strategy determines which of these records are packed each time it runs.

Note

This section describes generic behavior for update strategies.

If a specific outgoing feed states different behavior for update strategies, it means that it has implemented that specific behavior. Follow that documentation instead.

Available update strategies:

Append#

Pack only new and modified objects in datasets since the last time this feed was run.

Does not remove or revoke data from where the feed publishes to.

Replace#

Note

Not recommended. Can cause high resource use on EclecticIQ Intelligence Center and at publishing destination, and high volume of network traffic.

Pack all objects in datasets regardless of the last feed run.

Usually used for testing outgoing feeds, or when you want to replace all data at the feed publishing destination.

Data removal: Always republish all records from connected datasets. If you remove a record from a dataset, feeds using the replace update strategy always removes all previously published data from the feed destination and publishes all data from the current dataset(s).

This means that any records removed from those datasets are also removed from the destination.

Diff#

Note

Usually has behavior specific to content type or transport type. See respective feed documentation. See Integrations documentation.

Outgoing feeds using the diff update strategy always publishes:

  • Removals for records removed from connected dataset(s) since the last feed run.

  • New records from connected dataset(s) since the last feed run.

For example: Outgoing feeds using the EclecticIQ Entities CSV or EclecticIQ Observables CSV content types produces a CSV file with a diff column. This has 2 possible values:

  • del: Records removed from connected dataset(s) since the last feed run.

  • add: Records added to the connected dataset(s) since the last feed run.

This update strategies is most commonly used with the following outgoing feed content types:

  • EclecticIQ Entities CSV

  • EclecticIQ Observables CSV

Known issues for each of these content types:

  • Known issues for EclecticIQ Entities CSV

    • Modifying directly related observables always produces a new add record in the packed CSV.

    • Modifying an entity produces 2 records: a del record to remove the previous version of the entity, and an add record for the updated version of the entity.

  • known issues for EclecticIQ Observables CSV

    • If there are multiple entities related to the same observable, only one instance of entity + observable is packed, with the observable value as the anchoring value.

      • Which entity + observable combination gets packed is at random.

      • To make sure that a given entity + observable combination is packed, make sure that the entity + observable combination is unique in your dataset(s).

    • Modifications to entities and observables in the dataset(s) do not cause the feed to produce a package for those changes. Only added and removed data produce new feed packages.