Add an observable rule

Observable rules give you granular control over entity observables by automatically flagging them as malicious, safe, or irrelevant. This enables automating default reactions by triggering follow-up actions in other components of your prevention and detection toolchain.

When you ingest large quantities of data, you are likely to introduce noise that can clutter your database.
Noisy data can make analysis and research more time-consuming and labor-intensive. Wading through a large data soup that includes meaningful information, as well as unnecessary data that does not yield any relevant intelligence value can slow down analysts’ decision-making process, and it can make it more error-prone.
This has an impact, among others, on prevention and response timeliness.

Entity and observable rules are highly customizable to give you fine control over your data.
For example, you can create rules to target specific entities or observables from predefined data sources, and then automatically add them to a detection or prevention system, or mark them for exclusion to reduce data noise.

About observable rules

Observable rules enable automating default reactions by triggering follow-up actions in other components of your prevention/detection toolchain, based on automatically assigned ignore, safe, or malicious flags.

Observable rule filtering enables funneling observables to dedicated buckets for further processing.
You can create rule-driven automation tasks that use the observable rule output to instrument external systems or devices in the toolchain.

For example, you can set up observable rules to:

Add potentially malicious threats to a prevention and/or a detection system.
Exclude non-malicious observables that do not represent a potential threat for the organization.

Rules handle the flags, and they can initiate actions on observables — for example, by routing them to a prevention and/or a detection system, or by marking them as safe-to-be-ignored and filter them out to reduce unwanted data noise.

Create an observable rule

Required fields are marked with an asterisk ( * ).

To create a new observable rule:

In the side navigation bar click > Rules > Observable rule.
In the Create observable rule view, in the Rule name field, enter a name to identify the rule.
In the Description field enter a short description to clarify the purpose of the rule, and the type of data it applies to.
This is helpful when the amount of rules users create in the Intelligence Center grows over time. A short description provides context, and it is a reminder of the reasons why the rule is in place.
From the Action drop-down menu, select one of the following options:
- Ignore: all entity observables matching all the conditions defined under Criteria selection are ignored.
  If any observables are found that can be ignored, you can delete them in bulk from the Intelligence Center by selecting Delete all matching observables.
  It is advisable to review the specified observables before deleting them.
  
  If you manually create an entity in the entity editor, and add observables with a type or value that matches the criteria of an existing observable ignore rule, these observables may not be accessible after saving the entity.
- Mark as safe: all entity observables matching all the conditions defined under Criteria selection are flagged as safe, and therefore non-threatening.
- Mark as malicious: all entity observables matching all the conditions defined under Criteria selection are flagged as malicious.
  These are the ones you may want to examine.
  For example, by defining rules that trigger follow-up actions in external prevention/detection components, or by requesting further analysis on the potential threats.
  Setting a maliciousness confidence level makes it easier to triage and to prioritize threat severity.
  
  If an observable rule updates the maliciousness confidence level of an observable that belongs to an entity, the Intelligence Center produces a new minor version of the entity to incorporate the changes.
  
  If you select Mark as malicious, you can fine-tune the option by making a further distinction based on Confidence:
  - Malicious – Low confidence: based on the available intelligence, the threat the observable represents may or may not be malicious.
  - Malicious – Medium confidence: based on the available intelligence, the threat the observable represents is likely to be malicious.
  - Malicious – High confidence: based on the available intelligence, the threat the observable represents is malicious.
To enable the rule immediately after creating it, select the Enabled checkbox.

Add rule criteria

A valid rule must have a name, an action, and at least one rule criterion to define the logic of the rule.
You can select and configure these criteria under Criteria selection.

To add a new criterion:

Click Criteria.
From the Entity types > Types drop-down menu, select one or more entity types to apply the rule to.
This criterion applies the rule only to observables associated with the specified entity types.
To remove a selection, go to the item(s) you want to remove, and click the cross icon .
To remove all selections at once, click the cross icon next to the drop-down menu arrow in the input field.
Alternatively, click Unselect all options.
From the Observable types > Types
drop-down menu, select one or more observable types to apply the rule to.
For example, you may want apply the rule only to city names, threat actors, IP addresses, or telephone numbers.

From the Paths > Path drop-down menu, select one or more options to define which fields in the entity data structure you want to search for matching values in.

The available options map to corresponding JSON paths in the JSON data structure representing entities in the Intelligence Center.
JSON paths are relative to the top-level data field. The the top-level data field is the implicit root of a JSON path.
For example, to point to the title or to the description fields of an entity, specify title or description, instead of data.title or data.description.

Path defines the entity data structure location where you want to look for specific data values.
Path works together with Value matches, which enables specifying regexes or literals to define the data patterns the rule should use to retrieve the desired matching values from the fields defined in Path.

Path option	JSON path	Entity type
Information source, Identity	`information_source.identity.name`	All
Information source, References	`information_source.references[]`	All
Title	`title`	All
Affected assets, Properties affected	`affected_assets[].nature_of_security_effect_properties_affected`	Incident
Observables	`observable`	Indicator
Sightings	`sightings`	Indicator
Raw events	`raw_events`	Sightings
Security control, Identity	`security_control.identity.name`	Sightings
Security control, References	`security_control.references[]`	Sightings
Resources, Infrastructure	`resources.infrastructure.type`	TTP
Resources, Persona	`resources.persona`	TTP

You can select one or more predefined JSON paths from the available options, or you can enter custom, user-defined JSON paths by typing them in the input field.

If you manually type user-defined JSON paths, they must be valid JSON paths pointing to existing fields in the entity JSON structure.
If you enter invalid JSON paths or valid JSON paths pointing to non-existing fields, they are silently ignored.

To examine the JSON data structure of an entity:

Open the entity detail pane and click the JSON tab.

Alternatively:

In the open entity detail pane, click the menu icon , and from the drop-down menu select Export > JSON to download and save the entity in JSON format.

In the Value matches > Value match field, enter one or more literals, where supported, or regexes to define the data patterns the rule should use to retrieve the desired matching values from the fields defined in Path.
If you specify multiple values, enter one value per line.
- Currently supported wildcards: * and ?
- Escape character: \
- Avoid starting a wildcard query term with * or ? , as it may result in an extremely slow-running query.
  Search and rules use literal values and regex patterns to look for matching values assigned to the corresponding JSON path fields.
The regex syntax this field accepts is the Elasticsearch regular expression syntax.
The main peculiarities of the Elasticsearch DSL query syntax are:
- Anchors – ^ and $ – are implied at the beginning and at the end of the regex.
  Do not include them in the regex pattern input.
- If you insert explicit anchor characters in the value input field, they are interpreted as literal values.
- Escape special characters:
  
  ( . ? + | { } [ ] ( ) " \ / )
  
  To escape a special character, prepend a backslash \ to it.
  Example: \{ \}
  
  At the moment, Elasticsearch regular expression syntax optional operators are not supported.
  Although the regex syntax used in the UI of the Intelligence Center is that of ElasticSearch, the regex syntax used by the Intelligence Center's ingestion engine is Python regex. In the vast majority of cases this does not lead to conflicts, but if your discovery rule is not working the way you want it to, this may be the reason.
  
  Here are two differences that commonly cause problems:
  - ElasticSearch searches for whole strings, while Python searches everywhere in a string. For example,
    Input: Cert-BUND
    Pattern: ert
    ES result: <none>
    Python: will match
  - ElasticSearch regex syntax requires you to leave out anchors, whereas Python regex syntax requires you to include them.
  If you are experiencing problems in this area, contact Customer Support for help in writing the discovery rule you want.
From the Source > Source drop-down menu, select an incoming feed, an enricher, or a Intelligence Center user group to use as a data source for the rule.
Data sources can be existing incoming feeds and enrichers, as well as existing Intelligence Center user groups.
From the Link name filter > Link types drop-down menu, select one or more link name options to apply the rule only to observables with the specified link name values representing specific types of relationship between observables and their parent entities.

Setting link names to define relationships adds intelligence value by describing how entities and observables are related.
This information provides additional context, and it helps understand how a specific resource is used, or the purpose it serves for a potential attacker.
For example, it can clarify that an observable describes a vulnerability or a weakness related to its parent entity.

Therefore, observables with a Link name value are in general more relevant and more valuable than observables without a Link name value.

Link name options vary, based on the relationship the observable has with the specific entity type it belongs to.

This filter applies to bundled observables; that is, observables included inside entities.
It does not apply to enrichment observables.
Select the Include observables without a link name checkbox to apply the rule also to observables without a defined link name identifying the relationship type.
These observables may or may not have relationships with other entities or other observables.
In the former case, the relationships are undefined; therefore, they have lower intelligence value than link-named ones.

This filter applies to bundled observables; that is, observables included inside entities.
It does not apply to enrichment observables.
To store your changes, click Save; to discard them, click Cancel..