Discover entities#

Discovery helps you explore ingested intelligence. Use discovery filters and rule-based searches to retrieve specific cyber threat information.

Apply discovery filters to view specific entities#

The Discovery view returns a list with matching ingested entities, based on one or more rule-based search queries.

You can refine the results by applying one or more quick filters. They are available on the left-hand navigation sidebar:

On the top navigation bar click Discovery.
By default, quick filters are switched off. To toggle quick filter visibility click the filter icon.
On the side navigation bar click a filter group name to expand the corresponding sub-nodes:
1. Entity type: select one or more checkboxes to include in the filtered results only the specified entity types.
2. Source: select one or more checkboxes to include in the filtered results only the specified entity sources.
3. TLP: select one or more checkboxes to include in the filtered results only entities flagged with the specified TLP color codes.
4. Date: select a time interval to include in the filtered results only entities ingested between the specified start and end dates.
5. Reliability: select one or more checkboxes to include in the filtered results only entities with the specified level(s) of reliability.
6. Discovery rules: select one or more checkboxes to include in the filtered results only entities matching the specified rule criteria.
7. Dataset: select one or more checkboxes to include in the filtered results only entities belonging to the specified datasets.
  
  The Dataset filter is not available when the results do not include any entities belonging to at least one dataset.
You can stack and combine filters as you need.

For example, you can create a filter to retrieve only indicators ingested from Hailataxii in the first two weeks of last month, and whose reliability flag is either A (completely reliable) or B (usually reliable).

Apply discovery rules to retrieve specific entities#

The Discovery service is a rule-based feature looking for cyber threat information that satisfies specific search criteria. You define the search criteria in a search query.

The query sets the scope for the discovery rule. If you want, you can further restrict the discovery rule context by selecting one or more workspaces and/or workspace types.

Query task execution is capped: the response can return max. 500 matches.

In EclecticIQ Intelligence Center discovery rules work like configurable, specialized intelligence agents:

Configurable because you can define discovery rules as necessary.
Specialized because the rules use search queries to focus on a specific search scope.

When you execute a discovery rule for the first time, it runs incrementally as a provider: the first run returns matching data, up to a maximum of 500 entities, since the beginning of time; that is, there is no start time setting to limit the discovery scope to a specific starting point in the past.

Following runs execute the specified query starting from the previous successful run, and they discover only entities added since the previous successful execution of the same rule. Repeated runs return all discovered entities since the previous successful execution of the same query.

To run a discovery task without this temporal constraint, create a new discovery rule.

Editing a rule does not affect this behavior. If you want a query in an existing discovery rule to search all available data since the beginning of time, create a new rule, and then run it for the first time.

You can also edit a discovery rule, and then click Save and re-run for all time.

This option saves any changes, resets the execution time counter, and then it runs the rule task without applying any time constraint.

The run returns matching data for the rule, up to a maximum of 500 results, since the beginning of time; that is, there is no start time setting to limit the discovery scope to a specific starting point in the past.

Note

When a rule is active, it automatically runs every 15 minutes.
Query task execution is capped: the response can return max. 500 matches.
Discovery search queries use the Elasticsearch query syntax.

View discovery rules#

To view a list of all saved discovery rules, go to Data configuration > Rules > Discovery from the left navigation bar.

You can sort the items on the view by column header. To do so, click the column header you want to base the data sorting on. An upward-pointing or a downward-pointing arrow in the header indicates ascending and descending sort order, respectively.

Create discovery rules#

Note

Required fields are marked with an asterisk (*).

To create a new discovery rule, do the following:

In the left navigation bar, go to Data configuration > Rules > Discovery.
Select Create rule +.
Fill out the Rules > Discovery > Create form with the necessary details to create the new rule:
1. Name: enter a name to describe the rule.
  
  It should be descriptive and easy to remember.
  
  Example: China or Russia, 1 year till now
2. Description: enter a short description to briefly explain what the rule does, its purpose, and the type of data it looks for.
  
  Example: Discovers any indicator data types having either “China” or “Russia” as a tag, and whose creation date falls in the range “one year ago until now”.
3. Search query: the search query you want to run when executing the rule. It should do what you explain in the rule description field.
  
  Search queries for discovery rules and rules in general use the Elasticsearch query syntax.
  
  Example: data.type:indicator OR entity.tags:China OR entity.tags:Russia AND created_at:[now-1y TO now]
4. Correlated workspaces: you can select one or more workspaces to focus the search only on those entities that are associated with the selected workspaces.
  
  To remove a selection from the input field, click the icon corresponding to the item(s) you want to remove.
  
  Example: IOCs originating in China and Russia
5. Correlated workspaces types: if you want, you can specify one or more workspace types to focus the search only on those entities that are related to all workspaces of a specific type.
  
  To remove a selection from the input field, click the icon corresponding to the item(s) you want to remove.
  
  Example: Topic
6. Enabled: select or deselect this checkbox to enable or disable the rule.
Click Save to store your changes, or Cancel to discard them.

Save options#

To access additional save options, click the down arrow on the Save button:

Click Save and new to save the current data or configuration for the item you are working on, and to create a new item of the same type right away.

For example, a new dataset, feed, policy, rule, task, or workspace.
Click Save and duplicate to save the current data for the item you are working on, and to create a new prepopulated copy of the same item, which you can use as a template or a blueprint to speed up repetitive manual work.

Edit discovery rules#

To edit a rule, do the following:

In the left navigation bar, go to Data configuration > Rules > Discovery.
On the rule overview, click the row corresponding to the rule you want to modify.
An overlay slides in from the side of the screen. It displays detailed rule information in a flash-card format.
On the rule detail view, select Actions > Edit.
On the Rules > Discovery > Edit form, you can change the field inputs as appropriate.
Click Save to store your changes, or Cancel to discard them.

Alternatively:

In the left navigation bar, go to Data configuration > Rules > Discovery.
On the rule overview, click the dotted menu icon on the row corresponding to the rule you want to modify.
On the Rules > Discovery > Edit form, you can change the field input as appropriate.
Click Save to store your changes, or Cancel to discard them.

Note

You can also edit a discovery rule, and then click Save and re-run for all time.

This option saves any changes, resets the execution time counter, and then it runs the rule task without applying any time constraint.

Delete discovery rules#

To delete a rule, do the following:

In the left navigation bar, go to Data configuration > Rules > Discovery.
On the rule overview, click the dotted menu icon on the row corresponding to the rule you want to delete.
From the pop-up context menu, select Delete.
On the confirmation pop-up dialog, click Delete to confirm the action.
The discovery rule is deleted.

Enable and disable discovery rules#

To manually enable and disable an existing rule, do the following:

In the left navigation bar, go to Data configuration > Rules > Discovery.
On the rule overview, click the row corresponding to the rule you want to run manually.
An overlay slides in from the side of the screen. It displays detailed rule information in a flash-card format.

On the Details tab you can enable and disable the rule.

If the rule is disabled:
1. Click Enable.
2. The button name changes to Enabled to notify that the rule is active.
3. A pop-up dialog asks you whether you want to run the rule right away.
  
  A notification message confirms enabling the rule.
If the rule is enabled:
1. Click Disable.
2. The button name changes to Disabled to notify that the rule is inactive.
  
  A notification message confirms disabling the rule.

Manually run discovery rules#

You can bypass automatic execution and decide to manually run a rule, for example to test it immediately after creating it.

To manually run a rule, do the following:

In the left navigation bar, go to Data configuration > Rules > Discovery.
On the rule overview, click the row corresponding to the rule you want to run manually.
An overlay slides in from the side of the screen. It displays detailed rule information in a flash-card format.
On the Details tab, either click the Run now button, or select the Actions > Run now menu option.

After completing the run, you can review the outcome on the Details tab:

Under the Status column you can check the execution outcome.

Status	Description
Started	The task run has been initiated, it has been added to the queue, and it is waiting to be executed.
Success	The task run completed correctly.
Error	The task run failed. Click the status icon to view an error message and a traceback with more details about the failure. This information can be helpful to troubleshoot the issue.

Under the Results column you can see whether the discovery action yielded any new results matching the rule criteria.