Search | Query Syntax

Basic operators and syntax

Search fields in EclecticIQ Intelligence Center use the Elasticsearch DSL query syntax unless otherwise specified.

Query syntax element	Description
AND	Boolean operator between filters.
NOT	Boolean operator between filters.
OR	Boolean operator between filters.
"we want to look up all these keywords"	Use double quotes to group multiple search entries into one single search string. The search query looks for matches for the string as one token.
keywo*	Searches for words containing criteria. Use the * character for multiple character wildcards.
key?ord	Searches for words containing criteria. Use the ? character for single character wildcards.
keyword~	Searches for similar words. Fuzzy search.
"keyword1"^2 AND keyword2	Use the ^${integer} suffix to give more weight to a search term over another in a search query.
+keyword	Prefix + to include a keyword in a search (must be in the search).
-keyword	Prefix - to exclude a keyword from a search (must not be in the search).
/keyw?rd)/	Searches using Elasticsearch DSL query syntax regular expressions.

Wildcards

• Currently supported wildcards: * and ?

• Escape character: \

• Avoid starting a wildcard query term with * or ?, as it may result in an extremely slow-running query.

  Search and rules use literal values and regex patterns to look for matching values assigned to the corresponding JSON path fields.

Example

Use Boolean operators and wildcards to create more refined searches.

enrichment_extracts.kind:domain AND enrichment_extracts.meta.classification:high

Search with regular expressions

The regex syntax this field accepts is the Elasticsearch regular expression syntax.

The main peculiarities of the Elasticsearch DSL query syntax are:

• Anchors – ^ and $ – are implied at the beginning and at the end of the regex.

  Do not include them in the regex pattern input.

• If you insert explicit anchor characters in the value input field, they are interpreted as literal values.

• Escape special characters:

  ( . ? + | { } [ ] ( ) " \ / )
  

  To escape a special character, prepend a backslash \ to it.

  Example: \{ \}

  Note

  At the moment, Elasticsearch regular expression syntax optional operators are not supported.

  Note

  Although the regex syntax used in the UI of the Intelligence Center is that of ElasticSearch, the regex syntax used by EclecticIQ Intelligence Center’s ingestion engine is Python regex. In the vast majority of cases this does not lead to conflicts, but if your discovery rule is not working the way you want it to, this may be the reason.

  Here are two differences that commonly cause problems:

  • ElasticSearch searches for whole strings, while Python searches everywhere in a string. For example,

    • Input: Cert-BUND

    • Pattern: ert

    • ES result:

    • Python: will match

  • ElasticSearch regex syntax requires you to leave out anchors, whereas Python regex syntax requires you to include them.

  If you are experiencing problems in this area, contact Customer Support for help in writing the discovery rule you want.

Search in specific JSON fields

You can also specify JSON paths to search for the corresponding values in specific JSON fields of an entity data structure.

To do so, in the search input field enter the JSON path to the field whose value you want to look up, a semicolon, and the literal or data pattern to look up.

Format:

json.path:${search_string}

Example:

data.types.value:"IP Watchlist"

Search for observable properties

For example, the following JSON paths point to JSON fields holding observable values:

Field	Description	Example
extracts.kind	Search for a specific observable type.	extracts.kind:ipv4
extracts.meta.classification	Search for observables based on classification : good bad unknown	extracts.meta.classification:bad
extracts.meta.confidence	Search for observables based on confidence : high medium low	extracts.meta.confidence:high
extracts.meta.blacklisted	Search for observables based on blacklist status: true false	extracts.meta.blacklisted:false
enrichment_extracts.kind	Search for a specific enrichment observable type.	enrichment_extracts.kind:ipv4
enrichment_extracts.meta.classification	Search for enrichment observables based on classification : good bad unknown	enrichment_extracts.meta.classification:unknown
enrichment_extracts.meta.confidence	Search for enrichment observables based on confidence : high medium low	enrichment_extracts.meta.confidence:medium
enrichment_extracts.meta.blacklisted	Search for enrichment observables based on blacklist status: true false	enrichment_extracts.meta.blacklisted:true

Search for entities

Search for entiites by going to Search > Go to search and browse > Observables.

Tip

You can search for entities by querying the observables they are connected to. See the extracts or meta.bundled_extracts keywords in the Entity attributes table below.

Entity attributes

The table below lists common attributes you can use to search for entities.

Tip

There are more granular attributes you can search with. Use the search bar’s autocomplete to help you find attributes to use, see individual entity documentation for more information.

Query key	Looks for	Query value examples
created_at	Entities whose creation date matches the specified pattern or literal.	[now-1w TO *], [now-24h TO *], [now-1w TO *], [now-1y TO *], [* TO *]
created_by	Entities whose user ID integer value matches the specified pattern or literal.	*, 1
data.confidence.value	Entities whose observable maliciousness confidence value matches the specified pattern or literal.	high, medium, low, none, unknown
data.kill_chain_phases.kill_chain_name	Entities whose kill chain phase name matches the specified pattern or literal.	reconnaissance, weaponization, delivery, exploitation, installation, command and control, actions on objectives
data.kill_chain_phases.name	Entities whose official or standard name matches the specified pattern or literal.	LMCO Kill Chain
data.kill_chain_phases.ordinality	Entities whose integer order value matches the specified pattern or literal. This value defines the order of a kill chain phase within a kill chain.	1, 2, 3, 4, 5, 6, 7
data.observable.title	Entities with at least an observable whose title/header matches the specified pattern or literal.	Mirai botnet-related observable
data.producer.identity.name	Entities whose data producer name matches the specified pattern or literal. The Entity per producer gauge on the dashboard used this field to display the total amount of ingested entities, based on the corresponding producer.	phishtank, hailataxii
data.producer.time_produced	Entities whose creation time at the data producer matches the specified pattern or literal. Fractions of seconds are optional and may not always be included in the time value. Input format: yyyy-MM-ddTHH:mm:ssZ Description: year, month, day, hour, T (the date and time separator), minutes, seconds, time zone	2016-11-08T05\:04\:12Z, 2016-11-08T05\:04\:12\+00\:00, [now-24M/M TO 2016-01-01]
data.producer.time_received	Entities whose reception time at the data producer matches the specified pattern or literal. Fractions of seconds are optional and may not always be included in the time value. Input format: yyyy-MM-ddTHH:mm:ssZ Description: year, month, day, hour, T (the date and time separator), minutes, seconds, time zone	2015-03-26T14\:28\:24Z, 2015-03-26T14\:28\:24\+00\:00, [now-24M/M TO 2016-01-01]
data.sightings_count	Entities with at least an observable that has actually been sighted. This is a counter whose integer value reports the number of sightings recorded for the corresponding observable.	*, 1, 2, 3
data.timestamp	Entities whose data creation time matches the specified pattern or literal. Fractions of seconds are optional and may not always be included in the time value. Input format: yyyy-MM-ddTHH:mm:ssZ Description: year, month, day, hour, T (the date and time separator), minutes, seconds, time zone	2015-03-26T14\:28\:24Z, 2015-03-26T14\:28\:24\+00\:00, [now-24M/M TO 2016-01-01]
data.type	Entity type. See Table of possible entity types.	indicator
exposure.sighted	Entities with at least an observable that has actually been sighted. When entities are associated with a sighting, they are exposed. Entities with exposure.sighted:true have a data.sightings_count value of at least 1.	true, false
meta.source_reliability	Entities whose data source reliability matches the specified pattern or literal.	A, (A B C)
meta.tlp_color	Entities whose TLP color matches the specified pattern or literal.	RED, AMBER, GREEN, WHITE, NONE
meta.tags	Entities whose custom tag values match the specified pattern or literal.	malware, ransomware
tags	Entities whose custom tag and standard taxonomy values match the specified pattern or literal.	malware, ransomware
_exists_:"attached_files"	When added, returns only Report entities with files attached. When excluded, returns Report entities without files attached and all other entities.	_exists_:"attached_files", !(_exists_:"attached_files")

Table of possible entity types:

Entity type	JSON value
Attack Pattern	attack-pattern
Campaign	campaign
Course of Action	course-of-action
Exploit target	exploit-target
Sighting	eclecticiq-sighting
Identity	Identity
Indicator	indicator
Incident	incident
Infrastructure	Infrastructure
Intrusion set	intrusion-set
Location	location
Malware	malware
Malware analysis	malware-analysis
Report	report
Threat actor	threat-actor
Tactics, Techniques, Procedures	ttp
Tool	tool

Search entities by observables

Entities can have embedded observables. These are usually entities and observables ingested through incoming feeds.

You can search for entities that are directly related to or embed certain observables by using the extracts or enrichment_extracts keyword.

See Search for observables for a list of attributes to use with these keywords.

E.g. To search for entities containing an extract with an ID of 4:

extracts.id: 4

Search entities by outgoing feeds

Outgoing feeds are indexed as destinations. Search for entities and observables published through an outgoing feed based on the feed name, type, or ID.

Query key	Description
destinations.name	Outgoing feed name. Example: destinations.name: "Poll TAXII Stand" Note destinations.name is only set when an entity is packed for an outgoing feed. It is not updated when the name of that outgoing feed is changed, only when it is re-packed. This can result in entities not carrying the updated name of the outgoing feed it belongs to.
destinations.type	Outgoing feed transport type. Example: `destinations.type: “TAXII Poll”
destinations.id	(Deprecated. Search using destinations.name instead.) UUID of outgoing feed. To find the UUID of an outgoing feed, make a request to the /outgoing-feeds Public API endpoint. Tip Outgoing feeds have two IDs: a numeric ID. a destination ID, which is a UUID and is only available through a GET /outgoing-feeds request. To find the UUID of an outgoing feed: Find the numeric ID of the feed. Go to Data configuration > Outgoing feeds and select the feed to open it. Inspect the URL in your browser’s address bar. The numeric ID of the feed is the value of the detail parameter. E.g. In https://ic-playground.eclecticiq.com/main/configuration/outgoing-feeds?tab=detail&detail=5 the feed numeric ID is 5 Make a REST API request to retrieve the feed IDL Make sure you have an API token. Make a request to /outgoing-feeds/{id}, where {id} is the numeric ID of the outgoing feed. E.g.: curl -H "Authorization: Bearer $APITOKEN" \ https://ic-playground.eclecticiq.com/api/v2/outgoing-feeds/5 Inspect the response. The UUID of the feed is the value of data.destination

Search for observables

Search for observables by going to Search > Go to search and browse > Observables.

Observables are indexed and searchable using the extracts and enrichment_extracts keywords.

Tip

enrichment_extracts are extracts created as a result of running an enricher.

When searching for observables in Search > Go to search and browse > Observables, you can omit the extracts keyword and search directly using attributes.

E.g. To search for an observable with an ID of 4:

id: 4

Observable attributes

The table below lists common attributes you can use to search observables with.

Entity JSON path	Description	Example values
id	Unique ID of observable.	13
value	Entities with at least an observable whose value matches the specified pattern or literal. The value attribute has tokens and keyword attributes you can use. See Search | Using Tokenizers.	malware.win32.sample
kind	Entities with at least an observable whose data type matches the specified pattern or literal.	ipv4, name
meta.confidence	Confidence assigned to an observable. Used together with classification to determine maliciousness. See Observable maliciousness.	high, medium, low
meta.classification	Classification assigned to an observable. Used together with confidence to determine maliciousness. See Observable maliciousness.	good, bad, unknown
meta.blacklisted	Whether or not an observable has been ignored by an Observable rules.	true, false

Observables by link name

Use link names to search for specific observables, based on the type of relationship they have with their parent entity.

The type of relationship between an observable and and entity adds context, and it can help understand the function of the observable within the broader threat landscape it belongs to.

Let’s assume that an analyst is investigating a threat scenario where a threat actor exploits the CVE-2017-8793 vulnerability to gain access to the targeted victim’s assets.

The analyst may want to search EclecticIQ Intelligence Center for any exploit target entities containing observables that are related to the parent exploit target because they represent a vulnerability.

Example query:

data.type:exploit-target AND \
extracts.kind:domain AND \
meta.bundled_extracts.link_types:vulnerability OR \
extracts.instance_meta.link_types:vulnerability

In the search query example:

• meta.bundled_extracts.link_types is the JSON path pointing to the JSON field in the entity data structure that holds the link name value defining the relationship between entities and the corresponding bundled observables.

• extracts.instance_meta.link_types is the JSON path pointing to the JSON field in the entity data structure that holds the link name value defining the relationship between entities and non-embedded observables.

• vulnerability is the link name value defining the the type of entity-observable relationship you are looking for.

If the link name value search string contains multiple words separated by spaces, wrap the search string in double quotes (example: "my multiple word search string").

The following table maps the link name values you can enter in a search query to the corresponding options displayed in the GUI (campaign entities have no link names to define relationships with observables):

Search input value	GUI option	Linked to entity
parameter	Parameter	Course of action
affected	Affected	Exploit target
configuration	Configuration	Exploit target
vulnerability	Vulnerability	Exploit target
weakness	Weakness	Exploit target
affected-asset	Affected asset	Incident
related	Related	Incident
observed	Observable	Indicator
sighted	Sighted	Indicator
test-mechanism	Test mechanism	Indicator
malicious-infrastructure	Malicious infrastructure	TTP
targeted-victim	Targeted victim	TTP
observable	Observable	Report
identity	Identity	Threat actor

Relationships

From release 3.3.0 onward, use the relational query UI to search entity relationships.

Search with date and time

Use only ISO 8601 date-time formats:

• YYYY-MM-DD

• YYYY-MM-DDTHHZ

• YYYY-MM-DDTHH:mmZ

• YYYY-MM-DDTHH:mm:ssZ

As a complete example, 12:35 p.m. on 25 January 2020 in Bucharest, Romania would be expressed as 2020-01-25T12:35:00+0200

Entity date-time attributes

There are several entity attributes that you can use to search by date and time. The most commonly used are listed below along with their explanations.

Entity attribute	Description
created_at	The date and time that an entity was created in EclecticIQ Platform.
last_updated_at	The date and time that an entity’s details were last edited in EclecticIQ Intelligence Center.
data.producer.time_produced	The date and time that an incident was created. This field is specific to incidents. You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).
data.reporter.time_produced	The date and time that an incident was reported. This field is specific to incidents. You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).
data.time_first_malicious_action	The date and time of the first malicious of an incident. This field is specific to incidents. You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).
data.timestamp	A specific version of an entity.
data.valid_time_positions.start	The start of the time window during which an indicator is valid. This field is specific to indicators. You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).
data.valid_time_positions.end	The end of the time window during which an indicator is valid. This field is specific to indicators. You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).
meta.first_ingest_time	The date and time of an entity’s most recent ingestion into an organization’s cluster of platforms. You can only search using a single value.
meta.ingest_time	The date and time of an entity’s ingestion into a particular platform. You can search using a single value or a range of values.
meta.estimated_observed_time	The date and time when an entity is estimated to have been first observed. You can search using a single value or a range of values.
meta.estimated_threat_start_time	The date and time when the threat posed by an entity is estimated to have started. You can search using a single value or a range of values.
meta.estimated_threat_end_time	The date and time when the threat posed by an entity is estimated to have ended You can search using a single value or a range of values.

Date ranges

Elasticsearch allows you to specify date ranges in your queries. Please be aware however that Elasticsearch treats queries using single dates differently to queries using date ranges. In the case of single dates, Elasticsearch only compares date values; it ignores time values and timezones. That’s why if you use ranges in your search queries you are more likely to get the results you expect.

Tip

If you use single date values in your search queries, make sure that the timezone in your user profile is the same as your system timezone, which is also the timezone of all the entities and observables in your system.

To specify ranges, enclose them either square or curly brackets:

• [<min_datetime> TO <max_datetime>]

• {<min_datetime> TO <max_datetime>}

Note

The TO operator must be written in uppercase.

Examples:

Query string	Description
meta.ingest_time:["2020-01-01T00:00:00" TO *]	Returns all entities that were ingested on and after 1 January 2020
meta.ingest_time:{"2020-01-01T00:00:00" TO  "2020-01-04T00:00:00"]	Returns all entities that were ingested on 2 and 3 January
meta.ingest_time:["2020-01-01T13" TO "2020-01-01T14"}	Returns all entities that were ingested on 1 January 2020 from 13:00 up to but excluding 14:00

You can also use the following shortened syntax:

Query string	Description
meta.ingest_time:[now/d TO *]	Returns all entities that were ingested today.
meta.ingest_time:[now/M TO *]	Returns all entities that were ingested this month.
meta.ingest_time:[now-24h TO *]	Returns all entities that were ingested during last 24 hours

User timezone

The timezone you specified in your platform profile affects the value displayed for the timestamp of, for example, an entity, observable or relationship. It also serves as the default timezone for Elasticsearch queries.

If you specify a timezone in your query, Elasticsearch ignores your user timezone for the actual search. However, the search results are displayed according to your user timezone.

For example:

• User timezone: CET (+0100)

• Entity ingestion date-time: 21 Feb 2020 at 7:00 a.m. GMT (+0000)

• Search string: meta.ingest_time:["2020-02-21T06:30:00+0000" TO "2020-02-21T07:30:00+0000"]

The entity appears in the search results, but because the user timezone is an hour later, the entity is displayed with a timestamp of 8:00 a.m.

This may be confusing; unless you need to share results with users in different timezones, it is preferable to avoid specifying a timezone in a search query.