Search query syntax

Basic operators and syntax

Search fields in EclecticIQ Intelligence Center use the Elasticsearch DSL query syntax unless otherwise specified.

Query syntax element	Description
AND	Boolean operator between filters.
NOT	Boolean operator between filters.
OR	Boolean operator between filters.
"we want to look up all these keywords"	Use double quotes to group multiple search entries into one single search string. The search query looks for matches for the string as one token.
keywo*	Searches for words containing criteria. Use the * character for multiple character wildcards.
key?ord	Searches for words containing criteria. Use the ? character for single character wildcards.
keyword~	Searches for similar words. Fuzzy search.
"keyword1"^2 AND keyword2	Use the ^${integer} suffix to give more weight to a search term over another in a search query.
+keyword	Prefix + to include a keyword in a search (must be in the search).
-keyword	Prefix - to exclude a keyword from a search (must not be in the search).
/keyw?rd)/	Searches using Elasticsearch DSL query syntax regular expressions.

Wildcards

• Currently supported wildcards: * and ?

• Escape character: \

• Avoid starting a wildcard query term with * or ?, as it may result in an extremely slow-running query.

  Search and rules use literal values and regex patterns to look for matching values assigned to the corresponding JSON path fields.

Example

Use Boolean operators and wildcards to create more refined searches.

enrichment_extracts.kind:domain AND enrichment_extracts.meta.classification:high

Entity attributes

Query key	Looks for	Query value examples
created_at	Entities whose creation date matches the specified pattern or literal.	[now-1w TO *], [now-24h TO *], [now-1w TO *], [now-1y TO *], [* TO *]
created_by	Entities whose user ID integer value matches the specified pattern or literal.	*, 1
data.confidence.value	Entities whose observable maliciousness confidence value matches the specified pattern or literal.	high, medium, low, none, unknown
data.kill_chain_phases.kill_chain_name	Entities whose kill chain phase name matches the specified pattern or literal.	reconnaissance, weaponization, delivery, exploitation, installation, command and control, actions on objectives
data.kill_chain_phases.name	Entities whose official or standard name matches the specified pattern or literal.	LMCO Kill Chain
data.kill_chain_phases.ordinality	Entities whose integer order value matches the specified pattern or literal. This value defines the order of a kill chain phase within a kill chain.	1, 2, 3, 4, 5, 6, 7
data.observable.title	Entities with at least an observable whose title/header matches the specified pattern or literal.	Mirai botnet-related observable
data.producer.identity.name	Entities whose data producer name matches the specified pattern or literal. The Entity per producer gauge on the dashboard used this field to display the total amount of ingested entities, based on the corresponding producer.	phishtank, hailataxii
data.producer.time_produced	Entities whose creation time at the data producer matches the specified pattern or literal. Fractions of seconds are optional and may not always be included in the time value. Input format: yyyy-MM-ddTHH:mm:ssZ Description: year, month, day, hour, T (the date and time separator), minutes, seconds, time zone	2016-11-08T05\:04\:12Z, 2016-11-08T05\:04\:12\+00\:00, [now-24M/M TO 2016-01-01]
data.producer.time_received	Entities whose reception time at the data producer matches the specified pattern or literal. Fractions of seconds are optional and may not always be included in the time value. Input format: yyyy-MM-ddTHH:mm:ssZ Description: year, month, day, hour, T (the date and time separator), minutes, seconds, time zone	2015-03-26T14\:28\:24Z, 2015-03-26T14\:28\:24\+00\:00, [now-24M/M TO 2016-01-01]
data.sightings_count	Entities with at least an observable that has actually been sighted. This is a counter whose integer value reports the number of sightings recorded for the corresponding observable.	*, 1, 2, 3
data.timestamp	Entities whose data creation time matches the specified pattern or literal. Fractions of seconds are optional and may not always be included in the time value. Input format: yyyy-MM-ddTHH:mm:ssZ Description: year, month, day, hour, T (the date and time separator), minutes, seconds, time zone	2015-03-26T14\:28\:24Z, 2015-03-26T14\:28\:24\+00\:00, [now-24M/M TO 2016-01-01]
data.type	Entity type. See Table of possible entity types.	indicator
extracts.value	Entities with at least an observable whose value matches the specified pattern or literal.	malware.win32.sample
extracts.kind	Entities with at least an observable whose data type matches the specified pattern or literal.	ipv4, name
enrichment_extracts.value	Entities with at least an observable retrieved through enrichment whose value matches the specified pattern or literal.	www.w3.org
enrichment_extracts.kind	Entities with at least an observable retrieved through enrichment whose data type matches the specified pattern or literal.	domain, actor-id
exposure.sighted	Entities with at least an observable that has actually been sighted. When entities are associated with a sighting, they are exposed. Entities with exposure.sighted:true have a data.sightings_count value of at least 1.	true, false
meta.source_reliability	Entities whose data source reliability matches the specified pattern or literal.	A, (A B C)
meta.tlp_color	Entities whose TLP color matches the specified pattern or literal.	RED, AMBER, GREEN, WHITE, NONE
meta.tags	Entities whose custom tag values match the specified pattern or literal.	malware, ransomware
tags	Entities whose custom tag and standard taxonomy values match the specified pattern or literal.	malware, ransomware

Table of possible entity types:

Entity type	JSON value
Campaign	campaign
Course of Action	course-of-action
Exploit target	exploit-target
Sighting	eclecticiq-sighting
Incident	incident
Indicator	indicator
Report	report
Threat actor	threat-actor
Tactics, Techniques, Procedures	ttp

Observable attributes

Entity JSON path	Description
extracts.kind	Searches for types of observable.
enrichment_extracts.kind	Searches for types of enrichment observables.
extracts_nested.kind	Searches for types of observables embedded in parent entities.
meta.bundled_extracts.kind	Searches for types of observables bundled in an ingested package.
data.related_extracts.kind	Searches for observables with the specified type of relationship.

Observables (enrichment)

Field	Type	Description	Example
enrichment_extracts.id	String	The alphanumeric ID string that uniquely identifies the enrichment observable.	01h12x45-01q2-1234-od01-123456h78h90
enrichment_extracts.kind	String	The enrichment observable data type.	domain
enrichment_extracts.meta.blacklisted	Boolean	An observable is blacklisted when it is included in the results returned by an ignore extraction rule. Allowed values: true false	true
enrichment_extracts.meta.classification	String	This value is defined in Rules by selecting one of the available options under Action and Confidence. Allowed classification metadata values: good bad unknown	good
enrichment_extracts.meta.confidence	String	This value is defined in Rules by selecting the appropriate option under Action and Confidence. The selected action must be Mark as malicious for the Confidence drop-down list to become available. Allowed confidence metadata values: low medium high	high
enrichment_extracts.value	String	The actual value of the enrichment observable, based on the enrichment observable data type.	doom.dismay.biz

Observables by link name

Search observables by their link name to look up observed IOCs based on the function they have in a broader threat context, such as targeted victim, affected asset, malicious infrastructure, and so on.

You can use link names to search for specific observables, based on the type of relationship they have with their parent entity.

The type of relationship between an observable and and entity adds context, and it can help understand the function of the observable within the broader threat landscape it belongs to.

For example, a relationship can help identify an observable as a victim, and affected asset, a vulnerability, or as a component of the threat actor’s malicious infrastructure.

Let’s assume that an analyst is investigating a threat scenario where a threat actor exploits the CVE-2017-8793 vulnerability to gain access to the targeted victim’s assets.

The analyst may want to search EclecticIQ Intelligence Center for any exploit target entities containing observables that are related to the parent exploit target because they represent a vulnerability.

Example query:

data.type:exploit-target AND \
extracts.kind:domain AND \
meta.bundled_extracts.link_types:vulnerability OR \
extracts.instance_meta.link_types:vulnerability OR \
extracts_nested.instance_meta.link_types:vulnerability

In the search query example:

• meta.bundled_extracts.link_types is the JSON path pointing to the JSON field in the entity data structure that holds the link name value defining the relationship between entities and the corresponding bundled observables.

• extracts.instance_meta.link_types is the JSON path pointing to the JSON field in the entity data structure that holds the link name value defining the relationship between entities and non-embedded observables.

• extracts_nested.instance_meta.link_types is the JSON path pointing to the JSON field in the entity data structure that holds the link name value defining the relationship between entities and the corresponding embedded observables.

• vulnerability is the link name value defining the the type of entity-observable relationship you are looking for.

If the link name value search string contains multiple words separated by spaces, wrap the search string in double quotes (example: "my multiple word search string").

The following table maps the link name values you can enter in a search query to the corresponding options displayed in the GUI (campaign entities have no link names to define relationships with observables):

Search input value	GUI option	Linked to entity
parameter	Parameter	Course of action
affected	Affected	Exploit target
configuration	Configuration	Exploit target
vulnerability	Vulnerability	Exploit target
weakness	Weakness	Exploit target
affected-asset	Affected asset	Incident
related	Related	Incident
observed	Observable	Indicator
sighted	Sighted	Indicator
test-mechanism	Test mechanism	Indicator
malicious-infrastructure	Malicious infrastructure	TTP
targeted-victim	Targeted victim	TTP
observable	Observable	Report
identity	Identity	Threat actor

Outgoing feeds

Outgoing feeds are indexed as as destinations. Search for entities and observables published through an outgoing feed based on the feed name, type, or ID.

Query key	Description
destinations.name	Outgoing feed name. Example: destinations.name: "Poll TAXII Stand" Note destinations.name is set when an entity is packed for an outgoing feed. It is not updated when the name of that outgoing feed is changed, only when it is re-packed. This can result in entities not carrying the updated name of the outgoing feed it belongs to.
destinations.type	Outgoing feed transport type. Example: `destinations.type: “TAXII Poll”
destinations.id	UUID of outgoing feed. To find the UUID of an outgoing feed: Select an outgoing feed to open it. Inspect the URL in your browser’s address bar. It should display a URL like: https://ic-playground.eclecticiq.com/main/configuration/outgoing-feeds?detail=3&tab=detail The ID of this outgoing feed is the value of the detail query parameter: 3.

Relations

Note

Only available on EclecticIQ Intelligence Center. From the left navigation, select Search .

Search for entities with specific attributes that are connected with each other to explore hypotheses, and to validate cyber threat intelligence models as you build them during an investigation.

For example, you may want to explore if a threat actor is related to an incident or to a campaign to validate or to reject a hypothesis.

To search entities by relation, enter in search fields on EclecticIQ Intelligence Center:

# Relationship query syntax
?(<source_entity>)--(target_entity)

Query key	Description
?	Begins a relation query.
(<source_entity>)	Enclose a query within parentheses. This query should match one or more entities that will serve as the source entity in a relation query. Example: (data.type:indicator AND tags:["APT-X"])
--	Delimiter. Separates the source query from the target query.
(<target_entity>)	Enclose a query within parentheses. This query should match one or more entities that will serve as the source entity in a relation query. Example: (data.type:campaign AND sources.name:FireEye AND ingest_time:[now-7d TO *])

Example:

The following relation query returns all indicator entity types tagged with APT-X that have a relationship with campaign entity types ingested from the FireEye data source, and that have been created in the course of the past week:

# Search by relationship example
?(data.type:indicator AND tags:["APT-X"])--(data.type:campaign AND sources.name:FireEye AND ingest_time:[now-7d TO *])

Limitations

Currently, search by relationship has the following limitations:

• It is available only in the search input field.

• Relational search queries can return max 5000 matches.

• Relational search queries look for relationships by analyzing the entities stored in EclecticIQ Intelligence Center.

  They can examine up to max 100 000 items for each entity in the query.

  If a relational search query reaches this upper limit for one or both entities in the query, it stops searching for the entity or entities whose cap is reached.

Date and time

Use only ISO 8601 date-time formats:

• YYYY-MM-DD

• YYYY-MM-DDTHHZ

• YYYY-MM-DDTHH:mmZ

• YYYY-MM-DDTHH:mm:ssZ

As a complete example, 12:35 p.m. on 25 January 2020 in Bucharest, Romania would be expressed as 2020-01-25T12:35:00+0200

Entity date-time attributes

There are several entity attributes that you can use to search by date and time. The most commonly used are listed below along with their explanations.

Entity attribute	Description
created_at	The date and time that an entity was created in EclecticIQ Platform.
last_updated_at	The date and time that an entity’s details were last edited in EclecticIQ Intelligence Center.
data.producer.time_produced	The date and time that an incident was created. This field is specific to incidents. You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).
data.reporter.time_produced	The date and time that an incident was reported. This field is specific to incidents. You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).
data.time_first_malicious_action	The date and time of the first malicious of an incident. This field is specific to incidents. You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).
data.timestamp	A specific version of an entity.
data.valid_time_positions.start	The start of the time window during which an indicator is valid. This field is specific to indicators. You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).
data.valid_time_positions.end	The end of the time window during which an indicator is valid. This field is specific to indicators. You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).
meta.first_ingest_time	The date and time of an entity’s most recent ingestion into an organization’s cluster of platforms. You can only search using a single value.
meta.ingest_time	The date and time of an entity’s ingestion into a particular platform. You can search using a single value or a range of values.
meta.estimated_observed_time	The date and time when an entity is estimated to have been first observed. You can search using a single value or a range of values.
meta.estimated_threat_start_time	The date and time when the threat posed by an entity is estimated to have started. You can search using a single value or a range of values.
meta.estimated_threat_end_time	The date and time when the threat posed by an entity is estimated to have ended You can search using a single value or a range of values.

Date ranges

Elasticsearch allows you to specify date ranges in your queries. Please be aware however that Elasticsearch treats queries using single dates differently to queries using date ranges. In the case of single dates, Elasticsearch only compares date values; it ignores time values and timezones. That’s why if you use ranges in your search queries you are more likely to get the results you expect.

Tip

If you use single date values in your search queries, make sure that the timezone in your user profile is the same as your system timezone, which is also the timezone of all the entities and observables in your system.

To specify ranges, enclose them either square or curly brackets:

• [<min_datetime> TO <max_datetime>]

• {<min_datetime> TO <max_datetime>}

Note

The TO operator must be written in uppercase.

Examples:

Query string	Description
meta.ingest_time:["2020-01-01T00:00:00" TO *]	Returns all entities that were ingested on and after 1 January 2020
meta.ingest_time:{"2020-01-01T00:00:00" TO  "2020-01-04T00:00:00"]	Returns all entities that were ingested on 2 and 3 January
meta.ingest_time:["2020-01-01T13" TO "2020-01-01T14"}	Returns all entities that were ingested on 1 January 2020 from 13:00 up to but excluding 14:00

You can also use the following shortened syntax:

Query string	Description
meta.ingest_time:[now/d TO *]	Returns all entities that were ingested today.
meta.ingest_time:[now/M TO *]	Returns all entities that were ingested this month.
meta.ingest_time:[now-24h TO *]	Returns all entities that were ingested during last 24 hours

User timezone

The timezone you specified in your platform profile affects the value displayed for the timestamp of, for example, an entity, observable or relationship. It also serves as the default timezone for Elasticsearch queries.

If you specify a timezone in your query, Elasticsearch ignores your user timezone for the actual search. However, the search results are displayed according to your user timezone.

For example:

• User timezone: CET (+0100)

• Entity ingestion date-time: 21 Feb 2020 at 7:00 a.m. GMT (+0000)

• Search string: meta.ingest_time:["2020-02-21T06:30:00+0000" TO "2020-02-21T07:30:00+0000"]

The entity appears in the search results, but because the user timezone is an hour later, the entity is displayed with a timestamp of 8:00 a.m.

This may be confusing; unless you need to share results with users in different timezones, it is preferable to avoid specifying a timezone in a search query.