Search query syntax#

Basic operators and syntax#

Search fields in EclecticIQ Intelligence Center use the Elasticsearch DSL query syntax unless otherwise specified.

Query syntax element

Description

AND

Boolean operator between filters.

NOT

Boolean operator between filters.

OR

Boolean operator between filters.

"we want to look up all these keywords"

Use double quotes to group multiple search entries into one single search string.

The search query looks for matches for the string as one token.

keywo*

Searches for words containing criteria.

Use the * character for multiple character wildcards.

key?ord

Searches for words containing criteria.

Use the ? character for single character wildcards.

keyword~

Searches for similar words. Fuzzy search.

"keyword1"^2 AND keyword2

Use the ^${integer} suffix to give more weight to a search term over another in a search query.

+keyword

Prefix + to include a keyword in a search (must be in the search).

-keyword

Prefix - to exclude a keyword from a search (must not be in the search).

/keyw?rd)/

Searches using Elasticsearch DSL query syntax regular expressions.

Wildcards#

  • Currently supported wildcards: * and ?

  • Escape character: \

  • Avoid starting a wildcard query term with * or ?, as it may result in an extremely slow-running query.

    Search and rules use literal values and regex patterns to look for matching values assigned to the corresponding JSON path fields.

Example

Use Boolean operators and wildcards to create more refined searches.

enrichment_extracts.kind:domain AND enrichment_extracts.meta.classification:high

Search for entities#

Search for entiites by going to Search icon Search > Go to search and browse > Observables.

Tip

You can search for entities by querying the observables they are connected to. See the extracts or meta.bundled_extracts keywords in the Entity attributes table below.

Entity attributes#

The table below lists common attributes you can use to search for entities.

Tip

There are more granular attributes you can search with. Use the search bar’s autocomplete to help you find attributes to use, see individual entity documentation for more information.

Query key

Looks for

Query value examples

created_at

Entities whose creation date matches the specified pattern or literal.

[now-1w TO *], [now-24h TO *], [now-1w TO *], [now-1y TO *], [* TO *]

created_by

Entities whose user ID integer value matches the specified pattern or literal.

*, 1

data.confidence.value

Entities whose observable maliciousness confidence value matches the specified pattern or literal.

high, medium, low, none, unknown

data.kill_chain_phases.kill_chain_name

Entities whose kill chain phase name matches the specified pattern or literal.

reconnaissance, weaponization, delivery, exploitation, installation, command and control, actions on objectives

data.kill_chain_phases.name

Entities whose official or standard name matches the specified pattern or literal.

LMCO Kill Chain

data.kill_chain_phases.ordinality

Entities whose integer order value matches the specified pattern or literal.

This value defines the order of a kill chain phase within a kill chain.

1, 2, 3, 4, 5, 6, 7

data.observable.title

Entities with at least an observable whose title/header matches the specified pattern or literal.

Mirai botnet-related observable

data.producer.identity.name

Entities whose data producer name matches the specified pattern or literal.

The Entity per producer gauge on the dashboard used this field to display the total amount of ingested entities, based on the corresponding producer.

phishtank, hailataxii

data.producer.time_produced

Entities whose creation time at the data producer matches the specified pattern or literal.

Fractions of seconds are optional and may not always be included in the time value.

Input format: yyyy-MM-ddTHH:mm:ssZ

Description: year, month, day, hour, T (the date and time separator), minutes, seconds, time zone

2016-11-08T05\:04\:12Z, 2016-11-08T05\:04\:12\+00\:00, [now-24M/M TO 2016-01-01]

data.producer.time_received

Entities whose reception time at the data producer matches the specified pattern or literal.

Fractions of seconds are optional and may not always be included in the time value.

Input format: yyyy-MM-ddTHH:mm:ssZ

Description: year, month, day, hour, T (the date and time separator), minutes, seconds, time zone

2015-03-26T14\:28\:24Z, 2015-03-26T14\:28\:24\+00\:00, [now-24M/M TO 2016-01-01]

data.sightings_count

Entities with at least an observable that has actually been sighted.

This is a counter whose integer value reports the number of sightings recorded for the corresponding observable.

*, 1, 2, 3

data.timestamp

Entities whose data creation time matches the specified pattern or literal.

Fractions of seconds are optional and may not always be included in the time value.

Input format: yyyy-MM-ddTHH:mm:ssZ

Description: year, month, day, hour, T (the date and time separator), minutes, seconds, time zone

2015-03-26T14\:28\:24Z, 2015-03-26T14\:28\:24\+00\:00, [now-24M/M TO 2016-01-01]

data.type

Entity type. See Table of possible entity types.

indicator

exposure.sighted

Entities with at least an observable that has actually been sighted.

When entities are associated with a sighting, they are exposed.

Entities with exposure.sighted:true have a data.sightings_count value of at least 1.

true, false

meta.source_reliability

Entities whose data source reliability matches the specified pattern or literal.

A, (A B C)

meta.tlp_color

Entities whose TLP color matches the specified pattern or literal.

RED, AMBER, GREEN, WHITE, NONE

meta.tags

Entities whose custom tag values match the specified pattern or literal.

malware, ransomware

tags

Entities whose custom tag and standard taxonomy values match the specified pattern or literal.

malware, ransomware

Table of possible entity types:

Entity type

JSON value

Campaign

campaign

Course of Action

course-of-action

Exploit target

exploit-target

Sighting

eclecticiq-sighting

Incident

incident

Indicator

indicator

Report

report

Threat actor

threat-actor

Tactics, Techniques, Procedures

ttp

Search entities by observables#

Entities can have embedded observables. These are usually entities and observables ingested through incoming feeds.

You can search for entities that are directly related to or embed certain observables by using the extracts or enrichment_extracts keyword.

See Search for observables for a list of attributes to use with these keywords.

E.g. To search for entities containing an extract with an ID of 4:

extracts.id: 4

Search entities by outgoing feeds#

Outgoing feeds are indexed as destinations. Search for entities and observables published through an outgoing feed based on the feed name, type, or ID.

Query key

Description

destinations.name

Outgoing feed name.

Example: destinations.name: "Poll TAXII Stand"

Note

destinations.name is only set when an entity is packed for an outgoing feed. It is not updated when the name of that outgoing feed is changed, only when it is re-packed. This can result in entities not carrying the updated name of the outgoing feed it belongs to.

destinations.type

Outgoing feed transport type.

Example: `destinations.type: “TAXII Poll”

destinations.id

(Deprecated. Search using destinations.name instead.)

UUID of outgoing feed. To find the UUID of an outgoing feed, make a request to the /outgoing-feeds Public API endpoint.

Tip

Outgoing feeds have two IDs:

  • a numeric ID.

  • a destination ID, which is a UUID and is only available through a GET /outgoing-feeds request.

To find the UUID of an outgoing feed:

  1. Find the numeric ID of the feed.

    1. Go to Data configuration icon Data configuration > Outgoing feeds and select the feed to open it.

    2. Inspect the URL in your browser’s address bar. The numeric ID of the feed is the value of the detail parameter.

      E.g. In https://ic-playground.eclecticiq.com/main/configuration/outgoing-feeds?tab=detail&detail=5 the feed numeric ID is 5

  2. Make a REST API request to retrieve the feed IDL

    1. Make sure you have an API token.

    2. Make a request to /outgoing-feeds/{id}, where {id} is the numeric ID of the outgoing feed. E.g.:

      curl -H "Authorization: Bearer $APITOKEN" \
        https://ic-playground.eclecticiq.com/api/v2/outgoing-feeds/5
      
    3. Inspect the response. The UUID of the feed is the value of data.destination

Search for observables#

Search for observables by going to Search icon Search > Go to search and browse > Observables.

Observables are indexed and searchable using the extracts and enrichment_extracts keywords.

Tip

enrichment_extracts are extracts created as a result of running an enricher.

When searching for observables in Search icon Search > Go to search and browse > Observables, you can omit the extracts keyword and search directly using attributes.

E.g. To search for an observable with an ID of 4:

id: 4

Observable attributes#

The table below lists common attributes you can use to search observables with.

Entity JSON path

Description

Example values

id

Unique ID of observable.

13

value

Entities with at least an observable whose value matches the specified pattern or literal.

The value attribute has tokens and keyword attributes you can use. See Search using tokenizers.

malware.win32.sample

kind

Entities with at least an observable whose data type matches the specified pattern or literal.

ipv4, name

meta.confidence

Confidence assigned to an observable. Used together with classification to determine maliciousness. See Observable maliciousness.

high, medium, low

meta.classification

Classification assigned to an observable. Used together with confidence to determine maliciousness. See Observable maliciousness.

good, bad, unknown

meta.blacklisted

Whether or not an observable has been ignored by an Observable rules.

true, false

Relations#

Note

Only available on EclecticIQ Intelligence Center when searching for entities. From the left navigation, select Search Search icon > Entities.

Search for entities that are related to other entities using the relationship query syntax. This allows you to enter two queries – <source_query> and <target_query> – and display entities from source_query that have a relationship to results of target_query.

# Relationship query syntax
?(<source_query>) -- (<target_query>)

Query key

Description

?

Begins a relation query.

(<source_query>)

Enclose a query within parentheses.

This query determines the initial set of entities to filter with the subsequent target_query.

Example: (data.type:indicator AND tags:["APT-X"])

--

Delimiter. Separates the source query from the target query.

Note

There is one space character before and after --.

  • Tick Do: ?(query) -- (query)

  • Cross Don’t: ?(query)--(query)

(<target_query>)

Enclose a query within parentheses.

This acts as a filter for the source_query. Only entities resulting from the source_query that have a relationship to the results of target_query will be displayed as search results.

Example: (data.type:campaign AND sources.name:FireEye AND ingest_time:[now-7d TO *])

Example:

The following relation query returns all indicator entity types tagged with APT-X that have a relationship with campaign entity types ingested from the FireEye data source, and that have been created in the course of the past week:

# Search by relationship example
?(data.type:indicator AND tags:["APT-X"])--(data.type:campaign AND sources.name:FireEye AND ingest_time:[now-7d TO *])

Limitations#

Currently, search by relationship has the following limitations:

  • It is available only in the search input field.

  • Relational search queries can return max 5000 matches.

  • Relational search queries look for relationships by analyzing the entities stored in EclecticIQ Intelligence Center.

    They can examine up to max 100 000 items for each entity in the query.

    If a relational search query reaches this upper limit for one or both entities in the query, it stops searching for the entity or entities whose cap is reached.

Search with date and time#

Use only ISO 8601 date-time formats:

  • YYYY-MM-DD

  • YYYY-MM-DDTHHZ

  • YYYY-MM-DDTHH:mmZ

  • YYYY-MM-DDTHH:mm:ssZ

As a complete example, 12:35 p.m. on 25 January 2020 in Bucharest, Romania would be expressed as 2020-01-25T12:35:00+0200

Entity date-time attributes#

There are several entity attributes that you can use to search by date and time. The most commonly used are listed below along with their explanations.

Entity attribute

Description

created_at

The date and time that an entity was created in EclecticIQ Platform.

last_updated_at

The date and time that an entity’s details were last edited in EclecticIQ Intelligence Center.

data.producer.time_produced

The date and time that an incident was created.

This field is specific to incidents.

You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).

data.reporter.time_produced

The date and time that an incident was reported.

This field is specific to incidents.

You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).

data.time_first_malicious_action

The date and time of the first malicious of an incident.

This field is specific to incidents.

You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).

data.timestamp

A specific version of an entity.

data.valid_time_positions.start

The start of the time window during which an indicator is valid.

This field is specific to indicators.

You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).

data.valid_time_positions.end

The end of the time window during which an indicator is valid.

This field is specific to indicators.

You have to type the date value as a full string (for example: 2020-01-25T12:35:00+0200).

meta.first_ingest_time

The date and time of an entity’s most recent ingestion into an organization’s cluster of platforms.

You can only search using a single value.

meta.ingest_time

The date and time of an entity’s ingestion into a particular platform.

You can search using a single value or a range of values.

meta.estimated_observed_time

The date and time when an entity is estimated to have been first observed.

You can search using a single value or a range of values.

meta.estimated_threat_start_time

The date and time when the threat posed by an entity is estimated to have started.

You can search using a single value or a range of values.

meta.estimated_threat_end_time

The date and time when the threat posed by an entity is estimated to have ended

You can search using a single value or a range of values.

Date ranges#

Elasticsearch allows you to specify date ranges in your queries. Please be aware however that Elasticsearch treats queries using single dates differently to queries using date ranges. In the case of single dates, Elasticsearch only compares date values; it ignores time values and timezones. That’s why if you use ranges in your search queries you are more likely to get the results you expect.

Tip

If you use single date values in your search queries, make sure that the timezone in your user profile is the same as your system timezone, which is also the timezone of all the entities and observables in your system.

To specify ranges, enclose them either square or curly brackets:

  • [<min_datetime> TO <max_datetime>]

  • {<min_datetime> TO <max_datetime>}

Note

The TO operator must be written in uppercase.

Examples:

Query string

Description

meta.ingest_time:["2020-01-01T00:00:00" TO *]

Returns all entities that were ingested on and after 1 January 2020

meta.ingest_time:{"2020-01-01T00:00:00" TO  "2020-01-04T00:00:00"]

Returns all entities that were ingested on 2 and 3 January

meta.ingest_time:["2020-01-01T13" TO "2020-01-01T14"}

Returns all entities that were ingested on 1 January 2020 from 13:00 up to but excluding 14:00

You can also use the following shortened syntax:

Query string

Description

meta.ingest_time:[now/d TO *]

Returns all entities that were ingested today.

meta.ingest_time:[now/M TO *]

Returns all entities that were ingested this month.

meta.ingest_time:[now-24h TO *]

Returns all entities that were ingested during last 24 hours

User timezone#

The timezone you specified in your platform profile affects the value displayed for the timestamp of, for example, an entity, observable or relationship. It also serves as the default timezone for Elasticsearch queries.

If you specify a timezone in your query, Elasticsearch ignores your user timezone for the actual search. However, the search results are displayed according to your user timezone.

For example:

  • User timezone: CET (+0100)

  • Entity ingestion date-time: 21 Feb 2020 at 7:00 a.m. GMT (+0000)

  • Search string: meta.ingest_time:["2020-02-21T06:30:00+0000" TO "2020-02-21T07:30:00+0000"]

The entity appears in the search results, but because the user timezone is an hour later, the entity is displayed with a timestamp of 8:00 a.m.

This may be confusing; unless you need to share results with users in different timezones, it is preferable to avoid specifying a timezone in a search query.