Remove eclecticiq-extension-commons from custom extensions for 3.4#

Overview #

These instructions are for developers of custom extensions who use the eclecticiq-extension-commons package in their custom-built extensions for EclecticIQ Intelligence Center 3.3 and older.

You must migrate your custom extensions before attempting to upgrade to EclecticIQ Intelligence Center 3.4.

If your custom extensions do not import modules containing the path eiq.extensions.commons, you are not affected..

Changes #

Tip

You can make and test these changes on EclecticIQ Intelligence Center 3.3.

The eclecticiq-extension-commons package is no longer provided for EclecticIQ Intelligence Center 3.4 and newer.

eclecticiq-extension-commons contains helper methods for creating and validating data before submitting them the EclecticIQ Intelligence Center for ingestion.

From EclecticIQ Intelligence Center 3.3 onward, developers should use the eclecticiq-platform-extensions-api package instead. Most functionality provided by the eclecticiq-extension-commons package has been moved to this package.

To do this, change import paths from:

from eiq.extensions.commons import (...)

to new import paths using:

from eiq_ext import (...)

You can make and test these changes now on EclecticIQ Intelligence Center 3.3.

The following sections cover these changes in detail.

Remove eclecticiq-extension-commons package from dependencies #

Remove the eclecticiq-extension-commons package from your custom extension’s list of dependencies.

For example, in the following setup.py file, remove eclecticiq-extension-commons from the install_requires list:

from setuptools import find_packages, setup

setup(
    name="eclecticiq-extension-example_extension",
    version="3.3.1",
    description="EXAMPLE_EXTENSION",
    url="https://www.eclecticiq.com/",
    author="EclecticIQ",
    author_email="[email protected]",
    license="Proprietary License",
    long_description="EXAMPLE_EXTENSION",
    long_description_content_type="text/plain",
    classifiers=['Private :: Do Not Upload'],
    packages=find_packages(),
    namespace_packages=["eiq"],
    install_requires=[
        "eclecticiq-platform-extensions-api == 3.3.*",
        "eclecticiq-extension-commons == 3.3.*", ## <-- Remove this line
    ],
    include_package_data=True,
    entry_points={"eiq.extensions": ["example_extension = eiq.extensions.example_extension:ext"]},
)

common.entities module #

This module provided constructors for composing and validating data that is subsequently submitted to the ingestion engine.

Change old import paths using eclecticiq-extension-commons:

from eiq.extensions.commons.entities import (
    create_report,
    create_indicator,
    create_ttp,
    create_exploit_target,
    create_threat_actor,
    create_incident,
    create_course_of_action,
    create_campaign,
    create_sighting,
    create_identity,
    create_malware,
    create_tool,
    create_attack_pattern,
    create_location,
)

To new import paths using eclecticiq-platform-extensions-api:

from eiq_ext.entities import (
    create_report,
    create_indicator,
    create_ttp,
    create_exploit_target,
    create_threat_actor,
    create_incident,
    create_course_of_action,
    create_campaign,
    create_sighting,
    create_identity,
    create_malware,
    create_tool,
    create_attack_pattern,
    create_location,
)

common.utils module #

Helper functions.

Removed #

The following have been removed:

eiq.extensions.commons.utils.proccess_transformed_data: Usually used in enrichers to flatten and preprocess the entity stream to resolve relations. This is no longer needed. Instead, enrichers should return a EnrichmentResult object containing a single list of entities and relationship entities.
eiq.extensions.commons.utils.add_country: Logic is inlined in eiq_ext.entities.create_location.
eiq.extensions.commons.utils.get_attack_ids: Removed.
eiq.extensions.commons.utils._filter_kwargs: Removed.

Moved to eiq_ext.entities module #

The following have been moved from eiq.extensions.commons.utils to eiq_ext.entities:

Change old import paths using eclecticiq-extension-commons:

from eiq.extensions.commons.utils import (
    COUNTRY_LIST,
    create_extract,
    check_value,
    create_information_source,
    make_likely_impact,
    make_confidence,
    create_threat_actor_type,
    create_threat_actor_sophistication,
    create_motivation,
    create_intended_effect,
    add_basic_data,
    add_meta,
    add_incident_data,
    add_sighting_data,
    add_threat_actor_roles,
    ensure_aware,
)

To new import paths using eclecticiq-platform-extensions-api:

from eiq_ext.entities import (
    COUNTRY_LIST,
    create_extract,
    check_value,
    create_information_source,
    make_likely_impact,
    make_confidence,
    create_threat_actor_type,
    create_threat_actor_sophistication,
    create_motivation,
    create_intended_effect,
    add_basic_data,
    add_meta,
    add_incident_data,
    add_sighting_data,
    add_threat_actor_roles,
    ensure_aware,
)

Moved to eiq_ext._entity_utils module #

The following have been moved to eiq_ext._entity_utils and are meant for internal use only. Developers should not use these in extensions.

New import paths:

from eiq_ext._entity_utils import (
    add_time,
    remove_empty_extracts,
)

common.validators module #

Helper functions and patterns for validating data that should comply with certain formats or schema.

Change old import paths using eclecticiq-extension-commons:

from eiq.extensions.commons.validators import (
    ip_octet,
    BASE_URI_SCHEMES,
    schemes,
    url_regex,
    URL_PATTERN,
    domain_regex,
    md5_regex,
    sha1_regex,
    sha256_regex,
    to_unicode,
    valid_url,
    valid_domain,
    valid_ipv4,
    valid_ipv4_cidr,
    valid_ipv6,
    valid_ipv6_cidr,
    valid_email,
    valid_md5,
    valid_sha1,
    valid_sha256,
    valid_sha512,
)

To new import paths using eclecticiq-platform-extensions-api:

from eiq_ext.validators import (
    ip_octet,
    BASE_URI_SCHEMES,
    schemes,
    url_regex,
    URL_PATTERN,
    domain_regex,
    md5_regex,
    sha1_regex,
    sha256_regex,
    to_unicode,
    valid_url,
    valid_domain,
    valid_ipv4,
    valid_ipv4_cidr,
    valid_ipv6,
    valid_ipv6_cidr,
    valid_email,
    valid_md5,
    valid_sha1,
    valid_sha256,
    valid_sha512,
)

Do not use `eiq_ext.legacy`#

Developers should remove usage of the eiq_ext.legacy module. Its contents can change without notice.