Delete incoming feeds

Delete an incoming feed to remove from the platform any packages, as well as any entities and observables, ingested through the feed. Purge an incoming feed to completely remove any ingested data through the feed, as well as the feed configuration.

Delete incoming feeds

Delete an incoming feed to remove from the platform all the content ingested through the feed, while keeping the incoming feed configuration, in case you may want to ingest content from that data source again in the future.

Delete incoming feeds in the GUI

To delete any content ingested through an incoming feed, and to keep the feed configuration:

  1. In the top navigation bar click Data configuration > Incoming feeds.

  2. In the incoming feed overview, click anywhere in the row corresponding to the feed you want to delete.

  3. In the incoming feed detail pane, click > Delete.

  4. In the Delete the incoming feed dialog, the Delete feed content only option is preselected by default.
    This action deletes all the data ingested with the selected incoming feed since the beginning of time. It keeps the (empty) feed and its configuration.

  5. To delete the feed content, click Delete; to keep the data, click Cancel.

Monitor deletion progress

You can visually monitor the progress of the operation in the GUI:

  • A progress bar shows completed and queued steps.

  • To display an explanatory tooltip, hover the mouse over a square representing a step in the process.

  • The platform provides information about the current running time of the process, as well as the estimated time left before completion.

To view these details in the System jobs view:

  • In the pop-up notification message about the running incoming feed deletion process, click Show job.

Alternatively:

  1. In the side navigation bar click the settings icon > System jobs.

  2. In the System jobs view, click the eiq.utilities.delete_incoming_feed job whose progress you want to inspect to open the corresponding detail pane with the information.

Check completion status

Deleting or purging an incoming feed can produce the following end results:

  • Success: the process completes correctly.

    • Any content ingested through the incoming feed is deleted.

    • The incoming feed configuration is kept.

  • Revoked: a user aborted the process before it could complete.
    The notification message includes the user name of the user who revoked the process.

  • Failure: an unexpected error caused the process to abort and fail before it could complete.
    The notification message includes basic information about the step that failed. Inspect relevant logs to begin troubleshooting.

Delete incoming feeds in the command line

The eiq-platform purge-incoming-feed command enables deleting incoming feeds from the command line.
Before executing, it prompts users to specify whether they want to delete any ingested content and keep the feed configuration; or if they want to completely remove any ingested content, as well as the feed configuration.

Start a virtual environment

The eiq-platform command line interface offers a set of commands and utilities to diagnose and to address common operational issues that may occur over time.
The eiq-platform command line interface is available in the platform Python virtual environment: activate it and, if necessary, import the platform environment variables.

  • Open a terminal session.

  • In the terminal, log in to the platform through SSH:

    # The platform user name must have admin access rights
    ssh ${platform_user_name}@${platform_host_name}
     
    # Or:
    ssh ${platform_user_name}@${platform_ip_address}
     
    # Example:
     
    # Or:

  • Obtain root-level access by running sudo -i:

    # Root-access login shell
    sudo -i

    To access resources as a different user than the currently active one, append -u:

    # Grant the currently logged in user root-level access
    sudo -i
     
    # Grant root-level access to a different user
    sudo -i -u ${user_name}
     
    # Run a command as a different user, with root-level access
    sudo -i -u ${user_name} ${command} ${options}

  • Activate a Python virtual environment for the platform:

    source /opt/eclecticiq-platform-backend/bin/activate

Run the purge incoming feed command

  1. First, print a list of the available incoming feeds.

  2. Then, specify either the feed ID, or the source ID corresponding to the incoming feed you want to delete.
    Do not specify both.

  3. At the prompt requesting you to confirm whether you want to delete all the content ingested through the feed – downloaded packages, entities, and observables – press y to confirm content deletion.

  4. At the prompt requesting you to confirm whether you want to delete also the feed configuration, press n to keep the configuration.

For more information about the command, append --help to it, and then press ENTER on your keyboard.

Command


eiq-platform purge-incoming-feed

Parameters


Parameters

Type

Description

Required

Default

--list

-

The parameter takes no values.
It returns a table listing all incoming feeds currently available in the platform:

  • ID: displays the incoming feed ID.
    To purge a feed by specifying the feed ID, pass this value with the --incoming-feed-id parameter.

  • Source ID: displays the data source ID.
    To purge a feed by specifying the data source ID, pass this value with the --source-id parameter.

  • Entity count: displays the total amount of entities ingested with the incoming feed until then.

  • Feed name: displays the designated name to identify the feed in the platform.

No

-

--incoming-feed-id

Integer

To purge an incoming feed by specifying the feed ID, add this parameter to the command.

Assign the ID value corresponding to the feed you want to purge.
The incoming feed list the --list parameter generates includes feed IDs.

No

-

--source-id

UUID

To purge an incoming feed by specifying the data source ID of the feed, add this parameter to the command.

Assign the Source ID value corresponding to the feed you want to purge.
The incoming feed list the --list parameter generates includes data source IDs.

No

-

Example


# Request a list of all incoming feeds
eiq-platform purge-incoming-feed --list
 
# The response contains feed ID, data source ID, total number of entities ingested through the feed, and human-readable feed name
ID Source ID Entity Count Feed Name
---- ------------------------------------ -------------- -------------------
1 b056b69e-6dc4-4347-b37b-5071e7ddf81d 0 TAXII Stand Samples
 
 
# Either purge an incoming feed by feed ID
eiq-platform purge-incoming-feed --incoming-feed-id=1
 
# Or purge an incoming feed by data source ID
eiq-platform purge-incoming-feed --source-id=b056b69e-6dc4-4347-b37b-5071e7ddf81d
 
 
# At this prompt, press "y" to delete any content ingested through the feed
are you sure you want to delete all?
 
# At this prompt, press "n" to keep the feed configuration
Do you want to delete the incoming feed configuration?
 
 
# Background task name in the system jobs
eiq.utilities.delete_incoming_feed
 
# Logger name in the logs
eiq.platform.purging