Order of operations

The order of operations for every interval starts with:

  1. Read the global policy configuration and each individual policy configuration.

  2. Process the global tasks.

  3. Process each policy, and push content for that policy.

Process global tasks

See General settings for configuration options.

  1. Process enabled integrations.

  2. Download remote Suricata config files and store the in /etc/corelight-update/global/.

  3. Download new content and update the Global Source Cache.

  4. Remove content from the global cache for sources that are no longer configured.

Process policy tasks

See Policy configuration for configuration options

  1. Copy local suricata rulesets from /etc/corelight-update/configs/<policy>/local-suricata/ to the working directory.

  2. Copy global suricata rulesets from /etc/corelight-update/global/global-suricata/ to the working directory.

  3. Copy local intel files from /etc/corelight-update/configs/<policy>/local-intel/ to the working directory.

  4. Copy global intel files from /etc/corelight-update/global/global-intel/ to the working directory.

  5. Remove content from the policy cache for sources that are no longer configured.

  6. Download new content from policy sources.

  7. Add default Input files to /etc/corelight-update/configs/<policy>/local-input/ (if enabled - only runs once)

  8. Copy local YARA files from /etc/corelight-update/configs/<policy>/local-yara/ to the working directory.

  9. Copy global YARA files from /etc/corelight-update/global/global-yara/ to the working directory.

  10. Process enabled integrations based on their intervals. See Third-party integrations settings

  11. Process Input files and update the statefile.

  12. Process Suricata rulesets.

    1. Collect ruleset files

      1. Collect new source content and copy it to the suricata working directory.

        • Check the global cache first.

        • If not in the global cache, download new content directly and update the policy level cache.

      2. Check for global .rules or .rules.tar.gz files in /etc/corelight-update/global/global-suricata/ and extract/copy them to the suricata working directory.

      3. Check for local .rules or .rules.tar.gz files in /etc/corelight-update/configs/<policy>/local-suricata/ and extract/copy them to the suricata working directory.

    2. Merge all of the rulesets into a single ruleset, ignoring any ruleset file identified with File filters in the following:

      • Corelight recommended disable.conf (if enabled)

      • global disable.conf (if it exists)

      • policy disable.conf (if it exists)

    3. If enabled, process Corelight recommended disable.conf, enable.conf and modify.conf files in that order.

    4. If they exist, process global disable.conf, enable.conf and modify.conf files in that order.

    5. If they exist, process policy disable.conf, enable.conf and modify.conf files in that order.

    6. If enabled, extract selected atomic rules from the Suricata ruleset and generate a Zeek Intel file.

    7. If enabled and Suricata is installed on the same host, test the new ruleset with Suricata in test mode (see Suricata configuration for details).

    8. Publish the new Suricata ruleset - suricata.rules.

  13. Process Intel files

    1. Check for global intel files in /etc/corelight-update/global/global-intel/, and copy them to the intel working directory.

    2. Check for local intel files in /etc/corelight-update/configs/<policy>/local-intel, and copy them to the intel working directory.

    3. Evaluate all of the global, local, and integration intel files from all sources. If disable.intel is available, evaluate and remove indicators. Merge and dedupe results into a single file.

    4. Publish the new intel file - intel.dat

  14. Process YARA files

    1. Check for global YARA files in /etc/corelight-update/global/global-yara/, and copy them to the yara working directory.

    2. Check for local YARA files in /etc/corelight-update/configs/<policy>/local-yara, and copy them to the yara working directory.

    3. Evaluate all of the global, local, and integration YARA files from all sources. Merge and dedupe results into a single file.

    4. Publish the new YARA file - yara_rules.yar

Push content for policies

Corelight-update deploys content updates in a specific order:

  1. Push new Intel files.

  2. Push new Suricata ruleset.

  3. Push new Zeek Package bundle.

  4. Push new YARA source

  5. Push new Input files

By default, Corelight-update will push updates to the sensors concurrently. Corelight-update will open a connection to multiple sensors in a policy, push updated content, and cycle to the next sensor, up to the global configuration setting parallel_push_limit. See General settings.

Tip

Corelight-update only attempts to push new content to sensors. You can manually force a push of all existing content to a group of sensors using the CLI commands.