Filter generation in practice: from algorithms to LLMs

Security technologies

Filter validity criteria

Problems in filter generation

Filtering security alerts with an algorithm

Pros and cons of the algorithmic approach

Filter generation with an LLM

Pros and cons of generating filters with an LLM

Using multiple AI agents

Pros and cons of the agent-based approach

Evaluating the filter generation system

A modern organization protects its information systems with a whole suite of technologies that generate thousands of security alerts every day. Looking at these alerts in bulk, you’ll see that a significant share are false positives. Ideally, these alerts would be filtered out automatically so that analysts could focus on the ones that may point to a real incident. However, writing filters is a time-consuming task in itself. To create a correct filter, an analyst needs to:

Identify the source of the activity

Define filtering boundaries

Choose filtering criteria

Check that the filter is robust

This post looks at how filter creation itself can be automated to reduce analyst workload.

In the context of security alerts, a filter is a combination of event field values that marks a detection as a false positive and not worth the analyst’s attention. Filters can be generated for any combination of event fields, but not every combination yields a valid filter. That’s why we need to define validity criteria: conditions showing that a specific filter actually removes noise and doesn’t suppress useful alerts.

Detection rules in security systems typically contain a set of conditions, and each of them can generate an alert. A specific event triggers a given fragment of a detection rule. The set of events that a given fragment responds to is what we’ll call its context. A good filter should cover the entire context where possible, without going beyond it. Filters that are too broad or too narrow are useless. For example, a filter based solely on PowerShell would exclude too many events, including some that deserve the analyst’s attention. Most of the filtered events would also be unrelated to the rule in question. A filter on a specific ID, on the other hand, would cover only one event, which may never recur.

We can run into a number of problems when creating filters. First, events often contain an excessive number of fields (around 400 on average), and most of them aren’t needed for filtering. At the same time, generating and ranking combinations of these fields is also fairly resource-intensive. So, the first step is to discard the irrelevant fields and keep only those worth using in the filter. For example, out of 400 fields in an event, four may be enough for filtering.

Another problem is string normalization. Field values (for example, the path to the same application on different systems) can vary depending on system settings, user folder names, and so on. For a filter to account for all these variations, the strings must be normalized. For file paths, in particular, you can replace the drive letter and user folders with standard environment variables and convert all paths to a single case.

Finally, if a filter completely mirrors the query of a detection rule, it effectively disables that rule. To fix this, remove the event fields used in the detection rule from the filter. For example, if a rule triggers on a file path, there are two ways to filter out unwanted alerts:

In the filter, use fields other than the file path.

Use a part of the file path that the rule doesn’t use. For example, for the rule file_path: ".../something.exe" , you can set up a filter that excludes events with the user/unique_files/ directory in the file path.

Given the problems described above, automatically generating filters for security alerts should consist of the following steps:

An analyst receives a security alert that they assess as a false positive and uses the filter generator.

The generator takes the 400-field event from the alert, along with the rule that triggered on it, and determines the context.

The generator selects the fields that are relevant to the filter.

The generator normalizes the strings.

The generator builds different combinations of the selected fields. To do this, we applied the FP-Growth (frequent pattern growth) algorithm to the context determined earlier. FP Growth builds a tree of frequently occurring items (fields) from a dataset (in our case, the context) and discards items that aren’t frequent enough. This reduces the number of combinations to check, which lowers the cost of finding the most relevant ones.

The generator ranks the resulting combinations and offers the analyst the top three to choose from.

For example, consider an event described by the following JSON:

After generating and ranking the combinations, we got three filters based on file paths:

processcmdline:("nodejs\\node.exe" AND "*****\\appdata\\roaming\\npm\/node_modules\/pi-coding-agent\/dist\/cli.js") 1 processcmdline : ( "nodejs\\node.exe" AND "*****\\appdata\\roaming\\npm\/node_modules\/pi-coding-agent\/dist\/cli.js" )

file_path:"*****/.pi\\agent\\bin\\fd.exe" 1 file_path : "*****/.pi\\agent\\bin\\fd.exe"

processcmdline: "programfiles\\nodejs\\node.exe" 1 processcmdline : "programfiles\\nodejs\\node.exe"

The last filter turned out to be too broad, but the analyst marked the other two as valid.

The algorithmic approach to filter generation has several advantages. First, it’s simple and deterministic: the same alert always produces the same result. Second, it’s interpretable, meaning we can see why a given combination ranked near the top of the list and why a particular field didn’t make it into the list of candidate fields at all. Third, the stages of the process are independent of each other, so individual steps can be improved and tuned without the risk of breaking the algorithm elsewhere. Finally, it has no external dependencies or models, which makes it easy to deploy in any customer environment.

However, it also has drawbacks. The first is its dependence on context. If there’s no context for some reason — for example, the rule’s first-ever trigger — a filter can’t be created. The second problem is string normalization. There’s a high chance of running into atypical cases that the normalization rules don’t cover. Often such cases can’t be described correctly at all, because they contradict each other.

These problems can be solved with large language models (LLMs). When an LLM generates the filters, no context is needed, because the LLM doesn’t iterate over or rank combinations. The LLM also takes a more flexible approach to string normalization, including non-standard cases. That said, an LLM doesn’t have to be involved at every stage of the generation process. We decided to keep the algorithmic solution where it works well. For example, a simple algorithm selects suitable fields without errors, so we didn’t move this stage to the model.

The LLM-based filter generation looks like this:

The input is still an alert containing an event with 400 fields, plus the rule that triggered on that event. Then an algorithm selects the most suitable fields for the filter. Next, the LLM processes the input — the event, the selected fields, and the detection rule — and generates three filter options along with an explanation for them. The explanation helps the analyst understand why these particular filters were suggested and make an informed choice.

Consider the following event in JSON format:

The LLM suggested three filter options that all contain the same command line fragment:

file_path: "\\WindowsPowerShell\\powershell.exe" AND filecmdline: "Get-VM -Name kafka\*" 1 file_path : "\\WindowsPowerShell\\powershell.exe" AND filecmdline : "Get-VM -Name kafka\*"

filecmdline: "Get-VM -Name kafka\*" 1 filecmdline : "Get-VM -Name kafka\*"

filecmdline: "kafka*" 1 filecmdline : "kafka*"

As in the previous example, two of the three output filters were valid. The last one was too broad to be usable. The analyst chose the first filter with minor edits:

Using an LLM solved the problems of purely algorithmic filter generation: dependence on context and errors in string normalization. However, the single-request approach had its own drawbacks. First, it had no output validation. The LLM returned filters as-is, with no syntax check and no way to correct errors. Second, unlike in the algorithmic version, all filter generation tasks were handled by a single system prompt. As a result, a minor fix in one place could break the logic in another.

To address the problems of generating filters with a single model and a single system prompt, we tried distributing the tasks across different LLM agents. In this version, we brought in agents as early as the stage of selecting relevant fields. The chain includes four agents in total:

Selector: selects the relevant fields.

Generator: generates filters.

Validator: evaluates the generated filters. If the filters are found to be invalid, it returns the task to the generator. If the selected fields are found to be irrelevant, it starts a new field selection cycle. If the filters pass validation, or the limit on regeneration attempts is reached, the valid filters (or the best ones available) are passed to the analyst.

Commentator: analyzes the output filters and explains which field values were used and why.

For the following JSON event, the agents produce three valid filters based on the file path.

The filters differ in the additional parameters that support the detection:

file_path:"sysvol\\domain\\scripts\\logon.bat" 1 file_path : "sysvol\\domain\\scripts\\logon.bat"

file_path:"sysvol\\domain\\scripts\\logon.bat" AND file_md5:"0x7..6E" 1 file_path : "sysvol\\domain\\scripts\\logon.bat" AND file_md5 : "0x7..6E"

autorun_entry: "Group Policy\\State" AND file_path: "SYSVOL\\domain\\scripts\\logon.bat" 1 autorun_entry : "Group Policy\\State" AND file_path : "SYSVOL\\domain\\scripts\\logon.bat"

LLM agents give us a system that can validate and correct filters. Its modular architecture also lets us tune and improve different stages of generation independently. However, this approach makes more LLM calls than the previous ones, so the system is slower. It also depends on access to a model, which may not be available in the organization’s isolated environment.

To evaluate how well the filter generation system works, we use two main metrics:

Approval rate. For each security alert, the system suggests three filters. If the analyst selects at least one of them, we count the generation as successful. For example, in the case below, the approval rate is 2/3.

False positive reduction rate (FP drop). This metric shows how much a filter helps reduce the number of false positives. It can only be measured over time, because we need to compare the number of false positive alerts received during a set period before and after the filter is deployed.

Below is a comparison of the three filter generation approaches by these metrics:

As the results show, LLMs significantly improve the efficiency of filter generation. At the same time, the LLM agentless and agent-based approaches perform almost identically. So we chose the simpler and faster option: an LLM with a single system prompt. As a result, we cut false positives by 99%, increased the number of filters generated per day by 28%, and made analysts’ alert handling 17% more efficient.

Practice has shown that filter generation can be automated effectively. We’ve looked at several options for automatically generating filters for false-positive security alerts: a purely algorithmic approach, an LLM-based approach, and an approach built on a group of LLM agents. All of them can generate valid filters and make the analyst’s work much easier, but each has its own pros and cons.

The algorithmic approach scored lower on the evaluation metrics, but in two-thirds of cases, it still produces filters that satisfy the analyst. It works especially well when there’s already a set of similar events to filter out, meaning enough context has accumulated. The algorithmic generator can also be deployed in any environment. Both standalone LLM and multi-agent LLM approaches show high accuracy, and their results differ only slightly, so the slower agent-based approach looks excessive at this stage. If an LLM is available, the best option appears to be a single model that handles several stages of the generation cycle.

Artificial intelligence

Filter generation in practice: from algorithms to LLMs

This site uses Akismet to reduce spam. Learn how your comment data is processed.

GReAT Ideas. Balalaika Edition

GReAT Ideas. Green Tea Edition

GReAT Ideas. Powered by SAS: malware attribution and next-gen IoT honeypots

GReAT Ideas. Powered by SAS: threat actors advance on new fronts

GReAT Ideas. Powered by SAS: threat hunting and new techniques

In the same category

Wardriving assessment across Mexico: Preparing for the 2026 World Cup

How an image could compromise your Mac: understanding an ExifTool vulnerability (CVE-2026-3102)

Yet another DCOM object for lateral movement

Detecting DLL hijacking with machine learning: real-world cases

How attackers adapt to built-in macOS protection

安全技术

筛选有效性标准

滤波器生成中的问题

使用算法过滤安全警报

算法方法的优缺点

利用LLM生成滤波器

使用LLM生成滤波器的优缺点

使用多个人工智能代理

基于代理方法的优缺点

评估滤波器生成系统

现代企业使用一整套技术来保护其信息系统，这些技术每天都会生成数千条安全警报。批量查看这些警报，你会发现其中很大一部分是误报。理想情况下，这些误报应该被自动过滤掉，以便分析人员能够专注于那些可能指向真正安全事件的警报。然而，编写过滤器本身就是一项耗时的任务。要创建正确的过滤器，分析人员需要：

确定活动的来源

定义过滤边界

选择筛选条件

检查滤波器是否稳健

本文探讨了如何实现筛选器创建的自动化，以减少分析师的工作量。

在安全警报领域，过滤器是指事件字段值的组合，用于将检测结果标记为误报，从而避免引起分析人员的关注。过滤器可以针对任意事件字段组合生成，但并非所有组合都能产生有效的过滤器。因此，我们需要定义有效性标准：即证明特定过滤器确实能够去除噪声，而不会抑制有用的警报的条件。

安全系统中的检测规则通常包含一组条件，每个条件都可能触发警报。特定事件会触发检测规则的特定片段。特定片段所响应的事件集合称为其上下文。一个好的过滤器应该尽可能覆盖整个上下文，并且不超出上下文范围。过宽或过窄的过滤器都毫无用处。例如，仅基于 PowerShell 的过滤器会排除太多事件，包括一些值得分析人员关注的事件。大多数被过滤掉的事件也与相关规则无关。另一方面，基于特定 ID 的过滤器只会覆盖一个可能永远不会再次发生的事件。

创建筛选器时，我们可能会遇到一些问题。首先，事件通常包含过多的字段（平均约 400 个），而其中大部分字段并非筛选所必需。同时，生成和排序这些字段的组合也相当耗费资源。因此，第一步是舍弃无关字段，只保留筛选中真正需要的字段。例如，一个事件的 400 个字段中，可能只需要四个就足以用于筛选。

另一个问题是字符串规范化。字段值（例如，同一应用程序在不同系统上的路径）可能因系统设置、用户文件夹名称等因素而异。为了使过滤器能够处理所有这些变化，必须对字符串进行规范化。特别是对于文件路径，您可以将驱动器号和用户文件夹替换为标准环境变量，并将所有路径转换为单一大小写形式。

最后，如果过滤器与检测规则的查询完全一致，则该规则实际上会被禁用。要解决此问题，请从过滤器中移除检测规则中使用的事件字段。例如，如果规则基于文件路径触发，则有两种方法可以过滤掉不需要的警报：

在筛选器中，使用文件路径以外的字段。

使用规则未使用的文件路径部分。例如，对于规则 file_path: ".../something.exe"，您可以设置一个过滤器，排除文件路径中包含 user/unique_files/ 目录的事件。

鉴于上述问题，自动生成安全警报过滤器应包含以下步骤：

分析人员收到一条安全警报，经评估为误报后，使用过滤器生成器进行过滤。

生成器从警报中获取 400 字段事件，以及触发该事件的规则，并确定上下文。

生成器会选择与筛选条件相关的字段。

生成器对字符串进行规范化处理。

生成器会构建所选字段的不同组合。为此，我们对先前确定的上下文应用了 FP-Growth（频繁模式增长）算法。FP-Growth 会从数据集（在本例中为上下文）中构建一个频繁出现项（字段）的树，并丢弃出现频​​率不够高的项。这减少了需要检查的组合数量，从而降低了找到最相关组合的成本。

生成器会对生成的结果组合进行排名，并向分析师提供前三名供其选择。

例如，考虑以下 JSON 描述的事件：

在生成并对组合进行排序后，我们根据文件路径得到了三个过滤器：

processcmdline:("nodejs\\node.exe" AND "*****\\appdata\\roaming\\npm\/node_modules\/pi-coding-agent\/dist\/cli.js") 1 processcmdline : ( "nodejs\\node.exe" AND "*****\\appdata\\roaming\\npm\/node_modules\/pi-coding-agent\/dist\/cli.js" )

file_path:"*****/.pi\\agent\\bin\\fd.exe" 1 file_path : "*****/.pi\\agent\\bin\\fd.exe"

processcmdline: "programfiles\\nodejs\\node.exe" 1 processcmdline: "programfiles\\nodejs\\node.exe"

最后一个筛选条件过于宽泛，但分析师认为其他两个筛选条件有效。

算法生成的过滤器具有以下几个优点。首先，它简单且确定性强：相同的警报总是产生相同的结果。其次，它易于解释，这意味着我们可以了解为什么某个组合排名靠前，以及为什么某个字段根本没有进入候选字段列表。第三，该过程的各个阶段彼此独立，因此可以改进和调整各个步骤，而不会影响算法的其他部分。最后，它不依赖任何外部模型，因此易于部署到任何客户环境中。

然而，它也存在一些缺点。首先是它对上下文的依赖性。如果由于某种原因缺少上下文——例如，规则的首次触发——则无法创建过滤器。第二个问题是字符串规范化。很可能遇到规范化规则无法涵盖的非典型情况。通常，这些情况根本无法正确描述，因为它们彼此矛盾。

这些问题可以通过大型语言模型（LLM）来解决。LLM 生成过滤器时无需上下文信息，因为它不会遍历或对组合进行排序。LLM 还采用了一种更灵活的字符串规范化方法，能够处理非标准情况。也就是说，LLM 并非必须参与生成过程的每个阶段。我们决定在算法效果良好的地方保留它。例如，一个简单的算法就能准确无误地选择合适的字段，因此我们没有将此步骤移至模型中。

基于LLM的滤波器生成过程如下：

输入仍然是一个包含 400 个字段的事件警报，以及触发该事件的规则。然后，算法会选择最适合过滤器的字段。接下来，LLM 处理输入（事件、选定的字段和检测规则），并生成三个过滤器选项及其解释。这些解释有助于分析人员理解为什么建议这些特定的过滤器，从而做出明智的选择。

请考虑以下 JSON 格式的事件：

LLM 建议了三个筛选选项，它们都包含相同的命令行片段：

file_path: "\\WindowsPowerShell\\powershell.exe" AND filecmdline: "Get-VM -Name kafka\*" 1 file_path : "\\WindowsPowerShell\\powershell.exe" AND filecmdline : "Get-VM -Name kafka\*"

filecmdline: "Get-VM -Name kafka\*" 1 filecmdline : "Get-VM -Name kafka\*"

filecmdline: "kafka*" 1 filecmdline : "kafka*"

与前例类似，三个输出筛选器中有两个有效。最后一个筛选器范围过广，无法使用。分析师最终选择了第一个筛选器，并稍作修改：

使用 LLM 解决了纯算法生成过滤器时存在的问题：对上下文的依赖以及字符串规范化错误。然而，这种单请求方法也有其自身的缺点。首先，它没有输出验证。LLM 会按原样返回过滤器，既没有语法检查，也无法纠正错误。其次，与算法版本不同，所有过滤器生成任务都由单个系统提示符处理。因此，一处的小错误可能会导致另一处的逻辑出错。

为了解决使用单一模型和单一系统提示生成过滤器的问题，我们尝试将任务分配给不同的 LLM 代理。在这个版本中，我们最早在选择相关字段的阶段就引入了代理。该流程链总共包含四个代理：

选择器：选择相关字段。

生成器：生成过滤器。

验证器：评估生成的筛选器。如果筛选器无效，则将任务返回给生成器。如果所选字段不相关，则启动新的字段选择周期。如果筛选器通过验证，或者达到重新生成次数限制，则将有效的筛选器（或最佳可用筛选器）传递给分析员。

评论员：分析输出过滤器，并解释使用了哪些字段值以及原因。

对于以下 JSON 事件，代理会根据文件路径生成三个有效的过滤器。

这些过滤器之间的区别在于用于辅助检测的附加参数：

file_path:"sysvol\\domain\\scripts\\logon.bat" 1 file_path : "sysvol\\domain\\scripts\\logon.bat"

file_path:"sysvol\\domain\\scripts\\logon.bat" AND file_md5:"0x7..6E" 1 file_path : "sysvol\\domain\\scripts\\logon.bat" AND file_md5 : "0x7..6E"

autorun_entry: "Group Policy\\State" AND file_path: "SYSVOL\\domain\\scripts\\logon.bat" 1 autorun_entry : "Group Policy\\State" AND file_path : "SYSVOL\\domain\\scripts\\logon.bat"

LLM代理为我们提供了一个可以验证和修正过滤器的系统。其模块化架构还允许我们独立地调整和改进生成过程的不同阶段。然而，这种方法比以前的方法需要调用更多的LLM，因此系统速度较慢。此外，它还依赖于对模型的访问，而该模型在组织的隔离环境中可能无法使用。

为了评估过滤器生成系统的工作效果，我们使用两个主要指标：

批准率。对于每个安全警报，系统会建议三个筛选条件。如果分析人员选择了至少一个筛选条件，则该次生成视为成功。例如，在下面的例子中，批准率为 2/3。

误报率降低率（FP下降）。该指标显示过滤器在减少误报数量方面的效果。它只能通过一段时间来衡量，因为我们需要比较部署过滤器前后特定时间段内收到的误报数量。

下面根据这些指标对三种过滤器生成方法进行比较：

结果表明，LLM显著提高了过滤器生成效率。同时，无代理和基于代理的LLM方法性能几乎相同。因此，我们选择了更简单快捷的方案：仅需一次系统提示的LLM。最终，我们将误报率降低了99%，每日生成的过滤器数量增加了28%，分析师的告警处理效率提高了17%。

实践证明，过滤器生成可以有效地实现自动化。我们研究了几种自动生成误报安全警报过滤器的方案：纯算法方法、基于LLM的方法以及基于LLM代理组的方法。所有这些方法都能生成有效的过滤器，并大大简化分析人员的工作，但每种方法都有其自身的优缺点。

算法方法在评估指标上的得分较低，但在三分之二的情况下，它仍然能够生成满足分析师要求的过滤器。当已经存在一组相似事件需要过滤时，这种方法尤其有效，这意味着已经积累了足够的上下文信息。此外，该算法生成器可以部署在任何环境中。独立LLM和多智能体LLM方法都展现出了很高的准确率，且结果差异很小，因此，在现阶段，速度较慢的基于智能体的方法显得有些多余。如果LLM可用，最佳选择似乎是使用一个能够处理生成周期多个阶段的单一模型。

人工智能

滤波器生成实践：从算法到LLM

本网站使用 Akismet 来减少垃圾邮件。了解您的评论数据如何处理。

伟大的想法。巴拉莱卡版

绝妙创意。绿茶版

GReAT Ideas。由 SAS 提供支持：恶意软件归因和下一代物联网蜜罐

GReAT Ideas。由 SAS 提供支持：威胁行为者在新战线上不断扩张

GReAT Ideas。由 SAS 提供支持：威胁狩猎和新技术

同一类别

墨西哥全境巡逻评估：为2026年世界杯做准备

图片如何危害您的 Mac：了解 ExifTool 漏洞 (CVE-2026-3102)

另一个用于横向移动的DCOM对象

利用机器学习检测 DLL 劫持：真实案例

攻击者如何适应 macOS 内置的保护机制