Skip to Main Content
Back to blog

Finding and closing gaps in security logging with AI

by Sanya Kohli

November 18, 2026 - 11 min

Abstract dark blue banner featuring neon line-art icons of keys, devices, locks, a shield, cloud, gear, AI brain, and chatbot, connected by layered geometric lines. Finding and closing gaps in security logging with AI, 1Password.

Related Categories

A security incident is the worst time to discover that the logs you need were never collected.

Even with a security information and event management (SIEM) in place, critical logging gaps can go unnoticed. Some tools may not send logs at all, while others may appear to be integrated but have collection or parsing issues that make their logs difficult to use during an investigation. With multiple tools, it can be a challenge to know what's missing or whether the logs you have are actually usable.

At 1Password, finding these gaps meant manually comparing our tool inventory against SIEM log sources, one tool at a time. The process was time-consuming, difficult to keep up to date, and hard to scale as our environment evolved. We needed a more reliable way to identify gaps before they affected an investigation.

We saw an opportunity to use AI to change how we identify security logging gaps and onboard new log sources. Instead of relying on time-consuming manual reviews, we built an approach that uses AI to uncover potential gaps and guide engineers through onboarding, while keeping critical decisions in human hands. Here's how we built it, what we learned, and where AI makes a difference.

Collecting logs is only the beginning

As 1Password adopts new tools, maintaining security visibility requires more than simply sending logs to the SIEM. We need to determine which logs matter for security, collect them, and ensure they contain the searchable information engineers need during an investigation. Even when logs are successfully ingested, important details can remain buried in unparsed text. Without proper parsing, engineers may struggle to search those details or build reliable detection rules.

To address these challenges, we built an automated audit that compares our tool inventories against SIEM log sources and produces a prioritized report of potential logging gaps. For gaps we decide to close, we use AI skills: reusable instruction files that guide an AI tool through specific tasks, including required checks and approvals. These skills help engineers configure log collection and parsing pipelines, manage access controls, document integrations, and prepare detection rules. Engineers review every proposed action, keeping human oversight central to the onboarding process.

Missing logs are only part of the problem

The obvious gap is a tool whose logs we do not collect. The less obvious one is a source that appears to be onboarded because logs are arriving, but whose logs still cannot support the investigations or detection rules we need.

Our audit looks for both.

The first part compares the tools listed in our inventories with the log sources available in the SIEM. A tool with no matching logs is flagged as a potential coverage gap.

The second checks whether existing log sources show signs of collection or parsing problems. Examples include a source that has stopped sending logs or logs whose activity details are not being extracted into searchable fields. These checks help identify sources that need attention even though an integration is already in place.

These findings feed different decisions. A missing source might require a new pipeline to collect and deliver its logs. An existing source might need a parsing fix to correct how individual details are extracted from a log, such as who acted and what they did. Both belong in the same review.

How AI helps find the logging gaps

A scheduled workflow runs the audit automatically and produces a report for engineering review. It runs a Python script that requests vendor lists from our procurement system, reads our internal tool catalogs, queries the SIEM for log sources, and searches our ticketing system for related work. The script makes these requests directly through each system’s API. An AI model assesses which vendors are likely to have logs relevant to security monitoring and writes an explanation of each potential gap, why the logs would be valuable, and the recommended priority for onboarding them. The script handles data collection, comparison, and publication. This keeps the model focused on assessment while code handles repeatable operations.

The AI model assesses vendor names using instructions focused on the potential value of their logs for monitoring identity, sensitive data, infrastructure, code, security controls, or administrative access. The audit then checks whether logs from the tools identified as security-relevant are available in the SIEM. Because names alone can be ambiguous, this assessment can miss tools and remains a limitation of the approach.

The script builds a list of relevant tools from both inventories, standardizing names and removing duplicates from that list. It then compares that list with the names identifying tools in the SIEM logs. Matching accounts for naming differences, with safeguards against mistaking part of one tool’s name for a different tool. The result is an initial indication of coverage that an engineer can verify. The script also searches existing onboarding tickets so the report can show whether work to collect a tool’s logs is already underway.

The script saves each report as a dated file and opens a pull request for review. Once merged, the file becomes part of the audit history, allowing comparison with reports from earlier runs. The report identifies tools whose logs are not currently ingested into the SIEM, as well as existing SIEM log sources with potential log collection or parsing issues. Our priority framework considers how useful the logs would be for detecting suspicious activity, investigating an incident, and proactively searching for signs of attacker activity.

The workflow automatically sends a team notification with links to the report and audit run, along with a summary of potential gaps and log sources reviewed. A written review guide supports decisions about which findings need action. The report feeds planning; finding a gap does not trigger an emergency alert.

Can we trust the audit report?

An automated report can look convincing while silently omitting a tool, making completeness checks essential. The audit now stops if the vendor inventory or SIEM source list is empty, identifies tools without matching SIEM log sources, and verifies that every missing tool is assessed in the final report. Failed ticket searches are labeled “lookup failed” to distinguish them from searches that found no existing tickets.

Consistency between runs matters too. The script chooses a consistent name for each tool and instructs the AI model to preserve it exactly, so findings can be matched to their history from earlier audits. The model also receives the previous priority for each finding, and a change needs new evidence. We reuse saved assessments when the supporting information and report instructions remain unchanged, with periodic refreshes. Changes to the instructions trigger fresh analysis.

We save previous vendor classifications, tool assessments, and gap history in a cache, so each run can build on earlier work. Keeping this history makes it possible to inspect earlier assessments and correct them when needed.

Which logging gaps should we close first?

Before starting an integration, an engineer verifies that the reported gap exists. A tool may already send logs under a name the audit did not recognize, or the inventory may still list a tool that is no longer used.

Before a finding becomes an onboarding project, we check three things: the gap is real, the vendor can deliver useful audit logs, and the expected security value justifies prioritizing the work alongside the team’s other commitments.

For custom integrations, a classification skill reads the vendor's official audit log documentation and recommends a supported way to collect the logs, with a confidence rating. It can also identify cases our pipeline tooling cannot support. This happens before ticket creation, so we avoid generating a full onboarding plan for a source we cannot collect.

The reviewer considers how much damage a compromise could cause, what we can already monitor, which suspicious activities we could detect, and the effort needed to add the logs. During report review, an engineer may prioritize repairing an existing source over collecting logs from another tool, especially if the repair would restore visibility into important activity.

False gaps require a correction in the underlying inventory. Otherwise, the next audit simply asks the same question again.

Putting AI to work on log onboarding

For an approved source, a skill creates an onboarding plan in our ticketing system with standard tasks. Each task identifies which skill to use, explains the work it performs, and includes manual instructions as a fallback.

The workflow covers choosing how to collect the logs, gathering access and configuration details, building the pipeline, making logs searchable, and restricting access. It also documents what the logs contain, how they are collected, and how engineers can find them in the SIEM. The later steps prepare the repository structure and review process for managing detection rules as code. A detection-seeding skill then examines the available logs and identifies detection opportunities, creating tasks for engineers to develop and review the proposed rules.

A coordinating skill, the orchestrator, reads the onboarding plan and checks task status against evidence. For a task marked complete, it verifies the relevant result: infrastructure changes have merged, logs are arriving and searchable in the SIEM, or the log inventory has been updated with the collection method and instructions for finding the new source’s logs.

The orchestrator proposes one next action for an engineer to approve. Each skill retains its own approval requirements, and the orchestrator pauses at tasks that require manual work. An engineer reviews and applies infrastructure changes and configures the credentials used in production.

That onboarding boundary involves decisions with lasting consequences. An engineer needs to review which resources will be created or changed and decide whether the logs contain sensitive information that requires restricted access.

When is a log source really ready?

The skills check live evidence at several points in the workflow.

When building a pipeline that collects logs through a vendor API, the pipeline-builder skill first makes a live request to that API. It confirms authentication, the response structure, the timestamp field, whether the API returns logs from oldest to newest or newest to oldest, and which vendor account or tenant the logs belong to. It uses real sample logs to map details into the fields our monitoring expects. Each proposed detection idea must be grounded in logs available in the SIEM and fields engineers can query.

A successful deployment or an absence of alarms does not establish that log collection is working. Validation needs to follow logs through the pipeline and confirm that they are searchable. For teams building similar automation, this is a useful distinction: infrastructure status and evidence of log delivery answer different questions.

The routing skill checks the configuration that determines whether incoming logs become searchable in the SIEM. This matters because successful delivery to the platform does not always mean the logs are available for monitoring. Reviewing that configuration before making changes also helps preserve integrations already in use.

Access controls belong in the onboarding process before new logs become searchable. An engineer reviews the sample logs for sensitive information and decides who should be able to read them. Considering access at this stage helps avoid exposing sensitive logs while restrictions are still being arranged.

At the detection stage, a skill checks a standard set of activities worth monitoring, including granting administrator access, creating access tokens, exporting data, and changing security settings. It records whether each is already covered by a detection rule, needs a new detection rule, lacks the necessary logs, or still needs verification. Only a confirmed detection gap becomes a detection task. If the logs are missing, we need to collect them first.

We now have a recurring comparison of company tool inventories against SIEM log coverage, together with a documented workflow for collecting missing logs and preparing detection rules. The audit preserves history between runs, and the interactive workflow checks evidence before advancing.

There are still limits. A name match in the SIEM does not prove that logs capture every activity we need to monitor. A source that produces logs infrequently may need an additional check before being treated as missing. Human review remains part of interpreting the report.

Would your logs be ready for an incident?

A useful starting point is to compare the tools your organization uses with the logs your security team can actually search. Investigating the differences helps establish where coverage is missing and which gaps deserve attention. From there, the audit and onboarding approach described in this post provides a repeatable path from identifying a gap to collecting the logs and preparing detection rules, with AI assisting the work and engineers reviewing the changes.

During an incident, the logs available to investigators shape what they can establish about the activity involved. Finding missing logs, verifying that they can be searched, and building detection rules gives teams a stronger foundation for identifying and investigating threats before an incident puts that visibility to the test.

Jessica Mitchell - Staff CX Content Developer

Subscribe to the 1Password Developer newsletter

Stay up to date with the latest 1Password Developer product news, industry insights, and community contributions. Plus, learn best practices for becoming a better, more secure developer – both at work and at home.