Share via

Centralize access to security logs (Secure Future Initiative)

Pillar name: Monitor and detect threats
Pattern name: Centralize access to security logs

Security logs are essential for monitoring threats and supporting investigations, but without centralization and standardization, they can create more management overhead and slow down response times. Microsoft addressed these challenges by centralizing log access, standardizing data capture, and extending retention periods, enabling faster, more effective investigations. These measures, alongside AI-powered detection and expanded log retention for customers, improve overall security and forensic capabilities.

Context and problem

Security logs are critical tools for security staff, as they reveal who accessed what, when, and how. Logs support continuous monitoring of threat activity, speed up incident response, and provide forensic records for security investigations.

Yet without centralization and standardization, logs can create more problems than they solve. Inconsistent formats, disparate storage locations, and variable retention periods make it difficult to quickly piece together a complete picture of an attack. At Microsoft, investigations were slowed by fragmented log sources and gaps in retention that left security teams unable to trace initial access points or lateral movement paths.

Solution

As part of SFI, Microsoft centralized access to security logs and extended retention policies. Key measures include:

  • Standardized security logging library: Ensures consistent data capture across services, reducing observability gaps.

  • Centralized log collection: Specialized investigator accounts provide unified access to cross-service logs, simplifying correlation and speeding up investigations.

  • Extended log retention: Audit logs retained for up to two years across Microsoft services, to enable forensic investigation of long-term attack patterns.

  • Advanced detection analytics: Integration of machine learning and AI-powered models improves detection of complex attack techniques and reduces false positives.

  • Expanded customer logging: Microsoft increased standard audit log retention for Microsoft 365 customers to 180 days, with options for longer retention.

Guidance

Organizations can adopt a similar pattern using the following actionable practices:

Use case Recommended action Resource
Standardized logging
  • Use Microsoft Purview auditing solution for Microsoft Entra and Microsoft 365 applications logs (for example, Exchange Online, SharePoint, OneDrive).
  • Implement a common log library across applications and services to enforce consistent fields (identity, actions, timestamps).
Central log storage
  • Use Microsoft Sentinel's data lake to transform SIEM with AI and unified security data.
  • Direct all security logs into a central repository, accessible to authorized investigators.
  • Centralize access to logs with Azure's Monitor and Log Analytics to reduce blind spots.
  • Use Microsoft Sentinel's security information and event management (SIEM) solution to enhance visibility and context for security logs and data, threat investigation, and response.
Encryption and immutable storage
  • In Azure Monitor, you can secure log data in transit and encrypt Log Analytics data at rest in Azure Storage using Microsoft-managed keys (MMK)
  • Encrypt logs in transit and at rest, and store them immutably to preserve forensic integrity, using Microsoft Sentinel's data tiering for cost-efficient retention.
  • We recommend using immutable to protect from overwrites and deletes.
Real-time monitoring
  • Set up continuous monitoring and alerts with Azure Monitor and Microsoft Sentinel to detect suspicious activity or logging coverage gaps.
  • In Azure Monitor and Log Analytics, you can monitor, improve logging coverage, and alert on suspicious activities using KQL queries and alert rules
  • Microsoft Sentinel provides advanced monitoring and detection features such as real-time log ingestion, analytics rules to watch ingested data, log matching against threat intelligence, alert grouping into incidents, and security orchestration, automation, and response (SOAR) capabilities.
Advanced analytics
  • Correlate logs across services using AI/ML and integrate with threat intelligence feeds.
  • Correlate logs in Azure Monitor and Log Analytics using KQL queries to connect signals across data sources.
  • Apply advanced analytics in Microsoft Sentinel with prebuilt rule libraries that map to common attack patterns, user and entity behavior analytics (UEBA) for anomaly detection, AI-powered alert correlation, ML-driven anomaly detection, and logs enriched with threat intelligence.
Review cadence
  • Conduct quarterly reviews to validate coverage, retention compliance, and pipeline integration.
  • Optimize SOC processes with Microsoft best practices and guidance on security operations.

Benefits 

  • Improved visibility: Provides security teams with a unified view of activity across identities, infrastructure, applications and endpoint devices.
  • Faster investigations: Investigators can correlate events across services without manual searches or format conversion.
  • Forensic readiness: Extended retention ensures security teams can analyze long-term attack campaigns.
  • Proactive detection: AI-driven analytics surface emerging attack patterns earlier.
  • Regulatory alignment: Supports compliance with standards requiring centralized, retained audit evidence.

Trade-offs 

  • Increased storage costs: Extending retention and centralizing logs requires scalable infrastructure.
  • Access governance complexity: Centralized investigator accounts must be strictly controlled to prevent misuse.
  • Operational overhead: Teams must maintain log libraries, pipelines, and monitoring tools across services.
  • Signal-to-noise balance: Centralized logging increases data volume, requiring investment in advanced filtering and analytics.

Key success factors

To track success, measure the following: 

  • Percentage of services using the standardized logging library.
  • Number of pipelines and services successfully forwarding logs to the central repository.
  • Mean time to investigate (MTTI) reduced by centralized access.
  • Compliance with two-year audit log retention standard.
  • Detection accuracy improvements from AI/ML-enhanced analytics.

Summary

Centralizing access to security logs transforms fragmented monitoring data into actionable intelligence. By standardizing log formats, consolidating collection, extending retention, and applying AI-driven analytics, Microsoft has improved both the speed and effectiveness of its threat detection and investigation.

Organizations can adopt this approach by ensuring consistent log formats, enforcing central storage and immutability, extending retention for forensic readiness, and integrating logs with advanced analytics platforms. These steps enable security teams to monitor and detect threats with clarity and precision---turning logs from scattered records into a powerful, unified security capability.

Centralize your security logs today to reduce blind spots, accelerate investigations, and stay ahead of evolving threats.

  • Last updated on