Tips
In today’s rapidly evolving DevSecOps landscape, the need for real-time visibility into developer activity and security events has never been more critical. GitHub, as a central platform for code collaboration, generates a wealth of audit log data that can provide deep insights into repository changes, user actions, and potential security threats. However, these logs are often underutilized due to lack of native visualixation. To address this gap, Microsoft has introduced a powerful integration strategy that streams GitHub audit logs into Azure Log Analytics using a lightweight, low-latency pipeline. This blog explores the key observability trends, challenges, and strategic rationale behind this integration, and how it aligns with enterprise security goals.Key Observability Trends Around GitHub Security
Modern enterprises are increasingly adopting DevSecOps practices, integrating security into every phase of the development lifecycle. Key observability trends include:
- Unified Security Dashboards: Organizations seek a single pane of glass to monitor GitHub security events—code scanning, secret leaks, alongside audit logs.
- Real-Time Visibility: There is a growing demand for near real-time ingestion and visualization of GitHub audit logs and security alerts .
- Operationalizing GitHub Advanced Security (GHAS): While GHAS provides powerful scanning capabilities, enterprises want to surface these insights beyond GitHub’s UI to security and compliance teams.
Challenges in Displaying All Security in One Dashboard
Despite GitHub’s robust security features, customers face several challenges:
- Siloed Data: GitHub’s native alerts are confined to its interface, limiting visibility for security teams who operate outside the development environment.
- Limited Audit Log Retention and Analytics: GitHub retains audit logs for a limited time and lacks advanced analytics, making long-term trend analysis and compliance reporting difficult.
- High Volume and Noise: Audit logs generate large volumes of events, many of which are routine. Without filtering and categorization, meaningful insights are buried in noise.
- Alert Fatigue: Without proper tuning, teams may be overwhelmed by false positives or low-priority alerts, leading to desensitization.
Why Visualizing GitHub Audit Logs in Defender Makes Sense
Integrating GitHub audit logs into Microsoft Defender offers several advantages:
- Centralized Visibility: Security teams gain access to GitHub events without needing GitHub access, enabling broader insights.
- Enhanced Governance: Audit logs are archived centrally, enabling compliance tracking and forensic analysis.
- Real-Time Response: Alerts can trigger automated actions—like revoking secrets or creating incidents—via Power Automate or Logic Apps.
- Complement to GHAS: While GHAS provides detection, this solution ensures findings are acted upon and tracked across the enterprise.
Microsoft’s solution strategically aligns GitHub and Azure security under a unified framework:
- End-to-End Integration: GitHub (Dev), Azure (Cloud), and Microsoft Defender (Sec) are connected, creating a seamless DevSecOps pipeline.
- Modular Architecture: The solution supports both Log Analytics workspace and Sentinel for visualization and integrates with existing tools like Microsoft Defender for Cloud.
- Data Sovereignty and Security: All data remains within the customer’s Azure tenant, ensuring compliance and control.
- XDR Strategy Alignment: By bringing GitHub security signals into Microsoft’s Extended Detection and Response (XDR) ecosystem, the solution enhances threat correlation and response across the enterprise.
The rationale is rooted in addressing a critical visibility and response gap:
- Customer Demand: Enterprises have explicitly requested real-time GitHub audit analytics and integration with existing security workflows.
- High ROI: The cost of implementation is low compared to the potential savings from avoided incidents and improved compliance.
- Strategic Differentiation: Only Microsoft can offer this native, end-to-end integration across GitHub, Azure, and M365, making it a compelling value proposition for customers.
In this blog we will share how GitHub audit logs can be moved into Azure with seconds-level latency, strong observability, and full control. If you care about centralizing GitHub security signals in Azure, this path keeps the pipeline short and the feedback loop tight.
If you’re exploring DevOps security on Azure, this overview is a good primer: https://learn.microsoft.com/en-us/azure/defender-for-cloud/defender-for-devops-introduction
The Roads We Tried but Skipped
- Event Hub + Function App: We built this first. It worked, but end-to-end latency could be unpredictable, and troubleshooting across services felt a bit “black box”.
- Microsoft Sentinel GitHub connector: Useful, but it’s a pull model with minutes-level delay. If you want near-real-time monitoring and alerts, it’s not ideal.
The Path That Worked: GitHub Streaming → HEC Simulator → DCR → LAW
We use GitHub Audit Log Streaming (push) to hit a Splunk HEC–compatible webhook (a small “HEC simulator” service). The service stores a local copy and forwards events directly to Azure Log Analytics via Logs Ingestion API (DCR/DCE). Short path, low latency, and easy to see what’s going on.
Architecture:
Five Steps to Roll Out
Recommended schema:
created_atin milliseconds)_document_id)
-
- Keep these handy: DCE endpoint, DCR immutableId, stream name (often
Custom-Json-github_auditlogs_CL), and an Azure AD app with rights to write to the DCR.
- Keep these handy: DCE endpoint, DCR immutableId, stream name (often
4. Run the Splunk HEC Simulator service
-
- Open-source implementation: https://github.com/satomic/github-auditlogs-streaming-splunk-2-log-analytics
- This repo includes the same service. It exposes standard HEC endpoints (
/services/collector/event,/raw,/health), saves a local copy for audit, and forwards to LAW via DCR.
5. Enable GitHub Audit Log Streaming
-
- Path:
Settings -> Audit log -> Audit log streaming - Target type: Splunk (HEC). Provide your host/IP, port (default 8088), and token matching
VALID_TOKENS. TLS verification depends on whether you run the simulator with HTTPS.
- Path:
Finally, create the Dashboard in Defender and configure it appropriately.
What Helped Us In The Field
- Tokens and TLS: Use strong tokens and HTTPS in production; HTTP is fine to validate the pipeline in dev.
- DCR 401/403: Usually credentials or permissions. Double-check DCR immutableId, stream name, and the app permissions.
- Latency and throughput: The HEC service is async and handles concurrency well. Keep local archiving as a safety net and monitor LAW ingestion rate.
- Observability: Build small KQL views for ingestion rate and failure counts. The service exposes
/healthand/stats.
Solution Comparison
|
Approach
|
Model
|
End-to-End Latency
|
Observability
|
Pros
|
Cons
|
When to Use
|
|
Event Hub + Function App
|
Push (multi-service chain)
|
Medium; can fluctuate
|
Fair (cross-component)
|
Mature ecosystem, elastic, extensible
|
Longer chain, black-box feel, scattered debugging
|
Existing EH/FA footprint; not strict real-time
|
|
Microsoft Sentinel Connector
|
Pull
|
Minutes
|
Good (central in Sentinel)
|
Turnkey; rules/workbooks
|
Not real-time; weak for “seconds-level alerts”
|
Compliance and centralized analysis
|
|
GitHub Streaming → HEC → DCR
|
Push (short path)
|
Typically seconds
|
Strong (service-level visibility)
|
Low latency, controllable, easy to customize
|
Requires operating a small service and credentials
|
Near real-time and high customization
|
Conclusion
The integration of GitHub audit logs into Microsoft Defender for Cloud represents a significant leap forward in enterprise DevSecOps maturity. By overcoming the limitations of native audit log visualization, this solution empowers security and compliance teams with real-time, centralized, and actionable insights. It not only enhances visibility and governance but also aligns with Microsoft’s broader XDR strategy, unifying GitHub, Azure, and Microsoft Defender into a cohesive security ecosystem.
With a low-code, high-impact approach, this solution is both scalable and customizable, offering enterprises a cost-effective way to mitigate risks, improve compliance, and respond swiftly to security events. As organizations continue to prioritize secure software development, this integration stands out as a strategic enabler for modern, resilient, and intelligent security operations.
References
- https://learn.microsoft.com/en-us/azure/defender-for-cloud/defender-for-devops-introduction
- https://learn.microsoft.com/en-us/azure/azure-monitor/logs/quick-create-workspace?tabs=azure-portal
- https://learn.microsoft.com/en-us/azure/azure-monitor/logs/create-custom-table?tabs=azure-portal-1,azure-portal-2,azure-portal-3
- https://learn.microsoft.com/en-us/azure/azure-monitor/logs/tutorial-logs-ingestion-portal
- https://github.com/satomic/github-auditlogs-streaming-splunk-2-log-analytics
0 comments
Be the first to start the discussion.