Auditing outbound LLM traffic without breaking dev environments.

Pump the brakes. Block all services that aren’t secured. All your data and IP are being scattered to the wind, nothing is being kept in a silo. It’s the Wild West. Identify providers. Provision a tenant there. Carve out access and acceptable use policies. Block everything else. Use SSL inspection to handle prompt logging if necessary, but at this point you have a large mess to try to clean up. Hopefully no one’s stuck data out there that they shouldn’t have.

Counterpoint: YOLO!

Litellm, bifrost, llmgateway, portkey ai gateway There are too many to list. The first three I've listed are open source and have paid options for a business. At the very least, any of these will help you find others when googling.

Firewall (palo). Umbrella/Secure access also works.

An LLM gateway is another option to take into consideration.

Others got that covered before i could reply. When i researched, portkey seemed like a solid option.

Thanks chatgpt

Since this is a Microsoft-managed enterprise environment, I would use the Microsoft security stack before deploying a heavy SSL-inspection proxy. The key distinction is: Intune deploys configuration and security policies. Microsoft Defender for Endpoint records endpoint network activity. Defender for Cloud Apps classifies known cloud and generative-AI services. Defender Vulnerability Management can inventory supported browser extensions, with the required additional licensing. Microsoft Purview Endpoint DLP can inspect supported browser data-sharing scenarios. Intune by itself does not discover which API endpoints developers are using. 1. Onboard the developer devices into Defender for Endpoint Use Intune’s Endpoint Detection and Response policy to onboard the managed devices into Microsoft Defender for Endpoint. After deployment, verify the onboarding status in Intune rather than assuming that successful policy assignment means every device is actively reporting. 2. Find outbound destinations with Advanced Hunting In the Microsoft Defender portal, open: Hunting → Advanced Hunting Start with DeviceNetworkEvents: DeviceNetworkEvents | where Timestamp > ago(7d) | where RemotePort == 443 | where isnotempty(RemoteUrl) or isnotempty(RemoteIP) | extend Destination = iff(isnotempty(RemoteUrl), RemoteUrl, RemoteIP) | summarize Connections = count(), Devices = dcount(DeviceId), Users = dcount(InitiatingProcessAccountUpn), FirstSeen = min(Timestamp), LastSeen = max(Timestamp) by Destination, RemoteUrl, RemoteIP, InitiatingProcessFileName, InitiatingProcessCommandLine | order by Devices desc, Connections desc The important correction here is not to use: | where isnotempty(RemoteUrl) by itself. That would discard records where Defender captured the remote IP but did not populate the destination domain. This query can produce relationships such as: code.exe → api.vendor.example node.exe → extension-backend.example python.exe → model-gateway.example msedge.exe → chat-service.example chrome.exe → browser-sidebar.example Do not initially limit the query to well-known AI companies. An extension may send data through: Its own vendor backend A generic Azure, AWS, Google Cloud or CDN hostname A private relay A locally launched helper process A destination that has not yet been classified as an AI service Also, do not assume every VS Code extension connection will appear as code.exe. An extension can launch node.exe, python.exe, java.exe or another native helper process. 3. Understand what the telemetry proves Defender network telemetry can help establish: Which managed device made the connection Which user and process were associated with it The destination domain or FQDN when available The remote IP address and port The connection time and frequency The initiating process command line when available It generally does not reveal: The full HTTPS request path The API operation being called The prompt or request body The exact source-code lines submitted Whether a secret or token appeared inside the encrypted request Therefore, call these destination hosts or remote endpoints, not necessarily complete API endpoints. Without content inspection before encryption or TLS decryption, the network event cannot prove exactly what was submitted. 4. Use Defender for Cloud Apps for Shadow AI discovery Integrate Defender for Cloud Apps with Defender for Endpoint and review: Cloud Apps → Cloud Discovery → Discovered apps Filter the application category for generative AI. This can help identify recognized AI services, affected users and devices, usage patterns and Microsoft’s catalog classification. Applications can be tagged as: Sanctioned Monitored Unsanctioned A monitored application can warn users while allowing them to continue. An unsanctioned application can be blocked through the Defender for Endpoint integration. Blocking has prerequisites. These include the necessary licenses, Defender for Endpoint onboarding, Microsoft Defender Antivirus protections, cloud-delivered protection, Network Protection in block mode, custom network indicators and the Defender for Cloud Apps integration. Also remember that Cloud Discovery depends on recognizing the service or its domains. A new vendor, custom API, private relay or generic cloud endpoint might not be identified as generative AI. The raw Advanced Hunting results are still necessary. 5. Inventory browser extensions A browser connection may appear only as msedge.exe, chrome.exe or firefox.exe. The network event might not identify the individual extension that initiated it. Defender Vulnerability Management provides a browser-extension inventory for Windows devices covering Microsoft Edge, Chrome and Firefox. However, this feature requires either: Microsoft Defender Vulnerability Management Standalone, or The Defender Vulnerability Management add-on for eligible Defender for Endpoint Plan 2 customers The inventory can show the extension, browser, affected users and devices, whether it is enabled and its requested permissions. A powerful permission is a risk indicator, not proof of leakage. The extension still needs to be investigated and reproduced. 6. Perform a controlled reproduction For each suspicious extension: Select a test device. Record the exact start time. Close unrelated browser and IDE sessions. Perform one harmless, identifiable operation through the extension. Record the completion time. Review Defender network events from that time window. Disable the extension. Repeat the same workflow. Compare the resulting processes and destinations. This provides stronger evidence that an extension owns a connection than simply observing that the browser or IDE contacted a domain. A short pktmon or Wireshark capture can also help confirm timing, IP addresses and connection behavior during the controlled test. It still will not reveal the HTTPS request body without decryption. 7. Control Edge extensions through enterprise policy After the audit, use Intune or another supported device-management method to deploy Microsoft Edge policies. You can: Block extension installation by default Allow approved extension IDs Block specific extensions Force-install required extensions Restrict extension permissions Restrict the websites an extension may access Establish an extension-request and approval process Roll this out to a pilot group first. Blocking all extensions immediately can disrupt legitimate engineering workflows. 8. Control VS Code extensions directly VS Code supports an organization-managed AllowedExtensions policy beginning with VS Code 1.96. Organizations can allow or block extensions by: Publisher Specific extension ID Version Platform Once the policy is configured, unlisted extensions can be prevented from installing. An installed extension that becomes disallowed is disabled. This is one of the strongest controls for the original problem because it addresses the IDE extension itself instead of trying to maintain an incomplete list of vendor domains. Also consider an internal or private extension marketplace for approved extensions. 9. Use Purview only for the scenarios it actually covers Microsoft Purview Endpoint DLP can audit, warn or block sensitive information being pasted or uploaded through supported browser scenarios on managed devices. This can produce a useful signal such as: A user attempted to paste content matching a sensitive-information policy into an unmanaged generative-AI website. However, this should not be described as universal inspection of all AI traffic. A VS Code extension sending source code through code.exe, node.exe or its own helper process is not automatically covered by browser DLP. Purview coverage also depends on the operating system, browser, application, policy configuration and the organization’s applicable Purview and Endpoint DLP licensing. 10. Provide a sanctioned path for IDE-based AI For IDE integrations, the stronger long-term design is: Allow only reviewed IDE extensions Use company-managed identities or API credentials Provide an approved enterprise AI service Route sanctioned calls through a company-controlled AI gateway where practical Log metadata such as user, provider, model, time, repository classification and policy decision Avoid storing full prompts unless there is a clearly defined security and retention requirement Otherwise, blocking individual domains will become an endless exercise because extensions can change backends or route through shared cloud infrastructure. Recommended rollout Audit: Onboard a pilot group and collect Defender network events, discovered cloud applications and extension inventories. Correlate: Map users, devices, processes, extensions, destination domains and remote IP addresses. Reproduce: Validate suspicious extensions with controlled testing. Classify: Review vendor ownership, permissions, retention terms, authentication method and business purpose. Enforce: Apply Edge and VS Code extension policies, Defender for Cloud Apps governance and Purview DLP where supported. The Microsoft stack can provide strong visibility into devices, processes, extensions and outbound destinations without decrypting every developer connection. But it is important not to overpromise: network telemetry alone cannot prove exactly what source code or prompt was transmitted.

This comment was deleted.

for the discovery part you probably don't need SSL inspection at all - most of these tools phone home to known hostnames (api.openai.com, the copilot/cursor/anthropic endpoints, etc), so egress DNS logging plus a maintained domain list gets you most of the visibility with zero MITM. the IDE extensions and browser sidebars still resolve DNS even when they dodge your L7 blocklists. inspection only becomes worth the pain once you want to block payloads rather than just see who's hitting what.

Question one: what do your AUP / AI policies say that your staff are allowed or not allowed to do?

dns egress logging catches which endpoints got hit, just not how much data went with it. a quick question and someone pasting a whole repo read the same in the logs. if you need to know what actually leaked youre back to the proxy you're avoiding.

SSL inspection is probably the wrong starting point for this problem. It can provide deep visibility, but it is expensive and operationally heavy. You need inspection infrastructure, trusted certificate deployment, capacity planning, application exceptions, certificate-pinning bypasses, privacy controls, and ongoing maintenance. It can also introduce latency or break developer tools that do not tolerate intercepted TLS connections. For the initial goal—finding which AI services, browser extensions, and IDE tools are making outbound connections—full SSL decryption is disproportionate. A much faster and cheaper approach is to use the Microsoft tooling that is already designed for managed enterprise endpoints: Use Intune to onboard and configure the developer devices. Use Microsoft Defender for Endpoint to collect process-to-destination network telemetry. Use Defender for Cloud Apps to identify recognized generative-AI services. Use Edge and VS Code enterprise policies to inventory, allow, block, or disable unapproved extensions. This should quickly show which device, user, process, domain, IP address, and extension or agent are involved—without decrypting every HTTPS connection. It will not reveal the exact prompt or source code inside the encrypted request, but that level of inspection is not necessary for the first phase. First identify the tools and destinations, restrict unapproved extensions, and provide an approved enterprise AI route. For sanctioned AI usage that truly requires request-level auditing, route it through a company-controlled AI gateway. That gives much more reliable logging than trying to intercept every developer connection. So I would treat SSL inspection as a narrow, last-resort investigation tool—not the default architecture. Intune, Defender, and Microsoft enterprise extension policies should get useful results much faster, with lower cost and significantly less disruption.

you would get a lot AI slop in response. if you just want to see where your developers going - use dns. something like adguard dns will give you stats without intercepting traffic.