Detecting GenAI Attacks in Splunk: Five Use Cases from the OWASP Top 10 for LLMs

Almost every company I speak to these days has pushed some LLM application into production. A chatbot for support, a RAG assistant sitting on top of the internal wiki, maybe an agent that raises tickets on its own. Very good. But when I ask the SOC team one simple question — “if someone attacks that application tonight, will you see it?” — the room goes quiet.

That gap is what this post is about.

The OWASP Top 10 for LLM Applications (2025) gives us a common language for the risks. Splunk is where we convert that language into alerts an analyst can actually act upon. Below are five use cases I have built, with the SPL, the data you must onboard first, and the tuning traps I walked straight into so that you need not.

First things first: get the telemetry in

Detection content without logs is only a PowerPoint slide. Before writing a single search, make sure your LLM gateway or proxy — LiteLLM, Portkey, Azure API Management, Bedrock invocation logging, or whatever wrapper your team has written — is emitting one event per inference call with these fields:

Field	Why you need it
`user`, `tenant_id`, `session_id`	Attribution and cross-tenant checks
`app`, `model`, `endpoint`	Scoping and baselining
`prompt`, `response`	Content inspection (hash or truncate if policy demands)
`system_prompt_version`	Tying a leak back to a specific prompt build
`tokens_in`, `tokens_out`, `latency_ms`	Consumption baselines
`tool_calls`, `retrieved_doc_ids`	Agent and RAG traceability

A simple index layout that has worked well for me:

index=llm_gateway    sourcetype=llm:inference
index=llm_vectordb   sourcetype=vectordb:query
index=llm_pipeline   sourcetype=rag:ingestion

One word of caution before you rush ahead. Prompts and responses are user content. Please sit with your privacy and legal folks first and agree on retention, field masking and role-based access. In two of my engagements the final answer was to index only the matched snippet plus a hash of the full payload, and not the raw text at all. That conversation takes longer than the engineering, so start it early.

LLM01: Prompt Injection

The risk. Somebody crafts an input that overrides your system instructions. Either directly in the chat window, or indirectly through content the model retrieves — a poisoned document, a web page, the body of an email.

How I approach it. Keyword and regex matching on known jailbreak grammar, but scored rather than yes-or-no. One match is a weak signal. Three matches in a single prompt, or a dozen attempts from the same user in an hour, is a different story altogether.

Keep the patterns in a lookup so your analysts can tune without touching SPL:

llm_injection_patterns.csv

pattern,technique,weight
(?i)ignore (all |any )?(previous|prior|above) instructions,instruction_override,5
(?i)disregard (your|the) (system|initial) prompt,instruction_override,5
(?i)you are now (in )?(DAN|developer mode|unrestricted),persona_hijack,5
(?i)(repeat|print|reveal|output) (your|the) (system prompt|instructions|rules),prompt_extraction,4
(?i)\bdo anything now\b,persona_hijack,4
(?i)pretend (you are|to be) (an? )?(unfiltered|uncensored),persona_hijack,3
(?i)</?(system|instruction|admin)>,delimiter_injection,4
(?i)base64 decode and (execute|follow),encoding_evasion,4
(?i)for (educational|research) purposes only.{0,40}(bypass|ignore),social_framing,2

And the correlation search:

index=llm_gateway sourcetype=llm:inference
| rex field=prompt max_match=0 "(?i)(?<hit>ignore (?:all |any )?(?:previous|prior|above) instructions|disregard (?:your|the) (?:system|initial) prompt|you are now (?:in )?(?:DAN|developer mode|unrestricted)|(?:repeat|print|reveal|output) (?:your|the) (?:system prompt|instructions|rules)|do anything now|</?(?:system|instruction|admin)>)"
| eval hit_count=mvcount(hit)
| where hit_count>0
| eval severity=case(hit_count>=3,"high", hit_count=2,"medium", 1=1,"low")
| stats sum(hit_count) as total_hits,
        dc(session_id) as sessions,
        values(hit) as techniques,
        latest(severity) as severity,
        values(app) as apps
        by user
| where total_hits>=3 OR severity="high"
| sort - total_hits

The one that actually worries me

The dangerous variant never touches the user’s keyboard at all. It arrives inside the content your RAG pipeline retrieves. So look at retrieved content, not user input:

index=llm_gateway sourcetype=llm:inference retrieved_content=*
| rex field=retrieved_content max_match=0 "(?i)(?<inject>ignore (?:previous|all) instructions|new instructions:|assistant must|system override)"
| where mvcount(inject)>0
| stats count, values(inject) as markers, values(user) as affected_users by retrieved_doc_id, source_repository
| sort - count

If this fires, it means a document sitting in your knowledge base is carrying a payload that will be served to every user who asks the right question. That is an incident to be opened, not an alert to be tuned away.

Tuning notes. Your red teamers, your AI security channel and anyone writing prompt engineering documentation will set this off continuously. Maintain an allowlist of testing accounts. And do expect false positives from people quoting articles about prompt injection — which, quite amusingly, includes anybody who shares this very blog post in your internal chat.

LLM02: Sensitive Information Disclosure

The risk. The model returns something it had no business returning. PII from a fine-tuning set, another customer’s record surfaced through RAG, or an API key that somebody pasted into a Confluence page three years back and which got indexed last month.

How I approach it. Pattern matching on the response side, not the prompt side. The prompt only tells you what the person wanted. The response tells you what actually walked out of the door.

index=llm_gateway sourcetype=llm:inference
| rex field=response max_match=0 "(?<secret_aws>AKIA[0-9A-Z]{16})"
| rex field=response max_match=0 "(?<secret_pk>-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----)"
| rex field=response max_match=0 "(?<secret_bearer>(?i)(?:api[_-]?key|secret|token|passwd|password)\s*[:=]\s*[\"']?[A-Za-z0-9_\-]{20,})"
| rex field=response max_match=0 "(?<pii_email>[a-zA-Z0-9._%+-]+@(?!example\.com)[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})"
| rex field=response max_match=0 "(?<pii_iban>\b[A-Z]{2}[0-9]{2}[A-Z0-9]{11,30}\b)"
| rex field=response max_match=0 "(?<pii_natid>\b[12][0-9]{9}\b)"
| rex field=response max_match=0 "(?<pii_cc>\b(?:4[0-9]{12}(?:[0-9]{3})?|5[1-5][0-9]{14}|3[47][0-9]{13})\b)"
| eval secret_count = mvcount(secret_aws)+mvcount(secret_pk)+mvcount(secret_bearer)
| eval pii_count = mvcount(pii_email)+mvcount(pii_iban)+mvcount(pii_natid)+mvcount(pii_cc)
| where secret_count>0 OR pii_count>2
| eval disclosure_type = case(secret_count>0,"credential", pii_count>5,"bulk_pii", 1=1,"pii")
| table _time user app model disclosure_type secret_count pii_count retrieved_doc_ids session_id
| sort - _time

Do run your card matches through a Luhn check before they reach the analyst queue, or at least route them into a risk index instead. Order numbers and epoch timestamps throw up a surprising number of sixteen-digit false positives, and nothing kills a new detection faster than an analyst closing the same rubbish ticket five days running.

The higher-value query, honestly, is the quiet one — a single user slowly pulling records at volume:

index=llm_gateway sourcetype=llm:inference
| rex field=response max_match=0 "(?<pii_email>[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})"
| eval pii_this_response=mvcount(pii_email)
| bin _time span=1h
| stats sum(pii_this_response) as pii_returned, count as queries, dc(retrieved_doc_ids) as docs_touched by user, _time
| eventstats avg(pii_returned) as avg_pii, stdev(pii_returned) as stdev_pii by user
| eval zscore=round((pii_returned-avg_pii)/(stdev_pii+0.001),2)
| where pii_returned>50 AND zscore>3

LLM04: Data and Model Poisoning

The risk. Somebody modifies what the model learns from — a fine-tuning dataset, an embedding source, a document in the RAG corpus. After that the model very confidently serves the attacker’s content to every single user, and it looks perfectly legitimate while doing so.

How I approach it. This one has nothing to do with text analysis. It is plain old change monitoring, applied to a new class of asset. Your training buckets, vector collections and RAG source repositories are crown jewels now. Treat them exactly as you treat a domain controller.

Unauthorised writes to training and RAG storage:

(index=aws_cloudtrail eventSource=s3.amazonaws.com
   (eventName=PutObject OR eventName=DeleteObject OR eventName=CopyObject)
   requestParameters.bucketName IN ("ml-training-data","rag-corpus-prod","embeddings-source"))
| rename userIdentity.arn as actor, requestParameters.key as object_key
| lookup ml_authorized_writers.csv actor OUTPUT is_authorized, team
| eval is_authorized=if(isnull(is_authorized),"no",is_authorized)
| where is_authorized="no"
| stats count as changes,
        values(eventName) as actions,
        values(object_key) as objects,
        min(_time) as first_seen,
        max(_time) as last_seen
        by actor, requestParameters.bucketName, sourceIPAddress
| convert ctime(first_seen) ctime(last_seen)
| sort - changes

Off-hours and volume anomalies on the ingestion pipeline:

index=llm_pipeline sourcetype=rag:ingestion action=index_document
| eval hour=strftime(_time,"%H"), day=strftime(_time,"%A")
| eval offhours=if(hour<6 OR hour>19 OR day IN ("Friday","Saturday"),1,0)
| bin _time span=1h
| stats count as docs_ingested, sum(offhours) as offhours_docs, values(source_repo) as repos by ingest_user, _time
| eventstats avg(docs_ingested) as baseline, stdev(docs_ingested) as sd by ingest_user
| eval deviation=round((docs_ingested-baseline)/(sd+0.001),2)
| where (deviation>3 AND docs_ingested>100) OR offhours_docs>0

Please do change the weekend logic to suit your region. The query above assumes a Friday–Saturday weekend, which will be wrong in half the world.

And a content integrity check, which catches the silent edits that bypass your pipeline altogether:

index=llm_pipeline sourcetype=rag:ingestion
| stats latest(content_hash) as current_hash, latest(_time) as last_indexed, dc(content_hash) as hash_versions by doc_id
| where hash_versions>1
| lookup rag_baseline_hashes.csv doc_id OUTPUT approved_hash, owner
| where current_hash!=approved_hash
| table doc_id owner approved_hash current_hash last_indexed

LLM07: System Prompt Leakage

The risk. The model happily reveals its own instructions. And those instructions very often contain business logic, internal endpoint names, a description of your guardrails, and occasionally a credential that should never have been sitting there in the first place.

How I approach it. Canary tokens. Honestly, this is the single best detection in this entire post, and it is embarrassingly simple.

Embed a unique meaningless string in every system prompt version — something like REF-7Q4XZ-SYSGUARD. Then alert on any response containing it. No fuzzy matching, no false positives, and instant attribution to a specific prompt build.

index=llm_gateway sourcetype=llm:inference
| lookup system_prompt_canaries.csv system_prompt_version OUTPUT canary_token, prompt_owner
| eval leaked=if(match(response, canary_token),1,0)
| where leaked=1
| stats count as leak_events,
        dc(user) as users,
        values(user) as who,
        values(app) as apps,
        earliest(_time) as first_leak
        by system_prompt_version, prompt_owner
| convert ctime(first_leak)

Run a structural check alongside it, for responses that read like instructions rather than answers:

index=llm_gateway sourcetype=llm:inference
| rex field=response max_match=0 "(?i)(?<marker>you are (?:a|an|the) [a-z ]{3,40}(?:assistant|agent|bot)|your (?:instructions|rules|guidelines) are|never (?:reveal|disclose|mention) (?:that|your|the)|do not (?:discuss|answer|respond to)|system:\s|<\|system\|>)"
| eval marker_count=mvcount(marker)
| where marker_count>=2
| eval response_len=len(response)
| stats count as suspicious_responses, values(marker) as markers, avg(response_len) as avg_len by user, app, session_id
| where suspicious_responses>=2

Then pair it with the extraction attempts from LLM01. A user who asked to see the system prompt and then received a response stuffed with instruction markers is a confirmed leak, not a maybe:

index=llm_gateway sourcetype=llm:inference
| eval asked = if(match(lower(prompt), "(repeat|print|reveal|show|output|echo).{0,20}(system prompt|instructions|initial message|rules)"),1,0)
| eval leaked = if(match(lower(response), "(you are a|your instructions are|never reveal|do not discuss)"),1,0)
| where asked=1 AND leaked=1
| table _time user app model session_id prompt response

LLM08: Vector and Embedding Weaknesses

The risk. Your vector database is a datastore, and many teams simply never got around to putting proper authorisation in front of it. Attackers enumerate it, pull the neighbours of sensitive embeddings, invert those embeddings back into approximate source text, or — most commonly of all — read another tenant’s namespace because the tenant filter was applied in the application layer and not in the database.

How I approach it. Two very different behaviours: cross-tenant access, and enumeration.

Cross-tenant retrieval is the highest-fidelity detection in this whole post, because it compares two facts that should never, ever disagree:

index=llm_vectordb sourcetype=vectordb:query
| lookup user_tenant_map.csv user OUTPUT tenant_id as user_tenant
| eval queried_tenant=coalesce(filter_tenant_id, namespace, collection_tenant)
| where isnotnull(user_tenant) AND isnotnull(queried_tenant) AND user_tenant!=queried_tenant
| stats count as violations,
        values(queried_tenant) as accessed_tenants,
        dc(queried_tenant) as tenant_count,
        values(collection) as collections,
        values(src_ip) as source_ips
        by user, user_tenant
| sort - violations

Any result here is an authorisation failure. Please treat the very first occurrence as a P2 and resist all requests to tune it down.

Enumeration and bulk retrieval, where the giveaway is an abnormally large top_k or a rapid run of near-identical queries:

index=llm_vectordb sourcetype=vectordb:query
| eval top_k=coalesce(top_k, limit, n_results)
| bin _time span=10m
| stats count as queries,
        sum(top_k) as vectors_returned,
        max(top_k) as max_topk,
        dc(collection) as collections_touched,
        dc(query_vector_hash) as unique_vectors
        by user, _time
| eval repetition_ratio=round(queries/(unique_vectors+0.001),2)
| eventstats avg(vectors_returned) as baseline_vectors, stdev(vectors_returned) as sd_vectors by user
| eval zscore=round((vectors_returned-baseline_vectors)/(sd_vectors+0.001),2)
| where (max_topk>200) OR (vectors_returned>5000 AND zscore>3) OR (queries>100 AND repetition_ratio>10)
| eval reason=case(max_topk>200,"oversized_topk",
                   repetition_ratio>10,"probing_similar_vectors",
                   1=1,"bulk_retrieval")
| table _time user queries vectors_returned max_topk collections_touched reason zscore

And keep an eye on anybody hitting the vector store directly, bypassing your application entirely:

index=llm_vectordb sourcetype=vectordb:query
| lookup approved_vectordb_clients.csv user_agent OUTPUT is_approved
| where isnull(is_approved) OR is_approved="false"
| stats count, values(collection) as collections, values(user_agent) as agents by src_ip, user
| where count>10

Tying it together with risk-based alerting

Ten separate alerts will give you ten separate tickets and one thoroughly fed-up analyst. Risk-based alerting suits this problem much better, because LLM abuse is almost always a sequence of low-confidence signals from one identity, not a single screaming event.

Attach a risk annotation to each of the searches above:

... your detection logic ...
| eval risk_object=user, risk_object_type="user"
| eval risk_score=case(
    detection="llm01_prompt_injection", 20,
    detection="llm02_credential_disclosure", 60,
    detection="llm04_unauthorized_corpus_write", 70,
    detection="llm07_canary_leak", 80,
    detection="llm08_cross_tenant", 90,
    1=1, 15)
| eval risk_message="OWASP LLM detection: ".detection." for ".user
| collect index=risk

Then alert on accumulation instead of on individual events:

index=risk source="*llm*" earliest=-24h
| stats sum(risk_score) as total_risk,
        dc(detection) as distinct_detections,
        values(detection) as detections
        by risk_object
| where total_risk>100 AND distinct_detections>=2
| sort - total_risk

A user who probed for the system prompt, then tripped a PII pattern, then went poking at a collection they have never touched before has told you a complete story. Taken one by one, every single one of those events would have been closed as noise.

What I would do differently, if I were starting again

Onboard the gateway before building anything. I lost a full week writing detections against a schema I had assumed, then rewrote every last one of them when the real logs turned up with different field names. Map to the Common Information Model once, at the beginning, and save yourself the pain.

Settle the prompt retention policy on day one. Content inspection needs content. That discussion with privacy and legal will take longer than the engineering, and it will change your design, so start it before you write your first search.

Canary tokens beat clever regex every time. The canary-based leakage detection has given me exactly zero false positives. The heuristic version gave me dozens in the first week itself.

Baselines need a fortnight of data. Every anomaly query above quietly assumes you know what normal looks like. Run them in report mode first, see what fires, then set your thresholds. Do not ship z-scores on day one.

If you build only one thing, build the cross-tenant check. LLM08’s tenant mismatch query is trivial logic, it has almost no false-positive surface, and it catches a failure mode that maps straight to a breach notification. Best return on effort in the entire list.

Reference: OWASP Top 10 for LLM Applications 2025

If you are building similar content in your own environment, I would genuinely like to compare notes — particularly on the agentic side, which is where things are getting messy fast.