Splunk Configuration Precedence: The Complete Cheat Sheet
September 29, 202611 min read2,480 words
TL;DR
- Index-time (global context):
system/local→apps/*/local→apps/*/default→system/default. Apps tie-break in ASCII order (0-9 → A-Z → a-z); first wins. - Search-time (app/user context): user → current app → other apps (exported only, reverse-lexicographic) → system.
- props.conf stanzas:
[source::]→[host::]→[sourcetype]→[default]. - Index-time pipeline: line breaking → timestamp → SEDCMD → TRANSFORMS → RULESET → index.
- Search-time: EXTRACT → REPORT → KV_MODE → FIELDALIAS → EVAL → LOOKUP → eventtypes → tags.
- Parsing happens once, on the first heavy forwarder or indexer. Search-time settings belong on the search head.
- Verify everything with
splunk btool <conf> list --debug.
Why precedence matters
Splunk merges every copy of a .conf file into one runtime view, and when two copies set the same attribute, precedence decides which value wins. Most “my props.conf isn’t working” tickets are precedence problems, not syntax problems.
There are four layers of “order” to keep in your head:
- Directory precedence — which copy of a file wins (system/local vs app/local, etc.).
- Stanza precedence — which stanza inside a file wins (
[source::]vs[host::]vs[sourcetype]). - Processing order — the order Splunk applies settings at index time and search time (line breaking before TRANSFORMS, EXTRACT before FIELDALIAS, and so on).
- Location — which Splunk tier actually reads the setting (UF, HF/indexer, search head).
Directory precedence itself runs in one of two contexts:
- Global context — index-time and system-wide settings (inputs, outputs, index-time props/transforms, indexes, server).
- App/user context — search-time settings evaluated for a specific user in a specific app (search-time props, savedsearches, eventtypes, tags, macros).
When in doubt, ask Splunk directly. btool shows the merged result and the file each line came from:
$SPLUNK_HOME/bin/splunk btool props list <stanza> --debug
$SPLUNK_HOME/bin/splunk btool transforms list <stanza> --debug
$SPLUNK_HOME/bin/splunk btool check
btool check validates every stanza and setting name against the .spec files and reports typos or unknown keys. It checks names, not values, and Splunk also runs it automatically at startup.
1. Global context (index-time) precedence
In the global context, system/local wins over everything, and system/default loses to everything.
$SPLUNK_HOME/etc/system/local <-- highest
$SPLUNK_HOME/etc/apps/<app>/local (all apps, A → Z)
$SPLUNK_HOME/etc/apps/<app>/default (all apps, A → Z)
$SPLUNK_HOME/etc/system/default <-- lowest (never edit)
Tie-breaker between apps: lexicographic (ASCII) order of the app folder name. The app that sorts first wins.
- ASCII order is: digits
0-9→ uppercaseA-Z→ lowercasea-z. - So
000_org_indexesbeatsA_org_props, which beatsSplunk_TA_windows, which beatsorg_all_props. - This is why admins prefix override apps with
000_,100_or an uppercase letter.
Order of evaluation across apps: all apps’ local directories are checked before any app’s default directory. A value in zzz_app/local beats a value in aaa_app/default.
Files that live in the global context include inputs.conf, outputs.conf, indexes.conf, server.conf, limits.conf, authentication.conf, and the index-time parts of props.conf and transforms.conf.
Never edit
system/defaultor an app’sdefaultdirectory. Upgrades overwrite them.
2. Indexer cluster peers and search head clusters
On an indexer cluster peer, apps pushed by the cluster manager (peer-apps, formerly slave-apps) get an extra slot at the very top.
$SPLUNK_HOME/etc/peer-apps/<app>/local <-- highest
$SPLUNK_HOME/etc/system/local
$SPLUNK_HOME/etc/apps/<app>/local
$SPLUNK_HOME/etc/peer-apps/<app>/default
$SPLUNK_HOME/etc/apps/<app>/default
$SPLUNK_HOME/etc/system/default <-- lowest
Practical rules for clusters:
- Manage peer config only from the cluster manager:
etc/manager-apps(formerlymaster-apps) → bundle push →etc/peer-appson every peer. - A setting in a peer’s
system/localcan still lose topeer-apps/<app>/local, so don’t “hotfix” a single peer there. - Search head cluster: the deployer pushes from
etc/shcluster/appsinto each member’setc/apps. By default the deployer merges the app’slocalintodefaulton push, so anything a user saves in the UI (memberlocal) wins over deployer content. - Deployment server: apps land in the client’s
etc/appsand follow normal global-context rules above.
3. App/user context (search-time) precedence
At search time the user’s private copy wins, then the app you are currently in, then everything else.
$SPLUNK_HOME/etc/users/<user>/<current_app>/local <-- highest
$SPLUNK_HOME/etc/apps/<current_app>/local
$SPLUNK_HOME/etc/apps/<current_app>/default
$SPLUNK_HOME/etc/apps/<other_app>/local (exported settings only)
$SPLUNK_HOME/etc/apps/<other_app>/default (exported settings only)
$SPLUNK_HOME/etc/system/local
$SPLUNK_HOME/etc/system/default <-- lowest
Three things that trip people up:
- The current app beats every other app regardless of its name.
- Other apps count only if their objects are exported (
export = systemin the app’smetadata/*.meta). A non-exported sourcetype extraction inTA_foois invisible when you search from the Search app. - Between other apps, the order is reverse-lexicographic — the opposite of the global context. An app named
Bbeats an app namedA. Don’t rely on this; give conflicting stanzas one owner.
Files in the app/user context include savedsearches.conf, macros.conf, eventtypes.conf, tags.conf, datamodels.conf, ui-prefs.conf, and the search-time parts of props.conf and transforms.conf.
Check from a specific user and app’s point of view:
splunk btool props list <sourcetype> --debug --app=<app> --user=<user>
4. Stanza precedence inside props.conf
After the files are merged, Splunk applies every stanza that matches an event and resolves only the attributes that conflict. The most specific stanza type wins:
[source::<source>] <-- highest
[host::<host>]
[<sourcetype>]
[default] <-- lowest (applies to everything)
Within the same stanza type (e.g. two [source::] stanzas that both match):
- A literal match beats a wildcard/regex match (
[source::/var/log/app.log]beats[source::/var/log/*.log]). - Between two pattern matches, the stanza name that sorts first in ASCII order wins.
- Override that ordering with the
priorityattribute — higher number wins:
[source::.../app/*.log]
priority = 10
TRUNCATE = 20000
[source::.../app/debug*.log]
priority = 20
TRUNCATE = 0
priority only ranks stanzas of the same type. It cannot make a [sourcetype] stanza beat a [source::] stanza.
Other stanza rules:
- Attributes merge across stanzas. If
[source::x]sets onlyTRUNCATEand[mysourcetype]setsTIME_FORMAT, both apply. - A specific stanza overrides the
[default]stanza and any global settings at the top of the file. - The same stanza name in two files is merged attribute by attribute, and each conflict is settled by directory precedence (sections 1–3).
- In
transforms.confthere’s no stanza precedence: a transform runs only when aTRANSFORMS-/REPORT-line in props.conf calls it by name.
5. Index-time processing order
Index-time settings run in a fixed pipeline order, and you can’t reorder it: line breaking → timestamp → SEDCMD → TRANSFORMS → RULESET → index.
| # | Pipeline | What happens | Key props.conf settings |
|---|---|---|---|
| 1 | Input | Source, host, sourcetype, index assigned | inputs.conf, [source::] sourcetype= |
| 2 | Parsing | Character encoding, raw line breaking, truncation | CHARSET, LINE_BREAKER, TRUNCATE |
| 3 | Merging (aggregator) | Line merging, timestamp extraction | SHOULD_LINEMERGE, BREAK_ONLY_BEFORE*, MUST_BREAK_AFTER, MAX_EVENTS, TIME_PREFIX, TIME_FORMAT, MAX_TIMESTAMP_LOOKAHEAD, TZ, DATETIME_CONFIG |
| 4a | Typing | Regex replacement on _raw |
SEDCMD-<class> |
| 4b | Typing | Index-time transforms: masking, routing, filtering, indexed fields, INGEST_EVAL |
TRANSFORMS-<class> |
| 4c | Typing | Ingest actions rulesets | RULESET-<class> |
| 5 | Indexing | Output routing, write to disk | _TCP_ROUTING, _SYSLOG_ROUTING, indexes.conf |
SEDCMD always runs before TRANSFORMS. If a TRANSFORMS regex looks for text that a SEDCMD already removed or masked, it will never match. You can’t change this order; match on the post-SEDCMD text, or do the edit inside a transform instead.
TRANSFORMS always runs before RULESET. When both apply to the same sourcetype, rulesets see the already-transformed event.
Order among classes
- Multiple
TRANSFORMS-<class>lines run in ASCII order of the class name, e.g.TRANSFORMS-01_routebeforeTRANSFORMS-02_mask. - Inside one class, the comma-separated list runs left to right. For queue routing the last matching transform wins:
# props.conf
[my:sourcetype]
TRANSFORMS-filter = setnull, keep_errors
# transforms.conf
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[keep_errors]
REGEX = ERROR
DEST_KEY = queue
FORMAT = indexQueue
Everything is dropped first, then events containing ERROR are put back. Reverse the order and you keep nothing.
- Multiple
SEDCMD-<class>lines also run in ASCII order of the class name, each on the output of the previous one. - Rules inside one ingest actions ruleset run top to bottom, and a sourcetype can have only one ruleset.
The sourcetype-rewrite trap
If a TRANSFORMS rewrites the sourcetype (DEST_KEY = MetaData:Sourcetype), the new sourcetype’s index-time settings do not run — parsing is already finished. Only its search-time settings apply. To process the event again under the new sourcetype, use CLONE_SOURCETYPE.
6. Search-time operation order
Search-time knowledge objects run in a fixed sequence, and each step can only use fields created by the steps before it.
| # | Operation | props.conf / file | Can reference fields from |
|---|---|---|---|
| 1 | Inline field extraction | EXTRACT-<class> |
_raw and indexed fields only |
| 2 | Transform-based extraction | REPORT-<class> → transforms.conf |
Step 1 |
| 3 | Automatic key/value extraction | KV_MODE (auto, json, xml, none) |
Steps 1–2 |
| 4 | Field aliases | FIELDALIAS-<class> |
Steps 1–3 |
| 5 | Calculated fields | EVAL-<field> |
Steps 1–4 |
| 6 | Lookups | LOOKUP-<class> |
Steps 1–5 |
| 7 | Event types | eventtypes.conf | Steps 1–6 |
| 8 | Tags | tags.conf | Steps 1–7 (incl. eventtypes) |
Rules that follow from this order:
- An
EXTRACTcan’t use a field produced byREPORTorKV_MODE. Use thein <field>syntax only on fields that already exist. - A
FIELDALIAScan’t alias a field created byEVALor a lookup. - An
EVAL-can use aliases but not otherEVAL-fields. Calculated fields are evaluated independently, so chain the logic in one expression. - A lookup can use calculated fields and aliases as input keys.
- Event types can search on lookup output, and tags can be applied to event types.
Order within one step
EXTRACT-,REPORT-,FIELDALIAS-andLOOKUP-classes run in ASCII order of the class name.- Inside one
REPORT-class, transforms run left to right. FIELDALIAS-x = src AS destoverwrites an existingdest;src ASNEW destleaves an existingdestalone.- Lookups use
OUTPUT(overwrite existing fields) orOUTPUTNEW(fill only empty fields).
7. Where each setting actually runs
A perfect config on the wrong tier does nothing. Parsing happens once, on the first full Splunk instance (heavy forwarder or indexer) the data reaches.
| Setting | Universal forwarder | Heavy forwarder / indexer | Search head |
|---|---|---|---|
inputs.conf, outputs.conf |
Yes | Yes | Only for its own logs |
EVENT_BREAKER_ENABLE, EVENT_BREAKER |
Yes | No | No |
INDEXED_EXTRACTIONS (CSV, JSON, W3C…) |
Yes | Yes (if not already parsed) | Also needed here for field display |
CHARSET, LINE_BREAKER, SHOULD_LINEMERGE, TRUNCATE |
No | Yes | No |
TIME_PREFIX, TIME_FORMAT, TZ, MAX_TIMESTAMP_LOOKAHEAD |
No | Yes | No |
SEDCMD-, TRANSFORMS-, INGEST_EVAL, RULESET- |
No | Yes | No |
EXTRACT-, REPORT-, KV_MODE, FIELDALIAS-, EVAL-, LOOKUP- |
No | No | Yes |
| eventtypes, tags, macros, savedsearches | No | No | Yes |
Consequences worth remembering:
- Data from a heavy forwarder arrives already parsed (cooked). The indexer skips line breaking, timestamping, SEDCMD and TRANSFORMS for it, so those settings must live on the HF.
- Data sent from a UF with
INDEXED_EXTRACTIONSis also parsed at the UF; index-time props on the indexer won’t reapply. - Search-time settings on an indexer only matter if the search head distributes them in the knowledge bundle — put them on the search head.
- Safe practice: deploy the same TA everywhere and let each tier use the part it reads.
8. Other precedence rules worth knowing
inputs.conf
blacklistbeatswhitelist. A file matching both is skipped.- Put the most specific settings on the most specific stanza. A
[default]stanza in any app’s inputs.conf merges into every input on that instance, so use it sparingly. - Avoid two
[monitor://]stanzas that cover the same file — the file is read once and you can’t predict which stanza’sindex/sourcetypeit gets.
Routing (outputs)
outputs.confdefaultGroup— the fallback._TCP_ROUTINGon an input in inputs.conf overrides the default group.- A
TRANSFORMS-settingDEST_KEY = _TCP_ROUTING(on a HF/indexer) runs later and overrides both.
The same “later wins” logic applies to index (inputs.conf → DEST_KEY = _MetaData:Index) and host/source/sourcetype rewrites.
Permissions (.meta files)
metadata/local.metabeatsmetadata/default.meta.- Inside a .meta file, the more specific stanza wins:
[props/<stanza>/<attr>]→[props/<stanza>]→[props]→[](whole app). export = systemis what makes an app’s objects visible to other apps at search time (section 3).
Deployment server (serverclass.conf)
- With the default
filterType = whitelist, a client must match a whitelist and not match a blacklist. Blacklist wins. - Settings at
[serverClass:<name>:app:<app>]override[serverClass:<name>], which overrides[global].
9. One-page quick reference
| Topic | Order (highest / first → lowest / last) |
|---|---|
| Global context (index-time) | system/local → apps/*/local → apps/*/default → system/default |
| App tie-break, global | ASCII: 0-9 → A-Z → a-z (first wins) |
| Indexer cluster peer | peer-apps/local → system/local → apps/local → peer-apps/default → apps/default → system/default |
| App/user context (search-time) | users/<user>/<app>/local → current app local → current app default → other apps local → other apps default → system/local → system/default |
| App tie-break, app/user | Reverse lexicographic (B beats A); exported objects only |
| props.conf stanza type | [source::] → [host::] → [sourcetype] → [default] |
| Same stanza type | literal → pattern; higher priority wins; else ASCII |
| Index-time pipeline | Input → Parsing (LINE_BREAKER) → Merging (LINEMERGE, timestamp) → Typing (SEDCMD → TRANSFORMS → RULESET) → Indexing |
| Multiple TRANSFORMS-/SEDCMD- classes | ASCII order of class name; within a class, left to right |
| Search-time | EXTRACT → REPORT → KV_MODE → FIELDALIAS → EVAL → LOOKUP → eventtypes → tags |
| inputs / serverclass filters | blacklist beats whitelist |
| Output routing | defaultGroup → inputs _TCP_ROUTING → TRANSFORMS _TCP_ROUTING (last wins) |
| .meta permissions | local.meta → default.meta; [type/stanza/attr] → [type/stanza] → [type] → [] |
btool commands
# Merged view + source file of every line
splunk btool props list <sourcetype> --debug
splunk btool transforms list <stanza> --debug
splunk btool inputs list --debug | grep <path>
# Search-time view for one user in one app
splunk btool props list <sourcetype> --debug --app=search --user=admin
# Syntax / typo check across all .conf files
splunk btool check