Splunk: Core Concepts of SPL

Search commands tell Splunk what to do with the events retrieved from indexes. In Splunk, an index is a repository for data. When data comes in, Splunk processes it (a step called indexing), breaks it into events, and stores those events in indexes so they can be searched. An event is a single piece of data in Splunk, such as one line of a log file.

For more information about indexes, indexers, and indexer clusters

To see how these events are built from logs, metrics, and traces, you can check out what observability is.

Free Text Search

This is the most basic search option for SPL.

index=* kerem    (The AND operator is used implicitly and is case-insensitive)  
index=_* kerem   (To match internal indexes using a wildcard)  

Predicate expressions

A predicate is an expression that consists of operators or keywords that specify a relationship between two expressions. A predicate expression, when evaluated, returns either TRUE or FALSE.

Types of predicate operators

Types of predicate operators
Relational operators
Logical operators
Conditional and pattern-matching operators
Boolean functions

Relational operators

Relational operators are symbols that compare one expression to another.

The examples below evaluate against the sample event status = 404, bytes = 2048.

OperatorDescriptionExample
=Equal to — the field matches the valuestatus = 404 → TRUE
!=Not equal tostatus != 200 → TRUE
<Less thanbytes < 4096 → TRUE
<=Less than or equal tobytes <= 2048 → TRUE
>Greater thanbytes > 1024 → TRUE
>=Greater than or equal tostatus >= 400 → TRUE

Logical operators

The logical operators combine or negate predicate results (TRUE/FALSE). The examples below use the sample event status = 404, bytes = 2048.

OperatorDescriptionExample
ANDTRUE only when both sides are TRUEstatus = 404 AND bytes = 2048 → TRUE
ORTRUE when at least one side is TRUEstatus = 200 OR bytes = 2048 → TRUE
XORExclusive OR — TRUE when exactly one side is TRUEstatus = 404 XOR bytes = 512 → TRUE
NOTNegates (reverses) the result of an expressionNOT status = 200 → TRUE
Note: XOR is not supported by the search command; use it inside the eval or where commands. AND, OR, and NOT work everywhere.

Conditional and pattern-matching operators

The IN operator matches the values in a field to any of the items in the <expression-list>.

OperatorDescriptionExample
INField value equals any item in the liststatus IN (404, 500, 503) → TRUE

Boolean functions

Boolean functions are evaluation functions that return either TRUE or FALSE. For the full list, see Splunk's Evaluation functions reference.

Try it: build a predicate against a sample event and watch it evaluate to TRUE or FALSE.

Wildcard with SPL

A wildcard lets a single search match many values at once. Which character you use depends on the command:

WildcardWhere it's usedMatchesExample
*search and most commandsany run of characters (zero or more)sourcetype="access_*" matches access_combined and access_common
%where clause and like() in evalmultiple characterswhere like(ipaddress, "198.%")
_where clause and like() in evalexactly one characterwhere like(status, "40_") matches 400 through 409

A few rules worth remembering:

  • Wildcard strings must be wrapped in quotes — write sourcetype="access_*" with the quotes, not without them.
  • Mid-string wildcards can be unreliable. productID="S*G01" may miss SC-MG-G01, because Splunk segments that value into SC, MG, and G01, and no single segment matches S...G01.
  • Be specific instead of leaning on wildcards. Searching access denied is faster and lighter than access*, because the wildcard makes Splunk match every term starting with "access" instead of the exact phrase you want.
  • Search by field-value pair instead of raw text: status=404 is far more efficient than scanning the whole event (_raw) for 404.

Try it: type a pattern with * and watch which sample sourcetypes it matches.

Quotation marks

Quotation marks let us specify which string values to include and perform specific searches on field names. Put simply: when you search for a value, wrap it in double quotes, and use single quotes only when the field name itself is unusual (spaces, dashes, and so on).

QuotesUse them forExample
Double " "string valuesaction="purchase"
Single ' 'field names with spaces, dashes, wildcards, or other special charactersstats sum(bytes) AS 'Sum of bytes'
Back tick (`)a multi-term search literal, matched exactly as written`invalid user sshd[5258]`

You also need quotes whenever a value contains punctuation that Splunk might otherwise read as a word break — IP addresses and timestamps are the common cases: clientip="192.0.2.14", earliest="10/5/2020:20:00:00".

Note

A wildcard works inside those quoted values too. clientip="192.*" matches every address that starts with 192. — 192.0.2.14, 192.168.1.1, and everything else in the 192.0.0.0/8 range. Just note that this is very broad: narrow it with a longer prefix like 192.168.*, or, for exact subnet matching, use the cidrmatch() function — where cidrmatch("192.168.0.0/16", clientip).

Search commands

Splunk's Search Processing Language (SPL) allows analysts to apply filters and narrow down search results using search commands.

The reference below lists every SPL search command. Type to filter, and click a name to open its official documentation.

A few of the most common filtering commands — click one to see what it does and a full example:

Note

The - sign is a modifier, and its meaning depends on the command. In sort, - field sorts that field in descending order (ascending is the default). In fields, - field excludes that field from the results.

Tip

Splunk regular expressions are PCRE (Perl Compatible Regular Expressions) and use the PCRE C library. You can build and test your patterns online at regex101.com; for the details, see Splunk's about Splunk regular expressions reference.

Note

Beyond single commands, SPL can nest one search inside another with a subsearch. To dig deeper, see Splunk's join command and the use a subsearch tutorial.