Regular expressions for people who avoid them: the dozen constructs that cover most jobs

Regular expressions have a reputation problem: they look like line noise, most people learn them by copying from search results, and the copied pattern usually works on the three examples in front of them and fails on the fourth. But the working vocabulary is genuinely small — about a dozen constructs cover the great majority of real jobs — and the parts that go wrong are a short, predictable list. This guide covers those dozen constructs with patterns you would actually write, the greedy-matching and backtracking traps that account for most broken regexes, the differences between engines, and a working method using the regex tester here, which highlights matches live and breaks out every capture group as you type.

The dozen constructs

Two rules make the rest sensible. Anything with a special meaning is escaped with a backslash to match it literally — \. for a full stop, \$ for a dollar sign, \\ for a backslash. And inside square brackets, most special characters lose their powers: [.+*] matches those three punctuation marks, no escaping needed. The exceptions inside brackets are ^ at the start (which negates), - between characters (which makes a range) and ] itself.

Greedy by default

This is the single most common surprise. Quantifiers take as much as they can:

text     <a href="x"> and <b>
<.+>     matches the entire line
<.+?>    matches <a href="x">  then  <b>
<[^>]+>  matches the same, and faster

Adding ?after a quantifier makes it lazy: take the minimum, expand only when the rest of the pattern fails. Often the better fix is the third line — instead of “anything, but stop early”, say “anything that is not the terminator”. A negated character class cannot overshoot in the first place, so there is nothing to back up over, and it is dramatically faster on long lines.

Anchors are what stop a pattern matching everywhere

An unanchored pattern matches anywhere in the text, which is why \d{3} happily finds three digits inside a twelve-digit number. Anchors pin it down: ^ at the start, $ at the end, \bat a word boundary. For validation — “is this whole string a postcode?” — anchor both ends, or you will accept a postcode with rubbish either side of it.

\b is the underrated one. It matches the zero-width position between a word character and a non-word character, so \bcat\b finds the animal but not concatenate or catalogue. Most “my find-and-replace destroyed the file” incidents are a missing pair of \b.

Capturing what you came for

Parentheses do two jobs at once: they group, so a quantifier can apply to several characters, and they capture, so you can pull the matched text out afterwards. Groups are numbered left to right by their opening bracket. When you only need grouping, (?:…) avoids the cost and the renumbering.

Named groups are better for anything you will maintain. The tester’s starting pattern uses one — #(?<order>[A-Z]-\d+) against a sample of order confirmations — and the match table shows the named group beside the numbered ones, so you can see exactly what would land in match.groups.order. In replacements you refer to captures as $1 or $<order>, which turns a regex into a reformatting tool:

find     (\d{4})-(\d{2})-(\d{2})
replace  $3/$2/$1
result   2024-03-18  →  18/03/2024

Backreferences let a pattern refer to its own captures: \b(\w+) \1\b finds a doubled word, and ("|').*?\1 matches a quoted string that ends with the same quote character it started with. Lookarounds assert without consuming — (?=…) and (?!…) ahead, (?<=…) and (?<!…) behind — so \d+(?= USD)captures the number in “250 USD” without taking the currency.

Flags change everything

The tester exposes the JavaScript flags as checkboxes, and each one changes the meaning of the same pattern:

• g — global. Find every match rather than the first. On by default here, and the match count updates as you type.
• i — case-insensitive. Cheaper and clearer than writing [Ee][Rr][Rr][Oo][Rr].
• m — multiline. ^ and $ match at every line break instead of only at the ends of the text. Essential for log files.
• s — dotAll. . also matches newlines, so a pattern can span lines.
• u — unicode. Makes escapes like \u{1F600} and Unicode property escapes work correctly, and treats astral characters such as emoji as single units rather than two code units.
• y — sticky. Matches only at the current position; used when tokenising, rarely otherwise.

The m and s pair is worth internalising, because they are frequently confused: m changes what the anchors mean, s changes what the dot means.

Patterns worth keeping, and ones not to write

Useful patterns from real work:

ISO date        \d{4}-\d{2}-\d{2}
Time 24h        ([01]\d|2[0-3]):[0-5]\d
IPv4 (rough)    \b(?:\d{1,3}\.){3}\d{1,3}\b
UUID            [0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}
key=value       ^(\w+)\s*=\s*(.*)$
Log level       ^\[(?<level>DEBUG|INFO|WARN|ERROR)\]
Trailing space  [ \t]+$
Duplicate word  \b(\w+)\s+\1\b

And three things a regex should not be asked to do:

• Validate an email address. The grammar allows quoted local parts and comments; the truly correct pattern is unmaintainable and still cannot tell you the mailbox exists. Check for an @ with text either side and a dot in the domain, then send a confirmation.
• Parse HTML or JSON. Both nest arbitrarily, and regular expressions cannot count nesting. Use a parser; for JSON, the converters here will reformat it for you.
• Split a CSV. Quoted fields containing commas and newlines defeat every one-line pattern. Use a real CSV parser — the CSV guide covers why.

Catastrophic backtracking

A backtracking engine — which includes JavaScript, Python, Java, PCRE and .NET — tries alternatives when a match fails. Usually that is cheap. But when a quantifier contains another quantifier over overlapping characters, the number of ways to split the input grows exponentially:

(a+)+$          against  "aaaaaaaaaaaaaaaaaaaaaaaaX"
(\s*\w+)*$      against a long line with a trailing symbol
(.*,)*           against a long comma-heavy string

Each takes exponential time to fail. Thirty characters of adversarial input can freeze a thread for minutes — this is the ReDoS vulnerability class, and it has taken down real production services through a validation pattern applied to user input.

Three defences, in order of usefulness:

• Do not nest quantifiers over overlapping classes. (a+)+ should be a+; (\s*\w+)* should be rewritten so only one construct can consume each character.
• Prefer negated classes to lazy dots. [^"]* cannot backtrack the way .*? can.
• Anchor and bound. Anchored patterns fail sooner, and an explicit upper bound like {1,64} caps the work.

Some engines offer atomic groups or possessive quantifiers that forbid backtracking outright; JavaScript has neither, so restructuring is the only option. Go’s RE2and Rust’s regex crate sidestep the whole class by using an algorithm with guaranteed linear time — which is also why they refuse backreferences and lookaround.

How to build one that works

Write regexes the way you write tests: start from the data, not from the pattern. Paste a representative sample into the tester — including the awkward lines, not just the clean ones — and grow the pattern one construct at a time, watching the highlighting after each addition. When a change makes matches disappear, undo it and try the smaller step.

Three habits that save time. Watch the match count: an unexpected jump usually means a quantifier is greedier than you thought (the tester caps display at 2,000 matches, and says so, which is itself a signal your pattern is too loose). Test the failures: paste text that should not match and confirm it does not — most bad patterns are too permissive rather than too strict. And compare before and after: when a regex is reformatting a file, the text diff here shows exactly which lines changed, which catches the replacement that quietly mangled every twentieth row. For the specific job of changing identifier style rather than matching it, the case converter handles camelCase, snake_case, kebab-case and the rest without a pattern at all.

Finally, comment anything non-obvious. A pattern you understood perfectly on Tuesday is a puzzle by the following month; one line of prose above it is cheap.