regexps
regular expressions

Unwieldy video from distance lecture, 2017-feb-21 (1h35m)
but you can watch just [0:24:40,1:33:00) (1h08m) about regular-expressions.
(You can skip over: opening pause [0:00:00,0:00:30), some ERD review [0:00:30,0:04:30), git-review [0:04:30,0:22:00), and a few comments about projects-with-external-clients [0:22:00,0:24:40), as well as the final few "shutting down" minutes at [1:33:00,1:35:00).)

regular expression intro

regexps.pdf

Representing regular expressions: They're not the same type as strings ². Various languages have slightly different ways of representing regexps: Some …

… use strings only — this means the computer can't type-check for any mistakes.
… have a literal syntax for regular expressions (javascript /ab*c+?d/ or racket #rx"ab*c+?d" or python r'ab*c+?c') )
… have no literal syntax, but do have a class to represent regexps: Java new Pattern("ab*c+?d")
… will auto-promote a string-literal to a regexp, based on context. (handy, but also circumvents some of the type-system's checking.)
Use strings only, but regexps must start,end with a special delimiter (php: "/ab*c+?d/"; any delimiter will do).

PHP regular expressions

PHP regular-expression matching: In php, regular expressions are strings delimited by a special character (usually /). Some examples; see playing with regexps in PHP for details.

      echo preg_match( '/abcd/', 'abcd' );
      echo preg_match( '/abcd/', 'azcd' );      // false

      echo preg_match( '/a..d/', 'azcd' );      // . matches any single character (besides newline, null)

      echo preg_match( '/ab*cd/', 'abbbbcd' );  // b*  matches 0-or-more-b's.
      echo preg_match( '/ab*cd/', 'abcd' );  
      echo preg_match( '/ab*cd/', 'acd'  ); 

      echo preg_match( '/ab+cd/', 'abbbbcd' );  // b+  matches 1-or-more-b's.
      echo preg_match( '/ab+cd/', 'abcd' );  
      echo preg_match( '/ab+cd/', 'acd'  );     // false

      echo preg_match( '/ab.*cd/', 'abcd' );
      echo preg_match( '/ab.*cd/', 'abXd' );               // .* and .+ are a common patterns.
      echo preg_match( '/ab.*cd/', 'abBlahBlahBlahcd' );
      echo preg_match( '/ab.+cd/', 'abcd' );               // false
      echo preg_match( '/ab.+cd/', 'abXcd' );

      
      echo preg_match( '/ab?cd/', 'acd' );  // b?  matches 0-or-1 b
      echo preg_match( '/ab?cd/', 'abcd' );  
      echo preg_match( '/ab?cd/', 'abbbbcd'  );     // false

      echo preg_match( '/(ab)*cd/', 'ababababcd' );  // parens do grouping
      echo preg_match( '/(ab)*cd/', 'abbbbcd' );  // false


      // WARNING: preg_match looks to see if the string *contains* a match!
      echo preg_match( '/row/', 'How now, brown cow?' );  // true
      echo preg_match( '/.ow/', 'How now, brown cow?' );  // true

      // Use "^" to specify the start-of-string, and "$" to specify end-of-string.
      echo preg_match( '/^.ow$/', 'How now, brown cow?' );  // false
      echo preg_match( '/^.ow$/', 'Zow' );
      echo preg_match( '/^.ow/', 'Zowee' );
      echo preg_match( '/w.e$/', 'Wowee Zowee' );

      
      echo preg_match( '/[WZ]ow/', 'Wowee Zowee' );  // square-brackets match any one character from the set
      echo preg_match( '/[WZ]ow/', 'Yow' );          // false
      echo preg_match( '/[W-Z]ow/', 'Yowee' );       // square-brackets can contain a *range*
      echo preg_match( '/ab[0-9]+de/', 'ab789de' );


      echo preg_match( '/[0-9]*/', '00047' );        // Beware: matching just a * expression (w/o ^,$)!

See the manual.
Note: the name “preg” comes from "Perl compatible"; earlier PHP used the POSIX regexp's but PHP decided to deprecate that.

Your task: What is a regular expression to match...

The name "Jon"?
The name "Jon" or "John"?
Any name starting with "Jon"?
Any name starting with: "Jon" or "John"?
Any name starting with "Jon"; or, the name "John"?
Any name containing "Jon"?
The name "Jon" or "John" or "Jonathon" or "Jonathan"?
(For the following: we'll take “names” to mean containing only the 26 unaccented roman letters, and capitalized as in western culture ³).
A hyphenated name.
A possibly-hyphenated name.
A doubly-hyphenated name.
An exactly-doubly-hyphenated name.
An at-most-doubly-hyphenated name.

or an underscore (?!)

A word starting with a lower-case letter.
A word not starting with a capital letter.
A word ending with a digit.
A word ending with a roman numeral.

False positive, and false negative:

		test result
		negative	positive
is test accurate?	false	false negative	false positive
is test accurate?	true	true negative	true positive

In our setting, the “test” is preg_match, and “accurate” means whether the value returned is what we want it to be.

Warning: Beware matching a top-level * expression: the empty-string matches it, and any string contains the empty-string! Thus preg_match_all( "/(xyz)*/", "uh-oh") === 6!!, since "uh-oh" has zero "xyz"'s at the start, followed by 'u', followed by zero more "xyz"'s, followed by 'h', followed by ….

Atomic regexps

^ : matches start-of-string
$ : matches end-of-string
. : matches any one character
Most characters match themselves (e.g. 'b'); you can escape special characters with backslash.
Character classes: We'll tend to prefer the unicode properties, since the other two don't account for characters like “é”. See www.regular-expressions.info/posixbrackets.html.
- \p{L} is better than [[:alpha:]] is better than \w.
- \p{Lu} is better than [[:upper:]] is better than [A-Z].
- \p{Ll} is better than [[:lower:]] is better than [a-z].
- For digits, [0-9] or [\d] (or [[:digit:]]) is arguably okay; use \p{N} if you want to handle any digit-ish character like or “𝟜” (double-struck script) or “٤” (arabic 4) or “⁴” (superscript). You can use \p{Nd} to exclude the last (but not the former two).
  bug?: preg_match_all( '/\p{N}/', "⁴٤𝟜4" ) is returning 1 for me (not 4), in php 5.4.3.
- \p{P} is better than [[:punct:]] is the same as [!.?:;"'].
- […] — build your own character set, e.g. [aeiou] matches any one lower-case vowel.
  Use ^ just inside the [ to negate the set. (Note that this overrides the beginning-of-string meaning of ^; it's clear from context).
- Checking for space (one of the most common situations) is the one that is hardest: We have \p{Z}, as well as [[:space:]] and \s. The first, unicode-property version checks for separating space characters (thin-space, etc), but doesn't catch tabs (or newlines). On the other hand, \s does catch tabs, newlines etc., but doesn't catch thin-space or nonbreaking-space or other space-looking characters.
  
  Also, we are probably least able to ignore this problem, because in web forms people might paste in text from web pages or Word documents, and those exper-typesetting programs do tend to use the various special-spaces.
  
  Solution: Either
  - use the pattern [\p{Z}\n\t\r\v\f] — long to type and read, and easy to get wrong; or
  - $str2 = preg_replace( '/\p{Z}+/', ' ', $str ); … if( preg_match(/\s+/',$str2)) …
Remember with the character-sets: Each of these match one single character. They are often used in conjunction with * and + below.

Compound regexps

There are also ways of building bigger regexps out of smaller ones:

(α) — same as α; used to dis-ambiguate precedence.
α* — 0 or more occurrences of α
α+ — 1 or more occurrences of α
α? — 0 or 1 occurrence of α
α{…} — you specify a range of #repetitions of α (not very common)
α|β — α OR β. This is the one low-priority regexp operator: ab|cd is interpreted as (ab)|(cd), not a(b|c)d. Compare that with any of the others: ab*c is interpreted as a(b*)c, not (ab)*c.

NOTE: In php, no trailing “g” allowed, as in javascript!
(There are other trailing modifiers however — e.g. i for case-insensitive, and more.)

Three helpful functions:

preg_match($pat,$target) (also, note: preg_match_all — not as common, but it motivates why preg_match's output is 0,1 and not a raw boolean).
Warning: preg_match_all("/x/","abc",$results) will always have $results be non-empty — it will be an array containing an empty array. (In particular, the condition of if ($results) … will always be true.) You can instead use preg_match_all("/x/","abc",$results,PREG_SET_ORDER) which will have zero elements for zero matches (and if ($results) … will behave as you want). Each element of $results will still be an another entire array though -- the full match plus any sub-patterns.
preg_replace($pat,$target,$str)
split($pat,$target) Returns an array (indexed numerically from 0)
trim — nothing to do with regular expressions, but a function you probably want to call on most user-entered text.

regexps vs. unicode

In unicode, we separate the notion of a byte from that of a character. (The term “code point” is sometimes used to emphasize unicode-character-that-might-be-more-than-a-char.)
In unicode, we have more generalized notions of letters: not just A-Z, but also accented characters like “é”, chinese characters, etc.. Also, whitespace is more complicated: There is a regular ol' space, thin whitespace  , and more.
Use “properties” — generalized character classes, when supported by your language:
- \p{Letter} or \p{L} — any letter, any language.
- \p{Separator} or \p{Z} — space-characters, including thin-space etc.. But not other whitespace like newlines and tabs (!). Using [\p{Z}\t\r\n\v\f] is better than \s though, since people might paste in whitespace on input forms. (Or, do a regexp-replace of \p{Z} to , and then use \s.)
- \p{Punctuation} or \p{P}
There are more refined versions of these properties, too (e.g. \p{Ll} or \p{Lowercase_Letter}).
Corner case: If you want to compare strings case-independently, use a library function that is unicode-aware. Note that the common trick of converting both strings to (say) uppercase doesn't always work in unicode, since (e.g.) “ß” has no (traditional) upper-case version. Moreover, for comparing German strings, “ß” should be considered equivalent to “ss”.
Corner case: Beware that characters like “é” can actually be encoded either as a single character (and thus match “.”), or as a regular ol' “e” combined with the accent “́” (and thus match, say, “..”)! See this page for advice on how to write regexps to try and cope with this.

²C, ML, and Haskell let you introduce new type-synonyms, but not Java -- you have to introduce a new class that wraps strings. ↩

¹ Similarly, I'd love to have a language that lets me rename types ², so that I could have string (for raw-data), html-data (safe to concatentate to HTML), and sql-data (safe to splice into a SQL query as data). htmlspecialchars and mysql_real_escape_string can be thought of as consturctors for these new types, but the type-system doesn't help protect me from giving a raw (unsanitized) string when it was expecting some already-sanitized data. ↩

³ Okay, that's actually vague: is “de Soto” or “da Vinci” all last name? Names are notoriously hard to characterize, especially across multiple cultures. My advice is to liberally accept what characters people say their name is; trimming and collapsing whitespace is about all I'd do. ↩

home—lects—hws
D2L—breeze (snow day)

©2017, Ian Barland, Radford University
Last modified 2017.Oct.09 (Mon) Please mail any suggestions
(incl. typos, broken links)
to ibarlandradford.edu