ITEC 325 2017spring ibarland

home—lects—hws
D2L—breeze (snow day)

regexps
regular expressions

regular expression intro

Representing regular expressions: They're not the same type as strings². Various languages have slightly different ways of representing regexps: Some …

• … use strings only — this means the computer can't type-check for any mistakes.
• … have a literal syntax for regular expressions (javascript /ab*c+?d/ or racket #rx"ab*c+?d")
• … have no literal syntax, but do have a class to represent regexps: Java new Pattern("ab*c+?d")
• … will auto-promote a string-literal to a regexp, based on context. (handy, but also circumvents some of the type-system's checking.)
• Use strings only, but regexps must start,end with a special delimiter (php: "/ab*c+?d/"; any delimiter will do).

PHP regular expressions

PHP regular-expression matching: In php, regular expressions are strings delimited by a special character (usually /). Some examples; see playing with regexps in PHP for details.

      echo preg_match( '/abcd/', 'abcd' );
      echo preg_match( '/abcd/', 'azcd' );      // false

      echo preg_match( '/a..d/', 'azcd' );      // . matches any single character (besides newline, null)

      echo preg_match( '/ab*cd/', 'abbbbcd' );  // b*  matches 0-or-more-b's.
      echo preg_match( '/ab*cd/', 'abcd' );  
      echo preg_match( '/ab*cd/', 'acd'  ); 

      echo preg_match( '/ab+cd/', 'abbbbcd' );  // b+  matches 1-or-more-b's.
      echo preg_match( '/ab+cd/', 'abcd' );  
      echo preg_match( '/ab+cd/', 'acd'  );     // false

      echo preg_match( '/ab.*cd/', 'abcd' );
      echo preg_match( '/ab.*cd/', 'abXd' );               // .* and .+ are a common patterns.
      echo preg_match( '/ab.*cd/', 'abBlahBlahBlahcd' );
      echo preg_match( '/ab.+cd/', 'abcd' );               // false
      echo preg_match( '/ab.+cd/', 'abXcd' );

      
      echo preg_match( '/ab?cd/', 'acd' );  // b?  matches 0-or-1 b
      echo preg_match( '/ab?cd/', 'abcd' );  
      echo preg_match( '/ab?cd/', 'abbbbcd'  );     // false

      echo preg_match( '/(ab)*cd/', 'ababababcd' );  // parens do grouping
      echo preg_match( '/(ab)*cd/', 'abbbbcd' );  // false


      // WARNING: preg_match looks to see if the string *contains* a match!
      echo preg_match( '/row/', 'How now, brown cow?' );  // true
      echo preg_match( '/.ow/', 'How now, brown cow?' );  // true

      // Use "^" to specify the start-of-string, and "$" to specify end-of-string.
      echo preg_match( '/^.ow$/', 'How now, brown cow?' );  // false
      echo preg_match( '/^.ow$/', 'Zow' );
      echo preg_match( '/^.ow/', 'Zowee' );
      echo preg_match( '/w.e$/', 'Wowee Zowee' );

      
      echo preg_match( '/[WZ]ow/', 'Wowee Zowee' );  // square-brackets match any one character from the set
      echo preg_match( '/[WZ]ow/', 'Yow' );          // false
      echo preg_match( '/[W-Z]ow/', 'Yowee' );       // square-brackets can contain a *range*
      echo preg_match( '/ab[0-9]+de/', 'ab789de' );


      echo preg_match( '/[0-9]*/', '00047' );        // Beware: matching just a * expression (w/o ^,$)!
    

Your task: What is a regular expression to match...

• The name "Jon"?
• The name "Jon" or "John"?
• Any name starting with "Jon"?
• Any name starting with: "Jon" or "John"?
• Any name starting with "Jon"; or, the name "John"?
• Any name containing "Jon"?
• The name "Jon" or "John" or "Jonathon" or "Jonathan"?
• (For the following: we'll take “names” to mean containing only the 26 unaccented roman letters, and capitalized as in western culture³).
• A hyphenated name.
• A possibly-hyphenated name.
• A doubly-hyphenated name.
• An exactly-doubly-hyphenated name.
• An at-most-doubly-hyphenated name.
(For the following: we'll take “words” to mean containing only unaccented roman letters and digits, or an underscore (?!).)
• A word starting with a lower-case letter.
• A word not starting with a capital letter.
• A word ending with a digit.
• A word ending with a roman numeral.

False positive, and false negative:

Warning: Beware matching a top-level * expression: the empty-string matches it, and any string contains the empty-string! Thus preg_match_all( "/(xyz)*/", "uh-oh") === 6!!, since "uh-oh" has zero "xyz"'s at the start, followed by 'u', followed by zero more "xyz"'s, followed by 'h', followed by ….

Atomic regexps

• ^ : matches start-of-string
• $ : matches end-of-string
• . : matches any one character
• Most characters match themselves (e.g. 'b'); you can escape special characters with backslash.
• Character classes: We'll tend to prefer the unicode properties, since the other two don't account for characters like “é”. See www.regular-expressions.info/posixbrackets.html.
  • \p{L} is better than [[:alpha:]] is better than \w.
  • \p{Lu} is better than [[:upper:]] is better than [A-Z].
  • \p{Ll} is better than [[:lower:]] is better than [a-z].
  • For digits, [0-9] or [\d] (or [[:digit:]]) is arguably okay; use \p{N} if you want to handle any digit-ish character like or “𝟜” (double-struck script) or “٤” (arabic 4) or “⁴” (superscript). You can use \p{Nd} to exclude the last (but not the former two).

    bug?: preg_match_all( '/\p{N}/', "⁴٤𝟜4" ) is returning 1 for me (not 4), in php 5.4.3.

  • \p{P} is better than [[:punct:]] is the same as [!.?:;"'].
  • […] — build your own character set, e.g. [aeiou] matches any one lower-case vowel.
    Use ^ just inside the [ to negate the set. (Note that this overrides the beginning-of-string meaning of ^; it's clear from context).

  • Checking for space (one of the most common situations) is the one that is hardest: We have \p{Z}, as well as [[:space:]] and \s. The first, unicode-property version checks for separating space characters (thin-space, etc), but doesn't catch tabs (or newlines). On the other hand, \s does catch tabs, newlines etc., but doesn't catch thin-space or nonbreaking-space or other space-looking characters.

    Also, we are probably least able to ignore this problem, because in web forms people might paste in text from web pages or Word documents, and those exper-typesetting programs do tend to use the various special-spaces.

    Solution: Either

    • use the pattern [\p{Z}\n\t\r\v\f] — long to type and read, and easy to get wrong; or

    • $str2 = preg_replace( '/\p{Z}+/', ' ', $str ); … if( preg_match(/\s+/',$str2)) …

  Remember with the character-sets: Each of these match one single character. They are often used in conjunction with * and + below.

Compound regexps

• (α) — same as α; used to dis-ambiguate precedence.
• α* — 0 or more occurrences of α
• α+ — 1 or more occurrences of α
• α? — 0 or 1 occurrence of α
• α{…} — you specify a range of #repetitions of α (not very common)
• α|β — α OR β. This is the one low-priority regexp operator: ab|cd is interpreted as (ab)|(cd), not a(b|c)d. Compare that with any of the others: ab*c is interpreted as a(b*)c, not (ab)*c.

NOTE: In php, no trailing “g” allowed, as in javascript!
(There are other trailing modifiers however — e.g. i for case-insensitive, and more.)

• preg_match($pat,$target) (also, note: preg_match_all — not as common, but it motivates why preg_match's output is 0,1 and not a raw boolean).

  Warning: preg_match_all("/x/","abc",$results) will always have $results be non-empty — it will be an array containing an empty array. (In particular, the condition of if ($results) … will always be true.) You can instead use preg_match_all("/x/","abc",$results,PREG_SET_ORDER) which will have zero elements for zero matches (and if ($results) … will behave as you want). Each element of $results will still be an another entire array though -- the full match plus any sub-patterns.

• preg_replace($pat,$target,$str)
• split($pat,$target) Returns an array (indexed numerically from 0)
• trim — nothing to do with regular expressions, but a function you probably want to call on most user-entered text.

regexps vs. unicode

• In unicode, we separate the notion of a byte from that of a character. (The term “code point” is sometimes used to emphasize unicode-character-that-might-be-more-than-a-char.)
• In unicode, we have more generalized notions of letters: not just A-Z, but also accented characters like “é”, chinese characters, etc.. Also, whitespace is more complicated: There is a regular ol' space, thin whitespace  , and more.
• Use “properties” — generalized character classes, when supported by your language:
  • \p{Letter} or \p{L} — any letter, any language.
  • \p{Separator} or \p{Z} — space-characters, including thin-space etc.. But not other whitespace like newlines and tabs (!). Using [\p{Z}\t\r\n\v\f] is better than \s though, since people might paste in whitespace on input forms. (Or, do a regexp-replace of \p{Z} to , and then use \s.)
  • \p{Punctuation} or \p{P}
  There are more refined versions of these properties, too (e.g. \p{Ll} or \p{Lowercase_Letter}).
• Corner case: If you want to compare strings case-independently, use a library function that is unicode-aware. Note that the common trick of converting both strings to (say) uppercase doesn't always work in unicode, since (e.g.) “ß” has no (traditional) upper-case version. Moreover, for comparing German strings, “ß” should be considered equivalent to “ss”.
• Corner case: Beware that characters like “é” can actually be encoded either as a single character (and thus match “.”), or as a regular ol' “e” combined with the accent “́” (and thus match, say, “..”)! See this page for advice on how to write regexps to try and cope with this.

home—lects—hws
D2L—breeze (snow day)

©2017, Ian Barland, Radford University Last modified 2017.Feb.26 (Sun)

Please mail any suggestions (incl. typos, broken links) to ibarlandradford.edu