Specifying word delimiter attributes

As mentioned previously, to break down the contents of a word-indexed field into individual words, the AVM needs to know which characters delimit words and which do not. The distinction can be subtle and sometimes depends on context. For example, consider the function of the dot in the character strings in the following table.

Table 6. Is the dot a word delimiter?
Character string	Function of the dot	Is the dot a word delimiter?
"Balance is $25,125.95"	Decimal separator	No
"Shipment not received.Call customs broker"	Period at end of sentence	Yes

In the first character string, the dot functions as a decimal point and does not divide one word from another. Thus, you can query on the word "$25,125.95." In the second character string, by contrast, the dot functions as a period, dividing the word "received" from the word "call."

To help define word delimiters systematically while allowing for contextual variation, ABL provides eight word delimiter attributes, which you can use in word-break tables. The eight word delimiter attributes appear in the following table.

Table 7. Word delimiter attributes
Word delimiter attribute	Description	Default
LETTER	Always part of a word	Assigned to all characters that the current attribute table defines as letters.In English, these are the uppercase characters A–Z and the lowercase characters a–z.
DIGIT	Always part of a word	Assigned to the characters 0–9.
USE_IT	Always part of a word	Assigned to the following characters: Dollar sign ($) Percent sign (%) Number sign (#) At symbol (@) Underline (_)
BEFORE_LETTER	Part of a word only if followed by a character with the LETTER attribute; otherwise, treated as a word delimiter	–
BEFORE_DIGIT	Treated as part of a word only if followed by a character with the DIGIT attribute	Assigned to the following characters: Period (.) Comma (,) Hyphen (–) For example, "12.34" is one word, but "ab.cd" is two words.
BEFORE_LET_DIG	Treated as part of a word only if followed by a character with the LETTER or DIGIT attribute	–
IGNORE	Ignored	Assigned to the apostrophe ('). For example, "John's" is equivalent to "Johns."
TERMINATOR	Word delimiter	Assigned to all other characters.