Word

Defines classes and functions for parsing bodies of text to find words and prepare them for output to the user.

The top-level functions in this module are optimized to build Words from bodies of text containing more than a single Word. A chunking scheme based on the average length of words in the English language reduces the number of calls to Word.append and generally outperforms single character at a time processing (at least for English text).

Authors:: Peter Parente, Larry Weiss

Organization: IBM Corporation

License: The BSD License

All rights reserved. This program and the accompanying materials are made available under the terms of the BSD license which accompanies this distribution, and is available at http://www.opensource.org/licenses/bsd-license.php

getContextFromString(string, por, state=DefaultWordState)

source code

Gets the previous, current, and next Words relative to the given POR. If any word is missing, a None value is returned in its place. The string is considered to be at the zero offset of the Item indicated by the POR. Uses a default WordState object if none is provided.

Parameters:

string (string) - Text to parse for words
state (WordState) - Settings used to define a word
por - Point of regard indicating the source accessible and Item for the string

Returns: 3-tuple of Word

Previous, current, and next words surrounding the POR

buildWordsFromString(string, por=None, state=DefaultWordState, main_ob=None, trail_ob=None)

source code

Parses the given string to build a list of Words using the given state and the given POR. When no POR is given a dummy POR is constructed. Each Word constructed will use the provided or constructed POR to indicate it's position as if the string was from the same component and Item. The character offset from the given or constructed POR is not used. The string is always considered to be at the zero offset of the Item indicated by the POR. Uses a default WordState object if none is provided.

Parameters:

string (string) - Text to parse for words
state (WordState) - System settings used to define a word
por (POR) - Point of regard indicating the source accessible and Item for the string
main_ob (callable) - Function to invoke for each character in the main part of a word
trail_ob (callable) - Function to invoke for each character in the trailing part of a word

Returns: list of Word

Words parsed from the string

Module Word

getContextFromString(string, por, state=DefaultWordState)

buildWordsFromString(string, por=None, state=DefaultWordState, main_ob=None, trail_ob=None)