Archive for June 15th, 2005

Amazon.com Statistically Improbable Phrases

Amazon.com’s Statistically Improbable Phrases, or “SIPs”, are the most distinctive phrases in the text of books in the Search Inside! program. To identify SIPs, our computers scan the text of all books in Search Inside. If they find a phrase that occurs a large number of times in a particular book relative to all Search Inside books, that phrase is a SIP in that book.

SIPs are not necessarily improbable within a particular book, but they are improbable relative to all books in Search Inside. For example, most SIPs for a book on taxes are tax related. But because we display SIPs in order of their improbability score, the first SIPs will be on tax topics that this book mentions more often than other tax books. For works of fiction, SIPs tend to be distinctive word combinations that often hint at important plot elements.

For instance Handbook of the International Phonetic Association has the following SIPS: phonetic diphthongs, recorded passage, alveolar trill, transcribed passage, long counterparts, word final position, lexical stress, voiced obstruents, open syllables, stress placement, unstressed positions, nasalized vowels, educated speakers, velar nasal, front vowels, high vowels, vowel qualities, unstressed syllables

What puzzles me a little is that not all the books, even all the searchable books, have SIPS, and I’m not sure why.

Wednesday, June 15th, 2005

Mahnahmahna

Manahmahna

Which reminds me, why the hell is it that when every TV show under the sun is coming out in boxed sets (Hee-Haw, for crying out loud. Three’s Company. *Airwolf*), there’s still no season-by-season collections of the Muppet Show? There are a couple (four maybe?) DVD with some of the famous episodes, organized somewhat thematically (e.g. biggest name musical guests), but somebody somewhere is overlooking the opportunity to practically print money. Are the rights to the shows trapped in some legal Limbo?

hat tip Oxblog

Wednesday, June 15th, 2005