Digging for background

Let's say you're an on-air reporter and your assignment editor has asked you to prepare a series on the increase in Type-2 diabetes in your viewing area. Before you call a local expert, you may wish to research the phenomenon.

Most news reporters will launch an online search through a comprehensive Web utility, such as Google, or a paid aggregator of research, such as Nexis.

As a first step, I have a better way: a $125, 1,150-page semiannual volume called Fulltext Sources Online, compiled by Information Today Inc. (www.infotoday.com), a publisher of magazines and reference materials for online researchers and librarians.

It has more than 15,000 listings of newspapers, newswires, journals, magazines and even television broadcast transcripts that are available online, either through aggregators like Nexis or Dialog or directly on the publisher's Web site. Alphabetically arranged by publisher and indexed by topic, the listings note where each resource is available, the range of publication dates available for that outlet and the nature of the archives on the resource's Web site, if any.

On diabetes, for example, Fulltext Sources Online lists eight publications. The index indicates, for example, that Nexis has Diabetes magazine, though only through June 1989. The only place you can find Diabetes Technology & Therapeutics is through a rare database called ECO. Furthermore, none of the eight publications listed have their archives posted on the Web.

That got me weighing the relative merits of searching the Web for background story research versus performing the same task on aggregator resources, such as Nexis, Dow Jones/Factiva or Dialog.

For perspective, I contacted Reva Basch, author of Researching Online for Dummies and executive editor of the Super Searchers book series, including the just released Super Searchers in the News: The Online Secrets of Journalists & News Researchers from CyberAge Books.

"The Web doesn't have much historical source material at all," Basch told me. "If you want a newspaper story or journal article from earlier than the mid-1990s, you'll almost certainly have to use a commercial database service.

"The Web," she adds, "doesn't offer detailed data, like you'd get in a multiclient market study or any sort of custom research study. You may find teasers or highlights but not the in-depth numbers and analyses."

Basch isn't all that gung-ho about the Web's comprehensiveness for scientific and technical literature, preferring the proprietary online services.

Web search services-Google being the largest-send automated programs called spiders out to index the entire visible Web. By visible, I mean posted Web pages, not those generated when you perform a search of a site's database. When you perform a search on Google, it doesn't "go out" to the Web but returns a linked list of pages in its database of "crawled" pages, whose content matches one or more of the words you typed in in your search request.

Proprietary services actually index articles from each publication with which they have agreements. Then they charge you a fee to look at them on their sites.

Why pay for a Nexis search when you can find what you want on Google or AltaVista? The main reason is that, as Basch says, comparatively little research information gets put up on the Web in the form of posted pages that a Google can easily retrieve. Enter a search on Google, and, instead of white papers full of quotable research and interviewable sources, you are likely to find lots of pages from people selling their services.

Paid services do have their downsides, however. They are a pain to work with. "I don't think any of the major proprietary services have done a particularly good job of translating its powerful, command-driven functionality over to the Web."

As a result, Basch says, the "average user walks away from a search with incomplete and imprecise results and a huge bill to show for it. But," she adds, "there's always a tradeoff between power and ease of use."

In short, neither Web search engines nor proprietary research databases offer the ideal data-search environment.

I'd be interested in hearing from you television-newsroom info ferrets. Send me your thoughts on and experiences in online research. I'll gather your responses and turn them into a column.