Surfing the Web or spinning your own Web site? Millions are, but you're probably not evaluating your site to see what effects it has or using your Web site to evaluate your Web site. At Cornell University, however, an evaluation expert not only is using his elaborate Web site as an integral part of his teaching and research but also as a "field laboratory" to evaluate how well the site is accomplishing its goals.

In addition, William Trochim, Cornell professor of human service studies in the College of Human Ecology, is using his Web site to collect data on a wide variety of evaluation research projects, from whether students do better when their course resources are online to what traits a community wants in its next chief of police.

In his first online semester, Trochim developed a Web-based resource center complete with a hyperlinked text, downloadable lectures, sample exams, simulation workbooks, a sample term paper and resource lists, class bulletin boards, an online extra-credit scavenger hunt game and more. Working with a team of graduate and undergraduate research assistants, Trochim found that students liked the flexibility and worldwide access to resources the online course offered them, but they did not do any better on class exams than former students.

"This finding is important because government agencies, businesses and nonprofit organizations are making tremendous investments to adopt the new Web technologies but have no idea whether their investments are having an impact. In their haste to use this pervasive and revolutionary technology 'on the fly,' they have no idea what its effectiveness actually is or what unexpected consequences there might be," said Trochim, an evaluator and social science research methodologist. He is using the Web as a tool in conceptualizing and evaluating all kinds of projects and problems, including instructional technology delivery, community issues and social science research questions.

"The bottom line is that we need to evaluate and then evaluate again. Just because clients or students have more access to information doesn't necessary translate into performance improvements. Making such assumptions might prove very dangerous -- and expensive," he said.

At the moment, there are no established rules or theories to guide Web construction and use, Trochim pointed out. "We are weaving the Web out of thin air -- making it up as we go along. There has been a remarkable absence of research into how Web sites are developed and implemented and their effects and impacts. My work helps fill that gap."

Trochim is taking the tools of his trade -- the latest methodologies and technologies in evaluation and social science research methods developed over the past half-century -- and applying them to all phases of Web site development and assessment.

In studying his own Web site, for example, Trochim and Human Ecology junior Dawn Bonanno of Lake Huntington, N.Y., are analyzing the content of every single electronic communication between Trochim and his students over the course of a semester -- more than 500 separate messages. The analysis shows how students use the Web in their coursework and what problems they struggle with in accessing and making use of this new technology.

With another Human Ecology junior, Dominic Cirillo of Butler, Pa., Trochim is examining the different ways to analyze every "hit" to a Web page and working out the logistics of linking that data to course performance information.

Both students wrote papers describing their research that were presented at the American Evaluation Association Conference in Atlanta in November. (The full papers, of course, are posted on the Web site listed below).

Interestingly, Trochim and the students have detected some intriguing gender differences in Web site use.

"We found some tantalizing signs in the data that males and females respond very differently to the course-based Web site," Trochim commented. "Although the results are still preliminary, it seems that female students don't have the same level of access or utilization of Web technology as their male counterparts." Trochim plans to study this phenomenon intensively in the next few months; his progress may be viewed on the Web.

Much of Trochim's evaluation work uses a technique he has developed over the past 14 years called concept mapping. This method (which can now be implemented over the Web) uses state-of-the-art statistical analyses on input from many people (such as the staff of an agency, corporation managers, boards of directors or members of a community) to "map" their ideas, prioritize what's most important to whom and explore the degree of consensus across subgroups. Concept mapping involves brainstorming ideas relevant to the problem or issue, grouping and ranking the ideas, and feeding them into a specially designed multivariate and cluster analysis program. The end result is a picture or "map" that visually conveys what the groups involved think is most important and the degree of consensus across different groups.

Concept mapping has been used in a wide variety of applications, including strategic planning, community-based decision-making, training and technology evaluation, program and product development, and even hiring decisions for important positions.

For instance, Trochim has used concept mapping to evaluate staff goals for an employment program for the severely mentally ill; to help assess a multicultural awareness youth camp; to construct a volunteer training program for working with deinstitutionalized patients; to develop standards for accrediting graduate-level evaluation programs; to evaluate the effectiveness of corporate training programs; to determine what's most important to the local community in choosing a police chief; to explore the factors that enhance a congregation's spirituality; and to determine what's of concern to Cornell students. All of these projects can be reviewed on his Web site, and several of them were conducted over the Web.

Trochim's Web site is a valuable resource not only for his students and "eavesdroppers" but for anyone seeking help with social science research and methods. It includes open access to his lectures, textbook and course materials on evaluation methods as well as research papers and help in choosing which statistical analysis would be appropriate for a set of data. It also has links to many resources for help in creating a Web page, including clip art and graphic packages, and there is even an "O.J. Page" that argues that the legal system is a good metaphor for how applied social researchers collect and weigh evidence.

And to help in applying these concepts to any use, Trochim and his wife, Mary Kane, have launched a company, Concept Systems, Inc., that provides software, training and consulting services for planning and evaluation. The company's Web site -- also designed by Trochim -- can be viewed at and includes detailed information about concept mapping and a downloadable version of the software used to accomplish it.

Trochim's Cornell Web site can be viewed at . It was developed entirely with $5,000 seed money from Cornell's College of Human Ecology. His concept mapping project to help the Ithaca community select the next chief of police is funded by a Hatch grant from the U.S. Department of Agriculture.

On all his projects, Trochim invites the public "to stop by and follow our progress on the World Wide Web."