This chapter presents the results of two types of evaluations conducted on the
CodeBroker system. The first evaluation compares the retrieval
effectiveness of the Okapi-based retrieval mechanism and the LSA-based
retrieval mechanism (Section 6.1.1). The second
evaluation empirically studies how
well CodeBroker supports reuse-within-development (Section 4.2.2
through experiments with programmers.
The purpose of the empirical studies of CodeBroker was not to analyze the
quality of programs produced by programmers and the productivity of
programming, but to observe and analyze how the system promotes reuse
during programming.8.1 The empirical studies attempted to answer
the following questions:
- Are programmers able to reuse unknown software components with
the support of CodeBroker?
- Does CodeBroker encourage programmers to explore the possibility of reuse?
- Is the task modeling based on doc comments and signatures good enough to
find components relevant to the task at hand?
- Do discourse models improve the relevance of delivered components?
- Do user models contribute to the personalization of component
delivery?
Subsections
Ph.D. Dissertation by Yunwen Ye, April 20, 2001, Department of Computer Science, University of Colorado