![\includegraphics[width=0.8\linewidth]{figs/cognitiveModel.eps}](Timg13.png)
|
The gulf of evaluation is the gap from the tool output to the intended goal, and it must be bridged by another three consecutive efforts:
The cognitive model in Figure 3.1 is a refinement of the location-comprehension-modification cycle (LCM-cycle) of a reuse process in Figure 1.1 [Fischer et al., 1991], with special emphasis on the location process, which is the focus of this research. Although the LCM-cycle acknowledges and stresses the difficulty of formulating appropriate reuse queries in the location process, it does not point out that formulating reuse queries must be preceded by the forming of reuse intentions. Furthermore, the comprehension step in the LCM-cycle does not differentiate two different levels of comprehension: comprehension for choosing components and comprehension for integrating components. Comprehension for choosing is still a part of the location effort because it may result in the reformulation of queries.
Unlike the syntax of programming languages, which can be learned through schooling or tutoring before programmers start working, the mastery of reusable components cannot be completed in classrooms or merely by reading books [Ye, 1998]. Due to the large volume of reusable components and their constantly evolving nature, total coverage is impossible and obsolescence is unavoidable. Moreover, learning components is less effective when the components are separated from their use context; components are better learned when they are needed for a programming task. Therefore, component learning needs to be integrated with working where the components are reused, and programmers should learn components on demand--that is, learn the component when it is needed [Fischer, 1991]. However, a pitfall for the learning-on-demand model is that programmers need to learn a component because they do not know it, but because they do not know it, they may settle on a suboptimal solution by creating their own program instead of reusing the existing component. To support learning-on-demand, component repository systems should be able to identify learning (and reusing) opportunities by connecting programmers to the components that can be reused in their current task.
There are two types of conceptual gap between situation model and system model: vocabulary mismatch and abstraction mismatch. The vocabulary mismatch refers to the inherent ambiguity in most natural languages. Thanks to the richness of natural languages, people use a variety of words to refer to the same concept. Based on their systematic study of word use of ordinary people in different domains, Furnas et al. have found that the probability that two persons choose the same word to describe a concept is less than 20% [Furnas et al., 1987]. Even well-trained indexing experts have a 20% disparity on average in choosing terms to describe the same document [Harman, 1995].
The abstraction mismatch refers to the difference of abstraction levels in requirements and component descriptions. Programmers deal with concrete problems and thus tend to describe their requirements concretely, whereas reusable components are often described in abstract concepts because they are designed to be generic so they can be reused in many different situations. For example, in one experiment to evaluate the CodeBroker system, one subject described his task as follows: 3.4
/** This class contains methods for converting between
western-style numbers (three numbers in a set with a
comma) and Chinese style numbers (four numbers followed
by a comma). For example, 1,000,000--> 100,0000. */
Another subject initially described the same task in a similar way:3.5
/** Takes a string with a Chinese formatted number and
outputs a western formatted number. */
This task can be easily implemented by setting the group size to 4 with the method setGroupingSize of the class java.text.DecimalFormat. However, the description of this method (as follows) is abstract: It describes grouping of numbers without mentioning western style or Chinese style in particular.
public void setGroupingSize(int newValue) Set the grouping size. Grouping size is the number of digits between grouping separators in the integer portion of a number. For example, in the number "123,456.78", the grouping size is 3.
Many retrieval mechanisms have been proposed in the past (for more detailed descriptions, see related work in Section 9.2). There are three major approaches: text-based, descriptor-based and formal specification-based. In text-based approaches, components are represented by their textual documents and information retrieval technology is used to match components to queries [Maarek et al., 1991]. In descriptor-based approaches, components are represented by a set of selected descriptors. The semantic relationships among those descriptors are captured in a predetermined structure that can be specified by a semantic network [Henninger, 1997], an AI frame [Ostertag et al., 1992], a taxonomic category system [Devanbu et al., 1991], or a fuzzy set theory [Damiani et al., 1997]. In specification-based approaches, components are represented with formal specification languages, and automatic theorem-proving systems [Zaremski and Wing, 1997] or specification refinement systems [Mili et al., 1997a] are used to determine whether a component matches a query, written in formal specification languages too.
In terms of complexity of representation schemata, the text-based approach is the simplest and the specification-based approach the most complicated. In general, a complete and precise representation can make the matching more precise and retrieval more effective. However, because the same representation is also used by programmers to specify their reuse queries, the schema of representation is greatly limited by programmers' willingness to formulate long and precise queries. There is no point in representing every bit of relevant information about a component if a programmer barely has the patience for typing string search regular expressions [Mili et al., 1995].
Comprehension for choosing is focused on what the component does, and it is conducted in two stages: information discernment and detailed evaluation [Carey and Rusli, 1995]. At the stage of information discernment, programmers avoid spending too much time by quickly scanning the component and its description to decide whether this component is related to their current task, and thereby also avoid any deep understanding at this point [Lange and Moher, 1989]. The process of information discernment may result in the reformulation of queries if programmers find the retrieval results are not satisfactory. Only when a promising component is found do programmers start to evaluate the components extensively.
To integrate a component into their programs, programmers need to understand the component's functionality, its usage, and even its implementation details, especially in cases of white-box reuse and glass-box reuse (Section 3.1.2). Executable examples that use the component prove to be very useful to help programmers quickly understand how to reuse the component in their own programming task [Redmiles, 1992,Aoki et al., 2001].