After more than 15 years of development, field completion by more than 10,000 executives, and rigorous statistical evaluation, the collection of instruments incorporated in the ExecuSmart Talent Management System is highly reliable, valid and robust. Existing research shows that past attempts to predict leader behavior based upon a single cognitive mechanism has not demonstrated the predictability that business and Human Resource Leaders find useful in assessing and developing leadership capability. The Talent Management System is predicated on the idea that key cognitive dimensions have the ability to be predictive of performance only when they are specified together in a unified model. In this article, the scholarly background of the collection of instruments is presented and the statistical testing which demonstrates a robust nature is described.
DESCRIPTION OF THE SYSTEM
The Talent Management System is an integrated set of analytical tools for the creation and management of Talent Knowledge. The Research Team established the System on a theoretical foundation, enlightened by actual field experience and application, and concentrated on a process of continuous improvement based upon actual implication impact. It includes four modules: the Leadership Success Profile, the Leadership Capability Indicator, the Context Based 360[degrees] Leadership Assessment, and the Talent Director, each of which is an instrument based tool, and each of which contributes to the completion of a system of talent management for organizations involved in executive and leadership assessment, evaluation, training and development.
The first step in the process is the Leadership Success Profile. This instrument, which requires about 45 minutes to complete, is administered to an organization's subject matter experts: those individuals who have a high level of knowledge about the company and the specific position in the organization which is under review. The instrument assesses the shared perspectives of the top management team of the specific, ranked, leadership competencies which are necessary for success in a specific leadership position within that organization, taking into consideration the market and competitive environment of the organization, its strategic orientation and culture. These ranked competencies are therefore unique to the individual company, and form a foundation for understanding the leadership needs of a given position within that company. The process uses a forced choice format built upon a foundation of redundancy which avoids the potential for response bias and fosters the development of a set of Tier One, Two and Three, empirically defined specific competencies required for success by an individual executive in a specific leadership position within a specific organization.
The second step in the process is the Leadership Capability Indicator. This instrument, which requires about 45 minutes to complete, is administered to a specific individual who is the subject of assessment, evaluation, training or development. It is a forced choice format in 22 dimensions which assesses an individual's self evaluation of his or her leadership behavior predispositions with respect to Five Cs of Leadership: Leadership Capacity, Leadership Character, Leadership Communication, Leadership Collaboration and Leadership Change. It produces an empirically defined perspective of an individual's cognitive natural state with regard to the various leadership behaviors and compares those to benchmark averages of the other executives in the specific organization, and to a selected group of executives, which can consist of Fortune 1000 or Inc. 500 executives, or executives from any of more than 40 specific industries.
The third step in the process is the Context Based 360[degrees] Leadership Assessment. This instrument, which requires about 45 minutes to complete, is administered to a specific individual in a specific position within an organization, and to that individual's superiors, peers, and subordinates. The instrument is a forced choice format in 16 dimensions which permits the identification of perceived and preferred behaviors with respect to the Four Types of Leadership: Directive Leadership, Transactional Leadership, Transformational Leadership, and Empowering Leadership. It produces an empirically defined comparison of an individual's perspective of the intensity of a leadership behavior which is desired by the organization to that individual's self evaluation of his or her delivery of that leadership behavior. Further, the instrument produces a comparison of the individual's perceptions to those of his or her superiors, peers and subordinates.
The fourth step in the process is the Talent Director, which is a graphical interface of the previous three measures. Specifically, the Talent Director prioritizes graphical representations of the outcomes of the Leadership Capability Indicator and the Context Based 360[degrees] Leadership Assessment, to the prioritized leadership competencies identified by the organization through the Leadership Success Profile. The outcome is an empirically derived fit index which demonstrates the percentage of each specific desired leadership competency which the individual embodies in his or her cognitive predispositions.
Taken as a whole, the Talent Management System is a complete package which provides empirical support to an organization in the executive recruitment and assessment process, executive evaluation process, and executive training and development process. As it is customized to a given organization and position, it is useful in determining positional fit and in establishing experience ladders for targeted, long term, executive development.
THEORETICAL FOUNDATION, RELIABILITY AND VALIDITY OF THE LEADERSHIP SUCCESS PROFILE
The first component of the System, the Leadership Success Profile, began with a rejection of traditional competency model building. The traditional approach is to utilize leadership competency interviews with selected executives or teams of executives within an organization, or to rely on focus groups of executives within the organization. Such approaches are highly dependent on the expertise of the facilitator. They are extremely time consuming and can be prohibitively expensive. Further, they are subject to varying and unidentifiable levels of response bias. When groups are used, differences of opinion which arise within the groups are difficult to resolve and may lead to yet more response bias and political issues. When individuals are surveyed, establishing priorities among the resulting myriad of competencies becomes highly subjective.
The Research Team evolved an approach which eliminates response bias and empirically and independently establishes priorities of leadership competencies. The approach involves beginning with a predetermined set of leadership competencies. The Leadership Success Profile starts with a list of 45 specific leadership competencies which the research team identified as having strong theoretical support in the literature. The specific competencies have been identified over a period of years and are still evolving as research team members continue their research. These competencies are displayed in Table One, along with the specific theoretical support for each competency. Administrators of the System can expand the initial list when requested by a specific organization with competencies which its executives feel are omitted, however, training of the administrators emphasizes the need for extreme care in such an expansion, due to the potential for response bias identified above.
The Research Team recognized that a strong library of leadership competencies has limited value for a specific organization without empirical prioritization. The number of competencies which are identified as valuable is simply too large to be of practical value to the organization. Those competencies must be ranked in a way which establishes the most critical of skills without conflict of interest challenges and without participant bias. Consequently, the Team developed a Patent Pending process which is based on an advanced conjoint application. Such conjoint applications have been demonstrated to be effective in a range of situations that require an understanding of the importance of certain characteristics or capabilities (see Louviere, 1988; Wittink and Cattin, 1989; Green and Srinivasan, 1990; and Krishnan and Ulrich, 2001, for some key examples). As described above, the conjoint application involves a forced choice pairing of the various competencies in a redundant model which permits the independent, empirical determination of ranked outcomes.
The outcomes are tested for agreement using James, Demaree, and Wolf's (1984) RWGJ approach to testing and understanding agreement. This provides the administrator an understanding of agreement or consistency between the various executives who have been involved in the process. Further, the system produces specific measures of the consistency of each individual's responses across the large number of matched pairs. The combination of the two tests informs the administrator as to the level of agreement on each competency and the consistency of the responses of each executive participant. Administrators are trained to conduct outlier analysis when the results of the tests demonstrate the need. Administrators are cautioned, however, as to the danger of excessive interference in the process as each human intervention has the potential to introduce bias.
From a theoretical perspective, Hoyle, Harris and Judd (2002) were very clear that a field application with a larger number of subjects will have higher reliability and validity. Podsakoff and Organ (1986) reported that designs with multiple informants are capable drivers of reliability and validity. The Leadership Success Profile incorporates both...