Developing an instrument for the validation of competencies: the case of medical librarians.

Author:Anwar, Mumtaz Ali


We, the authors of this paper, planned to conduct a study of competencies required for medical librarians. We needed a validated and suitable instrument that could be confidently used in this research. An extensive search through various databases and the published research did not yield one which we could use. We knew that reliable and valid tests / instruments were the foundation of good research and that weak instruments would result in weak research. Our failure in finding a reliable instrument led us to decide to develop a new one. We also understood that the procedure for developing a new test / instrument was a long and a complicated one.

We went through that long and arduous process and developed an instrument which has already been published (Catalano, 2016, p. 109-113). The understanding of the process of designing an instrument is important as it helps in assessing its validity and reliability. This paper describes meticulously the process of developing the instrument that we used for the identification and validation of competencies required by medical librarians. It provides full details of the various steps that were undertaken to develop the instrument. The results of our research were published in 2012 (Ullah & Anwar, 2012). The present paper is being published with the intention of providing a roadmap for those who need to develop a new instrument for their research.

Previous work on competencies

Earlier work on competencies has been ably reviewed by Roper and Mayfield (1993a). The following paragraphs will briefly review some of the previous efforts made for the identification of competencies for medical librarians only.

Bowden, Bierschenk and Olivier (1989) conducted a survey in 1988 of the members of the Association of Academic Health Sciences Library Directors (AAHSLD) in order to determine important professional skills and personal qualities desired by the employers of fresh graduates. The instrument used consisted of nine skills and 13 personal traits using "very important", "important "and "not important" as the choices. It is not known how these skills and traits were identified. Bowden and Olivier (1995) repeated the same survey in 1992. It was found that 'problem solving/analytical skills', 'microcomputer skills', 'bibliographic instruction skills', 'online searching skills', 'reference/information service skills', and 'MEDLINE searching skills' were rated as 'very important'. The following personal qualities were considered as 'very important': 'communication skills', 'enthusiasm', 'self-esteem', 'flexibility', 'service orientation', 'willingness to be a team player', and 'interpersonal skills'.

Roper and Mayfield (1993a) report the results of a survey of knowledge and skills in the health information sciences conducted by the Medical Library Association using a sample of 750 health sciences librarians out of which 375 (50%) returned the questionnaire. The survey instrument listed 63 knowledge-bases categorizing these in seven areas derived from the survey of literature and expert review. The respondents were asked to indicate the importance of each knowledge or skill for effective performance on a 5-point Likert-type scale from 'essential' to 'no importance'. The results are reported in five tables indicating the level of importance of various competencies. They conclude that "If health sciences librarians are not willing to take on the responsibilities which their clientele feel are appropriate, they will be replaced by other professionals who can and will" (p. 38). A fuller version of this report was published in the Bulletin of the Medical Library Association (Roper & Mayfield, 1993b).

Giuse, et al. (1997), using a four-step needs assessment process, designed a survey instrument consisting of 96 competencies divided into 13 categories (the appendix actually lists 98 items). It was administered to 300 respondents (150 librarians and 150 library users) with 131 (77 librarians and 54 users) responding. The response rate was 43.7 per cent. Each skill was rated for its importance on a 1-7 point Likert-type scale. It was found that "all of the knowledge and skills in their categorical groupings were rated above the Likert mean and, in fact, often exceeded the mean considerably" (p. 62). These competencies are dated now.

A study on the role of the clinical librarian using content analysis of job advertisements, job titles, and duties, using a structured questionnaire administered to a sample of five practicing librarians, followed by an interview was conducted in the UK (Sargeant & Harrison, 2004; Harrison & Sargeant, 2004). It was discovered that there was "a moderate degree of congruence between recruitment specifications and working activities of clinical librarians" (Sargeant & Harrison, 2004, p. 180). This is an interesting study but limited by a small sample. Robinson et al (2005) analyzed the previous literature to identify a variety of competencies and prepared a model consisting of: Professional competency (Content knowledge and Technical knowledge), Teaching skills (Professionalism, Basis of training, Design, and Evaluation), and E-learning skills (Creation and Delivery).

The roles of academic medical librarians in Malaysia were studied by using face-to-face interviews of 18 respondents (Santra, 2007). The focus was placed on the respondents' current activities, awareness of the skills and knowledge of emerging roles, and their education and training.

The Medical Library Association (2007) in its educational policy statement listed seven areas under 'Professional Competencies for Health Sciences Librarians', each with several subareas, as follows (p. 4-5):

  1. Understand the health sciences and health care environment and the policies, issues, and trends that impact that environment

  2. Know and understand the application of leadership, finance, communication, and management theory and techniques

  3. Understand the principles and practices related to providing information services to meet users' needs

  4. Have the ability to manage health information resources in a broad range of formats

  5. Understand and use technology and systems to manage all forms of information

  6. Understand curricular design and instruction and have the ability to teach ways to access, organize, and use information

  7. Understand scientific research methods and have the ability to critically examine and filter research literature from many related disciplines

    These areas are followed by a set of recommendations for various agencies (p. 6), a list of personal attributes that contribute to success (p. 9), and then a long listing of 'health sciences information knowledge and skills' (p. 10-13). The last section could become the basis of developing competency statements.

    The published literature indicated that a variety of competencies were dealt with in many publications. However, none of these sources provided a comprehensive and satisfactory listing of competencies that we could use. It was felt that there was an urgent need for the preparation of an up-to-date and comprehensive checklist of validated competencies for medical librarians.

    Instrument development

    The survey instrument used for this study was designed using an extensive review of published literature, expert scrutiny and a pilot study as detailed below.

    The scanning of a large amount of literature resulted in a large number of raw competency statements. These statements were listed, edited, sorted, compared, and merged to form the first draft of potential competencies. This list of raw statements and the sources were carefully and critically reviewed by the researchers for refining of statements and selection of credible sources. This process resulted in a tentative list of 133 items which were derived from 24 sources.

    The 133 tentative competencies were initially grouped into the following eight subject domains: Health sciences environment and concepts (14 statements), management theory and techniques (28 statements), health sciences reference and information services (14 statements), health sciences resource management (18 statements), information systems and technologies (15 statements), instruction and teaching (14 statements), research methods (10 statements), and general and personal competencies (20 statements). Each of these 133 raw statements, followed by the sources derived from, is listed in Appendix A. Full bibliographical details of the 24 sources with the number of times each was used are listed in Appendix B. These statements were used as a raw list for critical review and study by the researchers with a view to reduce their number.

    Draft Instrument

    The raw list was revised several times in order to merge closely related and similar statements to reduce their number. The statements were merged keeping in mind that these should appear to be bona fide and natural units. This intensive exercise resulted in a draft instrument containing 87 competency statements which was submitted to the experts. Each statement was provided with a 5-point Likert scale and a column for comments by the panel of experts. Several rows were left blank at the end of each subject area if any of the experts might like to add another competency.

    Expert Scrutiny

    The draft instrument was submitted to the panel of six experts along with a covering letter. The panel consisted of two LIS faculty members (having a qualification of PhD and research contribution in the field of LIS competencies), two ex-library coordinators (medical faculty members having a research degree and contribution to the medical literature), and two retired medical librarians for content validity. The list was revised in response to feedback received from this panel. Keeping in view the comments of the panel of experts, some competencies were added and some were removed. The language of competency statements was also improved in light of changes suggested by the panel...

To continue reading