A test of the validity of the LCSW examination: quis custodiet ipsos custodes?

Author:Albright, David L.
Position:Licensed clinical social workers - Report
 
FREE EXCERPT

The authors administered to 59 first-year MSW students the practice test for the clinical-level examination developed by the Association of Social Work Boards (ASWB); the latter is used by most jurisdictions to qualify individuals to practice as licensed clinical social workers. To assess the practice test's validity, the authors blanked out actual questions, leaving only four choices (one correct answer and three incorrect ones) and instructed students to select correct answers. On the basis of chance alone, students should have answered an average of 25% of items correct. In reality, they correctly guessed 52% of the items, a statistically significant difference. This suggests that the practice test (said by ASWB to contain items like those on actual examinations) is excessively transparent or guessable and cannot justifiably be claimed to be a valid assessment of competence to practice social work. The authors encourage ASWB to publish in independent peer-reviewed .journals a series of psychometric studies on the reliability and validity of the licensing examinations they have developed. The multimillion dollar social work testing industry is big business; both the profession and the public, which relies heavily on the gate-keeping function of these tests, deserve greater transparency and accountability with respect to their legitimacy.

KEY WORDS: assessment; Association of Social Work Boards; LCSW; replication; validity

**********

In 2007, over 8,000 social workers paid for and took the clinical social work examination for the first time, and of these, fewer than 75% passed the test (Association of Social Work Boards [ASWB], 2008a, 2008b, 2008c). Over 12,000 additional social workers completed other ASWB examinations (associate, bachelor's, master's, or advanced generalist). With a fee of $175 per individual examination, the social work testing industry is a multimillion dollar business. ASWB (formerly the American Association of State Social Work Boards), the independent organization that prepares, markets, and administers the tests, was incorporated in 1979 under the auspices of state credentialing bodies to promote consumer protection and to

assist social work regulatory bodies in carrying out their legislated mandates, and to encourage jurisdictional efforts to protect a diverse public served by social workers who are regulated through common values, ethics, and practice standards. The Association will help foster public and professional understanding of the value, competency, and accountability of regulated social workers. (ASWB, 2007, p. 4) The ASWB developed and began administering social work examinations in 1983, and it claims that its tests "continue to be one of the most important assurances that a social worker possesses the competence to practice responsibly" (ASWB, 2007, p. 1). Prerequisites for being allowed to take the clinical examination typically are that the applicant have earned his or her MSW from a program accredited by the Council on Social Work Education and have had at least two years of postdegree supervised clinical practice experience. Each examination is composed of a total of 170 multiple choice questions, with each item consisting of a stem (or question), one correct response (the key), and three distractor (incorrect) answers. Only 150 of the 170 questions are used in determining the candidate's actual score; the remaining 20 questions are being pilot tested for future use.

The ASWB conducted its last practice analysis of social work in 2003.A new one will be released shortly. The practice analysis is a survey of licensed social workers that asks them to list their common professional tasks, rate how often they perform each task, the importance of the task regardless of the frequency of performance, and whether or not it is a necessary skill. The periodically updated practice analysis survey is developed and piloted, sent to participants, and the results are used to update examination content. The results of the practice analysis "determine the nature and distribution of questions that appear on the ASWB examinations" (ASWB, 2008c, p. 6; see also ASWB, 2008a).

The ASWB content outline for the clinical examination is currently as follows: human development and behavior in the environment (22% of the test's items); issues of diversity (6%); diagnosis and assessment (16%); psychotherapy and clinical practice (16%); communication (8%); the therapeutic relationship (7%); professional values and ethics (10%); clinical supervision, consultation, and staff development (4%); practice evaluation and the utilization of research (1%); service delivery (5%); and clinical practice and management (5%) (ASWB, 2007). This outline determines the types of questions that appear and how they are distributed on the ASWB clinical examination.

The face and content validity of the ASWB examination is assessed by an overview of the individual items (question, key, and distractors) by experts within the field. The ASWB examination committee is composed of social workers residing in jurisdictions that legally regulate social work, including the 49 U.S. states; Washington, DC; the Virgin Islands; and several Canadian provinces. ASWB Examination Committee members are required to be licensed and are trained to write items, as are other licensed social workers selected and trained by the ASWB to prepare new test items. ASWB states that items "are not created in a vacuum, [nor] are they intended to trick or purposely confuse candidates" (ASWB, 2007, p. 5). Questions generated by members and other trained item writers are pretested by mixing them in with regular examination questions and analyzed to ensure appropriateness for inclusion on the actual examination.

Licensed clinical social workers (LCSWs) have the right to provide important social, health, and mental health services, including (among others) the right to assess and diagnose; to...

To continue reading

FREE SIGN UP