Vocational Assessment essay

A case study on the assessment of learners in relation to a specific element of training: a skills set

            This case study presents the assessment of learners in relation to a specific element of training or education. The element selected for assessment is a skill set. Vocational assessment is a properly developed process that is aimed at appraising the level of functioning of learners in relation to their vocational preparation.

            Overview of the learners, learning element, and context

            The group of learners involved in the assessment process includes young people aged 16-18, both male and female students, coming from different cultural, racial, and ethnic backgrounds. The learning element is a skill set appropriate for the identified age group of learners. The skills set includes the major skills that are necessary to perform the tasks in the selected area. These skills include problem-solving skills, cooperation skills, and the skills related to compliance with vocational ethics and maintenance of health and safety, communication skills, aesthetic skills, and technical skills.

            Assessment plan

            In order to design and develop an assessment plan, it is crucial to determine the key areas that will be addressed in the assessment. Assessment of learners is “criteria based, which is to say that a student’s learning and competence is always compared to either the skills requirements of vocational qualification modules or the objectives of core subjects as well as the assessment criteria based on them” (Vocational Qualification in Beauty Care, 2009, p. 7). According to career development theory,

data should be collected in order to help learners in their planning of developmentally appropriate vocational goals. The assessment plan has three steps or stages, namely screening/needs assessment, exploratory and comprehensive vocational evaluation stages.

1. Screening/needs assessment stage

A needs assessment process is based on a systematic exploration of the needs of learners in developing an appropriate plan of action.

1. Interviewing learners
2. Collecting the proper information about learners’ background
3. Using subjective statements of learners, including vocational choice or interest, self-assessment of competence and reported work history data
4. Conducting functional assessment by means of personal skills and abilities questionnaire – the Myers-Briggs Type Indicator (MBTI)
5. Conducting limited unstandardized tests on learners’ interests by means of  Cards Sorts assessment instrument

1. Exploratory stage

This stage is based on conducting a case study of learners’ skills and their application to vocationally related circumstances. It is focused on the application of information obtained through stage 1 through the use of the following assessment approaches:

• Conducting personal interviews in the context of the selected assessment instruments,  the Myers-Briggs Type Indicator (MBTI) and Card Sorts instruments
• Collecting and analyzing documentation (e.g. academic achievement records, medical records)
• Exploration of career opportunities based on the results of the assessment of learners’ skills
• Providing vocational and adjustment counseling based on the results of the assessment of learners’ skills
• Psychometric/standardized testing (if necessary)
• Proving analysis of transferable skills of learners based on the results of the assessment of learners’ skills
• Conducting computerized job-matching based on the results of the assessment of learner’s skills
• Conducting job analysis and/or environmental assessment based on the results of the assessment of learners’ skills
• Applying assistance technology considerations (if necessary) (Vocational Assessment and Evaluation, 2016).

Actually, the above-mentioned assessment approaches are linked to the two assessment tools: the Myers-Briggs Type Indicator (MBTI) and Card Sorts instruments.

1.  Evaluation stage

Evaluation is based on the systematic observation of learners’ behavior, performance, and interests and the recording of relevant data by means of different methods and techniques, including:

• Using job samples 
• Conducting situational assessments and community-based assessment
• Conducting standardized testing
• Conducting interviews
• Assessment of transferable skills and job matching
• Providing analysis of background information of learners
• Providing career exploration 
• Applying assistance technology considerations
• Providing prescriptive recommendations.

            Rationalizing the plan using the principles of assessment for, as, and learning

            Applying the principles of assessment for, as, and of learning, it is possible to give a logical explanation of the assessment plan to justify its consistency. There are three principles of assessment that should be considered, including assessment for learning, assessment as learning, and assessment of learning. These principles can be viewed as effective approaches that help to collect valuable information about learners’ achievement. As a matter of fact, the principles of assessment for learning and assessment as learning share common elements that should be reflected in the assessment plan. These ones include self-assessment and peer-assessment strategies, the strategies developed for learners to ensure monitoring and evaluation of learning, and feedback that allows determining further steps in the process of learning. The proposed assessment plan provides an opportunity to assess current learning experiences and their ability to strengthen knowledge, enhance understanding and develop new skills (McCrae & Costa, 1998; Peterson, 1998; Osborn & Zunker, 2015).

            Presentation of assessment instruments

            There are two assessment instruments that should be used in the process of assessment of learners’ skills:

1. the Myers-Briggs Type Indicator (MBTI)

It is a formal guided exploration that is presented in the form of a standardized formal online assessment instrument specially developed to facilitate the assessment process (McCrae & Costa, 1998). The Myers-Briggs Type Indicator (MBTI)  is used to achieve the following goals in the assessment process:

• Ito identify the personality preferences of a learner
• to select a career option
• to provide a better understanding of the need for enhancing professional relationships
• to improve leadership skills and collaboration with other learners involved in teamwork
• to assess the learner’s personality type and have a good understanding of the personality traits and their role in shaping communication, data collection, decision-making, problem-solving and organizational culture (McCrae & Costa, 1998).
• Card Sorts

It is possible to select Card Sorts or card sorting instruments as an effective assessment tool used by career counselors in the process of vocational assessment. Card sorts can be viewed as interactive assessment instrument that helps to determine and prioritize the skills of learners to address the needed career values. This assessment instrument can be used in person during the assessment process. Skills Card Sorts helps to achieve the following goals:

• to identify the interests of learners in applying certain skills in the selected career
• to select a career option
• to have a good understanding of professional relationships
• to strengthen learners’ values in the selected career option
• to identify possible changes in the values of learners (Osborn & Zunker, 2015).

In addition, it is necessary to consider the benefits of using occupational card sorts in vocational assessment, including active participation of the client in the decision-making process, the immediacy of the assessment results, low-cost process, and informal environment compared to written or computerized assessment procedures (Osborn & Zunker, 2015). Besides, card sorts are based on fostering interpersonal communication and interaction, which contribute to the identification of certain interests, values, and attitudes. Researchers suggest that the card sorts instrument is “helpful when a client has obtained a flat profile on an interest inventory” (Osborn & Zunker, 2015, p. 168). In fact, this a “seven-step, 36-card, vocational card sort mapping task with concurrent verbalizations is presented which may be used in the initial stages of career exploration to assist counselors in assessing a client’s development and integration of occupational knowledge” (Peterson, 1998, p. 47).  Based on cognitive theory, the use of this assessment tool is based on certain steps in the process of assessment to obtain valuable information on the skills set developed by an individual (Peterson, 1998).

            Analysis of assessment methods

            The analysis of the selected assessment methods shows that they are consistent with the key outcomes of the assessment process. According to researchers, the Myers-Briggs Type Indicator is an effective tool, which measures the key aspects of the dimensions of normal personality (McCrae & Costa, 1998). Besides, this assessment tool helps learners to make career decisions or address the challenges associated with life circumstances. This tool contributes to the development of career adaptability and maturity through the use of the proper decision-making styles and dealing with personality-related issues (Swanson & D’Achiardi, 2005). According to experts, “Jung’s four dimensions represented in the Myers-Briggs Type Indicator are used in career counseling for the purpose of helping clients understand their own decision-making style” (Swanson & D’Achiardi, 2005, p. 361). Actually, the currently available vocational assessments are aimed at measuring an individual’s physical body functions and learning capacity, as well as can be used to predict work performance. There is much evidence that the key vocational assessments, including the Myers-Briggs Type Indicator, help to assess the functional capacity of learners (McFadden et al., 2010). Based on social cognitive career theory (SCCT), it is crucial to consider individual differences and contextual influences in the process of career decision-making. The role of personality and social support in vocational assessment is significant as it contributes to self-efficacy in career decision making and to positive outcome expectations in the career planning process (Rogers et al., 2008).

            Moreover, the Card Sorts method is known as a non-standardized and subjective assessment tool widely used in the field of career counseling to assist clients in clarifying their skills, abilities, and career interests. According to recent studies, “advancements in conceptualizations about career and career development and irreversible changes in the world of work have necessitated that career counselors reflect on their practice in order that it keeps pace and maintains relevancy” (McMahon & Patton, 2002, p. 51). Card Sorts is used in career assessment to benefit clients in their decision-making through the enhancement of relevant skills and abilities. Card sorts tool in the form of qualitative assessment which is essential in career counseling. As a rule, card sorts are based on particular themes and topics of interest in the selected field, such as occupations, work-related skills, values, motivation, job satisfaction, etc. At the same time, the process of the card sort assessment includes life situations, perceptions, beliefs, and attitudes (McMahon & Patton, 2002, p. 62).

            Appraisal of assessment instruments

            In order to appraise the two assessment instruments, namely the Myers-Briggs Type Indicator (MBTI) and Card Sorts, it is possible to explore how these methods of assessment of learners’ skills are consistent with the concepts of validity and reliability. In fact, the concepts of validity and reliability are used to explore the effectiveness of the assessment process, which should be sound and free of bias.  Reliability stands for the extent to which the selected assessment instrument is consistent in measuring learners’ skills.  The assessment instruments should be used several times a week or a month to ensure that there is a correlation between the results of the assessment. Validity stands for the accuracy of the assessment process. It is necessary to determine whether or not it provides a measurement of learners’ skills. The content of both the Myers-Briggs Type Indicator (MBTI) and Card Sorts matches the key vocational assessment objectives. Hence, the two assessment tools selected to measure the skill set of learners are both reliable and valid.

            In general, the key goals and objectives of the assessment process should be clearly defined to succeed in the use of assessment instruments. Besides, the application of qualitative assessment instruments has many benefits both for the counselor and the client. These assessment instruments are more informal and guarantee a flexible approach to resolving various problems. Besides, these instruments do not depend on statistical analysis that requires certain competencies in terms of scores, norms, and data associated with reliability and validity issues. The Myers-Briggs assessment instrument is insightful, reliable, valid, and thought-provoking. Card Sorts instrument is easy to use, engaging and accurate. It allows the clients to be aware of the results of the assessment due to active participation in the process of card sorting. Finally, it helps to identify the areas of strengths and weaknesses to ensure that the skills of learners are transferable. Although critics of the Myers-Briggs assessment tool state that it demonstrates significant psychometric deficiencies, such as poor level of validity that is the inability to measure what should be measured, poor level of reliability is the inability to provide the same results for the same individual on different occasions, this assessment tool can be used effectively if the goals of assessment are set and they are easy to understand and achieve.

            All in all, the two assessment tools discussed in this paper are interview-based assessment methods aimed at measuring learners’ skills in the selected area. Both assessment tools are used to gain accurate information regarding the individual’s interests, values, beliefs, attitudes, and goals to ensure active participation in professional development and personal growth activities.

            Discussion and application of accommodation as an assessment principle

            It is necessary to consider accommodation as an assessment principle that allows assessing learners effectively. The proper knowledge should be applied to have a good understanding of the assessment of the selected skills. The assessment instruments are effective in assessing learners’ knowledge, skills, and abilities. The selected accommodations address the needs of learners because of the necessity for considering the specific characteristics of the assessment tools. The major characteristics that should be addressed include: “the number of questions, the time allotted, if there is a lot of reading involved, the expected length of responses to be provided orally or in writing, etc.” (Principles for assessment accommodations, 2017).

            In addition, it is crucial to apply the major principles which contribute to the quality of assessment. These principles help to determine what accommodations are appropriate for the vocational assessment of learners’ skills (Principles for assessment accommodations, 2017). In other words, these principles are guided by values like accessibility, fairness,  representativeness, and transparency. As it is necessary to comply with the professional standards in the process of assessment of learners’ skills, the following principles should be followed:

1. providing learners with equal opportunities to ensure demonstration of their skills, abilities, and qualifications

This principle allows learners to demonstrate their competence in the selected area and identify qualifications for the proper assessment. Besides, this principle is designed to accommodate learners’ needs in the context of vocational assessment (Principles for assessment accommodations, 2017). There are some circumstances that may have a negative impact on learners’ participation in the process of assessment. It is crucial to make adjustments to the key procedures to allow full demonstration of skills and abilities by learners. This principle is based on promoting fairness in assessment. In the context of assessment, fairness stands for using fair approaches to the assessment of learners’ skills (Principles for assessment accommodations, 2017).

• Identifying relevant assessment accommodations

This principle is based on the three elements that ensure  the proper assessment accommodations:

• Considering the nature and the extent of functional limitations of individuals involved in the process of assessment
• Considering the type of assessment instrument selected 
• Considering the nature and level of skills being assessed (Principles for assessment accommodations, 2017).

Accordion to experts, “since the information on these elements varies from one person to another and from one assessment situation to another, each request for assessment accommodations must be considered individually, using a case-by-case approach” (Principles for assessment accommodations, 2017). Besides, it is necessary to consider the variability of these elements to select appropriate accommodations to deal with particular assessment situations (Principles for assessment accommodations, 2017).

• Avoid altering the nature and level of skills being assessed

This principle is designed to prevent making changes or modifications in the assessment procedures (e.g. changes in the content of the assessment, changes in the format of assessment, etc.) in order to avoid any challenges with an assessment of skills of learners. Recent studies on the impact of assessment accommodations point out the fact that any small changes or modifications performed in the standard assessment procedures may have considerable effects on the nature and level of skills being assessed (Thompson et al., 1999). This allows avoiding putting doubt on the validity and reliability of assessment instruments, as well as the results of the assessment (Thompson et al., 1999).

• Focusing on complete information to provide a high-quality assessment accommodations

This principle help to make the proper decisions while selecting the type of assessment accommodations appropriate for the assessment process. It is required to be reliable in the use of complete information, based on the identified elements to provide the case-by-case approach (Thompson et al., 1999).

            All in all, assessment accommodations have the key goal – “to provide equal access to academic and equal opportunities, not unfair and preferential access” (Lichtenberger et al., 2012, p. 23). Assessment accommodations should provide learners with fair advantages through adaptations or adjustments to task performance (Lichtenberger et al., 2012).

            Discussion of the holistic nature of assessment

            The process of vocational assessment is complicated because it consists of several assessment procedures, including the use of informal and formal instruments, interviews, and diagnostic evaluations. The assessment of learners’ skills is holistic in its nature. Multiple procedures are used in this process. It is closely linked to generic skills and the key competencies associated with the identified learning context. This vocational assessment is presented in the form of the process, which helps to determine the learner’s interests, preferences, abilities, values, and skills to foster the identification of the major strengths of learners, their needs, and career potential. In fact, vocational assessment depends on the usage of various standardized techniques, including testing, as well as non-standardized methods, such as interviews and observations. This vocational assessment is considered to be an essential part of the vocational guidance procedure that provides a set of reliable recommendations for further employment. According to researchers, “a holistic approach seeks to develop critical, confident and independent learners capable of action in their professions and in society generally” (Patel., 2005, p. 12). The assessment of learners is holistic in its nature as it promotes critical thinking skills in learners.

            In general, the process of vocational assessment can be characterized as an individualized, creative, interesting, encouraging, holistic process, which contributes to experiencing self-discovery. In other words,  the process of evaluation allows identifying both the individual’s strengths and his/her potential of acquiring a specific skill set (Power, 2013). Hence, the major principle that establishes the basis of vocational assessment acknowledges that “all human assessment should be holistic and humanistic” (Vocational Evaluation and Work Adjustment Association, 1993, p. 54). There is much evidence that this holistic process allows obtaining accurate information to provide a big picture of the learner’s skills  (Neukrug, 2015).

            Furthermore, the process of vocational assessment requires developing a close relationship between competence and performance assessments. Due attention should be paid to the increased value of knowledge, development of competence‐based assessment principles, and evaluation of assessment results. The purpose of this assessment process is to provide data on learners’ skills. Hence, the individuals who participate in the process of vocational assessment should be prepared to provide fair responses on their interests, values, skills, abilities, and attitudes. This vocational assessment is consistent with the educational background of learners. The major findings provided by the direct approach to vocational assessment can be applied in practice. The vocational assessment provides certain recommendations on the individual’s vocational goals to assist in future employment (McFadden et al., 2010; McMahon & Patton, 2002; Patel., 2005; Power, 2013; Swanson & D’Achiardi, 2005).

References

            Guide to the assessment of practical skills in  International Vocational Qualifications. Retrieved from: www.cityandguilds.com

            Lichtenberger, E. O., Mather, N., Kaufman, N. L. & Kaufman, A. S. (2012). Essentials of Assessment Report Writing. John Wiley & Sons.

            McFadden, S., MacDonals, A., Fogarty, A., Le, S., Merritt, B. K. (2010). “Vocational assessment: a review of the literature from an occupation-based perspective,” Scandinavian Journal of Occupational Therapy, 17(1): 43-48.       

            McCrae, R. R. & Costa, P. T. (1998). “Reinterpreting the Myers-Briggs Type Indicator From the Perspective of the Five-Factor Model of Personality,” Journal of Personality, 57 (1): 17-40.

            McMahon, M. & Patton, W. (2002). “Using Qualitative Assessment in Career Counselling,” International Journal for Educational and Vocational Guidance, 2(1): 51-66.

            Neukrug, E. S. (2015). The World of the Counselor: An Introduction to the Counseling Profession. Cengage Learning.

            Osborn, D. S. & Zunker, V. G. (2015). Using Assessment Results for Career Development. Cengage Learning.

            Patel., N. V. (2005). Critical Systems Analysis and Design: A Personal Framework Approach. Psychology Press.

            Peterson, G. W. (1998). “Using a Vocational Card Sort as an Assessment of Occupational Knowledge,” Journal of Career Assessment, 6(1): 49-67. 

            Power, P. W. (2013). A Guide to Vocational Assessment. Pro-Ed.

            Principles for assessment accommodations. (2017). Public Service Commission of Canada. Retrieved from: www.psc-cfp.gc.ca/plcy-pltq/guides/assessment-evaluation/apwd-eph/ch3-eng.htm

            Rogers, M. E., Creed, P. A. & Glendon, A. I. (2008). “The role of personality in adolescent career planning and exploration: A social cognitive perspective,” Journal of Vocational Behaviour, 73(1): 132-142.

            Swanson, J. L. & D’Achiardi, C. (2005). “Beyond Interests, Needs/Values, and Abilities, Assessing other important career constructs over the life span,” in Career Development and Counseling: Putting Theory and Research to Work, ed. by Steven D. Brown, Robert W. Lent. John Wiley & Sons.

            Thompson, S., Blount, A., & Thurlow, M. (2002). A summary of research on the effects of test accommodations: 1999 through 2001 (Technical Report 34). Minneapolis, MN: the University of Minnesota, National Center on Educational Outcomes.

             Vocational Evaluation and Work Adjustment Association. (1993). Sixth National Forum on Issues in Vocational Assessment: the issues papers: papers presented at the Sixth National Forum on Issues in Vocational Assessment, March 4-6, 1993, Virginia Beach, Virginia.

            Vocational Qualification in Beauty Care. (2009). Finnish National Board of Education. Retrieved from: www.oph.fi/download/140409_vocational_qualification_in_beauty_care_20009.pdf

            Vocational Assessment and Evaluation. (2016). P. W. Bosswell & Associates. Retrieved from: http://www.employment-training.org/vocational_assessment_index.html