Authentic assessment of English - Majored undergraduates’ English proficiency

of the thesis In theory, the thesis: - helps build upon the body of literature on authentic assessment of communicative language competence of EFL learners. - helps clarify the theoretical basis and develop a theoretical framework for the integration of authentic assessment into the assessment of English-majored undergraduates’ English proficiency at universities in Vietnam. - clarifies purposes, definitions, roles and features of authentic assessment. - describes and analyzes the procedure, methods, criteria and rubrics used in the incorporation of authentic assessment in the process of teaching, learning and assessment of English-majored undergraduates’ English proficiency. In practice, the thesis: - identifies and describes the common practices of the testing and assessment of English-majored undergraduates’ English proficiency at universities in Vietnam, and provides an analysis and evaluation of the practices, with focus placed on the assessment activities conducted in the teaching and learning process. - proposes a procedure (including three stages: designing, implementing and evaluating) and 4 measures (used to assess the four language skills of listening, reading, speaking and writing) for teachers to employ authentic assessment in their teaching and assessment of English-majored undergraduates’ English proficiency. - helps confirm that the proposed procedure and measures worked effectively by getting feedback from specialists, teachers and students in the areas of language teaching, learning and assessment and by experimental activities. 10. Organization of the thesis 6 In addition to the parts of Introduction, Conclusions and Recommendations, References and Appendices, the present thesis consists of 4 chapters: Chapter 1: Literature review Chapter 2: Current practices of integrating authentic assessment into the assessment of English-majored undergraduates’ English proficiency Chapter 3: Procedure and measures for integrating authentic assessment into the assessment of English-majored undergraduates’ English proficiency at universities in Vietnam Chapter 4: Experiment CHAPTER 1: LITERATURE REVIEW 1.1. A brief overview 1.1.1. Research on assessment in language instruction A lot of research has been done on instructional assessment with focus placed on the discussion of definitions, goals and objectives, roles and functions, characteristics, forms and methods of assessment. Based on the findings of various studies [11, 14, 23, 43, 97], assessment can be defined as a process of gathering information about learners, discuss, analyze and explain the information in a systematic way in order to understand what learners know and can do from the knowledge and experiences they gather, compare with the goals and objectives of the curriculum and provide stakeholders (learners, teachers, managers, employers, etc.) with the information so that they can make decisions (e.g. for recruitment) or make adjustments or changes to increase the quality of teaching and learning. With reference to the findings of various studies [11, 43, 54, 60, 61, 92], assessment can be used to serve: - administrative goals: for placement, selection, certification, decision- making by stakeholders (e.g. school managers, recruiters, employers); 7 - instructional goals: to gain an understanding of the teaching and learning process, learners’ achievements, learners’ competence, etc. so that teachers and learners can make adjustments or changes to improve the instructional process. Studies also discuss different forms of assessment categorized based on different criteria. For example, based on the criterion of time of assessing, assessment can be diagnostic (before the learning process), formative (during the learning process) or summative (after the learning process) [11, 23]. Methods used in the assessment of learning vary, but to a certain extent they can be summarized as follows: assessment of knowledge (e.g.: objective testing), performance-based assessment (e.g.: presentation, project-based assessment, portfolio) and observation [11]. 1.1.2. Research on communicative language competence and the assessment of communicative language competence There has been an enormous amount of research done on communicative language competence with focus placed on the clarification of its definition and structure. Early pieces of research propose a narrow definition of language competence which only covers the knowledge of a language that a person has. Together with that narrow definition of language competence, the term “language performance” is also introduced. Language performance is used to refer to learners’ ability to produce the language with the application of their knowledge of the language [49]. More recently, the definition was expanded by several linguists and reseachers to a broader notion of “communicative language competence” which means the ability to communicate in various contexts using the language, not only using elements of the language such as grammatical knowledge but also gestures, intonation, strategies and other non-verbal means of communication” [45, 46, 50, 68]. Two major approaches to the assessment of learner communicative language competence are: (1) testing; and (2) performance-based assessment tasks [44, 82]. 8 1.2. Research on authentic assessment in language instruction 1.2.1. Definition and characteristics of authentic assessment Researchers have come up with various definitions of authentic assessment. Based on their findings, authentic assessment can be summarized and defined as an approach to assessment in which learners are asked to do authentic tasks that demonstrate meaningful application of essential knowledge and skills in real-world contexts [57, 58, 65, 70, 72, 79, 82]. From the findings of various studies [10, 11, 44, 61, 70, 71, 82, 86], main characteristics of authentic assessment can be summarized as follows: - Resembling real-world tasks and activities; - Based on problem-solving activities; - Formative, supportive of learning; - Emphasis placed on interaction and collaboration; - Performance-based activities; - Requirements and scoring criteria made clear to students before learning begins; - Measuring both learning products and process; - Can be used together with or to replace standardized tests. Authenticity Authenticity is an important characteristic that distinguishes authentic assessment from other forms of assessment. Based on the studies of Bachman and Palmer, Cumming and Maxwell, Ozkan and Palm [42, 51, 85, 86], an assessment is regarded as authentic if: - students are engaged in cognitive processes that are important in successful behaviors in life beyond school; 9 - language input used in the assessment is authentic, i.e. materials which are produced for real life, not changed or simplified for instructional purposes; - the context in which the assessment takes place should be real-life situations or similar to real-life situations. 1.2.2. Goals and objectives of authentic assessment Authentic assessment is designed to serve three main objectives: - to measure learners’ competence; - to provide a statement of learners’ competence or language ability (e.g. certification) so that stake-holders can make appropriate decisions; - to promote learning (e.g. provide feedback on the teaching and learning process/activities so that teachers and students can make adjustments suitable for their own teaching and learning practices) [71]. 1.2.3. Principles in using authentic assessment During the process of using authentic assessment, it is important for teachers to take account of the following factors: (1) validity; (2) reliability; and (3) authenticity [82]. 1.2.3.1. Validity Validity is to do with whether teachers can assess what they want to assess. More specifically, an assessment task is considered valid if it can measure exactly what it is designed to measure [67]. 1.2.3.2. Reliability Reliability refers to how consistently a method measures a learner’s language proficiency. If the same result can be consistently achieved by using the same methods under the same circumstances, the result is considered reliable [44, 67]. 1.2.3.3. Authenticity Apart from validity and reliability, in order for an authentic assessment task/activity to be distinctive from other assessement activities, it is necessary for 10 the assessment to have authenticity. More specifically, in carrying out the task, students need to be engaged in cognitive processes that are important in successful behaviors in life beyond school, the language input used in the assessment need to be authentic (i.e. materials which are produced for real life, not changed or simplified for instructional purposes), the context in which the assessment takes place should be real-life situations or similar to real-life situations. 1.2.4. Some models of using authentic assessment in language teaching, learning and assessment According to various studies [40, 71, 72], within the framework of this thesis, it is suggested that teachers embed authentic assessment in the learning process using the following pentagon model: With focus placed on assessment, authentic assessment can be employed using the Quad model introduced by Anthony et al.[71]: 1. Observation of process Students immersed in - speaking - listening - reading - writing 2. Observation of product - Audio tapes - Selected pages from notebooks or journals - Reading logs - Writing folders - Group-work logs - Projects Interactive/collaborative activities Lectures Tests Authentic assessment Assignments 11 - Learning logs - Homework 3. Classroom measures - Text-related activities - Teacher-made tests - Comprehension questions 4. Decontextualized measures - Criterion-reference tests - District exams - Provincial or state exams 1.2.5. Procedure, approaches and types of authentic assessment With reference to the findings of various studies [44, 71, 82], it can be summarized that the procedure suggested for the employment of authentic assessment includes three main stages: (1) designing; (2) implementing; and (3) evaluating. It is also learned from various pieces of research that there are two major approaches to authentic assessment: (1) observation of assessment products (portfolios, reports, learning journals, research pieces, etc.); and (2) observation of performance (e.g.: interviews, conversations, roleplays, presentations, simulations, etc.) [71, 82]. 1.3. Research on communicative language competence 1.3.1. Definitions of communicative language competence Different researchers come up with different definitions of communicative language competence. The author of this thesis uses the definition introduced by the Canadian Language Benchmarks: English as a Second Language for Adults, adapted from Bachman (1990), Bachman and Palmer (1996, 2010) and Celce- Murcia, Dornyei and Thurrel (1993, 1995): communicative language ability refers to “the ability to communicate: to interact, to express, to interpret and to negiotiate meaning, and to create discourse in a variety of social contexts and situations” [45]. 1.3.2. Structure of communicative language competence With regard to the structure of communicative language competence, in this thesis the author uses the model provided by Celce-Murcia, Dornyei and Thurrell 12 (1993) in which communicative language competence consists of 5 components: (1) grammatical competence; (2) sociolinguistic competence; (3) discourse competence; (4) strategic competence; and (5) action competence [47]. The five components are structured as in the following picture: Picture 1.4.: The structure of communicative language competence All the five mentioned-above components are closely interrelated and integrated into communicative language competence, so in order to communicate effectively, the language user needs to employ all the five components at the same time [47]. In this thesis, the author agrees with the findings and conclusions of various researchers that a person’s communicative language competence cannot be measured directly and is exposed through their performance and communicative activities in the form of listening, speaking, reading and writing in various communicative contexts [45, 46, 47, 50, 82]. 1.3.3. Communicative language competence framework 1.3.3.1. Communicative language competence framework In this thesis, the Vietnamese Framework of Reference for Languages (abbreviated NLNNVN) with 6 levels of language proficiency is used [8]. The Discourse competence Sociolinguistic competence Grammatical competence Action competence Strategic competence 13 expected outcomes for English-majored undergraduates is level 5 (corresponding to level C1 of the CEFR). Table 1.1.: Communicative language competence framework NLNNVN Framework CEFR Framework Advanced Level 6 C2 (Proficiency/mastery) Level 5 C1 (Effective Operational Proficiency) Intermediate Level 4 B2 (Vantage) Level 3 B1 (Threshold) Elementary Level 2 A2 (Waystage) Level 1 A1(Breakthrough) 1.3.3.2. Descriptors of communicative language competence framework In the development of language frameworks, authors often place great emphasis on the construction of can-do descriptors that describe and illustrate what learners can do at each level of proficiency [8, 50]. 1.3.4. Learning outcomes and proficiency level set for English-majored undergraduates Learning outcomes describe what students are able to demonstrate in terms of knowledge, skills, and values upon completion of a course, a span of several courses, or a training program. The information of learning outcomes of a course at universities is often made public and informed to those concerned [45, 50]. In Vietnam, the expected outcome level of proficiency set for English- majored undergraduates is level 5 of the NLNNVN, corresponding to level C1 of the CEFR [8, 50]. 1.4. Research on the employment of authentic assessment in the assessment of English-majored undergraduates’ English proficiency 1.4.1. Important issues in the employment of authentic assessment in the assessment of English-majored undergraduates’ English proficiency Aims: Authentic assessment is designed to: 14 - be used as an alternative or supplementary method to help measure learners’ competence more comprehensively and to increase validity and reliability of assessment activities; - provide a statement of learners’ competence or language ability (e.g. certification) so that stake-holders can make relevant decisions; - promote learning (e.g. provide opportunities for collaborative learning and feedback on the teaching and learning process/activities so that teachers and students can make adjustments appropriate for their own teaching and learning practices) [71]. Approaches: The author of this thesis agrees with the findings of various studies that authentic assessment of English-majored undergraduates’ English proficiency is based on the performance of students through the activities of listening, reading, speaking and writing in real-life communicative contexts. With that philosophy, authentic assessment should be performance-based and within the framework of this thesis, though being authentic in some situations, testing is distinctive from the authentic assessment discussed in this thesis. Thus, the common forms of authentic assessment covered in this study include interviews, conversations, observations, information gap, assignments, learning projects, roleplays, simulations, presentations, portfolios, etc. [44, 82]. 1.4.2. Components of communicative language competence that need to be assessed As described in the previous sections, communicative language competence is structured into five components: (1) grammatical competence; (2) sociolinguistic competence; (3) discourse competence; (4) strategic competence; and (5) action competence [47]. These are the components that need to be trained, developed and assessed during the learning process of English-majored undergraduates [42]. Again, it is neccessary to emphasize that the five components are exposed through communicative acts conducted by students. Anchored in this reasoning, it 15 can be concluded that authentic assessment in actuality is the assessment of English-majored undergraduates’ English performance through communicative acts in real-life contexts [44, 45, 50, 82]. 1.4.3. Authentic assessment of English proficiency In order to carry out authentic assessment effectively and systematically, with reference to the findings of O’Malley and Pierce, it is proposed that teachers do the following activities: (1) identifying purposes; (2) planning for assessment; (3) setting standards; (4) developing scoring criteria and rubrics; (5) selecting assessment activities; (6) implementing assessment and collecting information; and (7) analyzing information and using assessment results [82]. 1.5. Factors that influence the employment of authentic assessment in the assessment of English-majored undergraduates’ English proficiency Based on the findings of various studies, it can be concluded that there are several major factors that influence the employment of authentic assessment in the assessment of English-majored undergraduates’ English proficiency. They include the curriculum and syllabus of the English course, outcome level of proficiency set for students, characteristics of students; the demand of the society and labour market. 1.6. Conclusions Studies on assessment in general and authentic assessment in particular deal with various issues such as definitions, purposes, roles, methods, tools and procedures. With regard to the introduction and employment of authentic assessment, a great amount of research focuses on the analysis of the weaknesses of testing and the need for alternative or supplementary methods. It can be summarized that the body of literature on assessment and authentic assessment has become more and more comprehensive and multi-dimensional. However, little research has been done on the employment of authentic assessment 16 in the measurement of English-majored undergraduates’ English proficiency. This makes room for such a piece of research to be conducted with a hope to provide valid, reliable and comprehensive measures to assess English-majored undergraduates’ English proficiency at universities in Vietnam. CHAPTER 2: CURRENT PRACTICES OF INTEGRATING AUTHENTIC ASSESSMENT INTO THE ASSESSMENT OF ENGLISH- MAJORED UNDERGRADUATES’ ENGLISH PROFICIENCY 2.1. Aims of the survey The survey aims to investigate the current practices of the assessment of English-majored undergraduates’ English proficiency in general and the employment of authentic assessment in the teaching, learning and assessment of their English proficiency in particular. The findings of the investigation can be used as the foundation for the introduction of measures with a hope to increase the quality of the assessment activities (with regard to validity, reliability and comprehensiveness). More specifically, the survey focuses on the investigation of the 4 main issues: (1) teachers’ and students’ knowledge of the two concepts of communicative language competence and authentic assessment; (2) the current practices of the assessment of English proficiency in general; (3) the current practices of authentic assessment in teaching, learning and measuring English-majored undergraduates’ English proficiency in particular; and (4) teachers and students’ opinion of the best method to assess English-majored undergraduates’ English proficiency. 2.2. Methods, population samples and data analysis 2.2.1. Methods The main methods of collecting data include: (1) Survey questionnaires; (2) interviews. 2.2.2. Population Two main groups of population: 17 - 518 English-majored undergraduate students are from 6 universities in Vietnam. - 91 teachers of English are from same 6 universities. Ways of analyzing data The app IBM SPSS 20 was used to analyze the data to answer the research questions. Also, based on the data collected and analyzed, the researcher also assessed, gave comments and made conclusions on the current practices. 2.3. Findings - Most of the teachers and students asked understood the two concepts of communicative language competence and authentic assessment. - English language curriculum consists of different components of assessment, which makes room for the implementation of various forms of assessment, including authentic assessment. - Teachers and students did employ authentic assessment in their teaching and learning. - The most common form of assessing students’ English proficiency is testing. Meanwhile most teachers and students surveyed believed that tests should be used together with other forms such as authentic assessment. - Teachers’ skills of designing authentic assessment tasks, building scoring criteria and rubrics fall short of expectations. They rarely design and use important assessment tools such as self-assessment and peer-assessment rubrics or checklists. - Teachers’ skills of implementing authentic assessment fall short of expectations. They rarely use important assessment forms such as projects, portfolios, self-assessment or peer assessment. - Teachers employ authentic assessment in their teaching and assessment, but only do this in a relatively informal way that does not provide systematic information about student learning or instructional goals. 18 - The results of authentic assessment activities do not reliably reflect both the students’ learning process and their learning achievements. - Teachers’ use of the results of authentic assessment are not up to the expectations of their students. - In general, teachers’ feedback on students’ performance is not timely provided. In order to help deal with the issues mentioned above, it is proposed in this thesis that a comprehensive precedure (including three stages: designing, implementing and evaluating) and 4 easy-to-use measures (each includes sample tasks, assessment criteria and rubrics) be used in the assessment of English-majored undergraduates’ English proficiency with focus placed on the four communicative skills of listening, speaking, reading and writing. CHAPTER 3: PROCEDURE AND MEASURES FOR INTEGRATING AUTHENTIC ASSESSMENT INTO THE ASSESSMENT OF ENGLISH- MAJORED UNDERGRADUATES’ ENGLISH PROFICIENCY AT UNIVERSITIES IN VIETNAM 3.1. Principles The procedure and measures were proposed based on the following factors: (1) goals and objectives of the curriculum and syllabus; (2) accountability; (3) feasibility; and (4) authenticity. 3.2. Procedure and measures It is proposed that the following procedure and measures be used. 3.2.1. Procedure for the employment of authentic assessment The proposed procedure consists of three main stages: Stage 1: Designing T and Ss work to design authentic assessment • Setting goals and objectives • Making a plan for assessment tasks • Setting standards, building criteria, rubrics, etc. 19 tasks • Making a plan for student involvement • Selecting assessment activities • Identifying methods of collecting information Stage 2: Implementing T organizes assessment activities and Ss do the activities T’s activity Ss’ activities • Informing and introducing the assessment tasks to Ss • Organizing the assessment activities for Ss to take part in • Observing and monitoring Ss’ performance • Giving feedback or comments when necessary • Receiving information about the assessment tasks • Working and discussing with T to clarify the tasks if necessary • Working individually, in pairs or groups to do the tasks • Doing self or peer assessment Stage 3: Evaluating T and Ss analyze, evaluate and use the information gathered T’s activity Ss’ activities • Collecting assessment evidence and materials (rubrics, learning products, etc.) • Analyzing and evaluating Ss’ assessment evidence (their performance, their learning process and products) • Scoring, giving feedback or comments on Ss’ performance and their learning process and learning products • Re