Luận văn Thiết kế và đánh giá một bài kiểm tra tiếng anh chuyên ngành cho sinh viên xây dựng dân dụng tại trường đại học dân lập Hải Phòng

, but primarily to the ways in which the test will be implemented in a given situation, and to whether the test will be developed and used at all. Practicality often affects a tester’s decisions during the development of a test, i.e., at every stage of his testing. Practicality can be defined as ‘the relationship between the resources that will be required in the design, development, and use of the test and the resources that will be available for these activities’. (Bachman & Palmer, 1996: 35). This relationship can be represented as in the figure below: Available resources Practicality= Required resources When practicality ³ 1, the test development and use is practical When practicality< 1, the test development and use is not practical. In a nutshell, when designing a test the tester should always bare in mind this quality- practicality-to ensure that the test is as economical as possible, both in time (preparation, sitting and marking) and in cost (materials and hidden costs of time spent). In other words, a practical test is the one which can minimize the use of the available resources, i.e., the required resources must not be more than the available resources. 1.4.4. Discrimination Finally, a discussion of the basic concepts behind testing would be incomplete without the treatment of the closely related idea of discrimination. According to Harrison (1994:14) discrimination is ‘the extent to which a test separates the students from each other.’ However, the extent of discrimination varies according to each kind of test. For instance, an achievement test should result in a wide range of scores because it is easier to make decisions about where to separate one group of students from another so that they can be awarded different grades. A diagnostic test, however, may be intended to show that nearly all students have learnt the material tested, and in this case they should all get fairly high scores. 1.5. Achievement tests Different researchers have different points of view of an achievement test. According to Harrison (1983: 65) ‘designing and setting an achievement test is a bigger and more formal operation than the equivalent work for a diagnostic test, because the student's result is treated as a qualification which has a particular value in relation to the results of other students. An achievement test involves more detailed preparation and covers a wide range of material, of which only the sample can be assessed.’ Heaton (1988) defines achievement tests What are “them” as the ones that are “based on what the students presumed to have learnt, not necessarily on what they have actually learnt nor on what has actually been taught.” In Brown’s point of view “an achievement test is related directly to classroom lesson, units or even a total curriculum within a particular time frame.” (Brown, 1994: 259). In other words, an achievement test measures a student’s mastery of what should have been taught. It is thus concerned with covering a sample (or selection), which accurately represents the contents of a syllabus or a course book. Unlike progress test, achievement test should attempt to cover as much of the syllabus as possible. If we confine our test to only part of the syllabus, the contents of the test will not reflect all that the student has learnt. Achievement test can be subdivided into class progress tests and final achievement tests. 1.5.1. The class progress test The class progress test is often conducted during the course and is developed by the teacher himself after each chapter or each term. He constructs such type of test to judge how successful his teaching is and also to find out what his students have achieved from his teaching. The class progress test is a teaching device and can be considered a good chance for the students to prepare for the final achievement test. 1.5.2. The final achievement test The final achievement test is more formal and intended to measure achievement on a larger scale (annual exams, entrance exams, final exams). The final achievement test is not written and administered by the teacher himself, but maybe by ministries of education, boards of examiners, or by members of teaching institutions. A final achievement test is often based on an adopted syllabus and its approach, either syllabus-content approach or syllabus-objective approach. If the test is based on the former, its contents should be based directly on a course syllabus or on the textbooks and other materials chosen. If it is based on the latter, its contents are based directly on the objectives of the course. Summary In this chapter I have briefly dealt with the concept of a language test, how it is defined and what is important in designing it. Moreover, I also mentioned the concept of communicative language ability in which communicative competence was also discussed. Also, in this chapter the definition of an achievement test as well as testing reading skills were presented because they play an important role in the process of doing this research. Chapter two: Methodology This chapter will include a brief introduction of a quantitative study, the selection of participants who took part in doing the test, and the materials from which the test items were taken. The methods of data collection and data analysis are presented afterwards. Finally come the limitations of the research. 2.1.A quantitative study Like qualitative research, quantitative research comes in many approaches including descriptive, correlational, exploratory, quasi-experimental, and true-experimental techniques. As a teacher of Civil Engineering English, I designed this reading test to understand better how things are really operating in my own classroom as well as to describe the performance of my learners in the reading skill. After 120 period reading course 50 How many? Be specific! students were chosen from three different classes (XD501, XD 502, XD 503) to do a reading test in the time given (60 minutes) and then the results collected from the testing papers would Em oi, phai consistent ve thi chu. Trong research report, khi ke ve nhung thu em da lam thi tat ca deu o thi qua khu hoac tuong lai trong qua khu. be described in different terms with the use of the descriptive statistics technique. The correlational research technique was also used to find out the reliability coefficient latter in the study. 2.2.The selection of Participants The students at Haiphong Private University mainly come from different towns and cities in the North of Vietnam. They are generally aged between 18 and 22, or older. At the university, they study for eight terms in four years. There students are classified into majors and non-majors of English. The latter usually have to learn a foreign language, in this case English, in only two years of their whole student lifeIn the first three terms, they study General English and in the fourth term English for Specific Purposes (ESP). After two years’ English learning, they are required to be able to read and translate their ESP at intermediate level. However, students often have varying English levels prior to the course due to the fact that at secondary school they learned different languages, including Russian, French, and Chinese. It is therefore important for teachers to apply appropriate methods in teaching them GE as well as ESP to help them become more proficient. It is also critical that teachers give them suitable tests which meet their need and Em nen de y den cau truc song song trong khi viet nhe. the requirements of society at the same time. 2.3.The Materials During the first three terms the CE students are required to learn all the 15 units in Elementary Headway, and the first 8 units in Pre-intermediate Headway. These three terms include 205 periods in all, 75 periods for each term. In the fourth term, they study 120 periods of ESP usig a the 15-unit textbook on English for Civil Engineering. 2.4.Methods of data collection and data analysis To collect data for the research, a 34-item test of Civil Engineering English reading was delivered to 50 students of the Construction Department. These non-majors did the test within the time frame given (60 minutes). Then the test papers were collected, and then were marked, analysed, and interpreted. Doing these things did point out how many students did the test well, how many performed badly, the most frequent scores the testees got, how these scores ranged, how many scores deviated from the mean, etc. 2.5.Limitations of the research Like in any other studies, some limitations cannot be avoided in this one. Firstly, because of the limitation of time as well as of ability, the author could design only one reading test to be conducted on 50 students, which might not be a large number. Yet, it is hoped that the results could be reliable and valid enough for the researcher to make inferences and come to certain conclusions. Secondly, instead of designing different types of test, the author was able to make solely one type, that is an achievement test to measure the progress her students had made in terms of reading skills after undertaking the course of English for Civil Engineering in their last term in 2004-2005 school year. From the results, the author could also measure the effectiveness of her teaching. Summary This chapter gives a brief account on a quantitative study, in which the author used the descriptive statistics and correlational technique to analyse the data. Following the methods, the selection of participants and materials has also been dealt with. A quick introduction of the data collection and data analysis methods was also presented and finally came the limitations of the research. Chapter three: Discussion This chapter is the discussions firstly about the content area of the test, how the test was divided, how to construct and mark the test. Afterwards, the whole test results and each test item would be analysed and then interpreted. Finally, the author will evaluate the test based on the four criteria of a good test as mentioned in the previous chapter. 3.1. The content area of the test The following topic checklist of the course book will help to point out the content area of the reading test. Help seize co nghia la gi va giup ai? The Topic checklist of the course book Topic Material Number of unit/ page Architectural composition Skeleton construction Concrete, reinforced concrete, prestressed concrete Ultimate carrying capacity and factor of safety Pre-cast products Breakwaters Conveying, placing, compacting, and curing Concrete and strength test Asphalt concrete Materials and properties Structure Location Actions and sequences Arch and arch beam bridges Shear forces and bending moments in beams Matrix methods in the calculation of structure The hinge English of Civil Engineering Unit 1-p.1 Unit 2-p.6 Unit 3-p.10 Unit 4-p.14 Unit 5-p.20 Unit 6-p.24 Unit7-p.1 (Book 2) Unit 8-p.8 Unit 9-p.12 Unit 10-p.20 Unit 11-p.26 Unit 12-p.29 Unit 13-p.32 Unit 14-p.35 Unit 15-p.35 Unit 16-p.42 Unit 17-p.46 3.2.The relative weights of the different parts of the test The test is composed of 5 parts, and the weighting of each part is illustrated in the following table: Test of reading Part Input Response/ Item type Scores Weighting 1 Factual text, approx.120 words 5 comprehension questions 10 20% 2 5 word columns Matching to make 5 sentences 10 20% 3 10 jumbled sentences Rearranging 10 20% 4 10 statements True / False 10 20% 5 Factual text, approx. 120 words with 5 blanks Blank filling 10 20% 3.3.Constructing the test To construct the reading test used in this research, the author went through the following procedures: Statement of the problem There was a need for this achievement test to be administered at the end of the course of training in the reading of Civil Engineering English (the students are graduates). The test was intended to find out what progress was being made after 120 period study and also what were the greatest difficulties in learning that the students still had at the end of the course. Thanks to that future courses may give more attention in these areas. Backwash is considered important; the test should encourage the practice of the reading skills that the students need in their university study. The time allowed was one hour. Specifications CONTENT Types of text: The academic texts were from the course book entitled ‘English for Civil Engineering’. One sample text is provided in Appendix 1. Addressees: Non-native speaker university students at HPU, or more specifically non-majors of CE at HPU. Topics: The topics were suitable for the candidates and the type of test, and the subject area were neutral. Operation: The test has 5 tasks and the candidates had to scan to locate specific information, to match words/ phrases to make correct statements, to arrange words/ phrases to make complete sentences, to decide whether the given statements are true or false, and finally to fill blanks with the given words. FORMAT AND TIMING Scanning: 1 passage with about 120 words in length. 5 short answer items, the items in the order in which relevant information appears in the texts. Responses were controlled. Time: 10 minutes. Detailed reading -5 columns of words. Responses were controlled. Time: 10 minutes. -10 jumbled sentences to be rearranged. Responses were controlled. Time: 20 minutes. -10 statements to be marked T or F. Responses were controlled. Time: 10 minutes. -1 passage with about 120 words in length. 5 gaps to be filled. Responses were expected. Time: 10 minutes. CRITICAL LEVELS OF PERFORMANCE All test items were written such that any student completing the course successfully would be able to respond correctly to all of them. Allowing for ‘performance errors’ on the part of candidates, a critical level of 80 percent was set. The students reaching this level would be the ones succeeding in terms of the course’s objectives. SCORING PROCEDURES There was a detailed key and the scoring was completely objective. SAMPLING The texts were chosen from a variety of topics in the course book. Draft items were written before the test was officially used. ITEM WRITING AND MODERATION All the items in the test were based on a consideration of what a competent non-major would be able to obtain from the texts. Considerable time was set side for moderation and rewriting of items. KEY There was a detailed key for the test results. The key is provided in Appendix 2 After having followed the above procedures, the test was designed as follow: Haiphong public university Achievement test Testee's full name:..................................................... Skill: Reading Mark: Level: Intermediate Time allowed: 60 minutes Question 1: Read the following passage then answer the questions given below Conveying devices may be wheelbarrow, bottom dump bucket, dump truck. If necessary concrete may be pumped through hoses and steel pipelines. The mode of transport depends on the quality of concrete to be placed, the equipment available and other factors. The method employed must prevent the separation of the materials, called segregation, and insure that concrete of good quality is deposited in the form. The forms are made of timber or metal of a size and shape suitable for the finished work. They must be of sufficient strength and rigidity to support the wet material and allow it to be properly compacted. They are so constructed that they may be easily removed when the concrete has hardened. The interior of the forms must be oiled or soaped to prevent the concrete from adhering to the forms. 1-What are the conveying devices mentioned in the passage? ................................................................................................................................... 2-What does the mode of transport depend on? ................................................................................................................................... 3-What are the forms made of? ................................................................................................................................... 4-Why must the forms be of sufficient strength and rigidity? ................................................................................................................................... 5-What must we do with the interior of the forms before placing? ................................................................................................................................... Question 2: Use the words/ phrases given below to make sentences describing the properties of materials. Steel Stone Glass wool Brick has the property of high tensile strength good sound isolation good thermal isolation high compressive strength This means it can resist high compressive forces it can resist high tensile forces it does not transmit heat easily it does not transmit sound easily 6-............................................................................................................................... ............................................................................................................................... 7-............................................................................................................................... ............................................................................................................................... 8-............................................................................................................................... ............................................................................................................................... 9-............................................................................................................................... ............................................................................................................................... Question 3: Arrange the following words/ phrases to make complete sentences 10-fire/ weather/ has/ high/ concrete/ resistance/ and. ............................................................................................................................... 11-lost cost/ durable/ is/ and/ concrete/ pre-cast/ at. ............................................................................................................................... 12-made/ different/ materials/ from/ concrete/ is. ............................................................................................................................... 13-solid/ reinforcement/ is/ widely/ in/ spaced/ slabs. ............................................................................................................................... 14-aggregate/ 20mm/ 40mm/ coarse/ in/ ranges/ to/ size/ from. ............................................................................................................................... 15-during/ segregate/ conveying/ may/ concrete. ............................................................................................................................... 16-be/ spread/ shall/ mixture/ by/ asphalt/ paver. ............................................................................................................................... 17-elastic/ clay/ rubber/ is/ plastic/ but/ is. ............................................................................................................................... 18-a/ done/ is/ concrete/ mixer/ mixing/ in. ............................................................................................................................... 19-vibrators/ driven/ be/ by/ electricity/ air/ or/ compressed/ may. ............................................................................................................................... Question 4: Use your knowledge of the subject to decide whether the following statements are true or false. (Write T or F) 20-Glass wool is a heavy material. 21-Rubber cannot be stretched or compressed. 22-Concrete is a light material so it is easy to lift. 23-We can burn wood because it is combustible. 24-Mild steel can resist corrosion. 25-Rubber is plastic while clay is elastic. 26-Because copper is a good conductor of heat so heat can be easily transferred through it. 27-We can easily scratch glass because it is soft. 28-Concrete cannot be bunt because it is non-combustible. 29-Stainless steel is corrosion-resistant. Question 5: Fill each blank below with ONE of the given words. multi-story minimum timber architecture maximum possible impossible low steel The modern skeleton structure is the result of rational use of steel and concrete in building. Among its characteristic features are the reduction of all load-carrying members to (30).................sizes and clear division between structural and non-structural elements. The skeleton is composed of rigidly connected beams and columns. It is a particular suitable form for (31) ................. buildings. The great strength of modern building materials makes it (32) ................ to build higher and higher, to meet today’s ever increasing demands. The pattern of our large cities is being determined by skeleton structures of steel and concrete just as decisively as the pattern of medieval cities was determined by the (33) .................frame. Widespread use has made the modern skeleton structure a central theme of contemporary (34) ................. 3.4.Administering the test In order to accomplish the two purposes of test administration for this reading test-collecting feedback to assess usefulness of the reading course and making inferences about test takers’ language ability-it is necessary to have some control over the procedures for administering it. These involve guiding the test takers through the following process of taking the test: Preparing the testing environment The first step in the test administration was preparing the testing environment to be consistent with the specifications in the test blueprint. This involved arranging the place of testing (rooms C101 and C102), the materials (50 test papers) and equipment (fans, tables and desks for the students, chairs for the examiners, lights), personnel (2 examiners), time of testing (60 minutes), and physical conditions under which the test is administered. The weather at that time was quite good for the students to do the test. Communicating the instructions ‘The second step in administering the test was to give the instructions in such a way that they would be understood by all the test takers. When administering the test it is essential that the test takers receive the full benefit of the instructions’ (Bachman and Palmer, 1981: 233). This included the obvious steps of providing suitable conditions (time, lighting, lack of distraction) for reading written instructions with the help of the two examiners. Maintaining a supportive environment. The next step is maintaining a supportive testing environment throughout the test. This includes avoiding distractions due to temperature, noise excessive movement, etc. Collecting the tests. The final step in the test administration was collecting the tests.The testing papers were collected by the examiners after the allowed time in each testing room was over. When they were being collected, the test takers left at their own peace. 3.5. Marking the test The testing papers were marked according to the band scores on the 0 – 10 scale as officially approved by the HPU board of examiners after they were collected. 3.6. Test scores interpreting and evaluation The results of language tests are most often reported as numbers of scores, and it is these scores, ultimately, that test users will make use of. The test scores of the 50 student participants were interpreted and analysed. This very analysis will simply provide a summary of how the students did the test, and to check on the test’s reliability and to have some idea of how dependable the test scores were. The following steps will provide the reader with an outline of how such analysis can be conducted. 3.6.1.The frequency distribution: Frequency distribution Em nen format lai bang di, vi bang thua nhieu cho trong qua, va cung hoi kho hieu. Minh khong hieu truc nao dung de ta cai gi, va em list duoc diem cua bao nhieu em. isa record of testees’ scores ranging from the lowest to the highest marks in a test. Raw marks are marks awarded by counting the number of correct answers on a test. The frequency distribution of the reading test that the author conducted is presented by the diagram below: (It is essential to remembe