Reflection of Unit 3

In this unit we learned how to create tests, the different types of testing and tests items and at the end of this unit we created a test of our own. It is for us one of the most useful and important unit of this subject because we feel all of it revolves around doing this type of instruments.

3.5 To design a language test

a) LEARNING OBJECTIVES AND CONTENTS

This test was designed to assess the two main skills of the English language which are widely used for academic purpose: reading and writing. After taking this test, students are going to be able to know their doing in both skills, each one with two different sub-skills.  For reading, comprehension of a text and scanning for specific information will be assessed. In the other hand, the writing section will assess the spelling related to the vocabulary of the previous classes and the use of linking devices within paragraphs.

The test will use a short text about Animals in the City as the main topic. The content will be adapted for each skill and for the student's level (intermediate).

The test itself will be short, with four items; two for writing and two for reading, making it one item for each sub-skill.

b) SPECIFICATIONS:

FIRST ITEM

Reading (10 minutes)

Task: Students read a text and answer the following questions.

Format: Text about “Animals in the city” and multiple choice questions about the main ideas of the text.

SECOND ITEM

Reading (10 minutes)

Task: Students read a text and answer the following questions.

Format: Text about “Animals in the city” and multiple choice questions about specific information which can be found in the text.

THIRD ITEM

Writing (8-10 minutes)

Task: Student completes a paragraph by using words from the vocabulary.

Format: Paragraph with fill-in-the-gaps exercise, where students write the correct word, taking care of the spelling.

FOURTH ITEM

Writing (8-10 minutes)

Task: Student completes a paragraph by using linking devices.

Format: Paragraph with fill-in-the-gaps exercise, where students write the correct linking device, selecting it from words in a box.

c) TEST:

Instructions for the test: Read carefully each item on this test and follow the instructions mentioned there.

READING SECTION

(I) Instructions: Read the text and answer by choosing the correct answer.

Animals in the City

One night in December 2011, a bear came into the city of Vancouver in Canada. It walked through the city streets past houses, shops and offices. Then it found some food in bins outside a restaurant and started eating. In the morning, someone saw the bear and called the police. The police came with a vet from the city zoo. They put the bear in a lorry and took it to the mountains outside the city. Luckily, the bear was safe. But what happens in other countries when big animals come into cities? In Vancouver it is unusual to see a bear, but in some cities you can see big animals on the city streets every day.

Big animals usually come into cities to find food. In Cape Town in South Africa baboons come into the city when they are hungry. They go into gardens and eat fruit from trees. They even go into houses and take food from cupboards and fridges! Baboons are strong animals and they can scare people. But the city can be dangerous for baboons too. Sometimes, cars and buses kill baboons in accidents. Human food is very bad for the baboons’ teeth because it has a lot of sugar. Now, there are Baboon Monitors working in Cape Town. Their job is to find baboons in the city and return them to the countryside.

In Berlin in Germany, pigs sometimes come into the city to look for food. They eat flowers and plants in parks and gardens. Sometimes they eat vegetables from gardens and they walk in the street and cause accidents. Some people like the pigs and they give them food and water to drink. Other people do not like the pigs and they want the government and the police to stop them entering the city.

(Robin Newton, britishcouncil.org)

1-What is the text's main idea?

a) Differences between wildlife in cities around the world.

b) People is angry about animals wandering in the streets and causing accidents.

c) The way some cities around the world react to uncommon animals in the urban environment.

2- How is it different the way the government is solving the issue about these animals in Vancouver and in Cape Town?

a) Cape Town is having trouble with more intrusive dangerous animals.

b) Cape Town created a special group of people who patrols the city looking for the animals while Vancouver uses the police.

c) People in Vancouver take the animals back to the forest and Cape Town uses the Baboon Monitors.

3-Choose the option which describes the best the situation in Berlin:

a) Some people like the pigs and feed them, while some of them think pigs can cause accidents in the streets. The government is not working properly on the issue and people get angry about it.

b) Some people like the pigs and feed them, while some of them think pigs can cause accidents in the streets. The government is not taking any action about it because some want the pigs to stay.

c) Some people like the pigs and feed them, while some of them think pigs can cause accidents in the streets. The government wants to solve the issue, but the police is inefficient.

4-Why the Vancouver Police Department called a vet from the zoo?

a) To calm the bear and take the animal to the lorry.

b) To do an examination of the animal and make sure he was safe.

c) To take the bear to the zoo before returning it to the forest.

(II) Instructions:  Answer true or false to the next statements according to what you read in the text "Animals in the City" by underlining the correct option.

1- The bear that was captured in Vancouver was injured.

TRUE FALSE

2-Babbon monitors return the baboons to the countryside

TRUE FALSE

3-Sometimes, baboons take food from the fridges.

TRUE FALSE

4-Some people in Berlin give water to the pigs.

TRUE FALSE

WRITING SECTION

(III) Instructions: Complete the next sentences by writing the correct word in the blank. A hint is being given at the end of the sentence.

1- I was feeding the ________ when I heard my Mom yelling at me. (Hint: plural of goose)

2-  The ______ was migrating to the South in Winter. (Hint: a large number of birds which fly together.)

3-I bought two _______ to feed my snake. (Hint: plural of mouse)

4-The __________ animals will face extinction if we do not protect them. (Hint: Adjective ending as a verb in past simple, used for species which are near to extinction)

5-You cannot walk a dog without a ________. It is a law. (Hint: Wit this device you can make sure that the dog is not going to escape from you when walking it)

6-Kangaroos are _______ to Australia. (Hint: Adjective used for animals and plants peculiar to certain area or region in the world)

(IV) Instructions:  Complete the paragraph by using the linking devices in the box below.  You can use them as many times as needed. However, some of the linking devices in the box are not going to be used in this exercise.

However/In the other hand/First/Despite/but/Furthermore/ While/Although/Also

 

 Dinosaurs were amazing creatures. (1)_________ not being able to study them alive nowadays, we can tell that they by just looking at their fossils.(2) _________ many people are happy with only analyzing their bones, there are some scientists who actually want to bring dinosaurs to life. That would be awesome!

(3)__________, it would be a very difficult task to perform. (4)__________, we need to find a piece of a complete genetic code. Then, find a way to bring a baby dinosaur to an egg. I think I am going to be waiting to see one of those animals for a long, long time.(5) ___________ waiting for it, I am going to continue reading about them.(6) __________, I want to go to a museum and see more fossils.

d) ANSWER KEY

(I)

1) c

2) b

3) a

4) a

(II)

1) FALSE

2) TRUE

3) TRUE

4) TRUE

(III)

1) Geese

2) Flock

3) Mice

4) Endangered

5) Leash

6) Endemic

(IV)

1) Despite

2) While

3) However

4) First

5) While

6) Also

e) SCORING/RATING GUIDE

This test has two sections, one for reading and one for writing. Each section includes two assessment items. Also, each item has a number of questions (four or six) and it has its own scoring.  Each section is an individual assessment tool for reading or writing, and the test's final score includes two individual results; the one for reading and the one for writing. To complete the assessment, you must follow the next scoring guide:

READING SECTION (50% of the test's final percentage)

(I) (50% of the section's percentage)

1) 25% of the item.

2) 25% of the item.

3) 25% of the item.

4) 25% of the item.

(II) (50% of the section's percentage)

1) 25% of the item.

2) 25% of the item.

3) 25% of the item.

4) 25% of the item

WRITING SECTION (50% of the test's final percentage)

(III) (50% of the section's percentage)

1) 16% of the item

2) 16% of the item

3) 16% of the item

4) 16% of the item

5) 16% of the item

6) 16% of the item

(If all the answers are correctly spelled, add 4%)

(IV) (50% of the section's percentage)

1) 16% of the item

2) 16% of the item

3) 16% of the item

4) 16% of the item

5) 16% of the item

6) 16% of the item

(If all the answers are correct, add 4%)

 Rating Guide: In order to give students a complete assessment, use the next rating guide to describe the student's skills:

READING SECTION:

25%: Insufficient. Student needs to focus on the development of reading skills. Provide more reading material for the learner.

50%: Not enough. The student can grasp the main idea of the text, but it is probable that they find unknown and unfamiliar words. Improving the student's lexicon would help to solve the issue.

75%: Basic. The student with this score is able to find the main idea and supporting ideas from a text. A large lexicon is expected from this student. However, the learner has problems identifying the message in between words.

100%:  Good and ready to write. This student has developed good reading skills. The learner can identify main and supporting ideas, the true intentions from the author in between words, grasp the context of the text and infer from titles.

WRITING SECTION:

25%: Insufficient. The student cannot take part of a proper writing process. Spelling and cohesion need to be developed. Reading can be a tool for improving writing.

50%: Not enough. Student is able to communicate basic ideas through written methods. However, spelling and use of linking devices in between paragraphs need to be taken care of.

75%: Basic. The learner can elaborate a regular text, using linking devices and a wider lexicon. Despite this, the spelling needs to be improved.

100%: Good. This student can produce good quality texts, with coherence, good spelling and use of the linking devices to join ideas.

f) COMMENTS BY MODERATOR

Full Checklist for Moderators

	Yes	No
Is the English grammatically correct?	X
Is the English appropriate for the level of students?	X
Is the English in accordance with the specs?	X
Does the item (exercise) test what it is supposed to test?	X
The correct response cannot be obtained without the appropriate grammar knowledge?		X
Is the item economical?	X
Multiple choice-Is there only one correct answer? Gap filling-Are there just one or two correct responses?	X
Multiple choice-Are the distracters really going to distract?	X
Is the key complete and correct?	X
Are the instructions clear? Not too long?	X
Is there an example?		X

g) NARRATIVE OF THE PROCESS AND SELF-EVALUATION

This test was made thinking about the specific needs of the intermediate level of English. The assessment that can take place after taking this test will help only to identify the student's level on certain skills. It was not intended to be a final test with a lot of weight into the final grade. Very specific sub-skills were used here (Reading comprehension, scanning for specific information, spelling and use of linking devices) just to help teachers to focus on some aspects of the learning process for a certain sub-skill. In this way, after completing a unit, the teacher can use this test to know if the learning objectives were accomplished.

The multiple choice items and close answers are being used for practicality of the evaluation. The direct way in which the answers have only one correct answer make easier the teachers' work. The grading system is based on percentages that can be adapted to the teacher's needs. 

3.4 To design a test item for each skill

-Speaking (3 minutes) Fluency

Format: Individual and Interview
Task: Teacher asks the student about his/her favorite holiday and why.

Instructions: Teacher asks each student individually about their favorite holiday and he/she will have to explain why that holiday is their favorite above the others.

-Listening (5 minutes) Listening for specific information

Format: True or false
Task: Audio about the customs and traditions of a foreign country and using the information to answer some questions.

Instructions: The teacher will give to each student a piece of paper that has a chart with some questions, a TRUE column and a FALSE column, the student will mark however they want their answer. The teacher will play an audio about customs and traditions of a foreign country and the students will answer according to what they are listening to.

-Reading (8 minutes) Getting the gist

Format: Article about traditions in Europe
Task: The students will be put in pairs and after reading the article, they will do a matching activity linking the name of the tradition and its description.

Instructions: The teacher will ask the students to be in pairs and will give an article to each pair, the article has information about Europe and some traditions that each country has, the students will have to do a matching activity on another piece of paper given by the teacher that has the name of the country and a short information about the tradition.

-Writing (10 minutes) Grammar and spelling
Format: Write a letter for someone of your family
Task: Student is going to write a letter/postcard to someone of their family, describing a visit to a foreign country, giving information about the customs and traditions of its people.

Instructions: The teacher will give a blank piece of paper to each student and ask them to imagine that they are in a different country and that they are going to write a postcard to their family describing the country and the traditions or customs that they saw people doing in different places, next they are going to put it in an envelope and think that they are sending it back home.

3.3 To explain core concepts of testing

Validity: A test has to be valid, that is true. However, what makes a test valid? Validity in a test is how a test relates to the language course in real life, if it meets the requirements and if it really helps the teacher to understand students' knowledge and abilities. If a test is valid, is going to be related to the class, is going to be helpful and it is not going to be out of place. There are many types of validity, such the validity of content and the validity of criterion. The first one is related to the fact of analyzing if the content on a test is linked to what is going to be assessed and the latter is related to if the test really measures what is promising.

Reliability: If we can trust the results of a test, even under different environmental characteristics and different time and place, we can say that a test is reliable. Human beings are not able to produce the same results and scores in a test under this different characteristics, however, if the scores are closely the same, this test would be an almost perfect tool and the scores would represent the real proficiency of a student.

In a reliable test, errors in measurement are not expected and mistakes by the scorer are not even thinkable.

Practicality: A test should be helpful for a teacher, not detrimental. Under this idea, we can look in detail how sometimes a test can be a nightmare for a teacher if such test was not developed to be practical. By practical we mean easy to score, easy to modify, easy to apply and easy to understand. The items in a practical test should be objective, handy and must be easy to revise and grade.

Backwash: What we know as backwash in testing is the effect of a test in teaching and learning. For teachers, testing some skills may be a hard task and they test just the easiest skills, as reading and writing, mostly because there is a stronger evidence of learning under this methods. Unfortunately, this is the awful truth of many language centers, which do not test listening or speaking due to the hard work that this represents. However, backwash can be beneficial under certain circumstances, like paying more attention in the assessment of these skills which are known for being difficult to score. 

3.2 To analyze a language test

Test being analyzed: Test #2

1-What is the purpose of the test?  To know if the students can use the knowledge they acquired during their classes.

2-Does it represent direct or indirect testing? Direct Testing.

3-Are the items discrete point or interrogative? Most of the activities are discrete point, including just some interrogative ones.

4-Which items are objective and which are subjective? Most of them are totally objective, just including a writing exercise and a grammar one with different answers.

5-Is the test norm referenced or criterion based referenced? Criterion based

6-Does the test measure communicative abilities? Would you describe it as a communicative tests? It is focused on reading and writing communicatively, specially exercise VI.

7-What relationship is there between the answers to question 6 and the answers to the other questions? They use all the knowledge that they are supposed to have in exercises 1-5 in an objective way, then in exercise 6 the student uses the language subjectively. It includes direct testing and interrogative points, using subjective exercises. Thus, it is communicative.

3.1 Types of Testing

Direct and indirect testing: Direct testing provides information that is not going to change depending of the person answering it. Just one possible answer. Indirect is the opposite, being the answers different and unique.

Example: For direct is multiple choice and for indirect an open paragraph

Discrete point and integrative testing: Discrete test just one skill at the time, while integrative as its name says, blends them together.

Example: For discrete, fill the gaps. For integrative, listen to an audio and take notes.

Norm-referenced and criterion referenced testing: norm-referenced testing is used to make comparisons between students’ performance and the other one is used when we want to know if students are able to perform certain things.

Example: For norm reference we compare between the students who passed the test and those who doesn’t. For criterion, it is just a regular exam to say if they have the knowledge or not.

Objective and subjective testing: The scorer acts objectively or subjectively when revising.

Example: True or false statements are objective, while an open paragraph is subjective.

Computer adaptive testing: Used to collect data from scores.

Example: Use a computer to create a digital exam

Communicative language testing: used to evaluate the performing of communicative skills.
Example: The students listen to an audio and then explains orally the content of it.

3.1 Types of tests

There are different types of tests. The most important ones are:

Proficiency tests: This type of test was made to test people's abilities in a language and show their skills and performance of the language, no matter if they have a previous training in the subject.

A good example is the TOEFL test, which shows the proficiency in the English language. The equivalent of this type of tests in Japanese is the JLPT, made by the Japanese government.

Achievement tests: Are tests made by authorities to get feedback about students' performance and what they have achieved. There are two subcategories under this type of tests: final achievement and progress achievement. The first one is applied at the end of the course in order to know if the objectives were achieved, and the second one is applied in the middle of the course.

"Prueba ENLACE" is clear example of this type of tests. With that test, the Mexican government can get results from students all over the country, compare them and change the educational strategies in order to get better results next time.

Diagnosis tests: This test does not have an impact on the students' grades and it is made just to know the abilities of the students at the beginning of a course, so the teacher can know the level of the students before starting the classes.

Diagnostic tests are commonly used at schools in the first days of classes, so each institution has their own tests for this. Also, many teachers prefer to made their own diagnostic tests according to their own needs.

Placement tests: This kind of tests are used to "place" students in group according to their proficiency at the beginning of the course, so the content of this one can meet the expectations and needs of the students taking the course.

Many institutions have their own tests. However, some tests as CENEVAL are used not only to decide if the students are admitted into school, but to place them into groups. In the other hand, speaking about language courses, a tests which shows a result based on the Common European Framework for Languages can be used to place students into groups.