On-Going Assessment: Article Review

Salam Alayk and Hello viewers! :) 

The selected article: 

The Utility of Article and Preposition Error Correction Systems for English Language Learners: Feedback and Assessment 

By: 

Martin Chodorow, Micheal Gamon and Joel Tetreault 
Language Testing 2010, 27:419

Source was retrieved from:

http://ltj.sagepub.com/

To view the copy of the article, please go to http://www.mediafire.com/view/?cju1uujwe19eoed

THE SUMMARY 

As the title of the article suggests, the three researchers, who were from the United States of America, attempted to take a closer look on the usability of computers in assisting language learners. This concerns both native and non-native speakers. Talking about the non-native speakers, it was found that most of them face hard times in mastering an important aspect of English Language - English Grammar - especially when it involves articles and prepositions. The difficulty is driven by two major factors. The first one deals with the usage of both articles and prepositions; in which is highly dependent on different contexts of meaning. Apart from that, the learners find it uneasy to understand the variability of the usage of both components. Prior to the problems mentioned, the researchers believed that computers can play a role in providing immediate feedback and assessment to learners who struggle with such difficulty. Therefore, two computerized systems had been selected to be described and evaluated to meet the purpose of this study.

The first system is called Criterion. Criterion is a type of system meant for helping language learners to detect language errors made in their writing. The article detection error was added in 2005. This was followed by the insertion of the preposition detection error three years later. This system was used in the study to assess the extent to which it assisted the participants involved to improve their writing over time. 

Criterion

The second system, ESL Assistant, is a prototype web application meant to enable language learners to get access to large statistical resources and web-based search in finding the explanations and appropriate sentences to correct grammatical errors made by learners in their writing. Learners were given choices either to make use of or to ignore the 'signals' given by the application each time an error is detected. The researchers used this application to examine the usage patterns logged from the web service.

ESL Assistant

Instead of focusing on the practicality of the systems, the researchers felt more interested to put an emphasis to the impacts made by the systems to the learning process experienced by the users. Hence, two separate experiments were conducted. The former analyzed the advantages of feedback provided to the learners; while the latter highlighted on the learners' attitude towards the importance of feedback to their writing process.

EXPERIMENT 1 (CRITERION): IS FEEDBACK IMPORTANT IN IMPROVING THE QUALITY IN STUDENT WRITING?

463 participants were selected, consisting of students who enrolled in an introductory psychology course at two different US universities. They were required to write a 500-word essay about motivation, which was one of the topics learnt in the stated course. Feedback was divided into several categories; which comprised of the detailed feedback automatically made by the system, the detailed feedback made by an instructor, and lastly, no detailed feedback. The participants came to the lab a week after the first session in order for the researchers to conduct the second part of the experiment. Upon reading the feedback provided to each of their essay, they were encouraged to revise their essay thus to produce the new version of the edited one. The essay was finally analyzed by the system and graded by two instructors.

The first experiment resulted in a reduction in terms of errors made by the participants in the final version of essay as compared to the earlier one. On the other hand, the results obtained were compared between native and non-native speakers. It was believed that the error detection module had helped both the native and non-native speakers to make them aware of the presence of article and preposition errors, thus to improve their final essay. However, the number of errors made between the native and non-native speakers depicted only a slight difference. Despite the significance of the findings obtained, the decrease of errors in the final version essay did not cater to a wide range of learners, since the scope of study was rather limited to a specific population. 


EXPERIMENT 2 (ESL ASSISTANT): DO USERS DISTINGUISH BETWEEN GOOD AND BAD CORRECTION SUGGESTIONS?

With the permission obtained from 107 selected participants, writing activities were analyzed by the researchers. 4550 sentences were assessed from 702 sessions conducted by the participants over a three month period of time; and the writing material was limited only to e-mail compositions. The logged data contained the information about the suggestions proposed by the system, along with the actions that might be performed by the users during the process of writing. The suggestions include possible corrections that could be possibly opted by the learners to replace the errors they had made. The researchers obtained necessary data from the site traffic based on the participants' writing activity.

The results showed that the learners were aware of the need to make use of the application to correct preposition and article errors in their writing. They were also able to be selective in choosing the most appropriate option from the suggestions available. That makes us to come into a conclusion which pronounces that the system somehow, in this case, did trigger the cognitive understanding of the learners, despite of the initial concern of the possibility for the system not to contribute to the process of learning.

MY REVIEW

Personally, I decided to choose this article because I am really interested with how the feasibility served by various computer applications can greatly assist language teachers. As a future language teacher, I would like to see the possibility of using the applications during my teaching days later. Undoubtedly, it is never easy for a language teacher to mark students' essays, especially when they are given a limited time to do so. Hence, my comments will be based on several aspects which I believe are important to be highlighted prior to the results of this study. 

Validity concerns with the degree to which the presented assessment able to really measure what is supposed to be measured. In this context, the researchers intended to specify their focus limited to testing preposition and article errors. Based on the article, it is agreeable to say that non-native speakers usually face difficulty in understanding the usage of article and prepositions. However, it is stated in the results section from the first experiment that the number of preposition and article errors made by the non-native participants was just a little bit higher than that of the ones made by the native group, specifically, after the second session. Though this results were obtained due to the small number of participants involved, I believe that native speakers too tend to experience confusion when dealing with the parts of speech tested in the study. 

This perhaps is closely related to the contexts; when the learners, either the native or non-native, are trying to adapt their knowledge about the functions of each of the part of speech tested to real situations. Based on the first experiment - to answer if feedback important in improving the quality in student writing, I will simply say yes, it does; therefore the assessment made can be considered valid. Though, more research need to be done to analyze if the nativity of the learner can make a difference in terms of the results obtained after being assisted with feedback from the same source; in this case, the source refers to the system and the instructor involved. How does the difference affect validity measures? This is the question left in my mind.

Secondly, I am quite interested with the way the researchers assessed the attitude of the language learners when choosing the suggestions. This is obviously based on the second experiment. As a learner, I find it pleasurable to have applications like ESL Assistant to help me to improve my writing. However, certain level of language competency is needed in order for me to pick up the best choice out of the suggestions proposed from the web-search engine. I try to imagine myself as a secondary school student, if I were to use this  application, and if I were an average language student, I rather choose one of the suggestions based on my intuition. This indicates that this application is not suitable to be used by all types of learners. This actually really happened to me during my school days, anyway! Henceforth, I am concerned with the practicality of this application in order for it to be applied in Malaysian school settings. Perhaps this application works best with advanced learners. 

In a positive side, I am confident that both of the applications used in this study play a major role in enhancing student motivation. Learners can enhance their creativity without feeling worried or afraid of making language errors and mistakes. Psychologically, some people just do not have the inner strength to receive verbal feedback due to shyness or any other reasons. Therefore, this is a good avenue for them to analyze the areas in the language they need to improve . Confusion and falseness are not left unattended; so this may implicitly promote autonomous learning.

In conclusion, error detection applications really help language teachers to make their work faster. Still, continuous revisions need to be done by the software developers so that the devices can work more efficiently. More samples of original pieces of writing need to be analyzed so that more errors can be detected automatically. Apart from that, it is hoped that the applications can work with all types of language learners. To reflect on the study discussed, further analysis should be done not only considering on specific syntactical aspects of the language, but to include assessment of the learners' usage of vocabulary and mechanics in writing.