What does TOEFL test sometimes?

Traditionally, Bulgarian takers of the TOEFL have very good results. It is mainly teenagers in their last year at high schools (most notably English Language schools) that take the test, the most common purpose being to continue their education at American universities. Many of them go in for courses offered for preparation for the TOEFL examination.

Written by: Vyara Istratkova & Ellie Boyadzhieva South-western University of Blagoevgrad – Neofit Rilski
[toc class=”toc-right”]
General Introduction
The choice of this topic has been provoked by some observations upon the preparation work and the successful accomplishment of some tasks by would-be Bulgarian takers of the TOEFL.
Traditionally, Bulgarian takers of the TOEFL have very good results. It is mainly teenagers in their last year at high schools (most notably English Language schools) that take the test, the most common purpose being to continue their education at American universities. Many of them go in for courses offered for preparation for the TOEFL examination.
The Bulgarian candidates’ approach to the test is based mainly on the presumption that their high level of language proficiency is the necessary prerequisite for success. It does hold true to some extent as it is necessary indeed. Yet, language proficiency is not a sufficient condition for success, and that is what we will try to prove below.

Aims of the TOEFL

The TOEFL in used as a standard measure of the ‘English proficiency’ of the candidates taking it. The TOEFL score is a compulsory requirement of the admission offices of different educational institutions and organizations so that foreign candidates are considered for admission. (In the 1996 edition on p.6 we read: “Many Universities use TOEFL scores to fulfill the foreign language requirement for doctoral candidates whose first language is not English” (TOEFL 1996:6). Actually, this applies to nearly all (American) universities and it refers to undergraduates as well. An interesting point is that the TOELF is to be taken even by British students.

The material in focus and the authors goal

As it is well known the TOEFL consists of three sections: Listening Comprehension, Structure and Written expression and Reading Comprehension & Vocabulary section and it tests the candidates’ five skills – listening, structure, written expression, vocabulary and reading (TOEFL 1989:8).
The material in focus presents samples from the 1st, 2nd and the 3rd part of the Listening Section of two different editions of TOEFL which (also) differ in their format – the Listening Section of the Sixth edition of Barron’s TOEFL (1989) consists of a statement-restatement part, conversations and mini-talks, whereas in the Eight edition of Barron’s TOEFL (1996) the Listening Section is comprised of short conversations, longer conversations and talks. The examples included (in the corpus) are illustrative, the authors’ goal being to show that sometimes something more than language proficiency and proficiency in how to approach the examination is needed for the successful accomplishment of the listening tasks of the test.

Theoretical background

Tests are measurement instruments and (according to Carrol, 68) a “procedure designed to elicit certain behavior from which one can make inferences about certain characteristics of an individual” (Carrol 1968:46). In fact, tests are (or, ideally, should be) simulated real-life situations. On the basis of the performance of the candidate one should be able to make predictions about his/her language behavior in real life. The test itself should be based on the real needs and serve as basis for predictions about the ability of the students to cope with people and matters in reality.
In the theory of testing and evaluation, it is a well-known fact that it is “neither theoretically nor practically possible to define either an absolutely perfect level of actual language performance or an individual with perfect language ability” (Bachman 1990:11). This can be regarded as a direct consequence of the fundamental dilemma in foreign language testing, namely that “the tools we use to observe language ability are themselves manifestation of language ability” (Bachman 1990: 9).

Analysis

Our personal observations show that in the TOEFL examination the most troublesome area appears to be the Listening Section of the test. Why should reading for example be easier than listening? After all, both are receptive skills. Yet, the phenomenon observed is not a surprising one – listening comprehension is found to be among the most difficult tasks for the learners due to several reasons. First, spoken language is ephemeral, and the examinees cannot refer to it whenever they want to as is the case with Reading comprehension texts. Consequently, they have to rely mainly on their memory abilities, which differ throughout individually. Second, examinees are pressed for time as the time is limited within the length of every particular text. Third, they have to develop different strategies for the different types of tasks depending on the types of exercises.
Accordingly, candidates sometimes fail to successfully accomplish the listening tasks. This may be owing to the fact that they do not know the meaning of a word or an expression. However, even if they did, it still might not make sense in the context, as the ideas of the text (the discourse) present a different point of view on the reality. When so, situations occur in which the students make wrong decisions even in cases when they know every single word in the string because of the fact that what they have heard merely does not meet their expectations, and sounds weird to them. Thus, candidates often fail to see the point because the language in use is strongly culturally biased.

Discussion

We shall now consider several types of most problematic discourses for Bulgarian takers of the TOEFL excerpted from the Listening sections of the two TOEFL textbooks already mentioned and classified as academic and political discourse, education, different kinds of services, everyday practices, and social behavior.

I. Academic environment

The examples connected with academic environment outnumber the rest. In general, the
main factors for making mistakes in items reflecting the specific American academic environment are a consequence of the crucial differences between the American educational system and the one we have in Bulgaria. Basically, these differences concern the syllabus structure, the types of courses offered, the existence/lack of credit system, the academic positions. Not rarely do students face the problem of the so called false friends, as it is the case with the word “professor” for instance.

e.g. 1. (Barron’s TOEFL, 1989)
A: I’d like to take Dr. Sullivan’s section of Physics 100, but my advisor is teaching it too, and I don’t want her to be offended.
B: Who cares?

The question is: What should the woman do?
The correct answer is A: The woman should not consider her advisor in the decision.


e.g. 2 (Barron’s TOEFL 1996)
A: So the course is closed. This is terrible. I have to have it to graduate.
B: You are o.k. Just Dr. Collin’s section is closed. There is another section that is still open, but nobody knows who’s teaching it. It’s marked staff.

The question is: What would the student probably do?
The correct answer is C: Enroll in the section marked ‘staff’
(cf. B. Graduate at a later date; D. Find out who’s teaching the other section – confusing)

In both examples, in order to work out the right answer Bulgarian test-takers should know what the duties of the advisor are and what his/her functions are (advisors are members of the academic staff at American Universities). Moreover, at American universities one and the same subject is taught by two (sometimes even more) people and the students are free to choose whose course to attend which presents different practice from what we have in Bulgaria. One is also to be aware of the existence of credit system at US universities.

e.g. 3 (Barron’s TOEFL 1996)
A longer conversation between a student and a professor from which it becomes clear that the student couldn’t register for the class of the instructor because the course was closed by the time he got to the front of the line. The two problematic questions are: 1. What’s Mike’s problem (The correct answer: He must have the permission of the instructor); 2. What does the professor decide to do(T the correct answer: Allow Mike to take the class this term).

Here again, different regulations at US universities are reflected which do not have equivalents within the Bulgarian educational system – Bulgarian students are to be aware of the fact that there is a competition among the students to register for a course on first-come first-serve basis as well as of the possibility for exceptions allowed by the professors themselves.

e.g. 4 (Barron’s TOEFL, 1996)
A talk concerning assessment and evaluation:
“You have a midterm examination the last week of October, and a final examination the second week of December. The midterm is worth 25 points and the final is worth 50 points. That leaves 25 points for the project that …… and you have several choices to fulfill that requirement. You either write a paper or make a half hour presentation”……

The question is: What are the course requirements?
The correct answer is D: A midterm, a project and a final exam
(cf. A. A midterm and a final exam; B. A midterm and either a final exam or a project; C. A midterm and a paper or a presentation)

Obviously, the language in use is again culturally biased – knowledge is required about the content of the notions midterm and project (written paper/oral presentation); projects do not constitute part of the continuous assessment, they are part of the final grade. Students themselves own up that the whole seems to be a complete mess and they take pot luck when answering the question.

II. Education

e.g. 5 (Barron’s TOFEL, 1996)
The text presents a talk – in fact, this is a public service announcement telling about possibilities offered by a college for distance learning provided by the help of video telecourses. The question is : What is the announcement mainly about? (The correct answer: Video telecourses). In the explanatory notes the other two options (The Sun up semester programme & The community college campus) are said to be ‘secondary’ as they have been ‘used to develop the main topic’.

Bulgarian takers of the TOEFL exam are generally not aware of the existence of distance education as these are usually young people at school age.This type of education seems to be quite extended in the US whereas in Bulgaria such a possibility has been provided for only a couple of years now by few institutions and does not have as long traditions as in the US.

III. Political discourse

e.g. 6 (Barron’s TOEFL 1996)
This is a lecture on history with the following key expressions:
Each political party… nominates a slate of electors pledged to support the party’s nominees… each state has the same number of electors in the college as it has members of Congress …. registered voters go to the polls to choose the electors. The ballots list only the names of the candidates… This vote by the people for electors is called the popular vote and the candidates who receive the most popular votes win all the electoral votes in a state.


The text includes a lot of specific terms and the test-takers need specific subject knowledge about the political electoral procedure. At short notice and after having heard the talk once only they have to identify all the names and to adjust them to the roles played by the different groups of people at the elections. The very word college is rather confusing – “An organized body of persons with shared functions and privileges” (Oxford Reference Dictionary). Thus, it is beyond doubt that language proficiency is far from sufficient if a Bulgarian student is to answer the questions: How are the people nominated for the electoral college? What is the popular vote?

IV. Everyday practices – food and drinks

e.g. 7 (Barron’s TOEFL 1996)
This example presents a longer conversation between two speakers in front of a coffee machine which has just stolen the man’s money. Finally they decide to go to the library to try the vending machine there. The questions to be answered are: 1. Why did they decide to go to the library? (The correct answer:To have coffee); 2. What prompted the conversation? (The answer: The speaker wanted coffee); 3. What do the speakers mainly discuss? (The correct answer: The coffee).

Vending machines are typical for all public places in the US and other countries, yet hardly ever seen in our country. So, you can go to any public place (library including) and be sure to find a machine there whereas to a Bulgarian a library is a place to only borrow or read books. An additional problem is the fact that the word “vending” has no equivalent in Bulgarian – no word for non-existing artifact.

e.g. 8 (Barron’s TOEFL 1996)
A: What do you want on that?
B: Everything and extra catsup, too, please.
The question is: What does the man mean?
The correct answer is: lettuce, pickles, onions, mustard, mayonnaise, and catsup

The TOEFL textbook explains: “Everything is an idiomatic (?) expression that means all
the condiments included”. Obviously, this item does not test language proficiency at all. You have to have lived in the US for some time to know the expression. As Bulgarian takers of the TOEFL examination are usually young people at school age, most would have no prediction as to what the phrase means.

V. Health services

e.g. 9 (Barron’s TOEFL 1989)
This is a conversation between a pharmacist and a patient of Dr. Williams’s, the doctor being supposed to give a prescription.
Key phrases: Aspirin is the strongest medication I can give without a prescription; I can call him…. Dr. Williams will give me a pain prescription over the phone.
The question to be answered: 1.Where does the conversation take place? (The correct answer: In the drug store); 2. Why did the man call Dr. Williams? (The correct answer: To get a prescription for the woman on the phone)

Something one should take into consideration here is the existence of different words in the two varieties – drugstore in American and chemist’s in British English. It is true though it does not usually cause problems for Bulgarians, but it might well be a problem for some native speakers of British English.
The main problem for a Bulgarian here might be the behavior of the chemist which seems rather strange since it is not possible to get a prescription on the phone in our country. What is more, in Bulgaria, no prescription would be needed a for a pain killer.

VI. Services

e.g. 9 (Barron’s TOEFL 1996)
A: You turn on the TV by pulling on this button. The heat control is on the wall. Will there be anything else, Ma’am?
B: No, thank you.
The question is: What can be inferred about the man?
A.   He’s a TV repairman
B.   He’s a bell boy (the correct answer)
C.  He’s a tailor.
D.  He’s a security guard.

In order to decide on which the correct answer is, one is to first identify the settings – deduction should be made about a place with both a TV and a heating device for which someone is responsible. This is more or less a matter of common sense, however only possible for experienced travelers since very few hotels provide bell boys services in Bulgaria and these are usually very expensive ones.

VII. Social behavior

e.g. 10 (Barron’s TOEFL 1989)
This is a restatement example:
My invitation has RSVP printed at the bottom.
A.   I should dress formally.
B.   I should tell the hostess whether I’ll go (the correct answer)
C.  I should take liquor.
D.  I should buy a ticket.

It is of crucial importance here that the abbreviation be interpreted correctly. The abbreviation itself is a borrowing from French and means repondez s’il vous plait. This phrase has almost no circulation in Bulgaria except on very formal occasions . We should not except the average Bulgarian to know it.

Some conclusions

  1. The TOEFL textbooks do never provide any information about cultural specifics and concentrate on grammar and lexis only. Consequently, many prospective test takers go in for courses which prepare them for the successful sitting of the examination.
  2. The courses but most often provide instruction only in grammatical and vocabulary issues and focus mainly on the techniques and building strategies to complete the test tasks within the time required.
  3. The material discussed above and the situation observed lead inevitably to some important consequences concerning the type of instruction necessary for the preparation for the TOEFL not only in BG but elsewhere. The basic presumption is that every candidate targets higher scores, hence a correct answer to every test item seems to be crucial for him/her.
  4. This puts additional load onto the instructors as they have to provide at least the minimum information on several cultural specifics, and especially in the academic discourse as this topic area presents abundance of different cultural practices included in the test content.
  5. From everything said so far one general question follows: who will make a better instructor for the TOEFL examination – a native or a non-native teacher? In the authors’ opinion a non-native teacher seems to be better suited to perform this task as cultural issues involve implicit or explicit comparisons with another culture and only a non-native teacher can possess the knowledge and intuition what the similarities and the differences in the target culture are so that to elicit the most problematic topics and issues.
  6. In the Bulgarian environment the above problem might sometimes have an alternative solution, namely – a British native-speaker teaching in TOEFL courses. Having in mind that sharing one mother tongue does not necessarily imply sharing one and the same culture, we could conclude that a British teacher is often a better choice than an American one because the former will also feel the specifics of the social practices adopted in the US.
  7. The analysis of the material shows that the TOEFL ( as well as many other standardized proficiency tests of English – cf. FCE) tests language proficiency as well as cultural awareness although it is often pointed out that culture should be subject to a different testing procedure. Still, in our opinion it is often absolutely impossible in cases of proficiency tests to divide language from cultural competence as it is intrinsically embedded into the use of the particular language.
  8. One additional point to be made here is that the preparatory TOEFL courses can successfully provide information about the social, educational, behavioral and political specifics in the US .
  9. Finally, we can conclude that the TOEFL aims at testing not only the language competence of non-native speakers of American English, but also their communicative competence which encompasses both linguistic competence and cultural awareness without which language cannot be used appropriately in different social contexts. So, the TOEFL reflects the general tendency in ELT which is to move further on from pure accuracy (or language competence) to pragmatic competence.

Pre-service teacher education in Bulgaria – critical evaluation

Written by: Diana Yankova, Irena Vassileva [toc class=”toc-right”] Aim.
The aim of the present study is to establish the correlation between the objective social needs for EL specialists, the students’ subjective motivation for taking up English language studies and their career expectations on the one hand, and the actual type of education in terms of knowledge and skills they are equipped with after graduating from the Bulgarian universities on the other, since, as is duly pointed out by Foldes (1994:3) “The market demand does not correlate automatically to the established social and individual needs as they may also be influenced, for example, by requirements of the educational system (administrative requirements, certificate recognition, etc.) or by tradition and prestige.”
The study is based on evaluation of: questionnaires to students and university lecturers, the existing curricula, job offers in terms of skills requirements, the need for teachers versus other career opportunities.

Critical analysis of ELT in Bulgaria.

High Schools.

The current ELT policy in Bulgaria has a long way to go. At present the bulk of practising teachers are either state-university graduates who have not been expressly trained for the job or people not qualified at all. At the same time, there is a marked shortage of EL specialists. It seems that Bulgarian educational authorities have no clear-cut FLT policy since there is no single compulsory FL in secondary and high school; students have to choose among available courses, contrary to tendencies in most European countries. The Ministry of Education has tried to prevent the influx of unqualified teachers into Bulgarian high schools by forbidding school authorities to appoint them. The students of English, on the other hand, are not fascinated by the idea of becoming secondary school teachers due to the low social prestige and payment of the job. The ensuing outcome is a further decline in the level of high school graduates as far as their knowledge of English is concerned, thus closing the vicious circle.

Pre-service teacher training.

The 50-year long tradition in English language and literature studies at university level has undoubtedly its strong points. Bulgarian universities have produced highly qualified English language specialists well-versed in linguistic theory and literary studies who have achieved considerable standards and international recognition of their academic research. All this, however, has been at the expense of more practically oriented teacher training programmes. Present curricula, adopted with practically no substantial variation by all state university English departments in the country, aim at providing students with comprehensive knowledge of the English language as a system and British and American literature in the course of 8 semesters (BA degree), followed by 2 semesters of specialized MA courses.

The BA course.

In contrast to current practice in other countries (e.g. Germany), graduate studies at Bulgarian Universities are English-language medium based with the aim of further developing students’ linguistic competence, which has proved to be both time-consuming and costly. This traditional practice is still adhered to in spite of the high general English admission level of the students (approximately Cambridge advanced).
The basic compulsory courses are covered during the first 6 semesters and the specialized courses (electives) appear as late as the 4th year. Compulsory courses may roughly be divided into the following categories with the respective percentage of overall teaching classes: General English (seminars on the lexical, grammatical and phonetic level of the language plus translation) – 47%; Linguistics (lectures and seminars) – 30%; Literature (British and American) – 20%; Culture studies – 3%.
The data presented above demonstrates clearly that the focus falls heavily on acquisition of linguistic competence, whereas recent ELT trends point to the precedence of pragmatic competence, thus courses such as “Shakespeare on Stage and Screen” have marked an economical attendance, to say the least.

The MA course

is based on the same assumption, at least as far as two of the electives are concerned: the ‘British studies’ programme includes predominantly British literature, the ‘American studies’ programme – American literature respectively, which is in total contradiction to the established fact that cross-cultural competence is an integral part of communicative competence.
Other MA courses include applied linguistics, and translation theory and practice. The applied linguistics course is highly theoretically oriented and far-fetched from any practical applications and no teacher-training course is offered at all.
Teacher-training qualification is compulsory for all university students of English as part of the BA degree and consists of general introductory courses in psychology, pedagogy, methods of ELT and observation of classroom lessons plus 5 weeks of teaching practice. The actual training students are subjected to reminds one of the English language education in Germany criticised by Edelhoff (1995:38) in the following way: “……very few (students) were educated to be intercultural learners and communicative classroom teachers”. And how could they be, having in mind the multitude and diversity of subjects they are made to study? Literature, linguistics and teacher training merged into one major turns graduates into Jacks-of-all-trade and least of all into FL teachers.
Attempts at ‘breaking the rules’ in terms of earlier specialization and orientation of students towards the teaching profession have recently been made by some private universities in Bulgaria. Their programmes pay particular attention to the specificities of ELT to various age groups, to learners with diverse needs (EGP, ESP) in different types of courses (intensive, extensive, etc.), which makes the students better equipped for teaching at all levels of the educational system up to university level. These universities seem to be on the right track in showing greater flexibility and adequate consideration of modern tendencies in teacher training by focusing on up-to-date interactive, co-operative, communicative and interdisciplinary approaches to foreign language teaching. Unfortunately, they are a drop in the ocean and besides, encounter a number of serious problems due to the limited number of students who enroll (as tuition is rather expensive) and the lower admission level of English.

Solutions and recommendations.

Obviously, there is a dire need for refocusing and reconceptualization of the whole basis on which ELT in Bulgaria functions. All the current socio-political, economic and technological changes have initiated new types of external and internal needs and motivation in the language learner. English-language specialists and educational authorities should reconsider the way English is taught. Language learning should get away from the static but still powerful, especially in Bulgaria, Latin or Greek model. A total discrepancy exists between students’ expectations and social demand for teachers on the one hand, and the qualifications of university graduates on the other. Questionnaires show that graduate students are not satisfied with the teacher qualification they obtain, neither are they fascinated by the idea of becoming high school teachers as the job has an unduly low social standing. University lecturers, on the other hand, do not seem particularly inclined to get away from the classical model of traditional philological instruction. Perhaps most of them simply feel comfortable enough in their old shoes?
At the same time, the conducted survey of job offers in Bulgarian media shows an ever increasing demand at present for EL teachers at all levels in both private and state institutions. High schools suffer an incredible shortage of ELT staff and are forced to offer other foreign languages, thus not being able to meet their students’ needs. Those needs, on their part, arise from the fact that it is practically impossible to get any decent job in Bulgaria without a good knowledge of English.
One way to remedy this situation seems to us to start with changes in syllabus design. From the onset of their university education students should be offered the possibility to choose between a more practically oriented teacher training course or predominantly theoretical linguistic/literary studies. This division of labor would benefit teacher training programmes immensely by providing time in the already tight students’ schedule for important courses such as ESP, materials design, culture studies, teaching English to various age groups, etc.
Besides being a comparatively new theoretical discipline, English for specific purposes finds more and more practical applications with the constant setting up of new specialized high schools and the springing up of new majors at Bulgarian universities such as public administration, management, business studies, etc. As our questionnaires show, there is a marked tendency for instrumental rather than integrative motivation with most Bulgarian learners of English. The needs of the different types of learners determine the content to be learnt, the skills to be acquired, the criteria for learners’ performance. Therefore, future teachers should be instructed how to teach English to lawyers, economists, engineers, etc. – something heretofore overlooked in the curriculum.
This leads to another very important aspect of teaching – that of materials design. With the exception of a few specialized EL schools in Bulgaria, other institutions make use of textbooks and materials produced mainly in Britain which, of course, cannot take into consideration students’ L1 and the possible points of collision between native and foreign language, i.e. those elements of L2 which are especially difficult for a speaker of Bulgarian to acquire. There is a number of English-Bulgarian contrastive studies that could be resorted to in designing textbooks with the aim of eliminating of L1-L2 negative transfer. We do not in any way wish to undermine the use of British and American textbooks, but rather endeavor to suggest that they should be complemented with teaching materials based on the established cultural and linguistic differences and similarities of the respective communities. Few educationalists, however, set about producing such materials perhaps because they find it time-consuming, low paid and not worth the trouble; and above all, because they have not been taught how to do it. Teachers are not equipped with the necessary knowledge and skills for materials development and on entering the teaching profession they tend to stick to the good old textbook approved by the Ministry of Education.
But as any good teacher is well aware, one way to keep learners interested, is going beyond the old chalk-and-talk method by providing more communicative, challenging, suited to learners’ needs activities. Instructing future teachers how to design their own materials and be creative would make the learning process more meaningful, more variegated, more made-to-measure. The choice of materials is not an end in itself, they should be made to work in the classroom and should be effective as representation of language as well as effective instruments of language learning. Teachers should be taught (!) that they themselves shoulder part of the responsibility of what is to be learnt, how it is to be learnt and how to take into account the particular target and learning situations. But first and foremost, language learning should be fun!
Yet another very important concept in FLT is cross-cultural learning. The crux of communication is the capacity of people from different cultural backgrounds to interact. Communicative competence encompasses not only linguistic competence but also cultural awareness without which language cannot be used appropriately in different social contexts. Teachers-to-be should be encouraged to take the role of intercultural interpreters by providing them with the knowledge and skills to digest and relay the socio-cultural environment of L2 community. They should have the capacity to deal with printed and electronic authentic materials which is the link of intercultural experience and facilitates the learning process.
Taking into account all the above considerations and incorporating them into pre-service teacher training programmes would raise the awareness of future teachers of learners’ needs in terms of age and professional interests, of teaching objectives and environment. It would make them more sensitive to ways of facilitating the transition from learning to acquiring a FL, and better equipped to show flexibility and creativity both in the immediate teaching context and in defining long-term goals. We are more than convinced that these measures will mark the right path towards raising the prestige of the teaching profession, which has for so long been looked down upon by society as a whole and neglected by state authorities in terms of funding.

Final remarks.

The general idea of the present study has been, in a nutshell, to locate, clarify and offer at least partial solutions to the current state of ELT and teacher-training courses in Bulgaria. We strongly believe that what is most needed is a substantial refocusing of curricula towards the inclusion of critical and social theory and the adoption of an interdisciplinary approach, without which ELT in this country would continue to lag behind global modern trends. And even more importantly, we do not consider this to be a problem only in Bulgaria, but also in many other countries all over the world. We sincerely hope that reading this article many educators might stop and think twice about the way English is taught.

REFERENCES:

Edelhoff, C. 1995: English Language Learning in Europe: Issues, Tasks and Problems. in Best of ELTECS, British Council.
Foldes, C. 1994: Deutsch als Fremdsprache in Mittel-, Ost- und Sudosteuropa. Uberlegungen zu Bestand und Bedarf. in: Deutsch als Fremdsprache 1, 3:12.