Measuring Science

Sacred code

Study shows that French and German researchers are not exempt from losing influence when they do not publish in English

Published in March 2011

Imagem: NELSON PROVAZI

The language barrier, which is responsible for the small amount of written science published in languages other than English, is a problem that does not only affect researchers from emerging countries such as Brazil. A study led by physicist Anthony van Raan, director of the Centre for Science and Technology Studies at Holland’s Leiden University, found that this barrier also negatively affects the scientific powers of France and Germany, which rank right behind the United States, China, the United Kingdom and Japan on the list of countries with the highest number of published scientific articles. Studies by French and German scientists, however, have less impact when they are published in their respective native languages.

Van Raan, an expert in scientometrics, a discipline that generates information to facilitate overcoming scientific challenges, was involved in the development of the Leiden Ranking, which is a collection of data generated by the Dutch university to analyze the scientific production of countries and of higher research and learning institutions. In the most recent Leiden Ranking, the University of São Paulo (USP) achieved fifteenth place in the list of universities with the highest volume of scientific production. The study on the language barrier focused on the list of the world’s 500 greatest universities, ranked according to the impact of their scientific articles as reflected in the Thomson Reuter Company’s Web of Science (WoS) database. The impact factor is a measure reflecting the average number of citations to articles published in science and social science journals. The Dutch researcher noticed that the modest performance of a number of French and German universities in the rankings was not consistent with their academic prestige. For comparison, he produced a second list in which only the scientific papers published in English-language scientific journals were considered, whereas articles written in the authors’ native languages were not. Van Raan found that the performance of the German and French universities was higher in the ranking of the list on articles in English because the impact of those papers was higher than that of articles published in the authors‘ native languages.

The University of Nantes, for example, ranks 106th on the list of articles published in English and 201st on the list that takes into account articles written in other languages as well. Germany’s Heidelberg University and Munich’s LMU ranked 109th and 114th, respectively, on the impact list based only on articles in English but dropped to 150th and 166th, respectively, on the list of all articles. “We found a dramatic and underestimated effect in measuring the impact,” said Van Raan. “The articles published in languages other than English dilute the average impact of countries such as Germany, Austria, and France. This effect is particularly pronounced in the applied sciences, such as clinical medicine and engineering, but also occurs in the social sciences and the humanities. Because medicine accounts for a major part of any country’s scientific production, this effect influences the ranking of a university.”

Tool
The basis of Van Raan’s concern refers to the use of bibliometric indicators linked to impact factors. Because citations carry considerable weight in the rankings of universities, such as those prepared by The Times Higher Education and China’s Shanghai Jiao Tong University, Van Raan suggests that one should be cautious when analyzing these lists and proposes a controversial alternative to remove the bias; he asserts that, for comparison purposes, one must take into account only the English version of the scientific production of the institutions and ignore articles written in other languages. “Calculating the indicators only on the basis of publications in English is the only fair procedure,” Van Raan states. There is nothing new about the fact that proficiency in English is a basic tool for researchers in all fields of knowledge. This was already true in the 1930s, when German researchers published a study, in German, that associated cigarette smoking with a higher incidence of lung cancer. Because of the language barrier, this fact remained almost unknown until the 1960s, when British and North American researchers reached the same conclusion. At present, resisting the supremacy of the English language is counterproductive says Sonia Vasconcelos, a researcher from the Federal University of Rio de Janeiro (UFRJ) and author of a doctoral thesis on the language barrier, which she presented in 2008 (see Pesquisa FAPESP, issue 162). “Countries in which English is the main language have a huge advantage, but there is an international movement among research institutions and scientific editors from various non-English speaking countries to narrow this advantage. In the case of Brazil, it is necessary to train our researchers, especially those working in the fields of science and technology, to write properly in English and to develop some independence in order to be able to communicate with their peers in international contexts,” she states.

Imagem: NELSON PROVAZI“In Germany, currently, a number of post-graduate courses are taught in English, which helps students overcome this barrier. This trend is also observed in France, which has always cultivated its language in academic settings, albeit with a strategic attitude towards English. Brazil has no strategy to deal with this challenge,” Sonia says. Van Raan’s suggestion that scientific papers written in native languages should be ignored to improve international comparisons might lead to another kind of bias that is caused by the absence of contributions in major fields of knowledge. “Production in a local language is a crucial part of the knowledge generated by countries and cannot be disregarded,” states Abel Packer, who is part of the team that coordinates the SciELO electronic scientific library in Brazil. Packer notes that papers on themes related to the health sciences and agrarian sciences have traditionally been written in Portuguese because it is important to convey this knowledge to professionals in these fields.

“The issue involves not only scientists, who generally speak English, but other users of scientific information who are not proficient in the language,” according to Packer. “Multilingualism is part of scientific communication and has its roots in the fact that science is part of culture. Science cannot be placed inside an ivory tower and kept apart from the rest of society; it must be regarded as a source of knowledge for economic and technological development. If the national scientific community doesn’t make an effort to create semantics in its own native language, the country and its culture will be unable to absorb ideas and knowledge that essentially serve its society,” he states.

In the opinion of Luiz Henrique Lopes dos Santos who is assistant coordinator at the Human and Social Sciences, Architecture, Economics and Business Administration of FAPESP and a professor in the Department of Philosophy of the School of Philosophy, Literature and Human Sciences at USP, the issue demands compromise because it is not limited to the issue of impact. “This is also a cultural issue,” he says. “Language is an essential element of a country’s culture and it is constituted and enriched by the interaction between its more common and more sophisticated uses – as in literature, science and philosophy. No country can afford to totally disregard its native language as a vehicle of knowledge production.”

Packer further proposes that the fact that the number of papers written in Portuguese is growing in the indexed journals must be considered. Until 2007, the percentage of articles published in Portuguese on the Web of Science database was 8.5 percent. This figure has since increased to 22 percent. “The percentage has increased because the list of indexed Brazilian journals has expanded from 34 in 2007 to 133 currently. Therefore, Brazil now ranks 13th in terms of scientific production. If we disregard the journals in Portuguese, then our ranking drops to seventeenth,” he states.

Outstanding detail – One must also consider that writing in English is not sufficient to ensure the insertion of citations and the desired prestige. A study by Rogério Meneghini, scientific coordinator of the SciELO Brasil library, states that even articles written in English and published in Brazilian journals produce fewer citations. Meneghini invited 9 Brazilian scientists who normally publish their papers in international journals to publish an original article in the May 2008 issue of the Anais da Academia Brasileira de Ciências journal. The intention was to evaluate the extent to which the related authors were able to transfer their prestige to the Brazilian journal, which is published in English. Two years after the publication, he found that the number of citations of those articles was higher than the number of citations of the journal’s other articles, specifically, 1.67 citations versus 0.76 citations from other articles. The 62 articles published by the same authors in international journals in 2008 each achieved an average of 4.13 citations. According to Meneghini, the difference can be attributed to the fact that the Brazilian journals have less international exposure even though the authors tend to send their best articles abroad. A striking fact was that the 9 authors did not cite articles from Brazilian journals. Only 1.52% of the citations mentioned by the authors in 2008 referred to papers published in Brazil. Meneghini suggests that quoting Brazilian journals does not lead to prestige. “It seems that the authors chose to neglect citations from Brazilian journals, assuming that they might convey the impression that the article was inaccurate,” he said.

This does not weaken the consensus that it is crucial to encourage the use of English. “When a researcher makes the effort to cite papers written by his fellow colleagues, it is frustrating to realize that the reference cannot be consulted abroad because the paper is available only in Portuguese,” says Sonia Vasconcelos. In the opinion of SciELO’s Abel Packer, the solution is to invest in the translation of articles written in Portuguese, making them available in English as well. “This would require heavy investments, but I see no other solution to increase the exposure of the work produced by Brazilian science,” Packer states.