Imprimir Republish

evaluation

A lot of heat, little light

Bias in the interpretation of data generates debate about the players involved in the increase in Brazil's scientific production

produção cientifica2BRAZBrazil’s scientific production has been progressing at a consistent speed since the 1980’s, but the announcement made by the Ministry of Education, Fernando Haddad, regarding last year’s growth generated a lot of heat but little light. At an event at the Brazilian Science Academy in Rio de Janeiro on May 5, Haddad stated that the number of indexed articles had risen by a spectacular 56% in that year. “We are experiencing times that have enabled an increase of more than 50% in Brazilian production, thanks to the work of the Ministry of Education (MEC) and of the Ministry of Science and Technology,” said the minister, according to the daily newspaper A Folha de São Paulo. The figures, which are far higher than those achieved by other countries, even China, whose scientific production is the fastest growing on earth, gave rise to some perplexity – and the experts in scientometry, the discipline that generates information to encourage the overcoming of scientific challenges, were the first to try and understand the minister’s figures.

Actually, the 56% increase did not reflect an increase in production, but was, rather, a reflex of the growth in the number of Brazilian publications on the Thomson Reuters Web of Science site. In 2006, there were 26 . This figure rose to 63 in 2007 and to 103 in 2008. An analysis of other databases to which the Ministry of Education also has access showed figures that were certainly more, but well beneath the 56%. The Science Citation Index, which also belongs to Thomson Reuters, but whose methodology did not change substantially, indicated 15% growth in the number of scientific articles in 2008. As for the Scopus database, the competitor from the Institute for Scientific Information (ISI), maintained by the Elsevier publishing house, reports a respectable, but realistic, growth of 8.9%.

In an article published on May 12 in the Folha de São Paulo newspaper, chemistry professor Rogério Meneghini, the scientific coordinator of the SciELO electronic library, brought to light the origin of the exaggerated interpretation of the data: what grew was the visibility of Brazilian research, driven by the inclusion of domestic scientific periodicals, whose quality was recognized by Thomson Reuters. Meneghini asserted, contrary to the minister’s announcement, that this advance occurred in a sector in which the federal government “only invests in very inexpressively,” since just a meager 0.4% of the budgets of CAPES (Coordinating Office for the Training of Personnel with Higher Education, a Ministry of Education bureau) and CNPq (National Scientific and Technological Council), equal to R$10 million in 2008, was earmarked for some 240 domestic journals. “The only Brazilian initiative to improve journals, besides a dedication from the editors, is the SciELO program,” wrote Meneghini. “In Brazil, SciELO has a role akin to that of ISI, the role of indexing the best Brazilian journals, selected on the basis of quality criteria; however, it goes beyond this, because it provides open access to the full articles. At present, it comprises 205 journals. It is important to stress that of the 103 Brazilian journals indexed by ISI, 81 are on the SciELO database. The program’s budget for 2009 is R$2.5 million, 80% of which comes from FAPESP (São Paulo State funds) and 10% from CNPq (federal funds),” concluded the professor.

The confusion after the announcement does not overshadow the fact that Brazilian scientific production is rising at an auspicious pace. “The growth is so major that it is only trailing behind China’s, which grew by 11% in 2008 vs. 2007. In other words, we outdid the annual growth of countries that have a history of strong expansion, such as India, South Korea and Taiwan,” said Eloisa Viggiani, a product manager with the Elsevier publishing house, which is responsible for the Scopus database. SCImago Journal and Country Rank put Brazil in 15th place among the countries with the greatest scientific production in 2007, out of a group that comprises countries with very different academic traditions and populations, such as the Netherlands and Russia (slightly ahead of Brazil) and Taiwan, Switzerland and Sweden (just behind). The index is a joint project of the SCImago group, formed by researchers from four Spanish universities, and of Elsevier, owner of the Scopus database. In 2008, the Web of Science ranking divulged by the Ministry of Education, Brazil ranks 13th, ahead of the Netherlands and Russia.

Still, if this confusion around the figures had one merit, it was to encourage discussion of the reasons for growth. “There are several reasons for the growth or reduction of the scientific production in a nation, region, institution and/or individual,” wrote the scientometry expert Jacqueline Let, a professor from the Federal University of Rio de Janeiro (UFRJ), in an article in which she commented on the figures. Mentioning a Capes press release, she listed reasons, such as the investment in human resources, by means of grants, and the access of researchers and graduate programs to international periodicals. “These functions – which are not exclusive to Capes – have been fundamental for a real advance of the activity in this country and they have been playing a decisive role for more than a decade,” she said.

028-031_prodcientifica_160-01Rankings
The contribution of São Paulo to Brazilian scientific production is a fundamental piece of data for the debate. The state, which has been producing half of the country’s scientific articles for three decades, stands out for having three large research universities – of which two, USP and Unicamp, are the only ones listed among the top 200 in the world in international rankings – and for the volume of resources invested in science and technology, as well as for the regularity of the investments (see table). USP, which grants some 2,000 doctorates a year, and Unicamp, which grants about 900, each generate more PhDs than any American university. Unesp does not lag very far behind: in 2000, it granted 765 PhDs.

There is a certain measure of agreement that the growth of the postgraduate system and the use of international productivity indicators to evaluate them has contributed largely to this progress, but the variables that determine the growth are multifaceted and do not fit into a linear system. Hernan Chaimovich, a professor at USP’s Chemistry Institute and vice-president of the Brazilian Science Academy, states that the recent growth of federal investments is yet to have a measurable impact on Brazilian scientific production. He compared the growth curves of the FNDCT (National Scientific and Technological Development Fund) resources with scientific production and found that, up to 2003, scientific production was growing despite the irregularity of FNDCT funding. “The two curves advanced independently of each other and only started climbing together as of 2003,” says Chaimovich.

The professor also draws our attention to another phenomenon: the percentage of growth in the number of scholars granted PhDs has been tapering off in the last two year, but scientific production has not. From the mid-1990’s to 2003, the average annual rate of growth of the number of PhDs was 16%. From 2003 onward, this rate plummeted to a much lower level, some 4% a year. This, however, did not lower the vigor of our scientific production. “One can state that for quite some time, the rise in the number of PhDs drove growth, but in the state of São Paulo this is reaching its limit, beyond which productivity, which is already extremely high, has no further room for growth,” he states. According to him, one probable hypothesis for the continued growth, especially regarding the São Paulo state example, is the growth of the number, not of PhDs, but of post-doctoral scholars – a new piece of information in the evolution of Brazilian research.

028-031_prodcientifica_160-02

São Paulo State’s contribution
Why the state produces half of the Brazilian articles

Science Citation Index data show that, despite the substantial growth of national scientific production over the last 30 years, one variable remained unchanged:São Paulohas accounted for half of the scientific articles published by Brazilian researchers. That was the case in 1980, when São Paulo accounted for 1,090 of the 2,215 Brazilian articles. And it was still the case in 2008, when 9,513 of the 18,783 Brazilian articles were also produced inSão Paulo. There are many reasons for this, although none, alone, can explain the phenomenon without the others. Money, undoubtedly, has been an important factor. However, in São Paulo, 70% of government investments come from the state, and only 30% come from the federal government. Likewise, other states created their own research aid agencies and undertook to produce knowledge with more tenacity. However, even this did not affect São Paulo’s proportional weight. “There are other elements that are as important, if not more important, than money, such as very stable state universities in São Paulo, a qualification policy that is at a far more advanced stage than that of the federal universities, besides the high regard for academic values that is built into the São Paulo universities and institutes,” states Carlos Henrique de Brito Cruz, FAPESP’s scientific director. “To this day, federal universities lack real autonomy, contrary to the São Paulo state universities,” he says.

According to physicist Daniel Pereira, dean of research at Campinas State University (Unicamp), from 2004 to 2009, São Paulo’s leading position was due to a set of factors ranging from the regularity of research funding, since 1962, when FAPESP was founded, to the training of highly qualified personnel in the state’s universities.

“FAPESP’s existence is a fundamental fact,” he says, referring to the 1% share of the state’s tax revenues that are regularly invested in research. “FAPESP also exerts an influence through its scientific policy decisions,” states the professor, mentioning, as an example, the new release by the Ministry of Science and Technology of the National Science and Technology Institutes, which are networks of excellence following the model first employed last decade by FAPESP’s Cepids (Centers of Research, Innovation and Dissemination). In any ranking of the largest Brazilian universities, USP and Unicamp lead. They concentrate a large number of graduate programs, most of which have achieved high grades in Capes assessments. One should highlight the substantial growth of Unesp’s productivity in the last few years,” states Pereira. He reminds us that the federal universities in São Paulo, such as Unifesp and UFSCar, have a far smaller scale of production than the state universities, although it is of high quality. “The most recent of them the Federal Universityof the ABC region, was staffed with teachers that came mainly from the state universities,” he says.

Hernan Chaimovich, a professor at USP’s Chemistry Institute, highlights the unique situation of the São Paulo state universities vs. the group of federal universities. “USP, Unicamp and Unesp became autonomous 20 years ago. And they made the decision to become research universities,” says Chaimovich, recalling an episode that affected USP deeply: the announcement in the media, in 1988, of a list of professors with no academic production in the preceding period, the so-called “unproductive list.” “The list had a major impact upon the university, namely, the consolidation of the idea that USP could not have people that did not produce knowledge. This spirit permeated the São Paulo state universities and, to this, the pressure of theCapesevaluation was added,” says Chaimovich.

The professor warns us that he does not mean to disregard the federal contribution to São Paulo science. “But with that contribution only, São Paulo could never maintain this pace of participation,” he evaluates. He also adds that this situation could change. “The relative weight of São Paulo in Brazilian research could fall if this type of environment is consolidated in other states and in the federal universities. Rio de Janeiro has significantly upped its research funding. It would be great if this proportion changed, but the investment must be regular to have an effect, as was the case in São Paulo,” he states.

Republish