MARCOS GARUTIPhysicist George Matsas, a professor from the Paulista State University (Unesp), has developed an original method for identifying scientific leaders and assessing their influence. It also measures how robust academic communities are. The method is the Normalized Impact Factor (NIF), an equation that is applicable to members of any scientific community whose production is measured in articles published in indexed international journals, as happens with the hard sciences. The NIF is based on a simple idea: the index ponders the number of times the researcher quotes others in his articles with the number of times he is quoted in the articles of others.
Whoever receives most mentions has a NIF bigger than 1 and can be considered a leader, because he exercises more influence over other members of his community than he is influenced by them. An NIF less than 1 is typical of someone who is led – his influence is still not sufficient to receive more mentions than those he produces. “In my opinion the index captures the essence of what it is to be a leader”, says Matsas, who is a professor at the Institute of Theoretical Physics at Unesp. He is a member of the physics area coordination team at FAPESP and the National Council of Scientific and Technological Development (CNPq).
The index was presented to the scientific community in the article “What are scientific leaders? The introduction of a normalized impact factor”, which is available on the service network, arXiv.org, a website with pre-prints from various fields of physics, mathematics, non-linear sciences, computer science and quantitative biology. The text has been submitted to the journal, Scientometrics, a point of reference in scientometry, the discipline that seeks to generate information to overcome the challenges of science, but it is still in the analysis process. It has, however, had a good reception from specialists and has helped fuel the eternal debate about the most reliable way of evaluating academic production.
For Rogério Meneghini, a specialist in scientometry and scientific director of SciELO (Scientific Electronic Library Online), Matsas’ proposal is innovative and has the potential to be used. According to Meneghini the NIF index, in general, is an improvement over the so-called h index, a method proposed by physicist Jorge Hirsch, from the University of California, in San Diego, in 2005, and which has been widely used by researchers, worldwide. “With the h index becoming popular several researchers have been suggesting ways of improving it, or avoiding its distortions”, says Meneghini. “Matsas’ proposal, to a certain extent, adds to this effort.” Easy to calculate, the h index combines productivity and impact and is defined as the “h” number of pieces of work that have at least “h” number of mentions for each one. In short: a researcher with a 30 h index is one who has published 30 scientific articles, each one of which has received at least 30 mentions in other works. The weighting excludes less quoted works and also avoids mentions in an author’s only article contaminating the overall count. So it provides the measure of the size and impact of a researcher’s academic production.
One of the advantages of the NIF is that, unlike the h index, it is little influenced by self-mention. A researcher who, in writing an article, quotes several of his own previous articles easily manages to raise his h index, without this necessarily qualifying him as a scientific leader. The NIF has an antidote to this, because self-mentions would be computed at the same time in the numerator, as a mention received, and in the denominator as a mention made, thereby minimizing its influence.
Edgar Dutra Zanotto, a professor from the Department of Materials Engineering at the Federal University of São Carlos (UFSCar), also praises Matsas’ proposal. “I think it goes in the right direction. It’s a universal and robust value, which manages to distinguish the leaders from the non-leaders”, says Zanotto, who two years ago proposed another method for identifying academic leaders. This was published in the Scientometrics journal and was based on the sum of objective factors, such as publications and mentions in indexed journals, qualitative factors, like the capacity for raising funds, the fact of working in internationally renowned centers or winning important prizes (see Pesquisa FAPESP nº 124).
Weighting
For Zanotto, the main quality of the NIF is that it supplies a standard reference value for various scientific communities, regardless of their traditions and customs. “Each area of knowledge has a number of particular mentions. If an article in the mathematics area mentions 6 other articles, and one from biochemistry mentions 30 or 40, this does not interfere in the NIF, because the weighting is always going to be based on a reference value equal to 1”, says the professor from UFSCar. The h index, on the other hand, is influenced by this distortion: leaders from different areas have h indices on different levels and cannot be compared. “This doesn’t mean that the mathematics community doesn’t have its leaders, or that biochemists are on average more competent that their other colleagues”, explains George Matsas.
To validate the methodology, on the ISI Web of Science database the Unesp researcher examined 223 physicists selected from a list of 531 individuals who were recognized in 2008 as outstanding referees by the American Physics Society (APS). Of the 223 researchers analyzed 31% of them were considered leaders and 69% followers, since their NIF was less than 1.
The NIF concept, of course, also has its limitations. The method is not applicable to human sciences, the academic production of which is not usually expressed in internationally indexed journals, but in books or book chapters, to mention just two examples. The method only produces reliable results in senior researchers, since researchers starting out on their careers have an NIF that is influenced by the academic production of their tutors or superiors, with whom they sign the articles.
Matsas admits that if his method was adopted by the scientific community it could change the behavior of researchers interested in raising their NIF, as happened with the h index. “A way of avoiding the NIF going down would be to try and not mention other articles. But there would be a limit to this. The editor of an academic journal would not accept any article that failed to make quotations, which are indispensable”, says the professor from Unesp. A positive aspect, according to Zanotto, is that it is highly unlikely that a researcher would make unnecessary mentions – those that are made merely to curry favor with his peers.
The creator of the method points out that its intention is not to make any individual judgments, but to assess the strength of the scientific communities in order to adjust scientific policies. “If we apply the method to Brazilian communities we’ll probably discover that the number of leaders is not as large as the robust increase in academic production would lead us to suppose”, he says. But this type of reference would help the formulators of policies to favor the formation of leaders. “We need more leaders. We want foreigners to come and work on the projects of Brazilians. Brazil has taken some positive steps. National science is increasingly strong, but it’s lacking that decisive step: we need policies that create more chances for Brazilian research to become a protagonist on the international stage.”
Republish