Researchers who follow the progress of scientific production in their fields know that good papers do not always have an instant impact. It is not unusual for innovative ideas to languish for some time before their importance is recognized. Winners of the Nobel Prize frequently receive the award for contributions made many years – and sometimes decades – earlier, just as applications based on well-known concepts may gain significance unexpectedly. Experts in scientometrics, the branch of science that studies the quantitative aspects of the production of knowledge, have coined the term “sleeping beauties” to describe articles that arouse interest years or even decades after they were first published. They have begun to study these articles as expressions of a delayed recognition of scientific production.
One famous—and extreme—case was that of American virologist Francis Peyton Rous, who in 1911 published an article demonstrating that some types of skin cancer observed in birds were caused by an RNA virus, one of the retroviruses. The importance of Rous’s work did not become apparent until 1951, after a leukemia virus had been isolated. That accomplishment marked the beginning of the association between cancer and infections caused by viruses. In 1966, Rous was awarded the Nobel Prize in Medicine. Similar occurrences have been analyzed in studies that attempt to understand the nature of sleeping beauties and identify the factors that help awaken them.
Physicist Anthony Van Raan, a researcher at Leiden University in the Netherlands, says that one must keep in mind that sleeping beauties with the potential to bring about paradigm shifts are fairly rare, which makes identifying them a complicated task. “The great majority of articles that pass unnoticed will remain so forever, simply because they are not interesting,” says Raan, who was the first to coin the term “sleeping beauties” to refer to papers that took time to be recognized and make an impact. Van Raan’s most recent works are intended to identify the “princes” responsible for breaking the spell and sparking an interest in long-dormant articles.
In an article published in February 2017 in the journal Scientometrics, Raan showed that in the field of physics, 16% of the sleeping beauties indexed in the Web of Science were awakened when they were mentioned in patents. He also observed that the average time lag between the publication year of a sleeping beauty and its first citation in a patent appeared to decrease in the early 1990s. “That would mean that sleeping beauties having technological importance, perhaps potential inventions, are being discovered increasingly earlier,” Raan suggests. He says it is common for good articles that present concepts or technologies that are ahead of their time to go unnoticed. In 1958, for example, an article was published that described an efficient way to obtain graphite oxide on a large scale. Citations of the study did not begin until 2007, when it was discovered that graphite oxide could be useful in obtaining graphene on an industrial scale. Graphene is an extremely hard and malleable material described as a carbon sheet with thickness at the atomic level and possessing electrical, mechanical and optical properties.
Physicist Ado Jório de Vasconcelos, a professor at the Federal University of Minas Gerais (UFMG), published a paper in 2002 in which he described the application of a technique known as Raman spectroscopy in the identification of the properties of carbon nanotubes, considered to be good conductors of heat. “The article did not begin to be cited actively until 2010, when the scientific community began to attach importance to the study of the Kohn anomaly, a characteristic of the vibration in atomic nuclei that are coupled with electrons. That phenomenon was known to appear in metallic materials. My work had already shown that it was also a characteristic of nanotubes,” says Jório, who in 2016 was included on the list of the “3,000 most influential scientists in the world,” published by Thomson Reuters.
Evaluation
Studies about the delayed recognition of papers also seek to perfect the evaluation systems employed in science, many of which are based on indicators that appear to favor papers whose impact is felt soon after publication. A paper published in April 2017 in the journal Nature suggests that scientific articles that made transformative contributions, even if they did not strictly fit the definition of sleeping beauties, usually took more time to produce repercussions than did articles that took incremental steps forward. Jian Wang, a researcher at The Catholic University of Leuven (KU Leuven) in Belgium and one of the authors of the study, told Pesquisa FAPESP, “We observed that truly innovative research studies did not receive citations until after a long period starting seven years after publication.” The survey concluded that bibliometric indicators that employ a citation period of only three years are clearly inefficient for evaluating research whose results need time in order to be understood.
NegreirosWang and his team examined the citations of more than 660,000 articles published in 2001 in all the fields of knowledge indexed in the Web of Science database. They found that 89% of manuscripts featured only a low degree of innovation. In order to determine which articles would be considered innovative, they selected papers that presented unusual bibliographic references, combining authors and fields of knowledge differently than had been the pattern in each field. “One way to verify whether an article contains new ideas and concepts is to look at its ability to combine different bibliographic references in an unprecedented fashion. That characteristic may point to those papers in which the researcher took more risks,” Wang explains.
It was found that during the first three years after publication, the likelihood that a very innovative article would be found among the 1% most frequently cited papers was lower than the probability ascribed to the other papers. According to the survey, papers that received many citations soon after publication and during the first three years tended to become obsolete. “But those that were considered disruptive, that introduced a high degree of novelty, accounted for 60% of the most-cited articles 15 years after publication,” Wang explains. He concludes that although funding agencies highlight the importance of investing in research of a transformative nature, their evaluation systems end up favoring incremental studies because they use the more popular indicators of impact. “Funding agencies’ and reviewers’ widespread use of parameters such as number of citations may discourage research efforts that have a potential for overturning paradigms,” Wang suggests.
As examples of agencies that make some use of bibliometric indicators in their evaluation procedures, the survey mentions the European Research Council (ERC), the National Natural Science Foundation of China (NSFC), the U.S. National Science Foundation (NSF), and the Brazilian Federal Agency for the Support and Evaluation of Graduate Education (Capes), which established the QUALIS system for classifying Brazilian scientific journals. Rita Barradas Barata, director of evaluation at Capes, explains that in order to supervise the approximately 4,200 graduate study programs in Brazil, the institution monitors scientific production by faculty and students. “Since it is impossible to measure the quality of each of the more than 800,000 articles produced by those programs, we designed a classification of the vehicles in which the works are published,” she reports. To that end, consideration is given to various indicator disciplines such as the Journal Impact Factor (JIF), identified in Wang’s study as one of the tools that disadvantages articles that take a long time to be recognized.
The scientific community is often shocked by ideas outside the mainstream, says Paulo Artaxo
Barata acknowledges that institutions have become accustomed to concentrating their attention on articles that are cited many times within a short interval. “There is a tendency to train the eye toward the ones that bibliometric indicators say are good at the time.” One idea being discussed, according to Barata, is that research agencies and institutions could adopt the practice of “prospecting” to unearth topics whose potential impact may be underestimated. In Wang’s opinion, agencies do not need to look for ways to favor researchers who are not often cited. “It’s enough to judge each proposal on its own merits, which is hard to do. Peer evaluation systems are a good counterpoint to the excessive use of metrics,” he says.
Impact factor
In a study published in 2015 in the journal PNAS, researchers from Indiana University in the United States analyzed 22 million papers published over a period of 100 years that were indexed in the archives of the American Physical Society and the Web of Science. They found that the bulk of the articles that remained dormant for long periods of time but then became celebrated in their respective fields dealt with chemistry, physics, and statistics. The study by Indiana researchers calls attention to the fact that the very concept of an “impact factor” remained hidden for years in an article published in 1955 by Eugene Garfield. In that article, Garfield, who died in February 2017, presents ideas and concepts that would later be used to consolidate the Web of Science database maintained by Thomson Reuters. “The paper had been dormant for almost 50 years until it became popular in the early 2000s and was cited in works on bibliometrics, some of them by Garfield himself,” the study reports.
Van Raan explains that although they are concentrated more in the exact sciences, sleeping beauties can be found in practically all areas of knowledge. “I am beginning to investigate medical fields as well as the social sciences and hope to discover some interesting things,” says Raan, who is counting on the development of software that would be able to identify the sleeping beauties of science.
Paulo Artaxo, a professor at the Physics Institute of the University of São Paulo (USP), sees it as natural that good articles do not receive proper recognition at first glance. “Research projects involving ideas well outside the mainstream take time to be digested by the scientific community, which often tends to be shocked by new ideas and may even exhibit some prejudice,” he explains. He says that studies that try to analyze sleeping beauties may play an important role. “They may provide us with paths to an understanding of why big ideas go unnoticed. This is an opportunity for us to alert publishers to come up with strategies to make articles more visible and readable, because today there is an excessive degree of specialization in these research papers, which makes them hard to understand even for someone who is versed in that same field of research,” Artaxo observes. However, Ado Jório argues that it is up to the authors of the research papers to make an effort to publicize them, especially when they know they are proposing something that conflicts with the prevailing paradigm. “It’s not enough to publish an article and pray that it is read, understood and cited.” He recommends that researchers attend conferences, lectures and debates and continually look for occasions to talk about their research with people who may be interested in it.
Republish