Illogical Scientists

I know that I am often illogical about lots of things; it's a trait that makes us human. However, this does not excuse sloppy thinking by scientists. Scientists often complain about the innumeracy of the general population, but we are the most egregious offenders by associating undue meaning to the numbers, such as our reliance on the h-index to quantify the performance of an individual researcher with a single number.

The h-index is a measure based on citations to a researcher's work. The Wikipedia page on the h-index gives a description of how it is computed, what it purportedly measures, and criticisms of its use. No one would disagree that there are many factors that affect the h-index, making it a highly inaccurate metric. So, why do so many people use it? I believe that it is sheer intellectual laziness. What could be simpler than using a single number for ranking a group of individuals?

One of my beefs with the h-index is that it favors scientists that manage large research groups that publish lots of papers and produce lots of PhD s who then get jobs doing similar work and generate more citations for their research adviser. Is it healthy to reward good managers of science for being good scientists? Perhaps.

Scientists are well aware of the importance of their own h-index on career advancement. It is all too common for reviewers to point out that some important references are missing in a manuscript. As you might guess, reviewers often push for their own publications to be cited. Placing such a high degree of importance on a single indicator has a perverse effect on the scientific enterprise.

Perhaps I am selfish, but I live to think and to produce new ideas. It makes me happy. I want to be in the trenches, scribbling equations with my mechanical pencil on smooth white sheets of paper and tinkering in the lab with neat equipment that I have designed and build with my own hands - all in the company of students who are excitedly doing the same. I understand that there needs to be a balance between managing a lab and doing the work. A PI who spends all the time working in the lab is not passing along his or her expertise to others. It is also inefficient to be involved with all the mundane activities that go along with doing research. More ideas can be investigated when the students have a large degree of autonomy, so while difficult, I try to keep a healthy distance.

On the other extreme is the professor who writes lots of huge grant proposals and uses his or her mega funding to hire an army of postdocs who generate a large chunk of the ideas, advise the graduate students, and implement the work. Such an individual is doing a service to society by concentrating a critical mass of intellectual capital to solve important problems. It is an efficient division of labor where grantsmanship brings in the funds and the most capable people do the work.

I want to make it clear that I am not criticizing the big research groups. They provide an important service to the scientific community of producing the next generation of scientists and adding substantially to the knowledge base. Given that some of my work is amenable to large-scale science, I often question my decision to limit the size of my research group. Am I letting down my employer by bringing in a few hundred thousand dollars of funding per year rather than millions? My decision to operate on a smaller scale is based on my conviction that my science produces value to both the university and to society. Arguably, my group produces more publications and more citations per research dollars than many others. Imagine if a researcher's productivity were measured in citations per dollar of funding.

If truth be told, I would be dissatisfied working in a field with hundreds of researchers who are working on a tiny piece of a puzzle. Rather, I derive satisfaction from thinking about things from a unique and broad perspective. As a result, I have far fewer colleagues doing similar work and my students, as a result, will have fewer job opportunities - unless our work leads to a breakthrough. But, I justify our work in my conviction that such work has the potential for making a large impact in the long term.

However, I am concerned that the emphasis on large research groups garnering huge research grants will squeeze out the smaller groups, where breakthroughs in new thinking are usually generated. Given the present-day economic challenges, researchers with larger grants will undoubtedly be held in even higher regard by their cash-strapped administration.

A second perversion of the scientific enterprise is recognition for simply publishing in a prestigious journal. The quality of a paper should be judged on its own merits, not by its venue. The viscous cycle is reinforced by "me too" papers that cite high-visibility journals to imply that the author's work is in an important research area.

Consider the work of Victor Veselago who in 1967 investigated the properties of materials with a negative refractive index. His work got little attention (and few citations for decades) until the recent explosion of research on meta materials, results of which are routinely published in Nature and in Science with sleek color graphics and exciting titles. Amongst the present-day superstars are their lesser-known brethren such as Veselago who are laying the foundations for future revolutions.

Recently, the program director of an agency that funds my work sent me the following email,

I am collecting information to prepare for my upcoming internal review and I am collecting interesting accomplishments for highlights. This request does not substitute the formal annual progress report specified in the grant document that is due at your grant anniversary date of each year. Please provide a (no more than) one page summary of the significant accomplishments and publications (such as Science or Nature publications) in your program...

Clearly, this program director deems it necessary to justify to his superiors his funding choices not necessarily based on the quality of the work, but on where the work is published. What I find more distressing is that scientists in increasing numbers believe that a publication in these high profile journals carries greater importance.

Many years ago, one of my colleagues suggested that I should package my work in a way to make it publishable in Science or Nature. I did not chose science as a career to focus on marketing. Rather, I am motivated on a daily basis by the promise of understanding something new about nature and to share my understanding with others who are similarly driven. Nothing beats the eager faces in the classroom, arguing with me about my rendition of nature's hidden treasures. Nor the excited chatter of my graduate students as they report on new results or insights. Even day-to-day difficulties bring to light the pleasures of seeking the truth. At the end of the day, the negative aspects of the scientific culture fade into the background, where they belong. But, on occasion, my bliss is interrupted by that unpleasant pang of realization that scientists hold positions that are not supported by reason. We should know better. Shame on us.