Self deception and academic honesty

The scientific method is a powerful paradigm for finding the truth. Still, it is very easy to get fooled, especially in fields where the data is noisy or the samples are unreliable. Polymer physics is one of these fields.

A polymer is like a bowl of spaghetti with intertwined strands of varying lengths. Between the strands are spaces, whose sizes and shapes are randomly distributed. The distribution of these voids can be mathematically modeled by what is called, not surprizingly, a distribution function.

The distribution function depends on the sample's history and how it was originally made. Imagine a bowl of hot steaming pasta that is slowly cooled until it forms a solid rubbery clump. The resulting structure will be relatively compact with few air spaces. On the other hand, toss the hot pasta in the air and let it land in an utra-cold bowl so that it freezes before it is able to settle down. In this case, one will find gaps of various sizes between the strands. Similarly, the properties of a polymer will depend on how it is made as well as its thermal history. Thus, even when two samples are made the same way, if one of them is subsequently heated and cooled, it can change its properties dramatically. To make things worse, polymer samples are notorious for not having the same properties from place to place within the material. This is called inhomogeneity.

Thus, when a measurement does not give an expected result, it is easy to attribute it to inhomogeneity or to issues with how the sample was made. In our lab, we try to reproduce the same measurements many times on different samples to make sure that we are not being fooled by an anomaly. However, when doing huge numbers of measurements on materials that are known to vary from sample to sample, it is far too easy to keep the ones that we believe are right and to throw away the data that does not conform to our expectations.

As I have mentioned in numerous posts in the past, our lab discovered many years ago that some polymeric materials self heal after being burned with a laser. This work was the topic of a masters thesis and a Ph.D. dissertation. Presently, there are two students working in my lab on this project under funding from the U.S. Air Force. After doing a series of very careful experiments, one of my students noticed a pattern that suggested that the polymers might not be self healing.

My initial response was that we had to verify the result. I suggested that the student try several variations of the experiment so that we could narrow down the problem, and indeed, we found that in the redesign of the experiment a year ago, the method of insuring that the laser was stable had been redesigned with a subtle flaw that allowed the laser power to slightly drift without detection. What appeared to be self healing was due to a variation of the laser power. When the laser power happened to be stable, we saw no self healing, which we attributed to a bad sample, and when the laser power varied in just the right way, the sample appeared to heal. Once we had taken all this into account in the whole set of data, self healing was no longer being observed.

This lead my mind racing, going over all our past results, wondering if it was possible that we had been fooling ourselves all along. I was picturing the humiliation of informing the funding agencies of our grand error and returning the unspent funds. As a result, I would lose support of two graduate students and a month of summer salary for myself. But, my greatest concern was for the student who had spent two years of his life working on the project -- being left with no dissertation. The thought was most devastating in light of the fact that he is one of the best students in our department, both bright and hardworking.

After settling myself, I began to carefully assess our body of work. The fact that so many independent experiments confirmed self healing (using other apparatus without this problem or totally different techniques), made me convinced that this was a temporary setback. Thinking it through, I realized that another change that we had made was in the way we were preparing our samples, so I had the student fix his experiment to properly stabilize the laser, and to dig up the older samples for experimentation.

Last evening, I got the excited email that he was again observing self healing.

This incident reminds me of how science is a dynamic and churning process; where mistakes are made and corrected. Like a fractal, this dynamic is found at the level of the individual, who adjusts to problems on a daily basis, and in the scientific community as a whole, where the work of many labs and scientists takes a seemingly random walk that eventually zooms in on the truth.

Science is like an adventure. If the path were well defined, not only would the process be dull, but the whole enterprise would be worthless. Too often, the public and even university administrators believe that discovery is a linear process. At our university, we are required to write a memo each year requesting an extension for a student if he or she has been a PhD candidate for more than 3 years. And, we are faulted for students who take a longer time to graduate. This mentality implies that the PhD degree is the result of a fixed amount effort and does not take into account the unexpected that scientists have learned to expect.

I have no idea where my scientific adventure will take me next, but I know that it will exceed my expectations. More important to me are the trials and tribulations of the process of discovery. It is truly a most exciting adventure.