How the Manuscript Review Process Should Work

The review process provides a level of quality control that insures that published papers are correct and of interest to the scientific community.  Since reviewers are themselves scientists with busy schedules, and being a reviewer provides no compensation aside from the satisfaction of being a good citizen, editors often find it difficult to get the best people for the job.  This has led to an increase in desk rejects by the editor, which avoids wasting reviewers' time on manuscripts that will most likely not be accepted.  The review system is frustrating to all parties involved.

I've been involved on all sides.  As an editor, I took lots of grief from angry authors.  In one case, I got a phone call from an irate author whose paper I had rejected.  He lectured me that as an editor of a prestigious journal decades prior, he would use at least one of the reviewers that the author had recommended.  Why had I not done so?  Because the review process is anonymous, I could not tell him that I used two of the three physicists that he had suggested, and they both recommended that the paper be rejected.  As a compromise, I selected his third choice of reviewer, along with yet another one.  Again, they all rejected the paper.  I could have avoided the next phone call and the indigestion that followed if I would have disclosed the fact that I had chosen at least one of his recommendations.  But I could not.

On another occasion, one of my colleges refused to act as a reviewer on a paper for which he was perfectly suitable.  That same colleague had an issue with one of his papers (not to my journal) where he needed my help, so I used it as leverage to get him to act as a reviewer for me.  There are all sorts of behind-the-scenes dynamics that are not always obvious to authors or reviewers.  The bottom line is that the process is far from perfect, which brings disdain from authors who are unhappy with the results.

I am writing this post to describe an example of a rainbow amidst the storm.

The American Journal of Physics is one of the coolest Physics publications on earth, so I read every monthly issue cover-to-cover.  There are always a few articles in each issue that surprise and delight.  They often point out subtleties in topics that you might think mundane, and bring insights that have been missed by the research community.

Being an author of a couple papers in AJP over the last two years, I have found the review process to be excellent.  The reviewers are knowledgeable and seem to spend lots of time trying to understand the work.  The exchanges are a real learning experience, and all parties are flexible -- admitting mistakes and savoring the process.  Here I describe an example of my experience with a paper that will be appearing in December (here is a link to the preprint).

The editor notified me that my paper had mixed reviews:

"Attached you will find copies of the reviewers' reports on your manuscript "Length as a Paradigm for Understanding the Classical Limit." Though the reports differ in recommendation, it is the content of the reports that is more important than the recommendation per se, and all three reviewers seem to be focusing on the same (or almost the same) issue: The justifiability of your model for what you call length. It will be necessary for you to address this issue in a revision. There are additional detailed corrections and suggestions that should also be carefully in a revision.

If you wish to revise your manuscript along the lines indicated, we would continue its editorial consideration once it has been resubmitted using the procedure indicated on the AJP website. If you do resubmit, please indicate in a single cover letter how you have responded to the various comments of the reviewers. DO NOT send separate replies for each reviewer.

Thank you for your interest in the American Journal of Physics."

The first thing that caught my eye was the fact that this was not the usual form letter used by most journals.  The editor had carefully read the reviews and noticed some common criticisms.  I eagerly dove into my revisions, finding that the reviewers' questions and confusion were the result of deficits in my paper.  I knew what I was trying to say, but my obtuse presentation of the material was only made obvious by their comments.  Most importantly, responding to the reviewers forced me to think more clearly about the physics.  As a result, I gained insights into my own work when making the revisions, increasing substantially the quality of presentation.

Exchange With Reviewers 

Particle in a Box

I will focus here on the common complaint made by all the reviewers, which centered on my use of the particle-in-a-box model.  Here are their complaints:

REVIEWER #1

My only concern is the basic assumption used to model the ``quantum'' systems.  The author is assuming the quantum system is inside a box with hard walls (it uses the infinite well model to derive the wave-function of one electron, and then generalizes it to non-interacting particles).  While this is a simple an intuitive model to work with, it is not clear that the results would carry on with more realistic potentials that might have different boundary conditions. 

REVIEWER #2

In the work, the focus lies on the electrons of a material and the nuclei are taken as a scaffolding for them, which is the usual approach for solid state physics. Yet, the wavefunction for the entire ruler will truly include the atoms as well. I would expect that the length of the ruler as calculated through this procedure will change when including these additional fermions, while I reality the length of the rod will remain the same. If this is so, it would jeopardize the numerical results in this work (although not its method)

REVIEWER #3

It is also strange the definition of the length of an object as the sum of the densities of the electrons confined in that potential, like the example of the particle in the box. The length of a real material should depend on the properties of the nuclei as well as of the electrons.

My Response:

Since all the reviewers brought up a similar point, I responded to them as a group.  Here is the verbatim response:

We interact with materials either by looking at them with our eyes (light scattering from electrons) or touching them (repulsion between electrons in the material and within us).  I believe that we all would agree that what we ``see" are the electrons, though their density does indeed depend on the presence of the nucleons.  So two materials with the same electron density but different nuclear positions would appear the same.  The mass, of course, is dominated by the nucleons, but we are viewing/touching the electrons when determining the length.  The length would be different if we did scattering experiments that are tuned to only probe the nucleons, but on the scale of human senses, we see only the electrons.  In either case, the lengths determined in these two ways would be similar for multi-atom quantum systems.

The positions of the nucleons are well represented by the Born-Oppenheimer Approximation, where the nuclear equilibrium positions are determined by the configuration with the lowest total energy.  Some of the electrons are involved in stabilizing the system -- which can be viewed as chemical bonds -- and in materials such as metals, the rest of the electrons are delocalized over the bulk material.  In the absence of defects, bulk metals appear smooth so each conduction electron moves approximately freely within the metal and encounters a large barrier at the edges.

This picture roughly holds for all materials and uniform electron densities are found is a variety of systems with delocalized electrons modeled by particles in a box.  These include small molecules such as the polyenes, as modeled by Kuhn in the 1940s, to metals as described in solid state textbooks, to nanoparticles that straddle the classical/quantum divide as recently reported by Scholl.

I would thus argue that the particle in a box is a good model that roughly holds well for many systems.  Much more sophistication is required to deal with the nuances.  I therefore believe that using the particle in a box model contains the correct physics that is assessable to a student.   Taking into account the reviewers' comments to explain this to the reader, I have added a third paragraph to Section III.A that reads:
 

"Models of materials with non-interacting electrons in a box roughly predict the electronic properties of small molecules such as the molecular class of polyenes,\cite{kuhn48.01} describe metals as found in solid state textbooks\cite{OpenStax20.01} and accurately portray the quantum to classical transition of nano-particles.\cite{schol12.01}  This shows that the effect of the nucleons on the electrons can be roughly taken into account with a box that confines the elections within.  We will thus model typical materials with uniform electron density as non-interacting electrons in a box.  The reader should keep in mind that this is a first step in modelling materials in which electrons are delocalized.  Later we treat materials made of such units that are ``pressed" together.  Then, the electrons are localized within domains rather than over the full material.  For simplicity, we will treat only one-dimensional systems.  Other potentials can be treated in the same way, but this exercise does not result in significant-enough insights about length itself  to make it worthwhile to treat in this paper."

The reviewers we satisfied with my response.  Here are excerpts from their second set of comments:

REVIEWER #2 - Second Response

I'm particularly satisfied with the argument to not include the
nucleons in the the total wavefunction. I've learned a new insight
here, using the Born-Oppenheimer approximation. Also, I think the
added paragraphs to section III add to the quality of the story.

REVIEWER #3 - Second Response

The author presented an improved manuscript that discusses the
difficult concept of quantum length. The language in the response
and the changes in the text greatly improved the manuscript.

Definition of Length

REVIEWER #3

This reviewer thought that my paper was wrong and should be rejected on the basis that I defined the length in a certain way that was arbitrary.  In the reviewer's own words:

It is my opinion that the manuscript is not technically correct as it starts with the definition of length of rod in terms of its uncertainty in the position.

 

The reviewer continues with technical details.  This comment led me to see that I was unclear in my presentation.  I responded with:

Your major criticism of this manuscript is with the ``definition of the length" and your argument against this definition is based on the fact that the coefficient $\sqrt{12}$ would change if the material were not uniform.  The original version treated only the uniform classical rod as the length element.  In the revised manuscript, an appendix describes how a non-uniform classical rod is treated, and is referred to in the main text.  Your argument is analogous to stating that the Pythagorean theorem can't be right because the expression would depend on the shape of the curve along the hypotenuse.  As with Pythagoras, where the length of the curve is obtained by dividing it into infinitesimal straight sections, so too the classical length is computed as the sum over uniform segments.  In retrospect, the original manuscript did a horrible job by neglecting this description.  I believe that using an appendix eliminates confusion yet maintains the flow of the narrative.

I have also added a couple paragraphs, as described above in the general section, which argues for the ansatz for the quantum length, a regime where it is no longer possible to subdivide a material without changing its properties.  I hope that these two major revisions remove confusion and makes you comfortable with the length expression that results from applying translational invariance and classical correspondence.

 

In response to my revisions, the reviewer adds: 

 

The added appendix lets the reader know that there are other
definitions that would converge to the proper classical limit. I
would like to see incorporated in the beginning of manuscript the
statement that the author made about how theories are developed.
This would be very helpful for the readership to explain how one
should approach making comparisons between new theories and their
classical limits. Incorporating these ideas on someone’s teaching
can help students navigate phenomena they are seeing for the first
time while relating them to things they are familiar with. These
ideas are incorporated in the Lessons Learned section, however,
helping the reader have this framework at the beginning of the
manuscript can guide the reader on understanding the assumptions
made, the development of the concepts, and finally understanding
the conclusions in the end.


It is my opinion that the manuscript might be published in the
present form, but it can still be improved by incorporating the
four points on theory developed in the introduction:


1. Setting the physical constraints. Here the length is required to
be translationally invariant and to give the correct classical
result.

2. Choosing the simplest ansatz that meets the constraints. The
uncertainty happens to meet the translational invariance criteria
but the length is NOT ad hoc defined as the uncertainty.

3. Demanding that the quantum theory in the classical limit gives
the classical result. Here, the quantum length and classical length
converge in the many-particle limit and for one particle in the
limit of it occupying the highest-energy state.

4. Investigating the Consequences. Here we apply the ansatz to
rulers and measurement.

In addition to being satisfied with my revisions, my comments to the reviewer led them to conclude that the response I had directed at the reviewer in my rebuttal was so useful that it should be added to the paper.  This was a great idea, and an example of what I was thinking when I was writing the paper, but something that I had not verbalized. This gave me the opportunity to carefully craft  what I think is an important takeaway message of my paper.

Conclusion

There are many other useful exchanges with the reviewers that would take too much time to summarize here and would add too much length to this post.  So instead, I have uploaded all the files with the reviewers' comments and my responses, to which I provide links below.  The end result is that I got great advice, which led to a much better paper not only in the presentation style, but in substantive additions to the content.  More importantly, I feel a deep kinship with these reviewers, who bared their minds to me in a frank dialog that gave me a more nuanced understanding of the topic.  I am indebted to these individuals who sacrificed their valuable time without compensation, other than to savor the satisfaction of learning about and understanding the subtleties of our world.

• Link to the reviewers' comments, my reply and their final comments.  Click here for the folder.
• Here is a link to the final version of the paper, which is scheduled to appear in December.

Meeting the Snob Factor

The best journals employ a snob factor as a first cut to limit the deluge of submitted manuscripts that go out for peer review.  The editor uses the "desk reject" for potential papers that don't look interesting.  Then, the reviewers are asked to evaluate a manuscript's significance to the field prior commenting on the technical details.  These two layers of subjective assessment can doom a manuscript, relegating it to a polite rejection: the work might be technically correct, but it is not of broad enough interest.

One such journal is Optics Letters, published by the Optical Society of America.  Though it is eclipsed by the new OSA journal Optica in its impact, it is still a highly selective and respectable publication.  Recently, we beat the odds by receiving an acceptance letter (subject to minor revision) along with the initial reviews.  The preprint of the paper can be viewed at https://arxiv.org/pdf/1809.01216.pdf

While the paper is based on some esoteric principles, it provides the experimentalist with a recipe for adding one state to the simple model commonly used in the field to correct for the infinite number of states that are omitted for bovious practical reasons.  This magical state is a proxy for those infinite numbers of states that are ignored.  The figure shows a plot corresponding to the uncorrected model (left) and the corrected one (right).  The nice smooth green background and the sharp red along the diagonal is the signature of success.  We thought it cool and useful that such a proxy state could fix a problem that has been plaguing nonlinear-optical measurements for decades.  For once, the editor and reviewers agree.

Here is a summary of the reviews:

Reviewer 1:

The manuscript represents an important advance in the calculation of nonlinear susceptibilities because it presents for the first time a method for dealing with the difficult continuum states present in realistic models of molecules. Ignoring these states leads, as the authors identify, to large errors in the calculations while, perhaps surprisingly, a single proxy state allows one to eliminate these errors to a large degree. This proxy state is not just a mathematical fudge, it is defined through physically measurable quantities. I therefore strongly recommend publication. 

Reviewer 2:

This is an interesting work discussing corrections to polarizability and hyperpolarizability calculations for limited state models that can be made using a single proxy state.  The conclusions are well supported by the calculations and this will find significant interest in its community.  

The nasty revewier strikes again - the third time is not a charm!

We sent a paper to a third journal and appear to have had the same reviewer again!  The review appears below.  Is it the same person?

The manuscript contains a  confusing and irrelevant approach to estimate maximal values that second and third order dipolar polarizabilities  can attain. The authors  also claim that this approach can be exploited to find   nonlinear  materials with optimal values for these coefficients without specifying  structural, chemical or other related  material characteristics: the Holy Grail in the quest of  nonlinear optical materials.

The approach is an extension of previous ones  with vaguely similar ingredients  and claims that  appeared in a series of publications  essentially by the same group and are extensively and exclusively referred  in the ms .  For a change this time the approach is disguised  with  “cartoons”   representing  “quantum graphs” (QG) and “star motifs”  that can be stressed and bent to any purpose with adjustable assumptions and parameters  to meet the authors  wishful  claims. These QG bear little, if any,  relation to chemical  structural  characteristics of the material as the usual quantum chemical  approaches  do and are far more complicated  to  estimate and guide the search for  nonlinear materials .

In a way  their approach is a disguised, unphysical  and complicated  version  of a  qualitative “assessment” of the nonlinear polarizabilities/susceptibilities based on  an expansion of the  induced  el-dipole/polarization in terms of the parameter  (E/Eat) where E is the el-field of the light  and Eat is an average  atomic(roughly the  ionization field) or cohesive el-field   of the atom (molecule)/solid.  This qualitative approach served  to qualitatively justify the use and range of the perturbation approach in powers of E and to also get a rough estimate of the susceptibilities   in the form of ?(n+1) = 1/(Eat)n; although the estimates  were  order of magnitude off  some  trends  were plausibly  accounted.  A short account of this approach is given in any respectable book on nonlinear optics (see  for instance introductory chapter in  Y.R. Shen,  The Principles of Nonlinear Optics, John Wiley). The present authors  in a cavalier manner  make no reference  to this approach  and  proceed  with  their complicated and useless to any purpose approach .

I shall accordingly not comment any longer on the inconsistencies of their  approach and  the  irrelevance of their quantum graphs  for  conceiving  nonlinear  materials  with optimal values for the second and third order coefficients. In fact the whole discussion in the ms proceeds with ill defined  terminology and unsubstantiated  vague statements. I do not recommend acceptance  of the present ms for publication in JOURNAL XXX.

August 31, 2013 update on the nasty review saga

The short update to the saga of the nasty reviewer is that another member of the editorial board reviewed our paper and it got rejected.  The fact that the paper was rejected is of minor significance compared with how the process was handled as well as the implications to my former students as well as to all my readers.  You could be next!

In my case, a rejected paper is a small delay in a long career, and its marginal impact small.  However, I have an obligation to make sure that the system remains fair, especially when my former students are being targeted, placing them at a disadvantage.

My letter below speaks for itself.  After reading it, please take the two-question survey by selecting the link below.  I spent a couple minutes designing the survey, so apologies for its unscientific design.  I am interested in getting feedback, either by a quick click of the survey page or by leaving a comment here on my Blog.  Thanks for reading and giving me all your input as a sanity check.  It is gratifying to see that a couple thousand of you have read these posts.

If you are so inclined, please pass a link to this page along to coworkers and colleagues.

Happy Labor Day weekend.  Today my students are coming over for a cookout.  We are wishing three of them the best of luck as they move on in their careers with their newly minted PhD's.  Good luck to them!

Take a web survey by clicking here.

Previous post as background, click here.










Dear Editor-in-chief,

I thank you for your time and effort on the issue of our manuscript.  Now that email communications between various authors, editors and staff have settled down, this is the time to calmly focus on the important issues that go beyond the fate of any one manuscript.  As I made clear in my first email, I wanted to decouple our paper from my broader complaint, but the large volume of chatter distracted us from this goal.

First is the issue of reviewer integrity.  It is inexcusable for a reviewer to recommend rejection of a paper on the basis of personal attacks.  This behavior is even more egregious when it is found that the same reviewer may have recommended that the paper of my former student be rejected, also without basis.  This brings up the issue of the integrity of the editorial process.

In the case of my student's paper in THE OTHER JOURNAL (I was not a coauthor), the editors immediately recognized that the review was inflammatory and technically baseless.  As a result, the editor discounted the review as unreliable and sent it to another reviewer.  In contrast, your editor concluded that our paper was wrong based on the vitriolic reviewer's assessment, totally discounting the positive review.  Even upon casual reading, a non-expert will conclude that the positive review was based on the technical contents of the paper and the other one was personal.

I am grateful that you have offered to act as the editor of our future papers that are submitted to your journal to insure that the process is fair; but, don't all authors have the right to a fair process?  The editor is responsible for upholding the integrity of the review process by mediating the discussions between the two parties, and I do not believe that this is possible unless that editor can publicly stand behind his or her decision.  Keeping confidential the identity of an editor who is deciding the fate of a manuscript, which appears to be your policy, is a bad idea.  Based on the tone of the editor's decision, many of us are under the impression that the reviewer is the editor or someone closely associated with the editor.  A more transparent system would avoid such suspicions.

Finally, the review process that followed our complaint was not objective in its execution, though I am sure that the editor who supplied the review did make an honest effort reading and understanding the work.  Discounting the nasty review, we have one positive review that recommends publication and a review from a board member, who made positive technical comments but felt that the paper was not suitable for Your Journal.  In reality, then, there are two positive reviews but a split decision of the suitability criteria.  The paper should have been sent to an independent reviewer, giving the editor two reviews to use in rendering a judgment.

Several questions remain unanswered.  First, the editors of THE OTHER JOURNAL informed me that they had contacted you to determine if the same reviewer was involved in both papers.  If so, this is irrefutable evidence that this individual has hijacked the review process for a personal vendetta and should be censored.  In the present model used by many physics journals, the reviewer's identity remains anonymous.  I believe that an individual who has repeatedly shown such extreme unethical behavior loses his or her right to anonymity so that future problems in other journals can be avoided.

I have a moral obligation to assure fairness to my past students, and I would hope to all young people who are starting new careers.  A lunatic reviewer who is out to sabotage the publications, grant proposals, and tenure packages of past students associated with a particular research group could ruin the career of a young person in a tenure track position.  While placing this reviewer on a do-not-use list is a good first step, it does not go far enough.  Reviewers should know that they must stick to the science when reviewing a paper and that there are consequences if they don't.

Secondly, we deserve closure on the actions of the first editor.  What steps have you taken to determine what failed in the editorial process and what safeguards are needed to prevent recurrence?  At minimum, I believe that the identity of the editor should be reveled and that his or her relationship with the reviewer disclosed.  In addition, we deserve an explanation from the editor of why he/she made the decision he/she made.  Assurances that all is well without disclosure perpetuates the problem.

This kind of behavior if ignored can seriously damage our field.  Unethical behavior by even a small number of individuals tarnishes our collective reputations, and being apathetic is inexcusable.  I have waited until the fate of our paper was determined to respond to your email to make it clear that I am not motivated by trying to get a paper published, but rather to contribute to the integrity of the process that I hope will benefit us all.  I was a topical editor at JOSA B for 5 years, and withstood the wrath of many angry authors and reviewers.  But in the end, I stood by my decisions not through a veil of secrecy, but through open discourse.  I would be comfortable if my editorial files were made public.  I hope that the editorial board at Your Journal can say the same in our case.

Please feel free to call upon me at any time.  I look forward to hearing from you in the near future.

Best,
Mark

SECOND BOARD MEMBER REPORT

The review below of the second board member is fair.  The individual clearly read the paper and knows about some to the topics.  Because the points are made clearly and logically, we would have been able to make a compelling case for publication.  Unfortunately, this review was not open for debate and was intended to close the door on this chapter of our paper.

I found this to be an interesting paper but I do not think it is suitable for Our Journal. Here is my full report.

The purpose of this paper is to compute the response to an applied electric field of electrons confined to connected one dimensional quantum wires under a wide variety of two dimensional geometries. The authors introduce the calculation in the context of nonlinear optics however it seems to me it would have been better to introduce the paper in the context of quantum chaos and periodic orbit quantisation.

The model is purely Hamiltonian. Real nonlinear optical  systems have dissipation and the approach of the paper ignores this entirely. Ignoring dissipation and dephasing assumes that the wires are in the ballistic regime in which only single particle states are excited in the conduction band.  This single particle regime ignores the possibility of Coulomb interactions resulting from multiple excitations and also phonon-electron scattering.   I assume this assumption is justified if the applied electric fields are kept weak? The authors should have made some comment on the physical applicability of the model.

Despite this idealisation the authors do obtain some highly non trivial results for optimal  second and third order susceptibilities using a novel methodology, he “star graph motif”.  I assume these optimal results must also bound what might be possible were dissipation included.

he analysis is very comprehensive, perhaps too comprehensive. One begins to loose sight of the key physical insights for example, that optimal response is usually associated with an effective three level structure. This is of course well known in nonlinear optics.  In the context of this paper I would have liked to have seen a connection made to the underlying periodic orbit quantisation: for example what are the conditions for which a three level model in quantum wires is appropriate?

There is also no attempt to link the results to possible experiments for example, the response of GaAs/AlGaAs confined wire structures to microwave driving.

In view of the rather restricted physical assumptions underpinning the model and the lack of any link to an experimental context, this paper is not suited to the Our Journal in my view. It is unlikely to be accessible to the broad readership of the physics community that Our Journal seeks to address. The authors should seek a more appropriate technical journal. Alternatively they could extract a  smaller more compact paper that addressed some key physical principles, such as links to periodic orbit quantisation or possible experiments in ballistic electronic systems with microwave driving.

Update on Nasty Review

My last post focused on the nasty reviews of our manuscript and my former student's paper at another journal.  Thanks for all of your comments.  The consensus is that the two nasty reviews were written by the same person and that the editor may indeed be the same reviewer.

I sent a letter on 8/1/13 to the journal editors of our paper, which is reproduced below, with all names removed, except mine:

Dear Editors,

I am writing to you in your capacity as editors of the JOURNAL X.  This communication is not an appeal of an editorial decision, but to report what I believe to be gross misconduct of an anonymous reviewer and an endorsement of this review by what appears to be an anonymous member of the editorial board.  I request that this complaint be kept separate from our manuscript file, which we are appealing through the normal channels.

One of the reviews borders on libel, stating "...they bend and distort basic procedures in perturbation theory and throw in out-of-context concepts and other spicy statements without the slightest concern for the basics and in fact the ethics," without supporting evidence for this serious allegation. All statements by this reviewer that are specific are totally off the mark and oblivious to the actual contents of the paper. More egregious is the statement from the anonymous editor that (s)he agrees with a review filled with personal attacks, flippant comments and unfounded accusations; and, chooses to ignore the positive review that accurately describes the technical details of the work.

A review should focus on pointing out specific scientific issues in a manuscript rather than being a ranting blog that attacks the reputation of the authors; and, the editors are responsible for upholding the integrity of the process. Shame on the editors and JOURNAL X for this unprofessional behavior.

I look forward to hearing the results of your investigation of this incident and the actions that you plan to take to avoid this unacceptable outcome in the future.

Sincerely,
Mark G. Kuzyk

As of the date of writing this post, I have not heard from the editors of this journal.  I next sent a letter to the editors of both journals on 8/5/13, shown below:

Dear Editors,

A potentially serious issue with the review process has come to my attention in which the evidence shows that a referee is rejecting papers based on the associations of the authors, independent of the quality of the science.  This does not appear to be an isolated incident, but is an intentional and systematic campaign to undermine the work of any researcher that is or has been associated with my research group.  Such inexcusable behavior undermines the integrity of the peer review process.

My former PhD student, Great Student, and I have noticed similarities between a reviewer's comments (attached) on one of his manuscripts, submitted to JOURNAL 1, and our paper (below), submitted to JOURNAL 2.  In particular, the referee(s) uses the same wording and makes similar complaints, attacking the character of the researchers without evidence.   Comments about the science are nonspecific, using colorful language and ranting rather than well reasoned arguments.  I have attached both reviews so that you can form your own conclusions.  At minimum, I would hope that the editors of both journals will share the identity of the reviewer(s) to determine if they are the same individual or related individuals, and if so, to take appropriate action to prevent recurrence.

A referee that systematically undermines the review process needs to be censured to ensure that this type of behavior does not become common practice.  I am deeply disturbed that such reviews passed through the editorial process.  I await the results of your investigation.

Note that I sent a letter to the editors of the JOURNAL 2 on Friday before Dr. Great Student alerted me to the review of his JOURNAL 1 paper.

Sincerely,
Mark G. Kuzyk

On 8/9/13, I got a letter from the editors of the journal that is considering the paper of my former student and junior colleague.

Dear Dr. Kuzyk,

I am writing in response to your recent email regarding the referee report on XXXXXXX "Great Paper" by Great Student et al.

Let me first thank you for bringing this issue to my attention. Of course you know, the referee process can only really function if editors, authors and referees all act professionally and in good faith. A breakdown of that professionalism is of course very disturbing and warrants our immediate attention.

I feel strongly that an author has the right to see all reports on his/her manuscript. Regardless of what an editor might think of a report, it is still information that plays into the editor's decision on a manuscript, whether consciously or subconsciously. Therefore, I do feel very strongly that no report should be suppressed 100%, although I have edited reports in the past to remove inflammatory statements. In this particular case, there was no way to edit the
report. However, the quasi-form letter I sent that accompanied the referee report did indicate that a new referee would be consulted. As you hopefully appreciate, editors are not "vote counters" but try to carefully weigh all evidence before rendering judgment on a manuscript and will often essentially discount a report (as I am willing to say I will in this case).

With regards to this particular report, I am in agreement with you that the tone of the reports are similar, as well as the language. I have written an email to be sent to the editors of the OTHER JOURNAL, sharing the referee report on XXXXX and your recent email, and requesting the name of that referee. Please rest assured that I will investigate this situation and, if I suspect unethical behavior, I will pass this up to the appropriate people within ORGANIZATION XXX.

Again, thank you for bringing this unpleasant situation to my attention.

Sincerely,
Reasonable Editor

I responded that same day, as reproduced below:

Dear Dr. Editor,
I fully agree with your analysis and the course of action that you are taking.  This gives me complete confidence that  JOURNAL 1 will continue to be a journal that I can trust will treat all manuscripts fairly and make decisions based on the science.
I thank you for taking quick action in this matter and look forward to its resolution.
Sincerely,
Mark G. Kuzyk

I believe that the silence of the of the journal that rejected our paper based on the nasty reviewer speaks volumes about that organization.  Rather than immediately acknowledging that there is a problem and to let us know that they are investigating the procedure that lead to this unacceptable outcome, they appear to be stalling while deciding how to save face.  I hope that I am wrong.

As usual, I welcome your opinions.

A refreshing review of our new paper on a model of self healing

There are times when one of my less stellar papers, in my opinion, gets accepted for publication without trouble; and, at other times, what I think are very significant papers have lots of trouble.  Perhaps this is a matter of taste, or perhaps papers that are outside the norm are misunderstood and therefore rejected.

As a case in point, I was invited to submit a paper to a special issue of a journal dedicated to self-healing polymers.  The focus of this special issue is on polymers that are made to self heal after mechanical cracking by incorporating tiny reserves of monomer that runs into cracks as they form, thus filling them and repairing the material.

Our work is very different, so I thought that our invitation was an effort by the editors to broaden the scope of the journal.  However, we were shocked to find that our paper was rejected without review based on the assessment that the work was "incremental."  We responded to the editors, reminding them that we were invited to write the paper.  The next email informed us that the editors had made an error,and that the paper would go out for review.

Two reviewers responded and one of them recommended that our paper be rejected on the grounds that the work was incremental.  However, the editor gave us the opportunity to respond.

In the meantime, we had found data at the extremes that were inconsistent with our model.  We fixed the model with one simple change in the underlying assumptions, and the new model fit all of our data. (This in itself is a very interesting story which I will report on later.) We revised the manuscript to include the new data and resubmitted it.

The editors sought an opinion from a forth reviewer.  An excerpt from his/her comments, follow,"One of the reviewers evidently commented that this work is incremental. However, I don’t agree. The authors are clearly refining their model, and this is an entirely new set of data and observations. The authors search for a better physical understanding will naturally require a significant amount of investigation, and it is helpful to the community to see the work as it unfolds, not wait 25 years for a definitive explain‐all paper that may never appear."

We, of course, agree with this reviewer, and are glad that this paper came to a happy ending, especially in light of the fact that I believe that our new results provide important insights that are taking us a step closer to understanding a new phenomena. 

The new version was accepted with minor revisions suggested, which we made.  The paper was then accepted and the page proofs arrived a couple weeks ago.  We fixed minor typos and now the paper is in the queue for publication in the early summer.  I ust learned today that the electronic version is already available online.  It's ironic that we were even invited to provide an artistic rendition of a figure that might be used as a cover photo.  What a difference a revision makes!  From incremental to cover story material with the change of one variable!

For the interested reader, below is the introductory paragraph, which describes our work and how it differs from the norm, "Structural damage and degradation of a polymer is usually associated with cracking. Mitigating damage or developing methods to promote healing in polymeric materials after cracking is an active area of research motivated by its practical utility. White and coworkers reported on a structural polymeric material with the ability to autonomically self-repair cracks. Such polymers incorporate a microencapsulated healing agent that is released in the cracking process with polymerization being triggered by contact of a catalyst with the healing agent, thus bonding the crack faces. White observed as much as 75% recovery in toughness. 

"Our work presented here is different in two regards. First, the sample is a dye-doped polymer rather than a neat polymer and the degradation process is through optically induced burning, so chemical changes are induced rather than solely mechanical/structural damage – though cracking can accompany burning. The dopant molecules thus mediate the phenomena. The degree of damage is observed using optical techniques, the simplest of which is the detection of a color change. Secondly, the healing process is a microscopic one, originating at a molecular level that we believe involves a cooperative process of aggregates of molecules. The polymers of interest to our work have applications as optical materials where photodegradation is a common cause of optical and optoelectronic device failures, either as catastrophic failure or a slow deterioration of performance."

The final paragraph in the conclusion succinctly states what we believe is cool about our work, as follows, "The concept that a material would exhibit such complex behavior without intentional design by the experimenter is an interesting one. Though self-healing is a process with great practical utility, it is intriguing that nature has been kind enough to provide an inherently smart material system that appears to behave in a way contrary to most others; it mediates recovery in a world in which irreversible damage is the norm. Further advances in understanding the physics underlying this phenomena will surely enable new applications that require materials to withstand high light intensities; and, may lead to new physics.

Now the next battle...

Wasting time, in a good way

Today my morning started early; responding to emails at 6:30 am and an 8:00 am search committee meeting. Various other administrative tasks delayed my arrival in the lab until about 9:45am. After doing the rounds in the lab, and then signing some more paperwork in the Physics office, I made it to my desk, where I spent the rest of the morning answering emails - with a short diversion to chat with the guys fixing our sprinkler system.

After lunch, I finally got back to the task of working on Nathan's cascading paper, which incidentally, I worked on a bit last night. As I was revising text in response to the reviewer's comments, I had a stroke of genius which I imagined would make a significant impact on the world of physics.

Without going into details, cascading is a process by which two molecules cooperate by exchanging a real photon. My insight provided the means for making the exchanged photon virtual. As a consequence, this photon's energy would not need to be conserved as long as the process were fast enough not to violate the uncertainty principle. This made the problem richly beautiful; and more importantly, it meant that a large area of nonlinear optics was flawed. I couldn't resist thinking about this problem with my full attention, so I placed my long "to do" list on the back burner.

I drew Feynman diagrams of the process and immediately realized that if the virtual photon did not conserve energy, it forced the cascading process to also not conserve energy. Thus, the photon must be real and my line of reasoning flawed. I am no genius after all!

However, by taking this detour, I found myself thinking about various cases where virtual processes contribute. To cut to the chase, my understanding of nonlinear interactions took a quantum leap. It made me appreciate the clever minds of great physicists such as Feynman, whose work embodies incredibly deep reasoning.

While most detours on the road waste time and make drivers frustrated, this kind was enjoyable and fulfilling. As I sit at my desk plowing through my work, I remain permeated with a calm happiness.

Until next time...

Me as a reviewer and a complainer about modern-day publication practices

In the past, I have complained about reviewers who have evaluated my papers. However, it is a time-consuming job with almost no rewards, so I do appreciate their efforts. It is a service that we are all expected to provide. Given the time others have spent on my papers, I feel obligated to return the favor.

The most rewarding reviews are those from which I learn. I spent this morning reviewing a paper by some very distinguished scientists in my field. While I admit to the possibility that I may be wrong, I believe that there are serious issues with their paper, which will require attention before it is suitable for publication.

As usual, one activity leads my mind jumping around to other thoughts. This paper is an example of one in a series that is trying to simultaneously correct errors in the literature while introducing new science. This got me thinking again about the curse of information overload.

I am concerned that modern-day science, with the huge number of venues available for disseminating research results, is producing too much information along with lots of junk. The signal to noise levels are dropping while the whole system is bursting at the seems. People are becoming more specialized and less aware of other work. Since researchers are being judged on numbers of publications and citations, they overload existing top journals with so many papers that editors often cut good papers based on arbitrary guidelines. More second-tier journals are popping up to meet the growing demands by authors.

It's getting difficult for me to find useful information in the literature. For example, when searching electronically using very specific keywords, I get too many irrelevant hits that take forever to sort. It is also frustrating to have done what I believe to be great work in the past, only for it to be ignored for 20 years. Even more annoying is seeing the same identical topic of my research appearing many years later in Nature or Physical Review Letters with no citations to my papers. It is even more irksome when the modern work is but a subset of my original research, yet gets lots of recognition.

Sometimes, I send these modern-day authors reprints of my older papers. Some will respond apologetically pleading ignorance of my research, then continue not to cite my work. Others ignore my emails. These are indicators of a system that is not serving its purpose in producing research that serves society.

My review reminded me of the past era of more responsibility in publishing. Perhaps I view the past with unfounded fondness. However, I can atest to the fact that the authors of the manuscript that I have just reviewed are interested in the seeking truth. I therefore feel confident that they will carefully consider my comments and will only move forward with a revised manuscript if they are certain that they can make a real contribution to the field.

Below, I include a copy of my review for all to read. I, of course, will not reveal the identity of the authors, nor the journal to which this paper has been submitted. I take the risk of being exposed as the reviewer, but, I am sure that they will have already guessed my identity based on the flavor of my review; and, I will not deny being the reviewer if asked. Having gotten this off my chest, I need to get back to writing a proposal and grading homework. Perhaps I can then squeeze in a few moments to think about physics, and achieve the bliss that accompanies such thoughts.

And now, finally, the review:

The authors do some combinatorial wizardry to determine the coefficients of the various orders of the nonlinear birefringence. I am not willing to check all of the math, but from what I have checked, I trust that this is done correctly. However, I have a serious concern that may invalidate the approach, as I describe below.

The fundamental property of a material is its nonlinear susceptibility, not the nonlinear birefringence. The nonlinear susceptibility is what governs the physics of light/light interactions while the nonlinear birefringence is the quantity measured. They are related through the constitutive relationship D = epsilon E = E + 4 pi P. In the process of relating the two, one takes a square root of a power series in the field with the susceptibilities as coefficients. The crux of what I believe to be the fallacy of this paper is that n_m is related only to chi^(n+1). In the process of doing the expansion of the square root, one gets cross terms that are products of various lower-orders of the nonlinear susceptibilities that coincidentally may look like expressions that one sees in cascading calculations. The authors have in effect only expanded the square root to the first term. I believe that if the calculations are done properly, then it may be impossible to define unique constants of proportionality. However, under certain approximations, it may be able to define unique constants in the spirit of the authors' original intention.

A second problem along these lines is the neglect of the imaginary parts of the susceptibility. While experiments are off-resonance, there is always a small imaginary part. The cross-terms that I mention above can include products of imaginary parts that give a real response. Since it is possible that effects due to the imaginary part may get large for higher-order susceptibilities, they also need to be considered in the calculation. The fact that in practice, higher-order susceptibilities are by necessity more resonantly enhanced is a problem with applying this theory to real experiments at ultra-high intensities, and should be mentioned.

Nonlinear dichroism can also lead to polarization rotation, a common way of measuring the nonlinear birefringence. This contribution might also be large in practical experiments. While a good experimentalist would take this into account, I am concerned that a blind application of your results could lead to the unintended consequence of more junk in the literature. Thus, if not accounted for specifically, I would suggest adding at least a cautionary note.

I believe that the above issue needs to be carefully addressed before the manuscript is reconsidered for publication.

As a more minor point, In the introduction, the authors mention that the proportionality constant depends on the number of eigenmodes. I usually associate this factor with the number of degenerate frequencies. Is it true that if the frequencies are the same in a pump-probe geometry one gets the factor of 2/3? I thought that if the prorogation directions are different in the non-collinear polarization geometry, the effect shows up in the tensor properties of the susceptibility. That is, if the polarizations are different, then one is probing that particular component of the nonlinear susceptibility tensor. In any case, the meaning of eigenmode needs to be clarified or the sentence needs to be reworded. The use of the expression "eigenmode" in the rest of the paper may also need to be reconsidered.

I find the issue of cross terms to be a major one. If the authors choose to argue that the results hold in some limiting cases, that might diminish the relevance of the paper in loss of generality. If the calculation includes the terms that I believe to be missing, the coefficients will no longer be well defined. In either case, I believe that this paper may need major revision before it is suitable for publication. If I am wrong in my assessment, then the paper may be suitable for publication after minor revisions. In this scenario, it would be useful if the authors provided a more detailed explanation of the relationship between the nonlinear birefringence and the nonlinear susceptibility.