Cutting life in half

When our son was born in 1985, we purchased a Sony Betamax camcorder. Back then, the Sony Beta format was king. Its quality was clearly superior to the competing VHS format. However, Sony made the mistake of keeping their technology proprietary, so the VHS format eventually won out. These days, VHS tapes are passe, as is its 8mm successor.

While I was in graduate school, our Egyptian friend introduced us to tabouli, a Lebanese dish that my wife and I loved from the minute we took our first taste. Our friend Saleem graciously volunteered to teach us to make this wonderful but labor intensive dish at our home, and that's how our Friday-night tabouli and pasta dinner ritual started.

I introduced my new Betamax camcorder to our guests at one of these dinners by taping the event. Later, while we excitedly watched ourselves on a tiny TV, Saleem commented, "This camera cuts life in half. We spend half our time living and the other half watching ourselves on TV." I was reminded of these past good times while preparing my annual review materials. It made me realize how much time we waste documenting what we do.

To waste more of our time, our university implemented an ill-conceived notion to computerize the whole process with a clunky system. The system, called WORQS, was designed to save time; but, it becomes more cumbersome when the reported activities become nuanced and diverse, as is often the case for productive faculty members. My extreme dissatisfaction with this system lead me to coin the phrase WORQS SUQS.

I have already spent a large chunk of my Saturday afternoon updating my CV so that I can then spend another day entering it again into WORQS. My aggravation with the process reminds me of all sorts of new well-intentioned but time-wasting demands on researchers. For example, in an effort to prevent scientific fraud, the National Science Foundation now requires a data management plan. I agree with transparency as well as forcing overly-protective scientists to publicly share their data, especially when the work is paid for by our citizens. However, such rules enforce undue additional time demands on all the scientists.

I have many complaints about regulations that waste time with little in the way of overall returns, but I will leave that for another day. For now, I am attaching, below, an exchange that I had with the administration a few years ago. You may wonder why I wasted time doing so. My assumption was that a university, populated by members of the intelligentsia, would respond positively to a logical argument, resulting in increased efficiency. Instead, I was perceived a spoiled brat who refused to do what little was asked of me. Here is the exchange, with names removed:

Dear Administrator,

I am surprised at the university's attitude that we are to fill out our annual reviews online because WSU pays us and that's what we are expected to do - case closed. This goes against what I understand to be the principle of shared governance. The purpose of my email to you and the WORQS committee was to point out how some parts of the system are poorly designed from the point of view of the faculty member and the chair who does the annual reviews. I was not attempting to shirk responsibility, but rather, was bringing to your attention the features of WORQS that are inefficient, inferior to the present system, and do not fulfill - as far as I can see - the needs of the administration and the mission of the university.

I wholeheartedly embrace the idea of a computerized system that both collects important data for the university; and saves time of faculty members and chairs when preparing annual reviews. In particular, the data mining features are excellent. Not having to enter my lists of grants and proposals; but rather having WORQS lift this information from OGRD saves lots of time and effort. And having WORKS download a list of the classes that we teach by pulling data from the registrar is another good idea.

Unfortunately, there are other aspects of WORQS that are cumbersome and poorly designed. Is the point of WORQS to make it less time-consuming to enter information? Is it to provide better information for the chair in evaluating performance? Are there certain key metrics that the university needs to collect for the state legislature for purposes of accountability or to give us ammunition for showing how great we are? If so, portions of WORQS need a serious overhaul.

I focus my criticism on the scholarship menu as an example of bad design.

Scholarship Menu

Time Savings?

Since the WORQS system is purported to save time - or at least to not take much extra effort, I timed the publication-entry process in the "scholarship" menu item. It took me, a fast typist and computer geek, 50 minutes to enter my 2006 publications (had I not timed myself, I would have guessed 10 minutes) compared with 55 seconds to cut and paste the same ones. Net time wasted: 49 minutes.

The major problems from the faculty member's perspective:

a) Having to add journals in one menu area first then pulling down a journal name from the publications area takes time. Note: One of the new journals in which I publish was not on the list, which I had to add. After adding it (I needed to first find its ISSN), I got an email a few minutes later that it was on the system.

b) The method of adding coauthors is perhaps the worst feature. In one case, I had a paper with 13 coauthors, most of whom will probably never coauthor another paper with me in the future. Your system erroneously assumes that collaborators remain static. Since the coauthor field was not required, I just left it blank. Going through this process would have wasted lots more time. In fact, I left most of the non-required fields blank.

c) There is no field to add the year of a publication. There are cases where a paper appears in print in the year following the "publication date" (the volume number, etc. is not known until after annual reviews), so those papers fall through the cracks.

d) The present system allows for double counting by allowing a talk and a conference proceedings paper to be entered individually.

Usefulness to Chair?

With regards to publications, the information that is not required (like the coauthor list) is perhaps one of the most important since it gives a snapshot of collaborations and the students involved in the research. As chair, I would refuse to force my faculty to waste their time making such entries since for my purposes, seeing a CV is enough to judge their work.

Usefulness to WSU

Getting the names of the journals entered correctly and free from typos appears to be a priority to the administration and WORQS is clearly designed in a way that gets this information accurately. Other parts of the form leave plenty of room for inaccuracies.

This one menu item illustrates the vast number of the kinds of scholarly works produced by our faculty and the unique regard for journal articles. On the one hand, entry of journal publications enforces strict compliance on getting the journal name right, while we can add whatever we wish in all the other fields. This suggests that journal names are held in high esteem while other fields are not, so these other sections could and should therefore be left to free-form entry.

If WSU truly is interested in the most efficient system, then the following entry method would be superior:

1. Set up WORQS to import from online databases (such as Web of Science, for example) an individual's publications that appear in refereed journals.

2. Allow the faculty member to enter missing journal articles by wholesale cutting and pasting. Units whose research is published in journals that are not covered by databases should be permitted to use free-form entry.

3. Have WORQS scan the list of WSU coauthors and mark the ones that are students (grad or undergrad) from WSU's personnel/student database.

4. Since WORQS does not seem to care about the accuracy of conference proceedings, etc, allow faculty to cut and paste such "other" scholarly work freestyle.

The above would be a smart use of technology and would better embrace what I assume to be in the true spirit of WORQS.

Other Menu items

There are many other areas that can be improved. Similar to the method of getting grant activity directly from OGRD, why not do the same for committee service? At the level of department, college, and university, there are lists of committee memberships in place, so lift that info electronically in one swoop. The same thing can be done with M.S. and Ph.D. committees from lists compiled by the grad school. Is the WORQS committee analyzing the system and implementing such features to save time? If there are difficulties with implementing these kinds of recommendations, the response should not be to stick the faculty with the chore. Rather, you should call upon the faculty for ideas to improve the system. We all have a common vested interest in making WSU a better place.

As a general observation, it does not make sense to design a uniform online system when there are so many permutations in the diversity of work that we do. Even if you were to take into account all contingencies, there are just too many variables to enter efficiently. In fact, I would argue that if a single efficient form could be designed that takes into account all that we do, it would be an indicator that we are a mediocre institution. WORQS should be a system that is designed to save time and to augment annual reviews, not to drive a process that encumbers us in a web of minutia and forces us into conformity. Given the present implementation of WORQS, it emphasizes the latter.

In your email, you state, "we are relying on a faculty committee to work on this form and they simply don't agree with your argument." The issue is not open to opinion. Rather, we should turn to factual data to support our conclusions. In your trials of WORQS, have there been objective studies that compare it with the old-fashioned CV in terms of time and flexibility? Have you timed faculty who are using WORQS to determine the time it takes to fill out various parts of the form? Would a totally automated system that gathers the info needed by WSU, augmented with free form entry for the chair's evaluation, not make the most sense? If such a system is going to be required of all faculty, a cost-benefit analysis needs to be done; and, if any unit or group takes the brunt of the cost, then the system should not be imposed on anyone. I find it unacceptable that we are being told that WORQS is great when there is evidence to the contrary. Faculty should have more input before use of the system is required.

A university's top priority is to foster an atmosphere that nurtures a dynamic interplay of education and the generation of new knowledge. The administration's responsibility is to remove burdens on faculty so that we can fulfill our mission. I believe that this whole business with WORQS and how it is being imposed on us sends the opposite message. Perhaps I am naive in believing that our faculty is so diverse and creative that one form can not do justice without becoming cumbersome.

On a closing note, we need to ask ourselves the question, "How are we to think outside the box if we spend so much of our time filling out boxes?" Let's use technology to save time and make us more productive.



Sincerely,

Mark G. Kuzyk
Professor and Associate Chair of Physics
and past Chair of the Materials Science Program

P.S. Most faculty in our department with whom I have discussed WORQS also found it quite cumbersome. I am attaching an email that I got from Faculty X. Also, you need to consider the reactions of the staff to this new system. They do not have the luxury of speaking up in the same way as faculty. Based on several conversations, my sense is that WORQS is demoralizing to the staff when most of the form is devoted to information that has no bearing on their work. Have you had broader input from staff beyond committee members?

My Original Email:

Thanks for your detailed response. I have been thinking about annual reviews from the point of view of a faculty member and as associate chair (I will be doing annual reviews this year in our department). I applaud the time saving features such as grants showing up automatically. I was credited with a $6 million dollar grant which I never got. So, that will make me look good! On the other hand, adding publications in the piecemeal way requested is very cumbersome. Even when the journal names are on a pull-down menu, it still takes lots of time to add the title, volume number, pages, type of pub, etc. - especially for productive faculty. Other parts of the form suffer from the same issues. The best solution is to give a choice of cutting and pasting items wholesale OR using the form.

The bottom line is that annual reviews are used to assess performance; and, the chair is best suited to do so. I can tell when one publication is of sufficient high quality that it trumps 10 others. So statistics are useless in my analysis of faculty performance. If WSU wants to gather statistics, then I would be happy to provide my CV for statistical analysis. But, as faculty members, we should not be REQUIRED to waste time entering information in a form that really has nothing to do with annual review, nor does it enhance the process in any way. In your email, you state, "…we can't do that if the system is to be useful." Useful to whom? It's certainly not useful to faculty nor is it to those of us who are doing evaluations.

Our time is chipped away to the extent that I am finding it impossible to do any research, so I do it in my times of "leisure." Yeh, this way of doing annual reviews may take only an extra ½ hour, but there are countless numbers of other such things that we are already doing that also don't make sense. Enough is enough!

My previous communication was sent to you through so and so because I was not aware of the committee working on this form. I hope that my comments are passed along to whatever group is working on this process and to whoever is making the decision to impose this online system.

At this rate, next we'll be requiring all teaching evaluations to be standardized. Though I would be a big beneficiary of such a system, this would be a huge mistake.

Your Response:

I understand your frustration about the many activities that faculty members are asked to do. Unfortunately, your comments have already been passed on to the faculty committee, and they disagree with you. WSU pays your salary all year. For that, the University only asks you to explain what you're doing once per year. Even if you have many, many, many publications and you are a completely incompetent typist, using the pulldown list will take you only minutes more than cutting and pasting. Your use of the pulldown list will allow the University to collect much better data. (So, for example, you won't get credited with the publication equivalent of a $6M grant that you didn't earn.)

I'm sorry that this issue is frustrating to you, but we are relying on a faculty committee to work on this form and they simply don't agree with your argument. So far, approximately half the colleges have decided to use WORQS for annual review this year even though it is still optional. To date, at least some of your colleagues believe that WORQS is a much needed effort saver for the entire University.