Misplaced rejoinder lacks support

By Flor Lacanilao

In his "Strange phenomenon: A response to Lacanilao" (Inquirer, 04/11/2011), Dr. Ramon Guillermo disagreed with several points in my commentary, "Democratic governance impedes academic reform" (03/14/2011). I showed that the use of peer judgment has been a major cause of declining academic performance in the Philippines; but this has been reversed by the use of objective measures. Guillermo challenged my article concerning the use of valid publication and citation counts (objective measures), but he discussed only their misuse instead of the useful information they provide.

The assessment tools are the ISI-indexed journals and the ISI indexes. These are internationally accepted indicators. They are widely used measures of research and S&T performance. His objections, however, centered on the misuse and abuse of data concerning publications in ISI-indexed journals. The usefulness of a tool -- like the kitchen knife or the gun -- can only be as good or as bad as the purpose or the person using it.

Dr. Guillermo favored the prevalent practice of peer judgment and democratic governance, instead of ISI measures, citing historical and emotional events of nationalist struggle for democracy and academic freedom. He failed to show how these relate to peer judgment or enhanced academic growth, like improved research and teaching. On the other hand, using hard data, I showed that the introduction of ISI measures improved research output after decades of decline.

Below are some important uses worldwide of ISI-indexed journals and ISI indexes. They will clarify the issues raised by Guillermo.

1. In developed countries, they supplement peer judgment of academic performance. In fast developing countries, they are the reliable measures of evaluating research and S&T performance.

2. They are commonly used in ranking nations, universities, and scientists, which are published in leading journals like Science and Nature.

3. The journal coverage of the three top ISI indexes are as follows: sciences (3,786), social sciences (2,876), and arts & humanisties (1603). The lower fraction of covered journals is a reflection of the research output from each major fields -- 75% average of journal content in the sciences, 50% in social sciences, and 25% in arts & humanities (ISI study). This disproves Guillermo's claim that ISI indexes are unfair to social sciences and humanities

4. Guillermo's claim that the dominance of US and UK in English-language journals is disadvantageous to non-English speaking countries has also no basis. The top six countries with the highest number of ISI-indexed publications are dominated by non-English speaking countries -- US, China, Japan, UK, Germany, and France -- with China increasing its number of publications twofold every 5 years in the last 2 decades, and predicted to overtake the US soon (Thomson ISI report and others).

5. In addition to titles and authors of published papers and books, ISI indexes also gives citation data, hence, solving Guillermo's worry of ISI's bias against books. The number of times a paper is cited is a recognized measure of quality. You can get the same information, but not quite as complete, from Google Scholar. A correction factor are used to remove distortions due to different citation rates in different disciplines, solving another problem raised by Guillermo.

6. Further, Fred Grinnell says in his book, Everyday Practice of Science, that the easiest way to assess if one has made any major contributions to one's field is with the ISI data base called Web of Knowledge.

7. The stature of top scientists in various fields is reflected by their scores in ISI indexes -- for quantity and quality of published work. On the other hand, most of our prominent academics, scientists, so-called experts, and even National Scientists -- selected by peer judgment -- lack the number and citations of their publications (click or Google search Celebrating the UP Centennial).

8. There is no question that the quality rather than the number of publications is a better indicator of research performance. Again, reminding Guillermo, we can only rely on the ISI citation indexes for valid citations because we lack experts to judge quality. For example, how can the quality of work done by a Filipino biogeographer be evaluated by his peers in the Philippines if he is the only well-published biogeographer in the country?

9. It is true that in western countries -- where all competent scientists publish in ISI-indexed journals -- there is much discussion concerning the misuse and abuse of “numerology.” This does not mean that numerical data are completely useless. Many who question the usefulness of the ISI-indexed journals or ISI indexes in measuring academic performance can be shown as poorly published.

10. The utility of numerical data can be seen, for example, in a recent paper ("Expert credibility in climate change") on Anthropogenic Climate Change (ACC) in the Proc. Natl. Acad. Sci. USA that reports, "The relative climate expertise and scientific prominence of the researchers unconvinced of ACC are substantially below that of the convinced researchers.”

Finally, my call for visionary leadership should not be confused with preference or support for fascist rule. Guillermo's appeal to Philippine nationalism is misplaced. Mediocrity has never been a UP tradition.

Florlaca

(Dr. Flor Lacanilao obtained his Ph.D. (specialization in comparative endocrinology) from the University of California at Berkeley. He served as chairman of the Zoology Department at UP Diliman, chancellor of UP Visayas, and chief of SEAFDEC in Iloilo. His email address is at: florlaca@gmail.com .)

Continuation of Philippine Monitoring and Evaluation Society responds to former U.P. Visayas Chancellor's comments on "grey literature"

(To return to the beginning portion of this article, please click here)

We agree that peer-reviewed publications are a strong indicator of accomplished researcher. But to make it as the lone criterion for saying that one who is not published is unfit as a researcher, administrator or a mentor would constitute total intellectual arrogance. People who have published may just have the circumstances favorable for publishing, but to say those who have not done much are not capable people would be a sign of narrow-mindedness. Who are we to say that given the circumstances, we, the lowly researchers would not be able to pass the rigid measure of peer-reviewed journal? People normally are of different circumstances and contexts -some virtually grow up in the lab or classroom, making it more conducive for them to contribute to journals. Some do not have that luxury –many are consumed by administrative work or industry practice, but it doesn’t make them lower caste individuals who will not be able to pass a refereed journal. There is no such thing as monopoly of intelligence. Published scientists like you are not the sole arbiters of what is correct and what is not in research.

Prof Romeo Santos has been in the industry practice for a number of years, simultaneous with his academic post in UP -and for more than 15 years had run a Japan-based Manila branch of a development and research consultancy firm. He was trained in the Japanese way of research and was exposed deeply onto Japan’s style of business management –for a long time managing that firm of more than 600 staff and employees at its peak. His recent research job commissions were from the UN & related organizations (UNIFEM, UNFPA, UNICEF, IDLO, others) and Canadian, Japan and US governments, among others. These may not be considered peer reviewed journal articles, but who are we to say that these commissioning agencies have lower standards of research? His works may not be of the same nature as your work and he is not used to a study ‘stimulating the bladder of marinus frogs’, http://www.facebook.com/l/30872/www.sciencemag.org/content/163/3873/1326.full.pdf, but certainly he also knows that research has ‘universal’ features regardless of whatever field one is working on.

When you refer to ‘Academician Emil Q. Javier, as President, Academician Ledivina V. Cariño as Vice President, and Academician Evelyn Mae Tecson-Mendoza Secretary, etc., etc.’ as non-scientists, http://www.facebook.com/l/30872/www.bahaykuboresearch.net/index.php?module=article&view=81, and scornfully claim that ‘No wonder the state of science in the country is so bad’, we can only smile in disbelief. Your doctrine would put other people as scratch and hopeless, if your measure of fitness were to be valid.

President Alfredo Pascual, our current leader, may not have published much in refereed journals. Would that make him unfit to run a Research University –which is UP? We ABSOLUTELY DON’T believe so. In the same line of argument, Dr. Patricia Licuanan, Director of CHED, may not have published much in your ‘Science Citation Index or Social Sciences Citation Index’ journals. But we are CONFIDENT that she is running the CHED in a way that research is brought to the forefront.

Publication in peer-reviewed journals is not the sole determinant of performance. In the field of evaluation, it is an Output, an indicator. But in socio-economic development, being just Output-oriented has been clearly shown as a wrong direction of practice, management or governance. We subscribe to the principle of Results-oriented way of measuring performance and success. It involves change of behavior and the attainment of enabling environment that supports positive change in the state of things. Output alone does not guarantee progress and achievement of Collective Well-being. It should be the Outcomes brought about by the outputs and the activities we put in our plan. How sure are we that peer-reviewed publications alone can improve the Philippine society?

For instance, you, as a well-published scientist had the opportunity of running the UP VISAYAS as a CHANCELLOR, and CHIEF of SEAFDEC in ILOILO in the past. Going to the counterfactuals… -How did your peer-reviewed publications work [for YOU] for the betterment of these two organizations? Where are they now? IS the state of science and research in these organizations now better off because of ‘Non-Academician but Scientist’ Flor Lacanilao? Well, based on the reality of what we see in these organizations even right after your stint, we suggest you better confer and compare notes with ‘Academician and Non-Scientist’ Emil Q. Javier, ‘Academician and Non-Scientist’ Ledivina V. Cariño, and ‘Academician and Non-Scientist’ Evelyn Mae Tecson-Mendoza, etc., etc.’ and see whether there are better ways of doing good for the country other than flipping your peer-reviewed articles in the air. And maybe you guys can give good advice to ‘Non-Scientist’ President Pascual and ‘Non-Scientist’ Director Licuanan on how to or not to run their organizations based on grey papers.

By the way, we feel sorry in knowing that the Philippine Star and other dailies in the past had declined printing your fifth paper “because many of our respected scientists found my articles adversarial and counter-productive”. BUT NO PROBLEM, DONT WORRY -you don’t have to fret a lot because they are GREY PUBLICATIONS anyway! Aren’t they?

Prof Lacanilao, we have a high respect for you, as we have mentioned early on. We still believe in your Cause and we trust that we can do much together to help our country. But we suggest we do it in a different way. 

Yes, “the truth hurts and it hurts only the culprits”, but we need the culprits on our side [to be together with us] in mending the tear in our society. Truths or facts alone may not settle disputes nor even put us in unity. Besides truths, we need to walk extra miles with our values and dignity intact to unite for the Common Good. Please put faith that we, the ‘culprits’, can change for the better -despite having less of your peer-reviewed journals. Please don’t just sit on the embankment, and from there - bark about how really stupid we fish, get right into the water and hold the line together with us - perhaps we could learn your trade better that way. And perhaps we can expertly dissect a Tetraodontidae soon -for submission to the Nature Journal.

Yours truly,

James Santos

P.S.

No one, not even published scientist, has an exclusive birthright to research. Research skills are cumulative. One hones skills through continues research undertaking. Even a well-seasoned researcher needs to retrofit in an ever-changing research environment.

For instance, in your article in SEAFDEC not too long ago -'Doing research for Development',
http://www.seafdec.org.ph/news_doing_research_for_devt.htm, we think, as a meticulous Scientist, you could present a better and less-faulty paradigm on representing how a right research is done. No researcher, in his right mind, would go straight into Publication by just having the Proposal and the Data Gathering –or even into writing the Report or the Thesis. All of us know there is something amiss in your model, and being a noted scientist that you are and an authority in this field, you should know what it is. It doesn't need a peer review to figure it out.

Dr. Lacanilao, if this is the level of expertise you boast you got as a highly published researcher, I’m afraid people should learn better from my graduate students than from you.