I’ve read the latest postings above, and they all seem to point in the same direction. Very few full text studies are being done, despite the potential benefits. That is my observation as well. Usually, in paradoxical situations like this, one has to examine incentives for deeper insights.

For an example from a completely different area of study, most people believe that interdisciplinary research has myriad benefits, but publications reflecting real interdisciplinary research are relatively sparse. I examined this paradox in a Bioscience paper in 2002, and showed that, despite the flowery words about the benefits of interdisciplinary research, the reality was mainly disincentives to pursue this type of research. There is far more ‘bang to the buck’ in publishing traditional focused research in a discipline, where every slight change in a parameter could result in another publication. While interdisciplinary research could result in great benefits to science not available through more narrowly focused research, the time and effort required to understand the different disciplines and their inter-relationships does not, in most cases, pay off in terms of the metrics used to evaluate research productivity.

As another example, climate change seems to be bearing down upon us, yet essentially nothing is being done to counter it. I’ve examined the motivations of all the major stakeholders related to climate change, and all are comfortable with the status quo, albeit for different reasons.

From the few studies I’ve done in full text mining, far more and richer information is possible than mining titles or Abstracts. However, mining full text is intrinsically more difficult than mining Abstracts, and as in the examples above, I’m not sure it provides more ‘bang for the buck’ of interest to most researchers. In other words, the incentives for going to full text may not be there.

However, I don’t buy the arguments of limited coverage as a valid reason for lack of studies. Many research studies could be categorized as proof-of-principle demonstration, and for that only limited coverage databases are required. Some of the original studies with Textpresso demonstrate that.

The best way to move full text mining forward is to use the limited full text databases available, compare full text results with Abstract only results, and show the benefits (and additional costs as well). If these studies could show that a benefit-cost advantage exists for full text, full text mining would be well on itsur way to gaining acceptance. Our limited full text studies in 2009, only part of which were published, convinced me the benefits far outweighed the costs, but that was only one data point. Far more is required to convince a wider public.

As a postscript, there is another advantage of full text that I haven’t seen mentioned elsewhere. Most publication information retrieval is based on text. Queries tend to be words and word combinations. One can then use these retrievals to explore citation networks and find additional relevant articles outside the text query terms used. But, full text especially contains much more than words/phrases. There are symbols and graphics of all types, like equations and curves. In theory, at least, these symbols could be used as search terms. One might have an interesting curve, and want to identify such curves in other literatures and how they were interpreted. Since curves typically are only presented in full text, this unexplored area has the potential of great payoff for full text mining.

Shashank Agarwal, Hong Yu “Figure summarizer browser extensions for PubMed Central” Bioinformatics. 2011; 27(12):1723-1724.

I already had the FigureSearch paper “Figure text extraction in biomedical literature” in the list, but if there are others, please let me know.

Best,
Casey

-Hong Yu

I don’t see a way to respond to the Nature Editorial online, but I’ll combine your blog response and the Editorial response here.

The Editorial includes the statement “The promise is yet to be backed up with concrete examples of scientific success”. That’s only true in part. Our research group has been generating potential discovery over the past five years. I addressed the issue in detail in an ARIST chapter [1], and more recently in a comprehensive update of our discovery approach [2].

However, the critics quoted in the Editorial may have a case. Useful text mining results have both a quantity and quality component. Typically, the computer and its built-in algorithmic rules do a reasonable job on the quantity component, but not on quality. For the latter, human judgment is essential. Much of the focus has been concentrated solely on the algorithms.
For full text, I have seen very little published in the literature-related discovery area. I did some full text mining when I worked at MITRE. I addressed two aspects: information retrieval and information extraction. I published the information retrieval results in Journal of Information Science in 2010 [3]. If queries are generated properly, full text information retrieval can provide orders of magnitude increase in relevant articles retrieved, depending on the category of interest. In fact, we have found that the types of queries required to retrieve relevant documents from searching the full text are the same types of queries required to pinpoint concepts/papers with high discovery potential. For intel work, many of the categories of interest (e.g., suppliers, hardware, software specifics, etc) can only be found in the full text.

For the information extraction, we examined a few approaches. The difficulty was extracting some of the important rare events. Operationally, important rare events (low frequency phenomena) had to be related to high frequency phenomena. Standard text mining procedures, such as in scientometrics, are based on statistical analysis of high frequency phenomena and are not really applicable to the low frequency extraction problem. I found a way to relate the important low frequency events to the high frequency and extract these rare events, but cannot comment on the downstream etiology of the results. The bottom line is that much useful information can be obtained from full text mining, if the right algorithms are used, and some human judgment is applied as well.

RNK

REFERENCES
[1]. Kostoff, R.N., Block, J.A., Solka, J.A., Briggs, M.B., Rushenberg, R.L., Stump, J.A., Johnson, D., Wyatt, J.R. “Literature-Related Discovery”. ARIST. 43. 243-285. 2008.
[2]. Kostoff RN. Literature-related discovery and innovation — update. Technological Forecasting and Social Change (2012). doi:10.1016/j.techfore.2012.02.002. Also, see appended below.
[3]. Kostoff RN. “Expanded information retrieval using full text searching”. Journal of Information Science. 36:1. 104-113. 2010.

APPENDIX – ANNOUNCEMENT LETTER FOR LRDI UPDATE PUBLICATION
When reference [2] went online, I distributed an announcement letter to a few colleagues. I reproduce the letter below; it summarizes the contents and provides access.

A recent publication updates the Literature-Related Discovery and Innovation (LRDI) technique, which identifies prevention and remediation measures for chronic and infectious diseases [1]. The information technology-based LRDI technique may be of interest to researchers in text mining, bioinformatics, and literature-based discovery, and the potential medical applications may be of special interest to researchers/clinicians focused on preventing, reducing, halting, or reversing progression of chronic and infectious diseases. To illustrate the potential power of LRDI, the article emphasizes the relationship between the results of our 2007 LRDI multiple sclerosis (MS) study and a recent demonstration of MS reversal.

The findings in the update [1] include:
* the role of comprehensive and precise information retrieval in discovery and innovation
* the value of interdisciplinary research in discovery and innovation
* the critical role of hormesis and synergy in preventative measures and accelerated healing
* the critical need for cause removal in reversal of chronic disease
* the severe under-reporting of critical variables in the clinical trials literature
* the severe under-utilization of the broad biomedical literature for reversing chronic disease
* concerns about the credibility and integrity of the medical literature in areas that concern commercial and government/political sensitivities

Dr. Ronald N. Kostoff

References
[1]. Kostoff RN. Literature-Related Discovery and Innovation – Update. Technological Forecasting and Social Change (2012). doi:10.1016/j.techfore.2012.02.002.
*Pre-print full text version can be accessed at (http://stip.gatech.edu/wp-content/uploads/2012/02/LRD-UPDATE_TFSC_7_REV.pdf).
*Journal posting access (http://dx.doi.org/10.1016/j.techfore.2012.02.002).

Need all these uses before concluding how useful PMC OA has been for text mining thus far.

> corpus-wide text-mining efforts are rare, and what this implies for the state of the art

ok, I’m with you on this. Good point.