In pursuit of impact

on

Impact of academic research is often measured through citation counts. Arguably, this is a more sensitive measure than just the number of publications, or even the number of publications in prestigious journals. Innovative work often gets published in venues with mixed reputations because prestigious journals and conferences may reject ideas that don’t fit well with the orthodoxy the discipline. In its heyday, for example, the ACM Hypertext Conference rejected Tim Berners-Lee’s paper on the World Wide Web because (among perhaps other reasons) that work contracted then-established standards of what makes interesting Hypertext research.

Thus it is useful to measure the citation counts of papers to understand their impact on the field. Traditionally, this has been the purview of librarians and citation indexes, but the proliferation of publication venues, and the desire to recognize work that was not published in the mainstream (or perhaps not officially published at all, as Daniel Lemire points out) makes the task of collation difficult.

Thus I was intrigued by a citation management tool called Citation Tracker written by Panos Ipeirotis. The tool uses Google Scholar to build a citation history of your (or someone else’s) papers by mining data from a variety of sources, including Google Scholar, DBLP, and others. For each paper, it identifies other publications that cite it, and allows you to clean up the metadata, to delete bogus citations, and in the end, to produce a living citation history. Panos describes it in some detail here.

I’ve used it long enough to clean up my list of papers and their citations. In the process, I discovered some other people who work I should read, and re-discovered some papers I had forgotten about. All in all, the exercise, although tedious for the more highly-cited papers, was worth it. I am now looking forward to using it to monitor for new citations, which strikes me as a good way to keep up with related work.

There are some features I think would be good to have to expand the utility of the tool:

  • When more than one of my papers is cited by some paper and I clean it up and accept it for one paper, it should automatically get accepted for all other papers it cites.
  • To help identify duplicate citations, it should be possible to sort citations by title and by author list (including partial matches) .
  • Certain sources (e.g., the ACM Digital Library) should be markable in some way to automate the selection of citations from these sources.
  • It should be possible to order papers by the number of citations, by publication date, and perhaps by the number of citations per year.
  • Metadata associated with the paper should be extracted automatically for entries for which a DOI is available.
  • It should be possible to export the bibliography en masse, including a report on the citation counts or even with the full citations.
  • The system should allow me to publish my publication list (perhaps filtered) so that I can share it with others. I should be able to put a link from my academic web page to my (suitably formatted) publication list in Citation Tracker to have it serve as a live bibliography.
  • The system should allow me to identify authors who cite my work frequently to increase possibilities for collaboration and to help track related work.

Overall, however, this is a great start and promises to be a useful tool for managing your publication list and discovering other people with similar interests. This tool reminds me of Mendeley, but in many ways it is simpler, more effective, and more stable. In particular, Citation Tracker’s ability to discover new papers automatically makes it more valuable. With the addition of some rudimentary data mining capabilities, Citation Tracker can become a truly indispensable tool for academics.

3 Comments

  1. Thanks for the detailed feedback!

    We will start working on the next version soon, trying to implement all these suggestions!

  2. Cool! Looking forward to it!

  3. […] of tools (often based on Google Scholar as the search/data mining engine) have been released. I reviewed CitationTracker earlier, and now got around to looking at […]

Comments are closed.