Monday, April 27, 2015

Week 13: Evaluating Digital Scholarship

Class session goal: develop our own list of criteria for evaluating digital projects.

  • What criteria should we use to assess digital scholarship? How do we know if a digital project is a success?
  • What kinds of problems do we need to take into consideration when evaluating digital history projects? How are these problems different from the problems associated with traditional history projects?
  • What is the context for most of the evaluations being discussed here? Are they talking about critical reviews, such as you might offer in a journal review of a website? Or are they talking about decisions about people’s jobs? Do some (or all) of these criteria apply in both contexts?
  • To what extent can academic tenure and promotion advice be translated to other contexts?
  • What criteria have we been using to evaluate websites? Based on the advice in these articles, what else could we be examining that we have not been talking about together?


Todd Pressner, “How to Evaluate Digital Scholarship,” Journal of Digital Humanities 1(4) (2012).
  • Why is it important to view a digital project in the medium in which it was created?
  • “New knowledge is not just new content but also new ways of organizing, classifying, and interacting with content.”
  • Are any criteria missing?
  • Do you find any of these criteria controversial?


Geoffrey Rockwell, “Short Guide to Evaluation of Digital Work,” Journal of Digital Humanities 1 (4) (2012).
  • What kinds of evaluations is he talking about?


James Smithies, “Evaluating Scholarly Digital Outputs: The Six Layers Approach,” Journal of Digital Humanities 1 (4) (2012).
  • Why would you want to pin a project into one of these categories?


 Laura Mandell, “Promotion and Tenure forDigital Scholarship,” Journal of Digital Humanities 1 (4) (2012).
  • Why did she write this document?
  • Why does she have to say explicitly that an article published in a digital journal is no different than an article published in a traditional print journal?
  • Do you agree that creating a platform for digital humanities scholarship (such as Hypercities) should count as much as offering a new interpretation?


Katherine D. Harris, “Explaining Digital Humanities in Promotion Documents,” Journal of Digital Humanities 1 (4): 2012.
  • What is a “legacy project”?
  • What does she mean by work that is ephemeral?
  • How can work that is ephemeral be integral to someone’s scholarly productivity?
  • What range of activities does she include in her case?



  • Why do “publicly engaged academic” historians need their own separate guidelines for tenure and promotion?
  • How is public history scholarship different from traditional academic scholarship?
  • What is the thrust of the list of best practices? Are any important considerations omitted?
  • What do they mean by “look beyond the traditional monograph”?
  • Why does this document give advice to departments and administrators as well as to digital scholars?


  • “Scholarship is a documented and disciplined conversation about matters of enduring consequence. Hiring, tenure, and promotion involve peer-based judgments evaluating the significance of a scholar’s contribution to one or more of those conversations.”
  • Why does this document give advice to departments as well as to digital scholars?
  • Why is digital scholarship so often “unfinished”?


Cohen and Rosenzweig, chapter 8 and “Some Final Thoughts
  • “The Ivar Aasen Centre of Language and Culture, a literary museum in Norway, lost the ability to use its large, expensive electronic catalog of holdings after the death of the one administrator who knew the two sequential passwords into the system.”
  • Is it obvious that all digital history projects should be preserved?
  • What is their general advice about how to choose the systems for coding and storing your project?
  • Why isn’t backing up your project on paper a really good idea?
  • What is “emulation” as a strategy for preserving digital materials? What are its advantages and disadvantages?
  • How can you strike a balance between being careful and being ponderously slow?


No comments:

Post a Comment