Self Evaluation

I am right in that in-between moment where the formal class meetings are over and my opportunity to assess the final projects has not quite come yet. I thought I would take a couple of minutes and review how the course went this second time through. The syllabus, should you want to check it out, is available here.

The "tools" presentations are far and away the most useful part of the course. They allow students to investigate an array of useful (and less useful) Big and Small digital tools that they can incorporate into their work right away. Collectively, they also teach the lesson that solutions are out there--or can be developed. I will keep this practice during the next iteration of the class.

I was very glad to have changed the final project into a collectively-prepared grant application rather than individual assignments. The class presentations on Tuesday night were terrific and really showcased how much the students have synthesized over the semester. This feature stays.

I was also pleased with how useful the readings were. I felt much better informed about the selections this second time through. I think that the order should be rejiggered next time through, however. The Big Data and Relational Databases weeks should definitely be flipped. Digital Archives and Full-Text Databases belong later, Twitter and Storify earlier. Related, I was disappointed that none of the students took to tweeting about the class. Almost all the Tweets with #HIS717S15 are my own.

One small change I have not yet been able to think through: when I assign students to make a change to Wikipedia, do I really need to run it by the Wikipedians first and set up a class page and sandbox? My theory is that I don't, for a couple of reasons. Students in this class are all graduate students; the assignment is minimal (make a change and watch what happens), no more than anyone might do on their own; I figured out this year to require that the change be "truthful," which helps to prevent vandalism. One of the most interesting points that came out of our discussion about this Wikipedia assignment was the way it caused the students to really review the site, as they looked for topics that needed fixing.

Finally, the portion of the course that I again feel is least successful is the blog. Although creating a dedicated blog site was an improvement over having each student maintain his or her own blog, the process was still not integrated into the class as a whole. In my wild pedagogical imagination, the students will keep up with their blog posts and responses and bring those conversations into the classroom. Although the students were fairly dutiful about posting, it did not feed back into the classroom in the way that I wished. More fodder for next time. And I hope there will be a next time (although not in 2015-2016).

Week 14: Conclusion

Last night was the last class for History 717, Spring 2015 version. In addition to two terrific presentations from the students about their semester DH projects, we discussed the course as a whole. Here are our class notes on What We Learned This Semester:

What did we learn about digital history this semester?

Evaluating DH projects in terms of metadata, funding, and front and backmatter of the websites.
Challenges DH faces in academia and public history
Perishability of DH
Emphasis on projects to market themselves: how do we build this into future curricula? What do you do about platforms being ephemeral?
problem of credit for tenure and promotion
How digital tools can expand your ability to work with historical documents, reconceive ways of working your topic--think outside the box
A lot of ways to be more organized and more efficient
Presentations on different tools
Learning a new tool doesn’t have to be scary
Blogging can be used professionally, not just LiveJournal
How conservative/luddite/resistant are we?
Potential for tools learned
Don’t have to be part of an institution to be part of a historical project (gold star)

What do you wish you had learned?

Closer connect DH with the public...how does it become public history
How does a database work?

Week 13: Evaluating Digital Scholarship

Class session goal: develop our own list of criteria for evaluating digital projects.

• What criteria should we use to assess digital scholarship? How do we know if a digital project is a success?
• What kinds of problems do we need to take into consideration when evaluating digital history projects? How are these problems different from the problems associated with traditional history projects?
• What is the context for most of the evaluations being discussed here? Are they talking about critical reviews, such as you might offer in a journal review of a website? Or are they talking about decisions about people’s jobs? Do some (or all) of these criteria apply in both contexts?
• To what extent can academic tenure and promotion advice be translated to other contexts?
• What criteria have we been using to evaluate websites? Based on the advice in these articles, what else could we be examining that we have not been talking about together?

Todd Pressner, “How to Evaluate Digital Scholarship,” Journal of Digital Humanities 1(4) (2012).

• Why is it important to view a digital project in the medium in which it was created?
• “New knowledge is not just new content but also new ways of organizing, classifying, and interacting with content.”
• Are any criteria missing?
• Do you find any of these criteria controversial?

Geoffrey Rockwell, “Short Guide to Evaluation of Digital Work,” Journal of Digital Humanities 1 (4) (2012).

• What kinds of evaluations is he talking about?

James Smithies, “Evaluating Scholarly Digital Outputs: The Six Layers Approach,” Journal of Digital Humanities 1 (4) (2012).

• Why would you want to pin a project into one of these categories?

 Laura Mandell, “Promotion and Tenure forDigital Scholarship,” Journal of Digital Humanities 1 (4) (2012).

• Why did she write this document?
• Why does she have to say explicitly that an article published in a digital journal is no different than an article published in a traditional print journal?
• Do you agree that creating a platform for digital humanities scholarship (such as Hypercities) should count as much as offering a new interpretation?

Katherine D. Harris, “Explaining Digital Humanities in Promotion Documents,” Journal of Digital Humanities 1 (4): 2012.

• What is a “legacy project”?
• What does she mean by work that is ephemeral?
• How can work that is ephemeral be integral to someone’s scholarly productivity?
• What range of activities does she include in her case?

 AHA, NCPH, OAH, “Tenure, Promotion, and the Publicly Engaged Academic Historian.” 

• Why do “publicly engaged academic” historians need their own separate guidelines for tenure and promotion?
• How is public history scholarship different from traditional academic scholarship?
• What is the thrust of the list of best practices? Are any important considerations omitted?
• What do they mean by “look beyond the traditional monograph”?
• Why does this document give advice to departments and administrators as well as to digital scholars?

American Historical Association, "(Draft) Guidelines for the Professional Evaluation of Digital Scholarship in History," April 2015, 

• “Scholarship is a documented and disciplined conversation about matters of enduring consequence. Hiring, tenure, and promotion involve peer-based judgments evaluating the significance of a scholar’s contribution to one or more of those conversations.”
• Why does this document give advice to departments as well as to digital scholars?
• Why is digital scholarship so often “unfinished”?

Cohen and Rosenzweig, chapter 8 and “Some Final Thoughts”

• “The Ivar Aasen Centre of Language and Culture, a literary museum in Norway, lost the ability to use its large, expensive electronic catalog of holdings after the death of the one administrator who knew the two sequential passwords into the system.”
• Is it obvious that all digital history projects should be preserved?
• What is their general advice about how to choose the systems for coding and storing your project?
• Why isn’t backing up your project on paper a really good idea?
• What is “emulation” as a strategy for preserving digital materials? What are its advantages and disadvantages?
• How can you strike a balance between being careful and being ponderously slow?

Week 12: Relational Databases (plus copyright)

Ansley T. Erickson, “Historical Research and the Problem ofCategories: Reflections on 10,000 Digital Note Cards,” Writing History in the Digital Age

• What is a relational database?
• How do you keep track of all your notes for a project? How should you do this?
• What was Erickson’s goal in using a relational database for her dissertation?
• How much reading of and thinking about documents should you do in the archives? How much do you do later?
• How did she take advantage of the possibilities of the database in her research and writing process?
• When in the process should you start writing?
• Do you actually organize your writing through categories, or through topics, or through narrative?

Gwendolyn Midlo Hall, “Africa and Africans in the African Diaspora: The Uses of Relational Databases,” American Historical Review (February 2010): 136-150.

• What motivates this article? Why does whether women milled rice during the Middle Passage matter?
• How do you know what is in a database you are working with? “Thousands of new Brazilian and Portuguese voyages have been added, correcting the Anglo-focused distortion of TSTD1.”
• What limitations do databases have?
• What does she mean by “unquantifiable data”?
• What cautions does she offer about databases? “to answer, they can be rigid and inflexible, locking in outmoded research and questions and not allowing for new ones. Databases are not a higher form of knowledge that can somehow trump other kinds of research. Scholarship is not a zero-sum game.”

Cohen and Rosenzweig, chapter 7, “Owning the Past”

• What do you need to know about copyright law? How much do Cohen and Rosenzweig want you to worry about it?
• Do course instructors still use course packets?
• What stance toward copyright do you expect from digital historians?
• What are the costs associated with acquiring permission to use published sources in a digital project?
• “Good copyright citizens—cooperative residents of the digital commons—don’t try to grab rights they don’t have.”
• What is Creative Commons?
• What is fair use?

Week 11: Big Data

The Historian's Macroscope

• What is topic modeling? How do they do it?
• What is network analysis? How do they do it?
• Can you tell how much “live writing” they actually did for this project? How does their writing in public approach compare to Moravec’s?
• What is github?
• Are you persuaded by their proposal that shifts in training and standards are needed for historical education?
• What kinds of historical questions can you ask with Big Data that you can’t with more traditional textual analysis methods?
• At what stage in the research process would you imagine visualizing your primary sources with a tool like Voyant?
• What do you think of this definition of Big Data?: “For us, big data is simply more data that you could conceivably read yourself in a reasonable amount of time – or, even more inclusively – information that requires computational intervention to make new sense of it.”
• Does this book motivate you to learn to code?
• Assuming you had a research problem that you needed a tool for…how would you find out what tools are available and which ones you might need to code yourself?
• What concerns of copyright underpinned Google’s digitization of the five big libraries?
• What kinds of historical questions do you think you can ask with Big Data?
• What did they do with the Dictionary of Canadian Biography?
• What do they mean by “distant reading”? (PhD Comics example: https://www.youtube.com/watch?v=tp4y-_VoXdA)
• Differences among “information visualization,” “scientific visualization,” and “infographic”
• Let’s compare the 3 kinds of written in public approaches that we have looked at so far this semester. What are their strengths and weaknesses?

Cohen and Rosenzweig, chapter 6, Collecting History Online

• How are the challenges of digital archiving different now than they were when Cohen and Rosenzweig wrote this chapter a decade ago?
• How do you know that you have a target project audience that is just the right size—not too narrow and not too broad?
• How do you feel the blog is working for our class?
• Do any of you run personal blogs? Why? Are they thematic in any way?
• What lessons do you take from the story of the Sept. 11 Digital Archive?

                                                                                                                

Week 10: Writing in Public

This week's topic is "Writing in Public," a topic I did not include in the first iteration of History 717. I thought to include it this year only after reading Twitter accounts of the 2015 AHA meeting, which included a session on Writing in Public. Michele Moravec (Rosemont College) shares her work-in-progress online for readers to engage as she writes.

General questions:

What are the reasons for writing in public?

Would you do it?

Should you advertise it if you do?

Touch on dissertation/thesis embargo question.

Michele Moravec, #writinginpublic

• How do you pick a platform for writing in public?
• Why does Moravec write in public? What are other reasons for writing in public? Would you?
• What are the risks of writing in public?
• Is writing in public the same as “live writing”?
• Would you do either?
• How do you interact with yourself while you write?
• How does writing in public work as a form of peer review?
• What kind of “behind the scenes” writing activity might support a written in public project like this? For example, how does the organization emerge? How does revising work? What if the author needs to move things around? How far along in a project do you need to be before you start “writing it in public?” What are the implications of writing in two platforms, using different word processing tools? How would you manage keeping track of two organizational systems? How would you decide what to post in public and what to do in private? How would you leave notes for yourself about what still needed to be done, what you were confused by?
• How could you use different platforms differently?
• I’m intrigued by the idea of live commenting. When I am a peer reviewer in a traditional process, the author doesn’t get my comments as I go along. I suppose I could go back and insert them later, but I am too lazy to refigure where they all should go!

Michele Moravec, Politics of Women's Culture manuscript:

• [Note that I end up with almost no substantive questions about this work, probably because it is so far afield for me. I can’t tell what might be known from previous scholarship and what is really new here, since I just am not immersed in that literature]
• How did you decide how to navigate through the book?
• Did you read the comments as you went along?
• If an author puts up images in a work in progress written in public, does she have to get permissions for them the same way she would with a “published” work?
• What argument could you get from the work in progress? Perhaps start with small groups to see what they think the argument is. Perhaps like trying to identify the elephant with a blindfold on?
• What is this project about?
• Do you feel like you need to read the finished book?
• How would you handle footnoting if writing in public? How would footnoting integrate with the use of a program like Zotero or RefWorks that handled many of the fine details for you?
• I wonder why comments are closed on the section on The Journal of Women’s History. I wanted to ask of paragraph 6 whether there are archives that would show the process by which authors decided what to do with their pieces and what to respond to. I’m sure I am only thinking of this because of reading in the open peer review format of this manuscript.

Cohen and Rosenzweig chapter 5, “Building an Audience”

• Do you respond to the idea of marketing in the negative way that Cohen and Rosenzweig assume?
• Should writing in public be regarded as a form of marketing by building an audience?
• How might you build an audience for the group projects you are working on?
• How does Google work, according to Cohen and Rosenzweig?
• What is the difference between “hits” and “visitors” and “users”?
• What are some reliable ways of figuring out why users come to your site?

Week 9: Peer Review

• How is open peer review different from traditional blind peer review, and what are the implications for academic publishing?
• How do digital media present credit problems for authors who are pursuing tenure?
• How can academic communities handle credit for collaborative work?
• What implications do these articles have for the way you are receiving credit for your group projects in this class?

Sheila Cavanagh, “Living in a Digital World: Rethinking Peer Review, Collaboration, and Open Access,” Journal of Digital Humanities 1 (4).

• How do digital media complicate traditional scholarly peer review?
• Why would digital contributions be difficult for traditional academic departments to assess? To include in tenure and promotion reviews?
• Who are the peers who can review digital projects? What standards do they and should they apply? Are those the same criteria that department scholarly evaluations use?
• Why is this true (or not)?: ‘“Self-publishing” on the web, for instance, does not correspond to traditional print “self-publishing” as closely as many non-digitally savvy faculty members believe.’
• What are the problems of traditional peer review for digital projects?
• How should graduate training adapt to the digital age?
• Why are humanities scholars reluctant to see collaboration as worthy of credit?
• Why do humanities scholars need institutional support for digital projects?

Bethany Nowviske, “Evaluating Collaborative DigitalScholarship (Or, Where Credit is Due),” Journal of Digital Humanities 1 (4).

• Why does she argue that Tenure and Promotion committees are not really qualified to assess collaborative scholarship?
• What does this mean?: “the activity I want to argue is actually the new responsibility of tenure and promotion committees. This is your responsibility to assess quality in digital humanities work — not in terms of product or output — but as embodied in an evolving and continuous series of transformative processes.”
• Why is so much of this essay about human relationships?
• What should we do about the problem of digital projects never being done?
• Is this just a lot of hand-wringing over problems that scientists and museums (and other public historians) have long-since solved?
• “Digital humanities practitioners don’t often say, but we all know that collaborative work involves a kind of perpetual peer review.”

Kristen Nawrotski and Jack Dougherty, “Introduction,” in Writing History in the Digital Age

• What is different about this approach to peer review?
• Take a look at some of the original comments: http://writinghistory.trincoll.edu/
• It might be hard to tell, since we have been reading this volume across multiple weeks: but did their review process produce a “better book”?
• What skills do you need to participate in (or run!) an open peer-review project such as this one?
• How does the timing problem affect participation in a project like this one?
• “As a result, higher education pays twice for scholarship produced by its own faculty: first, in the form of salary or sabbatical support for individual professors, and second, in fees for the right to distribute the work.”
• Should blogs count?
• Why doesn’t the Writing History in the Digital Age book have a comment section on the published version?
• What do they mean by “filter then publish” and “publish then filter”?

Jack Dougherty, Kristen Nawrotzki, Charlotte D. Rochez, and Timothy Burke, “Conclusions: What We Learned from Writing History in theDigital Age,” in Writing History in the Digital Age

• Can you think of any other project in which peer reviewers have become co-authors?
• Does this conclusion—the result of the open peer review process—hang together effectively as an essay?
• “But the most important lesson we learned was the power of a critical mass of contributors with their own social media connections. When we tweeted or blogged about new essay ideas on our edited volume, this information cascaded as several authors and commenters recirculated it on their Twitter, Facebook, and WordPress accounts.”
• Were you interested in the comments made on the book in progress? Did anyone go back and look at them?
• Would you publish an essay in progress on the internet?
• “In this way, the volume blurred the boundaries between a conference and a book.”

Jack Dougherty, “Lessons Learned from Open Peer Review for Digital Book Publishing,” media commons: a digital scholarly network, October 29, 2013

• What are the risks to publishers if they have an open peer review project? What are the reasons they might be inclined or disinclined to pursue such a project?
• What are the risks to authors? Why might they be inclined or disinclined to participate in an open peer review project?
• What difference does/should it make to authors in multi-paper projects to see the other essays as they develop?