Digitised Manuscripts to Europeana

We are excited to announce the second round of the Open Humanities Awards. *The deadline for submissions to the awards has been extended to Friday 6 June 2014.*

There are €20,000 worth of prizes on offer in two dedicated tracks:

• Open track: for projects that either use open content, open data or open source tools to further humanities teaching and research

• DM2E track: for projects that build upon the research, tools and data of the DM2E project

Whether you’re interested in patterns of allusion in Aristotle, networks of correspondence in the Jewish Enlightenment or digitising public domain editions of Dante, we’d love to hear about the kinds of open projects that could support your interest!

Why are we running these Awards?

Humanities research is based on the interpretation and analysis of a wide variety of cultural artefacts including texts, images and audiovisual material. Much of this material is now freely and openly available on the internet enabling people to discover, connect and contextualise cultural artefacts in ways previously very difficult.

We want to make the most of this new opportunity by encouraging budding developers and humanities researchers to collaborate and start new projects that use this open content and data paving the way for a vibrant cultural and research commons to emerge.

In addition, DM2E has developed tools to support Digital Humanities research, such as Pundit (a semantic web annotation tool), and delivered several interesting datasets from various content providers around Europe. The project is now inviting all researchers to submit a project building on this DM2E research in a special DM2E track.

Who can apply?

The Awards are open to any citizen of the EU.

Who is judging the Awards?

The Awards will be judged by a stellar cast of leading Digital Humanists:

• Professor Andrew Prescott, Kings College London

• Professor David Robey, University of Oxford

• Professor Melissa Terras, University College London

• Nicole Coleman, Stanford University Humanities Center

• Dr Laurent Romary, INRIA

• Sally Chambers, DARIAH-EU

What do we want to see?

Maphub, an open source Web application for annotating digitized historical maps, was one of the winners of the first round of the Open Humanities awards

For the Open track, we are challenging humanities researchers, designers and developers to create innovative projects open content, open data or open source to further teaching or research in the humanities. For example you might want to:

• Start a project to collaboratively transcribe, annotate, or translate public domain texts

• Explore patterns of citation, allusion and influence using bibliographic metadata or textmining

• Analyse and/or visually represent complex networks or hidden patterns in collections of texts

• Use computational tools to generate new insights into collections of public domain images, audio or texts

You could start a project from scratch or build on an existing project. For inspiration you can have a look at the final results of our first round winners: Joined Up Early Modern Diplomacy and Maphub, or check out the open-source tools the Open Knowledge Foundation has developed for use with cultural resources.

As long as your project involves open content, open data or open source tools and makes a contribution to humanities research, the choice is yours!

For the DM2E track, we invite you to submit a project building on the DM2E research: information, code and documentation on the DM2E tools is available through our DM2E wiki, the data is at http://data.dm2e.eu. Examples include:

• Building open source tools or applications based on the API’s developed

• A project focused on the visualisation of data coming from Pundit

• A deployment of the tools for specific communities

• A project using data aggregated by DM2E in an innovative way

• An extension of the platform by means of a practical demonstrative application

Who is behind the awards?

The Awards are being coordinated by the Open Knowledge Foundation and are part of the DM2E project. They are also supported by the Digital Humanities Quarterly.

How to apply

Applications are open from today (30 April 2014). Go to openhumanitiesawards.org to apply. The application deadline has been extended to 6 June 2014, so get going and good luck!

More information…

For more information on the Awards including the rules and ideas for open datasets and tools to use visit openhumanitiesawards.org.

This is the final blog in a series of guest blog posts by Robyn Adams and Jaap Geraerts, part of the project team that won the DM2E Open Humanities Award at the Center for Editing Lives and Letters. The final report is available here.

The blind spots of network visualizations

In our last blog we discussed some of the issues regarding the use of network visualizations as well as the limits of the information such visualizations might convey. We will examine this topic into more detail in this blog by employing some of the data derived from our own project. To reiterate a point we previously made, network visualizations often include only one type of relationship, which obscures the various links which connected people to one another. When analysing epistolary networks and the dissemination of information, the omission of such links can be of vital importance, for even when letters were sent to one person, often recipients were asked to pass on information to a third party. Often letters were packages, consisting of multiple letters or including other documents that sometimes were addressed to different people, thus turning a recipient (or the first recipient of the package) into a transmission agent, a person responsible for dispatching the documents or the information to their final destination. Besides the fact that people had different roles, how can we include the flow of information that resulted from people meeting in person to convey the information they received via letters, for instance?

We will illustrate the limits of a visualization of a binary epistolary network by focussing on a case-study, namely the correspondence and the transmission of information regarding the siege of Groningen, which took place in the late-spring and summer of 1594. The following image is a traditional network visualization, the edges representing the letters sent between Bodley and his correspondents regarding the siege of Groningen in the period May to August 1594.¹

This simple visualization tells us, for instance, that only a small number of people wrote about the siege and that Bodley was the main correspondent, yet there is much more information that is not represented in or by this image. The visualization also creates the impression that, while the siege was of great importance to the Dutch authorities, it nevertheless was only discussed by English correspondents who were part of the epistolary network. In order to show which people actually were involved in transmitting information, the way in which they did this, and the various links that were forged in the process of disseminating information about the siege, we returned to the primary sources. Extra research was necessary as we wanted to move beyond the basic relationship codified in our dataset, namely the authors and recipients of the letters, to include a large palette of connections which linked people to one another. The results are visualized in a SDL-diagram, which is normally used to depict processes within a particular system (e.g. a computer program), yet its format enabled us to track the flow of the information as well as the various actions and the ensuing relationships. The diagram follows after the figure below, which is a key to the symbols used.

Because of the density of the information included in the diagram, it might be initially more challenging to ‘read’ the image, but it immediately becomes apparent that a lot more people were involved in the exchange of information regarding the siege than has been shown in the network visualization, including the Dutch stadholder, Maurice of Orange, and Sir Francis Vere. Other relationships are also visible: in her letter to Bodley (letter id. 29), Queen Elizabeth asked him to convey the Queen’s message to the Council of State, the Estates General, and to Maurice of Orange. It is likely that Bodley went to see the members of these political bodies in person in order to pass on the information, hence links were created that existed outside of but are closely related to the epistolary network.

The diagram makes it clear that Bodley was not ‘just’ a correspondent, but also acted as a transmission agent, and the people who normally took care of transporting the letters, the bearers, can be easily included in the diagram as well (thus expanding the network and more clearly showing the different people who were involved in the transmission of information and who connected the various correspondents to one another). The sources from which Bodley derived his information are also shown: on July 14, 1594, Bodley wrote to Robert Cecil (letter id. 454) about the siege and he mentioned that he had received letters from the army camp at Groningen which provided him with information. The symbols indicate where other documents were enclosed with the letters sent between the correspondents: a letter package from Bodley to Burghley included a map about the siege of Groningen, one of the three maps about the siege Bodley sent to England.²

The SDL-diagram is one way of including various types of relationships and different flows of information that are difficult to include in many network visualizations (when using open source visualization software). Instead of depicting straightforward binary networks consisting of authors and recipients, we can zoom in closer, as it were, and show the material processes of collecting and disseminating information in more detail. Moreover, using such visualizations enables us to capture the complexity of the historical data as well as the diversity of the network. Arguably this comes at the cost, for it is difficult to visualize a large dataset in this way, but it opens up possibilities of visualizing networks without losing too much of the complexity and richness of the historical data which makes it so interesting to study in the first place. It also enhances our understanding of the often idiosyncratic process of gathering and spreading information and the fluid character of early modern information networks, aspects which tend to be ill-represented in neatly constructed network visualizations.

*This is the final blog in a series of posts from Dr Bernhard Haslhofer, one the recipients of the [DM2E Open Humanities Award](http://openhumanitiesawards.org). The final report is available here. *

Semantic Tagging in Maphub – Final Results and Lessons Learned

Maphub (http://maphub.github.io) is an open source Web application which allows people to annotate digitized historical maps. It pulls maps out of closed environments, adds zooming functionality, and assigns Web URIs so that people can talk about them on the Web. It has been built as a demonstrator for the W3C Open Annotation specification (http://www.w3.org/community/openannotation/), which currently works towards a common, RDF-based, specification for annotating digital resources. Here is a screenshot of the prototype application:

A first prototype (http://maphub.herokuapp.com) has been bootstrapped with a set of around 6,000 digitized high-resolution historical maps from the Library of Congress’ Map Division. It allows users to retrieve maps either by browsing or searching over available metadata and user-contributed annotations and tags.

Technical Details

Semantic tagging is part of Maphub’s annotation feature: to create an annotation, users markup regions on the map with geometric shapes such as polygons or rectangles. Once the area to be annotated is defined, they are asked to tell their stories and contribute their knowledge in the form of textual comments. While users are composing their comments, Maphub periodically suggests tags based on either the text contents or the geographic location of the annotated map region. Suggested tags appear below the annotation text. The user may accept tags and deem them as relevant to their annotation or reject non-relevant tags. Unselected tags remain neutral.

The screenshot in the next figure shows an example user annotation created for a region covering the Strait of Gibraltar. While the user entered a free-text comment related to the naming of the area, Maphub queried an instance of Wikipedia Miner (http://wikipedia-miner.cms.waikato.ac.nz/) to perform named entity recognition on the entered text and received a ranked list of Wikipedia resource URIs (e.g., http://en.wikipedia.org/wiki/Mediterranean_sea) in return. URIs should not be exposed to the user, so Maphub displays the corresponding Wikipedia page titles instead (e.g., Mediterranean Sea). Since page titles alone might not carry enough information for the user to disambiguate concepts, Maphub offers additional context information: the short abstract of the corresponding Wikipedia article is shown when the user hovers over a tag.

Once tags are displayed, users may mark them as relevant for their annotation by clicking on them once, which turns the labels green. Clicking once more rejects the tags, and clicking again sets them back to their (initial) neutral state. In the previous screenshot, the user accepts five tags and actively prunes two tags that are not relevant in the context of this annotation.

Sharing Annotations and Semantic Tags

Sharing collected annotation data in an interoperable way was another major development goal. Maphub is an early adopter of the Open Annotation specification and demonstrates how to apply that model in the context of digitized historic maps and how to expose comments as well as semantic tags. As described in the Maphub API documentation (http://maphub.github.io/api), each annotation becomes a first class Web resource that is dereferencable by its URI and therefore easily accessible by any Web client. In that way, while users are annotating maps, Maphub not only consumes data from global data networks – it also contributes data back. The following screenshot shows how the previous annotation could be represented following the Open Annotation specification.

Tagging Experiments

While working on Maphub, its semantic tagging functionality has become our core research interest. We conducted an in-lab user study with 26 participants to find out how semantic tagging differs from label-based tagging and learned that there was no significant difference in its tag production capacity, in the types and categories of tags added, and in overall user task load. Hence, semantic tagging as implemented in Maphub could produce the same result as a label-based tagging, with the main difference that semantic tagging gives references to unambiguous Web resources instead of semantically ambiguous labels. More details on the methodology and results of that experiment are described in our report available at (http://arxiv.org/abs/1304.1636).

Enabling Annotations and Semantic Tagging in other Applications

We found that semantic tagging might be useful for other application scenarios as well. Therefore, with the support we received from the Open Humanities Award, we added a semantic tagging feature to Annotorious (http://annotorious.github.io/), which is a JavaScript image annotation library that can be used in any Website. Annotorious is also compatible with the Open Knowledge Foundation’s Annotator (http://annotatorjs.org/) tool. Our next research and development steps will go into two main directions: (i) providing a more efficient and lightweight (semantic) tag suggestion service, and (ii) improving tag recommendation strategies.

*This is a guest blog post by Jaap Geraerts. Jaap works as part of the team that won the DM2E Open Humanities Award at the Center for Editing Lives and Letters.*

[In the previous update](https://dm2e.eu/open-humanities-awards-joined-up-early-modern-diplomacy-update-4/) we touched upon the relationship between the data and the software used for creating visualizations and this blog expands on this topic, for it is becoming increasingly apparent that -‘modern’- research techniques such as visualizations and network analysis have their own pitfalls. [In a perceptive blog post](http://www.scottbot.net/HIAL/?p=39600), Scott Weingart raises the question when the use of networks is and is not appropriate, and he rightfully states that network structures can be deceitful, partly because the algorithms used in network software are based on specific assumptions. Moreover, the data used for a network analysis can be biased or skewed, obscuring relationships between people or over-emphasizing the centrality of a person within a network. Networks also contain only so much information: the network structure of the data-set of our ‘Joined Up Early Modern Diplomacy project’, for instance, consists of the links between people that are solely based on their correspondence, leaving out the whole range of other relationships derived from family ties, friendships, et cetera. The relationship between Robert Cecil and William Cecil, for example, did not merely consist of the two letters the latter (father William) sent to the former (his son Robert). To give another example from the Bodley data-set: Anthony and Francis Bacon individually corresponded with Bodley, yet outside this epistolary network these brothers shared at least one link (kinship). This points at the existence of various networks of which people were part and the different positions people had in these networks (i.e. even though a certain person may have been a central figure in an epistolary network, this does not mean that he is in the centre of other networks in which he operated). For example, whereas person A is central in an epistolary network (he corresponded with B and C, two people who did not write to each other), in the social network of these three people, person B was central (having met person A once, and person C very often – B and C were cousins – while A and C never met). As such, network structures and visualizations of networks only show a part of ‘reality’, as networks are, naturally, dependent on the data with which they have been created.

Another difficulty when aiming to produce visualizations based on historical data can be the lack of context or points of reference for the viewers. Consider the following visualization, for instance:

Even without knowing what these blue dots mean, the viewer instantly recognizes that this image tells us something about the United States of America. The image, which shows the number of people who board or alight a flight (and where), can be explained very easily. Even though much is left unexplained in this image, the fact that the context is rather obvious and the fact that the data easily can be used to show density, makes this an effective visualization. However, in other cases the density of a network structure and the often resulting [hairballs](http://www.vpascual.org/blog/wp-content/uploads/2012/06/Screen-Shot-2012-06-17-at-20.16.02-.png), unintelligible clutters of nodes and edges, are more confusing than explanatory. When aiming, for instance, to include another layer of data into an epistolary network (such as the people mentioned in the letters), the result can become something like this:

This is an extreme example as the visualization has not been modified by using algorithms and filters, but it exemplifies how easily visualizations can become meaningless when using substantial layers of data in combination with the lack of a clear context. It therefore behoves the scholar to consider the particularities of the data and its possible limits as well as the aim of the visualization before enthusiastically pouring the data into computer programs. Furthermore, how to integrate the visualizations with the scholarship that underlies the visualization? And how do the visualizations tie in with the aims of the research and in what way do they enrich this research?
Possible ways of dealing with complex data can be to use different types of visualizations which show aspects or parts of the data set, perhaps to elucidate an aspect of the dataset which is otherwise difficult to perceive. Another option is to focus on a specific part of a larger visualization, or to approach the data from a specific angle (e.g. how did topic X flow through an epistolary network) in order to highlight the specifics of the network. Although countless options are imaginable, incorporating various visualizations into a narrative structure is a potential way of dealing with a complex data set, as the text can provide the much-needed (historical) context while also explaining the limits of the visualization to the reader. This does not mean that each visualization is accompanied by a lengthy explanation, but rather that the text and the visualizations support each other so that visualizations are not merely an addition to a story, but become part of it.

The point is that complex data can be visualized, but often at the cost of losing some of the complexity which makes the data (or sources) so interesting to study in the first place. When standing on their own, modern research techniques such as visualizations do not always add significantly to the existing scholarship: the crux is to combine these innovative techniques with more ‘traditional’ scholarship and to integrate the methodologies that are used for the gathering and mining of archival data in order to be able to push the boundaries of the research undertaken in the fields in which we are working.

*This is a guest blog post by Jaap Geraerts. Jaap works as part of the team that won the DM2E Open Humanities Award at the Center for Editing Lives and Letters.*

Since the [last update](https://dm2e.eu/open-humanities-awards-joined-up-early-modern-diplomacy-update-3/) about the Joined Up Early Modern Diplomacy project I have devoted a bit of time to the rationalization of the databases which have been created for this project. As the data of the Bodley project is stored in two databases (an ACCESS database and a MySQL database which powers the website) and both of the databases will be used to create the visualizations, it is important to ensure that both databases contain similar data. Moreover, we tried to see which data that was stored in the ACCESS database could be included in the MySQL database in order to enhance our understanding of Bodley’s correspondence network. The XML-files which contain the transcriptions of the letters that are visible on the website of the project had to be updated as well in order to keep them aligned with the updated databases, and all of this shows the work which precedes the creation of the actual visualizations – and I have not even started talking about the whole process of thinking about which visualizations are worth making, a topic which will be addressed in the next blog.

After the process of populating and updating the databases it is time to take the next step towards creating the visualizations, which is the prepare the data to be imported into GEPHI, the software we use to construct the visualizations. As GEPHI requires the data to be presented in a specific format which enables the software to connect the authors to the recipients of the letters and thus to construct the network, the data has to be exported from the database in a particular way.

Moreover, the way GEPHI thus looks at data poses interesting questions about how we view historical data ourselves. For instance, the issue of how to represent the fact that a letter had two or more authors in GEPHI raises the questions whether we should see these historical figures acting as one entity when writing such a letter. In other cases, especially when aiming to move beyond the ‘mere’ visualization of Bodley’s network by including other layers of information, such as the people and places mentioned in the letters, the question is how to capture the historical context and the wealth of the primary sources into a standardized piece of twenty-first-century software. Furthermore, the editorial decisions made by the research team in the development stage of the correspondence project meant that ‘correspondence’ was a fluid term: the bulk of the corpus comprises letters directly to or from Bodley, but also includes items sent in letter packets which, although epistolary in concept, do not necessarily have an addressee (or one that is immediately apparent, e.g. Bodley’s passport and cipher).

The examples given above bear witness to the fact that when using IT-software the researcher is obliged to engage in a dialogue between the software and the historical sources, and it is exactly at this point that IT-skills and the skills of a historian intersect. In addition, these examples serve as a reminder that while IT-software is able to create new insights and helps to address new research questions, a lot of extra work is necessary in order to gain the desired results, which in turn adds scholarly value to the technical resource. In this sense, it is important to remember that the tools embraced by the research taking place within the digital humanities do not magically provide extremely interesting results – rather, using some of these tools is like opening the box of Pandora. In this rapidly changing field of research, then, traditional skills such as scholarly diligence are needed more than ever.