» back to normal layout
WarMemoirSampo:
An instance of the VideoSampo Framework

Project Description

WarMemoirSampo is a Linked Open Data (LOD) resource of Finnish Second World War (WW2) veteran interview videos, as well as a semantic portal for easy access to them. The system is being realized using the Sampo model and by enriching the videos with related information from the WarSampo knowledge graph. WarMemoirSampo hosts a collection of video interviews of Finnish WW2 veterans, mostly reminiscing about their lives during and after wartime. Rough transcriptions of the interviews have been provided, which form the basis of the textual information presented in the portal and the metadata extracted from them.

A key technical challenge addressed in this work is how to search and access different temporal points in long videos, based on their time-stamped transcriptions. In order to achieve this, we created an RDF graph featuring data for the interviews as well as the interviewees. The main building blocks of the graph are coarse summary notes written by the interviewers, alongside their corresponding timestamps. However, the timestamps lack precision, since they may be repeated over multiple notes. These circumstances led us to our basic unit of data for the interviews: a group of notes that share the same timestamp, corresponding to a given stretch of the interview. They are of varying length, but typically several minutes long. The textual contents are enriched semantically using NLP techniques and knowledge extraction, resulting in new metadata about mentioned named entities - e.g., people, places, and events - and keywords generated via a pre-trained subject indexing tool.

The WarMemoirSampo portal is implemented using the Sampo-UI framework, which enables faceted search, exploration and analysis of the interviews. It is possible to identify interviews from specific interviewees and/or based on mentions of places, persons or subject matters (keywords) of interest. The results of the search take the user to the relevant parts of the video interviews, so that the veterans can be heard in their own voices. A semantic recommender system provides the user with links to related interview snippets present in the database, as well as additional information in WarSampo.

More features are planned for the future: named entities will be linked to the relevant resources in WarSampo and other Sampo portals, contributing to the growing web of Finnish linked data. An event detection tool is to be developed which extracts event information using times and places mentioned in the interviews. Moreover, when Finnish speech-to-text technology advances to the point that everyday dialectal speech can be automatically and reliably transcribed, the same tools could be used on the transcriptions, resulting in richer and more accurate metadata.

Video about WarMemoirSampo

Portal On-line

The portal WarMemisSampo was published on December 3, 2021, at the National Archives of Finland, and is in use at:

https://sotamuistot.arkisto.fi

More Information

More information is available at the Finnish homepage.


Publications

2023

Eero Hyvönen: Military History on the Semantic Web: Lessons Learned from Developing Three In-use Linked Open Data Services and Semantic Portals for Digital Humanities. Digital Humanities and Intelligent Computing of Cultural Heritage: Global Development and China Solutions, Routledge, September, 2023. Book chapter, in press. bib pdf
Eero Hyvönen: Digital Humanities on the Semantic Web: Sampo Model and Portal Series. Semantic Web – Interoperability, Usability, Applicability, vol. 14, no. 4, pp. 729-744, IOS Press, 2023. bib pdf link

2022

Eero Hyvönen, Esko Ikkala, Mikko Koho, and Rafael Leal, Heikki Rantala and Minna Tamper: How to Search and Contextualize Scenes inside Videos for Enriched Watching Experience: Case Stories of the Second World War Veterans. The Semantic Web: ESWC 2022 Satellite Events, Lecture Notes in Computer Science, vol. 13384, pp. 163-167, Springer, July, 2022. bib pdf link
Mikko Koho, Rafael Leal, Esko Ikkala, Minna Tamper, Heikki Rantala and Eero Hyvönen: Building Lightweight Ontologies for Faceted Search with Named Entity Recognition: Case WarMemoirSampo. Proceedings of the 1st International Workshop on Knowledge Graph Generation From Text and the 1st International Workshop on Modular Knowledge co-located with 19th Extended Semantic Conference (ESWC 2022) (Sanju Tiwari, Nandana Mihindukulasooriya, Francesco Osborne, Dimitris Kontokostas, Jennifer D’Souza and Mayank Kejriwal (eds.)), vol. 3184, pp. 19-35, CEUR Workshop Proceedings, May, 2022. International Knowledge Graph Generation From Text (TEXT2KG). bib pdf link
Rafael Leal, Heikki Rantala, Mikko Koho, Esko Ikkala, Markus Merenmies and Eero Hyvönen: WarMemoirSampo: A Semantic Portal for War Veteran Interview Videos. DHNB 2022 The 6th Digital Humanities in Nordic and Baltic Countries Conference, CEUR Workshop Proceedings, long papers, Vol. 3232, March, 2022. bib pdf link

2021

Eero Hyvönen: Sammon taontaa semanttisessa webissä (Forging Sampos on the Semantic Web). Tekniikan Waiheita, vol. 39, no. 2, pp. 87-105, Tekniikan Historian Seura ry, July, 2021. bib pdf link
/var/www/html/include/secoweb/utils.php; Thu, 28 Mar 2024 17:49:56 +0000