How to structure a corpus of text: An Introduction
Please find below the outline of the presentation prepared by Marie Bizais-Lillig (Université de Strasbourg, consortium HN distam) for the distam-RecentGlobe summer school in 2025. A selection of examples is meant to illustrate and give an overview of the possibilities and requirements of XML-TEI encoding.
What do we mean by “structure”? From a stream of characters, words, thoughts, informations, to a (documented, enriched) structure.
At the end of the session, it should be clear:
- – what we mean
- – why we want/need to structure
- – how we choose to do it
1. Available texts
Two main issues:
- – licenses
- – structure
1.1. Licenses
A few examples :
- – ctext
- – Invisible East
- – OpenITI
1.2. Structure
A few examples :
- – kanripo
- – wikisource 1
- – wikisource 2
- – CKP
For under-resourced languages, periods, genres, we might need to acquire/edit new texts, or at least “structure” them.
Once it’s done, we choose to share the results of our work to spare time for others – for an Open Science.
2. Full Text versus Structure
2.1. Displayed texts online, the aim?
- – OpenArabic
- – Racines
- – Strasbourg
2.2. Exploration, Text Mining
Two examples (among many other possibilities):
- – Overlap
- – Citations (who is cited when)
Organizing one’s dataset, for example:
- – texts
- – references
3. File format
We present a standard called TEI (Text Encoding Initiative) which relies on the XML file format, because it is a standard, commonly used for text editing. Other choices are possible, but one should thoroughly reflect on this issue before choosing any niche method (especially if it lacks documentation). For discussion?
3.1. Our XML files
Two examples:
Basic structure (a tree):
Each XML TEI File (the root) is composed of two parts:
- header
- text
Each part can the be subdivided.
The different parts, segments, elements are identified:
- – between tags
- – tag names are enclosed between angle brackets
- – tags are case-sensitiv
- – opening tag, text, closing tag: <l>this is a line in a poem</l>
Attributes (@): additionnal information enclosed withing the tags (within the angle brackets)
3.2. Choosing tags
The Guidelines: here
Choosing tags is:
- – not so straightforward
- – many possibilities, flexible
- – might be revised
*Do not forget to document your choices!*
4. Editing methods
4.1. Editing manually
Tagging can be done manually. It should be based on a precise documentation (.odd, relaxng files) to make sure that only the selected tags are used and that they are properly used.
Editing environment:
- – Oxygen (with license)
- – Studio Code/ Codium, with extensions, see our tutorial
4.2. Semi-automatic edition
Choice of tags cannot be shortened, but it can be quick if:
- – you choose a plain structure
- – you replicate choices you have made before
See:
There are also tools for specific tasks, ex. Markus for named entities (developed for texts in Classical Chinese, but that may be used in almost any language): Markus
To conclude
- – Very efficient once the corpus is structured
- – Time consuming (carefully choosing tags is mandatory)
- – Offers a new perspective on our body of text: raises questions throughout the editorial process
- – Reusable data if we stick to FAIR/Open principles (files might need restructuring to be used in other projects, not more)
OpenEdition vous propose de citer ce billet de la manière suivante :
Marie Bizais-Lillig (9 juillet 2025). How to structure a corpus of text: An Introduction. Carnet hypothèses du consortium Huma-Num DISTAM (DIgital STudies Africa, Asia, Middle East). Consulté le 14 janvier 2026 à l’adresse https://doi.org/10.58079/14ay1

