The Dark Archive Problem
Almost every library, broadcaster archive, university collection, museum, and public records office holds a substantial body of audiovisual material that is, functionally, invisible.
The material is catalogued at the item level — a title, a date, a creator, a duration, perhaps a paragraph of description written by whoever accessioned it. But the content is unindexed. Nobody knows which of four hundred oral history interviews mentions a particular factory closure, which of two thousand council meeting recordings contains a specific planning decision, or which lecture in a decade of recordings covers a particular topic.
A researcher cannot find any of it without watching, and watching four hundred hours to find one mention is not a research method. So the material sits, preserved at considerable expense, and used almost not at all.
This is the single largest and most tractable problem in audiovisual collections management, and the economics of solving it have changed fundamentally. Bulk transcription that would once have cost more than the collection's entire annual budget is now a batch job. The constraint has moved from cost to process design.
Transcription Is the Foundational Layer
Everything an archive wants to do with audiovisual material depends on having the words as text.
From a transcript you derive: full-text search across the collection, timecoded results that take a researcher to the exact moment, automatically suggested subject terms, named entity extraction for people, places, and organisations, accessibility captions, translation into other languages, and quotation with accurate citation.
Without a transcript, none of these are possible at scale, and every one of them requires a human to watch the material first.
The strategic implication is that bulk transcription should generally precede everything else. Institutions sometimes begin with selective high-quality treatment of highlight items, which produces excellent results for a tiny fraction of the collection and leaves the rest dark. Transcribing everything at moderate quality, then enhancing selectively, serves researchers far better — because an imperfect transcript makes an item findable, and findability is the binding constraint.
A related point worth making internally: an imperfect transcript is not a cataloguing statement. It is a finding aid. Archivists reasonably resist publishing anything below their descriptive standards, and the resolution is to label machine-generated transcripts clearly as such, distinct from human-verified description, rather than to withhold them.
Historical Audio Is Genuinely Hard
Archive material differs from contemporary content in ways that materially affect transcription quality, and expectations should be set accordingly.
Recording quality. Wax cylinder, acetate disc, open-reel tape, and early video formats carry noise, dropout, wow and flutter, and limited frequency response. Degradation from storage compounds this.
Historical speech patterns. Accents, dialects, and pronunciation shift across generations. Regional varieties that were common decades ago may be poorly represented in the data modern systems were built on.
Obsolete vocabulary. Institutional names, job titles, technologies, place names, and administrative terms that no longer exist. Systems will substitute plausible modern equivalents.
Recording conditions. Single microphones in rooms, crowd noise, no isolation between speakers, and equipment operated by people learning to use it.
Language change and code-switching. Older recordings frequently contain languages or dialects that have since declined, and speakers switching between them.
Realistic expectations follow. A clean 1990s studio recording transcribes close to contemporary accuracy. A 1940s field recording of a regional dialect will produce a transcript with substantial error, and that transcript may still be enormously useful as a finding aid — a researcher searching for a place name may well find it even in a flawed transcript, and then verify by listening.
The practical guidance: transcribe everything, record a confidence indication where available, label machine output clearly, and prioritise human verification by research demand rather than by attempting to verify the entire collection.
Terminology for Collections
Archives have an advantage most organisations lack: they already maintain authority files and controlled vocabularies. Connecting these to the transcription workflow substantially improves results.
Load as custom vocabulary:
- Personal name authority records for individuals likely to appear.
- Corporate and institutional name authorities, including historical forms of names that have since changed.
- Place name authorities, including historical and superseded names.
- Subject headings and thesaurus terms in use.
- Collection-specific terminology, technical vocabulary, and local jargon.
- Where a collection concerns a specific industry, community, or region, its particular lexicon.
This matters more for archives than for most content because the vocabulary is precisely what modern systems handle worst — a defunct company name, a renamed street, a job title from a vanished industry. Loading the authority file is often the difference between a usable and an unusable transcript for a specialised collection.
The reverse benefit is real too: entity extraction from transcripts surfaces names and places that were never catalogued, which can be reconciled back into the authority file. Collections routinely discover, through this process, that they hold material about people and topics nobody knew was there.
Metadata Generation
Once transcripts exist, a substantial amount of descriptive metadata can be derived, which addresses the perennial cataloguing backlog.
Derivable with reasonable reliability: named entities, topic and subject suggestions, timecoded segment boundaries, speaker counts and turn structure, language identification including code-switching points, and summary description.
The critical governance decision is how generated metadata is distinguished from professional description. The workable model:
- Generated metadata is stored in clearly separate fields, labelled by method and date.
- It is exposed to search — because that is the entire point — but presented to users as machine-generated.
- It is never silently promoted into authoritative descriptive fields.
- A workflow exists for a cataloguer to verify and promote specific elements.
- Provenance is recorded, including which system and version produced it, in line with normal archival practice around processing history.
This preserves descriptive integrity while making the collection searchable, and it is the model that most institutions adopting this approach have converged on.
Rights, Sensitivity, and Consent
Making a collection searchable changes its practical accessibility, and that can surface issues that physical obscurity was concealing.
Obscurity was doing work. Material that was technically open but practically unfindable becomes genuinely accessible once indexed. Content that nobody objected to while it sat unlistened may attract attention when a full-text search surfaces a specific passage.
Personal data in transcripts. Oral histories, meeting recordings, and interviews contain names, addresses, health information, and details about third parties who never consented to anything. A transcript makes this searchable in a way the recording did not.
Consent scope. Consent obtained decades ago for deposit in a physical archive frequently did not contemplate online full-text searchability. Whether it extends is a genuine question, not a formality.
Third parties. People discussed in a recording did not consent at all, and may be identifiable and living.
Culturally sensitive material. Indigenous and community collections may carry protocols about who may access what, which a general full-text index can override inadvertently.
Rights in the recording. Transcription creates a derivative work, and archives should confirm their rights position, particularly for deposited third-party material.
Practical approach: run sensitivity review as part of the transcription programme rather than after publication; support tiered access so a transcript can be indexed for staff and restricted for public search; provide a takedown and review process; and consult source communities for collections where protocols apply. Some material should be transcribed for preservation and finding-aid purposes without being publicly searchable, and the system needs to support that distinction.
Preservation Standards
Transcripts and derived metadata are themselves archival records and should be treated accordingly.
Practical requirements: store transcripts in open, well-documented formats rather than proprietary ones; retain timecode alignment to the source so text and media remain linked; record full provenance including method, system, version, and date; version transcripts so corrections do not destroy earlier states; store them alongside the media in the preservation system rather than only in an access platform; and ensure they are covered by the same backup and integrity-checking regime as the media.
The point about the access platform matters. A transcript that exists only inside a vendor's search index is not preserved. It needs to live in the institution's own repository, in a format that will still be readable when the current access system has been replaced.
Multilingual Collections
Many collections contain material in multiple languages, and some contain material in languages the institution has no staff competence in at all.
Approaches:
Transcribe in the source language first. The primary artefact is an accurate record of what was said. Translation is a secondary layer.
Translate for discovery. Translated transcripts allow a researcher who does not read the source language to determine whether an item is relevant, then seek proper translation if it is. This is enormously valuable and does not require the translation to be publication quality.
Label translation status clearly. A machine translation of a machine transcription of a degraded recording carries compounded uncertainty, and users must be able to see that.
Prioritise by demand and by risk. Material in languages with declining speaker populations may be a preservation priority regardless of current research demand.
Engage source communities. For community collections, involving speakers in verification improves quality and is the appropriate practice in its own right.
A Realistic Programme
Start with a pilot on one well-understood collection with reasonable audio quality, to calibrate expected accuracy and establish the workflow.
Load your authority files as custom vocabulary before processing anything at volume. This single step disproportionately affects quality on specialised collections.
Process broadly rather than deeply. Getting a moderate transcript of the whole collection serves researchers better than an excellent transcript of two percent of it.
Build the sensitivity review into the pipeline rather than treating it as a post-publication problem.
Publish with clear labelling of machine-generated content and an obvious correction path — researchers who use the material are your best verification resource and are generally willing to contribute corrections if there is a mechanism.
Then enhance selectively, driven by actual research demand rather than by curatorial assumption about what matters.
Frequently Asked Questions
Is machine transcription accurate enough for archival use?
For finding-aid purposes, generally yes, including on imperfect audio — a researcher searching for a name will often find it even in a flawed transcript and can then verify by listening. For authoritative description or quotation, no; those require human verification. The workable model is to transcribe everything at moderate quality, label it clearly as machine-generated, keep it separate from professional description, and verify selectively based on research demand.
How well does it handle historical recordings?
Variably, and expectations should be set by format and era. Clean recent studio recordings approach contemporary accuracy. Early field recordings with regional dialects, format degradation, and obsolete vocabulary produce substantial error. Loading your authority files and collection-specific vocabulary before processing significantly improves results on specialised material.
Does making a collection searchable create new rights problems?
It can surface existing ones. Material that was technically open but practically unfindable becomes genuinely accessible, consent obtained decades ago may not have contemplated online full-text search, and third parties discussed in recordings never consented at all. Build sensitivity review into the programme, support tiered access so material can be indexed internally without being publicly searchable, and provide a review and takedown route.
How should generated metadata relate to catalogue records?
Store it in clearly separate, labelled fields with recorded provenance, expose it to search, present it to users as machine-generated, and never silently promote it into authoritative descriptive fields. Provide a workflow for cataloguers to verify and promote specific elements. This keeps descriptive integrity intact while making the collection findable.
Where should transcripts be stored?
In your own preservation repository, in open documented formats, with timecode alignment to the source retained, full provenance recorded, versioning for corrections, and coverage by the same backup and integrity-checking regime as the media. A transcript that exists only inside a vendor's search index has not been preserved.
Related reading: Video to Transcript Guide | Audio to Text Converter Guide | Video Accessibility Guide
