How Ballarat's Digital Archives Fell Into the Duplicate Image Trap — and What It Cost to Fix

Ballarat's cultural institutions are midway through a painstaking audit of their combined digital image collections after years of ad hoc scanning projects produced a sprawling library riddled with duplicates — some records containing four or five versions of the same photograph, misfiled under different catalogue numbers and stored across incompatible systems.

The problem didn't emerge overnight. It is the direct result of how digitisation was funded in Victoria's regional centres: in short bursts, attached to grants, with little coordination between institutions operating just streets apart from one another.

A Patchwork of Grants and Good Intentions

The trajectory runs back at least to the mid-2000s, when organisations including the Ballarat Heritage Services unit within the City of Ballarat and the Art Gallery of Ballarat began digitising physical collections separately, each responding to different funding rounds from bodies such as Creative Victoria and, federally, the Australian Research Council's Linkage Infrastructure grants. Sovereign Hill also undertook its own photographic archiving work tied to the site's interpretive and tourism programs.

Each institution used the tools and metadata standards current at the time of their respective grants. Standards shifted. Staff turned over. By the time a more coordinated regional approach was discussed seriously — roughly around 2019 when the City of Ballarat published its Cultural Strategy — the underlying collections had diverged significantly. The same glass-plate negative of Sturt Street in the 1880s, for example, might exist as a high-resolution TIFF in one archive, a compressed JPEG in another, and a watermarked web-resolution copy in a third, none of them flagged as related records.

The scale matters for practical reasons beyond tidiness. Storage costs money. Cloud hosting for large cultural collections can run to tens of thousands of dollars annually for regional councils, and that bill rises with every redundant file retained. More importantly, duplicates degrade search quality: when a researcher or a teacher at a school in Wendouree queries an image database and receives six results that are essentially the same photograph, confidence in the archive drops.

Why the Problem Got Harder Before It Got Easier

Two factors accelerated the mess. First, the 2020–21 pandemic-era push to get collections online quickly — part of a broader response to closed venues — meant staff uploaded material fast, without the deduplication checks that slower, in-person projects allowed. Second, the rollout of new collection management software across several Victorian regional institutions created a migration window during which legacy files were bulk-imported rather than individually reviewed.

The Ballarat Mechanics' Institute on Sturt Street, which holds one of the most significant regional library collections in Victoria, flagged the problem formally to its board in late 2023 when a metadata review found a significant proportion of its digitised photographic holdings could not be distinguished from copies held elsewhere in the regional network. The institute does not have a dedicated digital archivist on permanent staff, which reflects a resource gap common across comparable institutions in the Central Highlands.

A coordinated remediation project is now underway, drawing on protocols developed by the Public Record Office Victoria and using automated deduplication tools cross-referenced against manual curatorial review. The work is being done in stages, prioritising the most-accessed parts of the collection first — the goldfields-era photographs that drive both tourism research and the heritage identity work tied to Ballarat's status as a UNESCO Creative City of Craft and Folk Art.

For institutions still holding physical-first collections with partial digital coverage, the lesson from Ballarat's experience is procedural rather than technological: deduplication policy needs to be written into grant acquittal requirements from the start, not treated as a cleanup task for the end of a project. The City of Ballarat's current draft digital asset management framework, expected to go before council later in 2026, reportedly includes exactly that kind of front-end requirement — though the detail will depend on what funding bodies are willing to mandate alongside their grants.

Until that framework is adopted and tested across a full grant cycle, archivists at institutions along Lydiard Street and beyond will keep doing what they have done for the past two years: working through backlogs, merging records, and trying to construct a single coherent picture from a decade of well-meaning but disconnected effort.