Ballarat's cultural institutions move to fix duplicate image problem plaguing digital archives

Ballarat's two largest public cultural repositories took concrete steps this week to address a long-running problem with duplicate images clogging their digital collections — a backlog that has frustrated researchers, slowed public access and, in some cases, caused the same photograph to appear under conflicting captions and dates.

The push comes as heritage digitisation projects funded through Regional Arts Victoria and the State Library of Victoria's community collections program have pushed thousands of new scans online since 2023, compounding an existing problem that predates the current digital era. When analogue collections were first migrated to online platforms in the mid-2010s, many institutions simply uploaded everything, duplicates included, rather than spend limited resources on deduplication before publication.

What happened this week

Staff at the Art Gallery of Ballarat on Lydiard Street North confirmed they have begun a structured audit of their online image repository, focusing first on the Eureka and goldfields photography collection, which holds more than 4,200 digitised items. The audit, which started on Monday, July 1, is expected to run through September. The gallery is using open-source image-matching software to flag near-identical files before human curators make final decisions on which version to retain and which to retire or merge.

Separately, the Ballarat Heritage Services team at the Ballarat City Council offices on Sturt Street flagged the same issue in a working paper circulated to councillors ahead of next week's ordinary meeting. The paper notes that the council's own historical photograph collection — accessible through the Central Highlands Regional Library Corporation's digital portal — contains an estimated 600 to 800 confirmed duplicate entries, some carrying different metadata that has misled researchers about the dates and locations of significant local events.

Sovereign Hill's archival team is also involved. The living museum on Bradshaw Street has been collaborating with the Ballarat and District Genealogical Society to cross-reference its own holdings against the library corporation's portal, identifying overlapping material from the 1850s to 1880s goldfields period. The genealogical society, which holds regular meetings at its Drummond Street North premises, flagged at least 140 image pairs as high-priority duplicates — photographs where two separately catalogued entries appear to show the same subject, scene or individual under different accession numbers.

Why the cleanup matters now

The timing is not accidental. Ballarat is mid-way through a four-year cultural infrastructure investment cycle that runs to 2027, and institutions receiving state grants are under growing pressure to demonstrate collection quality, not just collection size. The Regional Museum Partnership Program, administered through Creative Victoria, ties a portion of ongoing operational funding to measurable improvements in digital access and metadata accuracy — meaning duplicates are no longer just an inconvenience but a compliance issue.

There is also a practical cost. Server storage for uncompressed archival image files is not cheap: high-resolution TIFF files used for heritage photography typically run between 50 and 150 megabytes each, and the Art Gallery of Ballarat's repository alone occupies roughly 1.8 terabytes of managed cloud storage, according to figures the gallery has shared in previous annual reports. Eliminating confirmed duplicates could meaningfully reduce that overhead.

For Ballarat residents who use the library corporation's digital portal for family history research — one of the most popular non-circulation uses of the system — cleaner metadata will mean fewer dead ends. A photograph of Sturt Street in 1882 catalogued under three different dates is not a trivial error when a researcher is trying to establish when a particular building was constructed or demolished.

The council's working paper recommends allocating a one-off amount from the existing heritage maintenance budget to bring in a specialist metadata consultant, with a decision expected at the July 15 ordinary meeting. The Art Gallery of Ballarat's audit is proceeding with existing staff resources. Anyone with corrections to submit about specific historical images can contact the gallery directly through its Lydiard Street North office or via the Central Highlands Regional Library Corporation's online feedback form.