São Paulo's Digital Archives Face a Reckoning: The Key Decisions Ahead on Duplicate Image Replacement

São Paulo's municipal and state agencies are sitting on tens of millions of redundant digital image files, and the window for making smart decisions about how to replace and consolidate them is closing. A review process that began under the Secretaria Municipal de Inovação e Tecnologia in early 2026 has put duplicate image replacement firmly on the agenda — and the choices ahead will shape how the city manages its data infrastructure for the next decade.

The urgency is not accidental. Federal digitization directives from Brasília, combined with pressure on city hall under Mayor Ricardo Nunes to modernize municipal services ahead of the 2028 infrastructure review cycle, have pushed the issue out of the IT department and into budget discussions. Duplicate images — scanned documents, urban monitoring photographs, satellite and drone imagery used for flood mapping — are costing storage money that could fund frontline services. For a city that spent much of June 2026 managing the aftermath of another round of flash floods in the Zona Leste, efficiency in data management is no longer abstract.

What the Problem Actually Looks Like on the Ground

The Centro de Operações São Paulo, headquartered near the Consolação district, processes thousands of images daily from traffic cameras, weather stations and the city's expanding network of environmental sensors. Staff there have flagged that duplicate files — identical or near-identical images stored multiple times across different systems — are inflating storage demands and slowing retrieval times during emergencies, including the drainage crisis response events that hit Itaquera and Guaianazes this past rainy season.

At the Arquivo Público do Estado de São Paulo on Rua Voluntários da Pátria in Santana, archivists have been digitizing physical collections for years. The problem of duplicates emerged partly because multiple departments commissioned scanning projects without a unified deduplication protocol. The result: parallel copies of the same historical photographs sitting in separate servers, with no single system able to identify and remove the redundancies automatically.

Industry data compiled by the Brazilian tech sector association Brasscom estimated in its 2025 annual report that Brazilian public institutions waste between 15 and 30 percent of cloud storage budgets on duplicate or near-duplicate files. For a city the size of São Paulo, with municipal IT spending running into the hundreds of millions of reais annually, even the lower end of that range represents a significant fiscal drain.

The Decisions That Will Define the Next Phase

Three choices are now sitting on decision-makers' desks. First, which deduplication technology to adopt: hash-based exact matching, which is cheap and fast but misses visually similar near-duplicates, or AI-driven perceptual hashing tools, which are more accurate but require procurement processes that can take six months or longer under São Paulo's municipal contracting rules.

Second, who owns the process. The Secretaria de Gestão has historically controlled archival policy, while the Secretaria de Inovação holds the tech budget. Turf questions between those two bodies have delayed similar digitization projects before, including a 2023 initiative to standardize document formats across the 96 subprefeituras.

Third — and most contentious — what happens to images flagged as duplicates but not confirmed as exact matches. Deleting files from a public institution's archive requires legal sign-off under the Lei de Acesso à Informação, Brazil's freedom of information law. A hasty purge risks destroying images that are similar but not identical, potentially wiping out documentation relevant to litigation, urban planning appeals, or journalism investigations.

Tech companies already operating in São Paulo's unicorn ecosystem, including those based in the Vila Olímpia and Faria Lima corridor, are watching the procurement decision closely. A contract of this scale — replacing and consolidating image libraries across multiple secretariats — would represent significant revenue. Startups specializing in computer vision and archival automation have already submitted unsolicited proposals, according to procurement filings published on the city's transparency portal.

The Secretaria Municipal de Inovação e Tecnologia is expected to publish a formal request for information by September 2026. That document will signal which direction city hall is leaning. Organizations with a stake in the outcome — from civic tech groups like Rede Nossa São Paulo to state archivists — have until that window opens to make their case about what a responsible duplicate image replacement program should actually protect.