São Paulo's Duplicate Image Problem: The Key Decisions That Will Shape the City's Digital Archives

São Paulo's public institutions are sitting on a growing crisis that has nothing to do with flooding or potholes. Municipal agencies, cultural foundations, and the city's sprawling network of tech startups are wrestling with massive stores of duplicate digital images — photographs, scanned documents, satellite frames — that clog servers, inflate licensing costs, and make archival retrieval slower and less reliable. The question now is not whether to act, but who acts first and how.

The problem has sharpened in 2026 because two forces converged. Brazil's Lei Geral de Proteção de Dados, the national data privacy framework, entered a stricter enforcement phase this year, putting new legal weight on how institutions store and manage personal images. At the same time, cloud storage prices in Brazil have climbed in real terms following currency fluctuations that pushed dollar-denominated infrastructure costs higher. For agencies running large visual databases on tight orçamentos, every duplicated file now has a measurable price tag.

Where the Pressure Is Felt Most

The Arquivo Público do Estado de São Paulo, headquartered on Rua Voluntários da Pátria in Santana, holds decades of digitised photographic records and has been expanding its public-access portal since 2023. Staff there have acknowledged in public sessions that deduplication — the technical process of identifying and removing redundant image files — is unfinished work. The institution has not publicly committed to a completion timeline.

Across town at the Memorial da América Latina, near Barra Funda, the challenge is slightly different. The complex's cultural programming generates continuous new photographic output, and its digital management systems were designed in an earlier era before storage costs and compliance scrutiny reached current levels. The memorial, funded partly through the Secretaria de Cultura e Economia Criativa do Estado de São Paulo, faces pressure to modernise its asset management without a dedicated technology budget line for the current fiscal year.

São Paulo's tech unicorn ecosystem adds a commercial dimension. Startups operating out of hubs along Avenida Faria Lima and the Vila Olímpia corridor sell image-processing software to municipal clients across Latin America. Several of those companies have positioned AI-driven deduplication tools as a core product offering in 2025 and 2026, pitching city hall procurement officers directly. Whether Ricardo Nunes's administration — which has prioritised infrastructure spending — will fund those contracts is an open budget question heading into the second half of 2026.

What the Next Six Months Will Decide

Three decisions will determine the pace and shape of any resolution. First, the city's Secretaria Municipal de Inovação e Tecnologia must finalise its digital governance framework, a process that has been in consultation since late 2024. Without that framework, individual agencies will continue making incompatible choices about which deduplication standards to adopt, producing fragmentation rather than efficiency.

Second, the state legislature's technology committee is expected to take up a proposal before the August recess that would require public cultural institutions to conduct annual audits of their digital storage, including identification of redundant files. If passed, that proposal would give institutions like the Arquivo Público a legislative mandate — and theoretically a budget argument — for deduplication spending.

Third, federal procurement rules updated by the Lula administration in 2025 opened a faster pathway for municipalities to contract AI-based software tools without full tender cycles, provided the contracts fall below R$700,000. That threshold puts several deduplication platforms within reach of a direct-hire arrangement, which could move implementation from a planning document to a production environment within months rather than years.

For residents and researchers who depend on São Paulo's public digital records — journalists pulling archive photographs on Paulista Avenue demonstrations, historians tracing urban development in Brás and Mooca — the practical stakes are straightforward. Bloated, redundant archives return slower search results, produce false matches, and consume budget that would otherwise go toward expanding public access. The technical fix is not complicated. The governance will inside the city's institutions, and the political will at Prefeitura level, are where the real uncertainty lives.