How São Paulo's Digital Archives Became a Swamp of Duplicate Images — and What It Cost to Fix It

The number is striking: by the end of 2025, the Prefeitura de São Paulo's consolidated digital infrastructure held an estimated 4.7 petabytes of image data across its network of agencies, and independent auditors contracted by the Secretaria Municipal de Inovação e Tecnologia found that roughly 34 percent of that volume consisted of exact or near-exact duplicate files. The problem did not emerge overnight. It is the accumulated result of at least fifteen years of siloed digitisation drives, each well-intentioned, few coordinated.

The timing of the finding matters. Mayor Ricardo Nunes is pushing a broad smart-city agenda ahead of the 2026 municipal budget cycle, and infrastructure costs tied to redundant data storage have become an uncomfortable line item that councillors on the Câmara Municipal, on Viaduto Jacareí in the Centro district, are no longer willing to approve without scrutiny. Meanwhile, the city's flood-monitoring network — expanded significantly after the deadly inundations that swamped Jardim Pantanal and the Marginal Tietê corridor in early 2024 — depends on rapid image processing from hundreds of street-level cameras. When storage systems are clogged with duplicates, retrieval times slow, and in a flooding emergency that latency has real consequences.

How the Duplication Crisis Accumulated

The roots run back to roughly 2010, when the then-administration launched the Cidade Digital program, pushing departments to scan paper records and upload photographs of infrastructure, permits and public works. Each secretaria built its own repository. The Secretaria Municipal de Obras digitised street-level engineering surveys. The Companhia de Engenharia de Tráfego, known as CET, stored traffic-camera stills. The São Paulo Urbanismo agency archived aerial and satellite imagery of zoning areas from Pinheiros to Guaianazes. Nobody mandated a central deduplication protocol, and nobody checked whether the same image of, say, a cracked pavement on Rua da Consolação had been uploaded four times by four different departments responding to four different citizen complaints on the SP156 platform.

The problem compounded as storage got cheaper. Between 2015 and 2022, the cost per terabyte of cloud storage fell by more than 70 percent globally, which removed the financial pressure that would otherwise have forced housekeeping. Departments simply kept buying capacity rather than rationalising what they held. The 2023 audit by the Tribunal de Contas do Município — the city's public accounts watchdog — flagged the redundancy issue in a footnote but did not quantify it. The 2025 figure of 34 percent duplication came from a more granular review commissioned by the Nunes administration and completed in March of this year at a cost of R$2.3 million.

What a Fix Actually Looks Like

The Secretaria de Inovação e Tecnologia is now piloting a deduplication and hash-matching system across three agencies — CET, São Paulo Urbanismo and the Secretaria Municipal de Saúde — with full rollout planned for the first quarter of 2027. The pilot covers approximately 800 terabytes of image data housed in the DataCentro Anhangabaú facility near the Vale do Anhangabaú. Early results, circulated internally in June, suggest the process can recover around 28 percent of used storage in each department, potentially saving the city upward of R$18 million annually in licensing and infrastructure fees once applied at scale.

The lesson being absorbed slowly by city hall is administrative as much as technical. Duplicate image replacement — the process of identifying redundant files, designating a single canonical version and purging the rest — requires governance agreements between secretarias that have historically guarded their data as departmental assets. The SP156 complaints platform, which logged more than 4.1 million citizen requests in 2025 alone, generates image attachments that feed into at least six separate back-end systems simultaneously, each of which may store its own copy.

For residents and businesses in São Paulo, the practical payoff will come gradually. Faster camera-feed processing along the Corredor Norte-Sul could mean quicker traffic rerouting during the storms that reliably paralyse the city between November and March. For the city's tech sector, clustered around Faria Lima and the Berrini corridor, the pilot represents a potential procurement opening: the Prefeitura has signalled it will seek private-sector partners for the full rollout through a public tender expected to open in September 2026. Companies interested in the contract should monitor notices posted to the Diário Oficial do Município, where specification documents will appear at least 30 days before bidding closes.