São Paulo Tackles the Duplicate Image Problem Clogging Its Digital Infrastructure — and Asks Why It Took So Long

São Paulo's municipal digital archive holds an estimated 40 million scanned documents, photographs and georeferenced images accumulated since the city launched its digitisation push in 2018. A significant portion of that storage — city technology contractors have placed the figure at roughly 30 percent in internal assessments — consists of duplicate or near-duplicate image files generated by overlapping scanning projects, competing departmental workflows and years of uncoordinated data uploads. The price tag for unnecessary cloud storage alone has become a line item that the Nunes administration can no longer quietly absorb.

The problem matters now because São Paulo is mid-way through a R$2.1 billion smart-city infrastructure upgrade contracted through the Secretaria Municipal de Inovação e Tecnologia. Duplicate image data is not merely wasteful; it degrades search performance, slows emergency-response mapping systems and creates legal exposure when conflicting versions of the same cadastral record are retrieved during property or zoning disputes. The Paulista Avenue corridor redevelopment project, for instance, has seen planning documents scanned by at least three separate municipal departments since 2022, producing version conflicts that have delayed permit reviews.

The city's main response has been to pilot an automated deduplication program through CET — Companhia de Engenharia de Tráfego — which manages the largest single photographic dataset in municipal hands: roughly 1.4 billion traffic-camera stills archived annually. Working with the Instituto de Pesquisas Tecnológicas, the IPT facility on Avenida Professor Almeida Prado in Butantã, technicians began running perceptual-hash deduplication algorithms across CET's archive in March 2026. Early results shared at a municipal technology forum in May suggested the process had flagged more than 200 million redundant frames in the first two months of operation alone.

How São Paulo Compares to Peer Cities

New York City's Department of Information Technology and Telecommunications completed a citywide image deduplication audit across its 47 agency data repositories in 2023, recovering an estimated 18 petabytes of storage and reducing annual cloud expenditure by roughly $6 million, according to figures the agency published in its 2024 annual report. Amsterdam's municipal digital infrastructure office finished a similar exercise by 2022, consolidating its urban-planning image archive under a single content-addressable storage system that cut retrieval times for planners at the Ruimte en Duurzaamheid directorate by more than half. Nairobi, whose City Hall partnered with a Kenyan fintech consortium in late 2024 to deduplication-audit its land-registry image bank, recovered storage equivalent to four years of new data accumulation within six months.

São Paulo's effort is arriving later than those of its peer cities, and its structure is more fragmented. Rather than a single citywide audit authority, the work is being divided across individual secretariats. The Secretaria Municipal de Urbanismo e Licenciamento is running its own deduplication review of the building-permit image archive housed in the Lapa district data centre, while the Arquivo Histórico Municipal on Rua Voluntários da Pátria in Santana is conducting a separate exercise for its cultural heritage photograph collection, estimated at 3.2 million items dating back to 1892. Critics of this silo approach argue the city is recreating the coordination failures that generated the duplicate problem in the first place.

What Comes Next for Residents and Businesses

For residents filing building permits or environmental licensing requests in neighbourhoods like Vila Madalena or Pinheiros, the practical effect of successful deduplication should be faster online document retrieval through the SP156 platform and fewer instances of the system returning conflicting cadastral maps. The Nunes administration has indicated it aims to have a unified image-data governance policy drafted by the fourth quarter of 2026, which would require all new municipal scanning projects to route through a central deduplication check before data is committed to long-term storage.

São Paulo's tech community in the Berrini and Faria Lima corridors is watching closely. Several local startups specialising in document intelligence — including firms that have grown out of the USP and Insper ecosystems — have already submitted proposals to the municipal procurement portal to provide AI-assisted deduplication tools. The city has until the end of August 2026 to publish the results of the current pilot phases, at which point the scale of the project, and the competition for the contracts that follow, will become considerably clearer.