São Paulo's Digital Archives Are Drowning in Duplicate Images — and the Numbers Tell a Costly Story

São Paulo's municipal data infrastructure holds more than 40 million digitized image files, spanning urban planning permits, public health records, and decades of satellite mapping of the city's 1,521 square kilometers. A significant share of those files, according to internal assessments circulating among city technology contractors, are exact or near-exact duplicates — redundant copies that consume server space, inflate licensing costs, and slow the processing pipelines that São Paulo's secretariats depend on for everything from flood response to social benefit distribution.

The timing matters. Mayor Ricardo Nunes' city administration has pushed forward with a broad digitalization agenda since 2023, accelerating the migration of municipal records to cloud environments managed through contracts with vendors operating out of the Centro Empresarial Nações Unidas complex in Santo André and data centers along the Marginal Pinheiros corridor. As that migration accelerates, so does the scale of the duplicate-image problem — because files that lived in siloed departmental servers for years are now being consolidated into shared repositories where their redundancy becomes impossible to ignore.

What the Data Actually Shows

The economics are stark. Cloud storage in Brazil's tier-one enterprise market currently runs between R$0.08 and R$0.23 per gigabyte per month, depending on redundancy tier and provider — a range that São Paulo tech firms operating out of hubs like Cubo Itaú on Brigadeiro Faria Lima Avenue know intimately. For a municipal archive sitting on several petabytes of image data, even a 15 percent duplication rate — a conservative benchmark cited in data governance literature — translates to hundreds of terabytes of avoidable expenditure every billing cycle.

Brazil's National Data Protection Authority, the ANPD, which has been expanding its enforcement posture since the Lei Geral de Proteção de Dados came into full effect in August 2021, adds a compliance layer to the problem. Duplicate images of citizens — identity document scans, biometric photographs, health record imagery — mean that a deletion request under the LGPD requires finding and removing every copy. Organisations that cannot map their duplicates cannot reliably honor those requests, and the ANPD has signaled that enforcement actions are no longer confined to large private platforms.

The São Paulo State Court of Auditors, the Tribunal de Contas do Estado, flagged data storage inefficiency as a line item risk in its 2024 annual public accounts review, noting that overlapping digitalization projects across state and municipal agencies had generated structural redundancy in shared document repositories. That report did not publish a specific monetary figure for image duplication alone, but it identified storage rationalization as a priority recommendation for municipal technology spending.

How São Paulo's Tech Ecosystem Is Responding

The startup community concentrated around Paulista Avenue and the Vila Olímpia district has been building tooling around this problem for several years. Duplicate-image detection relies on a combination of perceptual hashing algorithms — which can identify visually identical images even when file metadata has changed — and machine-learning classifiers trained to flag near-duplicates where compression or format conversion has altered pixel values without changing meaningful content. Several São Paulo-based computer vision companies, including teams that have gone through the InovAtiva Brasil acceleration program run out of the Ministry of Development, are selling this tooling to public and private sector clients across Latin America.

The practical prescription for any organization managing a large image library is now well-established in the field: run a deduplication audit before, not after, a cloud migration. Establish a canonical master file for each unique image and redirect all references to that master. Implement ingestion-time hash checks so new duplicates cannot enter the repository. For São Paulo's municipal departments, that means the window to act cleanly is narrowing — the harder the migration is to reverse, the more expensive a retrospective cleanup becomes.

For residents and businesses navigating city services that depend on those archives — permit applications processed through the Secretaria Municipal de Urbanismo e Licenciamento, health records held by the Secretaria Municipal da Saúde — the downstream effect of a rationalized, duplicate-free image store is faster processing times and more reliable data retrieval. The numbers that built this problem are large enough that fixing it will take time. But the numbers that justify fixing it are larger still.