How to De-Identify Company Data Before Selling It

To de-identify company data, find every direct and indirect identifier, remove or replace it, test whether anyone could still be singled out, and record what you did. De-identification lowers risk but does not eliminate it, and whether the result counts as anonymous under the law is a legal question, not a technical one.

Most failed de-identification is not caused by missing names. It is caused by the things people forget: free-text fields, rare combinations of details, attachments and metadata. This article lays out a working pipeline in the order you would run it, explains why each stage exists, and is blunt about where the method stops being reliable.

What does de-identification actually mean?

De-identification means altering data so that it no longer points to specific people, customers or organisations, while keeping as much useful content as possible. It is a spectrum of techniques, not a single switch.

Three terms are often confused. Redaction removes content. Masking or pseudonymisation replaces identifiers with substitutes, such as tokens, so the same person is represented consistently but not by name. Anonymisation is the legal and practical outcome in which individuals can no longer be identified by any means reasonably likely to be used.

Under laws such as the GDPR, pseudonymised data is generally still treated as personal data, because the link back could be restored. Truly anonymous data is treated differently, but the standard is high. See anonymization vs pseudonymization for the distinction. This article is general information and not legal advice, so involve a qualified lawyer on the legal conclusion.

What counts as an identifier?

An identifier is anything that alone or in combination could single out a person, a customer or a confidential relationship. The list is wider than most teams expect.

• Direct identifiers: names, email addresses, phone numbers, postal addresses, account and order numbers, government and tax identifiers.
• Technical identifiers: IP addresses, device IDs, cookie values, user IDs, license keys, and URLs with tokens in them.
• Indirect or quasi-identifiers: job title, employer, location, dates, age ranges, and rare events that together point to one person.
• Organisational identifiers: client names, project codenames, contract references and internal system names.
• Content identifiers: signatures, faces and names in images, text in screenshots, and file names.
• Metadata: author fields in documents, timestamps, geotags and revision histories.

Do not forget that the identifiers of customers, vendors and partners matter as much as those of individuals. A dataset can be free of personal data and still expose a client relationship the seller was bound to keep confidential.

How do you de-identify data step by step?

Run the work as a pipeline with clear stages and records. A repeatable process is easier to audit, easier to improve and easier to explain to a buyer.

1. Define the purpose and the minimum fields. Delete every field the buyer does not need. Data you never ship cannot leak.
2. Map identifiers. For each field and for free text, mark whether it is direct, indirect, organisational or sensitive content.
3. Choose a treatment for each class: drop, mask, generalise, shuffle, or tokenise.
4. Apply automated detection for patterns such as emails, phone numbers and numeric IDs.
5. Apply entity recognition for names, places and organisations in free text, then add custom lists for your own clients, products and staff.
6. Review a human sample, larger for risky fields.
7. Test for re-identification, using the methods below.
8. Document the method, versions, decisions and residual risks.
9. Freeze the output and record a checksum so the delivered version is the reviewed version.

Rerun the whole pipeline whenever the source data or the rules change, even if the change seems minor. A one-time scrub of an old export tells you nothing about a new one, because new fields, new customers and new habits all introduce new places for identifiers to hide.

Which techniques work for which kinds of fields?

Different field types need different treatments. Choosing well preserves usefulness while reducing exposure.

Structured fields

Drop fields you do not need. Replace IDs with random tokens, using a different mapping per delivery if you do not want buyers to link datasets. Generalise dates to a month or a relative offset, ages to bands, and locations to a wider region. Remove very rare categories, since a value held by one person is an identifier.

Free text

Free text is where most leaks happen. Combine pattern matching, entity recognition and custom dictionaries, then review a sample by eye. Replace entities with labelled placeholders so that the sentence still makes sense. Expect to miss some, and design for that with a conservative scope.

Documents, images and attachments

Strip document properties and revision history. Remove embedded images or run them through a separate review. Text in screenshots is not caught by text scrubbing. Many sellers exclude attachments entirely, which is often the sensible choice.

Logs and telemetry

Look for user identifiers in URLs, query strings, error messages and stack traces. Remove or hash them, and consider whether the fine-grained timing alone could single someone out.

How do you test whether it worked?

Test by trying to break your own work. A check that only confirms the scrubber ran proves nothing about whether the result is safe.

• Sample audit: a person reads a random sample and counts what slipped through. Record the miss rate and the type of misses, not just a pass or fail.
• Seeded tests: insert synthetic fake identifiers into a copy and measure how many your pipeline catches.
• Uniqueness check: count how many records are unique on combinations of quasi-identifiers. A high proportion of unique combinations signals risk even without names.
• Linkage thinking: ask what public or purchasable data a motivated person could join with yours, and what would be revealed.
• Fresh eyes: ask someone outside the project, who knows the business, to look for client or project hints.

Any test result is a snapshot. State plainly in your documentation that residual risk exists, and give buyers the contractual rule that they must not attempt re-identification.

Who should do the work?

Assign it to people who understand both the data and the business. A data engineer knows the schema, but only someone from the operating team will recognise that a project codename gives away a client. Pair them, and give a third person from legal or privacy the authority to stop the release.

Where does de-identification fail?

It fails on rare combinations, on free text, on small datasets and on data that is intrinsically identifying. Some data cannot be made safe, however careful you are.

• Small populations: in a dataset of a few hundred people, a handful of attributes may be enough to find a person.
• Unique behaviour: location traces, writing style and detailed purchase histories can identify people without any labelled field.
• Context leakage: a sentence like "the CFO of our largest client in the city" can identify an organisation.
• Over-scrubbing: removing so much that the data loses its value, which is a commercial failure.
• Under-scrubbing: the opposite, which is a legal and reputational failure.
• Drift: the process worked last quarter, but a new field appeared in the export.

For some data, especially health records, formal methods and expert involvement are the proper route. In the US, HIPAA describes specific standards, discussed in HIPAA de-identification: Safe Harbor vs Expert Determination.

What does a worked example look like?

A worked example shows how the stages combine on one record. The following is a hypothetical illustration, not a real record.

Imagine a support ticket that includes a customer's full name, a work email, an order number, a city, a date and a message saying "I'm the finance lead at a mid-sized clinic and our login broke after the merger." The pipeline drops the email and order number, replaces the name with a stable token, generalises the city to a region and the date to a month, and replaces the employer type and merger reference with a neutral phrase. The reviewer notices that "finance lead at a clinic after the merger" could still single out one person in a small town, so the rule set is tightened to generalise job and sector details in that field.

Two lessons follow. First, automated detection caught the obvious fields, and a human caught the quasi-identifier. Second, the fix was a rule change, not a one-off edit, so the next export benefits too.

Keep a short log of rule changes like this. It becomes the evidence that your process improves with each review, which is exactly what a buyer's security team wants to see.

How should you describe the process to a buyer?

Describe it plainly: what was removed, which method was used, what checks were run, and what risk remains. Buyers trust sellers who explain limits more than sellers who make big claims.

Include a short statement in the data card. Say what categories of identifiers were removed or replaced, whether replacements are consistent across the dataset, and how the output was checked. Avoid words such as "fully anonymous" or "risk free". Your listing on LogitAtlas can attach license terms that prohibit re-identification and restrict onward sharing, and the message thread is the place to answer technical questions before a buyer bids.

If you are still deciding whether the data is worth preparing, the free ten-question valuation can help you decide, and a data inventory tells you where the riskiest fields sit.

Frequently asked questions

Is removing names and emails enough to de-identify data?

Usually not. Indirect identifiers, free text, metadata and rare combinations can still point to a person or client. Removing names is the first step. Testing for re-identification and documenting residual risk are what make the work defensible. Legal status as anonymous is a question for a lawyer.

Can I use an automated tool to do all the de-identification?

Automated tools are useful for a first pass, but none finds everything, especially in free text and images. Combine them with custom lists for your clients and staff, and always have a person review a sample. Treat tool output as a draft, not a guarantee.

Does de-identified data still need a license?

Yes. Even if individuals are no longer identifiable, the data remains your asset and may contain confidential business information. A license should define permitted use, limits on sharing, a ban on re-identification and what happens when the term ends.

How much should I budget for de-identification?

There is no fixed figure. Cost depends on volume, how much free text and how many attachments the data has, and how rigorous the review must be. Do a small pilot on a sample first, measure the effort per record, and compare that with a realistic price before committing.

When should I stop and not sell?

Stop if you cannot reliably remove identifiers, if the dataset is small enough that people stand out, or if contracts forbid sharing even de-identified data. If the remaining risk worries your lawyer or your security team, trust that instinct.