What Causes Dirty Data?

In an ideal world, data would go into your database correctly, completely and consistently; in other words, cleanly. Unfortunately, where transactional databases, contact managers and CRM systems are concerned, that’s rarely possible. Many of the more powerful contact management and sales database systems have algorithms built into them that perform data scrubbing operations on incoming data to nip as many potential data problems in the bud.
More information on data scrubbing.

Whether you pack that kind of processing power, or manage your data in a more conventional database, scrub your data before or after the fact, the following lists some of the reasons for good data going bad:

• Poor or "creative" data entry, including misspellings, typos, transpositions, and variations in spelling, naming or formatting.
  
  
• Missing data.



• Lack of company-wide or industry-wide data coding standards (e.g. a big problem in health care).



• Multiple databases scattered throughout different departments or organizations, with the data in each structured according to the idiosyncratic rules of each.



• Older systems that contain poorly documented or obsolete data.

As the list suggests, data scrubbing is aimed at more than eliminating errors and redundancy. The goal is also to bring consistency to various data sets that may have been created with different, incompatible business rules. Without data scrubbing, those sets of data aren't very useful when they're merged into a mailing list or warehouse that's supposed to feed business intelligence across an organization.

In the early days of computing, most data scrubbing was done by hand. And when performed by bleary-eyed humans, the laborious task of finding and then fixing or eliminating incorrect, incomplete or duplicated records was extremely costly - and often led to the introduction of new errors.

Northwest Database Services' specialized software tools use sophisticated algorithms to parse, standardize, correct, match and consolidate data. Our operations range from simple cleansing and enhancement of single sets of data to matching, correcting and consolidating database entries from different databases and file systems.

Contact us for more Data Scrubbing information