This is part 2 of a 3-part blog series. In this post, we provide tips on managing duplicate records and a worksheet to help implement processes in Insightly CRM.

Part 1: Backup & security

Part 3: User adoption

Dealing with duplicate data in a CRM

According to a 2015-2016 survey report from Experian Data Quality, “Duplicate data, in particular, jumped from 32 to 51 percent.” This is hardly a surprise due to the rapid growth of data, nonetheless it is a serious problem for many organizations, contributing to a poor customer experience, decreased efficiency, and misinformed decisions.

Insightly already detects duplicates when creating or importing new records, but sometimes these duplicate records sneak into your CRM usually because of human error, lack of internal resources, poorly implemented software, and even user permissions (see User Permissions).

5 ways to deduplicate records in Insightly

  • Smart Merge is an Insightly feature that finds duplicates with similar sounding or spelling of names, and gives you a thorough way of merging all related record objects for contact, lead, and organization records.
  • Dedupely integrates with Insightly to find duplicates for almost all your major record types using a variety of criteria customizable to you, and optionally, offers automatic duplicate merging.
  • Look to standalone data utilities like Excel for a complete duplicate scan. This may require programming to build a deduplication algorithm for your organization and to get your data into the tool for analysis (see Version History & Backups). Once you have identified your potential duplicates, manually merge your Project, Opportunity or Task records. SmartMerge will do a fantastic job at taking care of the rest!
  • The Deduplication Insight Card within Insightly automatically detects a duplicate within open lead records and displays an in-product alert showing the matching duplicates.
  • Insightly’s new AWS Lambda functionality lets you create a function to run a deduplicate check against all record types within Insightly. If it detects an email match, Insightly will flag that record as a duplicate.

Here is a chart showing the available features, record types, and related record objects from each tool:

(Click here for more up-to-date information)

As you can see we have a great set of tools to deduplicate records! However, this is not enough. We need to setup best practices and processes for decreasing the creation of duplicates and increasing the accuracy of matching more potential duplicates.

General best practices for data management include:

Train users to develop good habits for avoiding duplicates when creating or importing new records. When using Insightly CRM mobile, Gmail or Outlook Sidebar, use the search feature before adding a new record as the duplicate detection feature is only available on the web version.

Create data entry rules and train your users how to enter in data using the correct format. Insightly does not offer customizable required fields or validation rules, so try to use custom drop down menus instead of free text fields. When users accidentally enter the wrong format, you can update your data to the correct format using Zapier’s Formatter App, Insightly’s Workflow Automation feature, or manually using the bulk edit or bulk update import tools.

Get up-to-date information. Here are three ways to get this valuable information into your database:

Contact update form

Get information straight from the customer whether in-person, over the phone, or via email. This is hands down the most accurate way to get up-to-date information.

You can automate this process by creating a Zapier integration for Mailchimp and Insightly. Mailchimp’s Update Profile Link allows subscribers to update their contact information and preferences. When subscribers update their profile, Zapier updates your contacts in Insightly with the new information. Watch my CRM Contact Profile Update Form Integration Tutorial.

Profile enrichment tools

These tools enrich your data by validating and pairing your contact records with relevant information online using sources like social media profiles, company websites, list provider databases, and even from your email client. Collecting data that is updated often, such as email signatures, is going to be the next best way to get up-to-date information

Use Evercontact to automatically scan email signatures from your Office 365 Outlook or Gmail email client. Then let PieSync handle the rest by synchronizing any updated contact records with Insightly. I recommend reading, The Ultimate Guide to Outlook People By Piesync.

The next best integration for data enrichment is to integrate with a tool like Clearbit. With Clearbit’s Zapier integration you can automatically enrich Lead, Contact & Organization records with relevant information compiled from hundreds of data sources. This will help keep your data accurate and updated. Watch my CRM Contact Profile Enrichment Integration Tutorial

Manual updates

Many businesses have forms and documents stacked away that have yet to be transcribed into the database… Create custom filtered list views to find incomplete records and request the record owner to manually update the missing fields. This can be processed more efficiently with the use of the bulk edit and bulk update importtools.

  • Find patterns in your data and fill in the blanks. This is especially effective for email addresses.

For example, if all of an organization’s email addresses are in this format “[email protected]”, we can make an educated guess and apply this format to all other contacts in that organization with missing email addresses.

  • Never delete data. You may have incomplete records, but it is likely the data is of value to your organization and may be completed in the future.
  • Enforce your data entry rules for any data source integrated with Insightly such as list providers or software integrations. When creating new records in Insightly check for duplicates first. If the record is a duplicate, you will need to update the record instead of creating a new one.

In addition, multiple software integrations may result in duplicates because the integrations do not always play nice together. Try creating one workflow automation process or software integration at a time.

Measure, revise, and repeat

  • Create a standard operating procedure for finding duplicates and merging records. You can start by asking the right questions such as:
  • What fields in our database hold information related to the record’s identity i.e. name, email address, phone number, etc.?
  • Do we need to “normalise” all those fields before checking for duplicates?
  • Will analyzing those fields allow us to find more potential duplicate records?
  • Can we use third party tools to scan our database for potential duplicates?
  • Do we need a full duplication analysis report?
  • How do we identify the “winning” record?
  • What data will be transferred to the “winning” record?
  • What leftover data do we need to save from the “loser” record(s)?
  • How do we manage lost links and objects after merging records?
  • Is manual or automatic merging the best option for our organization?
  • Backup your data before and after each duplicate merge. If you are using Dedupely’s Automatic Merging tool, then look at backing up your data every ten minutes with Flatly.io (see Version History & Backups).
  • Monitor your deduplication analysis reports. Identify what record types have the largest duplicate rate and which fields are often incomplete. With the right business intelligence, you can find the source of those duplicates. Then you can create a resolution to revise your software solution, training material, and data entry rules.
  • Clean your database often.

Action Plan

Take the time to plan an effective data management strategy and procedure for cleaning your database. By doing so, you will allow for a better customer experience, increased efficiency, and more informed decisions.

Go to the Best Practices & Discussions forum in the Insightly Community, where I posted the CRM Best Practices for Insightly Worksheet. This post has everything you need to setup your best practices in Insightly along with more helpful tips to secure your CRM investment.

After implementing these best practices for your organization, I encourage you to leave feedback in the Insightly Community. With your help, my fellow Insightly Heroes and I can continuously improve the best practices for all Insightly users.

Stay tuned for Part 3 of “5 Best Practices for Insightly CRM” where we provide more helpful tips on dashboard reporting.

To be continued…

|