CRM is based on data. Garbage data in, Garbage data out. To resolve data management issues is something that all CRM customers should know. A solid data management and architecture structure to deal with large volumes of data sets you up for success now and in the future.
The key thing to remember is: You, yes you, should worry about Large Data Volume (LDV).
Though there’s no one measurement that can be used to determine whether or not your organization that has a large volume of data, both large and small orgs are at risk. While the number of records in an org is a reliable indicator, the way the data is structured is another factor. Organizations with smaller data volumes but poorly architected data models can still have performance issues related to data. In general, though, an organization will most likely be considered to have large volumes of data if meets one or more of the following qualifications:
Large amounts of data volume typically show themselves as performance-related issues. This includes issues around longer than expected search times, long wait times for fields on a screen to populate when a record is accessed, long record saving times, and other actions taking excessively long amounts of time to complete.
In addition to performance issues, large data volumes can also lead to record locking issues on parent records with large number of child records. Every time a child record is saved, its parent record is temporarily locked, so if thousands of child records are being accessed and updated simultaneously, their parent records will remain locked for extended periods of time. A huge inconvenience.
So let’s talk now about some solutions and proactive steps you can take to protect your org’s performance and see success with SaaS CRM.
Most SaaS CRM’s provide a wide variety of tools to help organizations with large data volumes get a handle on their data. These tools can do wonders for an organization, but they won’t be nearly as effective without an underlying plan for their proper utilization. So if you’re going to take one thing away from this blog post, it should be that the most important aspect of handling any amount of data is to create an effective data management and reporting strategy before implementing a technical solution.
Some of the data management related questions you’re going to want to consider are:
What does my current data model look like?
Create a data model diagram and roadmap that show your current and potential future states for any objects you’re utilizing. Then highlight areas where large numbers of records may be a concern (data modelling).
How long do I need to retain my data?
Think about retention requirements for each object. Donations may need to be retained for tax reporting purposes. Student Records may need to be retained for a certain period of time. Document your retention policy.
What other systems does my Salesforce data need to flow to?
Create a data flow diagram to show source and target applications, which data elements flow between them and their associated volumes other attributes, to ensure you won’t cause any downstream effects in other systems.
What are the source systems for my CRM data?
Create a data flow diagram to display data that’s created in other systems and replicated to the SaaS CRM, or whether external data really should remain in its source system and be accessed via CRM lookups.
How will archiving or purging an object affect related objects?
Document whether your organization has any scenario where child records would be required to be retained longer than their parents, and create a plan.
Does all of your data to be stored in the CRM or can it be stored externally and accessed through reporting?
Many CRM customers maintain a year of data in the CRM itself and store additional years of data in a data warehouse, accessing older records by simply running a report.
Once a data management strategy has been put in place, the next step is to determine the best way to execute it. Some CRM’s provide a number of tools that can help in this area. Examples:
Data Management Tool for Big Objects: Massive amounts of data, archive the data, but no standard reports, no UI, manual SQL queries
Off Platform Data Archiving: External cloud storage, centralised Data Warehouse, usually needs a tool to migrate the data in batches, external reporting tool needed
Indexes and Skinny Data; Don’t remove data from the CRM, Skinny data are table replications that contain frequently used fields to avoid joins, requires dev support, constant updates if fields or schemas are changed
