Data retention rules are a complex issue that all companies, large and small, have to deal with. As data creation and accumulation skyrocket, the complexity of data governance has become increasingly hard to manage. The result is a process that can be intimidating, unclear, and cumbersome. But it doesn’t have to be.
‘Data retention’ has become a buzzword (buzzphrase?) in the last several years. But what is data retention and why is it important?
At its core, data retention—also called records retention—is simply the practice of determining what data must be kept (retained), or otherwise disposed of. Data retention is motivated not only by business intelligence needs, but also by data security as well as compliance requirements. To make this process run smoothly, companies are encouraged to create data retention rules (also called data retention policies).
So…why should you care about data retention policies?
Data comes in all shapes and sizes and is used by businesses for all kinds of purposes. As a form of best practice, companies should identify what data must be retained and for what purpose as well as determine what length of time that data should be stored. These considerations are called data retention rules and they are essential to a healthy data governance policy.
But the question remains: why should a business care about data retention rules?
In short: because it’s too costly not to care.
If time equals money, then loss of operational efficiency due to either too much data or not enough data is equivalent to a bank vault. An additional concern is regulatory requirements: different regulatory bodies have strict requirements for what types of data need to be kept for certain minimum requirements—for example, the New York Department of Financial Services requires insurers to retain claim files for 6 years after closing a file. Regulators at state, national and international levels may have their own (sometimes conflicting!) requirements. Failure to comply with the requirements set out by regulators can result in huge fines. Likewise, failure to comply with legal retention requirements for discoverable material in the event of legal action can result in massive penalties up to and including a negative inference instruction from a judge.
But be careful not to swing too far in the other direction. In order to comply with any possible retention requirement that may spring up, it’s very tempting to just hold on to data indefinitely. While the strategy certainly looks attractive from a regulatory standpoint, it results in companies paying far more for storage than they need. Developing and enabling retention rules across the data landscape allows businesses to identify data that needs to be retained but does not need to be immediately accessible—thus allowing them to save money by moving that data to cheaper storage. Even better, data retention rules help companies identify data that can be deleted entirely—eliminating the need for storing that data altogether.
Considerations when building a data retention policy
What data should be retained?
Because data is the backbone of operating a successful business, it’s used every single day for a variety of reasons. On top of that, the sheer amount of data amassed can be staggering—it’s hard to know what data is important to keep and what data can be deleted. What data companies need to retain is a simple question with a deceptively complex answer: it depends. The company’s goals and internal processes as well as its industry and regulatory landscape will shape its requirements:
- Business Operation
Every day businesses generate information: information about their customers, their employees, their products and services, their industry, their competitors and on and on and on. Some of this data is superfluous but quite a bit of it is essential to everyday operation: business analytics, trend analysis, research, customer success and product development.
- Business Continuity
Data retention rules can ensure that businesses have and maintain access to historical information—critical to have in the case of disasters or system failures.
- Regulatory Requirements Depending upon their associated industry, many businesses are subject to stringent regulations that require certain data to be kept for pre-defined periods of time. The goal of these regulatory requirements is to ensure that the business is maintaining data that would be required during an audit or investigation, as well as ensuring that consumer data is accounted for and protected.
- Legal Requirements Nobody likes it, but everyone has to worry about it: legal action. When legal action is reasonably anticipated, companies must retain all associated data or risk hefty sanctions in court.
How long should data be kept?
Unfortunately, the answer here is, again, that it depends. Not all data needs to be kept for defined periods of time and not all industries require the same retention periods.
Where is the data?
The rate at which data is created is increasing exponentially every single year—as is the number of places to stash that data. Truly getting a handle on data retention and effectively managing data retention rules can’t be done without visibility into the entire data landscape. From a security standpoint, knowing what data resides in what sources can help companies more effectively protect sensitive data and guard against data breaches. From a regulatory standpoint, different jurisdictions may have rules about where data can reside and whether data is permitted to travel beyond that jurisdiction’s borders. Knowing where that data is helps manage these regulatory obligations.
Who owns the data?
Data retention rules can facilitate remedial action on data, whether that be securing data, deleting it, or migrating it to a different location—but they don’t exist in a vacuum. Just because data has aged beyond retention guidelines doesn’t necessarily mean that the data is not still in use by one or more departments within a company. Evaluating data to ensure that the proposed remedial actions are appropriate is a key step in the data lifecycle but different stakeholders within the business may own that data and be responsible for its disposition. This necessary oversight should be factored into data retention policies: who owns that data and have they approved the remediation workflow proposed for that data’s ultimate disposition?
Automation
These data retention rules should be established as part of a healthy data governance policy—but even more importantly, they must be implemented and followed consistently. The key to effective, consistent, and enforceable data retention policies (as well as peace of mind!) is simple: automation. Automating retention requirements across the organization lets companies rest easy knowing that their data isn’t going to be inadvertently deleted.
Without automation, enforcing retention rules across all enterprise data sources is a drain on time, money, and manpower. Manually sifting through each source to identify, classify, and retain data is a Herculean task for even the most efficient companies: it requires time, inter-departmental communication, and speed. Add in the need for legal holds to supersede retention policies and companies can be looking at a complete mess and a compliance-induced migraine. While not quite a ‘set-it-and-forget-it’ situation, as data retention requires close oversight, automation can handle the more manual processes, leaving resources open to handle the data that requires a more complex approach.
