Storage and Search Costs for Electronic Mail Archival
More Retention of Public Administrator E-mail?
Conventional records and information management (RIM) advice holds that each e-mail in an enterprise must be evaluated one-by-one. Presumably, this evaluation must be performed by an employee because software isn’t up to the task. If the employee deems the e-mail is a “record,” then (according to RIM) the employee should categorize it and place it in storage designated for that category. If the employee does not deem it to be a “record,” it should soon be destroyed.
This advice is not working in practice. A GAO study of senior federal government administrators found that many are not following the advice. The nonconformists are just
keeping all their e-mail – in good part because they believe that is the easiest way to avoid breaking law that requires the retention of records.
But the nonconformist administrators are keeping the records in e-mail systems (like Groupwise or Microsoft Exchange) that are not designed for long-term storage.
Conventional advice says these nonconformists either must work harder on sorting through their e-mail (spend more tedious hours in the office), or must hire someone to sort through it for them (incur higher labor costs).
The rationales behind conventional advice:
1. Storage is expensive, so we don’t want to keep more e-mail than necessary.
2. If you don’t put each e-mail into its category, you can’t find what you need when you need it.
3. If you store too many e-mails, it is too costly to search when you go looking for something.
Six Reasons Conventional Wisdom is Out of Date
I humbly argue the conventional advice is out of date – especially with respect to the e-mail of important people like executives and senior administrators. I offer six reasons.
First: The cost of digital storage is plummeting. It will continue to drop.
Second: The number of e-mails is growing; therefore, the labor costs in evaluating e-mails one-by-one grows.
Third: Search technologies enable easy retrieval of digital information. Keyword and other forms of computerized search are very sophisticated and are improving rapidly. They outperform the old-fashioned process of having people find things by peering into the categories in which they are stored.
Fourth: When people make records decisions, they can make mistakes. They can destroy records that someone else (a judge, an auditor, a prosecutor (district attorney), a whistleblower, an inspector general, a FOIA lawyer, a political opponent, an attorney general with a search warrant) thinks they should keep. Or they can place a record in the wrong category.
Ask 10 lawyers about whether any given e-mail is or is not a “record,” and if so which retention category (or categories) it belongs in, and you can get 10 different opinions.
Fifth: Employees don’t want the risk of making records mistakes. Nevada police (state equivalent to FBI) conducted a criminal investigation of its former state treasurer for destroying e-mails that some of his staff believed should be retained. The ex-treasurer defended himself saying he was just cleaning out insignificant stuff (what he called “clutter”). In other words, his defense was that he had evaluated the records, deemed them to be “non-records,” and decided to destroy them in accordance with conventional RIM advice.
The investigation did not conclude that the ex-treasurer had mismanaged records. Prosecutors never charged him with improper destruction of records, although they did indict him for other alleged crimes.
Here’s another way to describe the ex-treasurer’s story: a government administrator had to defend himself from allegations about his decisions to destroy “non-record” e-mail, and his defense prevailed. His decision to destroy "non-records" was correct. Yet, to defend himself, he had to hire a lawyer, at his own expense, and endure the peril of an invasive interrogation of his records management performance.
Which government administrator wants to defend her records decisions in a criminal (or any other kind of) investigation? If presented with this risk, wouldn’t the administrator just prefer to keep all the records?
Sixth: Although ordinary e-mail systems like Groupwise and Microsoft Exchange are not designed for long-term e-mail retention, other technology is designed for that purpose. I call the technology an e-mail archive appliance. It makes search and long-term storage of e-mail more practical and economical.
The availability of an archive appliance does not necessarily mean that an enterprise should retain all employee e-mail indefinitely. But it does suggest very generous retention of e-mail belonging to important people like top administrators.
--Benjamin Wright
Technical footnote: The conventional “RIM” advice as I portray it in this blog derives from lawyers and legal analysis. Although you hear RIM used in the records manager profession, I am not talking about the records management profession per se. When I argue against the application of RIM to e-mail, I am talking about the legal analysis – from lawyers and their publications – that has given rise to the conventional RIM understanding for e-mail. My disagreement is not with the records management profession.
Recent Comments