Electronic Data Records Law | How to Win E-Discovery

Electronic Mail Storage In-House or In the Cloud

E-mail Archive Economics

How expensive is it to store many terabytes of enterprise email so they are available for survey, eDiscovery, investigations, internal control and purposes of institutional memory?  The raw cost of data storage is dropping, so is it not cheap to retain lots of email?

I explored this topic in an email exchange with Dennis Vlasich, Director of the IT Department at the municipality of Fontana, California.  Following is our conversation.  

I started by pointing Dennis to my earlier post suggesting "content addressable storage" for inexpensive retention of large volumes of e-mail.  

DENNIS replied: “It [the earlier post] is basically a standard storage vendor's sales pitch and we have a SAN that can accommodate terabytes of ‘cheap’ storage.  You have to factor in the rack space, power and HVAC requirements to keep the storage system running and I can assure you that because such technology [content addressable storage] is less expensive, its MTBF [mean time between failures] will be much shorter than other forms of storage, so what happens when it crashes?  The bottom line is cost versus benefit versus risk.”  [Update:  Greg Smith of Messaging Architects responds to Dennis' comment.]

To which BEN said: “As a lawyer, here's what I don't understand.  I use Yahoo as my business email provider.  I pay Yahoo a paltry $20 a year for this email service, and Yahoo provides me unlimited storage.  Unlimited.  And since I pay for my service, Yahoo does not attach ads to my email. I store on Yahoo all my business e-mail (including attachments) dating back to 2004.  (I also back it up in zip files I download onto a local hard drive.)  The economics suggest that storage costs -- including all the infrastructure and management -- are just not a big deal to Yahoo.  Why can't an in-house IT department at a mid-sized enterprise get anywhere close to the same economics that Yahoo (or AOL or Hotmail) gets?”

Then DENNIS said: “Yahoo probably restricts the size of your attachments to 1MB and your mail client (probably not Outlook) does not have the search and archiving capabilities of an Exchange/Outlook environment.  I don't know what your SLA [service level agreement] is with them, but I would guess they aren't guaranteeing that all your email will be available forever.

"The City of Los Angeles just switched from an in-house mail system (using Groupwise) to Google's gmail for 30,000 users.  Google charges $50 per year per user for up to 250 gigabyte of disk space, however they do not have any archiving capabilities yet.  

“Google, Yahoo, and other large providers are using data center technologies on a massive scale that brings the cost per MB down below what those of us in smaller shops can come even close to. The big problem they have is security.  Google had to guarantee LA to use dedicated servers and storage for all their email so it wouldn't co-mingle with their other customers, and for $7.5MM Google said OK.  Most of the rest of us cannot negotiate on that scale.

“The administration overhead for managed disk space is quite high because it requires routine backups with take time and resources which all cost money.”

BEN responded: “My attachments, both incoming and outgoing, are limited to 20MB, which is reasonably large.  The mail client I use is Yahoo's web mail page.  It provides quite a few search capabilities.  I have my 'archives' arranged as large chronological zip folders on my local hard drive. Still, Yahoo stores a full copy of all my emails back to 2004.”

DENNIS: “[Regarding the 1MB attachment limit] I guess I was thinking of the free version [of Yahoo mail].  It's been a while since I looked at those kinds of mail services. We're looking seriously at Google Apps with gmail (waiting to see how things work for LA).  The 250GB limit might be a problem for some of our users, but I figure by the time we're ready to move, there will be more space available or they will have an archiving feature.”

Update July 2010:  The General Services Administration has certified Google Apps as qualified for use by government agencies.  Typically, a US government implementation of Google Apps would keep data on servers in the continental US.

–-Benjamin Wright.    A senior instructor at the SANS Institute, Mr. Wright regularly serves as a public speaker at lunch and dinner meetings for local professional groups, such as IIA and ARMA.

Native Format Electronically Stored Information?

Collection, Production Tactics | Ordinarily Maintained or Reasonably Usable Form

In eDiscovery, ESI can be produced in many forms.  For both the requesting party and the producing party, the form chosen can impact strategic issues like cost, search-ability and ease of manipulation.  Form of production can be a contentious issue between litigants.

California’s new electronic discovery rules (Assembly Bill 5 - amendments to the California Rules of Civil Procedure) set a framework for negotiating the form of ESI production.  

Amended Section 2031.030(a)(2) of the Code of Civil Procedure provides:  “A party demanding inspection, copying, testing, or sampling of electronically stored information may specify the form or forms in which each type of electronically stored information is to be produced.”  [Notice the words "testing" and "sampling". 

 They are a nod to the future of eDiscovery, where one-by-one review of records is impractical, so statistical methods are used instead.] 

Further, amended Section 2031.280(d) provides:  “Unless the parties otherwise agree or the court otherwise orders, the following shall apply: (1)  If a demand for production does not specify a form or forms for producing a type of electronically stored information, the responding party shall produce the information in the form or forms in which it is ordinarily maintained or in a form that is reasonably usable.”

Like the FRCP, this framework leaves lots of room for negotiation and disagreement.  

“When collecting a unit of ESI (such as an e-mail) within an organization, we might find it in more than one format,” says Greg Smith, eDiscovery specialist at Messaging Architects.  “It might be in network backup.  It might be an e-mail management system like Groupwise.  It might be in an archival system like M+Archive.  It might be in other formats and locations too.”

Leading courts like Williams v. Sprint/United have spoken of production of ESI in its “native format.” For courts like Williams, the rough idea is that a unit of ESI should come with its original meta data and structure, so as to enable better insight into what the ESI was and what it meant.  

According to Greg Smith, “The concept of native format might mean different things for different circumstances in an enterprise information system.  For example, the structure and metadata applicable to a given e-mail might be different depending on whether the email is collected from an electronic mail system like Microsoft’s Exchange or from an archive appliance like Symantec’s Enterprise Vault.  It’s my job to help trial counsel understand the different possibilities, evaluate options and then document/explain the format decisions and actions that are executed in ESI collection and production.”

–Benjamin Wright

Mr. Wright is an affiliate of messagingarchitects.com, experts in eDiscovery services.   Here is an ARMA podcast describing Messaging Architects' work in e-mail archiving, including its workshop for development of an email records policy in an enterprise.

Email Recovery | Network Back-up

E-mail Database Search and Forensics

Archival | What is a Subpoena?

As litigation and other requests for e-mail from backup tapes grow more frequent, an enterprise may be wise to restore, consolidate and archive all of its old e-mail held in backup.

Historically organizations did not think of e-mail as an asset worthy of preservation.  They did not archive it, though copies would end up in network backup.  

But experience has taught that e-mail evidence can be critical to defense of a lawsuit or enforcement of a contract.  Alternatively its disclosure might be required under a subpoena (see definition below), a court order, an administrative summons, an IRS tax audit or a freedom of information act request (FOIA).  Further, e-mail might be needed in an investigation (led internally or by outside criminal prosecutors) of a bribe, a kickback, an overcharge, tax evasion or a embezzlement of money, or it might be needed for an audit of alleged misallocation of funds. 

Archival

Greg Smith of Messaging Architects observes that repeated requests for e-mails from backup can warrant a change of thinking in the IT department. “While most backup software deals very efficiently with individual electronic records, it cannot provide the same level of access to email. This is because enterprise email exists as a record within a database (such as for Groupwise or Microsoft Exchange).  The database must be restored in its entirety and searched in its entirety in order to ascertain the contents of the database. With database sizes exceeding 100GB and scores if not hundreds of tapes, the cost to restore and search the tapes can be considerable. Recovering records for selective users and dates may satisfy the current discovery requirement, but with each successive lawsuit, or changes in discovery parameters for an existing lawsuit, restoration from tape can become a recurring financial liability.”

Greg continues, “The practical solution is to remove the information from tape permanently and place it in an electronic records format where information can be retained and managed as individual records and not conglomerates of information.  In such a format, individual email records can be sorted by user or date.”

Restored records can be archived in an open (non-proprietary) format, such as XML, which enables searchers to tap an ever-expanding array of search and forensics tools.  XML may or may not qualify as "native format" as that term is sometimes used in court decisions, though XML may be a more useful format than whatever the native format was.

–Benjamin Wright

Mr. Wright is an advisor to Messaging Architects, experts in e-discovery and consultants in e-mail investigations.

Post Script  What is a subpoena?  A subpoena is a legal demand that someone turn over information or evidence. Commonly the laws of litigation enable a party to subpoena other parties for records or other evidence. Sometimes the law also invests the power to subpoena in an official who conducts investigations, such as a government auditor or an inspector general. If a party abuses the power to subpoena, by demanding irrelevant records or by issuing a demand with no regard for the cost of compliance, a court may sanction the issuer of the subpoena.

Electronic Mail Keep-Destruction Practice

Storage and Search Costs for Electronic Mail Archival

More Retention of Public Administrator E-mail?

Conventional records and information management (RIM) advice holds that each e-mail in an enterprise must be evaluated one-by-one.  Presumably, this evaluation must be performed by an employee because software isn’t up to the task.  If the employee deems the e-mail is a “record,” then (according to RIM) the employee should categorize it and place it in storage designated for that category.  If the employee does not deem it to be a “record,” it should soon be destroyed.

This advice is not working in practice.  A GAO study of senior federal government administrators found that many are not following the advice.  The nonconformists are just keeping all their e-mail – in good part because they believe that is the easiest way to avoid breaking law that requires the retention of records.  

But the nonconformist administrators are keeping the records in e-mail systems (like Groupwise or Microsoft Exchange) that are not designed for long-term storage.  

Conventional advice says these nonconformists either must work harder on sorting through their e-mail (spend more tedious hours in the office), or must hire someone to sort through it for them (incur higher labor costs).

The rationales behind conventional advice:

1. Storage is expensive, so we don’t want to keep more e-mail than necessary.
2. If you don’t put each e-mail into its category, you can’t find what you need when you need it.
3. If you store too many e-mails, it is too costly to search when you go looking for something.

 

Six Reasons Conventional Wisdom is Out of Date

I humbly argue the conventional advice is out of date – especially with respect to the e-mail of important people like executives and senior administrators.  I offer six reasons.

First:  The cost of digital storage is plummeting.  It will continue to drop. 

Second:  The number of e-mails is growing; therefore, the labor costs in evaluating e-mails one-by-one grows.

Third:  Search technologies enable easy retrieval of digital information.  Keyword and other forms of computerized search are very sophisticated and are improving rapidly.  They outperform the old-fashioned process of having people find things by peering into the categories in which they are stored.

Fourth:  When people make records decisions, they can make mistakes.  They can destroy records that someone else (a judge, an auditor, a prosecutor (district attorney), a whistleblower, an inspector general, a FOIA lawyer, a political opponent, an attorney general with a search warrant) thinks they should keep.  Or they can place a record in the wrong category.

Ask 10 lawyers about whether any given e-mail is or is not a “record,” and if so which retention category (or categories) it belongs in, and you can get 10 different opinions.

Fifth:  Employees don’t want the risk of making records mistakes.  Nevada police (state equivalent to FBI) conducted a criminal investigation of its former state treasurer for destroying e-mails that some of his staff believed should be retained.  The ex-treasurer defended himself saying he was just cleaning out insignificant stuff (what he called “clutter”).  In other words, his defense was that he had evaluated the records, deemed them to be “non-records,” and decided to destroy them in accordance with conventional RIM advice.  

The investigation did not conclude that the ex-treasurer had mismanaged records.  Prosecutors never charged him with improper destruction of records, although they did indict him for other alleged crimes.  
Here’s another way to describe the ex-treasurer’s story: a government administrator had to defend himself from allegations about his decisions to destroy “non-record” e-mail, and his defense prevailed.  His decision to destroy "non-records" was correct.  Yet, to defend himself, he had to hire a lawyer, at his own expense, and endure the peril of an invasive interrogation of his records management performance.

Which government administrator wants to defend her records decisions in a criminal (or any other kind of) investigation?  If presented with this risk, wouldn’t the administrator just prefer to keep all the records?

Sixth:  Although ordinary e-mail systems like Groupwise and Microsoft Exchange are not designed for long-term e-mail retention, other technology is designed for that purpose.  I call the technology an e-mail archive appliance.  It makes search and long-term storage of e-mail more practical and economical.

The availability of an archive appliance does not necessarily mean that an enterprise should retain all employee e-mail indefinitely.  But it does suggest very generous retention of e-mail belonging to important people like top administrators.

--Benjamin Wright

Technical footnote:  The conventional “RIM” advice as I portray it in this blog derives from lawyers and legal analysis.  Although you hear RIM used in the records manager profession, I am not talking about the records management profession per se.  When I argue against the application of RIM to e-mail, I am talking about the legal analysis – from lawyers and their publications – that has given rise to the conventional RIM understanding for e-mail.  My disagreement is not with the records management profession. 

Financial E-Recordkeeping Law

National Instrument 31-101: E-mail, Collaboration, Online Chat, Instant Message, and Social Networking Records?

Canada is in the process of adopting new rules on the retention of electronic records by securities firms such as stock brokerages. The rules will impose rigorous requirements for preservation and availability of records for e-mail, text and other communications between firms and their clients.

The rules are part of proposed National Instrument (NI) 31-103, which will install a new nationwide regime for regulation of securities firms. Although authorities hoped to have made NI 31-103 effective as of March 31, 2009, they are still refining it. They anticipate publishing the next draft around April 1, 2009, with implementation beginning sometime after that.

Even though the rules are not yet final, firms can see that the new retention requirements are coming. They are wise to begin e-record retention now.  NI 31-103's new requirements are consonant with the larger trend in society to expect that enterprises keep plentiful electronic records.

Proposed NI 31-103 Section 5.15(1)(a) provides: “A registered firm must maintain records to accurately record its business activities, financial affairs, and client transactions.”

Proposed Section 5.5: “In most circumstances, registered firms should maintain the following records to satisfy subsection 5.15(1)(a) . . . all e-mail, regular mail, fax and other written communications with clients.”

What are the standards for record retention? Seven years is an important time period. Proposed Section 5.16:

"(1) A registered [securities] firm must keep its records safe and in a durable form.
(2) For a period of two years after the creation of a record, a registered firm must keep the record in a manner that permits it to be provided promptly to the regulator, and thereafter the record may be kept in a manner that permits it to be provided to the regulator in a reasonable period of time.
(3) A record provided under subsection (2) must be in a form that is capable of being read by the regulator.
(4) A registered firm must keep
  (a) an activity record for seven years from the date of the act, and
  (b) a relationship record for seven years from the date the person or company ceases to be a client of
the registered firm."

So 5.16 generally provides for shorter retention for “activity records” (seven years from the date of the act) than for “relationship records” (seven years after the client ceases its engagement of the firm).

5.16 goes on to define "relationship records" (which relate for example to the ongoing relationship between a firm and its customer) and “activity records” (which relate for example to specific purchase and sale transactions). In particular instances, however, the difference between an activity record and a relationship record could be indistinct and unclear. An e-mail can contain both activity information and relationship information.  Under a conservative interpretation, this lack of clarity could militate toward keeping even presumably “activity” records longer than seven years.

US securities firms have been dealing with similar record retention rules for a decade, and the US experience is a useful guide for Canadian firms. In 1997 the US Securities and Exchange Commission expressly required securities firms to store certain client communications such as e-mail under SEC Rule 17a-4. As new technologies have emerged, the requirements have been read to include them. For example, NASD Rule 03-33 specifically interpreted 17a-4 to include instant messages.

On several occasions US regulatory authorities have penalized firms for poor electronic records retention practices. For example the New York Stock Exchange fined J.P. Morgan $2.1 million.

As a matter of practice, compliance with rules like Rule 17a-4 and NI 31-103 can be easier if electronic messages are stored in a dedicated message archive server rather than in a production e-mail server or in network backup.

There is no reason in principle why such rules would not be read to include records of new forms of communications as they arise – such as postings on a message “wall” in a social networking environment like Facebook or Ning or txt or video communications via Skype.

–Benjamin Wright