Monday, August 11, 2014

Consumer Complaint Database and Public Narratives

Lou Holtz, the renowned football coach, is reported to have quipped “Never tell your problems to anyone...20% don't care and the other 80% are glad you have them.” Leaving aside the pleasure of Schadenfreude when competitors get their comeuppance, lingering in the shadows is our own fear that we just might be the next recipient of some imputation of blame! Indeed, lest we fall into the downward drift of reputation risk, the general modus operandi has been to resolve controversial issues affecting consumer complaints as quickly as possible. So, we are usually able to avoid hanging out our dirty laundry to dry in the acidic air of public opinion. Until now!

Before I jump into the deep pool of consumer complaint machinations of the Consumer Financial Protection Bureau (“Bureau”), I would like to offer a definition of a word. That word is “allegation.” Here’s my definition of an “allegation”, liberated from its legalistic moorings: ‘An accusation that someone has done something illegal or wrong, which may be true or may be false, typically made without proof, or sufficient proof, and eventually may or may not lead to somebody being found innocent or guilty of doing something illegal or wrong.’ Please keep my definition in mind as we explore together the Bureau’s new Proposed Policy Statement regarding consumer complaints, issued on July 16, 2014.[i]

For some time we have known about the Bureau’s “Consumer Complaint Database” (“Database”). The Bureau’s new proposal would expand the “public-facing database” to include “unstructured consumer complaint narrative data” (“Narratives”). The Bureau promises that only those Narratives for which an opt-in consumer consent has been obtained and a “robust personal information scrubbing standard and methodology” applied would be subject to disclosure. The expansion, therefore, supplements and extends the Bureau’s existing Policy Statements that established the Database.[ii]

The Database actually had the Narratives feature associated with it from the start. The Bureau planned to include Narratives from as far back as the original announcement of the Database on December 8, 2011, when it notified the public about its plans to disclose certain data about the credit card complaints that consumers submitted to the Bureau.[iii] Its final policy statement was issued on June 22, 2012.[iv] At the time that this policy became official, it also announced its plans to disclose data from consumer complaints about financial products and services other than credit cards.[v] Finally, the Bureau rendered its final policy statement for these other financial products and services on March 25, 2013.[vi] In effect, the July 16, 2014 supplements the Bureau’s existing Policy Statements.

Here is the timeline:

·       Credit Card Complaints:
o   December 8, 2011: “December 2011 Proposed Policy Statement”
o   June 22, 2012: “June 2012 Policy Statement”
·       Financial products and services other than credit cards:
o   June 22, 2012: “June 2012 Proposed Policy Statement”
o   March 25, 2013: “March 2013 Policy Statement”
·       Supplement to credit card and financial products and services other than credit cards:
o   July 16, 2014: “July 2014 Proposed Policy Statement”[vii]

It should be noted that the June 2012 Proposed Policy Statement did not propose the inclusion of public Narratives in the Database. The volume of comments in response to including the such Narratives was “significant”: there were consumer, civil rights, and open government groups, supporting disclosure “on the grounds that disclosing narratives would provide consumers with more useful information on which to base financial decisions and would allow reviewers to assess the validity of the complaints; and privacy groups wanted an opt-in, because of concerns about the risk of publishing “non-identifiable” data; but trade and financial industry groups “nearly uniformly” opposed the disclosure of consumer complaint narratives.[viii] 

Responding to these comments, the Bureau noted in the March 2013 Policy Statement that it would not post public Narratives to the Consumer Complaint Database – at least not until it could assess whether there were “practical ways” to disclose narrative data submitted by consumers without undermining consumer privacy.[ix] The stage was now set to determine how and when to expand the policy to include these Narratives.

AREAS OF INTEREST

The Bureau believes that there are three areas of interest that need to be considered in order to implement its plan to include the Narratives: (1) the direct and indirect benefits to consumers, (2) the benefit to the Bureau, and (3) the advancement of open government principles. Permit me to provide a synopsis of each of these vectors.

Direct Benefits to Consumers: Consumers may share their experience with other consumers.
Complainants would be able to provide information they deem useful to others who may be considering doing business with a particular financial institution. Or, the Narrative would be a means of letting others know about a company, offering and experiencing similar situations, thereby letting them “know that they are not alone.”[x] The Bureau contends that the public is not served if it only discloses the non-narrative portions of the complaint. It seems to me that the Bureau is crossing into the realm of the Confidence Fairy[xi] when it opines that “some consumers may choose to submit a complaint only if they will have the opportunity to share their story and other consumers may overcome their reticence to submit a complaint by reading the experiences of others.”[xii] The Bureau believes that this direct benefit may expand the number of complaints submitted to the Bureau, thereby improving the value of the Database.

Indirect Benefits to Consumers: The marketplace will be more responsive to consumers, because the effect of the Narratives will be to influence consumer purchasing decisions. The Bureau claims that research shows that “consumer word of mouth (which includes consumer reviews and complaints) is a reliable signal of product quality that consumers consult and act upon when making purchasing decisions.”[xiii] While the Bureau does not provide such research, it is a rudimentary premise of economic theory that, through their purchases, consumers signal competitive information to market participants. However, the device of a Database with Narratives is not itself the market. As the eminent semanticist, Alfred Korzybski, said, "the map is not the territory."[xiv] Or, as mathematician Eric Temple Bell said, "the map is not the thing mapped."[xv] The signal is not coming from within the market itself but mysteriously from a contrived database. This is what I would call the “Angie’s List Fallacy,” the notion that a list of pros and cons about vendors can substantially move the overall pricing and enhance customer service across a huge market. The theory seems wonderful; the practice does not deliver. Yet, the Bureau believe that the Narratives will be “responsive to the effect word of mouth can have on sales, adjust prices to match product quality and improve customer service in order to remain competitive.”[xvi] The Confidence Fairy reappears, when the Bureau asserts the indirect benefits of the “powerful first person voice of the consumer talking about their (sic) experience,” and the “ability for local stakeholders to highlight consumer experiences in their community,” and empowerment provided “by encouraging similarly situated consumers to speak up and be heard.”[xvii]

Given the direct and indirect benefits to the consumer, the Bureau seems to have arrived at the following algorithm: the aforementioned increase in benefits and utility leads to an increase in consumer contacts, which leads to a positive effect on Bureau operations, which leads to a “critical mass” of complaint data being achieved and exceeded, which leads to the representativeness of Bureau complaint data increasing. The July 2014 Proposed Policy Statement provides no information in support of effectiveness of this process nor does it offer how the “critical mass” will be sliced and diced, except to claim that the “complaint data” will be used by the Bureau’s Offices of Supervision, Enforcement, and Fair Lending, Consumer Education and Engagement, and Research, Markets, and Rulemaking.

Benefits to the Bureau: The Bureau sees itself as a vehicle to open and transparent government. As such, it takes the position that the expansion of the Database is needed in order to “further establishing itself as a leader in the realm of open government and open data.”[xviii] In support of this mission, it mentions the “Open Government Directive,” issued on December 8, 2009 by the Office of Management and Budget (“OMB”), which requires agencies to “take prompt steps to expand access to information by making it available online.”[xix] The Bureau indulges in a bit of spin when it asserts that “agencies have historically withheld data from the public due to privacy and cost controls, (but) with new technology comes new opportunities for openness without significant increases to privacy risk and costs.”[xx] I think it is fair to observe that agencies have withheld data from the public for numerous reasons, though the least of which seem to have been due to “privacy and cost controls.”

Advancement of Open Government Principles: In developing its thesis, the Bureau provides a list of agencies that are seeking to be more open and transparent, such as the Department of Health and Human Services, the Federal Trade Commission, and projects like HealthData.gov and Regulations.gov. These efforts are rooted in the OMB’s call for a “presumption of openness” standard.[xxi]

The OMB’s position is that this “presumption of openness” can be applied in utilitarian and economic terms.[xxii] The premise is that information should be considered “a valuable national resource and a strategic asset to the Federal Government, its partners, and the public,” and that “[m]aking information resources accessible, discoverable, and usable by the public can help fuel entrepreneurship, innovation, and scientific discovery – all of which improve Americans’ lives and contribute significantly to job creation.” While acknowledging that openness is “always subject to legal obligations such as those to protect privacy and confidentiality,” the government believes “public value” is created by treating information as a public asset, when made available to its public owners.[xxiii]

MINIMIZING (BUT NOT ELIMINATING) RISKS

The Bureau admits that publishing Narratives is not without risks. A principal risk of publishing Narratives is the potential harm associated with the possible re-identification of actual consumers within the Consumer Complaint Database. The term “re-identification” means removing personal information from a dataset, thereby obscuring individual identities. Re-identification generally occurs when separate datasets are combined to reestablish some number of individual identities. Individuals with personal knowledge of events described in a Narrative may also be able to identify consumers using de-identified narratives.

Research has shown that the sufficiency of de-identification suggests that the risks generally outweigh the benefits of sharing data.[xxiv] But, glossing over the risks, the Bureau opines that “many researchers espouse the sufficiency of de-identification and highlight the extremely low risk of actual re-identification and potential harm – suggesting a cost-benefit analysis where the benefits outweigh this risk.”[xxv]

Supporters of de-identification methodologies argue that modern scrubbing standards reduce risk, with the exemplar being the Health Insurance Portability and Accountability Act (known as “HIPAA”), in its Privacy Rule, which forms the basis of the Bureau’s own scrubbing standard for Narratives. This view holds that it is possible to decrease re-identification risk to acceptable levels and the number of known, successful attempts to re-identify publicly available datasets are de minimus.

We are now at the juncture where the Bureau must take on the allegations engendered by and embedded in the Narratives. It is this second risk that poses significant concerns to any financial institution unlucky enough to be impaled on the spike of misleading information. The Bureau realizes that the Narratives “may contain factually incorrect information as a result of, for example, a complainant’s misunderstanding or misrecollection (sic) of what happened.”[xxvi]

Here is the Bureau’s view of such risk:

“If consumers were to rely without question on all narrative data, it is possible that subsequent purchasing decisions may be based on misinformation. To the extent this risk may be realized, both consumers and the financial institutions that lose business due to misinformation would be disserved. Indeed, even absent any effect on consumer decision-making, there is a risk that financial institutions could incur intangible reputational damage as a result of the dissemination of complaint narratives.”[xxvii]

And here is the Bureau’s own reflection on the risks:

“To a large extent, this risk is inherent in any release of complaint data.”[xxviii] 

In effect, in deciding to release the structured complaint data, the Bureau believes that it has addressed this risk concerns and concluded that, while there is always a risk that market participants will draw erroneous conclusions from available data, the Bureau maintains that the “marketplace of ideas”[xxix] would be able to determine what the data shows and, mutatis mutandis, the Bureau believes that is true, as well, with respect to the complaint Narratives.  

Procedurally, the Bureau claims to be able to mitigate this risk by providing for the public release of the financial institution’s response, side-by-side and scrubbed of any personal information, to the consumer’s complaint. According to the Bureau, this process is supposed to ensure that, to the extent there are factual disputes, both sides of the dispute can be made public. Modern technology used to scrub public disclosure of unresolved disputes and allegations – what could possibly go wrong?

FEASIBILITY

Is the Bureau’s “open government” plan to release Narratives even feasible? Actually, the Bureau does not really know for sure! In deciding to release certain structured data, the Bureau stated that it would not disclose Narratives unless it is operationally feasible to do so without compromising consumer privacy.

In November 2013, its Consumer Response unit began piloting a comprehensive program to scrub all personal information from copied Narratives, using a scrubbing standard based on “government best practices.” This pilot is ongoing and the scrubbing standard is continually improved as lessons are learned and implemented.

The scrubbing process calls for the following standards:

(1) Consent for publication is obtained from the consumer;
(2) Narratives are scrubbed of consumer personal information consistent with a robust standard and methodology:
(a) that substantially meets government best practices for re-identification risk; (b) as written, results in a low risk of re-identification;
(c) as applied, maintains a low rate of operational error; and
(3) an independent, third party privacy expert conducts a review and operational test of the standard and methodology in support of the above conditions.[xxx]

SCRUBBING STANDARDS

A synopsis of the scrubbing standards follows, based on the foregoing procedures.

Consumer Narratives

Consumers share their individual stories with other consumers and the marketplace by consumer complaint narratives published in in the Database. Consent is first obtained from the consumer.

Consumer Consent to Disclose Narratives – Opt-In

The Bureau to disclosure Narratives (1) for which informed consumer consent has been obtained, and (2) that have been scrubbed of personal information. Consumers who submit a complaint will be given the opportunity to check a consent box giving the Bureau permission to publish the Narrative. The opt-in consent will state: (1) whether or not consent is given will have no impact on how the Bureau handles the complaint, (2) if given, the consumer may thereafter notify the Bureau to withdraw consent at any time and the Narrative will be removed from the Database, and (3) the Bureau will take “reasonable steps” to remove personal information from the complaint to minimize (but not eliminate) the risk of re-identification.

Company Response

Where the consumer provides consent to publish the Narrative, the related company will be given the opportunity to submit a narrative response for inclusion in the Database. The company will be instructed not to provide direct identifying information in its public-facing response, and the Bureau will take reasonable steps to remove personal information from the response to minimize (but not eliminate) the risk of re-identification. The Company Portal will include a data field into which companies have the option to provide narrative text that would appear next to a consumer’s narrative in the Database.

Personal Information Scrubbing Standard and Methodology

The publication of Narratives involves risks, so admits the Bureau, including the potential harm associated with the re-identification of actual consumers within the Consumer Complaint Database. So, in order to minimize the risk of re-identification, the Bureau will apply to all publically-disclosed Narratives, a “robust personal information scrubbing standard and methodology.”[xxxi] The Bureau believes it can exercise discretion, modifying data when privacy risks clearly and substantially outweigh the benefits of disclosure. Based on the foregoing “scrubbing standard” the Bureau believes the risks can be minimized (but not eliminated), with the result that publicly releasing “redacted narratives,” subject to consumer consent, will best protect all consumers without harming the protected privacy interests of any individual consumer. As to protecting the company whose reputation has been damaged by a factually inaccurate or misleading allegation, the Bureau provides no conclusory observation.

COMMENTS AND PROPOSAL

In the June 2012 Policy Statement and the March 2013 Policy Statement, the Bureau addressed comments received in response to the December 2011 Proposed Policy Statement and the June 2012 Proposed Policy Statement, respectively. The Bureau believes that it has sufficiently addressed comments concerning the Consumer Complaint Database generally, as well as comments regarding the current data fields, in the June 2012 Policy Statement and the March 2013 Policy Statement. These comments ranged from the very general, such as the Bureau’s authority to disclose consumer complaint data of any kind and the impact the database would have on consumers and covered persons, to the more specific, such as the impact of specific proposed data fields (i.e., company disposition) and the inclusion of other data fields (i.e., Narratives).

In both Policy Statements, the Bureau asserted that it was following its “open government” mandate, and, because of that, there would be forthcoming the inclusion of additional data fields. Consistent with the commitment, and in response to comments that urged the disclosure of Narratives, the Bureau has decided to propose the inclusion of Narratives in the Database.

Broadly, the Bureau now seeks comments that are related to the proposed extension of the policies to include the Narratives. It is worth noting that the July 2014 Proposed Policy Statement constitutes an agency statement of general policy and is exempt from notice and public comment.[xxxii]

Notwithstanding this procedure, the Bureau is inviting public comments on the July 2014 Proposed Policy Statement. But because no notice of proposed rulemaking is required, the provisions of the Regulatory Flexibility Act do not apply.[xxxiii]
Jonathan Foxx
President & Managing Director
Lenders Compliance Group



[i] Disclosure of Consumer Complaint Narrative Data, Notice of Proposed Policy Statement with Request for Public Comment, Bureau of Consumer Financial Protection, July 16, 2014. I will cite extensively from the Proposed Policy Statement, mirroring references therein as best as possible in the context of this article.
[ii] Idem. Summary
[iii] Op. cit. 1, 76 FR 76628, December 8, 2011
[iv] Op. cit. 1, 77 FR 37616, June 22, 2012
[v] Op. cit. 1, 77 FR 37616, June 22, 2012
[vi] Op. cit. 1, 78 FR 21218, April 10, 2013
[vii] The Bureau’s terminology is “Proposed Policy Statement”
[viii] Op. cit. 1, p 3
[ix] Op. cit. 1, p 4
[x] Ibid
[xi] The “Confidence Fairy” has been around a long time, casting fairy magic with its supernatural powers; most recently popularized by Paul Krugman, the economist and Noble Laureate.
[xii] Ibid
[xiii] Op. cit. 1, p 5
[xiv] Alfred Korzybski coined the expression in "A Non-Aristotelian System and its Necessity for Rigour in Mathematics and Physics", a paper presented before the American Mathematical Society at the New Orleans, Louisiana, meeting of the American Association for the Advancement of Science, December 28, 1931. The paper may be read in Science and Sanity, 1933, p. 747–61
[xv] Numerology: The Magic of Numbers, Bell, Eric Temple, Williams and Wilkins, Baltimore, 1933
[xvi] Op. cit. 1, p 5
[xvii] Ibid
[xviii] Op. cit. 1, p 6
[xix] Open Government Directive, Office of Management & Budget, December 8, 2009 , Peter Orszag, Director
[xx] Op. cit. 1, p 6
[xxi] Op. cit. 19
[xxii] Open Data Policy - Managing Information as an Asset, OMB Memorandum M-13-13, May 9, 2013
[xxiii] Op. cit. 1, p 7
[xxiv] Broken Promises of Privacy: Responding to the Surprising Failure Of Anonymization, Ohm, Paul, 57 UCLA L. Rev. 1701 (2010)
[xxv] Op. cit. 1, p 7
[xxvi] Op. cit. 1, p 8
[xxvii] Ibid
[xxviii] Ibid
[xxix] Ibid
[xxx] Op. cit. 1, p 9
[xxxi] Op. cit. 1, p 11
[xxxii] 5 U.S.C. 553(b)
[xxxiii] 5 U.S.C. Chapter 6