by Mary Levins – Sierra Creek Consulting Atlanta and Georgia Burleson –  Forest Rim Technology Denver

Every business knows – intuitively – that listening to the voice of the customer is a really good thing to do. When the customer speaks, the business responds, or ultimately goes out of business.

In today’s world, listening to the customer is a challenge for many reasons: there are a lot of customers; most customers are heard electronically (i.e. not in person); and customers speak in text, not in neatly well-defined pre-structured, little boxes that can be checked off. Customers talk about everything under the sun – complaints and bad experiences, questions, positive experiences, a desire to buy new things, questions as to how to install equipment, suggestions, and more. And ALL of the feedback from the customer is important.

A good source for listening to the voice of the customer is the Internet. The Internet is free and available to anyone. There are no privacy concerns for data from the Internet. People put their comments on the Internet for public scrutiny, and the Internet does not restrict what anyone can say. The customer can say anything that the customer wishes to say. All of these factors make the Internet a good place to find out what the customer is saying about a company and its products.

Using technology that reads text and organizes the text so that it can be analyzed by a computer, Forest Rim Technology (Denver, Colorado) and Sierra Creek Consulting (Atlanta, Georgia) went to the Internet and examined customer comments over several years for credit bureaus.

The examination of the customer data from the Internet was for these major credit bureaus – Equifax, Experian, and TransUnion.

Business events can affect credit bureaus and their customers. Most recently, there was a breach of the personal data found in the data bases of Equifax.

The Breach of Personal Data

On September 7, 2017 Equifax announced a data breach where personal information and credit card data was exposed. It has been called the most severe data breach in history, affecting more than 146 million consumers, or 40% of the US population. Equifax is one of the largest credit bureaus in the United States. However, Equifax is not the only consumer credit company to experience a data breach. Experian and TransUnion have also had instances where data was compromised in the past. For example, Iin 2013,, 200 million Experian customer records containing personal information were accessed and resold through an identity theft service from Vietnam. And on October 1, 2015, Experian reported a data breach which affected 15 million people. Names, addresses, SSNs, birth dates, various identification numbers (passport, driver license, military ID) were compromised. Individuals were notified about this Experian data breach on November 30, 2015.

So what are people saying about the top 3 consumer credit companies? Is there an increase of concern from the customers following these data breaches? Here’s what we found:

  • We collected 2963 Customer reviews from Consumer Affairs for the top 3 consumer credit companies – Equifax, Experian, and TransUnion.
  • Customers who submitted the reviews live in the US or Canada.
  • Reviews were dated from 2006 to Feb 21, 2018.
  • 84.81K Customer comments were identified and analyzed using Forest Rim Textual ETL.
  • Analysis and visualization of the results were done by Sierra Creek Consulting.

The primary (but not sole) source for reviews was Consumer Affairs. Experian had the highest number of customer reviews on Consumer Affairs overall.

Across all public reviews on Consumer Affairs for credit bureau data, Experian has the highest number of reviews posted and the highest amount of sentiment from 2006 to Feb 21, 2018.

Overall Sentiment by Company

Based on the ability to listen to and analyze the Voice of the Customer, there are more reviews and a higher level of sentiment (positive = green, negative = red, and neutral = gray) in the customer reviews for Experian over all reviews in the time period from 2006 through Feb 2018. Although Equifax is the largest consumer credit bureau, we didn’t expect Experian to have a higher level of sentiment across the three credit bureaus.

Overall Sentiment by Company chart can be found at https://charts.qlikcloud.com/5b0858e4351692000a599f7c/chart.html

Looking at the data at a high level of summarization is useful. But once the summary level data is examined, it is almost always useful to “drill down” more deeply into the data. Drilling down gives the opportunity for insight that can only be gained by looking at successively lower levels of detail. There are lots of ways the drill down analysis can occur. One way is to drill down in terms of time. You can narrow the timeframe of analysis. Another way to drill down is to look at a greater level of detail at how the summary data was constructed.

As with any professional sentiment analysis, it is necessary to break down the sentiment into finer categories of information in order to gain insight. The predominant pattern of sentiment for the credit bureaus is negative sentiment. So the next logical question is – what are the causes of the negative sentiment? Exactly why are customers dissatisfied with the credit bureaus? Just how much did the breach of customer data lead to dissatisfaction?

What is the negative sentiment about Consumer Credit Companies since 2006?

When we look at only the negative sentiment, we find that “communication” is the biggest concern across all 3 companies; and the category of “price sensitivity” unique to the customers of Experian. Top 5 Activity Class by Negative Sentiment chart can be found at https://charts.qlikcloud.com/5b05e28b0626e3001058b002/chart.html

An interesting (and relevant) question soon arises – where do the customers who have written these reviews live? Is there a geographical bias to the customers? An analysis of the locale of the customers generates a map that looks like –

Where the Reviewers Live:

The customers are scattered all over. The predominant pattern of customers who reviewed the credit bureaus is the East Coast of the US and California. However, the majority of the body of customers is widely and fairly evenly spread across the US and Canada.

Sentiment Over Time

The analysis starts to become more interesting when the analysis is viewed over time.

Here’s the link where you can view customer sentiment over time for credit bureaus with limited interactive capability: https://charts.qlikcloud.com/5b05df420626e3001058b001/chart.html

It’s interesting to see the sentiment trend across all 3 of the major consumer credit companies over time. We may draw conclusions looking back on the cause of some of these trends. For example, did the mortgage crisis in 2008 result in job losses, more people applying for credit, more people having issues with the consumer credit bureaus? Is this why there is an increasing trend in sentiment activity between 2008 and 2011? What are people saying in this time period? It’s clear that there are spikes in customer sentiment for each company over time. The strongest sentiment (with the highest spikes) appear to be from Equifax reviews.

Sentiment Trend in 2010

What happened in 2010? Why does EFX have so many spikes in 2010? Whenever there is a spike event it is useful to look at data at a more detailed level. This is done through drill down. Any professional sentiment analysis – such as that provided by Forest Rim – supports a deep drill down analysis. When looking at the Voice of the Customer in 2010, here is what is seen in the drill down.

The first phase of the drill down is to examine the data for just the time frame in question. The data for the period in question (2010) looks like –

For the period in question, a summary of the reviews looked like –

There were other spikes of sentiment activity. In the case of Equifax, there were spikes related to the breach of security.

Despite the concern of over the breach of customer information, when drilling down on the Equifax negative sentiment spikes, we find people talking about the categories of information such as “communication” and their “credit report” as the biggest concern. Ironically there was little conversation about the breach itself.

The Equifax 2010 Negative Sentiment spike chart can be viewed here: https://charts.qlikcloud.com/5b085d96ec2dd7000fd1b9e7/chart.html

As an example of the detail that relates to the spikes in consumer comments for Equifax, the following reviews were encountered –

It’s clear there are issues being voiced by the customer on the Consumerr Affairs review website.  Looking back, we can also find things in the news which may be related to the negative customer sentiment spikes.

Per the Scotsman Guide: In October 2010, Equifax agreed to pay a $1.6 million fine to settle a complaint with the Federal Trade Commission, after admitting to selling information on people who had been late in paying their mortgages. This affected 17,000 consumers. The company had two other smaller incidents in 2010 and 2006. 

The Voice of the Customer may be a strong indicator of a deeper root cause issue other than the breach of security. The customer is trying to tell Equifax (and the credit companies) something more troubling than concerns over the breach.

The Equifax Data Breach in 2017

In September 2017, Equifax Inc stock price dropped. Experian stock price appeared unchanged, while TransUnion dropped for a short period of time then quick recovered and outperformed both Equifax and Experian. This was most likely related to the world’s most severe data breach announced by Equifax in September 2017. But what are people saying about the consumer credit companies after the breach? Has the public sentiment changed before or after the breach? Here’s what we found when using Textual ETL to analyze the high volume of customer reviews from Consumer Affairs.

Before the Equifax Data Breach – Time Frame: 2006 through March 2017

Interestingly, before the Equifax Data Breach occurred, Experian had more negative sentiment before the Equifax Data Breach was announced to the public. The public was officially notified on September 7, 2017. The graph shows the Experian sentiment analysis leading up to the Equifax data breach.

An Explanation

In order to explain this phenomenon, the data needs to be examined before the Equifax Data Breach – Time Frame: 2006 through March 2017. Why does Experian have more negative sentiment in this time frame? What are the concerns? In order to explain what is going on, it is necessary to do a further drill down of the data. The sentiment data is organized by Forest Rim in a hierarchical manner. There is supercategory, which encompasses the highest level of classification of the sentiment found in the study created by Forest Rim. Then there is the generic classification of sentiment related text. Then next and lowest classification there is the  actual words that are created and used by the customer.

By looking at the Supercategory, we see PROCESS, PEOPLE, and COMMUNICATION are the biggest issues in the 2006 to 2017 time frame.

What about the Process? What is meant by PROCESS from the standpoint of the customer? We filtered by the Supercategory “Process” into a lower drill down category and we find the biggest concern is FRAUD.

We also look at the Predicate (the word or phrase for which action is taken in the sentence) for “Process” and find “Service, Scam, and Money”. By drilling down on the sentiment, the concerns over the Experian data leading up to the Equifax data breach are uncovered.

Further drill down to the actual comments looks like –

Before the Equifax Breach, it’s clear the biggest issues with the Credit Bureaus are with Experian’s service and the feeling of being scammed by this consumer credit company.

BEFORE the Equifax Data Breach Notification – Time Frame: April ’17 – June ‘17

Interestingly, customer negative sentiment spikes for Equifax in the period when it is believed the actual Equifax breach occurred, but before Equifax notified the public on September 7, 2017.  The Equifax negative sentiment timeline can be viewed here: https://charts.qlikcloud.com/5b0861adc823ee001107f37d/chart.html

Lesson Learned: An increase in customer call activity or on-line public reviews may indicate an issue.

A further analysis shows the pattern of sentiment (positive, negative, and neutral) analysis after the breach.

AFTER the Equifax Data Breach – Time Frame: April ‘17 through Feb ‘18

The total sentiment activity for Equifax increased slightly after the breach, beginning in April 2017.

There also appears to be spikes in customer sentiment in the month of September 2017 when the Equifax data breach was announced. By selecting the dates, we can easily review the total sentiment for each of the major credit bureau companies before and after the Equifax breach in 2017. Here is the change in customer sentiment found:

September 2017 – the month the breach was announced the number of Equifax reviews and the Equifax Sentiment activity went up significantly. This seems to correlate with the stock drop.

By filtering on negative sentiment for September 2017, we can easily see Experian had the lowest amount of negative sentiment than Equifax and TransUnion in this time period.

The top predicate for the month of September 2017 (the month the Equifax data breach was announced) clearly shows consumers collectively expressing concerns about the word ‘equifax’, ‘report’, and ‘service’.

A drill down analysis of the negative sentiment activity for this period of time yields the following analysis –

The negative sentiment drill down chart for September 2017 can be found here: https://charts.qlikcloud.com/5b08684c8f517e000ba155af/chart.html

Here is a sample of the reviews from Consumer Affairs for the month of September 2017 used for the sentiment analysis.

A further analysis shows that from September 2017 to February 2018 negative sentiment is still slightly higher for Experian with more reviews for Experian than Equifax or TransUnion.

The biggest activity classification for Experian related to negative sentiment between Sept 2017 and Feb 2018 is Communication. This breaks down to ‘Phone, Report, and Wait Time’.  We can make a safe assumption based on this analysis after the Equifax breach, that many people went to Experian asking how to protect their consumer credit file. Due to the number of people impacted, Experian also struggled with the volume resulting in phone, report, and wait time issues.

Sample Reviews for Experian under the ‘Communication’ classification are shown here for the time period after the Equifax breach was announced in September 2017:

Summary

The Equifax data breach did generate more customer reviews and sentiment on Consumer Affairs and other sentiment found on the Internet after the breach was announced in September 2017. Ironically, despite the breach, much of the customer sentiment did not directly reference the breach. Overall, Experian had a higher number of negative sentiment comments even though Equifax is the largest credit bureau.

By listening to the Voice of the Customer for the credit bureaus, we also learned that customer concerns are not only about the breach. Data Breaches happen all the time. Data breach, ransomware attack, hacking, employee theft of credit card numbers, cyber security incident, malware attack, phishing incident – something related is frequently in the news. By analyzing the customer reviews from the three large credit bureaus, it’s clear the breach triggered more customer reviews around the time or shortly after the breach was announced. But the sentiment expressed wasn’t about the breach itself. The sentiment expressed was more about the company’s process, specifically communication.

So what is the customer trying to tell the credit bureaus? The recommendations based on the Voice of the Customer analysis include the following:

  1. Equifax, Experian, and TransUnion should focus on improvements in the areas of Process and Communication
  2. Equifax – “Company” was the biggest category, which shows opportunities to improve reputation. Specifically, Equifax may want to improve training to improve call center communication when there are external public news stories.
  3. Experian – “Phone” was the biggest category under communication. Improvements in call center training may be needed.
  4. Experian should focus on their pricing strategy for improvements.
  5. Use Customer Sentiment trends to predict or confirm when facts aren’t yet known. An increase in consumer sentiment, may indicate something going on that should be investigated.

References:

2018, HEARING THE VOICE OF THE CUSTOMER, by W H Inmon, Technics Publications, NJ


[NOTE: The purpose of this article is to alert the reader to a customer sentiment study that has been done for the credit bureau community. The visualizations were created using Qlik Sense and have a web address attached so you can look in more detail at each visualization. The article focusses on some of the interesting aspects and analyses found in the creation of the study. However, like all complex studies, there are facets and other aspects of the data which are not covered in the article. The data that has been generated is open and available for public inspection and usage. You are free to inspect the data and analyze and interpret the data as you wish. There are many other aspects of the sentiment data that can be interpreted differently than the interpretations that are disclosed here.

The raw textual data was gathered from the Internet, analyzed, then turned into a standard data base by Forest Rim Technology. The interpretations, visualizations and analysis were done by Sierra Creek Consulting. All the customer comment data was based on publicly available customer comments found on the Internet.]

ConsumerAffairs.com Buyers Guide overview (https://www.consumeraffairs.com/online/credit-report/):

Best Credit Report Sites

by Barbara Friedberg, Personal Finance Contributing Editor for ConsumerAffairs

Updated on 03/01/2018

Credit reports are detailed accounts of an individual’s credit history officially prepared by credit bureaus. Lenders of various sorts, such as banks or landlords, refer to credit reports to determine a person’s creditworthiness when applying for a loan, mortgage, new bank account or credit card. The three major credit bureaus are Equifax, Experian and TransUnion.

Consumers who have been victims of identity theft can also turn to a credit report site to receive a current credit report and determine whether or not their private information has been compromised. Many credit report sites offer security options and credit monitoring to prevent identity theft.


Forest Rim Technology was formed by Bill Inmon in order to provide technology to bridge the gap between structured and unstructured data. Forest Rim Technology is located in Castle Rock, Colorado.