QUESTION
Our quality control reports provide defects and trendlines, but we have never been able to put together a list of which defects keep occurring. A trendline is useful; however, trends change all the time, and each quarter presents new defects to chase after and fix.
We outsource our monthly quality control audits, and we have an in-house quality control coordinator who monitors our policies and procedures for quality control. And I’m the in-house quality control coordinator.
For the audit method, we use a random sample. In our region, there is a huge competitor that uses discretionary samples. I wonder if they have an advantage in being able to identify defects better than we can by using the random sampling method.
My concern is determining how to stay informed of the frequency of recurring defects. Since defects change each quarter, I want to know how to list and track them so that I can anticipate and cure them going forward.
How can I anticipate defects and trends in quality control audits?
COMPLIANCE SOLUTIONS
ANSWER
I am going to show you how you can use the audit findings of discretionary quality control audits to anticipate defects and trends in random quality control audits.
First, I am going to provide some findings that Fannie has offered.[i] You likely have many investors, Fannie being one of them. Many of them utilize both random and discretionary analytics to determine if their relationship partners meet their quality control standards. Lenders often face many of the same challenges. Being aware of the defect trends across the industry allows for more dynamic QC, better action planning, and prevention of similar defects in your organization.
But don’t be too caught up in an investor’s report if your defect does not appear there. If you’re not experiencing the same defects, it may mean that you already have effective origination controls in place. There is also the possibility that your auditor is not picking up on the defect.
You should keep a list of the top defects occurring in your reports and investor reports. Keeping the list streamlines your ability to anticipate defects. Use the list to train on, too. You should be leveraging the list of defect trends to develop training opportunities for your staff, underwriters, and other participants in the loan flow process. This is a critical way to get ahead of the trending defects. Develop training to prevent these defects from occurring.
________________________________________________________
________________________________________________________
If you want to discuss your approach to quality control, please contact Brandy George, our Executive Director of LCG Quality Control. You can reach Brandy here.
My table below provides a means to anticipate defects as well as a learning tool for training. You can add more columns if you’d like. In our analyses, these are some of the defects that investors reported last year in random sampling. I will list them and add a Best Practice. The Best Practice is an essential component of your training experience. (I have left some blanks in the table to show other possibilities.)
|
RANDOM SAMPLE |
||
|
Description |
Best Practice |
|
|
Income and
Employment |
Calculation
errors |
-During
prefunding QC, be sure to target complex income streams, especially on higher
DTIs. -Assess the
year-over-year trends. The variable income requires evaluation of consistency
and predictability. |
|
Borrower
Eligibility |
Borrower not
employed |
-Perform
extra due diligence in addition to the verbal verification of employment. (i.e.,
Internet searches; email borrower at job address as close to closing as
possible; track and move the verification timeline up. -Be aware of
specific volatile jobs or industries that are more susceptible to workforce
disruptions. |
|
Appraisal |
Inadequate comparable
adjustments; condition and quality rating discrepancies |
-Utilize
value acceptance + property data (VA+PD), when applicable, to increase
certainty, better manage risk, and gain process efficiencies. |
|
Assets |
Insufficient
|
Using one-month
statements when two months are appropriate. |
|
Liabilities |
|
|
|
Credit |
|
|
|
Loan
Documentation |
|
|
|
Title/Lien |
|
|
|
Fraud |
|
|
Please note that the severity of certain defects may make the loan ineligible to certain investors. For instance, a Significant Defect is an issue that makes the loan ineligible for delivery to Fannie Mae and requires remediation or could result in a potential repurchase. An Initial Significant Defect occurs when a Significant Defect has been cited, but remediation activity is still in progress.
Discretionary sampling is valuable when you want to do a comparative analysis, even if you only do random sampling. As I mentioned above, you can get discretionary (or targeted) audit results directly from many investors.
Sometimes, the discretionary and random defects sync up. Take the category of appraisals, for example. In the random sample table above, inadequate comparables are a defect. However, Fannie reported in the fourth quarter of 2023 that their discretionary sampling listed appraisal errors amongst the highest of their defect trends.[ii] In this category example, importantly, the investor’s discretionary sample drills down into the appraisal defects to broaden out the findings, as the following table shows.
|
Discretionary Sampling |
|
|
Defect |
Description |
|
Appraisal |
Inadequate Comparable Adjustment(s) |
|
Failure to Adjust Comparables |
|
|
Inappropriate Comparable Sale(s) Selection Due to
Location |
|
|
Comparable Sale(s) Physical Features Reported
Inaccurately - Condition / Quality of Construction |
|
|
Use of Physically Dissimilar Comparable |
|
|
Sale(s) - Gross Living Area |
|
A discretionary sample intentionally looks for loans with a greater likelihood of being defective or ineligible. Your random defect trendline, however, may be low in a category yet high in the investor’s trendline. That information provides a possible anticipatory impact, so you should be monitoring that category closely, even though it is currently reporting a low defect and trend.
Taking a strategic approach to comparing random to discretionary sampling can lead to the ability to anticipate defects and trends. The opportunity to use this information to enhance your sampling findings, plus ongoing training, is established on both prefunding and post-closing quality control reviews.
Jonathan Foxx, Ph.D., MBA
Chairman & Managing Director
[i]
See, for instance, the quarterly quality control report provided by Fannie Mae,
which outlines random and discretionary audit findings, including defect and
trend analyses. Freddie Mac offers a similar report. And many investors will
provide information regarding their overall findings quarter-over-quarter or
other calendrical intervals.
[ii]
Idem
