Backtesting to satisfy SEC valuation rule

A common method of testing the accuracy of a fund’s valuation is backtesting – essentially comparing the price a security sold at with what the fund adviser’s fair valuation method came up with.

However, if you scour Investment Company Act rule 2a-5 (fair value determination and readily available market quotations), which took effect in 2022, you won’t find the word “backtesting.” But the rule does require fund boards or their designees (often a fund’s investment adviser) to periodically review “the appropriateness and accuracy of the [fair value] methodologies selected and making any necessary changes or adjustments.”

Gotta love ‘flexibility’

The final rule’s release gave the SEC an opportunity to “clarify” its position on backtesting. “While we believe that calibration and back-testing are methods that should be used for testing the appropriateness and accuracy of funds’ fair value methodologies in many circumstances, the final rule does not require calibration and back-testing… The final rule provides flexibility to allow funds to use new, appropriate testing methods.”

Flexibility carries the day. “The SEC does not require a specific method or frequency” of testing, states Paul Balynsky, CFA, CPA/ABV, the managing director at Valuation Research Corporation in New York. The onus falls upon the mutual fund board to lay out the valuation process – or its designee.

Backtesting is “the predominant way of doing testing,” states Benjamin Haskin, a partner with Willkie Farr & Gallagher in Washington, DC. “It’s the most intuitive testing that you could do,” comparing actual transactions with your fair-valued pricing.

Compliance around rule 2a-5 gains prominence because it’s a focus of examiners. The SEC’s exam division, in its 2024 priorities, promises to “review registered investment company valuation practices, particularly for those addressing fair valuation practices (eg, implementing board oversight duties, setting recordkeeping and reporting requirements and overseeing valuation designees), and, as applicable, will assess the effectiveness of registered investment companies’ derivatives risk management and liquidity risk management programs.”

An adviser CCO, who asked not to be identified, tells affiliate title Regulatory Compliance Watch of a recent Division of Examinations exam that deeply probed the adviser’s due diligence into its pricing services used for establishing valuations.

Best practices

The CCO’s firm relies heavily on a pricing vendor to establish valuations. He says the vendors expect adviser due diligence and readily share their SOC or SSAE16 reports. Others welcome site visits.

Your board and the adviser can decide on the frequency of such due diligence, but the CCO recommends that you risk assess and rank each of your pricing vendors. At his firm, the job of conducting the due diligence falls to the financial department and not compliance.

“We’re not backtesting the prices pulled down” through the pricing service the firm uses, says the CCO.

However, the adviser conducts valuation due diligence. Its valuation committee meets monthly to ponder the fair value of its securities. If securities are off by a certain percentage over time, say, 5 percent or 10 percent, the staff would peer into why the discrepancies exist.

In some cases, the adviser may even hire a second pricing vendor to work up its own value on a troubled security or its own auditors will delve into pricing discrepancies.

Variances in pricing, of course, are to be expected. Haskin even recommends that “testing in multiple environments probably adds a lot of values in making the process work well,” meaning to weigh “different market circumstances” eg, a sharp market drop or stable markets.

Your annual report to the board should include a summary of your testing around fair valuation, he counsels. Some advisers look at pricing variations when they periodically test their valuation services, he adds. Rule 2a-5 requires reporting to the board on these issues at least quarterly.

Go low, middle and high

Another approach that some advisers use is to gain a low-, mid- and high-point valuation for harder-to-value securities, notes Balynsky. Testing can spot which point came closest to accuracy after a security is sold or after market conditions change.

“If you’re seeing a consistent, material difference” between valuations and a security’s end price, that’s when the fund board needs to ask why, he advises. You also could consider additional testing of your process.

At the very least, the board should review your valuation P&Ps, update them as necessary and ensure the P&Ps are being followed and cross-referenced to the requirements of rule 2a-5, continues Balynsky.

The key is to take a reasonable, consistent and “a very well-documented approach,” he recommends. You should be able “to supply an audit trail for the SEC” if examiners ask, he counsels.

CCOs “should be a voice in the room” but the real responsibility falls upon the board, he maintains. However, a CCO can ensure that the adviser’s valuation P&Ps match the nomenclature of the rule. At a minimum, SEC examiners would want to see this.