Can you trust an AVM to “value”your KLCC Condominium?

The rapid adoption of Automatic Valuation Models (AVMs) to provide free estimates of Current Market Values (CMVs) by EdgeProp and PropertyGuru was I think a major driver of traffic to their phone apps and websites. But how many of their users understand the data science behind AVMs? What kind of data inputs do their AVMs use ? And perhaps more importantly, how "accurate" are the Current Market Values predicted by the consumer grade AVMs used by the 3 major property portals?

Yours truly found a couple of published scholarly articles on the use of AVMs in Malaysia but these don't tackle the questions above. But I also found an article on testing AVMs for accuracy by a leading US provider of professional and consumer AVMs. Yours truly suggests users might want to ask PropertyGuru, EdgeProp etc about the industry metrics used to measure the accuracy of their in-house AVMs.

At the time of posting, I think but cannot be 100% certain none of the major property search portals which deploy consumer AVMs as a marketing tool to get more FTHBs and property investors to use their apps are prepared to disclose their scores on the industry metrics of AVM accuracy. But I think an industry body- PEPS- has already reviewed some professional grade AVMs produced by software houses and presumably has also endorsed their use by registered valuers in Malaysia.

Industry metrics For AVM Accuracy:

Hit rate

The number of benchmark addresses the AVM was able to predict on. The hit rate is based on the AVM’s ability to locate the address, or how much confidence the AVM has on its prediction of value for the address. Often, AVMs do not (and should not) produce results when they don’t have strong confidence in the valuation.

P10 (or PPE10)

The percentage of time the AVM is within 10 percent of each benchmark. A standard measure for the AVM industry, but does not reflect on the AVMs that are very far off from the benchmark (the spread).

MAE (mean absolute error)

Calculating the percent variance between each AVM and benchmark, then taking the absolute value of each, and averaging them over the whole test set. A good measure if using consistent benchmarks since it accounts for the times the AVM was very wrong and can throw off the MAE. In other words, it accounts for the outlier predictions.

MdAE (median absolute error)

Similar to MAE, but since it used the median, it hides the times the AVM was really far off; the outliers. This is the most common measure for consumer-grade (not professional/lending-grade) AVMs

Based on an article by Clear Capital

Copyright PG Datasense