Online Property Valuation Models – how accurate are they?

by Alistair Helm in


iStock_000004287274Small.jpg

As might have been anticipated, my recent article providing a guide to the current portfolio of providers of online property valuations models triggered the inevitable question – "just how accurate are they?"

So I thought I would do some desk research. However before I unleash a barrage of criticism stating that there are heaps of examples where the Automated Valuation Models (AVM’s) are so wide of the mark to make them laughable, let me simply say this. There over 1.5 million AVM’s or potential AVM's for NZ properties – there will always be outliers and extremes. I do not have time nor patience to review thousands of properties, or even hundreds of properties. I chose to select just 12 properties.

The method I have used, is to track the latest auction results as published by the team at Interest.co.nz as the auction year started after Christmas. I simply took the first 12 I saw which comprised 8 properties in Auckland and 4 in Tauranga. So again I acknowledge that my sample is hardly representative nor truly random. It is made up of auction sales only, the sales are only for those 2 areas of the country and represented a very quiet period of the year.

With these 12 property sales results I went to each of the 5 providers:

I knew none of these providers had updated their valuations to take account of any of these actual 12 sales neither would the sale records have been picked up through local council sales or agent reporting so there was no bias of an AVM being influenced by these recent sales.

Another point to note is the analysis compared the sale price at auction to the mid-point of the price range of the AVM.

So here is the table of results. The colour code used is blue where the AVM equalled the sale price exactly, red signifies an AVM below the sale price with green where the AVM is above sale price. Finally, grey indicates that the provider had no AVM for the property.

As you can see, the visual skew towards red indicates that based on this sample set most AVM’s were below sale price.

The original version of this article I used an average variance measure, after receiving valuable feedback I have now used the calculation of Gross Median Error.

All providers achieved a gross median error of less than 10%, with Realestate.co.nz achieving less than 5% which is impressive. I would deduce that a factor in their accuracy, is they benefit from the very latest REINZ data each month of unconditional sales, whilst all other provides rely largely on settled sales which come through at least a month to 2 months later.

Another perspective I was keen to examine in respect of the accuracy of AVM's was the indicative range they provide to reflect the level of confidence. For each provider, for each property I assessed the range as a percentage of the midpoint price.

This analysis is very illuminating. The provider with the tightest range (in theory indicating confidence factor) is MyValocity, closely followed by Homes, both just under 10%. This effectively meaning that their AVM range is 5% below the midpoint to 5% above which I would judge as fairly acceptable given this is a computer based estimation with no detailed knowledge of the specifics of the property.

Of interest in this analysis is the very wide margin in the range from Trade Me Property at close on 30% with their tightest range being for a single property at just 19%. Similarly Realestate.co.nz seem to apply a standard c.21% to all AVM’s.


For completeness here are the raw numbers

 


Making sense of online property valuation models

by Alistair Helm in


couple engagement online shutterstock_3467780.jpg

Having cited the democratisation of property data as the most significant event to occur in the real estate industry over the past 3 years, I thought it would be a useful follow up, to provide some insight and perspective as to this new world of more accessible property data and by so doing provide more context as to these estimated valuations as compared to traditional valuation providers.

I do also propose in a follow up article to review each of these new providers and assess their relative strengths and weaknesses.

There are currently I judge five key players in the market offering online estimated valuations for NZ property. These are Homes, Trade Me Property, Realestate.co.nz, MyValocity and QV (a note, QV only provide a free estimated valuation model on their mobile app – their website still requires the purchase of an e-Valuer report at $49.95 per property).

All of these operations provide a free unlimited online automated valuation on pretty much all properties in NZ. Well actually not every property. The fact is all of these providers recognise that without sufficient proximate data from which to compute their algorithm they cannot attribute a reasonable estimate to every property, so not every property will have a valuation estimation. It is likely that the more remote the location, the more rural, the less frequent the number of local sales the less likely there will be for a estimated valuation.

Let me expand upon this as an insight as to how these Automated Valuation Models (AVM) work. Each of these companies leverage the now easily accessible massive computing power that only a few years ago was the reserve of major corporate and government agencies. The likes of Amazon Web Services, Google Cloud Platform and Microsoft Azure to name but a few, which offer massive computing capacity just when you need it – in this case allowing these local companies to rent a couple of hours of grunty computing power to run algorithms that analyse the impact of all recent local sale records for all properties. This is basically how the algorithm works. Each property record is assessed against recent sales of similar properties (similar by standard metrics of for example number of bedrooms, size of property being the two most important).

The key data point here is recent sales; the more recent the sale, then the more accurate the estimation. Naturally there is a lot more sophistication in each company’s algorithm than I have outlined here, including self-learning tools to assess the system’s accuracy by effectively going back and estimating a property sale before the actual sale is confirmed and then reviewing actual sale price against estimation.

Powerpoint_images_2018_WIP.png

AVM’s tend to be displayed on these various platforms as a price range with a mid-point. This is the result of statistical convention rather than a true sense of a predictive range. In my view look at the mid-point of the range as an indication of the AVM rather than the upper price! A range can often be as broad as 15% or even 20% either side of the mid-point which at times makes them seem very inaccurate. The fact is, the computer algorithms compute a single figure together with a confidence factor which then drives the scale of this range.

That is the complex part of Automated Valuation Models (AVM). The key question though is, can you, and should you, trust these estimation valuation models in the marketplace as a guide to better inform you as to an indication as to the likely selling price of a property?

Before I go into that, it is really important to lay out the difference between a number of data points that you are likely to come across in terms of assessing the value of a property. I have detailed below the 5 valuation data points in descending order of accuracy.

The Selling Price – this is ultimate statement of the true value of a property. This is the price at which a willing seller accepted an offer from a willing buyer. This valuation is 100% accurate, but at the same time ephemeral, as it is a moment-in-time judgement and will never be repeated because circumstances with the property market at a hyperlocal level change all the time.

A Registered Valuation – this is the most accurate estimate of a property's value and that is why it is insisted upon by banks and lending institutions who are prepared to take on the risk against which they lend. Registered Valuations are undertaken by a professional valuer, a person who has undergone extensive training and education spanning many years. Such valuations, often cost many hundreds of dollars. Registered valuers use recent sales and local knowledge to provide a very detailed written assessment of what a property is worth in today’s market. There is also professional indemnity that lies behind the valuation report.

A Real Estate Appraisal – a licensed real estate professional will provide a client with an appraisal as to what a property would expect to fetch in today’s market. Under the guiding rules of the Real Estate Agents Act 2008, it is a requirement that a salesperson provide a client with such an appraisal before signing an agreement to list and market their property. Such appraisals need to identify a price or a range, ideally not exceeding 5%. Such an appraisal is computed using a comparative market assessment of what properties of similar size and features have sold for recently. Additionally an appraisal will look at the hyper-local market conditions of supply and demand which a local agent is uniquely able to assess.

An Automated Valuation Model – as outlined above this computer based model is undertaken by the leading five online providers and is based on raw data with no human intervention or local insight.

A Rateable Value – this is a valuation developed for local authorities and undertaken every 3 years in order to provide a benchmark upon which local rates can be assessed. The Rateable Value is judged to be the likely selling price at the time the assessment is made and therefore this estimation decays pretty quickly afterwards. Largely the model used by the providers of this service for the local authorities matches the computer based AVM.

As a prospective buyer or seller the question is, which of these data points should you look at, when and why? Here is my opinion.


What a property seller should do?

Agent and clients shutterstock_165593015.jpg

If you are looking to sell a property it is very useful to keep an eye out for local sales results – many agents nowadays will provide such report at open homes, and of course Homes / Trade Me / Realestate / MyValocity can provide this data although not all provide email alerts of recent sales in your area (Homes does a great job of this). In addition, it does no harm to review the AVM estimate for your own property, it’s a valuable guide. Again Homes offers the ability for you to ‘own’ a property record and receive monthly emails of the latest valuation and market trends.

When you are ready to go to market with your property choose your licensed real estate salesperson and get them to provide an appraisal which will give you their valuation estimation which will be most likely based on selected comparable recent sales of properties that best match your property. Their appraisal report will identify these comparable properties thereby allowing you to discuss and debate the merits of your property versus others. This appraisal is the best indicative valuation you can get without investing in a registered valuation.

The estimated valuation in an appraisal will be either a single figure or a range and in this case best practice says that the range should be no more than 5% overall, which means between 2.5% above and below the mid-point. So for example a range of say from $535,000 to 560,000 would be acceptable.

The appraisal you receive may utilise a couple of well recognised models as well as comparable sales. These being net rate / replacement cost or capitalisation of income. I won’t dwell on these other methods here, aside than to say a professional real estate salesperson will used their skills and knowledge to arrive at an estimated valuation that is the best in the market.


agent buyers shutterstock_171921932.jpg

What a property buyer should do?

If you are looking to buy, I would recommend the same approach of keeping a watchful eye on local sales and see what properties, like the one you fancy buying are selling for. Certainly, review the online service providers to see what the properties that come on the market are valued at based on AVM’s. Trade Me is great at this, in providing a link from most listed properties to the AVM on their Property Insights section.

I would recommend that when you get closer to the decision-making process of buying you chat with the agent for the property you are interested in, and discuss with them the view they hold as to price range and how that may differ from the AVM online – they will be only too keen to share the reasons why they view that their price judgement is more reflective of the local market conditions. Listen closely as they are working every day in the market and their insight is critical.

If as a buyer you require finance on the property you choose, you will likely need a registered valuation as the bank or lending institution will insist upon it, however I would take that lead from the lender rather than rush into requesting a registered valuation before you are certain on the property purchase. Remember obtaining a valuation as part of the financial conditions of a conditional offer for a property is perfectly acceptable.

The one estimated valuation I have omitted to mention in this process is the Rateable Value. The role of the RV has now finally gone. It can finally be ignored and retired from the lexicon of property transactions – interestingly something I suggested back in 2013!