As might have been anticipated, my recent article providing a guide to the current portfolio of providers of online property valuations models triggered the inevitable question – "just how accurate are they?"
So I thought I would do some desk research. However before I unleash a barrage of criticism stating that there are heaps of examples where the Automated Valuation Models (AVM’s) are so wide of the mark to make them laughable, let me simply say this. There over 1.5 million AVM’s or potential AVM's for NZ properties – there will always be outliers and extremes. I do not have time nor patience to review thousands of properties, or even hundreds of properties. I chose to select just 12 properties.
The method I have used, is to track the latest auction results as published by the team at Interest.co.nz as the auction year started after Christmas. I simply took the first 12 I saw which comprised 8 properties in Auckland and 4 in Tauranga. So again I acknowledge that my sample is hardly representative nor truly random. It is made up of auction sales only, the sales are only for those 2 areas of the country and represented a very quiet period of the year.
With these 12 property sales results I went to each of the 5 providers:
- Trade Me Property Insights
- Realestate.co.nz Property Profile
- MyValocity (you do need to create an account to use the AVM)
- QV – mobile app for iOS and Android
I knew none of these providers had updated their valuations to take account of any of these actual 12 sales neither would the sale records have been picked up through local council sales or agent reporting so there was no bias of an AVM being influenced by these recent sales.
Another point to note is the analysis compared the sale price at auction to the mid-point of the price range of the AVM.
So here is the table of results. The colour code used is blue where the AVM equalled the sale price exactly, red signifies an AVM below the sale price with green where the AVM is above sale price. Finally, grey indicates that the provider had no AVM for the property.
As you can see, the visual skew towards red indicates that based on this sample set most AVM’s were below sale price.
The original version of this article I used an average variance measure, after receiving valuable feedback I have now used the calculation of Gross Median Error.
All providers achieved a gross median error of less than 10%, with Realestate.co.nz achieving less than 5% which is impressive. I would deduce that a factor in their accuracy, is they benefit from the very latest REINZ data each month of unconditional sales, whilst all other provides rely largely on settled sales which come through at least a month to 2 months later.
Another perspective I was keen to examine in respect of the accuracy of AVM's was the indicative range they provide to reflect the level of confidence. For each provider, for each property I assessed the range as a percentage of the midpoint price.
This analysis is very illuminating. The provider with the tightest range (in theory indicating confidence factor) is MyValocity, closely followed by Homes, both just under 10%. This effectively meaning that their AVM range is 5% below the midpoint to 5% above which I would judge as fairly acceptable given this is a computer based estimation with no detailed knowledge of the specifics of the property.
Of interest in this analysis is the very wide margin in the range from Trade Me Property at close on 30% with their tightest range being for a single property at just 19%. Similarly Realestate.co.nz seem to apply a standard c.21% to all AVM’s.
For completeness here are the raw numbers