As might have been anticipated, my recent article providing a guide to the current portfolio of providers of online property valuations models triggered the inevitable question – "just how accurate are they?"
So I thought I would do some desk research. However before I unleash a barrage of criticism stating that there are heaps of examples where the Automated Valuation Models (AVM’s) are so wide of the mark to make them laughable, let me simply say this. There over 1.5 million AVM’s or potential AVM's for NZ properties – there will always be outliers and extremes. I do not have time nor patience to review thousands of properties, or even hundreds of properties. I chose to select just 12 properties.
The method I have used, is to track the latest auction results as published by the team at Interest.co.nz as the auction year started after Christmas. I simply took the first 12 I saw which comprised 8 properties in Auckland and 4 in Tauranga. So again I acknowledge that my sample is hardly representative nor truly random. It is made up of auction sales only, the sales are only for those 2 areas of the country and represented a very quiet period of the year.
With these 12 property sales results I went to each of the 5 providers:
- Trade Me Property Insights
- Realestate.co.nz Property Profile
- MyValocity (you do need to create an account to use the AVM)
- QV – mobile app for iOS and Android
I knew none of these providers had updated their valuations to take account of any of these actual 12 sales neither would the sale records have been picked up through local council sales or agent reporting so there was no bias of an AVM being influenced by these recent sales.
Another point to note is the analysis compared the sale price at auction to the mid-point of the price range of the AVM.
So here is the table of results. The colour code used is blue where the AVM equalled the sale price exactly, red signifies an AVM below the sale price with green where the AVM is above sale price. Finally, grey indicates that the provider had no AVM for the property.
As you can see, the visual skew towards red indicates that based on this sample set most AVM’s were below sale price.
The average variance across the 5 providers for the 12 properties was just 3%, which is pretty amazing as the average sale price was $1,000,000 so that represents just a $30,000 variance.
The most accurate overall provider was MyValocity purely when averaging out overall accuracy with a -1.6% variance, although they did report the largest spread of +19% to -19%
Homes achieved the unique success of calling two properties on the nose! Whilst Realestate.co.nz was the tightest range across the portfolio of properties from -12% to +9%
The QV app was the worst performing for this sample set with an average variance of 6.3%, whilst all other providers were below 4%, with Realestate.co.nz and MyValocity below 2%.
Another perspective I was keen to examine in respect of the accuracy of AVM's was the indicative range they provide to reflect the level of confidence. For each provider, for each property I assessed the range as a percentage of the midpoint price.
This analysis is very illuminating. The provider with the tightest range (in theory indicating confidence factor) is MyValocity, closely followed by Homes, both just under 10%. This effectively meaning that their AVM range is 5% below the midpoint to 5% above which I would judge as fairly acceptable given this is a computer based estimation with no detailed knowledge of the specifics of the property.
Of interest in this analysis is the very wide margin in the range from Trade Me Property at close on 30% with their tightest range being for a single property at just 19%. Similarly Realestate.co.nz seem to apply a standard c.21% to all AVM’s.
For completeness here are the raw numbers