This article discusses several ways you can use Nearmap AI to automate your workflow, increase efficiency, and and ensure your data is more accurate and up-to-date.
Using AI Parcel Data to Refine an Existing Source of Truth
You may already have a source of truth. For example, a solar installer will have data on location of solar array installations that is sourced from smart meters, government, utility installers, or other sources on where solar arrays are installed.
These data sources are often infrequently updated, or even gathered as an initial one-off manual effort.
Artificial Intelligence (AI) from aerial imagery can:
- refine and correct the existing source of truth by intersecting it with one that relies on a fundamentally different perspective; what is visibly present at a particular point in time.
- continually and automatically keep that source of truth up to date over time, allowing you to highlight areas of change.
- be consistent across our coverage areas due to our capture program, where many other data sets have a high degree of regional variation.
How do I make the most of both sets of data?
It is very useful to have two different sources of truth with different perspectives. What is easy for one source of truth can be very challenging for another.
The advantages of comparing two different sources of truth are:
- Where both data sources agree, this boosts confidence that those results are correct.
- Aerial image features can highlight obvious errors in the existing source of truth, potentially allowing significant quality improvements, or identification of causes of error in your existing processes. We've often found customers surprised at the errors in their own data that are highlighted when they compare it with ours.
- Note that your existing source of truth may highlight areas where our aerial imagery derived features are incorrect. They will typically be low confidence results, which can be ignored.
How do I know which data is correct?
When you have two sources of truth, where there is a disagreement about a particular example, it is critical to distinguish who is correct.
Our AI Parcel Data provides you with a confidence score for a result based on machine learning from imagery. This is a probability which represents how confident the model is about whether it is correct for a given decision. You can read more here: Confidence.
Because our confidence measure tells you how confident we are about each and every decision, you can use what you know about your own data, and decide when to override it with our decision – a low confidence gentle suggestion, or a highly confident match that is very unlikely to be incorrect.
Here's how your workflow could look:
- Start by identifying the part of your data that overlaps with ours (e.g. presence of solar panel at an address).
- Identify a key on which to join the data sets (such as a parcel identifier, or address centroid), and join them.
- Filter for just the cases where our data disagrees with yours.
If they are well matched (on date, and definition), this may only be a few percent of cases.
If there is an approximate match only (e.g. your definition of construction is as soon as a DA is submitted, rather than commenced, or the data comes from different dates), then the level of disagreement may be much higher.
- Sort from highest to lowest confidence.
Nearmap AI highest confidence results are most likely to be correct, and present opportunities for you to update your data set. Our lowest confidence results are most likely cases where your data is strong, and should be left unchanged.
- Look at various examples in Map Browser as you work down the list, to determine a confidence cutoff where you are comfortable overriding your data with ours.
An example Nearmap AI parcel data CSV file
One practical example of this is with solar panels. If a utility has a data set based on documented installs, there will be examples where the panels were removed (due to a knock down rebuild, or the paperwork was lost). These are likely to come up as high confidence results with Nearmap AI – the reason it is challenging for the utility has nothing to do with the visual appearance.
On the other hand, there are cases where the utility's data set is likely to be stronger, and lower confidence scores will be reported by Nearmap AI. The Sydney Olympic village has many examples of very small solar arrays that were installed in the year 2000. These are likely to be well documented in the data set, but are visually challenging (the two panel, 20+ year old arrays with a very wide bezel look very much like skylights). See the image below.
Low confidence results on houses in Sydney Olympic Village, with small 20 year-old solar arrays.
While a workflow like the one described above can further enhance the value you get from Nearmap AI, it's important to ensure correct interpretation and a useful outcome, rather than a focus on where Nearmap AI “went wrong”.
Using AI Parcel Data to Scale Manual Effort
If you have no existing full scale source of truth and you are looking at having to create one manually, you can use Nearmap AI Parcel Data to create a stand alone source of truth.
You may, for example, work in a local government organisation, assessing which properties have been developed from one period to the next to assist with infrastructure planning and correct taxing. You may be mapping the change in an estuary over time to assist management of fishery stocks. In practice people are employed to perform the most valuable parts of this analysis by hand, with a high degree of compromise. Yet it is not feasible to keep this data updated in real time optimally via manual effort.
In order to obtain value in such a situation, it may be necessary to redesign business processes that interact with the existing manual effort. If your role is focussed on a manual task, consider whether you can use Nearmap AI results to do it more efficiently. Such an approach frees up time to spend on higher value activities such as assessing the quality of the automated results (and adjusting if needed), performing the analysis more frequently, or performing it at greater scale and precision.
Using AI Parcel Data to Detect Change Over Time
The built environment is a rapidly changing place, and you may have a strong interest in identifying which parts are changing. The major advantages of aerial imagery over satellite is the frequency and detail of data. With up to six captures a year in some locations, Nearmap allows you to refresh whole cities'-worth of content on a regular basis.
With Nearmap AI Parcel Data, it is very easy to detect change between two exported data sets (described below). The challenge is in defining what 'change' means to you. A comparison of exports at two dates will reveal legitimate changes due to construction, growth of trees, etc. However, it will also identify any results that changed due to things such as seasonality in vegetation and lighting, or temporary structures such as shade cloths. In practice, you'll want to first define what you mean by change (a vacant lot having a building appear, or perhaps the change in area of buildings on the lot). Comparing two exports can reduce the data set from say 100,000 properties, to a few thousand with potential changes to inspect.
Your first step is to sign up to Nearmap AI and get to know the data in your area - establish it as a source of truth. Our current Nearmap AI subscriptions give you the ability to do an annual refresh, so in a year's time, you will have a fully updated data set that you can compare against your first one. If you need to move faster, by getting two dates to compare when you sign up, please talk to your Nearmap Account Manager about offline delivery.
Once I have AI Parcel data from two dates, how do I detect change?
One method for reducing errors is to look at simple binary changes of your parcel data, for example when a property goes from having no solar array to having one, a swimming pool is removed, or a new building appears on a vacant lot. This is robust to a number of errors – slight shifts in location or extent of a detected object due to imagery shift, parallax error and lighting. You can make it even more robust by only looking at changes with high confidence, as shown in the example below.
Note that this simple form of change detection can only find major changes such as objects being created or demolished. More subtle changes will be ignored, such as when a solar array is expanded from 10 to 20 panels, or an extension is added to a dwelling.
You can extend this approach by taking advantage of the two tiers of AI content you can export from MapBrowser: the total area of objects of a particular type (and measuring change) from the spreadsheet file, or counting the number and area of individual objects through geospatial calculations on the rich geospatial file.
While still being robust to reasonably large shifts due to parallax error and on-the-ground accuracy, looking at variations in area in the spreadsheet file can reveal changes such as the installation of a new shed, an extension, or an expanded PV array.
You can use a workflow similar to the workflow described earlier to sort the magnitude of change (difference in object counts or area), where the largest area of change is a useful proxy for the confidence that the change is real. This workflow can determine the point at which you can ignore changes below a certain threshold, depending on your requirements.
A working example
Say you have two csv AI Parcel rollup files (exported from MapBrowser) from two different dates. By consolidating the two separate spreadsheet files, you can easily compare which parcels increase or decrease in total building area.
You can code a simple python script to consolidate the two csv files. A code example is shown below. Please note that Nearmap does not provide any guidance on coding and this is shown for illustration purposes only.
You can see that the third-last line of the sample code below shows how to combine the MapBrowser location URLs by separating the survey date with a pipe (e.g. https://apps.nearmap.com/maps/#/@-33.6763300,150.8855400,21.00z,0d/V/20180730|20190912) . This produces the very handy split view in MapBrowser (shown in the screenshot in step 4 that will help you visually inspect the potential change to see if it meets your criteria for a meaningful change).
Filter in excel.
- You can see in the area_change column, shown in the image above, both increase and decrease in area in area (which is the negative value) are pretty reliable in terms of building changes.
- Click the map_browser_change link to open in split view. (You may be able to view the example below, depending on the coverage of your Nearmap subscription.)
Want to know more?
We are here to help. Make sure you read out other AI articles to help you understand what packages are available with Nearmap AI and how to visualise AI layers and Export AI parcel data from MapBrowser.