Urban and socio-economic correlates of property prices in Dublin’s area

class: center, middle, inverse, title-slide

# Urban and socio-economic correlates of property prices in Dublin’s area
## 2nd GoutAI Seminar - Rennes School of Business
### Damien Dupré
### Dublin City University - November 19th, 2021

---

# My Journey into Data Science

#### Developement of the DynEmo Facial Expression Database (Master)

* Dynamic and spontaneous emotions
* Assessed with self-reports and by observers

#### Analysis of Emotional User Experience of Innovative Tech. (Industrial PhD)

* Understand users' acceptance of technologies from their emotional response
* Based on multivariate self-reports

#### Evaluation of Emotions from Facial and Physiological Measures (Industrial PostDoc)

* Applications to marketing, sports and automotive industries
* Dynamic changes with trend extraction techniques (2 patents)

#### Performance Prediction using Machine Learning (Academic PostDoc)

* Application to sport analytics
* Big Data treatment (> 1 million users with activities recorded in the past 5 years)

---
class: inverse, mline, center, middle

# 1. A Closer Look at Property Prices

---

# Market Characteristics

An housing market is endemic to a country in a specific context.

.pull-left[
Countries are very different in term of:
- Architecture Styles/Composition
- Building Materials
- Legislation
- Urban features
- Population characteristics
]

.pull-right[
<img src="goutai_2021_files/img/slides_insertimage_1.png" width="100%" style="display: block; margin: auto;" />
.center.tiny[Ratio of Housing prices adjusted for inflation (base year 2015)<br />Credit:  Jeff Desjardins - Visual Capitalist (2019) [🔗](https://www.visualcapitalist.com/mapped-the-countries-with-the-highest-housing-bubble-risks/)]
]

Ireland can be considered as a housing bubble as it has one of the highest ratio of Housing prices adjusted for inflation in the world.

Almost 40% of Irish population leave in greater Dublin (1.9M/4.9M inhabitant) spread on a 318 km2 urban area resulting in one of the lowest population density in an European capital (4,811 inhabitant/km2).

---

# Market History

Housing prices are included in an economic context. Despite, having one of European highest economic growth (5.5% in 2019), Ireland suffered badly form the economic crash between 2008 and 2012.

<img src="goutai_2021_files/slides_files/figure-html/unnamed-chunk-2-1.png" width="864" style="display: block; margin: auto;" />
.center.tiny[Distribution of properties sold in Dublin's area between  2010 and 2018]

But since 2013, price have constantly risen with an average of 7% leading to a social crisis in Dublin where the number of homeless people has gone to the roof.

---

# A Data Science Insight

All properties purchased has to be published online on the [Property Price Register](https://propertypriceregister.ie):

- Since January 2010
- Filed by the owners
- Contains only the address and the price

In its current form, the property price register is only used by buyers to compare the price of their future acquisition with the price that houses has be sold in the same street.  
 
However it could also be used to have an overall view of the price distribution by geocoding the addresses using sing [Nominatim REST API from Open Street Map](http://nominatim.openstreetmap.org):

```r
address <- "10 Downing St, Westminster, London SW1A 2AA, United Kingdom"
api_url <- "http://nominatim.openstreetmap.org/search/@addr@?format=json&addressdetails=0&limit=1"

api_url |> 
  stringr::str_replace("\\@addr\\@", stringr::str_replace(address, "\\s+", "\\%20")) |> 
  jsonlite::fromJSON() |>
  dplyr::select(lat, lon)
#           lat                  lon
# 1 51.50344025 -0.12770820958562096
```

---

# Spatial Density with GAM

Once the lat/lon has been obtained, it is possible to estimate the typical property price in a 2D space:
- Use of Generalized Additive Model (GAM, see [Wood, 2017](Generalized additive models: An introduction with R))
- Application of a soap film smoother to solve the "finite area smoothing" problem due to the coastal shape of Dublin's area

<img src="goutai_2021_files/img/slides_insertimage_2.png" width="60%" style="display: block; margin: auto;" />
.center.tiny[Illustration of the "finite area smoothing" problem.<br />Credit: Gavin Simpson (2016) [🔗](https://fromthebottomoftheheap.net/2016/03/27/soap-film-smoothers/)]

---

# Spatial Density with GAM

.pull-left[
**Step 1.** A matrix has to be drawn with key knots within the shape to predict
]

.pull-right[
<img src="goutai_2021_files/slides_files/figure-html/unnamed-chunk-6-1.png" width="144" style="display: block; margin: auto;" />
.center.tiny[Key knots within Dublin's coastal area.]
]

**Step 2.** A Bayesian spline smoothing using restricted maximum likelihood is applied to estimate the smoothness of the property price variation ([Wood, 2011](10.1111/j.1467-9868.2010.00749.x); [Lin & Zhang, 1999](10.1111/1467-9868.00183)):

```r
gam_pred <- 
  mgcv::gam(
    price ~ s(lng, lat, bs = "so", xt = list(bnd = bound)), 
    data = data_dublin_2018_features, 
    method = "REML", 
    knots = knots,
    family = gaussian(link = "identity")
  ) |> 
  gam_density(too.far = 0.05, n.grid = 100)
```

---

# Spatial Density with GAM

The prices differences between areas reveal that underlying factors are influencing these changes.

.center[**Can we predict the property prices with Urban and Socio-Economic features? and can we identify the most relevant features?**]

---
class: inverse, mline, center, middle

# 2. Urban and Socio-Economic Correlates

---

# Method

In order to predict the property prices with Urban and Socio-Economic features, a machine learning algorithm is used: XGBoost (https://xgboost.readthedocs.io).

> XGBoost is a decision-tree-based ensemble Machine Learning algorithm that uses a gradient boosting framework. In order to make the final prediction, the training continues iteratively, inserting new trees that predict the residuals or errors of previous trees that are then combined with previous trees. It's called gradient boosting since, when introducing new models, it uses a gradient descent algorithm to minimise the loss.

> XGBoost is a great choice for a wide variety of real-world machine learning problems and is the most used for Kaggle Data Science Challenges (see [🔗](https://towardsdatascience.com/xgboost-lightgbm-and-other-kaggle-competition-favorites-6212e8b0e835))

The price of each property is related with:
- Its closest distance to each of 161 urban elements (e.g., bar, university, park, stadium, ...)
- Each of the 48 socio-economic characteristics of the geographic small where the property is situated

---
class: title-slide, middle

## Analysis 1: Urban Correlates

---

# Urban Correlates

Are property prices correlating with their distance to urban landmarks?

Calculation of shortest distance to each property GPS location with one of 160 urban features from the [Overpass API of Open Street Map](https://wiki.openstreetmap.org/wiki/Overpass_API).

For example, here is the query for all the pubs in Dublin:

.pull-left[

```r
library(osmdata)

opq("Dublin") |>
  add_osm_feature(
    key = "amenity",
    value = "pub"
  ) |>
  osmdata_sp() |>
  use_series("osm_points") |>
  plot()
```

]
.pull-right[

<img src="goutai_2021_files/slides_files/figure-html/unnamed-chunk-9-1.png" width="288" style="display: block; margin: auto;" />
.center.tiny[Location of results obtained from the query `amenity = pub` in Dublin using the R package {osmdata}.]
]

---

# Urban Correlates

Use these distances as predictors of the property price in a XGBoost model (tree based boosted regression, see [Chen & Guestrin, 2016](Generalized additive models: An introduction with R)). The dataset was randomly split in a train and in a test dataset (80% vs. 20%).

The results reveal that the **model explains 44% of the variance of the property prices** of the test dataset ...

<img src="goutai_2021_files/slides_files/figure-html/unnamed-chunk-11-1.png" width="504" style="display: block; margin: auto;" />
.center.tiny[Model's prediction accuracy using urban features *vs.* test dataset.]

... but some features are more important than others!

---

# Urban Correlates

Identify the contributing importance of each feature to the property price estimate.

This contribution (also called "importance") is a measure of the improvement in accuracy brought by a feature.

<table class="table" style="font-size: 14px; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> Feature Category </th>
   <th style="text-align:left;"> Feature Type </th>
   <th style="text-align:left;"> Importance </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> amenity </td>
   <td style="text-align:left;width: 10em; "> embassy </td>
   <td style="text-align:left;"> 14.8% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> natural </td>
   <td style="text-align:left;width: 10em; "> grassland </td>
   <td style="text-align:left;"> 5.3% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> amenity </td>
   <td style="text-align:left;width: 10em; "> bar </td>
   <td style="text-align:left;"> 2.3% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> place </td>
   <td style="text-align:left;width: 10em; "> island </td>
   <td style="text-align:left;"> 2.2% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> route </td>
   <td style="text-align:left;width: 10em; "> bus </td>
   <td style="text-align:left;"> 2.0% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> boundary </td>
   <td style="text-align:left;width: 10em; "> administrative </td>
   <td style="text-align:left;"> 1.9% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> power </td>
   <td style="text-align:left;width: 10em; "> line </td>
   <td style="text-align:left;"> 1.7% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> boundary </td>
   <td style="text-align:left;width: 10em; "> political </td>
   <td style="text-align:left;"> 1.6% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> barrier </td>
   <td style="text-align:left;width: 10em; "> wall </td>
   <td style="text-align:left;"> 1.5% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> power </td>
   <td style="text-align:left;width: 10em; "> portal </td>
   <td style="text-align:left;"> 1.3% </td>
  </tr>
</tbody>
</table>
.center.tiny[Top 10 most important feature according their contribution to the model.]

The proximity to **embassies** and **grasslands** is a driver of property prices.

---
class: title-slide, middle

## Analysis 2: Socio-economic Correlates

---

# Socio-economic Correlates

Is the distribution of population characteristics correlating with property prices?

Localisation of the properties within the "small area" (i.e. highest administrative resolution map) used for the [Irish Census 2016](https://www.cso.ie/en/). Each small area is summarised according a percentage of 48 socio-economic features (e.g. age, employment, civil status, religion, ...).

<img src="https://shanelynnwebsite-mid9n9g1q9y8tt.netdna-ssl.com/wp-content/uploads/2017/10/electoral-divisions-dublin-city-sapmap.png" width="60%" style="display: block; margin: auto;" />
.center.tiny[Divisions of Dublin small areas.<br />Credit: Shane Lynn (2018) [🔗](https://www.shanelynn.ie/the-irish-property-price-register-geocoded-to-small-areas/)]

---

# Socio-economic Correlates

The census characteristics of the small area containing the properties are used as predictors of the property price in a XGBoost model. Again, the dataset was randomly split in a train and in a test dataset (80% vs. 20%).

The results reveal that **the 48 socio-economic features explain 42.8% of the property price variance** of the test dataset.

<img src="goutai_2021_files/slides_files/figure-html/unnamed-chunk-15-1.png" width="504" style="display: block; margin: auto;" />
.center.tiny[Model's prediction accuracy using socio-economic features *vs.* test dataset.]

---

# Socio-economic Correlates

<table class="table" style="font-size: 14px; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> Feature Category </th>
   <th style="text-align:left;"> Feature Type </th>
   <th style="text-align:left;"> Importance </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> carers </td>
   <td style="text-align:left;width: 20em; "> Provides No Care </td>
   <td style="text-align:left;"> 28.8% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> housing rooms </td>
   <td style="text-align:left;width: 20em; "> 8 or more Rooms </td>
   <td style="text-align:left;"> 17.6% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> carers </td>
   <td style="text-align:left;width: 20em; "> Total Care Providers </td>
   <td style="text-align:left;"> 10.2% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> religion </td>
   <td style="text-align:left;width: 20em; "> No Religion </td>
   <td style="text-align:left;"> 3.8% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> disabilty age group </td>
   <td style="text-align:left;width: 20em; "> Persons with a disability aged 0 - 14 </td>
   <td style="text-align:left;"> 3.2% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> population </td>
   <td style="text-align:left;width: 20em; "> Age 0 - 14 </td>
   <td style="text-align:left;"> 2.7% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> religion </td>
   <td style="text-align:left;width: 20em; "> Other Catholic </td>
   <td style="text-align:left;"> 1.7% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> housing tenure </td>
   <td style="text-align:left;width: 20em; "> Owner Occupier with Mortgage </td>
   <td style="text-align:left;"> 1.5% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> housing rooms </td>
   <td style="text-align:left;width: 20em; "> 7 Rooms </td>
   <td style="text-align:left;"> 1.4% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> housing tenure </td>
   <td style="text-align:left;width: 20em; "> Social Rented </td>
   <td style="text-align:left;"> 1.4% </td>
  </tr>
</tbody>
</table>
.center.tiny[Top 10 most important feature according their contribution to the model.]

The **density of individual not providing regular unpaid personal help** in the area is the most important socio-economic feature to predict property prices.

**Large properties** in the area and the proportion of **individuals reporting having no religion** are also relatively important.

---
class: inverse, mline, center, middle

# 3. Conclusion

---

# Conclusion

By preforming a feature analysis with urban and socio-economic features, it is possible to evaluate and predict the potential price of a property.

- The presence of embassies or parks are criteria that influence significantly the price of properties.

- Similarly, the characteristics of inhabitants in the area such as religion, health and age is correlated to the evolution of housing prices.

These results allow an understanding of why some areas have higher prices than others which is relevant information not only for real estate agents in charge of property valuations but also for buyers in order to estimate the real value of a property.

---

# Limitations and Perspectives

Still an ongoing work, more to follow in the coming weeks...

<span><i class="fas  fa-exclamation-triangle faa-flash animated faa-slow " style=" color:red;"></i></span> Essential features haven't been taken into account:
- House characteristics (e.g., square meter size, number bedrooms, ...)
- Macroeconomic trends (e.g., economic growth, unemployment, ...)

<span><i class="fas  fa-exclamation-triangle faa-flash animated faa-slow " style=" color:red;"></i></span> Essential features are difficult to assess:
- Market size
- Market demand

A more global view is required to confirm the results:
- Full country perspective
- Comparison between cities

A temporal perspective would also be interesting to study

---

# Acknoledgment

Many thanks to the contributors and developers of the OpenStreetMap project.

.pull-left[
Many thanks to the developers and contributors of R and RStudio as well as to those of the packages used:
- {tidyverse}
- {magrittr}
- {mgcv}
- {xgboost}
- {OpenStreetMap}
- {osmdata}
- {rgdal}
- {sf}
- {sp}

... to name the main ones.
]
.pull-right[

More in "Geocomputation with R" (Lovelace, Nowosad, & Muenchow, 2021), a book on geographic data analysis, visualization and modelling.

<img src="https://geocompr.robinlovelace.net/images/cover.png" width="50%" style="display: block; margin: auto;" />
.center.tiny[Geocomputation with R.<br />Credit: Lovelace, Nowosad, & Muenchow (2021) [🔗](https://geocompr.robinlovelace.net)]
]

---
class: inverse, mline, left, middle

# Thanks for your attention, find me at...

[<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"/></svg> @damien_dupre](http://twitter.com/damien_dupre)  
[<svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"/></svg> @damien-dupre](http://github.com/damien-dupre)  
[<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M326.612 185.391c59.747 59.809 58.927 155.698.36 214.59-.11.12-.24.25-.36.37l-67.2 67.2c-59.27 59.27-155.699 59.262-214.96 0-59.27-59.26-59.27-155.7 0-214.96l37.106-37.106c9.84-9.84 26.786-3.3 27.294 10.606.648 17.722 3.826 35.527 9.69 52.721 1.986 5.822.567 12.262-3.783 16.612l-13.087 13.087c-28.026 28.026-28.905 73.66-1.155 101.96 28.024 28.579 74.086 28.749 102.325.51l67.2-67.19c28.191-28.191 28.073-73.757 0-101.83-3.701-3.694-7.429-6.564-10.341-8.569a16.037 16.037 0 0 1-6.947-12.606c-.396-10.567 3.348-21.456 11.698-29.806l21.054-21.055c5.521-5.521 14.182-6.199 20.584-1.731a152.482 152.482 0 0 1 20.522 17.197zM467.547 44.449c-59.261-59.262-155.69-59.27-214.96 0l-67.2 67.2c-.12.12-.25.25-.36.37-58.566 58.892-59.387 154.781.36 214.59a152.454 152.454 0 0 0 20.521 17.196c6.402 4.468 15.064 3.789 20.584-1.731l21.054-21.055c8.35-8.35 12.094-19.239 11.698-29.806a16.037 16.037 0 0 0-6.947-12.606c-2.912-2.005-6.64-4.875-10.341-8.569-28.073-28.073-28.191-73.639 0-101.83l67.2-67.19c28.239-28.239 74.3-28.069 102.325.51 27.75 28.3 26.872 73.934-1.155 101.96l-13.087 13.087c-4.35 4.35-5.769 10.79-3.783 16.612 5.864 17.194 9.042 34.999 9.69 52.721.509 13.906 17.454 20.446 27.294 10.606l37.106-37.106c59.271-59.259 59.271-155.699.001-214.959z"/></svg> damien-datasci-blog.netlify.app](https://damien-datasci-blog.netlify.app)  
[<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M476 3.2L12.5 270.6c-18.1 10.4-15.8 35.6 2.2 43.2L121 358.4l287.3-253.2c5.5-4.9 13.3 2.6 8.6 8.3L176 407v80.5c0 23.6 28.5 32.9 42.5 15.8L282 426l124.6 52.2c14.2 6 30.4-2.9 33-18.2l72-432C515 7.8 493.3-6.8 476 3.2z"/></svg> damien.dupre@dcu.ie](mailto:damien.dupre@dcu.ie)