Tuesday, January 13, 2015
Need data? grab from wikipedia with sparql!
Recently We have an interesting requirement in one of the project that we have. The project basically need the web app that we build to be able to calculate the distance between a space with the nearest MRT station, and flag the space to be “near MRT” if the distance is within 500 meters.
One of the challenge for this is actually to find the location data of all MRT station in Singapore store it in our database and use it for calculation of distance.
Somehow it was very difficult to find a good and updated location data. And guess what, we found one of the best source of data is actually wikipedia!
this is one of wikipedia page showing Pasir Ris MRT Station. In the page reader will be able to find the exact location of the station in terms of latitude and longitude. And wikipedia have the data of more than 100 MRT stations covering all the lines in the MRT networks (unfortunately it doesn’t include the data of the upcoming new MRT stations).
So now how to grab all this data?
this is where sparql come to the rescue :).
Sparql stands for Sparql protocol and RDF query language. It’s basically a query language to retrieve and manipulate data that stored in RDF (Resource Description Framework) format, like how SQL is used to retrieve data from relational database.
to try sparql to grab wikipedia, there is an awesome website that have all the wikipedia data stored in RDF format. The website named DBpedia. Here is the same Pasir Ris MRT Station page in RDF format. This site also give a nice interface to use sparql to retrieve data.
This is the query that I used to retrieve all the Singapore MRT station latitude and longitude :
One of the challenge for this is actually to find the location data of all MRT station in Singapore store it in our database and use it for calculation of distance.
Somehow it was very difficult to find a good and updated location data. And guess what, we found one of the best source of data is actually wikipedia!
this is one of wikipedia page showing Pasir Ris MRT Station. In the page reader will be able to find the exact location of the station in terms of latitude and longitude. And wikipedia have the data of more than 100 MRT stations covering all the lines in the MRT networks (unfortunately it doesn’t include the data of the upcoming new MRT stations).
So now how to grab all this data?
this is where sparql come to the rescue :).
Sparql stands for Sparql protocol and RDF query language. It’s basically a query language to retrieve and manipulate data that stored in RDF (Resource Description Framework) format, like how SQL is used to retrieve data from relational database.
to try sparql to grab wikipedia, there is an awesome website that have all the wikipedia data stored in RDF format. The website named DBpedia. Here is the same Pasir Ris MRT Station page in RDF format. This site also give a nice interface to use sparql to retrieve data.
This is the query that I used to retrieve all the Singapore MRT station latitude and longitude :
awesome stuff!!
@deltawidyangga
Subscribe to:
Comments (Atom)