Streetfood Datastory
full version of the datastory in pdf link
With our analysis we tried to get in touch with the sadly more and more vanishing culture of Shanghai Street Food. Our first visualization shows the results of our research concerning people’s opinions and behaviour on that topic. The second graphic shows a more informal way of how street food influences the shape of the city.
Introduction
Our Data Journey starts with 6 different people, 6 different ideas, 3 different nations, 4 different ages, 2 different genders and 1 common goal: Finding a topic for our data visualisation. Since analytic skills seemed to be a likely skill for data visualisation, we decided to analyse and compare the 6 ideas concerning meaningfulness; data availability; data diversity; knowledge gathering potential. It took two turns of election to choose the winning topic between Tourism, Water, Happiness, Education, Marriage and Street Food（figure 1) . In the end Street Food took over Water the pole position. By choosing this topic we hoped to get a better understanding of this culture and why it is vanishing. We wanted to see the development over time, find out the peoples opinions on the topic and maybe try to find trends. This decision led us to some serious problems, apparently.
Where to get the data?
How to gather data about a topic which is barley illegal and greatly controversial? During our topic discussions we already figured that a good possible data source could be social media as well as a direct feedback from people via questionnaire.
Social Media
To choose the right targets to search for data, we needed to think about the way street food would/
might appear in social media. The first guess was as pictures. People like what they see, take
pictures of the food and share it somehow. So we decided to go for a picture-based social website,
in specific Flickr. Why Flickr? We thought that a website which focuses completely on pictures may
provide well structured amount of meta data of those pictures like Time, Location, ans Tags. On
other good option to share food experience we thought must be posting. People express their
experiences simply with words. One of the biggest platforms for this is Weibo. Which made the
website out second target. Of course these post might come along with pictures as well but since
we didn’t lay any focus on the pictures it self but only on the data they provide we decided that for
Weibo the posts themselves offer the greater amount of data. The posts also made it possible to
collect data manually. But of course we also had to use the APIs to get to the data. And since no
one of us ever used an API and only one has some proper experience with HTML this wasn’t
particularly an easy task. Which keyword to search? For flickr we use "Shanghai street food", and for Weibo, several
Chinese words like " 路路边摊 街边⼩小吃 街头⼩小吃 路路边⼩小吃 "( figure 2) are used to get the
datasets of street food. We choose 2013.05-2017.04 as targeting period for the Weibo
since four years ago there is still lots of street foods in Shanghai.And with the help of a
software Gooseeker, we are able to get enough data download from Weibo.
Questionnaire
Setting up a questionnaire is technically quite an easy task. But yet should be conceptual
thought through. We had to set it up as quick as possible in order to get as much responses
as possible in the short time. We also made 2 different language versions. It was necessary
to balance between getting enough data and keeping it convenient to answer. Our 10
questions contains basic demographic questions and strictly topic related ones. But the
most difficult part for questionnaire is actually to find people to answer it, especially if you
don’t hold large contact data bases. For the english survey the initial plan was to post the
link to the survey among expat-forum-websites and travellers websites. Unfortunately these
website have restrictions for posting links. So we needed to spread it manually.
This went down with a hoped to be kind of snowball system using WeChat , WhatsApp,
Facebook, Weibo, Chatrooms and Email to distribute our surveys.
-Where are you from?
-How long was your stay?
-Did you try Street Food?
-If No, why?
-What kind(s) did you try?
-By any chance, do you remember near which Metro Station it was?
-What is your gender?
-Please tell us your age.
-Which of the following best describes your current occupation?
-Street Food is more and more vanishing from the streets of Shanghai. What are your
thoughts on that?
The Flickr data mission
Exploring the Flickr API and gathering the data was the job for Antti as he had the most suitable
technical skills to maybe get something out of there. But the job was not near to an easy one.
There were many visiting speakers in the class and one of them mentioned Flickr as a source for
open data and gave an example of that. During the lesson we looked a bit into Flickr and decided
to explore it as a source for our data. We knew that we could also ask help from the visitor
speakers if we were to face some technical problems with Flickr. That was good to know, even
though I yet had no idea, what would our quest for Flickr data be like.
Soon after beginning to do research on Flickr and the APIs I realised that I didn’t really know much
about them in practise. My previous education in computer science lacks a bit in practical
understanding and skills even though I could do many things in theory. With Flickr I soon
understood the principles of API, but had hard time learning to use it in the Flickr case and getting
information out of there in practise was also problematic. At this point I went straight to Flickr and
explored websites that seemed to be relevant such as 1 | 2 | 3 | 4
During the next class, there was more discussion about the APIs and searching of data and I we
got some basic tips from Francesca about data collection. I would need an XML or JSON file and
an excel to support with analysis of the data later. We were encouraged to explore and try to solve
situations on our own before asking for help with technical problems. I had the feeling that I should
know the APIs and the technology behind a little better to help dealing with Flickr.
Knowing the Flickr API
I started the day with gathering knowledge on using APIs by doing a few short codecademy
courses and exploring websites and articles about the use of APIs and different data formats.
1 | 2 | 3
Now I felt more confident to dive more into the Flickr API and see what I could get out of it.
The exploration of the usage of Flickr API in a brief recreation was something like this:
• Documentation of the API https://www.flickr.com/services/api/
The search command seems to be interesting as I need to search for data about the pictures
• Flickr Photos search https://www.flickr.com/services/api/explore/flickr.photos.search
Well this is very wide and quite complicated actually, but yes I need photos info from the
Shanghai area so maybe I try to search with specific coordinates.
This lead to exploring many different ways to search for photos according to location and most
of them gave no good result.
• Test searching to see if can get photos that contain lat/long values
https://api.flickr.com/services/rest/?
method=flickr.photos.search&api_key=7622fe7cb0a248dc2339b9433de1e05e&tags=shanghai
%2C+street%2C+food&tag_mode=all&min_upload_date=2017&has_geo=&format=rest&auth_t
oken=72157682863353235-4f31e97e7b86d250&api_sig=b8a9948f020a50a03a734520475fbc
2e
• Test to see what kind of information is available of the photos
• https://api.flickr.com/services/rest/?
method=flickr.photos.getInfo&api_key=7622fe7cb0a248dc2339b9433de1e05e&photo_id=3401
4004772&format=rest&auth_token=72157682863353235-4f31e97e7b86d250&api_sig=892ea1
63356c52514cbfc4c3da53a1ae
• some search on web for how to find Flickr photos of certain location
• http://stackoverflow.com/questions/6474427/using-flickr-api-to-search-photos-by-location-orlanguage
• https://api.flickr.com/services/rest/?
method=flickr.places.find&api_key=7622fe7cb0a248dc2339b9433de1e05e&query=Shanghai&f
ormat=rest&auth_token=72157682863353235-4f31e97e7b86d250&api_sig=db4db1e1f9b8cf76
ac98c7b3135cd1d2
• Discovered the Flickr WOE (where on earth) id for Shanghai:
woeid="2151849" latitude="31.247" longitude="121.472"
or
woeid="12578012" latitude="31.113" longitude="121.416"
Finding photos from SH area
If we use the WOE id, the radius of search doesn’t have effect. What area does this WOE id of
Shanghai cover?
• The search results in 202 photos if the tag “streetfood” is used
https://api.flickr.com/services/rest/?
method=flickr.photos.search&api_key=7622fe7cb0a248dc2339b9433de1e05e&tags=streetfoo
d&tag_mode=all&min_upload_date=2000&woe_id=12578012&per_page=500&format=rest&au
th_token=72157682863353235-4f31e97e7b86d250&api_sig=c9eca65867227cdded449df4c44
ea032
• For “food” there are 9210 photos so it seems to be about right
https://api.flickr.com/services/rest/?
method=flickr.photos.search&api_key=7622fe7cb0a248dc2339b9433de1e05e&tags=food&min
_upload_date=2000&woe_id=12578012&radius=30&per_page=500&format=rest&auth_token=
72157682863353235-4f31e97e7b86d250&api_sig=0e817a39ff701cab8d4ade71db0ebccb
• Finding a good WOEid to search for photos of the regional size Shanghai area
https://api.flickr.com/services/rest/?
method=flickr.places.findByLatLon&api_key=7622fe7cb0a248dc2339b9433de1e05e&lat=31&l
on=121&accuracy=5&format=rest&auth_token=72157682863353235-4f31e97e7b86d250&api_
sig=07f3c97eb9fe29050160d689eec20962
• WOEid: woeid="15017039"
After doing research on the WOE id’s the re was a discussion with Francesca about the amount of
data we would need and how would it be good to search the data of photos. The process went on
and search for data was done maybe a week later.
Cleaning and analysing data
After 2 weeks of data hunting we gathered a great amount of data and formed our datasets.
But it also was very messy and needed to be properly refined.
Full details of the analysis in the datastory in pdf pp. 5-21
Questionnaire
Social Media
Flickr / This is the final search command for the XML file
Characteristic of the Flickr data set
The dataset contains information of 1555 photos, from beginning of year 2004 to the April 24th
2017, that have the tags “Shanghai” and “street food” linked to them. Each photo information
consists of 40 different values. The final dataset of Flickr seems to be quite large enough for analysis. It though has some
weaknesses that will decrease our possibilities of being plausible with some analysis methods. The data collection phase was not complicated but had a lot of “side quests” that lead to some
direction that was not relevant for getting the final dataset. In the end the things needed to be
able to get meaningful data from Flickr were: Basic understanding of APIs in general, basic
understanding of data types such as XML and JSON, good understanding of the Flickr API and
what can be done with which command. In the end the photos.search command gave all the
information that we required even though many other commands were explored along the way.
How we analyse the data
Weibo
Exploring visualisation methods with RAW
Metro Map
On other, visualisation we produced, was the deformed Metro Map of Shanghai. This visualisation
is less informative then the previous once. It is more understandable as an slightly artistic peace,
produced in order to playfully draw attention to the topic and therefore getting people to engage.
We collected the location data from all our sources. The questionnaires provided concrete metro
station information (one question asked for it). Some of the Weibo posts contained address
informations. Flickr provided coordinate data. We manually assigned via google maps to the
nearest metro station. 386 data points mentioned 80 different metro stations and at least one
station of each line.
How it was generated: To create this visualisation we used and combined two different programs. The first step was to
redraw the metro map. For this we simply imported a picture of the map in Rhino3d and set a point
for every station. Then we used the Rhino3d plugin Grasshopper3d to assign names and lines to
the points. We connected every station of every line with an adjustable circle component to be able
to offset circles according to the amount of times a station was mentioned in our sources. To
recreate the appearance of a metro map we connect the outcome of these circles with tangent
lines to get one closed shape for every metro line. After baking and cleaning the lines and circles in
Rhino, we imported everything to AdobeIllustrator to finalise the graphic. We preserved the original
colours of the metro lines, but tried out different shapes and transparencies.
After reviewing the graphics, figure 11 became the finalist due to the following. First is to mention
that because of overlapping and transfer stations with more than just one line, transparency was a
key method to lack a necessity of sorting out priority. In Figure 15 and 16 the priority was
generated simply by counting down the numbers of the lines. Line 1 in the very front and line 16 in
the very background. But this did not seem very reasonably, so we determined them. Figure 13
shows the graphic with a line style. Here it was nice to see single strokes interfering and merging
to the circles. But overall this graphic appears very light and fragile so it nether really fit to the topic
of
food
nor metro. Figure 14 show a version where we deleted all strokes and only left the circles. They were
given a transparency so overlapping generated a new colour. We liked that this version was the
most abstract but on the other hand it felt a bit more static and empty than the final version shown
in figure 11 and 12.
Conclusions
Gathering data for a topic which is totally of the books was an exciting journey. In the end we were very satisfied with the amount of data we collected. Of course the data contains a lot of bias due to the limitations of our research. But nevertheless it was informative and interesting to see certain peoples mindset about Shanghai Street Food. Along with the development and improvement of Shanghai City, sadly many street food vendors have been banned. With our research we detected that meanwhile more and more people began to miss it, complain about the disappearance and talk about it as one of the things which make the city lovable. With these results and websites like http://www.sh-streetfood.org and books like Shanghai Strassenküchen - Menschen, ihre Geschichten und Rezepte we hope that this declining trend of street food will soon make a turn and increase again.
Please check below our final visualizations in poster format
Project by Wu Shan | Jia Ye | Qin Zhen | Antti Kaivola | Zhao Dan | Lara N. Bohe
back to top