Last week I started a series of posts detailing how I scraped the iTunes store and how you might do the same for your needs.
Today I will add just one last thing and we will call this a wrap.
thedoedoeblog
Musings of a small game development team
blog
Scraping iTunes: Touch of Generalization
At this point our script (if you haven’t been following along) gives us the xhtml for five images which we can include in our webpage. This works nicely but requires that we name and store our images a certain way. What if we want to use different image names?
Scraping iTunes: Returning HTML
Today I want to take a look at how to modify our Perl script so that it returns the html we need instead of a number. The current version of our script can be found here. Before I get ahead of myself, I want to mention that we will continue to ignore the country code for now. Adding an option for grabbing the data from another iTunes store adds quite a bit of code so I will cover that in my last post.
So what we want to do today is add an option for grabbing the data as html instead of a number. It makes sense to leave the number as an option in the event someone still wants to use it for that purpose. At this point, we know we can add two arguments to the URL
Scraping iTunes: Deciding Direction
I finished off my last post by sharing a Perl script which will output the number of stars for an application given the app id. Today I will explain what I hoped to accomplish when I came up with the idea and then the direction in which I ended up moving. Feel free to grab the script and use it for your own purposes.
For the sake of demonstration, let’s look at an example using 3½ stars.
Scraping iTunes: Setup
With the release of Bailout America we wanted to find a way to display the game’s App Store rating on the website without editing it manually, so we decided to scrape the information from iTunes automatically. I figured if I could pull the number of stars as a numeral I could iterate and build out the star images.
What I’ve ended up with is much more and I thought someone else might benefit from it. Over the next several days I will post details about how I ended up with our current version as well as some idea how you can put it to use for your own benefit. Today we will start with the basics - how to scrape the iTunes store.
The first step is to decide what data we need and on what page we can find it. We launch iTunes and browse to our application. We find the link that reads 4 Reviews for all versions and click it which takes us to a page that shows the number of stars near the top right hand side of the page. If I can grab that, we’ll be in business. So, we click the back button, find that link again, control-click on it and copy the iTunes URL.