Fuck It: Let's Have Fun - Building a Top Torrents API

Like building API services? Like pirating things? Let’s do both!

In this talk, Randall Degges, Chief Hacker at Stormpath, will walk you through building and securing an API service that serves up the hottest 100 torrents on the web. You’ll learn how to scrape the world’s most popular torrent website, expose this data as a public API service using Heroku, and secure this API service using Stormpath.

This talk was given at Hack Reactor in San Francisco.

Transcript

1. Fuck It: Let’s Have Fun Building a Top Torrents API @rdegges
2. Hey, I’m Randall Chief Hacker, Stormpath Open Source Dude <333 t3h c0d3z
3. Story Time...
4. Let’s turn this Into an API
5. Arrr! Sounds cool, so what’s the plan?
7. https://github.com/rdegges/toptorrents-api
8. Step 1: Download the HTML
9. https://github.com/request/request
10. "use strict"; const request = require("request"); request("https://thepiratebay.org/top/207", (err, resp, body) => { if (err || resp.statusCode !== 200) { throw new Error("thepiratebay must be down! Arg!"); } console.log(body); });
11. Step 2: Parse the HTML The.Infiltrator.2016.720p.BRRip.x264.AAC-ETRG
12. https://github.com/cheeriojs/cheerio
13. const cheerio = require("cheerio"); let $ = cheerio.load(body); $("#searchResult .detName a").each(function() { let torrentName = $(this).text(); console.log(torrentName); });
14. Step 3: Define the API
15. Route: / { "categories": [ "https://toptorrents-api.herokuapp.com/otherPhysibles", "https://toptorrents-api.herokuapp.com/applicationsWindows", "https://toptorrents-api.herokuapp.com/videoMovieClips", "https://toptorrents-api.herokuapp.com/videoMoviesDVDR", "https://toptorrents-api.herokuapp.com/gamesPsx", "https://toptorrents-api.herokuapp.com/videoTvShows", "https://toptorrents-api.herokuapp.com/audioBooks", "https://toptorrents-api.herokuapp.com/applicationsMac", "https://toptorrents-api.herokuapp.com/audioFlac", "https://toptorrents-api.herokuapp.com/videoHandheld", "https://toptorrents-api.herokuapp.com/audioMusic", "https://toptorrents-api.herokuapp.com/gamesMac", "https://toptorrents-api.herokuapp.com/videoHdMovies" ] }
16. Route: /<category> { "torrents": [ { "name": "The.Infiltrator.2016.720p.BRRip.x264.AAC-ETRG", "size": "952.81 MiB", "url": "magnet:?xt=urn:btih:79fbe3251b40d18eb821be...", "seeds": "3272", "leeches": "473" }, { "name": "Star Trek Beyond 2016 1080p WEB-DL x264 AC3-JYK", "size": "3.36 GiB", "url": "magnet:?xt=urn:btih:65959583d7d7f90b98eb2464f...", "seeds": "2720", "leeches": "803" } ] }
17. Step 4: Set the Settings
18. Step 5: Scrape the Data
19. Step 6: Store the Data!
20. Step 7: Create the Routes
21. Step 8: Deploy It!
24. $ heroku create $ git push heroku master $ heroku open $ heroku addons:create heroku-postgresql Create a new app. Deploy your code to Heroku. Open your project in the browser. Create a Postgres database server.
25. Let’s See It In Action!
26. BONUS! (authentication with ...)
27. You’re awesome. @rdegges @gostormpath