Upgrade to Pro — share decks privately, control downloads, hide ads and more …

API Design Tips

daniellindsley
September 04, 2012

API Design Tips

Given at DjangoCon 2012. A collection of ideas (built through the pain of experience) on how to improve the code you write.

daniellindsley

September 04, 2012
Tweet

More Decks by daniellindsley

Other Decks in Technology

Transcript

  1. Who? •Daniel Lindsley •Toast Driven ( ) is my business

    •Wrote a couple Django apps •Haystack •Tastypie
  2. Why? •Think about how many times you’ve started with someone

    else’s library... •Used it some... •Then wanted to strangle them for the hoops they made you jump through
  3. Why? •Other people use your code all the time. •They

    might be wanting to strangle you! •It might even be future you wanting to strangle past you.
  4. But more people will be happy if they can tweak

    it. You can bet they’ll need to.
  5. And most people will be happy if it’s easy to

    tweak. I don’t have a clever followup here.
  6. They shouldn’t have to constantly refer to the docs. Especially

    not for things they use all the freaking time.
  7. Someone is going to do something weird/insane with your code.

    It’s inevitable, so design for it up front.
  8. Approaches On Design •Bottom-up sucks. •Sure, you built little pieces

    that work. •But do they work well together? •Likely not. •HOW IS BABBY PHP FORMED
  9. Approaches On Design •Top-down feels better. •Everything fits together right.

    •Less duplication. •Resist the urge to duct tape.
  10. Approaches On Design •With (some) instant TDD, you get your

    tests started from the get-go. •Fewer massive, painful refactorings down the road.
  11. Small Components class DoEverything(object): # ... def update_cache(self, timeout=600): #

    Stuff... def post_to_twitter(self, user): # More stuff... def make_me_a_sandwich(self, sudo=False): # Even more stuff...
  12. Small Components class Cachable(object): timeout = 600 def update(self): #

    Cache myself... class SocialPosts(object): def twitter(self, user): # Post a tweet class SandwichMaker(object): def __init__(self, sudo=False): self.sudo = sudo def make(self): # Maybe yes, maybe no.
  13. Composition >= Inheritance class CachedView(View): def get(self, request, **kwargs): #

    Cache the output & serve. class GhettoAPIView(View): def get(self, request, **kwargs): # Return either JSON or HTML. # Now try to use both. :/ class MyView(CachedView, GhettoAPIView): # Oh crap, they stomp on each other. pass
  14. Composition >= Inheritance class MyView(BetterView): def get(self, **kwargs): response =

    self.preprocess() # Do stuff. return self.render('template.html', {}) def preprocess(self): # Delegate! cached = ViewCache() return cached.get(self, self.request) def render(self, template, context): output = GhettoAPI(self.request) return output.render(template, context)
  15. Reflection class FakeDate(BadDate): def to_iso8801(self): return '-'.join( self.year, self.month, self.day

    ) def from_iso8801(self, date_string): date_bits = date_string.split('-') self.year = date_bits[0] self.month = date_bits[1] self.day = date_bits[2]
  16. Broad Familiarity If it’s a similar task to something else

    they’ll know, mimic that something else.
  17. Broad Familiarity def go_search(engine_url, query, limit=10 offset=0, just_values=False): # Hit

    the search engine... return results def crazy_raw_query(engine_url, raw_query): # Hope & pray the query is OK. return raw_results # Usage: kwargs = { 'limit': 20, 'offset': 20, } results = go_search('http://...', 'banana')
  18. Broad Familiarity class SearchQuerySet(object): # Make it look like ``QuerySet``.

    def __getitem__(self, key): # Handle the slice here. def filter(self, **kwargs): # Apply filters. def raw(self, query): # Do a raw query. # Usage: sqs = SearchQuerySet().filter(text='banana') results = sqs[20:20]
  19. Narrow Familiarity def go_search(engine_url, query, limit=10 offset=0, just_values=False): # Do

    stuff. return results def execute_raw_query(raw_query, offset, engine_url): # Do a raw query here. return { 'search_results': results, }
  20. Narrow Familiarity def query(engine_url, query, limit=10 offset=0, just_values=False): # Do

    stuff. return { 'results': results, 'total': total_count, } def raw_query(engine_url, query, limit=10 offset=0, just_values=False): # Do a raw query here. return { 'results': results, 'total': total_count, }
  21. Protocol class CoolKidBackend(object): # Whatever, as if we care. pass

    class RedisBackend(CoolKidBackend): def get(self, key): return self.conn.hget(key) class RiakBackend(CoolKidBackend): def get_hash(self, bucket, key): return self._conn.get(bucket, key) # You can't transparently switch backends. # Welp, time to go rewrite everything.
  22. Protocol class CoolKidBackend(object): def __init__(self, conn_string): # Parse the bucket

    out of # the conn_string, if present. def get(self, key): raise NotImplementedError('...') class RedisBackend(CoolKidBackend): def get(self, key): return self.conn.hget(key) class RiakBackend(CoolKidBackend): def get(self, key): return self._conn.get(...)
  23. Assume The Worst™! def parse_name(name): # It'll be fine. Names

    are always # "<first name> <single last name>" # and ASCII, right? return name.split(' ')
  24. Assume The Worst™! def parse_name(name, separator=' ', middlename_present=False, allowed_suffixes=SUFFIXES): if

    ',' in name: maybe_last_name = name.split(',', 1) if maybe_last_name in allowed_suffixes: # ... return { 'original': name, 'confidence': confidence_level * 100, 'first_name': first, # ... }
  25. Stop At A Low Level “This Low Level API ought

    to be good enough.” Yeah, right. And 640Kb was totally enough for everyone.
  26. Stop At A Low Level def get_page(socket, timeout): # Use

    the open socket to fetch # the web page. It's a open # socket to the correct place, # right?
  27. Stop At A Low Level def get_raw_page(socket, timeout=60): # Do

    the actual fetch. return headers, content def get_page(url, timeout=None, fetcher=get_raw_page): # Validate the URL. # Open a socket. head, content = fetcher(my_socket) if head.status == 500: raise OhNoes('...') # ...
  28. Wildly Different Return Values def query(engine_url, query, limit=10): # Make

    the user deal with the raw data. results = [res for res in results if res] return results, total def raw_query(engine_url, query, limit=10): # Make the user deal with the raw data. return { 'search_results': results, 'count': total, }
  29. Wildly Different Return Values def query(engine_url, query, limit=10): # We're

    doing a search query. # Return sane, similar structures. return { 'results': results, 'total': total_count, } def raw_query(engine_url, query, limit=10): # We're doing a search query. # Return sane, similar structures. return { 'results': results, 'total': total_count, }
  30. Useless “Implementation” Code class BaseBackend(object): # Full implementation. def __init__(self,

    url): raise NotImplementedError('...') def save(self, filename, data): raise NotImplementedError('...') def load(self, filename): raise NotImplementedError('...')
  31. Useless “Implementation” Code class FileBackend(object): # Does useful things. Subclasses

    can # override less. def __init__(self, url): self.url = url def save(self, filename, data): the_file = open(filename, 'w') return json.dump(the_file, data) def load(self, filename): data = open(filename) return json.load(data)
  32. If It’s Difficult To Test... def current_temperature(station): # Oh yeah,

    this totally works BTW. url = 'http://w1.weather.gov/obhistory/' + \ '%s.html' % station resp = requests.get(url) data = pq(resp.content) return float(data('table').eq(3)\ .find('tr').eq(4)\ .find('td').eq(6).text())
  33. If It’s Difficult To Test... class Weather(object): def build_url(self, station):

    return 'http://w1.weather.gov/obhistory/' + \ '%s.html' % station def fetch_page(self, url): return requests.get(url).content def parse_page(self, content): return pq(resp.content) def last_conditions(self, data): return data('table').eq(3).find('tr').eq(4) def current_temperature(self, station): url = self.build_url(station) content = self.fetch_page(url) data = self.parse_page(content) return float(self.last_conditions(data)\ .find('td').eq(6).text())
  34. The ORM Love it or hate it, there’s some great

    learning opportunities there to be had.
  35. Very Little Global State # Only defaults or things that

    absolute # have to be module-level. BASE_URL = 'http://w1.weather.gov/obhistory/' TIMEOUT = 10 # Make them defaults but allow passing # different in their place. def fetch_page(self, url=BASE_URL, timeout=TIMEOUT): return requests.get( url, timeout=timeout ).content
  36. Decrease Reliance On ``self`` class Weather(object): def build_url(self, station): #

    Make a URL. def fetch_page(self, url): # Get a page def parse_page(self, content): # Use the page content. def current_temperature(self, station): url = self.build_url(station) content = self.fetch_page(url) data = self.parse_page(content) # You could be using ``self.<variable>`` on # these, but by allowing data to be passed # in, you make it easier to test.
  37. Resist The Urge To Use Magic Be explicit first, then

    add shortcuts. (which can be a little more magical)
  38. So... •Use the Golden Rule. •Consistency is key. •Plan for

    the worst & include sweet shortcuts for the best.
  39. So... •Use the Golden Rule. •Consistency is key. •Plan for

    the worst & include sweet shortcuts for the best. •Write a thing you’d to use...
  40. So... •Use the Golden Rule. •Consistency is key. •Plan for

    the worst & include sweet shortcuts for the best. •Write a thing you’d to use... •...then make it even better.