12 Factor PHP Applications - The Remix

In many cases, developers will write code and assume it will perform exactly the same or better in production. However, as we've all seen, applications that build correctly locally do not necessarily perform well once they are being used by the world at large. In a previous incarnation of this talk, there was a brief overview of what 12 factor applications are and how to best apply those principles to CakePHP Applications. This talk will cover practical changes that can be made to a PHP application - in this case using the CakePHP framework, but easily applicable to other frameworks - in order to better support distributed, cloud-based environments, as well as various tactics developers can use to monitor and alert on application health.

Transcript

1. 12 Factor PHP Applications Jose Diaz-Gonzalez Hello everyone! My name is Jose Diaz-Gonzalez, and contrary to what has been announced on twitter, I will not be speaking about the 12 steps to programming. Instead, I’ll be covering 12 Factor PHP Applications.
2. about.me CakePHP Core Dev Infrastructure Lead @ SeatGeek Accidental Operations Terrible Cat Owner Before I get to the meat and potatoes of my talk, here’s a little bit about me. I’m a core developer on a super popular framework called CakePHP. You may have heard of it at your dayjob, and if you haven’t, I’m hiring! I’m also the infrastructure lead at SeatGeek, an event ticketing company for sports, concerts, and theater events (amongst others). It is a role I sort of fell into over the years since beginning my career writing little wordpress “extensions”. We deploy a dozen different languages over a couple hundred servers. Fairly small installation, but Its a lot of fun. Finally, I’m a terrible cat owner, but I’m pretty sure I setup the automatic feeder correctly, so I guess we’ll find out on Saturday. For anyone wondering, my cat’s name is Camilla.
3. What is this talk about? • Not much code, sorry • Not much CakePHP, sorry • Some dinosaurs • Mostly drawings and anecdotes So what is this talk about? Unfortunately, I won’t be going into too much code, as I don’t think that makes for an engaging talk - I’d probably personally also fall asleep, which maybe isn’t the best for you all. On that note, there likely won’t be too many references to CakePHP. In working on the talk, I realized that because there aren’t too many code samples, there isn’t a lot to show off the framework. If you want to talk about CakePHP, feel free to catch me after the talk. For those following me on twitter, I do have some dinosaur references. I also mostly drew these slides, and will be following the slides with anecdotes that I hope back up what I am talking about. I speak a bit quickly, so hopefully this talk lasts about as long as I have time for, but if it doesn’t, not much I can do about that. Finally, apologies in advance for the lack of transitions. I spent all the animation budget on the design work.
4. What is a Scalable App? apart from being built in laravel So just what is a scalable application? Why should any developer care if their app is scalable, and in what scenarios is that useful? What benefit does a scalable app bring the business, and will this result in a raise on Monday when I start building one?
5. picture this To begin with, lets start with an example! Picture this:
6. We have a single server running your glorious CakePHP app - your server may or may not look like an industrial waste barrel, but get it checked out if it does. Its running, serving requests, generally being awesome because its a CakePHP app. There is a database there, and a cache system, and some other stuff that I over-architected because its incredibly important that people be able to access my site. By the way, this app serves a dog photo sharing site, because I’m a terrible cat owner and somehow am sharing dog photos and not cat photos. I think dog photos are incredibly important internet content.
7. One day you wake up, check your mail, pour a bowl of cereal and you check on your server. Normal, everyday routine. Suddenly, you notice that there is a horde of pterodactyls attacking Fred with requests. Poor, poor Fred. His name is “fred”, btw, because you only have one and decided he needed a name. All our servers have cute names, some folks name them after holy swords, some after hobbits, others after bodies of water. We all have our preferences, and I’m not judging you for being wrong on naming things.
8. I don’t know about you, but I’d be panicking if I saw this happening. I mean, really this would be stressing me out. Stressed out enough that I would go on google images to search for “auxilio” - that means help in spanish - and write a talk about it.
9. Fred is being attacked by a horde of pterodactyls. Fred is just a server, and doesn’t have weapons, or friends, or really much of anything other than an internet connection. Fred is just a server, trying to serve requests. Fred is broken on the inside, and is somehow also on fire, because thats what he does when he’s upset. Your CakePHP app is now more of a Creme Brûlée app.
10. actual pterodactyls attacking your app Lets recap. There are actual pterodactyls attacking your app server. Not sure how that happened. You’ve never seen this before - pterodactyls are extinct, and they don’t have feathers like it said in the national geographic article.
11. how do we resolve this why are there even pterodactyls So what do we do? How do we get fred out of a jam? Why is his name Fred, and why are there even pterodactyls in this story? Your cereal is soggy, none of this makes any sense and you just want to have some breakfast
12. we build scalable apps hopefully far away from pterodactyls Well, we should start by making sure your creme brûlée cakephp app is scalable, so that we can withstand any sort of attack. And hopefully build it far away from pterodactyls, as I’m not really sure how they got into your datacenter since we didn’t build it on Isla Nublar. For those who don’t get the joke, Isla Nublar is apparently the island where Jurassic Park happened.
13. scalable apps • support more requests • ease rapid application development • protect your servers from pterodactyls So what defines a scalable app? What characteristics should we be looking for when we build them? What can my team/ manager come to expect from a scalable app? Well, first we should be able to handle a ton of requests. At least N requests, where N is the max you could ever think your dog photo sharing site could ever get in your wildest dreams. Second, we should be able to deploy this app quickly. What if you need a copy of this app in Dublin, or in Tunisia? I hear both of those places have dogs, and being able to service them quickly would be awesome. We also want to make it easy for many people to work on and with the app. As I start servicing people in Tunisia or Singapore, I’ll be hiring more people in various positions - engineering, support, business development, etc. In a sense, a scalable app makes the process of running the business easy. Finally, we almost certainly want to protect the servers from pterodactyls. Fred and his friends will be happy, and less on fire.
14. generally follow common patterns within an organization The definition of a scalable app changes from organization to organization, but generally they follow common patterns within that organization. These patterns can be reused across multiple applications, allowing multiple teams to build scalable applications. Why do we care about this? If we have a pattern, that means we can write down directions others can follow. This can then become a standard, and new things being built will all be fairly similar. Folks can jump from one app to another and be productive fairly quickly, because aside from the code itself, the way in which you work with the app is standard. A scalable application is therefore the culmination of organizational processes that allow teams of developers to rapidly develop applications. For what its worth, a scalable app also probably does dog photo sharing really well. At least it could, if your scalable app needed to do that.
15. How do we build scalable applications apart from building them in laravel So just how do we build scalable applications? What are the patterns we should be following? How do I get from Point A of having a waste barrel server on fire to Point B with no fire and a building over my server’s head?
16. I build them in CakePHP! I’m mostly kidding
17. Follow a pattern 12 factor apps One way is to follow an established pattern. The saying goes “No one gets fired for buying Microsoft”. Thats not completely true, but along those lines, following a predefined standard can give us a jump start on app development. And thats what this talk is about, a pattern called “The Twelve Factor App”.
18. 12 factor background • Coined by Heroku in 2011 • Describes practices for running apps on Heroku • Generally good advice regardless of your stack • 12factor.net I’d like to provide a bit of context about 12 factor apps. 12 Factor is a term coined by Heroku in 2011, and generally describes practices for running apps on Heroku. Despite the awesomeness that is Heroku, Heroku saw that many of their users were having trouble writing applications that worked properly on Heroku. This led to increased support requests, and more expenses in various ways. In an attempt to combat this, they sat down to codify the practices they were using internally for building Heroku apps. In essence, a manifesto for all of their best practices. It also just so happens that many of these practices are generally good, regardless of your stack, your environment, whether you deploy to the cloud or not, etc. I find them to be largely applicable 8 years later, and probably will in another 8. If you want to read the manifesto in depth - and I recommend you do, as I will almost certainly paraphrase some of the factors incorrectly - please see the 12factor.net website.
19. One note. 12 Factor, like any other set of guidelines, is not gospel. Just because I’m giving this talk, doesn’t mean anything I’m saying is necessarily a good idea. Please consult with your team, folks outside your organization, that one guy who makes your sandwiches at the deli, the twitters, etc. before implementing any of this stuff. If I’ve learned anything in attending conferences, its that going home and implementing everything at once is a bad idea. With that said, let me get right into the factors.
20. Factor 1: Codebase One codebase tracked in revision control, many deploys The first factor is “Codebase”. Seems straightforward. We have one codebase, tracked in revision control, that results in many deploys.
21. Factor 1: Codebase First off, 12 factor requires that you use version control. Use anything, whether that be subversion, mercurial, git, perforce, whatever. Just don’t use copies of a file as v1/v2/v3. I cannot stress enough how important this is. The simplest use case here is to be able to introspect on the state of your codebase at a given point in time. That is very difficult if you have PostsController.php.bak-2 lying around your folders. Use version control. Please come find me if you’d like resources for your version control system of choice.
22. Factor 1: Codebase Here is a simple example. I’m skipping ahead a bit, but I’ll create a CakePHP project via composer - you’re all using composer right? - called lolipop. I used lolipop as the name of my dog photo sharing site because dogs like lolipops. I’m also committing my initial work, before starting anything. When I make additional changes, I’ll commit those too. For anyone who doesn’t know what composer, git or a commit are, come find me after the talk :)
23. Factor 1: Codebase Next, we said that we have a single codebase for the app. This means that mono-repos are a violation of the 12factor manifesto. Yes, you can successfully use monorepos at your org - we did that at SeatGeek for 3+ years. It’s incredibly frustrating from a deployment standpoint if you don’t have the tooling in place, and at a certain size, you need a tooling team to handle this. So unless you’re DigitalOcean, Google, Facebook, etc., stick to one app per app. Semi-related, you can share libraries between apps. Those are dependencies, and we’ll get into that in a moment. Note that multiple codebases can be composed into a single product release, which is a separate concept unrelated to 12 factor apps.
24. Factor 1: Codebase A “deploy” is a running “instance” of an app, could even be “deployed” onto your laptop. Deploys are distinguished by the codebase version running in a specific “environment”. Once codebase, many deploys.
25. Factor 2: Dependencies Explicitly declare and isolate dependencies Dependencies! I said we’d come back to this topic. You want to declare your dependencies, and make sure you can retrieve them, regardless of where you are.
26. Factor 2: Dependencies I hope many in this room are acquainted with composer, but in essence you can use it to require that certain PHP libraries be made available for your application. You use a file called a composer.json to specify these. You should always version these dependencies, and never let that version float. I personally despise even floating patch versions, as you never know what a patch might break. If you need to know what is out of date, composer outdated is your friend. Use that when updating versions.
27. Factor 2: Dependencies Once you have dependencies setup, the big question is whether or not to “vendor” those dependencies. When I say vendor, I mean whether or not those dependencies should live inside your code repository in a “committed” form, versus always fetched from a remote. There are pros and cons to each method, but what I would say is that you should always have a backup should a downstream provider remove that dependency, or if the provider has an outage. Both have happened in various communities, and while I don’t have the time to get into it, I definitely recommend reading into the topic before making a decision.
28. Factor 2: Dependencies One last thing. Should you vendor os dependencies? 12 factor says yes. You 100% cannot depend on a dependency on a system being what you expect unless you yourself put it there - because you previously built it or copied it to a secure place. You can certainly pin versions of a debian or rpm package, but the upstream repositories may not always have those versions. Ubuntu does this fairly frequently when they release patched versions of a package. Heroku takes the tactic of needing to compile the libraries and having users download the compiled versions. Its pretty neat, but fairly heavyweight. This is also a longer discussion topic, but the easiest thing to do here is to not vendor, but have pinned dependencies so you know when they are no longer available. This will also allow you to test the upgraded versions with your application when new ones come out.
29. Factor 3: Config Store config in the environment The third letter of the alphabet is C, and today it stands for Config. How do you represent configuration for your application? 12-factor says you should use environment variables. I’ll start with a story first though.
30. Factor 3: Config You should never ship secrets with a publicly accessible app. In fact, if the secret is in the codebase in a raw, unencrypted form, you’re going to have a bad time. I had a client once that shipped their AWS keys with their iOS app. I was asked one day to look into why their VPN didn’t allow access to their prod database. Turns out that someone used that AWS account to delete all their servers - including the databases, which were regular servers on AWS - and ran a ton of bitcoin miners. Though we recovered, it was not fun to deal with. Don’t write secrets into the application code!
31. Factor 3: Config In CakePHP, you can represent a secret via environment variables. Here I’ve set the DATABASE_URL in shell, and then I am retrieving that in code. I can do this outside of running the server, meaning I can hot-swap this at will. It’s pretty great for deploying the app, because now I don’t need to have a hardcoded secret there, and can rotate the credentials for every deploy! Note that env is a special helper method in CakePHP. It takes a default in case you want to make the default something usable for local development.
32. Factor 3: Config By the way, please don’t mix different env config. For instance, do not point to a local mysql database and queue messages into your production rabbitmq. You’re almost assuredly going to have a bad time. Config in our case is meant so that different environments can exist, not so that you can mix and match.
33. Factor 4: Backing Services Treat backing services as attached resources Factor 4 is pretty related to Factor 3. Extract your services into boundaries so you can hot-swap them at will.
34. Factor 4: Backing Services This applies to both internally developed services and services your application uses that you have no control over. Maybe you’re running a local version of the api, or a mock version of the braintree payments api. Think of this as a higher-level form of dependency injection.
35. Factor 5: Build, release, run Strictly separate build and run stages Separate your deployment process! Outside of being my favorite slide to draw, this is a pretty short but important topic.
36. Factor 5: Build, release, run You want to segregate each section such that each can run independently without causing issues with the other. Build a deployment artifact once. Store it somewhere. Have a release process that pushes it out everywhere with the correct config for that environent. Finally, run a command within a specific release of your app. Separating these steps cleanly makes the concept of rollbacks possible, but also makes it fairly easy to stop the deploy process at a given step without affecting the rest of the system.
37. Factor 6: Processes Execute the app as one or more stateless processes When you have a larger app, you’re bound to run more than one thing. A background worker, webserver, cron tasks, etc. Run these processes in as stateless a method as possible.
38. Factor 6: Processes Build stateless services. They are easier to scale. Why? There is no need to reason about which instance of a thing you are speaking about. If you don’t have state, you can kill a specific instance of your app and bring up a new one without worrying if you lost some files.
39. Factor 6: Processes Scaling horizontally is cheaper as you can just add one as opposed to replacing the whole thing with a larger instance. This is not always applicable for memory or cpu intensive processes or singletons.
40. Factor 6: Processes Define your processes in a simple to consume manner. Heroku uses a Procfile to define each one in a separate stanza. Even if you have some messy format to contend with, having a simple way to list each command is extremely valuable.
41. Factor 6: Processes One thing to think about is that almost all apps - including our dog photo sharing app - has to deal with logged in users. Lots of folks out there may depend on the filesystem for sessions. This isn’t the greatest idea - though we can debate if storing the entire session in a cookie is any better.
42. Factor 6: Processes Store sessions in a distributed cache like redis or memcached. Your services and cache can go in and out, so design interaction here in a way that causes minimal issues should you lose cache data.
43. Factor 7: Port binding Export services via port binding Port binding! Very important topic here.
44. Factor 7: Port Binding Essentially, I don’t really care about this topic in the PHP world. Its a bit more useful in other languages, but just not useful in PHP. PHP servers typically run under nginx or apache, and those services are exposed via ports 90% of the time.
45. Not Gospel Sometimes just not applicable Sometimes things in a manifesto aren’t applicable. I feel as though this is one of those times.
46. Factor 8: Concurrency Scale out via the process model Concurrency! Once you have your processes running, how do you handle more requests? In a previous slide, I briefly mentioned scaling horizontally vs vertically. Regardless of which method you do, I think making it easy to scale horizontally is very important.
47. Factor 8: Concurrency In the base case, you’ll likely run a single web process for serving up dog photos.
48. Factor 8: Concurrency Under high load, maybe you manually scale the web process up to 3. Now you can handle more pterodactyls. Making this process trivial is important, as this means we’re exercising our ability to standup/teardown the application.
49. Factor 9: Disposability Maximize robustness with fast startup and graceful shutdown Which leads me into the next factor, disposability. Make it easy to discard an instance of your app processes, and handle that discard easily.
50. Factor 9: Disposability Have a defined process for starting a web or worker process, whether that be a generated init file, some command you run in your framework, or whatever. One thing that I won’t really get into is dealing with long running requests or message processing. Be careful about negligently killing such things - in a distributed cloud environment, this may be close to impossible to always get right, but my basic rule is to do as little work as possible so that the request and/or message has the highest likelihood of succeeding without being interrupted.
51. Factor 10: Dev/prod parity Keep development, staging, and production as similar as possible Next up is keeping parity across environments. This is pretty important, as it has implications for the types of bugs you’ll encounter.
52. Factor 10: Dev/prod parity If you run rabbit in production for message queuing, also run it in staging and dev for similar purposes. You’ve made a conscious decision to depend on this for mission critical processing, and you should not deviate from this.
53. Factor 10: Dev/prod parity Don’t use another system because its easier, only use another system for mocking out the service. While you can replace the backing service for a thing, that should only be done when testing out migrations to another service. One place you’ll see this often is a test suite that runs against sqlite, but the application is deployed against a postgres database. This is almost assuredly going to result in bugs in production, and is easily avoided.
54. Factor 10: Dev/prod parity Similarly, if you deploy to a specific operating system in production, try and make this how you deploy in staging, and what developers test on during development. Now, I’m not saying that everyone should be running linux on the desktop - its not yet that year. But the environment within which your app runs locally should model closely how you run it elsewhere.
55. Factor 10: Dev/prod parity Tools like docker and vagrant make this a bit easier - though with a ton of caveats depending on what the host operating system is and how you run it - but they are there. If you’ve ever heard “but it works on my machine”, you’ll understand why I am passionate about this. I can’t run your laptop in AWS, I can’t ask google to rack your Mac Trashcan.
56. Factor 11: Logs Treat logs as event streams Logs are a simple topic. Treat them as an event stream. If you send a message or write a metric, also write a log message down.
57. Factor 11: Logs I like to write logs in a standard format. The default formatter in many frameworks - cakephp included - is some arbitrary mess that is difficult to parse. Instead, I’ll lean on something like Monolog to write logs in json. In the above example, I’m configuring the default logger in cakephp. I’m setting it to log to a stream, in json format. Whenever I later retrieve a logger object, it’ll retrieve this one by default, and my logs will be written in json. As an aside, I only really use two log levels: Info and Error. If the log message is super verbose, I try to see how I can attach that data as a tag to another log message. So its either something I want to see - info - or something I hate seeing - error.
58. Factor 11: Logs Always include important metadata! Some things you should always include if available are: - the timestamp: use ISO8601, don’t make up a format. If you use a different format, I’ll be upset, but mostly parsing that format won’t be as fun later on. - any user info: be sure to keep PII such as credit card, email or addresses out of it. A user id is great, as that can be cross- referenced with your database, and customer support folks can look it up. - file/line where the log message occurred: If you have the same message several times in your app, having this information will help you figure out where in your app you’re getting the message from. - any tags that can be used to hunt down the specific log message: Things such as “purchase” or “refund” could be tags. - A message: I prefer tags for this, as tags are a bit more structured than a message, but sometimes the message is useful, such as in an error state.
59. Factor 11: Logs Or just use a standard format, such as gelf. Gelf is a format that is processable by many log processors such as graylog or filbeat. It is newline delimited json, with certain predefined fields, and everything else freeform. I wholeheartedly believe in depending on existing formats and patterns.
60. Factor 11: Logs As an aside, I can’t read json logs locally. Maybe you can, and I commend you for that. I recommend either leaving the log handlers as is when developing locally, or using a format called logfmt, which provides structured logs in a human-readable way. Logformat is a key=value format, which is easy to read and scan, but also somewhat machine parseable.
61. Factor 12: Admin processes Run admin/management tasks as on-off processes Home stretch, admin processes! Make it easy to run admin tasks as one off processes, and not ad-hoc sql queries or the like. Story time!
62. Factor 12: Admin processes One time I was tasked with deleting duplicate events in our database. I went to phpmyadmin, checked a few events, and then clicked the truncate button by accident. Wild panic ensued for the next 10 minutes. Ask me the longer version over drinks. I still have a job!
63. Factor 12: Admin processes Instead, I advocate making a command for this. What if instead I can just run make scale and run my truncation command? Now I’ve deliberately written code that another human has reviewed and has said “yeah, that truncation was a good idea, ship it”. That can give an extra bit of confidence over the random SQL statement.
64. Factor 12: Admin processes An even better way to do this would be to have an admin panel somewhere that a user can go into and click the button. This is super useful because now it doesn’t have to be a developer that runs the commands, but instead can be a member of support on a call with a customer, or a business analyst that is dealing with a client. Instead of taking valuable development time, we’ve made it trivial to scale this process across the organization.
65. Questions? And thats all I got. For anyone still here, my twitter handle is savant. I hope I’ve taught at least one person something, but if not, thats okay too. If we still have time: Does anyone have any questions I can answer? Thanks everyone!