Upgrade to Pro — share decks privately, control downloads, hide ads and more …

It's 10pm: Do You Know Where Your Writes Are?

It's 10pm: Do You Know Where Your Writes Are?

Presented October 12, 2017 at MongoDB.local San Francisco.

Jeremy Mikola

October 12, 2017
Tweet

More Decks by Jeremy Mikola

Other Decks in Programming

Transcript

  1. Errors and retry strategies Transient network error Persistent outage Command

    error Never retry May undercount Always retry Retry once
  2. Errors and retry strategies Transient network error Persistent outage Command

    error Never retry May undercount OK Always retry Retry once
  3. Errors and retry strategies Transient network error Persistent outage Command

    error Never retry May undercount OK OK Always retry Retry once
  4. Errors and retry strategies Transient network error Persistent outage Command

    error Never retry May undercount OK OK Always retry May overcount Retry once
  5. Errors and retry strategies Transient network error Persistent outage Command

    error Never retry May undercount OK OK Always retry May overcount Wastes time Retry once
  6. Errors and retry strategies Transient network error Persistent outage Command

    error Never retry May undercount OK OK Always retry May overcount Wastes time Wastes time Retry once
  7. Errors and retry strategies Transient network error Persistent outage Command

    error Never retry May undercount OK OK Always retry May overcount Wastes time Wastes time Retry once May overcount
  8. Errors and retry strategies Transient network error Persistent outage Command

    error Never retry May undercount OK OK Always retry May overcount Wastes time Wastes time Retry once May overcount OK
  9. Errors and retry strategies Transient network error Persistent outage Command

    error Never retry May undercount OK OK Always retry May overcount Wastes time Wastes time Retry once May overcount OK OK
  10. MongoDB 3.6 introduces logical sessions Sessions allow us to maintain

    cluster-wide state about the user and their operations.
  11. MongoDB 3.6 introduces logical sessions Sessions allow us to maintain

    cluster-wide state about the user and their operations. Sessions are not tied to connections.
  12. We can trust the server to Do the Right Thing™

    If the write already executed, return the result we missed.
  13. We can trust the server to Do the Right Thing™

    If the write already executed, return the result we missed. If the write never executed, do it now and return its result.
  14. Cursors have a timeout A er 10 minutes, the server

    will close a cursor due to inactivity.
  15. Cursors have a timeout A er 10 minutes, the server

    will close a cursor due to inactivity. Issuing a getMore resets the clock.
  16. Disabling cursor timeouts cursor = db.coll.find( { }, { noCursorTimeout:

    true } ); cursor.forEach(function() { // lengthy processing…
  17. Disabling cursor timeouts cursor = db.coll.find( { }, { noCursorTimeout:

    true } ); cursor.forEach(function() { // lengthy processing…
  18. A zombie cursor is born > db.serverStatus() { "metrics": {

    "cursor": { "open": { "noTimeout": 1, "total": 1
  19. Avoiding zombie cursors with logical sessions Sessions also have a

    timeout. We can associate queries with a session
  20. You’re running an operation that may never complete cursor =

    db.coll.find( { … } // table scans for days );
  21. Step 1: Find the operation ID > db.currentOp() { "inprog"

    : [ { "desc" : "conn2", "threadId" : "140181791471360", "connectionId" : 2, "client" : "127.0.0.1:49456", "appName" : "MongoDB Shell", "active" : true, "opid" : 132921,
  22. Cluster-wide killOp with logical sessions Any operation may be associated

    with a session. Terminating a session will end all of its associated operations.
  23. Terminating a session session = client.startSession(); cursor = db.coll.find( {

    … }, // table scans for days { session: session } );
  24. Resilence is primarily the driver’s domain Server discovery and monitoring

    Elections and failover recovery Load-balancing mongos connections
  25. Resilence is primarily the driver’s domain Server discovery and monitoring

    Elections and failover recovery Load-balancing mongos connections Routing queries by read preference
  26. Providing a relatively easy upgrade path No need to rewrite

    applications Opting in to retryable writes
  27. Providing a relatively easy upgrade path No need to rewrite

    applications Opting in to retryable writes New API for client session objects
  28. Providing a relatively easy upgrade path No need to rewrite

    applications Opting in to retryable writes New API for client session objects Pass session option as needed