Upgrade to Pro — share decks privately, control downloads, hide ads and more …

High Performance Core Data

High Performance Core Data

http://highperformancecoredata.com/

Learn how to analyze, debug, and squeeze every last bit of performance out of Core Data. The standard Core Data implementation is very powerful and flexible but lacks performance. In this advanced session we will cover various performance analysis tools, model optimization, and various high performance concurrency models. This is an advanced discussion that assumes you have used Core Data before.

Matthew Morey

November 15, 2013
Tweet

More Decks by Matthew Morey

Other Decks in Technology

Transcript

  1. Managed Object Context Managed Object Managed Object Persistent Store Coordinator

    Persistent Store (SQLite, XML, ...) Model Main Queue
  2. #define TICK NSDate *startTime = [NSDate date] #define TOCK NSLog(@"Elapsed

    Time: %f", -[startTime timeIntervalSinceNow]) ... TICK; [self methodYouWantToMeasure]; TOCK;
  3. $ sqlite3 UFO.sqlite sqlite> select * from sqlite_master; table|ZUFOSIGHTING|ZUFOSIGHTING|3|CREATE TABLE

    ZUFOSIGHTING ( Z_PK...VARCHAR ) table|Z_PRIMARYKEY|Z_PRIMARYKEY|4|CREATE TABLE Z_PRIMARYKEY (Z_ENT...INTEGER) table|Z_METADATA|Z_METADATA|5|CREATE TABLE Z_METADATA (Z_VERSION...BLOB)
  4. sqlite> SELECT t0.ZSHAPE, COUNT( t0.ZSHAPE ) FROM ZUFOSIGHTING t0 GROUP

    BY t0.ZSHAPE; changed|1 changing|1546 chevron|760 cigar|1782 circle|5271 cone|265 ... teardrop|595 triangle|6082 unknown|4490
  5. NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; NSArray *UFOSightingsArray = [self.managedObjectContext executeFetchRequest:fetchRequest

    error:&error]; NSMutableDictionary *uniqueShapesDictionary = [NSMutableDictionary dictionary]; for (UFOSighting *sighting in UFOSightingsArray) { ... // Count unique shape items ... } NSManagedObject
  6. NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; [fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"shape

    == %@", @"sphere"]]; NSUInteger sphereCount = [self.managedObjectContext countForFetchRequest:fetchRequest error:&error]; ... // Repeat for each unique shape ... countForFetchRequest:error:
  7. NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; NSExpressionDescription *expressionDescription = [[NSExpressionDescription alloc]

    init]; [expressionDescription setName:@"count"]; [expressionDescription setExpression: [NSExpression expressionForFunction:@"count:" arguments: @[[NSExpression expressionForKeyPath:@"shape"]]]]; [fetchRequest setPropertiesToFetch:@[@"shape", expressionDescription]]; [fetchRequest setPropertiesToGroupBy:@[@"shape"]]; [fetchRequest setResultType:NSDictionaryResultType]; NSExpression
  8. NSExpression NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; NSExpressionDescription *expressionDescription = [[NSExpressionDescription

    alloc] init]; [expressionDescription setName:@"count"]; [expressionDescription setExpression: [NSExpression expressionForFunction:@"count:" arguments: @[[NSExpression expressionForKeyPath:@"shape"]]]]; [fetchRequest setPropertiesToFetch:@[@"shape", expressionDescription]]; [fetchRequest setPropertiesToGroupBy:@[@"shape"]]; [fetchRequest setResultType:NSDictionaryResultType];
  9. [backgroundContext performBlock:^{ NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; fetchRequest.resultType = NSManagedObjectIDResultType;

    NSArray *managedObjectIDs = [backgroundContext executeFetchRequest:fetchRequest error:nil]; [mainQueueContext performBlock:^{ for (NSManagedObjectID *managedObjectID in managedObjectIDs) { UFOSighting *UFOSighting = [mainQueueContext objectWithID:managedObjectID]; // // Update UI on main queue // } }]; }];
  10. [backgroundContext performBlock:^{ NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; fetchRequest.resultType = NSManagedObjectIDResultType;

    NSArray *managedObjectIDs = [backgroundContext executeFetchRequest:fetchRequest error:nil]; [mainQueueContext performBlock:^{ for (NSManagedObjectID *managedObjectID in managedObjectIDs) { UFOSighting *UFOSighting = [mainQueueContext objectWithID:managedObjectID]; // // Update UI on main queue // } }]; }];
  11. [backgroundContext performBlock:^{ NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; fetchRequest.resultType = NSManagedObjectIDResultType;

    NSArray *managedObjectIDs = [backgroundContext executeFetchRequest:fetchRequest error:nil]; [mainQueueContext performBlock:^{ for (NSManagedObjectID *managedObjectID in managedObjectIDs) { UFOSighting *UFOSighting = [mainQueueContext objectWithID:managedObjectID]; // // Update UI on main queue // } }]; }];
  12. [backgroundContext performBlock:^{ NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; fetchRequest.resultType = NSManagedObjectIDResultType;

    NSArray *managedObjectIDs = [backgroundContext executeFetchRequest:fetchRequest error:nil]; [mainQueueContext performBlock:^{ for (NSManagedObjectID *managedObjectID in managedObjectIDs) { UFOSighting *UFOSighting = [mainQueueContext objectWithID:managedObjectID]; // // Update UI on main queue // } }]; }];
  13. NSFetchRequest *request = [NSFetchRequest fetchRequestWithEntityName:”MyEntity”]; NSAsynchronousFetchRequest *async = [[NSAsynchronousFetchRequest alloc]

    initWithFetchRequest:request completionBlock^(id result) { if (result.finalResult) { ... } }]; [context performBlock: ^() { NSError *error = nil; asyncResult = [moc executeRequest:asyncRequest error:&blockError]; }]; iOS 8 | OS X Yosemite
  14. Fetching •Don’t fetch more than you need •Set a batch

    size •Let SQLite do the calculations •Prefetch required relationships •Fetch in the background
  15. [fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"shape == %@ AND duration >

    %i", @"sphere", 30]]; [fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"duration > %i AND shape == %@", 30, @"sphere"]];
  16. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Primary Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Search Tokens
  17. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Primary Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Search Tokens URIRepresentation
  18. Data Model •Don’t overnormalize •Use external storage for large attributes

    •Large blobs as separate entity •Less data takes less time to fetch
  19. NSBatchUpdateRequest *batchRequest = [NSBatchUpdateRequest batchUpdateRequestWithEntityName: [RSSItem entityName]]; batchRequest.propertiesToUpdate = @{NSStringFromSelector(@selector(read)):

    [NSNumber numberWithBool:YES]}; batchRequest.resultType = NSStatusOnlyResultType; batchRequest.predicate = [NSPredicate predicateWithFormat:@"..."]; NSError *requestError; NSBatchUpdateResult *result = (NSBatchUpdateResult *)[self.managedObjectContext executeRequest:batchRequest error:&requestError]; if (result == nil) { NSLog(@"Error: %@", [requestError localizedDescription]); } else { // Batch update succeeded } iOS 8 | OS X Yosemite
  20. Main Queue vs Private Queue NSManagedObjectContext *privateManagedObjectContext = [[NSManagedObjectContext alloc]

    initWithConcurrencyType:NSPrivateQueueConcurrencyType]; [privateManagedObjectContext setPersistentStoreCoordinator: self.managedObjectContext.persistentStoreCoordinator]; [privateManagedObjectContext performBlockAndWait:^{ [self import]; }];
  21. Typical Import Algorithm 1) JSON to NSDictionary 2) Find existing

    NSManagedObject 3) Optionally, create new NSManagedObject 4) Set attributes of new/updated NSManagedObject 5) Repeat for every JSON item
  22. Typical Import Algorithm 1) JSON to NSDictionary 2) Find existing

    NSManagedObject 3) Optionally, create new NSManagedObject 4) Set attributes of new/updated NSManagedObject 5) Repeat for every JSON item
  23. Efficient Import Algorithm 1) Sort import objects by ID 2)

    Execute a single fetch request for all matching IDs 3) Iterate through both import and existing objects 4) Update or create
  24. 1 2 3 4 ... New Data 1 4 ...

    Existing Data Update
  25. 1 2 3 4 ... New Data 1 4 ...

    Existing Data Update
  26. 1 2 3 4 ... New Data 1 4 ...

    Existing Data Insert
  27. 1 2 3 4 ... New Data 1 4 ...

    Existing Data Insert
  28. 2 1 2 3 4 ... New Data 1 4

    ... Existing Data
  29. 2 1 2 3 4 ... New Data 1 4

    ... Existing Data
  30. 1 2 3 4 ... New Data 1 4 ...

    Existing Data 2 Insert
  31. 1 2 3 4 ... New Data 1 4 ...

    Existing Data 2 Insert
  32. 3 1 2 3 4 ... New Data 1 4

    ... Existing Data 2
  33. 3 1 2 3 4 ... New Data 1 4

    ... Existing Data 2
  34. 1 2 3 4 ... New Data 1 4 ...

    Existing Data 2 3 Update
  35. 1 2 3 4 ... New Data 1 4 ...

    Existing Data 2 3 Update
  36. Efficient Import Algorithm With Batching 1) Sort import objects by

    ID 2) Execute multiple fetch request for matching IDs - Iterate through both import and existing objects - Update or create 3) Save Batch
  37. // Grab sorted persisted managed objects, based on the batch

    NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] initWithEntityName:@"UFOSighting"]; NSPredicate *fetchPredicate = [NSPredicate predicateWithFormat: @"%K IN %@", @"GUID", jsonBatchGUIDArray];
  38. // Grab sorted persisted managed objects, based on the batch

    NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] initWithEntityName:@"UFOSighting"]; NSPredicate *fetchPredicate = [NSPredicate predicateWithFormat: @"%K IN %@", @"GUID", jsonBatchGUIDArray];
  39. Importing Data •Do import work on private queues •Use efficient

    find-or-create algorithm •Work in batches to keep memory low •Use pre-populated SQLite file
  40. Managed Object Context Managed Object Managed Object Persistent Store Coordinator

    Persistent Store (SQLite, XML, ...) Model Main Queue
  41. Managed Object Context Managed Object Managed Object Persistent Store Coordinator

    Persistent Store (SQLite, XML, ...) Model Main Queue
  42. Managed Object Context Managed Object Managed Object Persistent Store Coordinator

    Persistent Store (SQLite, XML, ...) Model Main Queue
  43. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue
  44. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue
  45. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue
  46. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Merge Changes From Notification
  47. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Merge Changes From Notification
  48. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Refetch and Reload
  49. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Managed Object Context Private Queue UI Work
  50. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Managed Object Context Private Queue UI Work Persisting to Disk
  51. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Managed Object Context Private Queue UI Work Persisting to Disk
  52. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Managed Object Context Private Queue UI Work Persisting to Disk
  53. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Managed Object Context Private Queue UI Work Persisting to Disk
  54. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Managed Object Context Private Queue UI Work Persisting to Disk
  55. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue
  56. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates
  57. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work
  58. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk
  59. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk
  60. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk
  61. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk __block NSManagedObjectContext *temporaryContext = self.workerObjectContext; __block NSManagedObjectContext *managedObjectContext = self.managedObjectContext; __block NSManagedObjectContext *writerObjectContext = self.writerManagedObjectContext; [temporaryContext performBlock:^{ NSError *error = nil; if (![temporaryContext save:&error]) { // TODO: Handle error } [managedObjectContext performBlock:^{ NSError *error = nil; if (![managedObjectContext save:&error]) { // TODO: Handle error } [writerObjectContext performBlock:^{ NSError *error = nil; if (![writerObjectContext save:&error]) { // TODO: Handle error } // Success!!! }]; // writerObjectContext }]; // managedObjectContext }]; // temporaryContext
  62. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
  63. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
  64. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
  65. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
  66. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification
  67. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Merge Changes From Notification
  68. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Merge Changes From Notification
  69. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Refetch and Reload
  70. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Refetch and Reload •Cannot pass objects between persistent store coordinators •Cannot pass objectIDs between persistent store coordinators •You can use URIRepresentation •You can use mergeChangesFromContextDidSaveNotification
  71. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Refetch and Reload
  72. Concurrency Models •Use the simplest model that meets your needs

    •Update main context after large imports have completed
  73. In general... •Use the tools, and measure, measure, measure •Don’t

    load more than you need to •Don’t use [cd] •Be smart with your data model •Import on a private queue in batches •Pick the correct (but simplest) concurrency model
  74. Questions? HighPerformanceCoreData.com MatthewMorey.com | @xzolian BuoyExplorer.com - Marine Conditions App

    WristPresenter.com - Presentation App ChaiOne.com - Hiring remote iOS engineers