Several Tips on Core Data Concurrency Programming

Swift 5.5 has finally provided the long-awaited async/await function, bringing unprecedented convenience to multi-threaded development. However, due to its unique concurrency rules, using Core Data carelessly can easily lead to uncontrollable code, causing many developers to shy away from multi-threaded development in Core Data. This article will provide some tips on several common issues in Core Data concurrency programming, so that developers can better understand the concurrency rules of Core Data and fully enjoy the powerful features provided by Core Data.

Enable Core Data Concurrency Debugging Parameters

Developers using concurrent programming in Core Data often encounter the following situation: the program runs without issues during debugging. However, after the program is released and used by more users, unexpected program exceptions or crashes may occur, which are difficult to reproduce and locate. Many of these issues are caused by incorrect use of concurrent programming in Core Data.

To minimize problems caused by violating Core Data’s concurrency rules during development, it is necessary to add -com.apple.CoreData.ConcurrencyDebug 1 to the startup parameters when using the Core Data framework. This flag will force the program to immediately throw an error when executing Core Data code that may cause concurrent exceptions in theory. This allows for timely discovery and early resolution of any issues.

Part of the code snippets in the following text will only throw errors when this flag is enabled, otherwise there is a probability of over 90% that there will be no abnormal behavior (keeping the hidden danger).

Reduce main thread blocking with private context

No matter how fast hardware develops, operating systems, API frameworks, and various services will always find ways to utilize and exhaust their capabilities. Especially with the continuous increase in device display refresh rates, the pressure on the main thread (UI thread) is also increasing. By creating a background managed object context (private queue context), Core Data’s occupation of the main thread can be reduced.

In Core Data, we can create two types of managed object contexts (NSManagedObjectContext) - main queue context and private queue context.

• Main Queue Context (NSManagedObjectContextConcurrencyType.mainQueueConcurrencyType)

Defined and can only be used for managed object contexts on the main queue. It is mainly used to obtain data required for UI display from persistent storage related to the same UI. When using NSPersistentContainer to create the Core Data Stack, the viewContext property of the container corresponds to the main queue context.

• Private Queue Context (NSManagedObjectContextConcurrencyType.privateQueueConcurrencyType)

As the name suggests, a private queue context creates its own queue when it is created and can only be used on the queue it creates. It is mainly used for operations that take a long time and may affect UI responsiveness if run on the main queue.

There are two ways to create a private queue:

let backgroundContext = persistentContainer.newBackgroundContext() // Method 1

persistentContainer.performBackgroundTask{ bgContext in  // Method 2
    ....
}

If the lifecycle of the operation is long and the frequency is high, Method 1 is usually used to create a dedicated private queue for the transaction (such as Persistent History Tracking).

If the frequency of this operation is low, you can use method two to temporarily create a private queue that can be used and discarded as needed (such as file importing).

Using different queue contexts for data operations is the most common concurrency application scenario for Core Data.

Managed object context and managed objects are bound to queues

Core Data is designed for multithreaded development. However, not all objects under the Core Data framework are thread-safe. The managed object context (NSManagedObjectContext) and managed object (NSManagedObject), which developers most frequently encounter and use, are not thread-safe.

Therefore, when programming with concurrency in Core Data, please make sure to follow the following rules:

• The managed object context is bound to the thread (queue) it is associated with when initialized.
• The managed objects retrieved from the managed object context are bound to the queue where the context belongs.

In plain terms, the context is only safe to execute on the queue it is bound to, and so are the managed objects.

Use Xcode to create a Core Data template, add code in ContextView.swift, and turn on the Core Data concurrency debugging flag.

The following code will immediately throw an error when executed:

Button("context in wrong queue") {
    Task.detached { // Push it to another thread (not the main thread)
        print(Thread.isMainThread) // false not on the main thread
        let context = PersistenceController.shared.container.viewContext
        context.reset() //  When calling the main queue context method on a non-main queue, most operations will result in an error
    }
}

When calling the viewContext method on a non-main thread, the program will immediately crash.

Button("NSManagedObject in wrong context"){
    // The view runs on the main thread
    let backgroundContext = PersistenceController.shared.container.newBackgroundContext() // Create a private queue
    // An operation that should have been performed on a private thread is performed on the main thread
    let item = Item(context: backgroundContext) // Create item in private context, and the item is bound to the private queue
    item.timestamp = .now // Assign on the main queue
}

If the Core Data concurrency debugging flag is not turned on, the above code will run normally in most cases, which is why such errors are difficult to detect.

Use perform to ensure the correct queue

To eliminate errors in the code above, we must place the operations on the managed object context and managed objects in the correct queues.

For the main queue context, since its queue is explicit and fixed - the main thread queue, it is sufficient to ensure that the operations are performed on the main queue. For example:

Button("context in wrong queue") {
        print(Thread.isMainThread) // true the view queue is the main queue
        let context = PersistenceController.shared.container.viewContext
        context.reset() // operating on the main thread context on the main thread is not a problem
}

Or by using DispatchQueue.main.async or MainActor.run to ensure that the operations are performed on the main thread.

However, for private contexts, since the queue is private and exists only within the NSManagedObjectContext instance, it can only be called through the perform or performAndWait methods. The difference between perform and performAndWait is the way in which the specified block of code is executed, asynchronously or synchronously.

Starting from iOS 15 (macOS Monterey), Core Data provides async/await versions of the above methods. The two are combined into one, and the task type is set through the schedule parameter. immediate schedules the task immediately, and enqueued schedules the task to be queued.

perform<T>(schedule: NSManagedObjectContext.ScheduledTaskType = .immediate, _ block: @escaping () throws -> T) async rethrows -> T

Put the code that caused the crash into the perform block to resolve the error.

Button("context in wrong queue") {
    // main queue
    Task.detached { // pushed to another queue (not main queue)
        print(Thread.isMainThread) // false
        let context = PersistenceController.shared.container.viewContext
        await context.perform { // switch back to context queue (main queue in this example)
            context.reset()
        }
    }
}

Button("NSManagedObject in wrong context"){
    // view is on main thread
    let backgroundContext = PersistenceController.shared.container.newBackgroundContext() // created a private queue
    backgroundContext.perform {  // executed on the private queue where backgroundContext resides
        let item = Item(context: backgroundContext)
        item.timestamp = .now
    }
}

Unless the developer can guarantee that the code is running on the main queue and calling the main queue context or managed objects that belong to that context, the safest way to prevent errors is to use perform.

Get context through NSManagedObject

In some cases, only the managed object (NSManagedObject) can be obtained, and it is ensured that it is operated on the correct queue by obtaining the managed object context from it.

For example:

// Item is NSManagedObject
func delItem(item:Item) {
    guard let context = item.managedObjectContext else {return}
    context.perform {
        context.delete(item)
        try! context.save()
    }
}

The managed object context corresponding to the managed object is declared as unowned(unsafe), please use this method only when confirming that the context still exists.

Using NSManagedObjectID for Transfer

Because managed objects are bound to the same queue as the context that manages them, NSManageObject cannot be transferred between contexts on different queues.

For scenarios where the same data record needs to be operated on different queues, the solution is to use NSManagedObjectID.

Taking the code for deleting an Item as an example: assuming the managed object was obtained on the main queue (through @FetchRequest or NSFetchedResultsController in the view), clicking the view button triggers the delItem method to perform data deletion on a private queue in order to alleviate the pressure on the main thread.

The adjusted code:

func delItem(id:NSManagedObjectID) {
    let bgContext = PersistenceController.shared.container.newBackgroundContext()
    bgContext.perform {
        let item = bgContext.object(with: id)
        bgContext.delete(item)
        try! bgContext.save()
    }
}

Or still use NSManagedObject as a parameter.

func delItem(item:Item) {
    let id = item.objectID
    let bgContext = PersistenceController.shared.container.newBackgroundContext()
    bgContext.perform {
        let item = bgContext.object(with: id)
        bgContext.delete(item)
        try! bgContext.save()
    }
}

Some careful readers may wonder, isn’t it impossible to call a managed object on another thread? Will there be any problems when obtaining objectID or managedObjectContext from managed objects? In fact, although the managed object context and most properties and methods of the managed object are not thread-safe, there are still some properties that can be safely used on other threads. For example, the objectID, managedObjectContext, hasChanges, isFault, etc. of the managed object. The persistentStoreCoordinator and automaticallyMergesChangesFromParent of the managed object context are also thread-safe.

NSManagedObjectID serves as a compact universal identifier for managed objects and is widely used in the Core Data framework. For example, in batch operations, persistent history tracking, context notifications, and more, NSManagedObjectID is used as a data identifier. However, it is not absolutely immutable. For example, when the managed object is created but not yet persisted, it will first generate a temporary ID, which will be converted back to a persistent ID after persistence; or when the version of the database or some meta information changes, it may also cause it to change (Apple has not disclosed its generation rules).

Unless it is necessary to use it as the unique identifier of the managed object during runtime, it is best to create your own ID property (such as UUID) to implement it.

If there is a need to archive the ID, NSManagedObjectID can be converted to a URI representation. For specific use cases, please refer to Showcasing Core Data in Applications with Spotlight.

In the example above, object(with: id) was used to obtain the managed object. Other context methods for obtaining managed objects through NSManagedObjectID include registeredObject and existingObject. They have different applicable scenarios

Merging Changes Across Different Contexts

Using the delItem code above, when a managed object is deleted in the background context, the managed object still exists in the main thread context. If the data is displayed in the interface at this point, no changes will occur. Only when changes from one context (in this case the background context) are merged into another context (the main context), will the changes be reflected in the interface (@FetchRequest or NSFetchedResultsController).

Prior to iOS 10, merging context changes required the following steps:

• Add an observer to listen for the Core Data sent context saved notification (Notification.Name.NSManagedObjectContextDidSave).
• In the observer, pass the userInfo of the notification and the context to be merged as parameters to mergeChanges.

 NotificationCenter.default.addObserver(forName:Notification.Name.NSManagedObjectContextDidSave, object: nil, queue: nil, using: merge)

func merge(_ notification:Notification) {
    let userInfo = notification.userInfo ?? [:]
    NSManagedObjectContext.mergeChanges(fromRemoteContextSave: userInfo, into: [container.viewContext])
}

You can also use the mergeChanges method of the NSManagedObjectContext instance to merge contexts one by one.

In the iOS 10 version, Core Data added the automaticallyMergesChangesFromParent property to the NSManagedObjectContext.

Setting the automaticallyMergesChangesFromParent property of the context to true automatically merges changes made to other contexts. In Core Data with CloudKit: Syncing Local Database to iCloud Private Database, you can see how to use automaticallyMergesChangesFromParent to reflect changes in network data in the user interface.

Set the Correct Merge Strategy

When using multiple contexts or multiple persistence coordinators, conflicts may occur when saving managed objects in different environments.

The merge in the merge strategy in this section does not refer to the context merge in the previous section. It refers to the merge strategy set when persisting managed objects to resolve save conflicts caused by inconsistent versions of optimistic locking in managed objects.

Although concurrency is not a necessary condition for save conflicts, they are easily encountered in concurrent environments.

For example, to help you understand save conflicts intuitively:

• The main context uses fetch to get managed object A (corresponding to data B in the database).
• Use NSBatchUpdaterequest (without going through the context) to modify data B in the database.
• Modify managed object A in the main context and attempt to save it.
• When saving, the optimistic lock version number of A is inconsistent with the new version number of database B, and a save conflict occurs. At this time, the merge strategy set determines how to resolve the conflict.

Use mergePolicy to set the merge conflict strategy. If this property is not set, Core Data defaults to using NSErrorMergePolicy as the conflict resolution strategy (all conflicts are not handled and errors are reported directly), which will cause data to fail to be saved correctly to the local database.

viewContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy

Core Data has four preset merge policies:

• NSMergeByPropertyStoreTrumpMergePolicy

  Compare properties one by one. If both the persistent data and the in-memory data have been modified and there is a conflict, the persistent data wins.

• NSMergeByPropertyObjectTrumpMergePolicy

  Compare properties one by one. If both the persistent data and the in-memory data have been modified and there is a conflict, the in-memory data wins.

• NSOverwriteMergePolicy

  The in-memory data always wins.

• NSRollbackMergePolicy

  The persistent data always wins.

If the preset merge policies do not meet your needs, you can also create custom merge policies by inheriting from NSMergePolicy.

Using the example above:

• Data B has three properties: name, age, and sex.
• The name and age were modified in the context.
• The age and sex were modified in the NSBatchUpdateRequest.
• The current merge policy is NSMergeByPropertyObjectTrumpMergePolicy.
• The final merge result is that the name and age are modified in the context, and the sex remains modified by the NSBatchUpdateRequest.

Summary

Core Data has a set of rules that developers must follow. If violated, Core Data will teach you a painful lesson. However, once you have mastered these rules, the obstacles that once hindered you will no longer be a problem.