WWDC 2023, What’s New in Core Data

Although at WWDC 2023, Apple will mainly focus on introducing the new data framework SwiftData, Core Data, as the cornerstone of SwiftData, has also been enhanced to some extent. This article will introduce the new features that Core Data has gained this year.

Composite attributes

Composite attributes are a new custom attribute provided by Core Data for entities. With it, developers can encapsulate complex data types in a customized way.

For example, let’s say we have a Restaurant entity:

public class Restaurant:NSManagedObject {
    @NSManaged public var address: String?
    @NSManaged public var name: String?
    @NSManaged public var phoneNumber: String?
    @NSManaged public var rating: Double
}

Before the appearance of composite attributes, we have three optional solutions to add the latitude and longitude information for this restaurant:

1. Create separate attributes for longitude and latitude, and use a computed attribute called “location” to improve code readability.
2. Create a Location entity, including longitude and latitude attributes, and create a one-to-one relationship between the Restaurant entity and the Location entity.
3. Create a Location structure and declare it as a Transformable attribute in the Restaurant entity.

There are advantages and disadvantages to each of these three solutions:

• Solution 1: The best performance, and both latitude and longitude attributes can be used as separate predicate conditions. However, when multiple entities have the same requirements, repetitive setup work is required for each entity. The more complex the composite type (e.g. Location), the more repetitive operations are required.
• Solution 2: Both latitude and longitude attributes can be used as separate predicate conditions, but the performance during retrieval is slightly lower than the first solution.
• Solution 3: Latitude and longitude cannot be used as predicate conditions (data has been converted into an unsearchable state), and there will be some performance loss due to encoding and decoding when saving and reading data.

Composite attributes provide developers with a brand new option.

First, we need to customize a Composite Attribute in Xcode’s Data Model Editor.

Then, in a similar way to defining an Entity, add attributes to the custom Composite Attributes.

When defining Composite Attributes, we can use any property provided by Core Data for the Entity, such as String, Double, Date, etc., and we can also use other pre-defined Composite Attributes. Supporting nesting is also a very prominent feature of Composite Attributes.

Finally, we can use the custom Composite Attributes in the Entity just like we use other pre-built properties provided by Core Data.

It should be noted that Custom Composite Attributes are merely an abstract description of the Entity Attribute type, and Core Data will not generate a corresponding type for it in the code. In SQLite, Composite Attributes use the same storage pattern as solution 1 (expanding all the properties of Composite Attributes in the table corresponding to the Entity, and creating independent fields for them).

In the code, Composite Attributes are declared as [String:Any]? type:

public class Restaurant:NSManagedObject {
    @NSManaged public var address: String?
    @NSManaged public var name: String?
    @NSManaged public var phoneNumber: String?
    @NSManaged public var rating: Double

    @NSManaged public var location: [String: Any]?
}

Currently, we still need to use a dictionary to set and read the content of this property in the managed object:

let newRestaurant = Restaurant(context: viewContext)
newRestaurant.address = address
newRestaurant.name = name
newRestaurant.phoneNumber = phoneNumber
newRestaurant.rating = rating
newRestaurant.location = [
    "latitude": 39.90469,
    "longitude": 116.40528,
]

However, when setting predicates, it is possible to directly access the keyPath with a namespace:

let predicate = NSPredicate(format:"location.latitude > %f AND location.latitude < %f",31.3,40.0)

Hint: In the official documentation about Composite Attributes, the following demonstration code appears. We hope that in future updates, it will be possible to access sub-attributes in Composite Attributes directly through this approach.

// Use property-like setters and getters to manage the underlying attributes directly.
quake.magnitude.richter = 4.6
print(quake.magnitude.richter)

Using New Predicate in Core Data

For a long time, Core Data developers have been hoping to create safe and easy-to-understand predicates in a more Swift-friendly way. This wish has finally been fulfilled this year thanks to the new Foundation.

Developers can create predicates for SwiftData in the following ways:

let today = Date()
let tripPredicate = #Predicate<Trip> {
    $0.destination == "New York" &&
    $0.name.contains("birthday") &&
    $0.startDate > today
}

Fortunately, in this update of Predicate, Core Data has not been abandoned. Developers can convert Predicate to NSPredicate through the new NSPredicate construction method.

For example:

let p = #Predicate<Restaurant>{
    $0.rating > 3.5
}

let predicate = NSPredicate(p)

Two things to note:

• Only predicates created for subclasses of NSObject can be converted to NSPredicate, which means that predicates created for SwiftData cannot be used as predicates for corresponding managed objects in Core Data.
• Currently, it is not possible to directly access properties of composite attributes through keyPath in predicates.

VersionChecksum

In this year, NSManagedObjectModel has added a new property called versionChecksum. This property corresponds to the 128-bit model version hash value of the data model, which is Base64 encoded.

This value can also be found in the VersionInfo.plist file and in Xcode build logs.

This value has two purposes:

• It is used in staged migration to create NSManagedObjectModelReference for different versions of the data model. More details are provided below.
• It is used in projects that run SwiftData and Core Data in parallel to compare whether they are using the same version of the data model.

For example, we can obtain the versionChecksum value currently used by SwiftData by using the following code. Then, in CoreDataStack, we can compare it with that value to determine whether the two are using the same data model.

@main
struct PredicateTestApp: App {
    let container = try! ModelContainer(
        for: Item.self
    )
    var body: some Scene {
        WindowGroup {
            ContentView()
                .onAppear {
                    if let versionChecksum = container.schema.makeManagedObjectModel()?.versionChecksum {
                        print(versionChecksum)
                    }
                }
        }
        .modelContainer(container)
    }
}

Deferred Migration

During the Core Data model migration process, if the data set is large and the migration operation is complex, the application may become unresponsive, resulting in a poor user experience.

In this update of Core Data, Apple has added the Deferred Migration feature to alleviate the discomfort caused by the above situation to some extent.

Precautions:

• Delayed migration can only apply to some operations in the lightweight migration process.
• Any operation that may cause data model incompatibility cannot be delayed.
• Delayed migration is only applicable to SQLite storage types.
• Delayed migration is backward compatible and can be used in both iOS 14 and Big Sur.
• Delayed migration also applies to phased migration added this year.

In simpler terms, after enabling the Delayed Migration feature, Core Data will determine which operations from lightweight migration can be skipped during the migration process without affecting the application’s ability to operate on the final completed data model version database (e.g. updating indexes, deleting unnecessary attributes, changing from ordered to unordered relationships, etc.). Core Data will skip these operations until the developer finds an appropriate time in the application to explicitly execute these “cleanup” operations through code.

To enable delayed migration functionality, set NSPersistentStoreDeferredLightweightMigrationOptionKey to true in the storage options.

let options = [
    NSPersistentStoreDeferredLightweightMigrationOptionKey: true,
    NSMigratePersistentStoresAutomaticallyOption: true,
    NSInferMappingModelAutomaticallyOption: true
]

let store = try coordinator.addPersistentStore(
    ofType: NSSQLiteStoreType,
    configurationName: nil,
    at: storeURL,
    options: options
)

After completing the necessary migration operations, developers can perform “clean-up” work at an appropriate time by calling the finishDeferredLightweightMigration method (Apple recommends doing this in BGProcessingTask):

let metadata = coordinator.metadata(for: store)
if (metadata[NSPersistentStoreDeferredLightweightMigrationOptionKey] == true) {
    coordinator.finishDeferredLightweightMigration()
}

Staged migration

In previous versions of Core Data, developers most commonly used the following three data migration methods:

• Lightweight migration

If the changes between two data model versions are simple enough for Core Data to infer a mapping model, developers do not need to provide additional information, and Core Data will automatically perform data migration between the two versions.

• Custom mapping model

If developers make deeper adjustments to the data model, causing Core Data to be unable to automatically infer a mapping model, they can create a mapping model (Mapping Model) for two specific versions using the Xcode Model Editor. By providing additional information in the custom mapping model, developers can help Core Data complete data migration between two versions.

• Custom entity mapping strategy

If the expressions provided by the custom mapping model still do not meet the needs of migration, developers need to create a custom entity mapping strategy (a subclass of NSEntityMigrationPolicy). NSEntityMigrationPolicy provides some methods that can override the default data migration operation.

Although Core Data itself provides a highly controllable progressive migration method, it is rarely used in practical development due to its unfriendliness to developers and the need to write a lot of code.

Since SwiftData does not use Xcode’s Model Editor, Apple needs to provide a migration method that does not rely on Mapping Model files. At the same time, the original method of writing custom entity mapping strategies is not very friendly to developers. Therefore, SwiftData uses a data migration method based on stage migration. As the foundation of SwiftData, Core Data naturally also added this migration mode.

This article will not provide a detailed explanation of phased migration, and we will explore it further in future articles.

Stage migration includes two migration modes: lightweight migration (NSLightweightMigrationStage) and custom migration (NSCustomMigrationStage). It encourages developers to decompose non-lightweight migration tasks into a series of lightweight migration steps. By creating multiple stages, data models can be migrated to the latest version with minimal code.

Generally, stage migration can be divided into the following steps:

Describing Promises of Data Model Versions

The promises of a specific version of NSManagedObjectModel are described by declaring multiple NSManagedObjectModelReference classes for phased migration. During migration, Core Data will adhere to these promises.

let v1ModelChecksum = "kk8XL4OkE7gYLFHTrH6W+EhTw8w14uq1klkVRPiuiAk="
let v1ModelReference = NSManagedObjectModelReference(
    modelName: "modelV1"
    in: NSBundle.mainBundle
    versionChecksum: v1ModelChecksum
)

let v2ModelChecksum = "PA0Gbxs46liWKg7/aZMCBtu9vVIF6MlskbhhjrCd7ms="
let v2ModelReference = NSManagedObjectModelReference(
    modelName: "modelV2"
    in: NSBundle.mainBundle
    versionChecksum: v2ModelChecksum
)

let v3ModelChecksum = "iWKg7bxs46g7liWkk8XL4OkE7gYL/FHTrH6WF23Jhhs="
let v3ModelReference = NSManagedObjectModelReference(
    modelName: "modelV3"
    in: NSBundle.mainBundle
    versionChecksum: v3ModelChecksum
)

The above code creates promises for three different versions of data models. Core Data ensures the correctness of data versions by checking the versionChecksum.

The staged migration feature of Core Data is designed to provide foundational support for SwiftData migrations, thus adopting a somewhat opaque approach when accessing model files. It is recommended that developers, during migration, directly access the mom files of different model versions to construct model instances, ensuring the accurate acquisition of model data.

For instance, let’s say we have created two model versions, named Model and Model 2 respectively:

The corresponding NSManagedObjectModelReference instances can be constructed as follows:

guard let momdURL = Bundle.main.url(forResource: "Model", withExtension: "momd") else { fatalError() }
let model1URL = momdURL.appendingPathComponent("Model.mom")
let model2URL = momdURL.appendingPathComponent("Model 2.mom")
guard let model1 = NSManagedObjectModel(contentsOf: model1URL) else { fatalError() }
guard let model2 = NSManagedObjectModel(contentsOf: model2URL) else { fatalError() }

let v1ModelChecksum = model1.versionIdentifiers
let v1ModelReference = NSManagedObjectModelReference(model: model1, versionIdentifiers: v1ModelChecksum)

let v2ModelChecksum = model2.versionIdentifiers
let v2ModelReference = NSManagedObjectModelReference(model: model2, versionIdentifiers: v2ModelChecksum)

This method allows developers to handle the migration process of data models more flexibly, ensuring data is accurately transferred to the new version.

Since SwiftData does not rely on data model files, the way promises are made in SwiftData is slightly different (by representing each version of the model through code).

enum SampleTripsSchemaV1: VersionedSchema {
    static var models: [any PersistentModel.Type] {
        [Trip.self, BucketListItem.self, LivingAccommodation.self]
    }

    @Model
    final class Trip {
        var name: String
        var destination: String
        var start_date: Date
        var end_date: Date

        var bucketList: [BucketListItem]? = []
        var livingAccommodation: LivingAccommodation?
    }

    // Define the other models in this version...
}

enum SampleTripsSchemaV2: VersionedSchema {
    static var models: [any PersistentModel.Type] {
        [Trip.self, BucketListItem.self, LivingAccommodation.self]
    }

    @Model
    final class Trip {
        @Attribute(.unique) var name: String
        var destination: String
        var start_date: Date
        var end_date: Date

        var bucketList: [BucketListItem]? = []
        var livingAccommodation: LivingAccommodation?
    }

    // Define the other models in this version...
}

enum SampleTripsSchemaV3: VersionedSchema {
    static var models: [any PersistentModel.Type] {
        [Trip.self, BucketListItem.self, LivingAccommodation.self]
    }

    @Model
    final class Trip {
        @Attribute(.unique) var name: String
        var destination: String
        @Attribute(originalName: "start_date") var startDate: Date
        @Attribute(originalName: "end_date") var endDate: Date

        var bucketList: [BucketListItem]? = []
        var livingAccommodation: LivingAccommodation?
    }

    // Define the other models in this version...
}

Describing the Required Migration Stages

In the previous section, we created three versioned models for Core Data staged migration: V1, V2, and V3. As a result, we need to describe two migration stages: V1 → V2 and V2 → V3.

let lightweightStage = NSLightweightMigrationStage([v1ModelChecksum])
lightweightStage.label = "V1 to V2: Add flightData attribute"

let customStage = NSCustomMigrationStage(
    migratingFrom: v2ModelReference,
    to: v3ModelReference
)

customStage.label = "V2 to V3: Denormalize model with FlightData entity"

When migrating from V1 to V2, we believed that lightweight migration would suffice, so no additional code was needed. However, when migrating from V2 to V3, we realized that lightweight migration was not enough, and custom code was required for the migration.

customStage.willMigrateHandler = { migrationManager, currentStage in
    guard let container = migrationManager.container else {
        return
    }

    let context = container.newBackgroundContext()
    try context.performAndWait {
        let fetchRequest = NSFetchRequest<NSFetchRequestResult>(entityName: "Aircraft")
        fetchRequest.predicate = NSPredicate(format: "flightData != nil")

        do {
           var fetchedResults: [NSManagedObject]
           fetchedResults = try viewContext.fetch(fetchRequest)

           for airplane in fetchedResults {
                let fdEntity = NSEntityDescription.insertNewObject(
                    forEntityName: "FlightData",
                    into: context
                )

                let flightData = airplane.value(forKey: "flightData")
                fdEntity.setValue(flightData, forKey: “data”)
                fdEntity.setValue(airplane, forKey: "aircraft")
                airplane.setValue(nil, forKey: "flightData")
            }
            try context.save()
        } catch {
            // Handle any errors that may occur
        }
    }
}

In the above code, the airplane property of the existing data was read (which is of Transformable type) before performing the migration from version V2 to V3. We used this data in airplane to create a new FlightData entity (which has a one-to-one relationship with Aircraft).

SwiftData also has similar corresponding operations:

enum SampleTripsMigrationPlan: SchemaMigrationPlan {
    static var schemas: [any VersionedSchema.Type] {
        [SampleTripsSchemaV1.self, SampleTripsSchemaV2.self, SampleTripsSchemaV3.self]
    }

    static var stages: [MigrationStage] {
        [migrateV1toV2, migrateV2toV3]
    }

    static let migrateV1toV2 = MigrationStage.custom(
        fromVersion: SampleTripsSchemaV1.self,
        toVersion: SampleTripsSchemaV2.self,
        willMigrate: { context in
            let trips = try? context.fetch(FetchDescriptor<SampleTripsSchemaV1.Trip>())

            // De-duplicate Trip instances here...

            try? context.save()
        }, didMigrate: nil
    )

    static let migrateV2toV3 = MigrationStage.lightweight(
        fromVersion: SampleTripsSchemaV2.self,
        toVersion: SampleTripsSchemaV3.self
    )
}

Enabling staged migration

Create an NSStagedMigrationManager with lightweight migration stages and custom migration stages, and add it to the NSPersistentStoreDescription options to enable staged migration in Core Data.

let migrationStages = [lightweightStage, customStage]
let migrationManager = NSStagedMigrationManager(migrationStages)

let persistentContainer = NSPersistentContainer(
    path: "/path/to/store.sqlite",
    managedObjectModel: myModel
)

var storeDescription = persistentContainer?.persistentStoreDescriptions.first

storeDescription?.setOption(
    migrationManager,
    forKey: NSPersistentStoreStagedMigrationManagerOptionKey
)

persistentContainer?.loadPersistentStores { storeDescription, error in
    if let error = error {
        // Handle any errors that may occur
    }
}

Code for setting up stage migration in SwiftData:

struct TripsApp: App {
    let container = ModelContainer(
        for: Trip.self,
        migrationPlan: SampleTripsMigrationPlan.self
    )

    var body: some Scene {
        WindowGroup {
            ContentView()
        }
        .modelContainer(container)
    }
}

Compared to the previous migration method, staged migration has a clearer structure; the amount of code needed to implement custom migration operations is less and the difficulty is lower. However, on the other hand, this also requires developers to have a better understanding of the migration process and to create data models in a timely manner as needed (decomposing non-lightweight migration tasks into a series of lightweight migration steps).

Conclusion

As the cornerstone of SwiftData, Apple will continue to add new APIs to Core Data in the coming years, whether intentionally or unintentionally. Considering that SwiftData still needs a few years to mature, many developers will need to use both SwiftData and Core Data in the same project in the future. Therefore, it is still valuable to keep up with the new features and trends of Core Data in a timely manner.