SwiftUI and Core Data: Data Fetching

In this article, we will explore ways to fetch Core Data data in bulk within SwiftUI views, and attempt to create a FetchRequest that can use mock data. As this article will involve many techniques and methods introduced in previous articles, it is best to read them together.

Creating a FetchRequest that can use Mock data

Is FetchRequest breaking the unidirectional data flow?

For every developer who uses Core Data in SwiftUI, @FetchRequest is an unavoidable topic. FetchRequest greatly simplifies the difficulty of getting Core Data data in the view. With @ObservedObject (the managed object conforms to the ObservableObject protocol), developers can implement real-time response to data changes in the view with just a few lines of code.

However, for developers who adopt the unidirectional data flow approach, @FetchRequest is like the Sword of Damocles hanging over their heads, always worrying them. Class Redux framework usually recommends that developers compose the entire app’s state into a single structural instance (State, conforming to Equatable protocol), and the view responds to changes in the state by observing them (some frameworks support slice-like observation to improve performance). However, @FetchRequest splits a large part of the state composition in the app into multiple views from the independent structural instance. In recent years, many developers have tried to find replacement solutions that are more in line with the spirit of Redux, but the effects are not well understood.

I have also made many attempts, but ultimately found that FetchRequest still seems to be the best solution in current SwiftUI. Let me briefly introduce my exploration process (using TCA framework as an example):

• Get and manage value data in Reducer

In the task (or onAppear) function, start a long-term Effect by sending an Action to create an NSFetchedResultsController to fetch the data set with a specified predicate from Core Data. In the NSFetchedResultsControllerDelegate implementation, convert the managed objects to the corresponding value types and pass them to the Reducer.

Use the IdentifiedArray type in State to store the data set for splitting the Reducer with .forEach.

The above approach is indeed a way that fully conforms to the spirit of Redux, but as we give up the lazy-loading feature of Core Data when converting managed objects to value types, it will lead to serious performance and memory usage issues once the data volume is large. Therefore, it is only suitable for scenarios where the data set is small.

• Getting and managing AnyConvertibleValueObservableObject in Reducer

Similar to the above method, but omitting the process of converting to value types, wrapping the managed object as AnyConvertibleValueObservableObject, and directly saving the reference type in State. However, considering that TCA will move Reducer out of the main thread later, from a thread-safe perspective, this plan was eventually abandoned.

Since we ultimately need to use AnyConvertibleValueObservableObject (managed objects) in the view, the data acquisition process must be performed in the context of the main thread (the context of data binding is ViewContext). Once Reducer is moved out of the main thread, it means that AnyConvertibleValueObservableObject will be saved in a non-threaded State instance. Although in practice, holding a managed object in a thread other than the one that created the managed object will not cause a crash as long as the non-thread-safe properties of the managed object are not accessed, out of caution, I finally gave up this approach.

• Get and manage WrappedID in Reducer

Similar to the method above, only save the thread-safe WrappedID (the wrapped NSManagedObjectID) in the State. In the view, use WrappedID to obtain the corresponding AnyConvertibleValueObservableObject or value type. Although it may increase the code in the view, this method is almost perfect from both the data flow processing and thread-safe perspectives.

However, the reason I gave up on all of the above attempts in the end was due to performance issues.

• Any changes to Core Data will result in a change in the app’s single State. Although TCA has splitting mechanisms, as the complexity and amount of data in the application increases, the performance issues caused by comparing State will become more serious.
• The operation of creating NSFetchedResultsController and obtaining the initial batch of data is initiated from onAppear. Due to TCA’s Action handling mechanism, there is a perceivable delay in the initial display of data (far less effective than obtaining data through FetchRequest in the view).
• Due to TCA’s Reducer being unable to automatically bind with the view’s lifecycle, the perceivable delay mentioned above will occur every time onAppear is triggered.

Finally, I decided to let go of my worries and still use the @FetchRequest-like approach to retrieve data in the view. By creating a new FetchRequest that can use mock data, I achieved the goal of being testable, previewable, and modular as mentioned in the article ”SwiftUI and Core Data: The Challenges“.

NSFetchedResultsController

NSFetchedResultsController retrieves a specific data set from Core Data through NSFetchRequest and sends the data set to instances that conform to the NSFetchedResultsControllerDelegate protocol to implement methods for displaying data on the screen.

Simply put, NSFetchedResultsController automatically updates the data set in memory in response to NSManagedObjectContextObjectsDidChange and NSManagedObjectContextDidMergeChangesObjectIDs notifications after the initial data set is obtained (performFetch) based on notification content (insert, delete, update, etc.). To improve the efficiency of UITableView (UICollectionView) updates, NSFetchedResultsController decomposes changes in data into specific actions (NSFetchRequestResultType) to allow developers to quickly adjust the display content of UITableView (without refreshing all data).

Unfortunately, the optimizations based on NSFetchRequestResultType prepared by NSFetchedResultsController for UITableView do not work in SwiftUI. In SwiftUI, ForEach automatically handles view additions, deletions, and other operations based on data identifiers (Identifier). Therefore, when using NSFetchedResultsController in SwiftUI, only the controllerDidChangeContent(_ controller:) method in the NSFetchedResultsControllerDelegate needs to be implemented.

Custom Types that Conform to DynamicProperty Protocol

In SwiftUI, common types that can serve as a source of truth conform to the DynamicProperty protocol. The DynamicProperty protocol provides access to the SwiftUI-managed data pool for data. With the help of the private _makeProperty method, data can request space in the SwiftUI data pool for storage and retrieval. This serves two purposes:

• Changes in the data will trigger updates in the views that are bound to it.
• Since the underlying data is not stored in the view, SwiftUI can create new view description instances at any time during the view’s lifetime without worrying about data loss.

Although Apple has not disclosed the details of the _makeProperty method, developers cannot request data storage addresses from SwiftUI on their own. However, similar effects can be achieved by using types that conform to the DynamicProperty protocol, such as State, in custom types that also conform to the DynamicProperty protocol.

When creating a custom DynamicProperty type, it is important to keep the following in mind:

• Environment values or objects can be used in custom types

Once a view is loaded, all types that conform to the DynamicProperty protocol in the view will have the ability to access environment data. However, attempting to access environment data before the view is loaded or without providing environment values (such as forgetting to inject an environment object or not providing the correct view context) will result in a crash.

• Custom types will also be recreated when SwiftUI recreates the view description instance during the view’s lifespan

During the view’s lifespan, if SwiftUI recreates the view description instance, then it will recreate all properties regardless of whether they conform to DynamicProperty or not. This means that persistent data (consistent with the view’s lifespan) must be saved in the DynamicProperty type provided by the system.

• The update method will only be called after the view has been loaded by SwiftUI

The only method exposed by the DynamicProperty protocol is update. SwiftUI will call this method when the view is first loaded and when data that can trigger a view update changes in a type that conforms to DynamicProperty. Since instances of the type may be repeatedly created during the view’s lifespan, data preparation (such as fetching NSFetchedResultsController data for the first time, creating subscription relationships) and update work should be performed in this method.

• Data that triggers a view update cannot be synchronously changed in the update method

As with updating the Source of truth in SwiftUI views, in a view update cycle, the Source of truth cannot be updated again. This means that even though we can only change data in the update method, we must find a way to stagger this update cycle.

Usage of MockableFetchRequest

MockableFetchRequest provides the ability to dynamically retrieve data similar to FetchRequest, but with the following characteristics:

• MockableFetchRequest returns data of type AnyConvertibleValueObservableObject.

The NSFetchedResultsController in MockableFetchRequest directly converts data to AnyConvertibleValueObservableObject type. This allows for the various benefits introduced in the previous section to be directly enjoyed in the view. In addition, when declaring MockableFetchRequest in the view, the use of specific managed object types can be avoided, which is conducive to modular development.

@MockableFetchRequest(\ObjectsDataSource.groups) var groups // Code is not polluted by specific managed object types

• Switch data sources through environment values.

In the previous section, we provided a preview capability without a managed environment for a view containing a single AnyConvertibleValueObservableObject object by creating data that conforms to the TestableConvertibleValueObservableObject protocol. MockableFetchRequest provides the ability to preview a set of data without a managed environment for a view that retrieves a data set.

First, we need to create a type that conforms to the ObjectsDataSourceProtocol protocol and specifies the data source by making the property of FetchDataSource type.

// Included in MockableFetchRequest code
public enum FetchDataSource<Value>: Equatable where Value: BaseValueProtocol {
    case fetchRequest // retrieves data through NSFetchedResultsController in MockableFetchRequest
    case mockObjects(EquatableObjects<Value>) // uses provided Mock data
}

public extension EnvironmentValues {
    var dataSource: any ObjectsDataSourceProtocol {
        get { self[ObjectsDataSourceKey.self] }
        set { self[ObjectsDataSourceKey.self] = newValue }
    }
}

// Code that developers need to customize
public struct ObjectsDataSource: ObjectsDataSourceProtocol {
    public var groups: FetchDataSource<TodoGroup>
}

public struct ObjectsDataSourceKey: EnvironmentKey {
    public static var defaultValue: any ObjectsDataSourceProtocol = ObjectsDataSource(groups: .mockObjects(.init([MockGroup(.sample1).eraseToAny()]))) // Sets the default data source to be from Mock data
}

Real-time modifications can be made to the data during preview (see the GroupListContainer code in Todo for more details).

When the application is running in a hosted environment, simply provide the correct view context and modify the property values in dataSource to fetchRequest.

• Allow not providing NSFetchRequest in the constructor.

When using @FetchRequest in a view, we have to set NSFetchRequest (or NSPredicate) when declaring the FetchRequest variable. As a result, when extracting the view to a separate Package, we still need to import the library containing the specific Core Data managed object definitions, making it impossible to achieve complete decoupling. In MockableFetchRequest, there is no need to provide NSFetchRequest when declaring, and the required NSFetchRequest can be dynamically provided for MockableFetchRequest when the view is loaded (detailed demo code).

public struct GroupListView: View {
    @MockableFetchRequest(\ObjectsDataSource.groups) var groups
    @Environment(\.getTodoGroupRequest) var getTodoGroupRequest

    public var body: some View {
        List {
                ForEach(groups) { group in
                    GroupCell(
                        groupObject: group,
                        deletedGroupButtonTapped: deletedGroupButtonTapped,
                        updateGroupButtonTapped: updateGroupButtonTapped,
                        groupCellTapped: groupCellTapped
                    )
                }
        }
        .task {
            guard let request = await getTodoGroupRequest() else { return } // Dynamically obtain the required Request through the environment method when the view is loaded
            $groups = request // Dynamically set MockableFetchRequest
        }
        .navigationTitle("Todo Groups")
    }
}

• Avoid updating data sets with operations that do not cause ID changes.

MockableFetchRequest does not update the dataset even if the attribute values of the data change, as long as the ID sequence or quantity of the dataset remains unchanged. Because AnyConvertibleValueObservableObject itself conforms to the ObservableObject protocol, even if MockableFetchRequest does not update the dataset, the view will still respond to changes in the properties of AnyConvertibleValueObservableObject. This reduces the frequency of changes in the ForEach dataset, improving the efficiency of SwiftUI views.

• Provides a lighter Publisher to monitor data changes

The original FetchRequest provides a Publisher (through the projection value) that responds to each dataset change. However, this Publisher responds too frequently and even if only one data attribute in the dataset changes, it will issue all the data in the dataset. MockableFetchRequest simplifies this by issuing an empty notification (AnyPublisher<Void, Never>) only when the dataset changes.

public struct GroupListView: View {
    @MockableFetchRequest(\ObjectsDataSource.groups) var groups

    public var body: some View {
        List {
           ...
        }
        .onReceive(_groups.publisher){ _ in
            print("data changed")
       }
    }
}

To achieve the same effect as @FetchRequest, only the permission of the sender property needs to be raised.

The following image is a preview demonstration created entirely with mock data:

Explanation of MockableFetchRequest Code

This section only explains part of the code, please see here for the complete code.

• How to avoid data updates overlapping with update cycles

In MockableFetchRequest, we manage two different data sources through a publisher of type PassthroughSubject<[AnyConvertibleValueObservableObject<Value>], Never>. By using the delay operator, we can achieve staggered data updates. If necessary, we can also use Tasks to asynchronously update the data.

cancellable.value = sender
    .delay(for: .nanoseconds(1), scheduler: RunLoop.main) // delay for 1 nanosecond is enough
    .removeDuplicates {
        EquatableObjects($0) == EquatableObjects($1)
    }
    .receive(on: DispatchQueue.main)
    .sink {
        updateWrappedValue.value($0)
    }

• Wrap the data that needs to be modified with a reference type to avoid unnecessary updates to the view.

By creating a reference type with wrapping purposes to hold the data that needs to be modified (holding the reference in @State), the following goals can be achieved: 1. Ensure that the data’s lifecycle is consistent with the view’s lifespan; 2. Allow the data to be modified; 3. Changing the data will not trigger view updates.

  extension MockableFetchRequest {
      // Wrapper type
      final class MutableHolder<T> {
          var value: T
          @inlinable
          init(_ value: T) {
              self.value = value
          }
      }
  }

  public struct MockableFetchRequest<Root, Value>: DynamicProperty where Value: BaseValueProtocol, Root: ObjectsDataSourceProtocol {
      @State var fetcher = MutableHolder<ConvertibleValueObservableObjectFetcher<Value>?>(nil)

      func update() {
          ...
          // fetcher is persistent, modifying fetcher.value will not trigger view updates
          if let dataSource = dataSource as? Root, case .fetchRequest = dataSource[keyPath: objectKeyPath], fetcher.value == nil {
              fetcher.value = .init(sender: sender)
              if let fetchRequest {
                  updateFetchRequest(fetchRequest)
              }
          }
          ...
      }
  }

• How to compare two [AnyConvertibleValueObservableObject<Value>] for equality?

Due to Swift’s inability to directly compare data containing associated types, an intermediate type EquatableObjects is created and made to conform to the Equatable protocol to facilitate comparison of two [AnyConvertibleValueObservableObject<Value>] data, avoiding unnecessary view refreshes.

  public struct EquatableObjects<Value>: Equatable where Value: BaseValueProtocol {
      public var values: [AnyConvertibleValueObservableObject<Value>]

      public static func== (lhs: Self, rhs: Self) -> Bool {
          guard lhs.values.count == rhs.values.count else { return false }
          for index in lhs.values.indices {
              if !lhs.values[index]._object.isEquatable(other: rhs.values[index]._object) { return false }
          }
          return true
      }

      public init(_ values: [AnyConvertibleValueObservableObject<Value>]) {
          self.values = values
      }
  }

  // in MockableFetchRequest
  if let dataSource = dataSource as? Root, case .mockObjects(let objects) = dataSource[keyPath: objectKeyPath],
     objects != EquatableObjects(_values.wrappedValue)  // remove duplicates
  {
      sender.send(objects.values)
  }

  ...

  cancellable.value = sender
      .delay(for: .nanoseconds(1), scheduler: RunLoop.main)
      .removeDuplicates {
          EquatableObjects($0) == EquatableObjects($1) // remove duplicates
      }
      .receive(on: DispatchQueue.main)
      .sink {
          updateWrappedValue.value($0)
      }

• Solving the problem of not being able to introduce self in closures by manipulating underlying data

Using the underlying data in the subscription closure can bypass the problem of not being able to introduce self in the struct.

let values = _values  // Corresponding to the type State, which is the underlying data of the values property of MockableFetchRequest
let firstUpdate = firstUpdate
let animation = animation
updateWrappedValue.value = { data in
    var animation = animation
    if firstUpdate.value {
        animation = nil
        firstUpdate.value = false
    }
    withAnimation(animation) {
        values.wrappedValue = data // Operate on the underlying data
    }
}

SectionedFetchRequest

I haven’t made any changes to the SectionedFetchRequest method for retrieving data for the time being. The main reason is that I haven’t figured out how to organize the returned data.

Currently, SectionedFetchRequest has serious performance issues when dealing with large amounts of data. This is because once multiple ForEach are present in SwiftUI’s lazy container, the optimization ability of the lazy container for child views will be lost. Any changes in the data will result in the lazy container updating all child views instead of only updating the visible ones.

The data type returned by SectionedFetchRequest is SectionedFetchResults, which can be viewed as an ordered dictionary with SectionIdentifier as the key. Reading its data will inevitably use multiple ForEach in the lazy container, causing performance issues.

@SectionedFetchRequest<String, Quake>(
    sectionIdentifier: \.day,
    sortDescriptors: [SortDescriptor(\.time, order: .reverse)]
)
private var quakes: SectionedFetchResults<String, Quake>

List {
    ForEach(quakes) { section in
        Section(header: Text(section.id)) {
            ForEach(section) { quake in
                QuakeRow(quake: quake)
            }
        }
    }
}

I currently have two ideas:

• Return all data as an array (with sectionIdentifier as the primary sorting condition) and provide the starting offset (or ID) for each section in the returned array, as well as the amount of data in each section.
• Return all data as an array (with sectionIdentifier as the primary sorting condition) and insert specific AnyConvertibleValueObservableObject data at the beginning and end of each section (since WrappedID exists, we can easily create mock data).

In either case, developers must give up using the native Section feature of SwiftUI and, in a lazy container, handle segmenting the data display based on the provided additional data.

Core Data itself does not have the ability to directly retrieve grouped records from SQLite. The current implementation is to retrieve all data with sectionIdentifier as the primary sorting condition. Then, group sectionIdentifier by propertyToGroupBy to obtain the amount of data for each group (count). Calculate the offset of each section using the returned statistics.

Summary and Introduction to the Next Article

In this article, we have created a FetchRequest that supports mock data and briefly discussed things to keep in mind when customizing a type that conforms to the DynamicProperty protocol.

In the next article, we will explore how to safely respond to data in SwiftUI, how to avoid unexpected data loss that can lead to abnormal behavior and application crashes.