Deep Dive into iMessage: Behind the Making of an Agent

Foreword

At Photon AI, we’ve been constantly thinking: What should the future of AI and Agents look like? Are we going to type URLs into browsers to use Agents, or download apps to chat with so-called AI boyfriends/girlfriends? This doesn’t just sound uncool — it’s completely different from the future we’ve seen in all Sci-Fi.

We believe that in the future world, AI shouldn’t appear as “features” or “tools,” but should be like a form of life, deeply integrated into our social fabric. When our children see AI, they won’t be surprised like we are, nor will they view it as cold programs. They’ll look at AI the way they look at their friends and classmates — as First-Class Citizens in our society.

With this vision in mind, we began thinking about what we should do today to make Agents feel like part of our lives. One day, we had a flash of inspiration: why not bring Agents into iMessage? In the United States, almost everyone uses iMessage daily, with millions of messages flowing through it. It might just be the most natural, native way to interact with Agents in this era — letting Agents appear in your conversation list like friends, even joining your group chats.

So we began turning this idea from inspiration into reality, developing infrastructure that allows AI to exist in a truly “social” way. Ultimately, we built imessage-kit — an open-source iMessage control framework based on TypeScript, enabling developers to send, receive, and manipulate iMessage messages programmatically.

In building imessage-kit, we overcame numerous technical challenges and reimagined how AI communicates with humans.

1. Understanding iMessage Data Architecture

1.1 Database Location and Structure

All iMessage data is stored in a SQLite database located here:

~/Library/Messages/chat.db

This database has existed since iMessage was born. If you’re a long-time user, the database file might have grown to hundreds of MB, or even over 1GB.

Core table structure:

chat.db contains the following key tables:
├── message          # Main message table
├── chat             # Conversation table (group/individual chats)
├── handle           # Contact identifier table (phone/email)
├── attachment       # Attachment table
├── chat_message_join        # Message-conversation association table
├── chat_handle_join         # Conversation-contact association table
└── message_attachment_join  # Message-attachment association table

1.2 Timestamp Conversion: Mac Epoch Time

The timestamps in the database aren’t the Unix timestamps we’re familiar with. For example:

408978598

If you directly use new Date(408978598) to convert it, you’ll definitely get the wrong time.

Key point: macOS uses its own epoch time, starting from 2001-01-01, not Unix’s 1970-01-01.

The correct conversion method:

const MAC_EPOCH = new Date('2001-01-01T00:00:00Z').getTime()
function convertMacTimestamp(timestamp: number): Date {
    // macOS stores nanoseconds, need to divide by 1000000
    return new Date(MAC_EPOCH + timestamp / 1000000)
}

This is the standard time representation for macOS and iOS systems (Core Data timestamp) — simple once you get used to it.

1.3 Message Content Encoding: NSAttributedString

iMessage text messages are stored in two fields:

• message.text: Plain text messages
• message.attributedBody: Rich text messages (NSAttributedString stored as binary plist format)

In the imessage-kit project, we used two strategies to parse attributedBody:

Strategy 1: Direct string matching (fast but rough)

const bufferStr = buffer.toString('utf8')
// Match readable characters (ASCII + Chinese)
const readableMatches = bufferStr.match(/[\x20-\x7E\u4e00-\u9fff]{5,}/g)

Extract readable text directly from binary data using regex, then filter out plist keywords (like NSAttributedString, NSDictionary, etc.).

Strategy 2: Using the plutil tool (accurate but slower)

plutil -convert xml1 -o - "temp.plist"

The built-in macOS plutil tool can convert binary plists to XML format, then we extract content from <string> tags in the XML.

Each method has its pros and cons:

• Method 1 is fast but occasionally extracts garbled strings
• Method 2 is accurate but requires creating temporary files and calling system commands

In practice, imessage-kit tries Method 1 first, falling back to Method 2 if needed — a good compromise.

2. Breaking Through macOS Security

2.1 Full Disk Access: The Required Pass

Since macOS Mojave (10.14), Apple has tightened privacy protection. The ~/Library/Messages directory is listed as a protected resource — unauthorized programs can’t access it at all.

Symptom:

$ sqlite3 ~/Library/Messages/chat.db
Error: unable to open database "chat.db": Operation not permitted

Solution:

1. Open System Settings → Privacy & Security → Full Disk Access
2. Click the ”+” button to add your commonly used terminals or IDEs (like Terminal, iTerm, VS Code, Cursor)
3. Restart the application (This step is crucial — many people forget to restart, resulting in permissions not taking effect)

It’s recommended to add permissions for both development tools and terminals to ensure normal operation in different environments.

2.2 SQLite WAL Mode Specifics

The iMessage database uses SQLite’s WAL (Write-Ahead Logging) mode, so you’ll see three files:

chat.db
chat.db-shm   # Shared memory file
chat.db-wal   # Write-ahead log file

Important characteristic: When new messages arrive, chat.db-wal updates immediately, but the main database file chat.db might lag several seconds or even minutes (waiting for checkpoint triggers).

This significantly affects real-time message monitoring. If you directly watch for changes in the chat.db file, the delay will be noticeable. Better approaches:

1. Use periodic polling of the database instead of file change monitoring
2. Open the database in read-only mode, letting SQLite handle WAL files itself

// The correct approach
const db = new Database(path, { readonly: true })

2.3 Concurrent Access Considerations

SQLite supports multiple readers, but write operations lock the database. imessage-kit deliberately opens the database in read-only mode (readonly: true) to avoid conflicts.

3. Sending Messages: The Art and Compromise of AppleScript

3.1 Why AppleScript?

Apple hasn’t provided an official API for iMessage. As developers, the only official automation tool we can use is AppleScript — this ancient scripting language born in 1993.

A simple sending example:

tell application "Messages"
    set targetBuddy to buddy "+1234567890"
    send "Hello from automation!" to targetBuddy
end tell

But in practice, there are quite a few tricks to it.

3.2 Character Escaping Issues

AppleScript is particularly picky about special characters and requires proper escaping:

// Wrong approach
const text = 'He said "Hello"'
const script = `send "${text}" to targetBuddy`
// Will directly throw a syntax error!

// Correct approach
function escapeAppleScriptString(str: string): string {
    return str
        .replace(/\\/g, '\\\\')  // Backslash
        .replace(/"/g, '\\"')    // Double quote
        .replace(/\n/g, '\\n')   // Newline
        .replace(/\r/g, '\\r')   // Carriage return
        .replace(/\t/g, '\\t')   // Tab
}

3.3 Sandbox Restriction Workarounds

If you want to send file attachments, you’ll encounter an even more troublesome problem: macOS sandbox restrictions.

Problem: Messages.app runs in a sandbox and can only access specific directories (like Documents, Downloads, Pictures). If your file is elsewhere, sending will fail directly.

Solution: Temporarily copy the file to the ~/Pictures directory

-- Bypass sandbox: copy to Pictures directory
set picturesFolder to POSIX path of (path to pictures folder)
set targetPath to picturesFolder & "imsg_temp_1234567890_file.pdf"
do shell script "cp " & quoted form of "/restricted/path/file.pdf" & " " & quoted form of targetPath

-- Send file
set theFile to (POSIX file targetPath) as alias
send theFile to targetBuddy

-- Delay to ensure upload completion (especially for iMessage)
delay 3

We specifically created a TempFileManager in imessage-kit that automatically scans and cleans up imsg_temp_* files in ~/Pictures.

3.4 File Sending Delays

Different file sizes need different delay times to ensure iMessage can successfully upload to iCloud:

function calculateFileDelay(filePath: string): number {
    const sizeInMB = getFileSizeInMB(filePath)
    if (sizeInMB < 1) return 2      // < 1MB: 2 seconds
    if (sizeInMB < 10) return 3     // 1-10MB: 3 seconds
    return 5                         // > 10MB: 5 seconds
}

3.5 Group Message ChatId Handling

Sending messages to group chats is more complex than individual chats because you need to use chatId:

tell application "Messages"
    set targetChat to chat id "chat45e2b868ce1e43da89af262922733382"
    send "Hello group!" to targetChat
end tell

How to get chatId?

Query directly from the chat table in the database:

SELECT
    chat.guid AS chat_id,
    chat.display_name AS name,
    (SELECT COUNT(*) FROM chat_handle_join
     WHERE chat_id = chat.ROWID) > 1 AS is_group
FROM chat
WHERE is_group = 1

ChatId format explanation:

• Group chats: Use GUID (like chat45e2b868ce1e43da...)
• Individual chats: Could be iMessage;+1234567890 or just +1234567890
• AppleScript format: iMessage;+;chat45e2b868... (needs standardization)

For developer convenience, we built smart chatId standardization logic into imessage-kit that automatically handles these format differences.

4. Real-time Monitoring: Polling and Performance Optimization

Unlike traditional iMessage automation, iMessage Agents require us to retrieve iMessage messages promptly and have the Agent respond. Research shows that delays of around 500 milliseconds are perceptible to humans and cause discomfort. In practice, we discovered many “pitfalls” in real-time message monitoring and ultimately chose polling with incremental queries.

4.1 Why Polling Instead of Event Listening?

Many people ask: Why not use filesystem monitoring (like fs.watch) to detect new messages?

Several reasons:

1. WAL mode causes chat.db update delays
2. File monitoring triggers too many false positives (internal database operations also modify files)
3. Polling combined with timestamp queries is more reliable

Optimal polling interval:

// Too fast: wastes CPU
pollInterval: 500
// Too slow: noticeable delay
pollInterval: 10000
// Sweet spot: 2 seconds
pollInterval: 2000  // Default value

2 seconds is the balance point we found between response speed and system load after many attempts while developing imessage-kit.

4.2 Incremental Queries and Deduplication

Each poll only queries for new messages since the last check:

// Dynamic overlap time adjustment: take the minimum of 1 second and polling interval
const overlapMs = Math.min(1000, this.pollInterval)
const since = new Date(lastCheckTime.getTime() - overlapMs)
const { messages } = await db.getMessages({
    since,
    excludeOwnMessages: false  // Get all messages first (including own)
})

// Deduplicate using Map
const seenMessageIds = new Map<string, number>()
const newMessages = messages.filter(msg => !seenMessageIds.has(msg.id))

Why overlap time? To prevent message loss at time boundaries (clock precision and database write order issues).

5. Cross-Runtime Support: Bun vs Node.js

5.1 Database Driver Selection

imessage-kit has a clever design that automatically detects the runtime environment:

async function initDatabase() {
    if (typeof Bun !== 'undefined') {
        // Bun runtime - use built-in SQLite
        const bunSqlite = await import('bun:sqlite')
        Database = bunSqlite.Database
    } else {
        // Node.js runtime - use better-sqlite3
        const BetterSqlite3 = await import('better-sqlite3')
        Database = BetterSqlite3.default
    }
}

Different runtimes have different characteristics:

• Bun (bun): Built-in driver, zero dependencies, faster startup, smaller memory footprint
• Node.js (better-sqlite3): Mature and stable, good community support, complete ecosystem

5.2 Bun’s Zero-Dependency Advantage

The biggest advantage of using Bun: zero external dependencies.

// Node.js
"dependencies": {
    "better-sqlite3": "^11.0.0"
}

// Bun
"dependencies": {}  // Completely zero dependencies!

For projects that prefer a minimalist style, this is a significant advantage.

6. Real-World Scenarios and Use Cases

After developing imessage-kit, the Photon AI team used it for some internal experiments. We even used it to take over our iMessage accounts to introduce our company to investors. During usage, we added many more developer-friendly capabilities to imessage-kit, such as more intuitive syntax and chaining. Here are some functional examples achievable with imessage-kit.

6.1 Automated Reply Bot

Intelligent replies based on message content (can connect to an Agent for truly intelligent replies):

// Start watching
await sdk.startWatching({
    onDirectMessage: async (message) => {
        await sdk.message(message)
            .ifFromOthers()
            .matchText(/urgent|emergency/i)
            .replyText('Got it! I'll handle this ASAP.')
            .execute()
    }
})

6.2 Message Data Analysis

Using the SDK for message data analysis is particularly convenient:

const result = await sdk.getMessages({
    since: new Date('2024-01-01'),
    limit: 10000
})

// Count most active contacts
const senderCounts = new Map()
for (const msg of result.messages) {
    senderCounts.set(
        msg.sender,
        (senderCounts.get(msg.sender) || 0) + 1
    )
}

// Sort by message count
const sorted = Array.from(senderCounts.entries())
    .sort((a, b) => b[1] - a[1])
    .slice(0, 10)
console.log('Top 10 most active contacts:', sorted)

You can further analyze message time distribution, group chat activity, attachment type statistics, and more.

6.3 Webhook Integration

Forward iMessage notifications to other systems (like Slack, Discord):

const sdk = new IMessageSDK({
    webhook: {
        url: '<YOUR_WEBHOOK_URL>',
    },
})
await sdk.startWatching()

This is particularly useful for team collaboration or monitoring important messages.

6.4 Message Sending Tracking Mechanism

imessage-kit implements OutgoingMessageManager to track sent messages. By starting the watcher, you can get the message object immediately after sending:

// Start watcher
await sdk.startWatching()

// Send message and get confirmation
const result = await sdk.send('+1234567890', 'Hello!')
if (result.message) {
    console.log('Message ID:', result.message.id)
    console.log('Send time:', result.message.date)
}

How it works:

1. Create MessagePromise before sending, recording send time, content, and chatId
2. AppleScript executes the send
3. Watcher polls for new messages, matching by timestamp and content
4. On successful match, resolve Promise, returning complete Message object

The matching logic considers various chatId formats (iMessage;-;recipient vs recipient), automatically extracting core identifiers for matching.

6.5 Automatic Temporary File Cleanup

To bypass sandbox restrictions, files are temporarily copied to ~/Pictures when sending attachments. TempFileManager handles automatic cleanup of these files:

Workflow:

1. File naming convention: All temporary files prefixed with imsg_temp_
2. Cleanup on startup: SDK initialization cleans up old leftover files
3. Periodic cleanup: Scans every 5 minutes, deleting files older than 10 minutes
4. Cleanup on shutdown: SDK cleans all temporary files when closing

// TempFileManager configuration
const DEFAULT_CONFIG = {
    maxAge: 10 * 60 * 1000,           // Keep files for 10 minutes    
    cleanupInterval: 5 * 60 * 1000,   // Clean every 5 minutes
}

This mechanism ensures temporary files are cleaned on next startup even if the program exits abnormally.

6.6 Message Deduplication Mechanism

Watcher uses Map<string, number> to record processed message IDs, preventing duplicate processing:

private seenMessageIds = new Map<string, number>()
// On each check
const newMessages = messages.filter(msg => !this.seenMessageIds.has(msg.id))
// Mark as processed
for (const msg of newMessages) {
    this.seenMessageIds.set(msg.id, Date.now())
}
// Periodic cleanup (keep records from last hour)
if (this.seenMessageIds.size > 10000) {
    const hourAgo = Date.now() - 3600000    
    for (const [id, timestamp] of this.seenMessageIds.entries()) {
        if (timestamp < hourAgo) {
            this.seenMessageIds.delete(id)
        }
    }
}

Key points:

• Use Map instead of Set, storing timestamps for cleanup
• Threshold-triggered cleanup (when exceeding 10,000 records)
• Keep only last hour’s records to prevent memory leaks
• Set overlap time (1 second) during polling to prevent boundary loss

7. Pitfalls and Solutions

7.1 Getting Messages Immediately After Sending

Problem: Can’t immediately get the sent message object after calling send(), return value is undefined.

Reason: AppleScript sends messages asynchronously, takes time to write to database.

Solution:

// Start watcher
await sdk.startWatching()

// Send message (watcher will capture and return)
const result = await sdk.send('+1234567890', 'Hello!')
if (result.message) {
    console.log('Message ID:', result.message.id)
}

imessage-kit implements OutgoingMessageManager to correlate sent messages through timestamp and content matching.

7.2 Attachment Paths with ~ Symbol

Attachment paths stored in the database might contain ~:

~/Library/Messages/Attachments/abc/def/IMG_1234.heic

Solution:

import { homedir } from 'os'

const fullPath = rawPath.startsWith('~')
    ? rawPath.replace(/^~/, homedir())
    : rawPath

8. Performance Optimization

8.1 Database Query Optimization

Slow query example:

-- Bad: full table scan
SELECT * FROM message WHERE text LIKE '%keyword%'

Optimized:

-- Use index + limit time range
SELECT * FROM message
WHERE date >= ? AND text LIKE '%keyword%'
ORDER BY date DESC
LIMIT 100

8.2 Concurrent Send Control

When sending multiple messages simultaneously, use a Semaphore to limit concurrency:

class Semaphore {
    private running = 0    
    private waiting: Array<() => void> = []
    
    constructor(private readonly limit: number) {}
    
    async acquire(): Promise<() => void> {
        while (this.running >= this.limit) {
            await new Promise<void>(resolve => this.waiting.push(resolve))
        }
        this.running++        
        // Return release function        
        return () => {
            this.running--            
            const next = this.waiting.shift()
            if (next) next()
        }
    }
    
    async run<T>(fn: () => Promise<T>): Promise<T> {
        const release = await this.acquire()
        try {
            return await fn()
        } finally {
            release()
        }
    }
}

// Usage
const sem = new Semaphore(5) 
// Maximum 5 concurrent
await sem.run(() => sendMessage(...))

In imessage-kit, we’ve provided default concurrency limit support for developers, with a default upper limit of 5, preventing Messages.app from crashing due to sending too many messages at once.

8.3 Long-Running Memory Management

For listening services that need to run for extended periods, imessage-kit uses a Map in the watcher to record processed message IDs. The project implements an automatic cleanup mechanism:

// Trigger cleanup when Map size exceeds threshold
if (this.seenMessageIds.size > 10000) {
    const hourAgo = Date.now() - 3600000  // 1 hour ago    
    for (const [id, timestamp] of this.seenMessageIds.entries()) {
        if (timestamp < hourAgo) {
            this.seenMessageIds.delete(id)
        }
    }
}

This strategy effectively controls memory usage when message volume is high, keeping only the last hour’s message records, helping developers reduce the burden of managing memory themselves.

9. Current Limitations and Solutions

9.1 Known Limitations

imessage-kit is based on AppleScript and local database reading. As an open-source SDK, it can already accomplish most core operations. However, since Apple hasn’t opened related APIs, it still has some system-level limitations that can’t be overcome, such as:

1. Message editing - Cannot edit sent messages
2. Message recall - Cannot recall messages within 2 minutes
3. Tapback reactions - Can only read reactions, cannot send (hearts, likes, haha, etc.)
4. Typing indicators - Cannot send/receive real-time typing status
5. Message effects - Cannot send messages with effects (fireworks, confetti, balloons, etc.)
6. Read receipts - Cannot mark messages as read/unread
7. Sticker sending - Cannot send iMessage stickers
8. Voice messages - Cannot send voice messages with waveform display
9. FaceTime integration - Cannot create FaceTime links or monitor call status

Additionally, AppleScript inherently has issues with stability and weak concurrency capabilities; the entire system depends on the user’s iCloud account and needs to be deployed on a fixed Mac, limiting scalability.

9.2 Advanced iMessage Kit: Next-Generation Infrastructure Breaking System Limitations

After conducting in-depth research on these issues, we created Advanced iMessage Kit. It bypasses AppleScript limitations through a completely new technical architecture, truly implementing almost all iMessage capabilities, and achieves higher concurrency and more stable service by building an entirely new infrastructure to support the underlying services.

In other words, we gave the Agent a phone. We’ve open-sourced part of the Advanced iMessage Kit code and welcome more people to bring Agents into iMessage.

Conclusion

iMessage automation is a technically challenging yet fascinating field. From underlying database structures to AppleScript scripting, from system permission control to performance optimization, every aspect requires deep understanding of macOS’s operating mechanisms.

To enable more developers to easily explore this field, we’ve packaged all these capabilities into a free, open-source TypeScript SDK. It simplifies complex iMessage operations into easy-to-understand, easy-to-integrate APIs, helping you accomplish in minutes what used to take days.

For developers who need stronger capabilities but don’t want to self-host on their own Mac, we also provide the more comprehensive Advanced iMessage Kit. It not only unlocks more system-level features but also significantly reduces deployment and configuration complexity.

We’re also continuously exploring more possibilities for AI Agent interaction, including making Agents have a smoother, more comfortable experience in iMessage, proactively sending messages, knowing when to stop, and not sending long replies all at once. We’re also internally experimenting with some bolder, more interesting directions, which we’ll continue to share with the community when mature.

If you find any issues with imessage-kit, please feel free to submit Issues or PRs on GitHub. If this project helps you, a Star is enough to support us in continuing to improve it.

About the Author

• LingJueYa: Photon AI engineer, an INTJ who does everything, passionate about exploring code, documenting life, and polishing products, being a lifelong learner forever.