Building Machine-Aware Plugins

Machine-aware plugins can interact with the user's local computer through PIE Connect. This guide shows how to declare machine capabilities in your manifest and use the context.machine API.

Declaring Machine Capabilities

Add a machineCapabilities array to your plugin manifest:

name: my-machine-plugin
displayName: My Machine Plugin
description: A plugin that reads machine info and clipboard
tier: tool
version: 1.0.0
manifest:
  trigger: auto
  machineCapabilities:
    - machine.info
    - clipboard.read
  tool:
    name: my_machine_tool
    description: Reads machine info and clipboard
    parameters:
      type: object
      properties:
        action:
          type: string
          enum: [info, clipboard]
      required: [action]

When a user installs this plugin, they'll be asked to approve the requested capabilities.

Available Capabilities

Capability	Risk	Description	Platform
machine.info	Low	Read hostname, OS, uptime, memory	All
clipboard.read	Medium	Read clipboard contents	All
clipboard.write	Medium	Write text to the clipboard	All
notifications.send	Low	Send desktop notifications	All
messages.read	High	Read iMessages	macOS
screenshot.capture	High	Capture screenshots	macOS
desktop.control	Critical	Control mouse and keyboard	macOS
shell.run	Critical	Execute shell commands	All
filesystem	High	Read, write, list, search, move, copy, delete files	All
apps.automate	High	Automate macOS apps via scoped AppleScript	macOS
browser.data	High	Read browser tabs, history, bookmarks	macOS
contacts.read	High	Read contacts from macOS Contacts	macOS
calendar.read	High	Read events from macOS Calendar	macOS
system.control	Medium	Volume, dark mode, wifi, battery, processes, etc.	macOS
file.transfer	High	Upload files from Mac to cloud or download files to Mac (up to 100MB)	All

Using the context.machine API

Check if a machine is online

const online = await context.machine.isOnline();
if (!online) {
  return { error: true, message: 'No machine connected. Install PIE Connect on your Mac.' };
}

Execute a capability

// Basic machine info
const info = await context.machine.execute('machine.info', {});

// Read clipboard
const clip = await context.machine.execute('clipboard.read', {});

// Write to clipboard
await context.machine.execute('clipboard.write', { text: 'Hello from PIE!' });

// Send notification
await context.machine.execute('notifications.send', {
  title: 'My Plugin',
  message: 'Task completed!'
});

// Read iMessages
const messages = await context.machine.execute('messages.read', { limit: 5 });

// Execute shell command
const result = await context.machine.execute('shell.run', {
  command: 'ls -la ~/Documents',
  timeout: 10000
});

// Read a file
const file = await context.machine.execute('filesystem', {
  action: 'read',
  path: '~/Documents/notes.txt'
});

// List directory
const dir = await context.machine.execute('filesystem', {
  action: 'list',
  path: '~/Desktop',
  recursive: false
});

// Automate an app (Spotify)
const track = await context.machine.execute('apps.automate', {
  app: 'Spotify',
  script: 'get name of current track'
});

// Get browser tabs
const tabs = await context.machine.execute('browser.data', {
  action: 'tabs',
  browser: 'safari'
});

// Read contacts
const contacts = await context.machine.execute('contacts.read', {
  search: 'John',
  limit: 10,
  fields: ['name', 'email', 'phone']
});

// Read calendar events
const events = await context.machine.execute('calendar.read', {
  from: '2025-03-14',
  to: '2025-03-21',
  limit: 20
});

// System info
const battery = await context.machine.execute('system.control', { action: 'battery' });
const volume = await context.machine.execute('system.control', { action: 'volume' });
const apps = await context.machine.execute('system.control', { action: 'running_apps' });

// Set volume
await context.machine.execute('system.control', { action: 'set_volume', level: 50 });

// Toggle dark mode
await context.machine.execute('system.control', { action: 'set_dark_mode', enabled: true });

// Upload a file from Mac to cloud (convenience method)
const uploaded = await context.machine.upload('~/Documents/report.pdf');
console.log(uploaded.fileId, uploaded.url);

// Download a file from cloud to Mac (convenience method)
await context.machine.download('https://example.com/file.pdf', '~/Downloads/file.pdf');

// Upload via execute (equivalent to context.machine.upload)
const result = await context.machine.execute('file.transfer', {
  action: 'upload',
  path: '~/Desktop/screenshot.png'
});

// Download via execute (equivalent to context.machine.download)
await context.machine.execute('file.transfer', {
  action: 'download',
  url: 'https://example.com/data.csv',
  savePath: '~/Documents/data.csv'
});

List registered machines

const machines = await context.machine.list();
// Returns: [{ machineId, capabilities, connectedAt }]

Handling Offline Machines Gracefully

Always check if the machine is online before executing commands. If the machine is offline, return a helpful error message so the AI can guide the user:

async function handler(input, context) {
  const online = await context.machine.isOnline();
  if (!online) {
    return {
      error: true,
      message: 'Your machine is not connected. Please make sure PIE Connect is running on your Mac.',
    };
  }
  // ... proceed with capability execution
}

Risk Levels

Capabilities are categorized by risk level, which the user sees during plugin installation:

• Low — Read-only system info, notifications
• Medium — Clipboard access, system settings
• High — File access, app automation, personal data (contacts, calendar, messages, browser)
• Critical — Shell commands, direct mouse/keyboard control

Security Model

PIE uses a three-layer permission system:

1. Plugin manifest — Plugins must declare every capability they use in machineCapabilities
2. User approval — Users see the requested capabilities and risk levels at install time and must approve
3. Desktop enforcement — The desktop app only executes capabilities from its fixed registry

The apps.automate capability rejects scripts containing: do shell script, run script, system attribute, path to startup disk, do script, POSIX file, call method, store script, load script, scripting additions. Use shell.run for terminal commands and filesystem for file operations.

Every apps.automate response includes a dictionary field containing the app's scripting dictionary (sdef), fetched in parallel and cached. This tells you exactly what commands and classes the app supports — especially useful for fixing scripts after errors.

Error Handling

Machine commands can fail for several reasons:

• MACHINE_OFFLINE — No machine is connected
• MACHINE_TIMEOUT — Machine didn't respond within 60 seconds
• Permission denied — The capability requires additional OS permissions (e.g., Full Disk Access for messages.read)
• Capability not found — The capability is disabled or not supported

try {
  const result = await context.machine.execute('messages.read', { limit: 5 });
  return result;
} catch (err) {
  return { error: true, message: err.message };
}

Security Considerations

• Only request capabilities your plugin actually needs
• The security scanner will flag plugins that request unnecessary machine capabilities
• shell.run and desktop.control are critical-risk — users must explicitly enable them
• All machine commands are logged for auditing
• apps.automate scripts are sandboxed to tell application blocks