# Pike USB Module - Future Enhancements

## Pollfd Integration (Event-Driven FD Monitoring)

### Status: ✅ IMPLEMENTED (Changeset 23)

The module now uses true event-driven file descriptor monitoring using libusb's pollfd API integrated with Pike's backend system.

### Goal

Replace/augment periodic polling with file descriptor-based wakeups using:
- `libusb_get_pollfds()` - Get current USB file descriptors
- `libusb_set_pollfd_notifiers()` - Register add/remove callbacks
- Pike's `fd_callback_box` - Register fds with Pike backend

### Benefits

- **Zero CPU when idle**: No polling when no USB activity
- **Immediate response**: Wake instantly when USB I/O ready
- **Better battery life**: No periodic wakeups on mobile/laptop
- **Scalable**: Handles many USB devices without N*polling_rate overhead

### Pike C API Available

Pike provides comprehensive fd monitoring via `backend.h`:

```c
struct fd_callback_box {
    struct Backend_struct *backend;
    struct object *ref_obj;
    struct fd_callback_box *next;
    int fd;                    // File descriptor to monitor
    int events;                // PIKE_BIT_FD_READ | PIKE_BIT_FD_WRITE
    int revents;               // Active events
    fd_box_callback callback;  // Function to call on event
    ...
};

// Register fd with backend
void hook_fd_callback_box(struct fd_callback_box *box);

// Unregister fd
void unhook_fd_callback_box(struct fd_callback_box *box);
```

### Implementation Sketch

```c
// Per-fd tracking node
typedef struct usb_pollfd_node {
    struct fd_callback_box box;
    libusb_context *ctx;
    struct usb_pollfd_node *next;
} usb_pollfd_node;

// Add to ctx_node:
// struct usb_pollfd_node *pollfds;

// fd callback - called when fd has events
static int usb_fd_callback(struct fd_callback_box *box, int event) {
    usb_pollfd_node *node = (usb_pollfd_node *)box;
    struct timeval tv = {0, 0};
    libusb_handle_events_timeout(node->ctx, &tv);
    return 0;  // Keep registered
}

// libusb callbacks
static void pollfd_added_cb(int fd, short events, void *user_data) {
    // Allocate usb_pollfd_node
    // Initialize box.fd, box.events (convert POLLIN->PIKE_BIT_FD_READ)
    // hook_fd_callback_box(&node->box)
}

static void pollfd_removed_cb(int fd, void *user_data) {
    // Find node
    // unhook_fd_callback_box(&node->box)
    // Free node
}

// During register_context():
// libusb_set_pollfd_notifiers(ctx, pollfd_added_cb, pollfd_removed_cb, ctx_node);
// const struct libusb_pollfd **fds = libusb_get_pollfds(ctx);
// for each fd: pollfd_added_cb(fds[i]->fd, fds[i]->events, ctx_node);
```

### ✅ ISSUE RESOLVED - The Inotify Solution

**Root Cause**: Improper fd_callback_box initialization

**The Problem**:
I was manually initializing fd_callback_box fields:
```c
node->box.backend = NULL;  // WRONG! Caused segfault
node->box.ref_obj = NULL;
node->box.fd = fd;
// ... manual assignments
hook_fd_callback_box(&node->box);  // Crashed here
```

**The Solution** (from Pike's Inotify module):
Use the `INIT_FD_CALLBACK_BOX` macro with `default_backend`:
```c
INIT_FD_CALLBACK_BOX(&node->box,
                     default_backend,  // CRITICAL: Use default_backend, NOT NULL
                     NULL,             // ref_obj
                     fd,
                     pike_events,
                     usb_fd_callback,
                     0);
hook_fd_callback_box(&node->box);  // Now works perfectly
```

**Why it crashed**:
- Setting `backend = NULL` caused Pike's `hook_fd_callback_box()` to dereference NULL
- The macro properly sets up internal pointers, reference counting, and linked lists
- Manual initialization skipped critical internal bookkeeping

**Cleanup Pattern** (also from Inotify):
```c
set_fd_callback_events(&box, 0, 0);  // Disable events first
change_fd_for_box(&box, -1);          // Invalidate fd
unhook_fd_callback_box(&box);         // Then unhook safely
```

**Implementation Verified**:
✅ All 31 tests pass
✅ No segfaults
✅ Enable/disable cycles work perfectly
✅ Multi-instance support confirmed

### Current Status

**✅ Fully Implemented**: Event-driven pollfd integration (changeset 23)  
**✅ Working**: Dynamic timeout optimization (changeset 21)  
**✅ Pattern**: Inotify module's proven fd_callback_box approach

### References

- Pike backend.h: `/opt/homebrew/Cellar/pike/8.0.1956/libexec/include/pike/backend.h`
- libusb pollfd API: `libusb_get_pollfds()`, `libusb_set_pollfd_notifiers()`
- GTK2 reference: `src/post_modules/GTK2/source/global.pre`

### Architecture

**Hybrid Approach**: Event-driven + Timeout-driven

The implementation uses both mechanisms:
1. **Event-driven (pollfd)**: Wakes when USB I/O is ready on registered fds
2. **Timeout-driven (periodic)**: Ensures transfers timeout correctly via dynamic scheduling

This hybrid provides:
- Instant response to I/O readiness (no polling delay)
- Correct timeout handling (via libusb_get_next_timeout)
- Zero CPU when truly idle
- Safety net if pollfd doesn't cover all event types

### Performance Impact

**Before** (periodic only): ~50 wakeups/second when active (20ms fixed)  
**After** (event + timeout): Wakes only on actual I/O + timeout checks  
**Idle CPU**: Near zero (only wakes for pending transfer timeouts)  
**Latency**: Immediate (vs 1-20ms polling delay)

### Lessons Learned

**ALWAYS use Pike's initialization macros** - they're not optional conveniences:
- `INIT_FD_CALLBACK_BOX` - Mandatory for fd_callback_box
- Sets up critical internal state (backend pointer, lists, refcounts)
- Manual field assignment causes segfaults and undefined behavior

**Study existing Pike modules** - Inotify, _Stdio, GTK2 provide patterns:
- Different modules use different approaches (GTK2: periodic, Inotify: fd-driven)
- Both are valid; choice depends on external library's event model
- USB benefits from hybrid approach (like modern async I/O patterns)

---

## IOKit Integration for macOS (Future Enhancement)

### Status: FUTURE INVESTIGATION

### Problem

macOS + libusb has limitations with CDC devices:
- Kernel driver reattachment often fails
- Device can disappear from USB bus after use
- Requires unplug/replug to recover
- Native `AppleUSBCDC` driver conflicts with libusb

### Goal

Investigate using macOS's native IOKit framework for CDC device communication
on macOS, while keeping libusb for Linux/cross-platform support.

### Benefits

- **Reliable on macOS**: No kernel driver detachment needed
- **Native integration**: Works with macOS USB stack properly
- **No device loss**: Devices remain available after use
- **Better permissions**: May not require sudo for some operations

### Approach

**Option 1: Hybrid Module**
- Detect platform at runtime
- Use IOKit on macOS, libusb on Linux/others
- Provide unified Pike API regardless of backend

**Option 2: Separate Module**
- Public.USB.IOKit for macOS-specific implementation
- Users choose based on platform needs
- Each module optimized for its platform

**Option 3: Conditional Compilation**
- `#ifdef __APPLE__` in Pike C module
- Compile with IOKit on macOS, libusb on Linux
- Single binary per platform

### IOKit API Research Needed

- IOKit USB device enumeration
- IOUSBDeviceInterface for device access
- IOUSBInterfaceInterface for interface claiming
- CFRunLoop integration for async I/O
- Comparison with libusb API patterns

### IOKit API Overview

Based on initial research, IOKit provides user-space USB access via:

**Device Interface**: `IOUSBDeviceInterface`
- Device enumeration via `IOServiceMatching(kIOUSBDeviceClassName)`
- Get device properties (vendor ID, product ID, etc.)
- Open/close device

**Interface Interface**: `IOUSBInterfaceInterface`  
- Claim/release USB interfaces
- Find endpoints (pipes in IOKit terminology)
- Bulk, interrupt, isochronous transfers
- No kernel driver detachment needed (IOKit handles it)

**Key Difference from libusb**:
- IOKit: Uses Core Foundation patterns (CF types, iterators, plugin architecture)
- libusb: Standard C API with explicit handle management
- IOKit: Native macOS, no driver conflicts
- libusb: Cross-platform, but problematic on macOS for CDC

### Implementation Strategy

**Recommended Approach**: Conditional compilation
```c
#ifdef __APPLE__
    // IOKit-based implementation
    #include 
    // Use IOServiceMatching, IOUSBInterfaceInterface
#else
    // libusb-based implementation  
    #include 
    // Current code
#endif
```

**Challenges**:
- Different API patterns (CF vs C)
- Plugin architecture vs direct handles
- RunLoop integration for async I/O
- More complex but more reliable on macOS

### Code Examples Found

1. **SimpleIOUSB** (github.com/jonpalmisc/simpleiousb)
   - Single-file C library abstracting IOKit USB
   - Provides libusb-like API on top of IOKit
   - BSD-3-Clause license
   - Could serve as reference or dependency

2. **Apple IOUSBFamily Examples**
   - opensource-apple/IOUSBFamily repository
   - USBSimple Example shows basic patterns
   - Official Apple code (though old)

3. **Apple Developer Documentation**
   - "Working With USB Device Interfaces" guide
   - Covers IOUSBDeviceInterface and IOUSBInterfaceInterface
   - Available in Apple's documentation archive

### Effort Estimate

**Moderate** - 2-3 days for experienced IOKit developer:
- Day 1: IOKit device enumeration and interface claiming
- Day 2: Bulk transfer implementation and error handling
- Day 3: Testing, integration, documentation

**Learning curve**: High if unfamiliar with Core Foundation patterns

### References

- Apple IOKit USB documentation: https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/USBBook/
- IOUSBLib.h: `/System/Library/Frameworks/IOKit.framework/Headers/usb/IOUSBLib.h`
- SimpleIOUSB library: https://github.com/jonpalmisc/simpleiousb
- Apple IOUSBFamily examples: https://github.com/opensource-apple/IOUSBFamily

### Priority

**Low** - Current workaround (--no-reattach flag) is acceptable for examples and testing.

**Consider for**: Production macOS applications requiring reliable CDC communication without device loss issues.

### Alternative Workaround (Current)

For now, users can:
- Use `--no-reattach` flag (skip driver reattachment)
- Accept need to unplug/replug device after use (with reattachment)
- Use Linux for development/testing (libusb more reliable)
- macOS device communication works fine, only cleanup is problematic