Autofill AI Ninja

Architecture

Content script owns the DOM; the service worker owns credentials and HTTP; the side panel is configuration and progress.

Runtime contexts

Context Entry point Responsibility
Content script src/content/index.ts Scans, applies, validates, navigates, streams progress
Service worker src/background/index.ts Message router; holds keys; builds prompts; all network calls
Side panel src/sidepanel/main.ts Provider config, custom request, run trigger, live log

The page never sees an API key: the content script asks the worker for values and receives only the values back.

System diagram

flowchart TB
  subgraph Page["Host page"]
    DOM[DOM / forms]
  end

  subgraph CS["Content script"]
    Entry[index.ts]
    Orch[fillOrchestrator]
    RunState["runState: step epochs"]
    FieldId["fieldId: content-hash sids"]
    Scan[scan]
    Widgets["widgets: native + aria adapters"]
    Settle["settle: MutationObserver"]
    Validate[validate]
    Nav[navigation]
  end

  subgraph BG["Service worker"]
    Router[message router]
    LLM[llm]
    Adapters["provider adapters"]
    Vault["keyVault: AES-GCM"]
    Store[storage]
  end

  subgraph SP["Side panel"]
    Panel[main.ts]
    Log[progress log]
  end

  Panel -->|RUN_FILL| Entry
  Entry --> Orch
  Orch --> RunState
  Orch --> Scan
  Scan --> FieldId
  Scan --> Widgets
  Widgets --> DOM
  Orch --> Settle
  Orch --> Validate
  Orch --> Nav
  Nav --> DOM
  Orch -->|RUN_PROGRESS| Log
  Orch <-->|LLM_FILL| Router
  Router --> LLM
  LLM --> Adapters
  Router --> Store
  Store --> Vault
  Adapters -->|HTTPS| Cloud[(Provider APIs)]

Fill workflow

sequenceDiagram
  participant U as User
  participant SP as Side panel
  participant CS as Content script
  participant DOM as Page DOM
  participant BG as Service worker
  participant API as Provider

  U->>SP: Fill this page
  SP->>CS: RUN_FILL
  CS->>BG: GET_SETTINGS
  BG-->>CS: settings (no key material)

  loop Each form step
    CS->>CS: beginEpoch, fresh applied map
    CS->>DOM: scan via widget adapters
    CS->>CS: chunk by fieldset / role=group / heading
    loop Each chunk, bounded by maxRounds
      CS->>BG: LLM_FILL(snapshot + run context)
      BG->>API: chat completion
      API-->>BG: JSON values
      BG-->>CS: values
      CS->>CS: validate, resolve option labels
      CS->>DOM: apply via native setters + events
      CS->>DOM: waitForDomQuiet (reveals conditionals)
      CS-->>SP: RUN_PROGRESS
    end
    CS->>DOM: click next, compare field fingerprint
    alt Page reported validation errors
      CS->>BG: LLM_FILL(corrections)
      CS->>DOM: repair, click next again
    end
  end

  CS-->>SP: fields filled, steps completed

Why reactive forms need special handling

1. Frameworks revert direct writes. React installs an instance-level value property. Assigning el.value = x updates that tracker too, so the following input event looks like a no-op. Writes go through the prototype’s setter instead, then the result is verified.

export function setNativeValue(el: HTMLElement, value: string): void {
  const descriptor =
    Object.getOwnPropertyDescriptor(Object.getPrototypeOf(el) as object, "value") ??
    Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value");
  if (descriptor?.set) descriptor.set.call(el, value);
  else (el as HTMLInputElement).value = value;
}

2. DOM nodes are recycled between steps. React often reuses the same <input> elements across wizard steps. Synthetic ids are s{epoch}_{hash}; the epoch increments on every step change so a recycled node cannot collide with a previous step’s applied map. Framework-generated ids (:r3:, mui-1234, radix-…) are excluded from the hash.

3. Option values are not option labels. For <option value="13">Tokyo</option>, prompts carry [value, label] pairs. Resolution matches from strictest to loosest — exact value, exact label, normalized, then unique containment — and refuses ambiguous loose matches.

Project layout

ai-form-filler/
├── manifest.config.ts
├── _locales/{en,de,ja,es,pt,bn}/
├── docs/                     # This site (GitHub Pages)
└── src/
    ├── background/           # Router, LLM, storage, key vault
    ├── content/              # Orchestrator, scan, apply, navigation, widgets
    ├── shared/               # Types, providers, heuristics
    └── sidepanel/            # UI

Development

npm install
npm run build      # tsc --noEmit && vite build
npm run dev        # rebuild on change
npm run check      # typecheck only

Load dist/ via chrome://extensions → Load unpacked.

New widget families: implement match / describe / read / apply in src/content/widgets/ and register in widgets/index.ts. ARIA adapters are registered before native ones.