Security model for AI-generated programs

Jo's static capability system enforces declared least-privilege boundaries at compile time — before any code runs.

Threat model

Attacker controls prompts and attempts to make generated code exceed authorized behavior.

The trust boundary is the platform capability interface for untrusted programs. LLM-generated code is untrusted.

  • Access other users' data
  • Escalate privileges beyond granted permissions
  • Exfiltrate data to external endpoints

Compile-time guarantees

Jo's capability model builds on object-capability security research with a long academic history, from Lampson's confinement problem (1973) to Mark Miller's E language, David Wagner et al's Joe-E, and recent research on contextual capabilities.

No unauthorized I/O

All side effects require explicit capabilities. LLM-generated code cannot access filesystem, network, or other resources without a declared capability.

No capability amplification

Attenuated capabilities cannot recover broader access. A user-scoped DB interface cannot be downcast to full DB access.

No language-level escape hatches

No reflection, no ambient state, no FFI in untrusted code. The type system enforces the security boundary.

Timing side-channel attacks

Spectre-class attacks exploit speculative execution to read memory across a privilege boundary via cache timing.

Jo's capability system provides meaningful protection here: the attack requires a high-resolution timer (or performance counters) to measure cache access times, and these are capabilities that must be explicitly granted to LLM-generated code. Without timer access, the timing channel cannot be read and the attack cannot proceed.

Most processor vendors and operating systems have also shipped firmware updates and kernel mitigations that significantly reduce the attack surface.

How Spectre attacks work

The CPU speculatively executes instructions ahead of branch resolution to improve performance. The attack uses a bounds-check guard as the target: by repeatedly calling with valid indices, the attacker trains the branch predictor to expect index < array.length to be true. Then passing an out-of-bounds index causes the CPU to speculatively execute the body — reading beyond the array into memory which should be inaccessible — before the check resolves and the branch is rolled back.

The CPU rolls back before the program ever sees the value. But before the speculative read, the attacker sets up a 256-page probe array. The speculative path uses the secret byte as an index into it:

if index < array.length:          # guard — attacker mistrains branch predictor to predict true
    secret = array[index]          # out-of-bounds, speculative — rolled back
    _ = probeArray[secret * 4096]  # encodes secret into which cache line is warmed

After rollback, secret is gone from any accessible variable. But one of the 256 probe pages is now in cache. The attacker times all 256 accesses — the fast one reveals the byte:

for i in 0..256:
    t0 = performance.now()
    _ = probeArray[i * 4096]
    t1 = performance.now()
    if t1 - t0 is fast: secret = i   # cache hit — this i is the secret byte

The value is never read directly — it is inferred entirely through which memory access is fast. This is the side channel.

How this differs from runtime isolation

Runtime isolation restricts environments. Jo restricts what generated programs are allowed to express and use, with violations caught at compile time.

Approach File system
protection
Compile-time
enforcement
Credentials
protection
User-scoped
data access
Type-based
security
cgroups/seccomp
VMs/microVMs
Deno
WASM + WASI ✓(link-time)
Jo

What the guarantees depend on

Jo's compile-time guarantees hold under the following assumptions. Understanding these is necessary for correct deployment.

Correct type checker

The guarantees are as strong as the Jo type checker. Jo is open source and tested with an adversarial test suite that includes cases designed to violate capability contracts. Compiler bugs would weaken the guarantees.

Reviewed capability interface implementations

Trusted code that implements capability interfaces must correctly encode the intended access control rules — for example, scoping database access to the current user. Jo enforces the boundary at the interface; correctness of what is inside that boundary is a code review responsibility.

Correct deployment controls

Jo verifies that generated code only uses declared capabilities. Infrastructure controls — network policies, resource limits, access credentials — remain the platform operator's responsibility and complement Jo's compile-time enforcement.

What this does not guarantee

Security guarantees assume correct capability interface implementations and correct deployment controls.

Resource exhaustion

Jo does not prevent long-running programs or memory overuse. Use infrastructure-level controls (timeouts, memory limits) alongside Jo for deep defense if needed.

Implementation bug

Capability boundaries are only as strong as the interface implementations. The implementation of business access control rules requires security review.

Prompt injection misuse

A manipulated prompt may cause generated code to misuse a granted capability within its declared scope. Jo enforces capability boundaries; it cannot enforce correct intent.

Responsible disclosure

To report a security vulnerability in Jo or TypeScope, use GitHub's private vulnerability reporting at github.com/typescope/jo/security.

Need a buyer-ready security walkthrough?