Python interop: call Python from Osprey, and Osprey from Python

Osprey goes all in on Python. Call it Python interopmaxxing: keep your Python libraries and habits, and add a compiled, statically typed language whose algebraic effects make it easy to test. If your work runs on NumPy, pandas, PyTorch, scikit-learn or SciPy, you keep them.

There are three ways in:

1. Write Python that is Osprey. Much ordinary Python compiles as the Python flavor, and NumPy, SciPy, PyTorch, Transformers and more are already ported to it.
2. Call Python from Osprey. A Python effect calls into CPython, so Osprey can use any Python library.
3. Call Osprey from Python. Compile Osprey to a native library and import osprey, without writing C.

A lot of Python is already Osprey

With # osprey: flavor=python at the top, this file prints the same three lines under python3 and osprey --run:

# osprey: flavor=python

def label(score):
    match score:
        case 0:
            return "none"
        case _:
            return "high" if score > 50 else "low"


def report(name, score):
    return f"{name}: {score} is {label(score)}"


print(f"{report("alice", 0)}\n{report("bob", 12)}\n{report("carol", 80)}")

alice: 0 is none
bob: 12 is low
carol: 80 is high

Osprey checks the types before it runs, then compiles natively. It isn't CPython: field mutation and exceptions are rejected, failures are Result values, and imports name Osprey modules. See how Python flavor differs from Python. A flavor converter is planned.

Call Python from Osprey (embed a Python library)

Python is a capability, so it is an effect. Code performs Python.call(module, function, jsonArgs); the live handler sends it to CPython, and a test handler answers without Python:

// Python is a replaceable capability: live code and tests share the same call.
// Implements [PYTHON-CALL]; run using `python -m osprey`, not the native target.
extern fn osprey_python_call(moduleName: string, name: string, args: string) -> string
extern fn osprey_python_object(operation: string, objectId: int, name: string, args: string) -> string

effect Python { call: fn(string, string, string) -> Result<string, string> }

fn decodedValue(doc) = match jsonGet(doc, "ok") {
    Success { value } => Success { value: value }
    Error => Error { message: jsonGet(doc, "error.message") ?: "invalid Python reply" }
}

fn decode(reply) = match jsonParse(reply) {
    Success { value } => {
        let result = decodedValue(value)
        let released = jsonFree(value)
        match released {
            Success(_value) => result
            Error => Error { message: "could not release Python reply document" }
        }
    }
    Error => Error { message: "invalid Python reply JSON" }
}

let live = handler Python { call moduleName name args => decode(osprey_python_call(moduleName, name, args)) }
let fake = handler Python { call _ _ _ => Success { value: "test answer" } }

fn squareRoot(value: float) = perform Python.call("math", "sqrt", "[${toString(value)}]")

fn squareRootText(value) = match live(fn() => squareRoot(value)) {
    Success { value } => value
    Error { message } => "error: ${message}"
}

fn mockedSquareRoot() = match fake(fn() => squareRoot(81.0)) {
    Success { value } => value
    Error { message } => "error: ${message}"
}

// Returns a result-object handle envelope; the tensor payload stays in Python.
fn scaleTensor(objectId) = osprey_python_object("call", objectId, "mul", "{\"args\":[{\"value\":2.0}]}")

fn main() = print("live: ${squareRootText(81.0)}; mock: ${mockedSquareRoot()}")

squareRootText runs squareRoot under live; mockedSquareRoot runs it under fake, a test double. Python exceptions come back as Error values. The program runs inside a Python process, started as shown next.

Call Osprey from Python (native speed, no C extension)

Compile Osprey into a native library (.so on Linux):

python3 -m pip install -e src/toolchain/python
osprey src/osprey/examples/python/interop.osp --compile --target=python --memory=arc -o interop.dylib

Then load it from Python:

import osprey

library = osprey.load("./interop.dylib")
print(library.squareRootText(81.0))
print(library.mockedSquareRoot())

9.0
test answer

Calls flow both ways: squareRootText calls math.sqrt back in the same process. osprey.build compiles and loads in one step; python3 -m osprey runs a file.

The Python-flavor version uses typed @extern signatures:

# osprey: flavor=python
# Osprey source: run with Osprey's compiler or Python's osprey.build, not CPython.

@extern
def osprey_python_call(module: str, name: str, args: str) -> str:
    ...


@extern
def osprey_python_object(operation: str, object_id: int, name: str, args: str) -> str:
    ...


def greeting(name: str):
    return f"hi {name}"


def square_root_json(value: float):
    return osprey_python_call("math", "sqrt", f"[{value}]")


def scale_tensor_json(object_id):
    return osprey_python_object("call", object_id, "mul", '{"args":[{"value":2.0}]}')

Real use: an ML data pipeline

In LLM dataset audit, Python loads datasets and tokenizers and native Osprey decides the policy for every record. It has run on all 15,011 Dolly records with the GPT-2 tokenizer.

Speed: in Codon's range

Codon also compiles Python-like code natively. On the benchmarks, Osprey with ARC takes 0.96× Codon's time (geometric mean) and is faster on 18 of 24 programs. Of the rest, four are up to 2× slower and two allocation-heavy sorts are 11–20× slower. Codon ran on a busy machine, so this is approximate. Performance improves all the time.

Limits

• Linux and macOS today; Windows coming soon. Exported functions take and return scalars, strings and JSON text.
• One loading thread, the GIL is held, and the library uses ARC.
• Objects from CPython libraries stay in Python, as handles from library.keep(obj) that you release(id). The ported packages, including @osprey/torch and @osprey/transformers, are pure Osprey and never cross this bridge.
• These calls cross Osprey's C memory-safety boundary.

The Python interop specification defines the exact protocol. Run the integration tests with cargo test --manifest-path src/toolchain/Cargo.toml -p osprey-cli --test python_interop.