#opg $OPG runs two different inference node types under one roof. LLM Proxy Nodes are housed within a TEE enclave and route requests to external providers like OpenAI and Anthropic. Node operators can't see your prompts, can't log your responses, and the entire interaction is cryptographically signed before it leaves. Local Inference Nodes are different. They run open-source models directly on GPU hardware from the Model Hub, with verification ranging from full ZKML proofs to lightweight signatures depending on what the use case actually requires.
What stops me is what this separation means in practice. When you call GPT-4 through a centralized API today, the provider sees everything. Through the LLM Proxy Node, the TEE enclave is the only entity touching your data, and the operators running the hardware are locked out by design.
This silently changes things. Node operators earn fees for running infrastructure they literally can't read.
The part I keep mulling over is the routing question. When a request comes in, what determines which type of node handles it? And if LLM proxy nodes distribute requests across multiple nodes for anonymity, does that distribution introduce latency or consistency issues that need to be addressed?
If node operators can't see your data but still profit from processing it, what does that model look like at scale as the network grows?
What stops me is what this separation means in practice. When you call GPT-4 through a centralized API today, the provider sees everything. Through the LLM Proxy Node, the TEE enclave is the only entity touching your data, and the operators running the hardware are locked out by design.
This silently changes things. Node operators earn fees for running infrastructure they literally can't read.
The part I keep mulling over is the routing question. When a request comes in, what determines which type of node handles it? And if LLM proxy nodes distribute requests across multiple nodes for anonymity, does that distribution introduce latency or consistency issues that need to be addressed?
If node operators can't see your data but still profit from processing it, what does that model look like at scale as the network grows?