Secure, locally deployed LLMs and the governance around them
Delivered language-model workflow control in environments where no commercial API was allowed, with access, segregation, and oversight controls built in from the start.
- Organization
- ORBIS, Inc. · DoD portfolio
- My role
- Director of Artificial Intelligence
- Period · Focus
- 2023 – Jan 2026 · Governance, Platforms
In brief
Defense programs could not send data to a commercial model API, which ruled out most generative AI as it is usually sold. I designed, trained, and deployed language models that ran entirely inside secure local environments to drive workflow control, and established the governance around them: secure model-access patterns, segregated AI environments, human oversight in AI-enabled workflows, and risk-based deployment criteria. The outcome evidence is operational and qualitative; no public metrics are available.
Context and challenge
Teams were told AI was off the table for security reasons. The real constraint was narrower: no data egress. The question was how much useful language-model work could run inside a boundary the organization controlled, and under what controls.
My mandate and scope
- Designed, trained, and deployed locally secured LLMs for workflow control, on-premises end to end.
- Established governance and risk-based deployment practice across the DoD portfolio: secure model-access patterns, segregated environments, human oversight, and controlled access to AI capability.
Key decisions and why
- 1
Design from the boundary inward
Where embeddings live, what gets logged, who can query what, and which data may share an index were settled before any workflow was built. Building first and presenting to security review afterward reopens every component the review rejects.
- 2
Size the model to hardware the organization controls and invest in retrieval
On grounded tasks, output quality tracks the relevance of retrieved context more closely than model scale, and retrieval is fully under local control. Reindexing a corpus takes hours; new inference hardware takes a budget cycle.
- 3
Start with bounded tasks and name the ones that need frontier scale
Retrieval over internal documents, structured extraction, and workflow control have narrow inputs and checkable outputs. Identifying the tasks that genuinely need frontier-scale capability before committing to an architecture prevents an expensive discovery later.
What changed
- AI-enabled workflows operated inside the boundary with human oversight at defined decision points.
- Access to model capability became governed and segregated by environment.
Outcome and evidence
Outcomes here are delivered milestones and operational results. No public metrics are available.
- Language models deployed and operating entirely inside secure local environments.
- Governance practice (model access, environment segregation, oversight, risk-based deployment) established and applied across the program portfolio.
What to expect from me
I translate a blanket security objection into the specific requirement underneath it and then design to that requirement. Governance works best as part of the architecture, decided before the first workflow ships.
Technical perspective
- Retrieval-augmented generation and prompt orchestration over sensitive corpora; agentic patterns for workflow control without third-party exposure.
- Operations, monitoring, and hardware lifecycle are owned locally, and so is evaluation; no vendor measures output quality for you.
- Model training ran on-premises end to end.
Artifacts
Confidentiality: Environments, data, and workflow specifics are controlled. This account is limited to approach and publicly shareable outcomes.