Skip to content
Noy Oliel
esc
  • Home
  • All systems
  • Writing
  • Resume (printable)
  • AI assistant for a security SaaS
  • MCP server and OAuth for outside AI clients
  • SARA: agentic software factory
  • AI development toolkit for the team
  • In-house graph engine
  • Copy email addressoliel.noy@gmail.com
  • Open LinkedIn
  • Book an intro call
  • Signal
  • Overview
  • Timeline
  • Architecture
  • Inside
  • Hard problems
  • Decisions
  • Outcomes
  • VoidDefault. Near-black with signal blue.
  • DaylightLight, for bright rooms and printing.
  • ContrastHigh contrast, 7:1 text. Best on TVs.
  • TerminalPhosphor green on black.
  • InfraredWarm graphite with infrared orange.
  • Room: team review on a large screen
  • Desk: one reviewer
  • Calm: stop ambient motion
  • Full motion

00Case study · Feb 2026 – present

AI development toolkit for the team

The team’s shared setup for AI coding agents: skills, role agents, hooks and three custom MCP servers.

LeadInternal tool

My roleLead: designed and built the workspace, its agents, skills, hooks and MCP servers. Teammates added a few skills of their own.

  • 20 agent definitions
  • 67 skills in 13 plugins
  • 3 MCP servers
  • 9 hooks

01Overview

What it is, and why

I built the shared workspace the development team uses to work with AI coding agents (Claude Code, with basic Codex support), so every developer starts with the same codebase knowledge, workflows and safety rails. The whole team adopted it at first; it later branched into environment-specific setups, and about four developers still use the original.

Instead of one giant prompt, knowledge lives in skills that agents load on demand, and one planning session delegates the work to role agents in a fixed order, with review gates.

Two needs came later: memory that carries corrections from one session to the next, and a way for several agents to test real backend code at the same time without sharing one cloud account.

At a glance

How it works, in short

  • Knowledge as skills: a catalog of skills per repository, technology, process and operation, which agents load on demand instead of carrying everything in their prompt.
  • skill-discovery (BM25), web-crawl (documentation RAG) and persistent memory on an extended open-source server.
  • A conductor workflow: a planning session delegates to role agents (investigator, architect, developer, reviewers, tester) in a fixed order with review gates.
  • Hooks recall memories at session start, capture the user’s corrections, and block destructive commands and pushes to protected branches.
  • Claude Code
  • MCP
  • BM25
  • RAG
  • Docker

02Timeline

How it came together

Feb 2026 to present.

  1. Feb 2026

    The workspace, the conductor and its agents; skill-discovery.

  2. Mar 2026

    web-crawl and the memory server.

  3. Jul 2026

    Memory through the Claude CLI, for subscription sign-in.

  4. Aug 2026

    Safety hooks; basic Codex support.

  5. Sep – Oct 2026

    Per-agent AWS environments on floci.

03Architecture

How the parts fit

A conductor session plans and delegates; role agents pull skills, documentation and memory through three MCP servers, and test against their own AWS sandbox.

toolkit · one session

  • Developer

    • Conductor sessionPlans and delegates; never writes the code
  • Agents and hooks

    • Role agentsInvestigator, architect, developer, reviewers, tester
    • HooksMemory recall and capture, safety guards
  • MCP servers and sandboxes

    • skill-discoveryBM25 over the skill catalog
    • web-crawlDocumentation, searched by meaning
    • MemoryFacts and preferences across sessions
    • AWS sandboxOne per agent, on the floci emulator
  1. Conductor session to Role agentsDelegates in a fixed order, with review gates and send-back
  2. Conductor session to HooksSession events: start, prompt, tool use, stop
  3. Role agents to skill-discoverySearch and load the skills a task needs
  4. Role agents to web-crawlSearch crawled documentation
  5. Hooks to MemoryRecall at session start; store the user’s corrections
  6. Role agents to AWS sandboxDeploy and test real backend code, in isolation
Point at a box or a flow to trace it. The numbers match the list.

04Inside the system

The parts that matter

  • 01

    Knowledge as skills

    • 67 skills in 13 plugins, 62 of them mine: per repository, technology, process and operation.
    • Skills inherit: a repository skill pulls in its technology skills.
    • skill-discovery ranks them with BM25 and reloads when skill files change.
  • 02

    A conductor, not one long context

    • 20 agent definitions, run as Define, Staff and Execute: investigate and clarify, give each agent its skill manifest, then build with forward flow and send-back.
    • Requirements and functionality reviews run in parallel, then flows review, testing and refinement.
    • Sub-agents start blank, so every spawn prompt carries its skills and constraints.
  • 03

    Memory across sessions

    • Extends an open-source memory server (its core is not mine): a Claude-subscription provider and local embeddings, so no extra API keys.
    • A 16-tool MCP server and 6 lifecycle hooks: recall at session start, capture corrections, record failed commands that were later fixed.
    • A consolidation agent merges duplicates and promotes repeated lessons into skills.
  • 04

    Safety rails

    • Hooks keep rm inside the workspace and block pushes to protected branches, with explicit flags that turn a deny into a question for a person.
    • Evidence grading: every bug an agent reports is marked confirmed (reproduced) or a conclusion (inferred).
  • 05

    Parallel agents, real backends

    • Tooling on the open-source floci emulator (floci itself is not mine) gives each agent its own disposable copy of the AWS backend.
    • 15 template transform rules and 6 deploy profiles; 121 test files.

05Deep dive

Hard problems

What stood in the way, what I did, and what came of it. Where nothing was measured, it says so.

Hard problem 01

More knowledge than a prompt can hold

Problem
Repository, technology and process knowledge for a large codebase is far more than an agent can carry in its prompt.
What I did
  • Split the knowledge into skills, with frontmatter and inheritance.
  • A BM25 skill-discovery MCP server with 5 tools, reading skills straight from disk.
Result
Each agent loads the repository, technology and process skills its task needs, and nothing else.

Hard problem 02

Several agents deploying at once

Problem
Agents testing real backend code at the same time would collide in one shared cloud account.
What I did
  • One emulator container per agent, each with its own ports, caches and locks.
  • The real infrastructure templates rewritten to run locally: cloud-only networking removed, unsupported resources stubbed.
  • Three planes: real credentials only read templates; deploys and data use fake ones.
Result
Several agents deploy and test on one laptop without touching the shared account. Built in about four weeks.

Hard problem 03

Autonomy without accidents

Problem
Agents with a shell in a monorepo of submodules can delete or push what they should not.
What I did
  • Pre-tool hooks deny rm outside the workspace, and pushes to protected branches, including by refspec, delete or mirror.
  • Escape hatches that turn a deny into a prompt for a person.
Result
A destructive command needs a person, by default.

06Decisions

Trade-offs I made

What I chose, what I passed on, and why.

  • Decision · Context

    On-demand skills, found by search

    OverOne long system prompt

    Agents start small and pull what the task needs; the catalogue can grow without crowding every session.

  • Decision · Memory and emulation

    Extend open-source projects

    OverWrite a memory server and an emulator from scratch

    The cores exist and are maintained. The team needed the integration: subscription auth, hooks and per-agent isolation.

  • Decision · Skill search

    BM25 keyword ranking

    OverAn embedding index

    Skill names and descriptions are short and specific, and keyword ranking needs no model to run.

07Outcomes

What came of it

Measured or owner-stated

Results

  • Adopted by the whole team at first; about 4 developers still use the original (owner-stated).
  • 20 agent definitions, 67 skills in 13 plugins, 3 MCP servers and 9 hooks.

Said plainly

Limits

  • There is no retrieval-quality or productivity benchmark; the claims are about the architecture.