MDC Platform

Installation

git clone https://github.com/saji1970/ModelDB.git
cd ModelDB/mdc

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Two ways to run it:

python -m mdc db init seeds the full spec-minimum synthetic payments dataset (10,000 merchants / 50,000 customers / 500,000 transactions, deterministic, seed 42) if you want the larger analytics dataset rather than the small dev seed.

Core concepts

MDC is two systems sharing one storage layer, kept deliberately separate because they solve different problems:

• The polymorphic object store - classifies any uploaded bytes (a model checkpoint, an image, a document, a log file) by real content signature, not filename, and routes it through a tiering decision. This is what § AI models and § Other objects cover.
• The database layer - unlimited named databases, each with its own schema-validated tables, created and queried entirely through chat or the CLI. This is what § The database layer covers.

Both are driven by the same principle: an LLM proposes an interpretation, but a deterministic validator - never the LLM itself - decides what operation actually reaches storage. No natural-language command generates raw SQL.

Nobody on your team has to learn SQL

Every example on this page - filtering by a threshold, scoping to a specific database, searching across tables - is plain sentences, not query syntax. That's not a simplified subset for beginners standing next to "the real interface"; it is the interface. Support staff, analysts, and anyone else who needs an answer from the data can just ask for it, with no SELECT/WHERE/JOIN training required, and nothing to get subtly wrong the way a hand-written query can.

Storing & retrieving AI models

Upload a .safetensors file and MDC parses its real header, splits each tensor into row-aligned blocks, and stores every tensor as its own independently addressable, independently tiered object - not one opaque blob.

curl -X POST "http://127.0.0.1:8000/objects?filename=model.safetensors" \
  --data-binary @model.safetensors
# {"object_id":"AIM-58F0BB747C","type":"AI_MODEL","tensor_count":2,"total_parameters":16}

curl http://127.0.0.1:8000/models/AIM-58F0BB747C
# manifest: tensor_count, total_parameters, dtype, checksum, ...

curl http://127.0.0.1:8000/models/AIM-58F0BB747C/tensors/linear.weight
# raw bytes of just that one tensor - the rest of the model is never touched

The same thing through chat:

move on a model cascades to every one of its tensor blocks - each tensor gets its own independent tier decision at upload time, so moving just the model's manifest would silently leave the actual weights wherever they were.

Storing other object types

The same upload endpoint handles images (real dimension parsing from file headers), documents (text extraction + chunking), logs, tensors, and tabular files - classified from content, never from a file extension.

Every object - model, image, document, or table row - carries a SHA-256 checksum verified on every read. A tampered or corrupted read never silently returns bad bytes; it fails with a clear integrity error instead.

Storage tiers

Tier is a real consequence, not a label - only HOT maps to genuinely different physical storage (an in-memory backend) today; ARCHIVE is the one exception that goes further still, actually encoding bytes to simulated DNA base sequences (2 bits per base, A/C/G/T - a real, working prototype encoder, not physical synthesis).

The core concept: 2 bits per base

Every byte in the ARCHIVE tier is mapped directly to 4 DNA bases - the same information-density idea real DNA-storage research is built on:

00 → A      01 → C      10 → G      11 → T

A real, working error-correction scheme (N-way repetition with majority voting across independently-corrupted copies) and a seeded corruption simulator sit on top of the encoding - modeling substitution, insertion, deletion, and dropout errors the way real DNA storage research does. This is a simulation: a string of ACGT characters in a database, not a molecule - no physical synthesis or sequencing anywhere in this codebase.

Every payload is AES-256-GCM encrypted before it's DNA-encoded - the ACGT sequence is ciphertext-as-base-letters, not plaintext-as-base-letters, so it's unreadable without the key this backend holds. The mapping above is public (you're reading it), so encoding plaintext directly would only be obfuscation; encryption is what actually makes the archive tier unreadable outside MDC's own API/CLI. mdc-lite stores every entry the same way.

Read the full whitepaper → for the error-correction design, the corruption model, and an honest, specific answer to "does this use quantum encryption?" (short version: no - and here's what quantum computing actually does and doesn't threaten about long-retention archival data).

Phase 2 (planned, not yet built): extending DNA-tier features further on the platform side - stronger ECC, a Windows CLI bridge that can open an mdc-lite store directly when a phone/wearable is connected, and research into applying the same tier beyond archival storage.

Multiple databases

No hardcoded limit on how many databases exist - each is a fully isolated DuckDB file with its own schema registry, created lazily the moment you ask for it, entirely from chat:

Tables & queries

Read another database's table without switching to it, using an explicit qualifier:

show data in products in database inventory
describe table products in database inventory

Table creation only ever produces a validated, typed SchemaRegistry collection - never raw SQL DDL generated from chat text, the same injection-avoidance principle as everything else in MDC.

Universal search

find searches every database's tables and documents at once, with deterministic clause extraction for amount constraints and explicit scoping - no need to know which database or table something lives in:

REST API reference

Every route below requires Authorization: Bearer <token> - see Connect your own NLU or UI for how to issue one.

Chat command reference

Connect your own NLU or UI

MDC's own chat layer doesn't have to be the only front door. Two integration patterns, depending on whether your system has already done its own language understanding by the time it talks to MDC:

Authenticate first — every route but the built-in UI needs a token

CORS is deliberately wide open (allow_origins=["*"]) so a third-party integration can call this API from a different origin without a browser rejecting the response first - that's the point of everything on this page. Open CORS was never itself a security boundary against a non-browser client though, so every route except the built-in browser UI's own HTML page requires a bearer token:

$ mdc token issue rasa-integration --role editor
mdc_kP3x...   (shown once - store it now)

Send it as Authorization: Bearer <token> on every request. Revoke it later with mdc token revoke rasa-integration, or set MDC_API_TOKENS (comma-separated) to accept fixed tokens without the on-disk store, for CI/deployment.

--role scopes what that specific token can do - omit it and a token defaults to admin (full access, matching the original all-tokens-are-equal behavior), or pick something narrower for an integration that shouldn't have it:

A token with an insufficient role still authenticates - a valid-but-underprivileged token gets 403, not the 401 reserved for a missing or invalid token.

User accounts and roles (CLI)

Separately from API tokens, the CLI itself now has real accounts. The first time you run mdc or mdc serve with no user configured yet, it interactively asks for an admin username and password and creates that first account as admin - the same four roles as tokens, since both share one vocabulary. Manage accounts afterward with:

$ mdc user create alice --role editor
Password: ********
Repeat for confirmation: ********
Created user alice with role editor.

$ mdc user list
  admin  (admin)
  alice  (editor)

$ mdc user set-role alice db_admin
$ mdc user delete alice

Passwords are never stored in a recoverable form - hashlib.scrypt derives a key from the password and a random per-user salt, and only that salt and derived key are persisted (mode 0600), the same reasoning already used for API tokens.

Pattern 1 — pass-through (simplest, works today, zero new code)

Forward raw user text to POST /chat and let MDC's own deterministic parser do the work - the same pipeline the browser chat panel and CLI already use. This is the right choice for most integrations: your system doesn't need to understand databases, models, or storage tiers at all, just detect "this is a data question" and relay the text.

# A RASA custom action (actions.py) - the entire integration
import requests
from rasa_sdk import Action

class ActionQueryMDC(Action):
    def name(self):
        return "action_query_mdc"

    def run(self, dispatcher, tracker, domain):
        reply = requests.post(
            "http://localhost:8000/chat",
            json={"message": tracker.latest_message.get("text"), "session_id": tracker.sender_id},
            headers={"Authorization": "Bearer mdc_kP3x..."},
        ).json()
        dispatcher.utter_message(text=reply["message"])
        return []

Note session_id: tracker.sender_id - reusing RASA's own conversation id keeps MDC's per-session state (which database you switched to, pronoun resolution) aligned with RASA's own multi-turn state, so "switch to database inventory" followed later by "show data in products" behaves correctly across turns.

Pattern 2 — structured (for a system that already extracted intent)

If your NLU has already parsed "add Widget at $9.99 to inventory" into its own slots, reconstructing MDC's sentence syntax from those slots is unnecessary indirection. Call the structured REST endpoints directly with the extracted values instead:

# RASA custom action calling the structured endpoint directly,
# using slots RASA's own NLU already extracted
requests.post(
    f"http://localhost:8000/databases/inventory/tables/products/rows",
    json={"values": {"name": tracker.get_slot("item_name"), "price": tracker.get_slot("price")}},
    headers={"Authorization": "Bearer mdc_kP3x..."},
)

Every structured endpoint runs through the exact same validation as the chat/CLI path it mirrors (schema-registry-only table creation, typed field coercion on insert) - there's no separate, less-checked code path for programmatic callers.

A custom "ChatGPT-style" UI

A minimal chat interface is one fetch() call - the same POST /chat endpoint the Storage Explorer's own chat panel uses:

async function ask(message) {
  const res = await fetch("http://localhost:8000/chat", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "Authorization": "Bearer " + MDC_TOKEN,
    },
    body: JSON.stringify({ message, session_id: crypto.randomUUID() }),
  });
  const { message: reply, data } = await res.json();
  return { reply, data };  // data is a table's rows / a search result list, when there is one
}