Llama.CPP Failed to parse Grammar - MCP Regex Fix Proxy

The Problem: When llama.cpp meets Enterprise MCP Schemas

If you are trying to connect a local llama.cpp inference server to a complex, official MCP (Model Context Protocol) server (I was trying the official Apollo.io MCP), you will likely hit a hard crash loop that looks like this:

parse: error parsing grammar: unknown escape at \d"{4,4} "-\d"{2,2} "-\d"{2,2}) "\"" space

Why is this happening? Enterprise MCP servers often expose dozens of tools at once, and their JSON schemas frequently use standard regex patterns (like \d{4}-\d{2}-\d{2}) to validate inputs like dates.

llama.cpp has a built-in C++ engine that attempts to convert these JSON tool schemas into a strict GBNF (Guided Backus-Naur Form) grammar to force the AI to output valid XML/JSON. The problem? GBNF does not understand standard regex escapes. It doesn’t know what \d (digit), \w (word), or \s (whitespace) are, and it instantly panics and crashes the server before the model can generate a single token.

What Didn’t Work

1. Fixing it via Jinja Templates (chat_template.jinja): I tried adding | replace('\\\\d', '[0-9]') filters to the Jinja template. It failed. Why? The C++ grammar compiler intercepts the raw tools array directly from the HTTP POST request body before it ever reaches the Jinja templating phase. I tried this fix on top of the Qwen Fixed Chat Templates which u/ex-arman68 posted about in r/Qwen_AI.
2. Dropping the —jinja flag entirely: I tried letting the model run raw to bypass the grammar. It failed. The llama-server binary automatically triggers the grammar compiler if it detects a tools array in the request, regardless of your template settings.

The Solution: The Regex-Scrubbing Proxy

Since the hardcoded C++ binary was crashing on the raw network request, the only way to fix it without recompiling llama.cpp from source was to intercept the HTTP payload mid-flight.

This tiny Node.js proxy sits between your chat UI (like OpenCode/Open WebUI) and your llama-server. It intercepts the /chat/completions payload, safely translates all incompatible Python/JSON regex escapes into GBNF-friendly character classes (\d becomes [0-9]), and forwards the clean schema to llama.cpp.

The C++ server parses it perfectly, builds the grammar, and the AI runs flawlessly.

The Code (mcp-proxy.js)

const http = require('http');
 
// Parse CLI arguments
const args = process.argv.slice(2);
if (args.length < 2) {
  console.error('❌ Usage: node mcp-proxy.js <LISTEN_PORT> <TARGET_BASE_URL>');
  console.error('💡 Example: node mcp-proxy.js 9010 http://192.168.1.163:9009');
  process.exit(1);
}
 
const PROXY_PORT = parseInt(args[0], 10);
let targetUrl;
 
try {
  targetUrl = new URL(args[1]);
} catch (e) {
  console.error(
    '❌ Invalid Target URL. Make sure it includes http:// or https://',
  );
  process.exit(1);
}
 
const TARGET_HOST = targetUrl.hostname;
const TARGET_PORT =
  targetUrl.port || (targetUrl.protocol === 'https:' ? 443 : 80);
 
http
  .createServer((req, res) => {
    let body = [];
    req.on('data', (chunk) => body.push(chunk));
    req.on('end', () => {
      let buffer = Buffer.concat(body);
 
      console.log(`[PROXY] Incoming request: ${req.method} ${req.url}`);
 
      // Intercept completion requests and scrub the GBNF-crashing patterns
      if (req.method === 'POST' && req.url.includes('/chat/completions')) {
        let payload = buffer.toString();
 
        // Apply GBNF-compatible translations directly to the raw JSON payload
        payload = payload.replace(/\\\\d/g, '[0-9]');
        payload = payload.replace(/\\\\w/g, '[a-zA-Z0-9_]');
        // Using a safe, JSON-escaped whitespace array for \s
        payload = payload.replace(/\\\\s/g, '[ \\\\t\\\\n\\\\r]');
 
        buffer = Buffer.from(payload);
        req.headers['content-length'] = buffer.length;
      }
 
      const options = {
        hostname: TARGET_HOST,
        port: TARGET_PORT,
        path: req.url, // Keep the original path (e.g., /v1/chat/completions)
        method: req.method,
        headers: {
          ...req.headers,
          host: targetUrl.host, // Ensure host header matches target for routing
        },
      };
 
      const proxyReq = http.request(options, (proxyRes) => {
        res.writeHead(proxyRes.statusCode, proxyRes.headers);
        proxyRes.pipe(res);
      });
 
      proxyReq.on('error', (err) => {
        console.error('[PROXY] Error forwarding to llama.cpp:', err.message);
        if (!res.headersSent) {
          res.writeHead(502, { 'Content-Type': 'application/json' });
          res.end(
            JSON.stringify({
              error: 'llama.cpp server unreachable or refused connection.',
            }),
          );
        }
      });
 
      proxyReq.write(buffer);
      proxyReq.end();
    });
  })
  .listen(PROXY_PORT, () => {
    console.log(`🚀 [MCP Regex Fix] Proxy listening on port ${PROXY_PORT}`);
    console.log(`📡 Forwarding traffic to ${targetUrl.origin}`);
  });

How to use it

1. Save the code above as mcp-proxy.js.
2. Run it in your terminal, providing the port you want the proxy to listen on, and the base URL of your actual llama.cpp server:

   node mcp-proxy.js 9010 http://127.0.0.1:8080

3. In your LLM Client/UI settings, change your OpenAI Base URL to point to the proxy instead of the direct llama.cpp port:

   http://127.0.0.1:9010/v1

4. Fire your tool-calling prompt. The proxy handles the regex translation silently in under 5ms, and your local inference keeps running.