Chivox Speech Evaluation MCP · chivox-local-mcp
Large language models (GPT, Claude, Gemini, Qwen, …) are fundamentally text brains: they excel at reading and writing, but they cannot hear audio, open your database, or call third-party APIs on their own.
Model Context Protocol (MCP) is an open standard (led by Anthropic, adopted across the industry). It is like a walkie-talkie: the model can invoke external tools, data sources, and services when needed, then continue the conversation with the results.
Picture an IELTS speaking coach mini-app built on Claude with Chivox MCP. When a user sends a recording, the flow looks like this:
Configure once (developer)
Add the Chivox MCP server URL. On startup the model learns there is a new speech evaluation tool.
User message
The user uploads a self-introduction clip and asks: “Where is my pronunciation weak?” The backend forwards text + audio reference to the model.
Model reasoning
chivox_voice_eval.”MCP tool call
The model issues a standard tools/call with the audio reference and reference text. You do not hand-write RPC code.
Chivox scoring
The engine returns structured scores: overall, phoneme issues, fluency, and more — back through MCP.
Natural-language feedback
The model turns numbers into coaching text for the learner.
In practice you only need to:
Add server URL and API key. New scoring capabilities appear without redeploying your app.
{
"chivox_voice_eval": {
"type": "streamable-http",
"url": "https://mcp.cloud.chivox.com",
"apiKey": "sk-••••••••••"
}
}Tell the model what the tool does, when to call it, and how to phrase feedback for users.
You are an English speaking coach.
When the user sends audio:
1. Call chivox_voice_eval
2. Focus on phoneme families below 70
3. Give kind, actionable drillsUploads, auth, discovery, invocation timing, and response shaping are handled by MCP runtimes and the model — not bespoke glue code for every integration.
Pick the path that matches your stack. Under the hood everything speaks the same MCP protocol.
Best for PMs and operators. Add an MCP server in the platform UI, paste URL + key, test in chat.
| URL | https://mcp.cloud.chivox.com |
| Auth value | Bearer sk-•••••••••• (must include the Bearer prefix and a single space before the key — same as Authorization: Bearer sk-…) |
If the UI already has a separate Bearer / scheme dropdown, paste only sk-… in the token field. If there is a single “API key / auth” box that is copied verbatim into the header, use the full string Bearer sk-….
tools/list and exposes tools to the bot.Add a block to mcp.json (or equivalent) so Cursor, Claude Code, Dify, LangChain, etc. load tools at startup.
{
"mcpServers": {
"chivox_voice_eval": {
"type": "streamable-http",
"url": "https://mcp.cloud.chivox.com",
"env": { "API_KEY": "your-key" }
}
}
}For OpenAI-compatible APIs (Doubao, DeepSeek, GPT, Qwen, …), fetch tool schemas from MCP instead of hand-maintaining JSON Schema for all 16 tools (10 English + 6 Chinese).
chat.completions, prefer MCP list_tools → OpenAI tools[] instead of maintaining schemas by hand.import asyncio, os
from openai import AsyncOpenAI
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client
client = AsyncOpenAI(
base_url="https://ark.cn-beijing.volces.com/api/v3",
api_key=os.getenv("ARK_API_KEY"),
)
async def main():
async with streamablehttp_client(
"https://mcp.cloud.chivox.com",
headers={"Authorization": f"Bearer {os.getenv('CHIVOX_KEY')}"},
) as (read, write, _):
async with ClientSession(read, write) as session:
await session.initialize()
mcp_tools = (await session.list_tools()).tools
tools = [{
"type": "function",
"function": {
"name": t.name,
"description": t.description,
"parameters": t.inputSchema,
}
} for t in mcp_tools]
resp = await client.chat.completions.create(
model="ep-xxx-doubao-tools",
messages=[
{"role": "system", "content": "You are a coach; give gentle feedback."},
{"role": "user", "content": "Score my Apple clip: https://demo.com/u1.mp3"},
],
tools=tools,
tool_choice="auto",
)
# execute tool_calls via session.call_tool, then second LLM pass…
asyncio.run(main())Use the official MCP Python SDK, mcp-use, or vendor SDKs that wrap the same bridge.
Chivox exposes two parallel front doors to the same scoring engine, so pick whichever fits your client runtime:
MCP mode (preferred)
chivox-local-mcp enables microphone streamingTypical clients: Cursor / Claude Desktop / Coze / Dify / LangChain / custom backend agents
Function-calling mode (non-MCP · cvx_fc)
resume_token, intermediate results, backpressure framesTypical clients: native Android / iOS / Flutter, WeChat / Alipay mini-programs, legacy Java / PHP backends
Which one should I pick?
| Your client environment | Recommendation |
|---|---|
| Cursor / Claude Desktop / AI IDE that accepts MCP | MCP mode |
| Python / Node / Java backend able to pull MCP SDK | MCP mode |
| Coze / Dify / n8n visual platforms | MCP mode |
| Native Android / iOS app that wants raw HTTP + WS | Function-calling mode |
| WeChat / Alipay / Douyin mini-programs | Function-calling mode |
| Legacy / sandboxed runtime that can't add MCP deps | Function-calling mode |
/ws/audio/ and function-calling /ws/eval/ endpoints live in different session-ID namespaces.MCP mode has two deployment shapes — both speak JSON-RPC 2.0, the only difference is whether there's a local proxy between client and server:
Shape A · Remote direct (zero-code)
For Cursor / Coze / Dify / n8n and any other Streamable HTTP-aware client. No local dependency.
AI client
Cursor / Coze /
Dify / n8n …
⇄
Streamable HTTP
JSON-RPC 2.0
Chivox MCP server
/mcp · /ws/audio
scoring engine
Shape B · Local proxy (microphone streaming)
For Claude Desktop / Claude Code and any stdio-based MCP host that needs live mic capture.
AI client
Claude Desktop /
Claude Code
⇄
stdio
chivox-local-mcp
local proxy · SoX capture
⇄
HTTP / WS
Remote Chivox
MCP server
scoring engine
Standard data flow
initialize → negotiate protocol version & capabilitiestools/list → fetch the JSON Schema of all 16 evaluation toolstools/call → server runs evaluation → JSON-RPC responsecreate_stream_session → push PCM over /ws/audio/{session_id} → receive stream_eval_resultRaw JSON-RPC requests (copy-paste for custom clients)
If you don't use the official MCP SDK and instead hand-craft JSON-RPC 2.0 frames against POST /mcp, here's the minimal three-step handshake. Every request must carry Authorization: Bearer <your_api_key>.
① Initialize · initialize
{
"jsonrpc": "2.0",
"id": 1,
"method": "initialize",
"params": {
"protocolVersion": "2024-11-05",
"capabilities": {},
"clientInfo": { "name": "my-client", "version": "1.0.0" }
}
}On success the server returns its protocol version, server capabilities, and server info; only then may you call other methods.
② Fetch the tool catalog · tools/list
{
"jsonrpc": "2.0",
"id": 2,
"method": "tools/list",
"params": {}
}Returns all 16 evaluation tools (EN 10 + CN 6) plus the 2 streaming tools, each with a name, description, and parameter JSON Schema. Cache the response on the client, then render / bridge it to your LLM as needed.
③ Invoke a tool · tools/call
{
"jsonrpc": "2.0",
"id": 3,
"method": "tools/call",
"params": {
"name": "en_word_eval",
"arguments": {
"ref_text": "hello",
"audio_base64": "<base64 encoded audio>",
"accent": 2,
"rank": 100
}
}
}Success payload lives in result.content[0].text (a JSON string whose contents match the evaluation-result schema); failures surface as error.code + error.message.
session.initialize() / session.list_tools() / session.call_tool(). This section is only for bare-protocol implementors and debugging.No proxy, no SDK — just two links between client and server: REST for session/file control, WebSocket for audio frames.
Client
Native app / mini-program /
legacy backend / pure web
⇄
HTTP REST
/v1/call · /v1/functions
cvx_fc gateway
function-calling API
scoring engine
⇄
WebSocket
/ws/eval/{id}
Client (streaming)
microphone / chunk buffer
Three typical flows
a. One-shot file eval
POST /v1/call (audio_url or audio_base64) → synchronous score
b. Streaming eval
POST /v1/call → start_stream_eval → open /ws/eval/{id} → push PCM → receive intermediate / backpressure → send stop → receive result
c. Resume & fallback
Network glitch within 60s → reconnect with resume_token; unrecoverable → POST /v1/call → get_stream_result
Distinctive capabilities
suggested_interval_ms and the client slows down accordinglySESSION_EXPIRED / INVALID_STATE / … for deterministic retry logicFull usage examples live in the Non-MCP REST (cvx_fc) section below.
| Dimension | MCP mode | Function-calling (cvx_fc) |
|---|---|---|
| Protocol | JSON-RPC 2.0 | REST + WebSocket |
| Entry path | POST /mcp | POST /v1/call |
| List tools | tools/list | GET /v1/functions |
| Call tool | tools/call | POST /v1/call + name |
| Streaming WS | /ws/audio/{session_id} | /ws/eval/{session_id} |
| Local stdio proxy | yes (chivox-local-mcp) | n/a |
| Reconnect | SDK auto-reconnect | resume_token (60s suspend) |
| Intermediate results | stream_eval callback | intermediate frames |
| Backpressure feedback | — | backpressure frames |
| Client dependency | MCP SDK (official / framework) | any HTTP + WS client |
chivox-local-mcp is a local MCP proxy for Chivox speech scoring. It bridges the hosted scoring service to assistants such as Claude Desktop, Claude Code, and Cursor via Model Context Protocol.
Two evaluation modes cover interactive and batch workflows:
Audio flows over WebSocket while you speak — no intermediate file, lowest latency for tutoring UX.
Local path, Base64, or URL — ideal for batch jobs and async pipelines.
| Dependency | Version | Notes |
|---|---|---|
| Node.js | >= 18 | Required at runtime |
| SoX | any | Only for live microphone capture |
Install SoX when you need streaming capture:
# macOS
brew install sox
# Ubuntu / Debian
sudo apt-get install sox
# Windows: download from https://sox.sourceforge.net/ and add the install folder to PATHQuick checks before you build or wire a client:
node -v # need >= 18
rec --version # optional; missing SoX only affects live mic captureSource: public github.com/chivox-developer/cvx_local_mcp; optional internal mirror git.chivox.com/CLOUD_DEV/cvx_local_mcp. Clone whichever you can reach — same project.
Option A — npm install + build (matches upstream README)
git clone https://github.com/chivox-developer/cvx_local_mcp.git
# or: git clone https://git.chivox.com/CLOUD_DEV/cvx_local_mcp.git
cd cvx_local_mcp
npm install
npm run buildOption B — build script
git clone https://github.com/chivox-developer/cvx_local_mcp.git
cd cvx_local_mcp
bash scripts/build.sh(Optional) Global chivox-local-mcp command
After npm run build, expose the CLI so clients can reference a short command instead of node …/dist/index.js:
cd cvx_local_mcp
npm link
# or: npm install -g .args at an absolute path to dist/index.js, or set command to chivox-local-mcp when installed globally. For manual debugging, run npm run start from the repo after exporting MCP_REMOTE_URL / MCP_API_KEY as needed.| Parameter | Type | Required | Description |
|---|---|---|---|
| MCP_REMOTE_URL | string | — | Remote Chivox MCP URL; when unset, defaults to https://mcp.cloud.chivox.com |
| MCP_API_KEY | string | — | API key when the remote deployment requires authentication |
What these setups share
{
"name": "chivox-speech-eval",
"type": "streamable-http",
"url": "https://mcp.cloud.chivox.com"
}Edit ~/Library/Application Support/Claude/claude_desktop_config.json on macOS:
{
"mcpServers": {
"chivox": {
"command": "node",
"args": ["/path/to/cvx_local_mcp/dist/index.js"],
"env": {
"MCP_REMOTE_URL": "https://mcp.cloud.chivox.com",
"MCP_API_KEY": "your-api-key"
}
}
}
}claude mcp add chivox -- env MCP_REMOTE_URL=https://mcp.cloud.chivox.com MCP_API_KEY=your_chivox_api_key node /path/to/cvx_local_mcp/dist/index.jsUse the same Streamable HTTP JSON as Cursor; only the settings location differs.
{
"mcpServers": {
"chivox-speech-eval": {
"type": "streamable-http",
"url": "https://mcp.cloud.chivox.com"
}
}
}In Coze, add a custom MCP extension with transport streamable-http and URL https://mcp.cloud.chivox.com. Set authentication so requests send Authorization: Bearer <your_api_key> — in JSON configs use "Authorization": "Bearer sk-…" (note the space after Bearer). After saving, bots can call scoring tools from prompts.
Use an HTTP node to POST audio to https://your-audio-host.com/upload, then call MCP JSON-RPC with the returned audioId.
For WeCom smart bots, register https://mcp.cloud.chivox.com as an external MCP service in the admin console. For WorkBuddy (Tencent CodeBuddy), paste the same mcpServers JSON as in Claude Desktop / Cursor. DingTalk / Feishu product UIs for third-party MCP vary by version and rollout; this page only documents the two paths above so readers are not sent to menus we cannot verify.
Add an MCP client node, point it at the same URL, and drag tool calls into LangChain / n8n graphs for batch scoring + reporting.
Any MCP-compatible host can use:
| Field | Value |
|---|---|
| Transport | streamable-http |
| URL | https://mcp.cloud.chivox.com |
| Auth (optional) | Authorization: Bearer <MCP_API_KEY> |
Typical traits
list_tools, never copy-paste sixteen schemas by hand.Connect once, call list_tools, forward schemas to any LLM. Example (Python):
# pip install mcp openai
import asyncio, os
from openai import OpenAI
from mcp.client.streamable_http import streamablehttp_client
from mcp import ClientSession
MCP_URL = "https://mcp.cloud.chivox.com"
llm = OpenAI(api_key=os.getenv("DOUBAO_KEY"),
base_url="https://ark.cn-beijing.volces.com/api/v3")
async def main(audio_id: str, ref: str):
async with streamablehttp_client(MCP_URL) as (reader, writer, _):
async with ClientSession(reader, writer) as sess:
await sess.initialize()
tools = await sess.list_tools()
openai_tools = [{
"type": "function",
"function": {"name": t.name, "description": t.description,
"parameters": t.inputSchema},
} for t in tools.tools]
resp = llm.chat.completions.create(
model="doubao-1-5-pro-32k",
messages=[{"role": "user",
"content": f"Score audioId={audio_id}, reference: {ref}"}],
tools=openai_tools,
)
# handle tool_calls → sess.call_tool → second LLM pass
asyncio.run(main("abc123", "I think therefore I am"))Point any MCP-aware adapter at https://mcp.cloud.chivox.com (Streamable HTTP). The framework keeps the tool loop: discovery → tool_calls → execution → follow-up messages.
LangChain / LangGraph
# pip install langchain-mcp-adapters langgraph
from langchain_mcp_adapters.client import MultiServerMCPClient
from langgraph.prebuilt import create_react_agent
client = MultiServerMCPClient({
"chivox": {
"transport": "streamable_http",
"url": "https://mcp.cloud.chivox.com",
}
})
tools = await client.get_tools()
agent = create_react_agent("openai:gpt-4o-mini", tools)
result = await agent.ainvoke({"messages":
[("user", "Score audioId=abc123 ref: think")]})
print(result["messages"][-1].content)Mastra
import { MCPClient } from "@mastra/mcp";
import { Agent } from "@mastra/core/agent";
import { openai } from "@ai-sdk/openai";
const mcp = new MCPClient({ servers: {
chivox: { url: new URL("https://mcp.cloud.chivox.com") }
}});
export const speechCoach = new Agent({
name: "speech-coach",
instructions: "Use Chivox tools to score speech and give feedback.",
model: openai("gpt-4o-mini"),
tools: await mcp.getTools(),
});OpenAI Agents SDK (Python)
# pip install openai-agents
from agents import Agent, Runner
from agents.mcp import MCPServerStreamableHttp
chivox = MCPServerStreamableHttp(
params={"url": "https://mcp.cloud.chivox.com"},
name="chivox-speech-eval",
)
async with chivox:
agent = Agent(name="coach",
instructions="Professional speaking coach",
mcp_servers=[chivox])
r = await Runner.run(agent, "Score audioId=abc123")
print(r.final_output)LlamaIndex (llama-index-tools-mcp), AutoGen, CrewAI, and Spring AI ship similar bridges — same URL, same 16 tools (10 English + 6 Chinese).
Vendors such as DeepSeek, Zhipu GLM, Moonshot KIMI, Doubao (Volcengine), Qwen, and Baidu expose OpenAI-compatible endpoints. The flow is always: list_tools → map to OpenAI tools → model returns tool_calls → call_tool → feed results back for a second completion.
base_url and API keys — Chivox MCP code stays identical.Upload audio and obtain audioId
Most scoring tools expect an uploaded clip reference. Minimal HTTP upload:
import requests
with open('audio.mp3', 'rb') as f:
r = requests.post(
'https://your-audio-host.com/upload',
data=f,
headers={'Content-Type': 'audio/mp3'}
)
result = r.json()
print(f"audioId: {result['audioId']}")
print(f"size: {result['size']} bytes")
print(f"expires: {result['expiresIn']}")const fs = require('fs');
const audioData = fs.readFileSync('audio.mp3');
const res = await fetch('https://your-audio-host.com/upload', {
method: 'POST',
headers: { 'Content-Type': 'audio/mp3' },
body: audioData,
});
const result = await res.json();
console.log('audioId:', result.audioId);Example: Doubao + MCP tool loop
When the hosted LLM has no native MCP runtime, pull schemas once and pass them through chat.completions.
# Doubao / DeepSeek / Qwen / Moonshot / GLM — OpenAI-compatible
import json
from mcp.client.streamable_http import streamablehttp_client
from mcp import ClientSession
# doubao = OpenAI(base_url="https://ark.cn-beijing.volces.com/api/v3", api_key=os.getenv("ARK_API_KEY"))
async with streamablehttp_client("https://mcp.cloud.chivox.com") as (r, w, _):
async with ClientSession(r, w) as mcp:
await mcp.initialize()
tools = (await mcp.list_tools()).tools
oa_tools = [{"type": "function", "function": {
"name": t.name,
"description": t.description,
"parameters": t.inputSchema,
}} for t in tools]
messages = [{"role": "user", "content": "Score audioId=abc123"}]
resp = doubao.chat.completions.create(
model="doubao-1-5-pro-32k", messages=messages, tools=oa_tools,
)
msg = resp.choices[0].message
if msg.tool_calls:
messages.append(msg)
for c in msg.tool_calls:
result = await mcp.call_tool(
c.function.name, arguments=json.loads(c.function.arguments))
messages.append({"role": "tool", "tool_call_id": c.id,
"content": result.content[0].text})
final = doubao.chat.completions.create(
model="doubao-1-5-pro-32k", messages=messages)
print(final.choices[0].message.content)list_tools().Most production setups already have an app server — add a long-lived MCP session in the process lifespan, call tools directly when latency matters, and only then involve an LLM for natural-language coaching.
from fastapi import FastAPI
from mcp.client.streamable_http import streamablehttp_client
from mcp import ClientSession
from contextlib import asynccontextmanager
mcp_session: ClientSession | None = None
@asynccontextmanager
async def lifespan(app):
global mcp_session
async with streamablehttp_client("https://mcp.cloud.chivox.com") as (r, w, _):
async with ClientSession(r, w) as s:
await s.initialize()
mcp_session = s
yield
mcp_session = None
app = FastAPI(lifespan=lifespan)
@app.post("/api/evaluate")
async def evaluate(audio_id: str, ref_text: str):
r = await mcp_session.call_tool(
"evaluate_english_sentence",
{"audioId": audio_id, "sentence": ref_text},
)
return {"result": r.content[0].text}The sections below drill into path ③ (chat.completions) per LLM vendor
Pick the vendor you use today — SDK / base_url / auth differ slightly, but the Chivox MCP side stays identical.
Latest model is DeepSeek-V3.2 (Dec 2025) — the first model to natively fuse thinking + tool use. Use deepseek-chat for non-thinking mode and deepseek-reasoner for thinking mode — both support function calling with a 128K context.
import os, json, asyncio
from openai import OpenAI
from mcp.client.streamable_http import streamablehttp_client
from mcp import ClientSession
client = OpenAI(
api_key=os.environ["DEEPSEEK_API_KEY"],
base_url="https://api.deepseek.com/v1",
)
async def run(audio_id: str, ref_text: str):
async with streamablehttp_client("https://mcp.cloud.chivox.com") as (r, w, _):
async with ClientSession(r, w) as mcp:
await mcp.initialize()
tools = (await mcp.list_tools()).tools
oa_tools = [{"type": "function", "function": {
"name": t.name,
"description": t.description,
"parameters": t.inputSchema,
}} for t in tools]
messages = [{"role": "user", "content":
f"Score this clip audioId={audio_id}, reference: {ref_text}"}]
resp = client.chat.completions.create(
model="deepseek-chat", messages=messages, tools=oa_tools)
msg = resp.choices[0].message
if msg.tool_calls:
messages.append(msg)
for call in msg.tool_calls:
result = await mcp.call_tool(
call.function.name,
arguments=json.loads(call.function.arguments),
)
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": result.content[0].text,
})
final = client.chat.completions.create(
model="deepseek-chat", messages=messages)
return final.choices[0].message.content
asyncio.run(run("audio_abc123", "Hello, nice to meet you."))Latest flagships are GLM-5 and GLM-5.1 (Feb 2026, 744B-A40B MoE, 200K context, on par with Claude Opus 4.5), with glm-4.7 / glm-4.6 still available and glm-4-flash kept as the free dev tier. OpenAI-compatible base URL: https://open.bigmodel.cn/api/paas/v4/.
import os, json, asyncio
from openai import OpenAI
from mcp.client.streamable_http import streamablehttp_client
from mcp import ClientSession
client = OpenAI(
api_key=os.environ["ZHIPU_API_KEY"],
base_url="https://open.bigmodel.cn/api/paas/v4/",
)
async def run(audio_id: str, ref_text: str):
async with streamablehttp_client("https://mcp.cloud.chivox.com") as (r, w, _):
async with ClientSession(r, w) as mcp:
await mcp.initialize()
tools = (await mcp.list_tools()).tools
oa_tools = [{"type": "function", "function": {
"name": t.name,
"description": t.description,
"parameters": t.inputSchema,
}} for t in tools]
messages = [{"role": "user", "content":
f"Score audioId={audio_id}, reference: {ref_text}"}]
resp = client.chat.completions.create(
model="glm-4-flash", # free dev tier; production: glm-5 / glm-4.7
messages=messages, tools=oa_tools)
msg = resp.choices[0].message
if msg.tool_calls:
messages.append(msg)
for call in msg.tool_calls:
result = await mcp.call_tool(
call.function.name,
arguments=json.loads(call.function.arguments),
)
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": result.content[0].text,
})
final = client.chat.completions.create(
model="glm-4-flash", messages=messages)
return final.choices[0].message.content
asyncio.run(run("audio_abc123", "How are you today?"))glm-4-flash (free) for dev,glm-4.7 for everyday tutoring diagnosis, and glm-5 / glm-5.1for K12 / multi-step agentic orchestration with 200K context.Latest flagship is Kimi K2.5 (Jan 27, 2026) — MoE, 256K context, native multimodal, agent-cluster collaboration. Variants: kimi-k2.5, kimi-k2-thinking, kimi-k2-turbo-preview. Legacy moonshot-v1-8k/32k/128k still works but the K2.5 family is recommended. Base URL: https://api.moonshot.cn/v1 (China) or https://api.moonshot.ai/v1 (global).
import os, json, asyncio
from openai import OpenAI
from mcp.client.streamable_http import streamablehttp_client
from mcp import ClientSession
client = OpenAI(
api_key=os.environ["MOONSHOT_API_KEY"],
base_url="https://api.moonshot.cn/v1",
)
async def run(audio_id: str, ref_text: str):
async with streamablehttp_client("https://mcp.cloud.chivox.com") as (r, w, _):
async with ClientSession(r, w) as mcp:
await mcp.initialize()
tools = (await mcp.list_tools()).tools
oa_tools = [{"type": "function", "function": {
"name": t.name,
"description": t.description,
"parameters": t.inputSchema,
}} for t in tools]
messages = [{"role": "user", "content":
f"Score English audioId={audio_id}, reference: {ref_text}"}]
resp = client.chat.completions.create(
model="kimi-k2.5", # flagship; or kimi-k2-turbo-preview / moonshot-v1-128k
messages=messages, tools=oa_tools)
msg = resp.choices[0].message
if msg.tool_calls:
messages.append(msg)
for call in msg.tool_calls:
result = await mcp.call_tool(
call.function.name,
arguments=json.loads(call.function.arguments),
)
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": result.content[0].text,
})
final = client.chat.completions.create(
model="kimi-k2.5", messages=messages)
return final.choices[0].message.content
asyncio.run(run("audio_abc123", "Nice to meet you."))kimi-k2-thinking for deeper reasoning,kimi-k2-turbo-preview for low-latency live feedback.ByteDance's Doubao Seed 2.0 family (released Feb 2026) ships Pro / Lite / Mini / Code variants on Volcano Ark, all behind an OpenAI-compatible /api/v3/chat/completions endpoint with native function calling, JSON Schema structured output, and chain-of-thought thinking mode. Plugging it into Chivox MCP follows the exact same dynamic-tools pattern as DeepSeek / GLM / KIMI.
en.sent.score, then drill weak words with en.word.score" in a single turn — a great fit for "evaluate → diagnose → re-test" agent loops. Thinking mode is on by default; disable it with thinking: { type: "disabled" } when latency matters.Recommended variants
| model id | tier | use case |
|---|---|---|
| doubao-seed-2-0-pro-260215 | Pro · flagship | Multi-step evaluation orchestration + deep diagnosis agents |
| doubao-seed-2-0-lite-260215 | Lite · balanced | Online tutoring single-turn diagnosis — best price / quality |
| doubao-seed-2-0-mini-260215 | Mini · compact | High-concurrency / mobile / edge — low-latency feedback |
| doubao-seed-2-0-code-preview-260215 | Code · preview | Code generation / tool DSL — not the primary speech pick |
| doubao-seed-1-8-251228 | 1.8 · deep thinking | Tunable reasoning_effort (minimal/low/medium/high), previous-gen still active |
| doubao-seed-1-6-251015 | 1.6 · classic | 256K context classic — keep using for already-shipped projects |
Environment
# Volcano Ark console → API key management
ARK_API_KEY=ak-xxxxxxxxxxxxxxxxxxxxxxxx
CHIVOX_APP_KEY=your_chivox_app_key
CHIVOX_SECRET_KEY=your_chivox_secret_keyFull example (Python · dynamic tools + tool-call loop)
import os, json, asyncio
from openai import OpenAI
from mcp.client.streamable_http import streamablehttp_client
from mcp import ClientSession
# Volcano Ark OpenAI-compatible endpoint
client = OpenAI(
api_key=os.environ["ARK_API_KEY"],
base_url="https://ark.cn-beijing.volces.com/api/v3",
)
MODEL = "doubao-seed-2-0-pro-260215" # or lite / mini
async def evaluate_with_doubao(audio_id: str, ref_text: str):
async with streamablehttp_client("https://mcp.cloud.chivox.com") as (r, w, _):
async with ClientSession(r, w) as mcp:
await mcp.initialize()
# 1. Inject all 16 Chivox tool schemas dynamically
tools = (await mcp.list_tools()).tools
ark_tools = [{
"type": "function",
"function": {
"name": t.name,
"description": t.description,
"parameters": t.inputSchema,
},
} for t in tools]
messages = [
{"role": "system",
"content": "You are a professional English speaking coach who actively calls Chivox evaluation tools to read real scores."},
{"role": "user",
"content": f"Evaluate this recording. audio_id={audio_id}, refText: {ref_text}. "
f"Run sentence-level scoring first; if any word is weak, follow up with word-level."},
]
# 2. Round 1 — let Doubao pick tools
resp = client.chat.completions.create(
model=MODEL,
messages=messages,
tools=ark_tools,
tool_choice="auto",
# extra_body={"thinking": {"type": "disabled"}}, # toggle off for low latency
)
msg = resp.choices[0].message
messages.append(msg)
# 3. Tool-call loop — Doubao may emit multiple tool_calls per turn
for call in (msg.tool_calls or []):
result = await mcp.call_tool(
call.function.name,
arguments=json.loads(call.function.arguments),
)
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": result.content[0].text,
})
# 4. Round 2 — Doubao reads the real evaluation JSON and produces coaching prose
final = client.chat.completions.create(
model=MODEL,
messages=messages,
)
return final.choices[0].message.content
print(asyncio.run(evaluate_with_doubao(
audio_id="audio_abc123",
ref_text="The quick brown fox jumps over the lazy dog.",
)))Thinking mode (optional)
Seed 2.0 enables thinking by default — chain-of-thought before answer — which clearly improves multi-step orchestration but adds 30%–80% latency. Toggle it via extra_body:
# Disable thinking — faster, ideal for live feedback
client.chat.completions.create(
model="doubao-seed-2-0-lite-260215",
messages=messages,
tools=ark_tools,
extra_body={"thinking": {"type": "disabled"}},
)
# Enable thinking — default; ideal for Pro + multi-step agents
client.chat.completions.create(
model="doubao-seed-2-0-pro-260215",
messages=messages,
tools=ark_tools,
extra_body={"thinking": {"type": "enabled"}},
)doubao-seed-2-0-lite with thinking off for 1-on-1 tutoring;doubao-seed-2-0-pro with thinking on for K12 / exam prep that needs multi-task coordination + level-up suggestions;doubao-seed-2-0-mini for mobile / real-time conversational scoring.Alibaba's Qwen 3.6 series (Apr 2026) is the latest — flagship qwen3.6-plus, fast tier qwen3.6-flash, and the open-weight MoE qwen3.6-35b-a3b. Older aliases qwen-max / qwen-plus still resolve. Served via Model Studio with OpenAI-compatible endpoints:https://dashscope.aliyuncs.com/compatible-mode/v1 (China Beijing), https://dashscope-intl.aliyuncs.com/compatible-mode/v1 (Singapore), https://dashscope-us.aliyuncs.com/compatible-mode/v1 (US Virginia).
import os, json, asyncio
from openai import OpenAI
from mcp.client.streamable_http import streamablehttp_client
from mcp import ClientSession
client = OpenAI(
api_key=os.environ["DASHSCOPE_API_KEY"],
base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)
async def run(audio_id: str, ref_text: str):
async with streamablehttp_client("https://mcp.cloud.chivox.com") as (r, w, _):
async with ClientSession(r, w) as mcp:
await mcp.initialize()
tools = (await mcp.list_tools()).tools
oa_tools = [{"type": "function", "function": {
"name": t.name,
"description": t.description,
"parameters": t.inputSchema,
}} for t in tools]
messages = [{"role": "user", "content":
f"Score audioId={audio_id}, reference: {ref_text}"}]
resp = client.chat.completions.create(
model="qwen3.6-plus", # flagship; or qwen3.6-flash / qwen-long
messages=messages, tools=oa_tools)
msg = resp.choices[0].message
if msg.tool_calls:
messages.append(msg)
for call in msg.tool_calls:
result = await mcp.call_tool(
call.function.name,
arguments=json.loads(call.function.arguments),
)
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": result.content[0].text,
})
final = client.chat.completions.create(
model="qwen3.6-plus", messages=messages)
return final.choices[0].message.content
asyncio.run(run("audio_abc123", "The weather is nice today."))qwen3.6-plus (flagship, agentic coding & reasoning), qwen3.6-flash (fast / cheap), qwen3-max (previous flagship), and qwen-long (1M context). All support function calling; pass extra_body={"preserve_thinking": True} to keep thought history across agent turns.OpenAI's flagship is GPT-5.1 (released 2025-11-13) — 400K context, 128K max output, with a configurablereasoning_effort parameter (none / low / medium / high). Also available: gpt-5.1-chat-latest (the snapshot ChatGPT uses, 128K context, optimised for chat) and gpt-5-mini (cost-efficient). All natively support Function Calling and Structured Outputs.
Environment
OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxx
CHIVOX_APP_KEY=your_chivox_app_key
CHIVOX_SECRET_KEY=your_chivox_secret_keyFull example (Python · GPT-5.1 + Chivox MCP)
import os, json, asyncio
from openai import OpenAI
from mcp.client.streamable_http import streamablehttp_client
from mcp import ClientSession
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
async def evaluate_with_gpt(audio_id: str, ref_text: str):
async with streamablehttp_client("https://mcp.cloud.chivox.com") as (r, w, _):
async with ClientSession(r, w) as mcp:
await mcp.initialize()
tools = (await mcp.list_tools()).tools
oa_tools = [{"type": "function", "function": {
"name": t.name,
"description": t.description,
"parameters": t.inputSchema,
}} for t in tools]
messages = [
{"role": "system",
"content": "You are an English speaking coach. Always call the Chivox eval tool first to get real scores."},
{"role": "user",
"content": f"Score this take. audioId={audio_id}, reference: {ref_text}"},
]
resp = client.chat.completions.create(
model="gpt-5.1",
messages=messages,
tools=oa_tools,
tool_choice="auto",
reasoning_effort="medium", # GPT-5.1 only: none / low / medium / high
)
msg = resp.choices[0].message
messages.append(msg)
for call in (msg.tool_calls or []):
result = await mcp.call_tool(
call.function.name,
arguments=json.loads(call.function.arguments),
)
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": result.content[0].text,
})
final = client.chat.completions.create(
model="gpt-5.1", messages=messages)
return final.choices[0].message.content
print(asyncio.run(evaluate_with_gpt(
"audio_abc123", "The quick brown fox jumps over the lazy dog.")))gpt-5.1with reasoning_effort="medium" for tool + diagnosis loops; use gpt-5.1-chat-latest for pure conversational coaching; switch to gpt-5-mini for high-volume / batch scoring; crank reasoning_effort to high for deep multi-step agent planning.Anthropic's current flagship is Claude Opus 4.7, with Claude Sonnet 4.6 (balanced) and Claude Haiku 4.5 (fast). Claude uses the Messages API (different from OpenAI Chat Completions) and a tool_use / tool_result block protocol. Tool schemas use the input_schema field (equivalent to OpenAI's parameters). Opus 4.5+ adds a new effort control (low/medium/high) — medium gets you Sonnet 4.5 quality with ~76% fewer tokens.
Environment
ANTHROPIC_API_KEY=sk-ant-xxxxxxxxxxxxxxxxxxxxxxxx
CHIVOX_APP_KEY=your_chivox_app_key
CHIVOX_SECRET_KEY=your_chivox_secret_keyFull example (Python · native Anthropic SDK + MCP bridge)
# pip install anthropic mcp
import os, json, asyncio
from anthropic import Anthropic
from mcp.client.streamable_http import streamablehttp_client
from mcp import ClientSession
client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
async def evaluate_with_claude(audio_id: str, ref_text: str):
async with streamablehttp_client("https://mcp.cloud.chivox.com") as (r, w, _):
async with ClientSession(r, w) as mcp:
await mcp.initialize()
# Convert MCP tools to Anthropic shape (uses input_schema, not parameters)
tools = (await mcp.list_tools()).tools
anth_tools = [{
"name": t.name,
"description": t.description,
"input_schema": t.inputSchema,
} for t in tools]
messages = [{"role": "user",
"content": f"Score this take: audioId={audio_id}, reference: {ref_text}"}]
# Tool loop — Claude may emit multiple tool_use blocks across turns
while True:
resp = client.messages.create(
model="claude-opus-4-7", # or claude-sonnet-4-6 / claude-haiku-4-5
max_tokens=2048,
tools=anth_tools,
messages=messages,
system="You are an English speaking coach. Always call the Chivox eval tool first.",
)
# Append the full assistant turn (including tool_use blocks)
messages.append({"role": "assistant", "content": resp.content})
if resp.stop_reason != "tool_use":
return "".join(b.text for b in resp.content if b.type == "text")
# Extract every tool_use block, run it, return tool_result
tool_results = []
for block in resp.content:
if block.type == "tool_use":
result = await mcp.call_tool(block.name, arguments=block.input)
tool_results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": result.content[0].text,
})
messages.append({"role": "user", "content": tool_results})
print(asyncio.run(evaluate_with_claude(
"audio_abc123", "The quick brown fox jumps over the lazy dog.")))claude-opus-4-7for deep diagnosis & complex agent orchestration (step-change in agentic coding over 4.6);claude-sonnet-4-6 for daily tool + diagnosis loop (best price/quality);claude-haiku-4-5 for high-throughput live feedback. Claude is also natively available on AWS Bedrock and Google Vertex AI if you need data residency.Google's newest is Gemini 3 Flash (gemini-3-flash-preview, Dec 17 2025) — Pro-grade reasoning at Flash latency / cost. Also available: Gemini 3.1 Pro Preview (frontier flagship) and Gemini 3.1 Flash-Lite Preview. All support Function Calling, parallel/compositional calls, and a new thinking_level parameter. We recommend the OpenAI-compatible endpoint so you can reuse your existing OpenAI SDK + MCP bridge code with zero rewrites.
Environment
GEMINI_API_KEY=AIzaSyxxxxxxxxxxxxxxxxxxxxxxxxxxx
CHIVOX_APP_KEY=your_chivox_app_key
CHIVOX_SECRET_KEY=your_chivox_secret_keyOption 1 — OpenAI-compatible endpoint (recommended, zero migration cost)
import os, json, asyncio
from openai import OpenAI
from mcp.client.streamable_http import streamablehttp_client
from mcp import ClientSession
# Gemini OpenAI-compatible endpoint — full OpenAI SDK reuse
client = OpenAI(
api_key=os.environ["GEMINI_API_KEY"],
base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
)
async def evaluate_with_gemini(audio_id: str, ref_text: str):
async with streamablehttp_client("https://mcp.cloud.chivox.com") as (r, w, _):
async with ClientSession(r, w) as mcp:
await mcp.initialize()
tools = (await mcp.list_tools()).tools
oa_tools = [{"type": "function", "function": {
"name": t.name,
"description": t.description,
"parameters": t.inputSchema,
}} for t in tools]
messages = [{"role": "user",
"content": f"Score audioId={audio_id}, reference: {ref_text}"}]
resp = client.chat.completions.create(
model="gemini-3-flash-preview", # or gemini-3.1-pro-preview
messages=messages,
tools=oa_tools,
tool_choice="auto",
)
msg = resp.choices[0].message
messages.append(msg)
for call in (msg.tool_calls or []):
result = await mcp.call_tool(
call.function.name,
arguments=json.loads(call.function.arguments),
)
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": result.content[0].text,
})
final = client.chat.completions.create(
model="gemini-3-flash-preview", messages=messages)
return final.choices[0].message.content
print(asyncio.run(evaluate_with_gemini(
"audio_abc123", "The quick brown fox jumps over the lazy dog.")))Option 2 — Native google-genai SDK (thinking_level / streaming function calls)
# pip install google-genai
from google import genai
from google.genai import types
client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])
# Convert MCP tools to Gemini function_declarations
fn_decls = [{
"name": t.name,
"description": t.description,
"parameters": t.inputSchema,
} for t in tools]
resp = client.models.generate_content(
model="gemini-3-flash-preview",
contents=f"Score audioId={audio_id}, reference: {ref_text}",
config=types.GenerateContentConfig(
tools=[types.Tool(function_declarations=fn_decls)],
# Gemini 3+: thinking_level replaces the older thinking_budget
# thinking_level="high",
),
)
# Tool call lives in response.candidates[0].content.parts[0].function_call
fc = resp.candidates[0].content.parts[0].function_call
print(f"call: {fc.name}({dict(fc.args)}) id={fc.id}")gemini-3-flash-preview (3× faster than 2.5 Pro at ~$0.50/M input — already the default in the Gemini app); use gemini-3.1-pro-preview for complex multi-step agents and coding; pick gemini-3.1-flash-lite-preview for the cheapest large-context multimodal batches.function_call carries a unique id; you must echo it back in the function_response for multi-turn tool use, or the request is rejected. The OpenAI-compatible endpoint handles this automatically.| Model | Suggested id | Function calling | Notes |
|---|---|---|---|
| DeepSeek V3.2 · non-thinking | deepseek-chat | Yes | Best default for tool + diagnosis loop |
| DeepSeek V3.2 · thinking | deepseek-reasoner | Yes (in chain-of-thought) | Deep analysis + error attribution |
| GLM-5 · flagship | glm-5 / glm-5.1 | Yes | 200K context + agentic orchestration |
| GLM-4.7 | glm-4.7 | Yes | Best price/quality for daily diagnosis |
| GLM-4-Flash | glm-4-flash | Yes | Free dev tier |
| Kimi K2.5 | kimi-k2.5 | Yes | 256K + multimodal + agent cluster |
| Kimi K2 Thinking | kimi-k2-thinking | Yes | Multi-step reasoning + progress reports |
| Kimi K2 Turbo | kimi-k2-turbo-preview | Yes | Low-latency live feedback |
| Doubao Seed 2.0 Pro | doubao-seed-2-0-pro-260215 | Yes (thinking) | Multi-step tool orchestration & deep diagnosis |
| Doubao Seed 2.0 Lite | doubao-seed-2-0-lite-260215 | Yes | Online tutoring · best price/quality |
| Doubao Seed 2.0 Mini | doubao-seed-2-0-mini-260215 | Yes | Mobile / edge — low-latency live feedback |
| Qwen 3.6 Plus | qwen3.6-plus | Yes | Alibaba flagship — agentic coding & reasoning |
| Qwen 3.6 Flash | qwen3.6-flash | Yes | Fast / cheap balanced tier |
| GPT-5.1 (flagship) | gpt-5.1 | Yes (reasoning_effort) | 400K context + deep reasoning + complex agents |
| GPT-5.1 Chat | gpt-5.1-chat-latest | Yes | ChatGPT snapshot — conversational coaching |
| GPT-5 mini | gpt-5-mini | Yes | High-volume / batch scoring |
| Claude Opus 4.7 | claude-opus-4-7 | Yes (tool_use) | Anthropic flagship — agentic coding & deep diagnosis |
| Claude Sonnet 4.6 | claude-sonnet-4-6 | Yes | Best price/quality for daily diagnosis loop |
| Claude Haiku 4.5 | claude-haiku-4-5 | Yes | High-throughput live feedback |
| Gemini 3 Flash | gemini-3-flash-preview | Yes (parallel / compositional) | Pro-grade reasoning at Flash latency / price |
| Gemini 3.1 Pro Preview | gemini-3.1-pro-preview | Yes (thinking_level) | Frontier flagship for complex multi-step agents |
tool_use / tool_result protocol) can be called through the OpenAI Python SDK by swapping base_url + api_key — no changes needed on the Chivox MCP side.When this section applies
cvx_fc (OpenAI function-calling style). It targets the same evaluation engine as MCP — they are two parallel entry pointsRule of thumb: always prefer MCP; drop down to this section only when your client genuinely cannot host an MCP SDK.
Alongside MCP, Chivox exposes an OpenAI-compatible Function-calling REST API (internal name cvx_fc). It hits the same scoring engine as MCP mode, but speaks pure HTTP / WebSocket with zero MCP dependencies.
POST /mcp, JSON-RPC 2.0) and function-calling (POST /v1/call, REST) are two parallel entry points with equivalent capabilities and essentially identical tool/function catalogs.API endpoints
| Method | Path | Purpose |
|---|---|---|
| GET | /v1/functions | List all available evaluation functions (equivalent to MCP tools/list) |
| POST | /v1/call | Invoke a function (equivalent to MCP tools/call) · also used to create streaming sessions and fetch streaming results |
| WS | /ws/eval/{session_id} | Streaming audio WebSocket |
Authentication
Every request carries the API key via the HTTP header:
Authorization: Bearer <your_api_key>Auth failures return 401 (missing header) or 403 (invalid key / not entitled). Structured error codes: see API reference · error codes.
Hit GET /v1/functions to fetch every evaluation function your API key is entitled to. The response is shaped like an OpenAI function-calling tools catalog, so you can feed it directly into an LLM's tools argument:
GET /v1/functions
Authorization: Bearer <api_key>{
"object": "list",
"data": [
{
"type": "function",
"function": {
"name": "en_word_eval",
"description": "English word pronunciation eval; returns overall score and per-phoneme scores.",
"parameters": {
"type": "object",
"properties": {
"ref_text": { "type": "string", "description": "Reference text" },
"audio_base64": { "type": "string", "description": "Base64-encoded audio (either this or audio_url)" },
"audio_url": { "type": "string", "description": "HTTP(S) URL of audio (either this or audio_base64)" },
"accent": { "type": "number", "description": "1 = British / 2 = American / 3 = any (default)" },
"rank": { "type": "number", "description": "Scoring scale: 4 or 100 (default)" }
},
"required": ["ref_text"]
}
}
}
/* … more functions … */
]
}tools/list. Each API key sees only the functions it's entitled to — rely on the live response rather than hard-coding all 19 functions on the client.English evaluation functions (10)
| Function | core_type | Purpose |
|---|---|---|
| en_word_eval | en.word.score | Word pronunciation |
| en_word_correction | en.word.pron | Word-level pronunciation correction |
| en_phonics_eval | en.nsp.score | Phonics evaluation |
| en_sentence_eval | en.sent.score | Sentence reading |
| en_sentence_correction | en.sent.pron | Sentence-level pronunciation correction |
| en_vocab_eval | en.vocabs.pron | Batch vocabulary |
| en_paragraph_eval | en.pred.score | Paragraph reading |
| en_realtime_eval | en.rltm.score | Real-time reading |
| en_choice_eval | en.choc.score | Spoken multiple-choice |
| en_semi_open_eval | en.scne.exam | Semi-open (scenario dialogue, etc.) |
Chinese evaluation functions (6)
| Function | core_type | Purpose |
|---|---|---|
| cn_word_pinyin_eval | cn.word.score | Pinyin / character pronunciation |
| cn_word_raw_eval | cn.word.raw | Character pronunciation |
| cn_sentence_eval | cn.sent.raw | Sentence reading |
| cn_paragraph_eval | cn.pred.raw | Paragraph reading |
| cn_rec_eval | cn.rec.raw | Bounded-branch recognition |
| cn_aitalk_eval | cn.recscore.raw | Spoken expression (AI Talk) |
Streaming functions (3)
| Function | Role |
|---|---|
| start_stream_eval | Create a streaming session; returns session_id / ws_url / resume_token |
| get_stream_result | Fallback fetch: when the WS didn't deliver result, pull the final score via HTTP |
| stream_eval_result | (MCP-side alias; cvx_fc clients use get_stream_result) |
Each function's business semantics / parameters / response fields are identical to its same-named MCP tool — refer to API reference · English tools and Chinese tools below.
Pass the complete audio via audio_base64 or audio_url (either/or) to /v1/call, and the score returns synchronously (semantically equivalent to MCP tools such as evaluate_english_word):
curl -X POST https://fc.cloud.chivox.com/v1/call \
-H "Authorization: Bearer your_api_key" \
-H "Content-Type: application/json" \
-d '{
"name": "en_word_eval",
"arguments": {
"ref_text": "hello",
"audio_url": "https://example.com/hello.mp3",
"accent": 2,
"rank": 100
}
}'Success:
{
"name": "en_word_eval",
"result": {
"overall": 85.5,
"details": { /* phonemes / prosody / fluency — identical schema to MCP mode */ }
}
}Failure:
{
"name": "en_word_eval",
"error": { "message": "human-readable description" }
}Single audio payload ≤ 50MB. Recommended: 16 kHz / mono / 16-bit.
Create session: POST /v1/call with function name start_stream_eval
Response returns session_id / ws_url / resume_token.
WebSocket push: connect ws_url, wait for {"type":"ready"}, then push binary audio frames.
You receive intermediate partials while streaming; send {"type":"stop"} at the end and catch the final result.
Fallback fetch (optional): if the WS dropped before the result arrived, call POST /v1/call again with get_stream_result.
start_stream_eval parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
| core_type | string | ✓ | Scoring type, e.g. en.sent.score / cn.sent.raw (see English & Chinese tool tables above) |
| ref_text | string | ✓ | Reference text |
| audio_type | string | — | Audio format: wav / mp3 (default mp3; wav recommended for streaming to skip codec overhead) |
| sample_rate | number | — | Sample rate, default 16000 |
| channel | number | — | 1 = mono (default) / 2 = stereo |
| sample_bytes | number | — | Bytes per sample, default 2 (16-bit) |
| accent | number | — | English accent: 1 = British / 2 = American / 3 = any (default) |
| age_group | string | — | Chinese cohort: adult (default) / child |
| rank | number | — | Scoring scale: 4 or 100 (default) |
WebSocket frame protocol
Client → Server
Binary — raw audio frames; no handshake, start sending once open{"type":"stop"} — end recording{"type":"ping"} — keepaliveServer → Client
ready — connection is live; begin sending audiointermediate — partial results in real timeresult — final scoring resultbackpressure — slow down; payload carries suggested_interval_mserror — structured code + message (see error codes)pong — keepalive replyJavaScript (browser / Node.js)
// 1. Create session
const resp = await fetch('https://fc.cloud.chivox.com/v1/call', {
method: 'POST',
headers: {
'Authorization': 'Bearer your_api_key',
'Content-Type': 'application/json',
},
body: JSON.stringify({
name: 'start_stream_eval',
arguments: {
core_type: 'en.sent.score',
ref_text: 'Hello world',
audio_type: 'wav',
sample_rate: 16000,
},
}),
});
const { result } = await resp.json();
// 2. WebSocket: push audio
const ws = new WebSocket(result.ws_url);
ws.onmessage = (ev) => {
const msg = JSON.parse(ev.data);
switch (msg.type) {
case 'ready': startRecording(); break;
case 'intermediate': console.log('partial', msg.data); break;
case 'result': console.log('final', msg.data); ws.close(); break;
case 'backpressure': console.warn('slow down ~', msg.suggested_interval_ms, 'ms'); break;
case 'error': console.error(msg.code, msg.message); break;
}
};
function onAudioChunk(pcm) {
if (ws.readyState === WebSocket.OPEN) ws.send(pcm); // raw binary
}
function stopRecording() {
ws.send(JSON.stringify({ type: 'stop' }));
}Android (OkHttp · Kotlin)
val wsUrl = "ws://your-host:8080/ws/eval/$sessionId"
val client = OkHttpClient()
val req = Request.Builder()
.url(wsUrl)
.addHeader("Authorization", "Bearer ${apiKey}")
.build()
val ws = client.newWebSocket(req, object : WebSocketListener() {
override fun onMessage(ws: WebSocket, text: String) {
val msg = JSONObject(text)
when (msg.getString("type")) {
"ready" -> startRecording()
"intermediate" -> updateUI(msg.getJSONObject("data"))
"result" -> handleFinalResult(msg.getJSONObject("data"))
"error" -> handleError(msg.getString("code"), msg.getString("message"))
}
}
})
// Forward PCM bytes from your recorder
fun onAudioData(pcm: ByteArray) {
ws.send(ByteString.of(*pcm))
}
fun stop() {
ws.send("""{"type":"stop"}""")
}ws_url, stream PCM bytes, and route server messages by the type field.resume_token · 60-second reconnect window
When the client disconnects unexpectedly, the session is parked for 60 seconds on the server; reconnect with the resume_token returned at session creation:
GET ws://your-host:8080/ws/eval/{session_id}?resume={resume_token}
Authorization: Bearer <your_api_key>ready — keep streaming audio. The server issues a new resume_token and invalidates the old one immediately.Fallback fetch · get_stream_result
If the final result never arrived over WS, fetch it over HTTP:
curl -X POST https://fc.cloud.chivox.com/v1/call \
-H "Authorization: Bearer your_api_key" \
-H "Content-Type: application/json" \
-d '{
"name": "get_stream_result",
"arguments": {
"session_id": "stream-1713340800-a1b2c3",
"auto_stop": true,
"timeout": 30
}
}'| Parameter | Type | Required | Description |
|---|---|---|---|
| session_id | string | ✓ | Session ID returned by start_stream_eval |
| auto_stop | bool | — | Auto-send stop and wait for the result (default true) |
| timeout | number | — | Wait timeout in seconds (default 30) |
/ws/audio/ and cvx_fc /ws/eval/ against the same session; the two paths have separate session-ID namespaces.Call create_stream_session, then start_recording, then stop_recording. Requires SoX and a local stdio proxy.
| Parameter | Type | Required | Description |
|---|---|---|---|
| core_type | string | ✓ | e.g. en.word.score, cn.sent.raw |
| ref_text | string | ✓ | Reference transcript |
User: “Please score my pronunciation for the word Hello.”
The model calls create_stream_session(core_type="en.word.score", ref_text="Hello") and receives a session_id.
The model calls start_recording(session_id).
Microphone audio streams over WebSocket to the scorer.
User says “stop” (or equivalent). The model calls stop_recording(session_id).
Structured scores (overall, phones, fluency, …) return through MCP.
Provide audio_file_path, audio_base64, or audio_url.
Formats: mp3, wav, ogg, m4a, aac. Limits: up to 50MB per file; files larger than ~500KB may be transcoded to 16 kHz mono for transport.
| Tool | What it does | Typical use | Details |
|---|---|---|---|
| en_word_eval | Word scoring | Single-word pronunciation | Docs |
| en_word_correction | Word correction | Detect omissions/extra/wrong phones | Docs |
| en_vocab_eval | Vocabulary scoring | Multiple words in one clip | Docs |
| en_sentence_eval | Sentence scoring | Accuracy + fluency | Docs |
| en_sentence_correction | Sentence correction | Per-word feedback | Docs |
| en_paragraph_eval | Paragraph read-aloud | Long passage quality | Docs |
| en_phonics_eval | Phonics scoring | Letter-sound rules | Docs |
| en_choice_eval | Oral multiple choice | Constrained answers | Docs |
| en_semi_open_eval | Semi-open prompt | Scenario speaking | Docs |
| en_realtime_eval | Realtime read-aloud | Streaming feedback | Docs |
| Tool | What it does | Typical use | Details |
|---|---|---|---|
| cn_word_raw_eval | Character scoring | Hanzi pronunciation | Docs |
| cn_word_pinyin_eval | Pinyin scoring | Syllable-level | Docs |
| cn_sentence_eval | Phrase scoring | Short utterances | Docs |
| cn_paragraph_eval | Paragraph scoring | Long text | Docs |
| cn_rec_eval | Constrained recognition | Pick-one answers | Docs |
| cn_aitalk_eval | AI Talk scoring | Open dialog evaluation | Docs |
{
"overall": 85,
"accuracy": 82,
"pron": 88,
"fluency": { "overall": 78, "speed": 65, "pause": 2 },
"integrity": 95,
"details": [
{
"char": "hello",
"score": 85,
"phone": [
{ "phoneme": "h", "score": 90 },
{ "phoneme": "ɛ", "score": 82 }
]
}
]
}Common symptoms
| Symptom | Mitigation |
|---|---|
| Invalid or expired audio id | Re-upload; IDs expire ~5 minutes after upload. |
| Server busy | Retry with exponential backoff. |
| WebSocket disconnect | Proxy reconnects automatically; check network if persistent. |
HTTP status codes (function-calling mode)
| Status | Meaning |
|---|---|
| 400 | Malformed request (missing fields / wrong types) |
| 401 | Unauthenticated (missing Authorization header) |
| 403 | Forbidden (invalid key or no permission for the requested core_type) |
| 404 | Session not found (wrong or already-reaped session_id) |
| 409 | Invalid state for current operation (e.g. audio sent after stop) |
Structured streaming error codes
WebSocket error frames and the error payload of get_stream_result share the same set of codes, making it easy to dispatch client-side handling:
| Code | Meaning | Suggested action |
|---|---|---|
| SESSION_NOT_FOUND | Session does not exist | Recreate with start_stream_eval / create_stream_session |
| SESSION_EXPIRED | Session expired (idle > 60s) | Recreate session |
| INVALID_STATE | Operation not allowed in current state | Check call order (e.g. audio after stop) |
| INVALID_PARAMS | Invalid parameters | Check core_type / ref_text / sample rate |
| RESUME_INVALID | resume_token invalid or expired | Recreate session (each resume issues a fresh token) |
| AUDIO_TOO_LARGE | Audio payload exceeds 50MB | Compress or segment before retry |
| UPSTREAM_CONNECT | Scoring engine unreachable | Retry later; contact Chivox if persistent |
| UPSTREAM_TIMEOUT | Scoring engine timed out | Shorten audio / check network |
| UPSTREAM_EVAL_ERROR | Scoring engine returned an error | Inspect the message field |
| CAPACITY_FULL | Concurrent sessions are full | Back off and retry, or contact sales to expand |
These templates mirror the coaching prompts used inside the live demo. They follow the same five-part structure — persona / task / method / output format / tone — and pipe MCP tool results into the model via the placeholder {mcp_response}. Mount them as the system message, then pass the raw tool JSON in the user turn.
overall / dp_type / phonemes etc.) as {mcp_response}. Pre-summarising loses the very details the coach needs.1. Word · phoneme diagnosis
Tools: en.word.score / cn.word.raw. Lean on phonemes[].dp_type to find mispronounced phones.
You are a professional English pronunciation coach. Below is **word-level** exam-grade data from the Chivox MCP API (en.word.score): one headword, overall + pron scores, and a phoneme array with per-phoneme score and dp_type.
## Task
Write a **learner-facing** diagnosis of this single word. Tie every claim to phoneme scores (cite IPA symbols).
## Method
- Group phonemes by dp_type; treat **mispron** first, then borderline **normal** scores below 70 if they explain the word score.
- For each problem phoneme: typical L1 interference, what it sounds like now, one concrete articulation fix (jaw, tongue, airstream, lip rounding).
- Respect lexical stress (ˈ): mis-stress counts as a rhythm issue even when individual phones look fine.
- Cap at four numbered issues, ordered weakest → strongest impact on intelligibility.
## Output
Short Markdown: optional one-line summary, then numbered items, then a single-line **🎯 Priority** ranking.
## Tone
Friendly, concrete, actionable — no empty cheerleading.
## MCP payload
```json
{mcp_response}
```2. Sentence · prosody + stress
Tools: en.sent.score. Open with a compact scoreboard, then walk phones → words → prosody.
You are a professional English pronunciation coach. Below is **sentence-level** exam-grade data from Chivox MCP (en.sent.score): refText, overall, pron sub-scores (accuracy / integrity / fluency / rhythm), optional speed (WPM), and details[] with per-token scores, dp_type and optional phonemes[].
## Task
Open with a compact **scoreboard line** (overall + sub-scores + speed if present), then move from **phones → words → prosody**.
## Checklist (cover each; say "none material" if not applicable)
1. **Segmental**: for each token with score < 70 or dp_type ≠ normal, open phonemes when present; name the likely error (e.g. /ð/ realised as /d/) and one fix.
2. **Omissions / insertions**: flag dp_type omit / insert and how they affect integrity.
3. **Prosody**: use rhythm + stressed syllables in mispronounced words — misplaced lexical stress, flat declarative intonation, choppy chunking.
4. **Fluency vs accuracy**: if fluency is high but accuracy low, say so explicitly so the learner knows what to prioritise.
## Output format
Markdown subheads such as **Pronunciation**, **Prosody**, **Study priority** (one short ranked line, highest ROI first).
## MCP payload
```json
{mcp_response}
```3. Paragraph · phoneme families + integrity
Tools: en.pred.score. Cluster recurring errors into "phoneme families" instead of replaying every word.
You are a professional English pronunciation coach. Below is **paragraph-level** data from Chivox MCP (en.pred.score): long refText, overall, pron sub-scores, optional speed / integrity, and many details[] rows (tokens may repeat across clauses; some rows may be empty char for omit).
## Task
Diagnose the **whole passage in context**:
(a) completeness / skipping,
(b) recurring **phoneme families**,
(c) rhythm & chunking,
(d) how sub-scores agree or disagree.
## Checklist
1. **Integrity / dp_type**: list every omit / insert with clause context; explain why function words disappear in long reads.
2. **Segmental clusters**: group weak tokens by shared phoneme issue (/θ/ chain, /e/ chain, /ɒ/, weak stress on ˈe…); cite example words from details.
3. **Prosody**: use rhythm + punctuation — flag over-pausing inside phrases, or missing boundaries before new clauses.
4. **Global coherence**: if fluency looks fine but accuracy is weak (or vice versa), call it out and say what that implies for practice design.
## Output format
Markdown sections such as **Summary**, **Completeness**, **Pronunciation (families)**, **Rhythm**, **Study priorities** (≤ 4 bullets, most impactful first).
## MCP payload
```json
{mcp_response}
```4. Semi-open · IELTS-style 4-axis rubric
Tools: en.pqan.score. Combine learner transcript + 4 sub-scores into a band estimate plus next-band upgrades.
You are an **IELTS-style / classroom speaking** coach. Below is **semi-open** data from Chivox MCP (en.pqan.score): prompt refText, learner transcript, overall, four rubric scores (grammar / content / fluency / pron), speed (WPM if present), and structured issues[] flags.
## Task
Give a **rubric-grounded** diagnosis: estimate a 0–100 overall band in plain language, then explain **each dimension** with **quoted snippets** from the transcript (short phrases, not the whole text). End with **3–5 concrete upgrades** to reach the **next band** (sentence stems / patterns, not vague "practice more").
## Weights (communicate these to the learner)
- Content ~30% — idea depth, elaboration, specificity
- Grammar ~25% — range, agreement, subordination
- Fluency ~25% — pace, pausing, fillers, cohesion
- Pronunciation ~20% — segmentals + intelligibility
## Cross-dimension checks
- If content is the lowest score, prioritise elaboration templates before micro-pron drills.
- If fluency lags with low WPM, separate planning pauses from articulation issues.
- If pron flags specific phones, tie them to actual words in the transcript.
## Output format
Markdown: **Band estimate**, a compact **score table**, then **"To reach the next band"** with numbered pattern upgrades.
## MCP payload
```json
{mcp_response}
```5. Mandarin · tone / erhua / sandhi
Tools: cn.word.raw / cn.sent.raw / cn.pred.raw. Tone errors carry the highest weight and need a dedicated pass.
You are a professional Mandarin speaking coach. Below is sentence-level data from Chivox MCP (cn.sent.raw): whole-sentence scores plus per-character details, optional phonemes, and fluency / rhythm fields.
## What matters in Chinese evaluation
- Tone errors carry the highest weight — wrong tone can change meaning.
- dp_type=mispron often comes from wrong tone, retroflex vs alveolar mix-ups, or front / back nasal confusion.
- Correct pinyin with wrong tone still sounds like a "foreign accent".
## Checklist (say "none notable" if absent)
1. **Erhua**: when phonemes include r(儿化), confirm the syllable is read as one beat (mǎr), not "main syllable + separate 儿".
2. **Neutral tone**: reduplications, suffixes, sentence-final particles — second syllable short, light, no full tone contour.
3. **Sandhi**: 3rd-tone chains, 不 / 一 + following tone — match Standard Mandarin rules.
4. **Prosody**: use rhythm + comma positions to flag pauses that are too long or too choppy; suggest chunk boundaries.
## Output
Markdown subheads: **Summary / Erhua / Neutral tone / Sandhi / Rhythm / Study priorities** (1–3 bullets, biggest impact on natural Mandarin first).
## MCP payload
```json
{mcp_response}
```6. Word / sentence drills
Run as the second pass, after the diagnosis exists, so the model targets only the named weaknesses — no generic packs.
You are still the same learner's coach. The **second-pass diagnosis** already lists weak phonemes — generate **micro-drills** that map 1:1 to those weaknesses (no generic packs).
## Output JSON (single valid JSON array only; no prose outside it)
Root must be an array of objects, each with category (string), icon (one emoji), and items (array of { "label", "content" }):
```json
[
{ "category": "Phoneme drills", "icon": "🎯", "items": [ { "label": "…", "content": "…" } ] },
{ "category": "Awareness", "icon": "🗣️", "items": [ { "label": "…", "content": "…" } ] },
{ "category": "Whole word", "icon": "📝", "items": [ { "label": "…", "content": "…" } ] }
]
```
## Rules
- **Phoneme drills**: for each mispron phoneme from the diagnosis, at least one minimal pair or near-minimal pair AND one short tongue-twister or repetition line.
- **Awareness**: one item that targets stress or jaw / tongue posture if stress was weak.
- **Whole word**: 2–3 natural sentences that include the headword in different positions (initial / medial / final).
- All content strings must be speakable aloud; no nested JSON inside strings.
## Context
Previous diagnosis: {previous_diagnosis}
Raw MCP:
```json
{mcp_response}
```7. Paragraph · chunk map + metacognition
Lift drills from "phone" to "phrase / passage / record-replay loop". Pairs naturally with diagnosis #3.
You are still the same learner's coach. The **second-pass paragraph diagnosis** already names omission spots and phoneme families — build **passage-length** practice that chains those families instead of isolated minimal pairs only.
## Task
Return drills that move sound → phrase → paragraph: family chains, **chunk maps** that glue weak function words, and a **metacognitive** record–replay step.
## Output JSON (single valid JSON array only; no prose outside it)
```json
[
{ "category": "Phoneme families", "icon": "🎯", "items": [ { "label": "…", "content": "…" } ] },
{ "category": "Chunking", "icon": "🎵", "items": [ { "label": "…", "content": "…" } ] },
{ "category": "Metacognition", "icon": "🧠", "items": [ { "label": "…", "content": "…" } ] }
]
```
## Rules
- **Phoneme families**: at least three items total, grouping minimal pairs / word chains that reuse the diagnosed families (/θ/, /e/, /ɒ/…).
- **Chunking**: include one bracketed chunk map of the passage (use [ ] and / // with rough pause hints in seconds) and one item that targets any omitted preposition / function word by embedding it inside a phrase (not drilling it alone).
- **Metacognition**: one item that prescribes record → replay → compare to MCP details for N passes (N explicit, 2–3).
- All strings are plain display text; no nested JSON.
## Context
Previous diagnosis: {previous_diagnosis}
```json
{mcp_response}
```8. Weekly report
Aggregate the past N days of MCP results before sending; let the LLM produce a report that students or parents can read.
You are an English speaking learning advisor. Below are the last 7 days of Chivox MCP evaluation history for this learner (chronological). Each item carries question type, coreType, overall, sub-scores, and a weak_phonemes summary.
## Task
Produce a **weekly report** for the learner + parent / teacher. Emphasise **trend** and **next steps**, not a play-by-play of every score.
## Must include
1. **Overall trend**: improved / flat / regressed (one sentence + brief evidence)
2. **Dimension deltas**: any of accuracy / fluency / rhythm / integrity / speed with data — "this week avg vs last week"
3. **Highlights**: question type / phoneme / sentence pattern with the biggest jump
4. **Sticky issues**: any phoneme or dimension that stayed below 70 for ≥ 3 sessions
5. **Next week plan**: 3 concrete actions (each tied to one MCP tool or drill type)
6. **Closing**: 1–2 lines of genuine encouragement — no platitudes
## Style
- A focused but warm private tutor's voice
- Use the placeholder {student_name} for the learner's name
- Round all numbers to integers
## History
```json
{history}
```9. Error attribution · JSON summary
For backend aggregation / dashboards: have the LLM compress one MCP response into a structured row that's easy to store and chart.
You are an evaluation-result classifier. Below is one full MCP response. Extract the key signals and **return ONLY valid JSON** (no surrounding prose).
## Schema
```json
{
"core_type": "string, e.g. en.sent.score",
"overall": 0,
"sub_scores": { "accuracy": 0, "integrity": 0, "fluency": 0, "rhythm": 0 },
"weak_phonemes": [
{ "ipa": "string", "avg_score": 0, "occur": 0, "example_word": "string" }
],
"omissions": [{ "after": "string", "missing": "string" }],
"stress_issues": [{ "word": "string", "expected": "string", "actual": "string" }],
"primary_issue_tag": "phoneme|stress|fluency|integrity|content|none",
"next_action_hint": "string, ≤10 words, e.g. \"drill /θ/ minimal pairs\""
}
```
## Rules
- weak_phonemes: only avg_score < 70, max 5 items, ordered by occur desc.
- Do not echo refText, no markdown.
- Use null or empty arrays for missing fields; never drop a key.
## MCP payload
```json
{mcp_response}
```chat.completions + dynamic tools as an example: round 1 the LLM picks en.sent.score; you take that tool message verbatim and slot it into any diagnosis template above as {mcp_response}, send it as the next user turn, then pass that diagnosis into a drills template for the third turn. Three calls give you the full evaluate → diagnose → practice loop.| Path | Method | Purpose |
|---|---|---|
| /upload | POST | Audio upload (used by MCP-mode file evaluation) |
| /mcp | POST | MCP mode · JSON-RPC (Streamable HTTP) |
| /ws/audio/{session_id} | WS | MCP mode · streaming audio WebSocket |
| /v1/functions | GET | Function-calling mode · list all scoring functions |
| /v1/call | POST | Function-calling mode · one-shot eval / create stream session / fetch stream result |
| /ws/eval/{session_id} | WS | Function-calling mode · streaming audio WebSocket |
| /health | GET | Health check |
| Item | Default | Notes |
|---|---|---|
| Scratch storage | 500MB | Temporary audio cache |
| Queue depth | 10 | Pending jobs |
| Concurrency | 3 | Parallel scoring workers |
| Audio TTL | 5 min | Expires if not scored |
| Max upload | 50MB | Per file |
These defaults describe technical guardrails on the hosted endpoint. Commercial usage (concurrency tiers and call volume) follows the Membership plans page — the two are independent dimensions.