Autoblocks × Val Town

Every AI team eventually hits the same wall: the built‑in “accuracy” or “helpfulness” metrics just don’t cut it once real users—and real edge‑cases—show up. You need evaluations that understand your domain, your data, your definition of done.

Today we’re making that dead simple. 🪄

Autoblocks now pairs seamlessly with Val Town—the delightfully lightweight serverless playground—to let you spin up custom evaluators with nothing more than a URL and a few lines of JavaScript.

Why we built this

1. Your rules, your way - Regulated industries (hi, healthcare friends 👋) can’t rely on fuzzy generic scores. You need hard “pass/fail” logic and auditable reasoning.

2. No infra tax - Standing up a microservice just to grade a prompt feels… overkill. Val Town gives you an always‑on endpoint without touching Kubernetes.

3. Instant feedback loop - Kick off a simulation in Autoblocks, watch Val Town log the payload, return a score, and see the result light up in the UI—all in one coffee’s time.

How it works

1. Create a Val - Drop the code below into a new Val. (Yes, that’s the whole thing.)

// https://www.val.town/v/yourname/autoblocksWebhook
import type { Evaluation } from "@autoblocks/client/testing";

export default async function httpHandler(req: Request): Promise<Response> {
  if (req.method !== "POST") return Response.json({ error: "Use POST" }, { status: 405 });

  const { input, output } = await req.json();

  // TODO: your logic here — LLM-as-judge, regex checks, whatever
  const score = 0.9;

  const evalObj: Evaluation = {
    score,
    threshold: { gte: 0.8 },
    metadata: { notes

2. Paste the URL into Autoblocks - Head to Evaluators → Webhook → Settings, drop in your Val URL, add an auth header, hit Save.

3. Simulate away - Run an Agent Simulation. Autoblocks ships the full scenario + transcript to your Val; Val Town runs your code and hands back the verdict in milliseconds. The score, threshold, and metadata pop straight into the Evaluations tab.

That’s it—custom grading logic without spinning up a single server.

What you can build

• HIPAA‑grade phone‑call checks (Did the agent verify DOB before sharing PHI?)

• Multi‑turn tone audits (Flag any response with CAPS LOCK RAGE)

• Security red‑teamers (Catch prompt‑injections before they hit prod)

• Fun stuff (Give bonus points for using y’all in a reply—why not?)

If you can express it in code, you can evaluate it.

Get started

• Read the docs: Webhook Evaluators →

• Fork the sample Val: autoblocks/Autoblocks_Webhook_Evaluator

• Show us what you build: Tag @AutoblocksAI and @ValDotTown on X / LinkedIn—we love a good demo thread.

This integration turns Autoblocks into a choose‑your‑own‑rubric adventure. No waiting on us to ship the “perfect” metric. You’ve already got the perfect metric in your head; now you can encode it in a Val and see it live in Autoblocks in under five minutes.

Happy grading. 🚀