Pipecat produced replies. Our local audio setup still took too long.
Nine offline audio runs completed. The eight warm runs reached first generated audio at a median of 2.9 seconds, before playback or telephony.
Read the story ↗
The test bench / 9 field reports
Looks good in the README.
What happens in use?
We try a specific task with a tool or repo, and keep the evidence. Here’s what worked, what didn’t, and what a small test can actually tell you.
01 / The setup
The version, task and conditions.
02 / The result
Measurements you can inspect.
03 / The limits
What we didn’t establish.
Our website-only AI agent accepted a fake cancellation policy
Sixty answers from one AnythingLLM configuration revealed repeatable failures, including a user message that overrode the real cancellation policy.
Read the story ↗
The browser automation worked. The token savings are still unproven.
A signed-in extension found an existing HeyGen output. That proves a useful task worked; it does not establish cheaper browser automation.
Read the story ↗
The extra design checklist lost five of six blind comparisons
Six matched pairs gave the existing setup five wins. Here is how the blind review worked and why we are skipping this particular extra checklist.
Read the story ↗
Perchance gave us 11 usable images out of 15. That is a narrow win.
Sam accepted 11 of 15 generic backgrounds. The paid comparison is unfinished, and the access terms keep this out of our automated pipeline.
Read the story ↗
Five AI ideas tested: what earned another look
A bonus field-notes series on image generation, design prompts, browser automation, website-grounded answers and voice latency. Small tests, specific decisions, and the gaps still open.
Read the story ↗
DBX blocked our tested writes. One response flag still needed a closer look.
A disposable PostgreSQL database, synthetic records, and two read-only setups. The rows stayed unchanged, but a batch’s outer success flag did not tell the whole story.
Files included ↗
Impeccable made a busy page calmer. Here is the change we could measure.
One Shossip guide page, a clearer starting point, and a coverage note that moved from 2.59:1 to 7.35:1 contrast. A design case study with the before and after.
Files included ↗
Ponytail wrote less code. Our small test did not prove it was better.
A paired coding experiment produced a smaller diff and the same passing tests. The cost result is why we are keeping the verdict open.
Files included ↗