Penny 1 guide

Ollama, from Hugging Face

ollama pull hf.co/pocketsflow/penny-1:Q4_K_M
# then use the model name "hf.co/pocketsflow/penny-1:Q4_K_M" (raw prompts, see 9. Prompt format)

Ollama, from a file

curl -L -o pocketsflow-penny-1-q4_k_m.gguf https://d372ebr33sbljd.cloudfront.net/research/pocketsflow-penny-1/pocketsflow-penny-1-q4_k_m.7437e57448.gguf
curl -L -o Modelfile https://d372ebr33sbljd.cloudfront.net/research/pocketsflow-penny-1/Modelfile.c1ccb000bf
ollama create penny -f Modelfile

LM Studio

1. Search "pocketsflow/penny-1" in Discover, or download pocketsflow-penny-1-q4_k_m.gguf (link above).
2. lms import pocketsflow-penny-1-q4_k_m.gguf   (or drag it into My Models)
3. Developer tab → load the model → Start server (http://localhost:1234)

llama.cpp

llama-server -m pocketsflow-penny-1-q4_k_m.gguf --port 8080 -c 2048

Client (pick one)

curl -LO https://d372ebr33sbljd.cloudfront.net/research/pocketsflow-penny-1/penny.005d5a5a60.py     # Python 3.9+, no dependencies
curl -LO https://d372ebr33sbljd.cloudfront.net/research/pocketsflow-penny-1/penny.72aeb5f0db.mjs    # Node 18+, Deno, Bun, browsers

Python

from penny import Penny

penny = Penny()                                   # Ollama on localhost:11434
# Penny(host="http://localhost:1234", api="openai")   # LM Studio
# Penny(host="http://localhost:8080", api="openai")   # llama-server

result = penny.yes_no(
    {"seller_age_days": 9, "payout": "$4,800", "disputes_30d": 3},
    "Should we release this payout now?",
)
if result["confidence"] >= 0.8:
    release() if result["yes"] else hold()
else:
    send_to_review_queue(result)

JavaScript / TypeScript

import { Penny } from "./penny.mjs";

const penny = new Penny();   // { host, api: "openai" } for LM Studio / llama-server
const { choice, confidence } = await penny.choose(order,
  "What should we do with this refund request?",
  ["Refund in full", "Offer store credit", "Decline politely"]);

curl (raw)

curl http://localhost:11434/api/generate -d '{
  "model": "penny", "raw": true, "stream": false,
  "logprobs": true, "top_logprobs": 20,
  "options": {"num_predict": 1, "temperature": 0},
  "prompt": "<state>\nPayout $4,800, seller 9 days old, 3 disputes in 30 days\n</state>\n<question type=\"yes_no\">\nShould we release this payout now?\nA. Yes\nB. No\n</question>\n<answer>\n"
}'
# read logprobs[0].top_logprobs: the probability of "A" vs "B"

Routing pattern

THRESHOLD = {"refund": 0.9, "route_ticket": 0.6}

def decide(kind, state, question, options):
    r = penny.choose(state, question, options)
    if r["confidence"] >= THRESHOLD[kind]:
        return r["choice"]                      # automated
    review_queue.add(kind, state, suggestion=r) # a person decides, with Penny's suggestion
    return None

Docker (CPU server)

docker run -d -p 11434:11434 -v ollama:/root/.ollama --name ollama ollama/ollama
docker cp pocketsflow-penny-1-q4_k_m.gguf ollama:/root/
docker cp Modelfile ollama:/root/
docker exec ollama sh -c "cd /root && ollama create penny -f Modelfile"

<state>
{anything: JSON, prose, a message, a table row}
</state>
<question type="choice">
What should we do with this refund request?
A. Refund in full
B. Offer store credit
C. Decline politely
</question>
<answer>