Every morning a duo of two AI models sorts a Gmail inbox. Jev judges each email quickly and cheaply. Claude runs the routine, steps in when in doubt and writes a summary. Emails that need attention get a label; nothing else is changed. Since October 2026 Microsoft-Decision-1 (MD1) has judged along in the shadow, so I can compare two models live.
A scheduled task in the Claude desktop app starts a new Claude session with fixed instructions. If the laptop is off, the task runs the next time the app opens.
The script triage.py daily reads the inbox of the past 14 days, with read-only access. For each conversation it keeps the sender, subject, date and the short preview Gmail itself shows, plus the earlier messages in that conversation. Full email bodies and attachments are not fetched.
For each conversation the script asks Jev six questions in a single request. Jev does not write text back: it returns a choice or a probability, so the code can carry on with it directly. A request takes about a quarter of a second.
| Question | Type of answer | Example |
|---|---|---|
| Which category? | Choice of 6 | newsletter · 0.97 sure |
| Is action needed? | Probability 0–1 | 0.91 |
| Is a reply expected? | Probability 0–1 | 0.06 |
| Security-sensitive? | Probability 0–1 | 0.12 |
| Is it a newsletter? | Probability 0–1 | 0.88 |
| How urgent, counted from today? | Score 0–3 | 2.2 (within a week) |
At the same time the script asks Microsoft-Decision-1 the same six questions, with the same thresholds, but through OpenRouter. MD1 gets no Claude review and decides nothing: its result is only compared with the final judgment of Jev and Claude. A request takes about 0.4 seconds. If OpenRouter is unreachable or the key is missing, the script skips the shadow and the rest of the run carries on as normal.
Fixed thresholds, set with a test on 243 labeled emails, decide what Jev may handle on its own. A probability below 0.3 counts as no, from 0.7 as yes. Everything in between, and a category with confidence below 0.7, goes to Claude.
Usually it is a handful of conversations per day. Claude decides each doubtful case and writes down a short reason. If the preview is not enough, Claude reads the full email through the Gmail connector, up to a maximum of 6 per run. That is the only moment a full email body is read.
triage.py finalize combines Jev's answers with Claude's decisions. The result is a summary and a list of which labels go where, plus an overview of the emails on which MD1 judges differently.
There are three kinds of labels: Action for emails that need something done, Security for login alerts, codes and possible phishing, and News for periodic subscription mail. Urgent emails also get Action.
The name of the label shows who decided. Jev/Action means Jev was confident. Claude/Action means Jev was unsure and Claude decided; these are nested labels under Claude. A thread never gets both: if Claude later decides differently from Jev, the old label is replaced. Nothing is ever archived, deleted, marked as read or sent.
Next to those come the shadow labels MD1/Action, MD1/Security and MD1/News (nested labels under MD1). If a thread has both Jev/Action and MD1/Action, the models agree; if only one of the two is on it, the judgments differ. MD1 labels are only material for comparison and decide nothing.
One rule for all labels: if a label under Jev/, Claude/ or MD1/ sits on an email you have archived or deleted, Claude removes it. That includes newsletters. A new label in those three groups therefore does not need to be added separately. Labels outside those groups are left alone, and so is any email still in the inbox.
The run ends with a short summary in the app, divided into: important today, action needed, security, and labels that can probably come off. It also says how many newsletters were labeled and how many labels came from Claude. It also says on how many threads MD1 judged differently from Jev and Claude. A push notification is sent only if something new and urgent turns up.
A so-called System One model: it answers short, well-defined questions with a choice or a probability, without writing text. That makes it fast, and a whole inbox costs less than a cent. The probabilities are usable: when Jev says 0.9, it is right about 9 times out of 10.
A large language model that can reason, read full emails and write in plain language. Claude runs the routine, decides in cases of doubt, applies the labels and writes the summary. Claude also built the code and set up the test.
Microsoft-Decision-1 is a decision model from Microsoft that, like Jev, answers short questions with a choice or a probability instead of text. Here it judges along in the shadow: it decides nothing, but shows where a second model would choose differently. The script reaches the model through OpenRouter.
Microsoft-Decision-1 (MD1) is a so-called decision model, like Jev. It does not write text. It answers a fixed question with a choice from options, a yes/no probability or a score on an ordered scale, with a probability for every answer. Microsoft built it for routing, classifying, prioritizing and verifying: work where software has to carry on directly with the answer.
The question format is very similar to Jev's (a state and questions of the type choice, noul and score). That means exactly the same questions can go to both models.
| Feature | Microsoft-Decision-1 |
|---|---|
| Provider | Microsoft |
| Base | Qwen3.5-9B, post-trained by Microsoft for decisions |
| Released | October 9, 2026 |
| Input | text only, up to 32,000 tokens |
| Output | no text and no explanation, only choices and probabilities |
| Price | $0.042 per million input tokens; output is free (like Jev) |
| Available through | Microsoft Foundry (Azure) and OpenRouter |
Both models got the same emails and the same questions, first on the development set and then on the final test with 116 new emails. Claude is the judge.
| Measurement (final test) | Jev | MD1 |
|---|---|---|
| Same category as Claude | 89% | 77% |
| Same, with a sharpened category question | 87% | 82% |
| Yes/no questions: correct per question (five questions) | 89–99% | 85–97% |
| Calibration error (ECE, average; lower is better) | 0.13 | 0.08 |
| Urgency after redesigning the question (MAE) | 0.18 | 0.20 |
| When the models disagree on the category (19 emails): who was right | 84% | 11% |
| Cost per 10,000 emails | $0.63 | $0.44 |
| Median response time per request | 220 ms | 430 ms |
Jev does better on category. On the yes/no questions and on urgency they are close, and MD1's probabilities are slightly better calibrated. MD1 is cheaper (fewer tokens per email) but twice as slow; the next part explains why. The sharpened category question helped MD1 (77% to 82%) and cost Jev a few percent: that question was built with Jev, so it does not automatically suit another model. The numbers are small, so a difference of one or two emails is noise.
That is why MD1 runs as a shadow and not as a replacement: Jev keeps the main role, and for a few weeks I watch live the emails on which the two differ.
OpenRouter is a service that gives access to many AI models from different providers through one API and one key. You add credit at OpenRouter, and OpenRouter forwards each request to the provider that runs the model. For decision models OpenRouter has its own Decisions API, for now still in alpha, with the same question format as Jev's.
triage.py sends the questions for MD1 with a plain HTTP call to OpenRouter (model microsoft/microsoft-decision-1), next to the TypeSafe SDK that Jev uses.api.typesafe.ai): the Python script that Claude Code runs sends the questions to that API with the Python SDK typesafe-sdk. The API key is stored locally on the laptop, outside the code. Cost: $0.042 per million input tokens; output is free.openrouter.ai): the script sends the same questions with a plain HTTP call to OpenRouter's Decisions API (microsoft/microsoft-decision-1). The API key is stored locally outside the code, like the TypeSafe one. Cost: $0.042 per million input tokens; output is free..claude/settings.local.json: the run may execute exactly two commands and perform a few Gmail actions, without anyone having to click.gmail.readonly), through google-api-python-client and google-auth-oauthlib.triage.py (the daily run), gmail_api.py (access), cleanup.py (cleanup advice) a test set in jevbench and compare_models.py (comparison of the models).There are two ways to reach Gmail, and the run uses both, each for what suits it best. Claude's Gmail connector works right after logging in to Claude, with no setup of your own. The Gmail API requires your own app at Google, but it is faster, more predictable and read-only.
| Gmail API | Claude Gmail connector | |
|---|---|---|
| Used for | Fetching the inbox (step 2) and, once, the cleanup advice over the whole inbox. | Reading a full email when in doubt (step 6) and applying labels (step 8). |
| Speed and volume | 45 conversations in about ten seconds, 3,079 emails in about five minutes. | At most 50 conversations at a time, and everything passes through the conversation with Claude. That is slower and costs Claude tokens. |
| Where the data ends up | The script writes it straight to a file. Nothing is retyped. | The result lands in the conversation with Claude. For further processing Claude had to copy it into a file, which can introduce errors. |
| Approvals in the morning | One fixed command (triage.py daily). It is in the permission rules, so the run goes through without clicks. | With the connector as the source, the run created its own helper scripts with changing names every day. Every new command asked for manual approval. |
| What it may do | Read only (gmail.readonly). By definition the script cannot change anything. | Can also apply labels, delete and send. The instructions limit it to reading and labels under Jev/, Claude/ and MD1/. |
| Setup and maintenance | Own Google Cloud project with an OAuth app, published once. Grant consent again after a password change. | No setup: connect once in Claude. Anthropic maintains the connection. |
Why the labels still go through the connector: that keeps the own script strictly read-only. Otherwise the Google app would need broader permissions to be allowed to modify Gmail.
credentials.json stays local, outside the code.There was no website or domain of my own yet. GitHub Pages solves all four requirements for free.
<username>.github.io is published automatically by GitHub at https://<username>.github.io/, with https.github.io itself is rejected by Google, because it is shared by everyone. The own subdomain <username>.github.io is accepted as an authorized domain.After that the branding could be filled in and the app published. After one new consent, a token came out without the 7-day expiry. Later the same approach can be applied to a domain of my own such as scher.nl, with a DNS verification instead of the tag.
The routine now runs as a scheduled task in the Claude desktop app on my own laptop. Claude Code can also run scheduled tasks in Anthropic's cloud. The run would then happen on time, even when the laptop is off. That switch has not been made yet, and a number of points need to be investigated first.
The run only happens when the app is open. If the laptop is off at 07:30, the run follows at the next start. Code, Python environment, API key, Gmail token, review memory and all email data are on the laptop itself.
The run starts at the fixed time in an Anthropic environment, independent of the laptop. Every run starts in a clean environment and fetches code and settings again.
api.typesafe.ai and the Gmail API?The advantage is reliability: the summary is ready on time every morning. As long as the laptop usually starts later in the morning, the desktop version with a catch-up run is a safe and simple choice.
Jev/, Claude/ and MD1/.First Claude judged 127 emails by hand. Jev got the same emails and the same questions. Then came a final test on 116 older emails that neither had seen before.
| Measurement | Result |
|---|---|
| Same category as Claude | 94% · final test 91% |
| Wrongly "urgent" after improving the question | 15 → 1 (final test) |
| Urgency with today's date, 25 date tests | 12 → 1 error |
| Newsletter: correct where Jev was confident (243 emails) | 95% · ± 18% to Claude |
| Share Jev handles without Claude | ± 70–95% per day |
| Cleanup advice over 3,079 emails in the inbox | 32 seconds · ± $0.04 |
The comparison is against Claude as the judge, not against an absolute truth. In a few of the differences Jev was right rather than Claude.