scher.nl
Personal project · September 2026

Gmail triage with Jev

Every morning a duo of two AI models sorts a Gmail inbox. Jev judges each email quickly and cheaply. Claude runs the routine, steps in when in doubt and writes a summary. Emails that need attention get a label; nothing else is changed. Since October 2026 Microsoft-Decision-1 (MD1) has judged along in the shadow, so I can compare two models live.

Emails per run
± 45 threads
Duration
± 1 minute
Jev cost
± $0.002 per day
Labels
3 kinds
Shadow model
MD1
In one line

From inbox to label

From inbox to label The Claude routine starts daily at 07:30. The Python script fetches the inbox from Gmail through the Gmail API, Jev judges each email, the policy in code decides what counts as doubt, Claude decides the doubtful cases and puts labels back in Gmail through the Claude Gmail connector. In parallel, the script sends the same questions through OpenRouter to Microsoft-Decision-1 (MD1), which judges along in the shadow without review; Claude also applies the MD1 labels. Claude routinedaily 07:30 Gmailinbox API Python scripttriage.py API Jev6 questions Policy in codesure or doubt? Claudedecide doubt apply labels · via Claude Gmail connector API OpenRouterAPI gateway API MD16 questions MD1 labelsno review shadow track alongside Jev
The process flow

One morning run, step by step

  1. 1

    Start at 07:30

    Claude app

    A scheduled task in the Claude desktop app starts a new Claude session with fixed instructions. If the laptop is off, the task runs the next time the app opens.

    Claude desktop · scheduled taskClaude Code
  2. 2

    Fetch the inbox

    Python scriptvia Gmail API · read-only

    The script triage.py daily reads the inbox of the past 14 days, with read-only access. For each conversation it keeps the sender, subject, date and the short preview Gmail itself shows, plus the earlier messages in that conversation. Full email bodies and attachments are not fetched.

    Gmail APIOAuth · gmail.readonlygoogle-api-python-client
  3. 3

    Jev judges every email

    Jev

    For each conversation the script asks Jev six questions in a single request. Jev does not write text back: it returns a choice or a probability, so the code can carry on with it directly. A request takes about a quarter of a second.

    QuestionType of answerExample
    Which category?Choice of 6newsletter · 0.97 sure
    Is action needed?Probability 0–10.91
    Is a reply expected?Probability 0–10.06
    Security-sensitive?Probability 0–10.12
    Is it a newsletter?Probability 0–10.88
    How urgent, counted from today?Score 0–32.2 (within a week)
    TypeSafe API (https)Jev 1.13typesafe-sdk (Python)Choice · Noul · Score
  4. 4

    MD1 judges along in the shadow

    MD1via OpenRouter

    At the same time the script asks Microsoft-Decision-1 the same six questions, with the same thresholds, but through OpenRouter. MD1 gets no Claude review and decides nothing: its result is only compared with the final judgment of Jev and Claude. A request takes about 0.4 seconds. If OpenRouter is unreachable or the key is missing, the script skips the shadow and the rest of the run carries on as normal.

    OpenRouter Decisions API (https)Microsoft-Decision-1Choice · Noul · Score
  5. 5

    Policy in code decides

    Python script

    Fixed thresholds, set with a test on 243 labeled emails, decide what Jev may handle on its own. A probability below 0.3 counts as no, from 0.7 as yes. Everything in between, and a category with confidence below 0.7, goes to Claude.

    Memory. A conversation that Claude has already reviewed, and in which nothing new has arrived since, does not need to go to Claude again. The code calculates dates itself. If an email mentions an appointment on October 3, it becomes urgent from 7 days before and calms down again afterwards.
    Python 3.14 · asynciolocal JSON files
  6. 6

    Claude reviews the doubtful cases

    Claudevia Claude Gmail connector

    Usually it is a handful of conversations per day. Claude decides each doubtful case and writes down a short reason. If the preview is not enough, Claude reads the full email through the Gmail connector, up to a maximum of 6 per run. That is the only moment a full email body is read.

    Claude OpusGmail connector (MCP)
  7. 7

    Wrap up

    Python script

    triage.py finalize combines Jev's answers with Claude's decisions. The result is a summary and a list of which labels go where, plus an overview of the emails on which MD1 judges differently.

  8. 8

    Apply labels

    Claudevia Claude Gmail connector

    There are three kinds of labels: Action for emails that need something done, Security for login alerts, codes and possible phishing, and News for periodic subscription mail. Urgent emails also get Action.

    The name of the label shows who decided. Jev/Action means Jev was confident. Claude/Action means Jev was unsure and Claude decided; these are nested labels under Claude. A thread never gets both: if Claude later decides differently from Jev, the old label is replaced. Nothing is ever archived, deleted, marked as read or sent.

    Next to those come the shadow labels MD1/Action, MD1/Security and MD1/News (nested labels under MD1). If a thread has both Jev/Action and MD1/Action, the models agree; if only one of the two is on it, the judgments differ. MD1 labels are only material for comparison and decide nothing.

    Gmail labelsGmail connector (MCP)
  9. 9

    Cleanup

    Claudevia Claude Gmail connector

    One rule for all labels: if a label under Jev/, Claude/ or MD1/ sits on an email you have archived or deleted, Claude removes it. That includes newsletters. A new label in those three groups therefore does not need to be added separately. Labels outside those groups are left alone, and so is any email still in the inbox.

    Gmail connector (MCP)
  10. 10

    Summary

    Claude

    The run ends with a short summary in the app, divided into: important today, action needed, security, and labels that can probably come off. It also says how many newsletters were labeled and how many labels came from Claude. It also says on how many threads MD1 judged differently from Jev and Claude. A push notification is sent only if something new and urgent turns up.

Who does what

Three models, each with its own job

Jev · TypeSafe

Fast judgments in bulk

A so-called System One model: it answers short, well-defined questions with a choice or a probability, without writing text. That makes it fast, and a whole inbox costs less than a cent. The probabilities are usable: when Jev says 0.9, it is right about 9 times out of 10.

Claude · Anthropic

Directing, settling doubt, explaining

A large language model that can reason, read full emails and write in plain language. Claude runs the routine, decides in cases of doubt, applies the labels and writes the summary. Claude also built the code and set up the test.

MD1 · Microsoft

A shadow judgment for comparison

Microsoft-Decision-1 is a decision model from Microsoft that, like Jev, answers short questions with a choice or a probability instead of text. Here it judges along in the shadow: it decides nothing, but shows where a second model would choose differently. The script reaches the model through OpenRouter.

The shadow model

Microsoft-Decision-1: what is it?

Microsoft-Decision-1 (MD1) is a so-called decision model, like Jev. It does not write text. It answers a fixed question with a choice from options, a yes/no probability or a score on an ordered scale, with a probability for every answer. Microsoft built it for routing, classifying, prioritizing and verifying: work where software has to carry on directly with the answer.

The question format is very similar to Jev's (a state and questions of the type choice, noul and score). That means exactly the same questions can go to both models.

FeatureMicrosoft-Decision-1
ProviderMicrosoft
BaseQwen3.5-9B, post-trained by Microsoft for decisions
ReleasedOctober 9, 2026
Inputtext only, up to 32,000 tokens
Outputno text and no explanation, only choices and probabilities
Price$0.042 per million input tokens; output is free (like Jev)
Available throughMicrosoft Foundry (Azure) and OpenRouter

How MD1 does next to Jev

Both models got the same emails and the same questions, first on the development set and then on the final test with 116 new emails. Claude is the judge.

Measurement (final test)JevMD1
Same category as Claude89%77%
Same, with a sharpened category question87%82%
Yes/no questions: correct per question (five questions)89–99%85–97%
Calibration error (ECE, average; lower is better)0.130.08
Urgency after redesigning the question (MAE)0.180.20
When the models disagree on the category (19 emails): who was right84%11%
Cost per 10,000 emails$0.63$0.44
Median response time per request220 ms430 ms

Jev does better on category. On the yes/no questions and on urgency they are close, and MD1's probabilities are slightly better calibrated. MD1 is cheaper (fewer tokens per email) but twice as slow; the next part explains why. The sharpened category question helped MD1 (77% to 82%) and cost Jev a few percent: that question was built with Jev, so it does not automatically suit another model. The numbers are small, so a difference of one or two emails is noise.

That is why MD1 runs as a shadow and not as a replacement: Jev keeps the main role, and for a few weeks I watch live the emails on which the two differ.

The connection to MD1

OpenRouter: what is it and how is it used here?

OpenRouter is a service that gives access to many AI models from different providers through one API and one key. You add credit at OpenRouter, and OpenRouter forwards each request to the provider that runs the model. For decision models OpenRouter has its own Decisions API, for now still in alpha, with the same question format as Jev's.

How it is used here

  1. triage.py sends the questions for MD1 with a plain HTTP call to OpenRouter (model microsoft/microsoft-decision-1), next to the TypeSafe SDK that Jev uses.
  2. The OpenRouter key is stored locally in a file in the user folder, outside the code and outside the conversation with Claude, just like the TypeSafe key.
  3. OpenRouter reports in the answer which provider handled the request. In the measurements that was Azure.
  4. If a call fails, or the key is missing, the shadow is skipped. The triage with Jev and Claude simply carries on.
  5. Jev deliberately does not go through OpenRouter, even though it is available there too. It keeps going straight to TypeSafe, because that is the fastest.

Advantages

  • No Azure needed. Directly through Microsoft Foundry you need an Azure subscription, a project and a deployment. With OpenRouter one key with credit is enough.
  • One setup for more models. Trying another model means another model name in the same call.
  • The same price. According to OpenRouter, MD1 costs $0.042 per million input tokens there, the same as at Microsoft.

Disadvantages

  • Delay. There is an extra link between the script and the model. In the measurements a request to MD1 through OpenRouter took a median of 430 ms (p95 820 ms), against 220 ms (p95 550 ms) for Jev directly at TypeSafe: about twice as long. Part of that may be down to the model itself; that was not measured separately. For the daily run it hardly matters, because the requests run at the same time and the shadow is done within a few seconds. For anything where every millisecond counts it would be a nuisance.
  • One more party. Sender, subject and preview of every email also pass through OpenRouter and the provider behind it. They see that data.
  • An alpha endpoint. OpenRouter's Decisions API may still change. That is why a failure must never stop the rest.
  • Another key and credit to manage.
Technology

What is used

AI models

  • Jev 1.13 from TypeSafe. Jev is connected to the Claude Code routine through TypeSafe's API (api.typesafe.ai): the Python script that Claude Code runs sends the questions to that API with the Python SDK typesafe-sdk. The API key is stored locally on the laptop, outside the code. Cost: $0.042 per million input tokens; output is free.
  • Claude Opus from Anthropic, through Claude Code in the Claude desktop app.
  • Microsoft-Decision-1 from Microsoft, as the shadow model, through OpenRouter (openrouter.ai): the script sends the same questions with a plain HTTP call to OpenRouter's Decisions API (microsoft/microsoft-decision-1). The API key is stored locally outside the code, like the TypeSafe one. Cost: $0.042 per million input tokens; output is free.

Automation

  • Scheduled tasks in the Claude desktop app: daily at 07:30, or the next time the app starts.
  • Permission rules in .claude/settings.local.json: the run may execute exactly two commands and perform a few Gmail actions, without anyone having to click.
  • Gmail connector from Claude (MCP) for applying labels and reading full emails.

Google

  • Gmail API with read-only access (gmail.readonly), through google-api-python-client and google-auth-oauthlib.
  • Own OAuth app in Google Cloud (Google Auth Platform), published for personal use. This keeps the access from expiring every week.
  • Search Console to prove that the website with the privacy policy belongs to the owner.

Own code and data

  • Python 3.14 in a virtual environment: triage.py (the daily run), gmail_api.py (access), cleanup.py (cleanup advice) a test set in jevbench and compare_models.py (comparison of the models).
  • Local JSON files for the inbox snapshot, the runs, the review memory and the summaries. There is no server and no database.

Website

  • GitHub Pages for the homepage and privacy policy that Google requires before an app can leave the testing phase.

Gmail API or Claude Gmail connector?

There are two ways to reach Gmail, and the run uses both, each for what suits it best. Claude's Gmail connector works right after logging in to Claude, with no setup of your own. The Gmail API requires your own app at Google, but it is faster, more predictable and read-only.

Gmail APIClaude Gmail connector
Used forFetching the inbox (step 2) and, once, the cleanup advice over the whole inbox.Reading a full email when in doubt (step 6) and applying labels (step 8).
Speed and volume45 conversations in about ten seconds, 3,079 emails in about five minutes.At most 50 conversations at a time, and everything passes through the conversation with Claude. That is slower and costs Claude tokens.
Where the data ends upThe script writes it straight to a file. Nothing is retyped.The result lands in the conversation with Claude. For further processing Claude had to copy it into a file, which can introduce errors.
Approvals in the morningOne fixed command (triage.py daily). It is in the permission rules, so the run goes through without clicks.With the connector as the source, the run created its own helper scripts with changing names every day. Every new command asked for manual approval.
What it may doRead only (gmail.readonly). By definition the script cannot change anything.Can also apply labels, delete and send. The instructions limit it to reading and labels under Jev/, Claude/ and MD1/.
Setup and maintenanceOwn Google Cloud project with an OAuth app, published once. Grant consent again after a password change.No setup: connect once in Claude. Anthropic maintains the connection.

Why the labels still go through the connector: that keeps the own script strictly read-only. Otherwise the Google app would need broader permissions to be allowed to modify Gmail.

What you need for your own Gmail API connection

  1. A project in Google Cloud, with the Gmail API enabled.
  2. An OAuth consent screen (Google Auth Platform) for an app of type External, with an app name, a support email address and a contact address.
  3. An OAuth client of type Desktop app. The downloaded credentials.json stays local, outside the code.
  4. The first time, grant consent in the browser. The script then stores a token so it can work without a browser later.
  5. Publish. In the Testing status that token expires after 7 days, and an unattended morning run cannot log in again. After Publish app (status In production) it only expires after a password change, after 6 months without use, or when access is revoked.
What Google requires for publishing. The Publish app button stays grey as long as the branding has no homepage and no privacy policy, both on a domain listed under Authorized domains and which you have proven in Search Console to be yours. A link to terms of use is also requested. What Google does not require. For an app that only the owner uses, an official verification by Google is not needed. When granting consent, a notice then appears that the app is not verified, and the app may have at most 100 users. Uploading a logo does make verification mandatory, so that was left out.

Solved with GitHub Pages

There was no website or domain of my own yet. GitHub Pages solves all four requirements for free.

  1. Website: a repository named <username>.github.io is published automatically by GitHub at https://<username>.github.io/, with https.
  2. Homepage, privacy policy and terms of use: three simple pages describing what the app does, which data it reads, where that data goes and which services process it.
  3. Domain: github.io itself is rejected by Google, because it is shared by everyone. The own subdomain <username>.github.io is accepted as an authorized domain.
  4. Proving ownership: add a property of type URL prefix in Search Console, and put Google's verification tag on the homepage.

After that the branding could be filled in and the app published. After one new consent, a token came out without the 7-day expiry. Later the same approach can be applied to a domain of my own such as scher.nl, with a DNS verification instead of the tag.

Where the routine runs

Claude desktop app or the cloud?

The routine now runs as a scheduled task in the Claude desktop app on my own laptop. Claude Code can also run scheduled tasks in Anthropic's cloud. The run would then happen on time, even when the laptop is off. That switch has not been made yet, and a number of points need to be investigated first.

Now: Claude desktop app

The run only happens when the app is open. If the laptop is off at 07:30, the run follows at the next start. Code, Python environment, API key, Gmail token, review memory and all email data are on the laptop itself.

Possible: Claude in the cloud

The run starts at the fixed time in an Anthropic environment, independent of the laptop. Every run starts in a clean environment and fetches code and settings again.

Limits

What the system may not do

Tested before it went live

How well it works

First Claude judged 127 emails by hand. Jev got the same emails and the same questions. Then came a final test on 116 older emails that neither had seen before.

MeasurementResult
Same category as Claude94% · final test 91%
Wrongly "urgent" after improving the question15 → 1 (final test)
Urgency with today's date, 25 date tests12 → 1 error
Newsletter: correct where Jev was confident (243 emails)95% · ± 18% to Claude
Share Jev handles without Claude± 70–95% per day
Cleanup advice over 3,079 emails in the inbox32 seconds · ± $0.04

The comparison is against Claude as the judge, not against an absolute truth. In a few of the differences Jev was right rather than Claude.