# SemIf

> Read typed option probabilities from an open model instead of generating JSON

- Category: App Orchestration / Structured Output
- Language: Python
- Licence: MIT
- Repository: https://github.com/TheoLeeCJ/SemIf
- Website: https://openjev.com/
- Canonical page: https://ai-tldr.dev/tools/semif/
- Updated: 2026-09-18

## In short

SemIf is an open-source baseline for what its author calls a semantic if: a small language model answers a runtime-defined question by reporting the probability of each option you allow, rather than writing an answer sentence that your code parses back into a conditional. It sits in the Structured Output category of the App Orchestration tools and is open-source, written in Python and licensed MIT.

## Overview

SemIf is an open-source baseline for what its author calls a semantic if: a small language model answers a runtime-defined question by reporting the probability of each option you allow, rather than writing an answer sentence that your code parses back into a conditional. One forward pass carries the unstructured state, the criteria and the typed options, and the declared option logits are read back directly. No answer token is sampled, so there is no JSON to repair and no decoding loop.

The project was formerly called OpenJev and is an independent reproduction of the interface pattern published by TypeSafe for its Jev service. Its author states that it does not reproduce Jev's undisclosed model or training, and that it is not affiliated with or endorsed by TypeSafe. What SemIf shows is that frozen, openly available models in the 0.6B to 4B range can serve the same shape of request on hardware you already own.

SemIf sits in the structured-output layer of an LLM stack, alongside constrained-decoding libraries such as Outlines, XGrammar and LM Format Enforcer. The difference is that those tools shape the tokens a model generates, while SemIf skips generation entirely for the narrow case of choosing among a fixed set of described options. Its results, fixtures, runners, raw timings, prompt hashes and known failure rows are committed to the repository so the published numbers can be rerun.

## What it does

- Direct scoring mode that reads declared option logits in a single forward pass, emitting zero output tokens
- Runtime-defined criteria and option descriptions supplied with each request, so no fine-tune is needed per decision
- Shared-state modes that prefill one long state once and branch it across many criteria, serially or in parallel
- Browser demo that runs quantized GGUF builds of Qwen3-0.6B, MiniCPM5-2B and Qwen3.5-4B on WebGPU
- Native MLX backend for macOS on Apple Silicon
- Committed fixtures, runners, row-level predictions, raw timings and model revisions for reproduction

## Getting started

SemIf needs Python 3.10 or newer, CUDA and a GPU that can hold a 4B BF16 model. On Apple Silicon, install the MLX extra instead and pass --backend mlx.

1. Create a virtual environment

   Set HF_HOME to a drive with room for the model weights before installing.

```bash
python -m venv .venv
. .venv/bin/activate
export HF_HOME=/path/to/large-drive/huggingface
```

2. Install SemIf

   Install from a checkout of the repository. Add the mlx extra on Apple Silicon.

```bash
pip install -e '.[test]'
```

3. Score the bundled examples

   Direct mode reads typed option scores for each row. Each result carries timing, the exact model revision and a prompt hash.

```bash
CUDA_VISIBLE_DEVICES=0 semif-score \
  --mode direct \
  --model Qwen/Qwen3.5-4B \
  --revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a \
  --input examples/decisions.jsonl \
  --output results.jsonl
```

4. Reuse one state across many criteria

   When every row shares the same state, shared mode prefills it once and evaluates the criteria in parallel.

```bash
semif-score --mode shared --model Qwen/Qwen3.5-4B --input examples/decisions.jsonl --output results.jsonl
```

## When to use it

- Routing, triage and retry decisions inside an agent loop where a full generated answer is wasted work
- Scoring one long document or conversation against many yes/no criteria at once
- Running typed decisions locally or in a browser when sending the state to a hosted API is not an option
- Benchmarking whether a small open model is good enough before paying for a hosted decision service

## Version history

- 2026-09-18: OpenJev was renamed to SemIf, and MiniCPM5-2B and Qwen3.5-4B were added to the browser demo alongside an alternative conventional interface. The project reached 621 points on Hacker News the same day.