Jev vs ScaleDown Classify: The Models I’d Use to Mine AI Citation

A lot of SEO and GEO work is still treated as a generation problem when it is really a classification, extraction and evidence-selection problem.

If I need to know whether an AI-cited page is documentation, editorial content, community discussion, research or a competitor page, I do not need another model to write a paragraph about it.

I need a reliable decision.

That is why models such as Jev and ScaleDown Classify-1 are becoming interesting for GEO infrastructure.

Not every language task needs a general-purpose language model.

We use generative models for too many things

A typical AI analytics pipeline still looks something like this:

Raw data
    ↓
Large language model
    ↓
"Analyze this and return JSON"
    ↓
Parse JSON
    ↓
Validate output
    ↓
Retry broken output
    ↓
Store result

That makes sense when the task requires reasoning, synthesis or writing.

But consider some of the operations inside an SEO or GEO workflow:

• Classify the intent of a prompt.
• Assign a page to a topic.
• Determine whether a cited domain belongs to a competitor.
• Classify a citation as editorial, community, documentation or first-party content.
• Determine whether a source actually supports a claim.
• Extract statistics from a long page.
• Extract products, companies, experts or studies from cited content.
• Decide whether an item should be automatically accepted or sent to review.

Most of these are not writing tasks.

They are bounded decisions.

Yet we often send them to a frontier LLM anyway.

That is the architectural mistake decision models make interesting.

What is Jev?

Jev is a decision model from TypeSafe AI.

Instead of asking the model to generate an open-ended response, you provide the state, define a bounded question and let the model assign probabilities to the available decisions.

Conceptually:

INPUT

Domain:
example.com

Question:
What type of source is this?

OPTIONS

A = Community / UGC
B = Vendor
C = Editorial
D = Documentation
E = Research

OUTPUT

A: 0.02
B: 0.08
C: 0.81
D: 0.06
E: 0.03

The important part is not the output format.

The model is not writing an explanation that happens to contain a decision.

The decision is the output.

That is useful when the same bounded judgment needs to happen thousands or millions of times.

It is also useful because probabilities can become part of the workflow itself.

confidence ≥ 0.95
→ accept automatically

0.75–0.95
→ optional second-stage check

confidence < 0.75
→ stronger model or human review

Now confidence is not just something displayed in a dashboard.

It controls the architecture.

An independent 2026 evaluation of Jev tested 346,009 requests across 37 datasets, covering tasks such as classification, routing, natural-language inference, moderation and rubric scoring. The researchers also found that Jev's probability outputs could support selective prediction, where high-confidence decisions are automated and uncertain cases are escalated. The full evaluation is available on arXiv.

Where ScaleDown Classify fits

ScaleDown uses the term Constrained Decoding Models, or CDMs, around its specialized model family.

I would be careful with the terminology.

"Constrained decoding" already has a broader technical meaning in language-model research. For practical purposes, what interests me here is not the branding but the interface:

Can the model cheaply and reliably make a bounded decision over my production data?

The model relevant to this comparison is ScaleDown Classify-1.

Like Jev, Classify-1 can handle bounded classification questions and return scores instead of normal open-ended prose.

That makes the two models natural candidates for:

source classification
domain tagging
intent detection
relevance scoring
routing
yes / no decisions
ordinal scoring

ScaleDown also has specialized extraction and summarization models, which suggest a broader pipeline:

Extract what exists
        ↓
Classify what it means
        ↓
Aggregate the results
        ↓
Summarize the evidence
        ↓
Use a frontier LLM for actual reasoning

That is much more deliberate than sending every document directly to a large model and asking:

"Tell me the insights."

Jev vs ScaleDown: generic benchmarks are not enough

It is tempting to find one benchmark table, identify the model with the highest score and call the comparison finished.

I do not think that works well for decision models.

Their behavior can change substantially depending on:

• the exact taxonomy,
• the number of choices,
• how ambiguous the labels are,
• how the question is written,
• the domain,
• the confidence threshold,
• and the distribution of the production data.

So the better rule is:

Benchmark the decision contract you actually plan to automate.

I recently did exactly that.

I tested both models on a real production workload

I wanted to see what happens when Jev 1.14 and ScaleDown Classify-1 are placed inside the exact same production pipeline.

The benchmark contained 20,646 unique real-world classification contexts.

Everything except the model stayed the same:

Same input data
Same classification request
Same scoring logic
Same downstream code
Same worker
Same concurrency

Only the model changed.

That matters because otherwise a model comparison quickly turns into a prompt comparison or an infrastructure comparison.

Here, the model was the variable.

Speed was not close

Jev completed the full run approximately 3.2× faster.

The tail-latency difference was even larger.

Jev's p95 latency was roughly 5.7× lower.

That matters in batch analytics. Average latency is nice. Tail latency is what makes a large pipeline painful.

Jev was also cheaper

On this workload, Jev was approximately 31% cheaper.

But the more interesting observation is how low both numbers are.

Once the task is reduced to a bounded decision, tens of thousands of classifications cost less than a dollar.

That completely changes which kinds of enrichment become economically reasonable.

Quality mattered more than cost

Speed and cost are simple to compare.

Classification quality is harder.

Instead of manually labelling more than twenty thousand examples, the pipeline used existing operational outcomes as a weak-supervision layer.

One group represented examples with confirmed positive outcomes.

Another represented examples the existing system had already rejected.

I then measured how well each model ranked the positive group above the negative group using AUC.

An AUC of 0.5 represents roughly random ranking. An AUC of 1.0 represents perfect separation.

Jev produced stronger ranking on both tests.

This does not prove that Jev is universally more accurate than ScaleDown Classify.

It proves something more useful:

Jev was better on this decision contract.

That is why I would not choose either model from a public leaderboard alone.

The score distributions were very different

This was one of the more interesting results.

ScaleDown placed roughly two-thirds of the workload in the 60–79 range.

Jev produced a much wider distribution and committed more strongly.

Neither behavior is automatically correct.

But it shows why copying the same decision threshold between models is dangerous.

Calibrate thresholds against your own data, per model.

A score of 75 from Jev should not automatically be interpreted the same way as a score of 75 from ScaleDown.

ScaleDown won on clean completion

There was one area where ScaleDown had the cleaner result.

ScaleDown completed 100% of requests in the benchmark.

Jev completed 99.93% during the initial run.

A very small number of requests hit rate limiting or exhausted the retry window, although every remaining item succeeded when retried.

Jev

+ 3.2× faster
+ 31% cheaper
+ stronger class separation
+ stronger score differentiation
- requires rate-limit handling


ScaleDown Classify

+ clean completion
+ bounded classification interface
- slower in this benchmark
- more expensive in this benchmark
- heavily concentrated in the middle of the score range

This was not an AI-citation benchmark

This distinction matters.

The production experiment was a classification benchmark, not a citation-mining benchmark.

I would therefore not turn the result into:

"Jev is better for GEO."

That would be an overreach.

The useful finding is:

Two models exposing very similar decision interfaces can behave very differently when tested against real data, a real taxonomy and real production constraints.

Which brings us to the GEO use case I actually want to test.

The GEO use case: mine the citation layer

Imagine tracking hundreds of prompts across ChatGPT, Perplexity, Gemini, Claude and Google's AI search surfaces.

For every answer, collect every cited URL.

A basic GEO platform might calculate:

brand citations
citation share
cited domains
brand mentions
competitor mentions

Useful, but mostly descriptive.

The more interesting question is:

What kinds of sources does each AI engine select for each topic?

That requires enrichment.

For every citation, I would classify fields such as:

source_type
page_type
topic
prompt_intent
citation_role
brand_relationship
evidence_type
commerciality
content_format
confidence

A source taxonomy might look like:

FIRST_PARTY_VENDOR
COMPETITOR
INDEPENDENT_EDITORIAL
COMMUNITY_UGC
TECHNICAL_DOCUMENTATION
ACADEMIC_RESEARCH
GOVERNMENT
REVIEW_PLATFORM
MARKETPLACE
VIDEO
SOCIAL
OTHER

Suddenly, a citation database becomes much more than a list of URLs.

From citation tracking to citation fingerprints

There is already evidence that AI engines do not behave like one homogeneous retrieval layer.

In Prefer's September 2026 study of 960 AI answers, researchers asked ChatGPT, Perplexity, Gemini and Claude the same 80 questions three times.

Across 1,329 cited sites, 72.7% appeared in only one of the four engines.

Only 2.2% appeared across all four.

Citation volume was also radically different:

That makes raw citation counts dangerous to compare across engines.

An engine that cites twenty sources per answer has much more citation inventory than an engine that typically cites three.

A separate seven-month analysis from Conductor found persistent differences in source preference across major AI engines and search surfaces.

That is why I think the better abstraction is:

engine × topic citation fingerprint

What would a citation fingerprint look like?

Suppose we track one topic across four AI engines.

We collect every cited URL and classify the source type.

Instead of asking only:

"How many citations did we get?"

we can ask:

Which source classes does each AI engine over-select for this topic relative to the other engines?

This is where citation tracking starts becoming retrieval research.

If one engine repeatedly prefers first-party technical pages while another over-selects independent editorial content or community discussions, those are materially different GEO environments.

Simply counting citations hides that distinction.

Measure preference, not just citation share

One metric I would test is Source Preference Lift.

Source Preference Lift =

P(Source Type | Engine, Topic)
──────────────────────────────
P(Source Type | All Engines, Topic)

Imagine editorial sources represent 42% of one engine's citations for a monitored topic.

Across all engines, editorial sources represent only 21%.

Engine editorial share = 42%

Cross-engine editorial baseline = 21%

Preference Lift = 42 / 21

Preference Lift = 2.0x

That engine is selecting editorial sources at twice the cross-engine baseline for that topic.

Repeat that calculation for:

Vendor
Competitor
Editorial
Documentation
Community
Research
Government
Review
Video
Social

Now each topic starts to develop a source-selection fingerprint.

One AI response is not a dataset

Citation selection is stochastic.

This is one of the easiest things to underestimate in GEO measurement.

In Prefer's repeated-query experiment, two runs of the same question shared the following proportion of cited sites on average:

A domain appearing once is therefore weak evidence.

A domain appearing repeatedly across several prompts and independent runs is much more interesting.

A useful citation dataset should preserve:

topic
prompt
engine
model
run_id
timestamp
answer
cited_url
cited_domain
source_type
page_type
citation_role
brand_relationship
confidence

Measure citation stability

One simple way to measure repeatability is Jaccard similarity.

A = domains cited in run 1
B = domains cited in run 2

Jaccard(A, B) = |A ∩ B| / |A ∪ B|

If run one cites:

domain-a.com
domain-b.com
domain-c.com
domain-d.com

and run two cites:

domain-a.com
domain-b.com
domain-e.com
domain-f.com

the intersection contains two domains and the union contains six.

Stability = 2 / 6 = 0.33

That pattern should be interpreted very differently from an engine returning nearly the same source set every time.

Where Jev or ScaleDown Classify fits

The decision layer can enrich every citation before a more expensive reasoning model ever sees it.

AI answer
    ↓
Extract citations
    ↓
Normalize URLs
    ↓
Fetch cited pages
    ↓
CLASSIFY
    ├── source type
    ├── page type
    ├── topic
    ├── intent
    ├── brand relationship
    ├── evidence role
    └── relevance
    ↓
Store probabilities
    ↓
Confidence gate
    ├── high confidence → database
    └── low confidence → stronger model / review

This is where a cheap decision model becomes useful even if it is not perfect.

We do not need it to solve every edge case.

We need it to reliably remove most easy cases from the expensive layer.

Extraction is the next useful layer

Classification tells us what a source is.

Extraction tells us what is inside it.

Suppose an AI engine repeatedly cites a long technical report.

Instead of asking a general-purpose model:

"Read this page and tell me why an AI engine might like it."

we can first extract:

definitions
statistics
named entities
products
benchmarks
expert quotes
research claims
dates
methodology
comparisons
recommendations

Now we are analyzing evidence rather than vibes.

This becomes especially useful when comparing repeatedly cited pages with our own content.

REPEATEDLY CITED PAGE

Contains:
✓ definition
✓ benchmark data
✓ methodology
✓ practitioner quote
✓ implementation example
✓ original statistic


OUR PAGE

Contains:
✓ definition
✓ product description
✗ benchmark data
✗ methodology
✗ practitioner quote
✗ implementation example
✗ original statistic

That is a more useful content-gap analysis than word count, keyword density or generic "content quality" scoring.

Summarization should come late

Imagine collecting thousands of citation passages across many monitored topics.

The tempting approach is:

4,000 documents
    ↓
LLM
    ↓
"Tell me the insights"

I would rather do:

4,000 documents
    ↓
Extract
    ↓
Classify
    ↓
Aggregate
    ↓
Summarize
    ↓
Reason

The difference is that the final reasoning layer receives structured evidence rather than a pile of raw pages.

The citation benchmark I want to run next

My production benchmark made one thing clear:

I would not trust a generic leaderboard to pick the decision model for a citation-mining pipeline.

I would create a citation-specific ground-truth dataset:

500–1,000 manually reviewed AI citations

Labels:

source_type
page_type
brand_relationship
topic_relevance
citation_role
claim_support

Then run exactly the same examples through:

Jev
ScaleDown Classify
Frontier LLM
Optional traditional classifier

And measure:

Macro F1
Precision
Recall
Calibration
Latency
Cost
Coverage at confidence threshold

The metric I care about most

I would not optimize primarily for average accuracy.

I would ask:

What percentage of the citation corpus can this model classify automatically while maintaining at least 95% precision?

That tells us how much work the decision model can safely remove from the expensive layer.

A model that confidently handles 90% of the corpus and escalates the remaining 10% can be more useful than one with slightly higher average accuracy but poor calibration.

The same decision layer is useful across SEO

Citation mining is not the only place this architecture makes sense.

Content clustering

Page
    ↓
Primary topic
    ↓
Search intent
    ↓
Funnel stage
    ↓
Pillar / supporting content
    ↓
Cluster confidence

SERP classification

Ranking URL
    ↓
Vendor
Publisher
Forum
Documentation
Marketplace
Government
Academic
Video

Content-gap detection

External evidence
    ↓
Extract entities / claims / statistics
    ↓
Compare against own page
    ↓
Present / Partial / Missing

These are not primarily writing problems.

They are structured decision and extraction problems.

The bigger GEO insight

The first generation of GEO tools mostly borrowed the mental model of SEO rank tracking.

Keyword ranking
became
Prompt visibility

Ranking URL
became
Citation URL

Share of voice
became
AI visibility score

That was a logical first step.

But it leaves a more interesting layer underneath:

Prompt
    ↓
Retrieval
    ↓
Source selection
    ↓
Evidence selection
    ↓
Answer synthesis
    ↓
Citation
    ↓
Brand inclusion

Tracking the final citation tells us the outcome.

Classification and extraction help us investigate the selection process.

That is where GEO starts moving from reporting toward actual retrieval research.

What does each AI engine actually choose?

AI engines do not literally "want" a type of content.

But repeated behavior can reveal persistent source-selection patterns.

For every topic, we can model:

Engine
×
Topic
×
Intent
×
Source Type
×
Page Type
×
Citation Role
×
Repeatability

That gives us something much closer to an AI retrieval fingerprint.

Instead of broad recommendations like:

"Create more community content."

the analysis could become:

This engine consistently over-selects independent editorial sources for this topic, while another engine disproportionately selects first-party technical pages.

That is much more useful for content strategy.

Decision models do not replace frontier LLMs

There is an equally bad version of this argument where specialized models suddenly replace everything.

They do not.

Frontier models remain better tools for:

open-ended research
complex reasoning
strategy
hypothesis generation
cross-document synthesis
writing
explanation
planning

The useful architecture is a cascade:

Deterministic code
        ↓
Decision / extraction models
        ↓
Frontier LLM
        ↓
Human review when necessary

The goal is not to eliminate large language models.

The goal is to stop paying a reasoning model to perform every tiny decision inside the system.

So, Jev or ScaleDown?

Based on my current testing, I would not choose one model globally.

For classification specifically:

Run Jev and ScaleDown against your own labeled workload.

My benchmark favored Jev.

A different taxonomy may not.

From benchmark to citation-mining pipeline

The experiment I want to build next looks like this:

1. Choose a set of commercially relevant topics.

2. Create realistic prompts for each topic.

3. Run them repeatedly across:
   ChatGPT
   Perplexity
   Gemini
   Claude

4. Collect every cited URL.

5. Fetch and normalize the cited pages.

6. Manually label a ground-truth citation sample.

7. Benchmark:
   Jev
   ScaleDown Classify
   Frontier LLM

8. Extract:
   statistics
   claims
   entities
   experts
   evidence
   comparisons

9. Calculate:
   source preference
   citation stability
   citation diversity
   competitor inclusion
   citation-role distribution

10. Build an engine × topic citation fingerprint.

This answers two different questions.

First:

Which model is actually good enough to classify AI citations at production scale?

Second:

What types of sources does each AI engine repeatedly choose for each topic?

The first is an infrastructure question.

The second is a GEO strategy question.

Combining them is where this gets interesting.

The model should not always write

The current AI stack has a strange default.

Whenever a problem involves language, we give it to a language model and ask it to generate more language.

Jev challenges that assumption.

ScaleDown pushes in the same direction with specialized classification and document-processing models.

For SEO and GEO, the timing is useful.

We are accumulating increasingly large datasets containing prompts, answers, citations, pages, passages, entities and competitors.

The next bottleneck is not generating more text.

It is turning that mess into structured evidence.

That means answering small questions reliably:

What is this source?
What role does it play?
Which topic does it belong to?
Does it support the claim?
Is the pattern repeatable?
How confident are we?

Those are decision problems.

And for GEO, solving them may be more valuable than adding another chatbot to the stack.

Sources and further reading

• TypeSafe AI: Jev 1.14
• Evaluating and Benchmarking the System One Model Jev
• ScaleDown Classify
• Prefer: How AI Engines Search and Cite — 960 Answers Measured
• Conductor: How AI Engines Choose and Cite Sources — A 7-Month Analysis
• Original benchmark: 20,646 identical production classification contexts tested with Jev 1.14 and ScaleDown Classify-1.