CrossNER entity extraction

Comparative evaluation across five domains

English · 2,505 sentences · Five domains · jpt-pairs-v1

Cross-domain results

ProvisionalF1 (%)

Rank	System	Configuration	Mean F1	AI	Literature	Music	Politics	Science	Status
1	Graphiti + Codex	Graphiti 0.31.2 / gpt-6.1-sol	75.66%	61.10%	77.23%	85.67%	80.40%	73.89%	Provisional
2	JPT-4B	Author-released prediction snapshot	75.26%	68.64%	72.70%	84.87%	78.80%	71.29%	Provisional
3	GLiNER	gliner_large-v2.1 / local FP32	66.70%	57.57%	63.43%	74.98%	69.93%	67.61%	Provisional
4	GLiNER Bi-large	gliner-bi-large-v2.0 / FP32	66.16%	58.75%	65.80%	78.68%	58.22%	69.37%	Provisional
5	NuNER Zero	NuNER_Zero / FP32	65.74%	51.90%	62.95%	74.48%	75.93%	63.45%	Provisional
6	ModernGLiNER Bi	modern-gliner-bi-base-v1.0 / FP32	64.08%	56.00%	59.55%	73.48%	62.19%	69.19%	Provisional
7	GLiNER2.5	gliner2.5-base-v1 / FP32	60.55%	52.32%	57.38%	70.18%	62.36%	60.50%	Provisional

On narrow screens, focus the table and use arrow keys to scroll.

Evaluation protocol

Normalized entity-pair F1

Dataset: CrossNER test splits for AI, literature, music, politics and science: 2,505 aligned English sentences. Our retained configurations originally ran on 2,506 sentences; scoring omits politics/test/650 to align with the released JPT gold using source text only.

Scoring: jpt-pairs-v1 matches normalized (name, type) sets within each sentence. Span offsets and entity identities are not evaluated. The reported mean averages five unrounded domain F1 scores equally. Rows follow descending mean order. Bar length represents F1 (0-100); color runs red to green within each column. Bold marks each column's observed maximum, including exact ties, without a significance claim.
Experimental conditions: Single retained runs; no estimated confidence intervals or significance tests. JPT uses rich type definitions; our retained configurations use original type names including misc. Schema effects are unmeasured.
Interpretation: The 0.40-point difference between the first two means is descriptive. JPT is scored from the current author cache, rather than the historical paper row. This selected comparison is provisional and does not establish a general SOTA ranking.

Contenders

Methods, publications and the configurations behind the measured results.

Graphiti + Codex

Temporal knowledge-graph framework with LLM-based extraction. This row evaluates its entity extraction stage with Codex gpt-6.1-sol, rather than graph construction quality.

Zep: A Temporal Knowledge Graph Architecture for Agent Memory (related architecture paper)
Preprint: 20 Jan 2025

Graphiti 0.31.2 released: 20 Oct 2026

Configuration & evidence

Saved predictions remain private. Independent public rescoring of this row is not yet possible.

Full-precision F1: AI 61.09734513274335; Literature 77.23449001051524; Music 85.67293777134589; Politics 80.4011170347804; Science 73.89028881566195
Mean F1: 75.65923575300937
Evidence: retained local inference predictions rescored locally
Reproduction coverage: Saved predictions remain private. Independent public rescoring of this row is not yet possible.
Prediction SHA256: 4cbf4ed0049ad07dc3eca877f33d952ea912fcb81c11bb60101b05fab29865e3
Dataset input SHA256: da3f74f360ca5690a8447287a65290f8771bcb0dffe3b88beca69ed490302079
Original scoring revision: 78ba4c2b032a2ffc4a03e80ff0f715c0111ffe74
Source lock SHA256: 475e47870af4132776833e8d3a7e46e157cd8f8da9647f699ee0ead4415b2274
Mapping SHA256: e9e0735430311a6b8ec6aed18ff0b7fbbb32cdf75bd65520095119bb287fc3f3
Upstream jpt-archive: jpt-benchmarks.zip; SHA256 df7dacfabe26e7080da99103c3b0d09b70c08366138c3ae8da41495d836cad2b; source https://drive.google.com/file/d/1D8aKQfvJQ65ELrwW0oITvFogWy96szFW/view
Upstream jpt-scorer: jpt-evaluator-source.txt; SHA256 d75050e5a086ca12301c8ebf000b3bae76fa3a419756d6aba375ceb0277c8794; source https://witness-ai-jpt-ner.hf.space/ static evaluator download, retrieved 2026-11-02
Snapshot date: 2026-11-03
Model checkpoint: Unknown; served model identifier only
Artifact integrity: Validated locally against pinned snapshot bytes
Code review: Original JPT path and standalone scorer reviewed; reviewers did not rerun evaluations
Benchmark status: Provisional comparison; no independently verified inference badge
Graphiti Core: 0.31.2
Openai Sdk: 2.54.0
Chat Model: gpt-6.1-sol
Chat Controls: provider_defaults
Retry Policy: single_attempt_stop_on_any_error
Schema: original-train-names-including-misc
Extraction Source Head: 50bec31822367bd3da02fa77fc2a44e012ea196f
Bridge Revision: d745731e90c9cfefbb5591ee6091e60502e7521e
Immutable Provider Checkpoint: not available; served model identifier only

Back to results

JPT-4B

Just Pass Twice uses repeated input to give a causal language model bidirectional context for token classification. This row rescores the author-released predictions.

Just Pass Twice: Efficient Token Classification with LLMs for Zero-Shot NER
ACL: Jul 2026 | First preprint: 6 Apr 2026

Measured cache retrieved: 2 Nov 2026 | Exact checkpoint release date unknown

Configuration & evidence

Author cache rescored locally; JPT inference was not rerun. See the public reproduction guide.

Full-precision F1: AI 68.6431014823261; Literature 72.70114942528737; Music 84.86545341421683; Politics 78.80189942773652; Science 71.28825085001888
Mean F1: 75.25997091991715
Evidence: Author-released current downloadable cache, rescored locally; predictions are covered by jpt-archive SHA256 and consumed member pins in the original source lock; historical paper checkpoint identity unconfirmed.
Reproduction coverage: Author cache rescored locally; JPT inference was not rerun. See the public reproduction guide.
Prediction SHA256: Covered by the pinned author archive and member digests
Dataset input SHA256: da3f74f360ca5690a8447287a65290f8771bcb0dffe3b88beca69ed490302079
Original scoring revision: 78ba4c2b032a2ffc4a03e80ff0f715c0111ffe74
Source lock SHA256: 475e47870af4132776833e8d3a7e46e157cd8f8da9647f699ee0ead4415b2274
Mapping SHA256: e9e0735430311a6b8ec6aed18ff0b7fbbb32cdf75bd65520095119bb287fc3f3
Upstream jpt-archive: jpt-benchmarks.zip; SHA256 df7dacfabe26e7080da99103c3b0d09b70c08366138c3ae8da41495d836cad2b; source https://drive.google.com/file/d/1D8aKQfvJQ65ELrwW0oITvFogWy96szFW/view
Upstream jpt-scorer: jpt-evaluator-source.txt; SHA256 d75050e5a086ca12301c8ebf000b3bae76fa3a419756d6aba375ceb0277c8794; source https://witness-ai-jpt-ner.hf.space/ static evaluator download, retrieved 2026-11-02
Snapshot date: 2026-11-03
Model checkpoint: Unknown; author-attributed cache, historical checkpoint unconfirmed
Artifact integrity: Validated locally against pinned snapshot bytes
Code review: Original JPT path and standalone scorer reviewed; reviewers did not rerun evaluations
Benchmark status: Provisional comparison; no independently verified inference badge

Back to results

GLiNER

A bidirectional transformer that identifies entities from supplied type descriptions. The measured gliner_large-v2.1 checkpoint runs locally in FP32 with a 0.5 threshold.

GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer
NAACL: Jun 2024

Model repository created: 10 Apr 2024 | Exact release announcement date unverified

Configuration & evidence

Saved predictions remain private. Independent public rescoring of this row is not yet possible.

Full-precision F1: AI 57.56639901173565; Literature 63.42660775650466; Music 74.9802683504341; Politics 69.93278179937953; Science 67.60714285714286
Mean F1: 66.70263995503936
Evidence: retained local inference predictions rescored locally
Reproduction coverage: Saved predictions remain private. Independent public rescoring of this row is not yet possible.
Prediction SHA256: 01e767b4ce9cf2123e64c6ce4901cdd2b1f484738bb4032dbef1d99b5115867a
Dataset input SHA256: da3f74f360ca5690a8447287a65290f8771bcb0dffe3b88beca69ed490302079
Original scoring revision: 78ba4c2b032a2ffc4a03e80ff0f715c0111ffe74
Source lock SHA256: 475e47870af4132776833e8d3a7e46e157cd8f8da9647f699ee0ead4415b2274
Mapping SHA256: e9e0735430311a6b8ec6aed18ff0b7fbbb32cdf75bd65520095119bb287fc3f3
Upstream jpt-archive: jpt-benchmarks.zip; SHA256 df7dacfabe26e7080da99103c3b0d09b70c08366138c3ae8da41495d836cad2b; source https://drive.google.com/file/d/1D8aKQfvJQ65ELrwW0oITvFogWy96szFW/view
Upstream jpt-scorer: jpt-evaluator-source.txt; SHA256 d75050e5a086ca12301c8ebf000b3bae76fa3a419756d6aba375ceb0277c8794; source https://witness-ai-jpt-ner.hf.space/ static evaluator download, retrieved 2026-11-02
Snapshot date: 2026-11-03
Model checkpoint: abd49a1f1ebc12af1be84d06f6848221cf96dcad
Artifact integrity: Validated locally against pinned snapshot bytes
Code review: Original JPT path and standalone scorer reviewed; reviewers did not rerun evaluations
Benchmark status: Provisional comparison; no independently verified inference badge
Checkpoint: urchade/gliner_large-v2.1
Revision: abd49a1f1ebc12af1be84d06f6848221cf96dcad
Backbone: microsoft/deberta-v3-large
Backbone Revision: 64a8c8eab3e352a784c658aef62be1662607476f
Package: 0.2.30
Threshold: 0.5
Flat Ner: True
Multi Label: False
Device: cuda
Precision: float32
Batch Size: 1
Max Words: 384
Max Encoder Tokens: 512
Seed: 0
Labels: original-train-names-including-misc
System: gliner-large-v2.1/0.2.30/fp32/defaults/original-schema-v1

Back to results

GLiNER Bi-large

Bi-encoder entity extraction with separate label representations.

The Million-Label NER: Breaking Scale Barriers with GLiNER bi-encoder
First preprint: 11 Feb 2026

Snapshot evaluated: 3 Nov 2026 | Exact checkpoint release date unverified

Configuration & evidence

Saved predictions remain private. Independent public rescoring of this row is not yet possible.

Full-precision F1: AI 58.748317631224765; Literature 65.79973992197658; Music 78.67918203151191; Politics 58.215547703180206; Science 69.36758893280633
Mean F1: 66.16207524413997
Evidence: Complete retained local GPU predictions, scored with the unchanged released JPT normalized-pair evaluator. No new inference for this publication.
Reproduction coverage: Saved predictions remain private. Independent public rescoring of this row is not yet possible.
Prediction SHA256: 2ac0c1a491771cdb181bc3e675e7951ccf1155071540c0f91232dd21150557e3
Dataset input SHA256: da3f74f360ca5690a8447287a65290f8771bcb0dffe3b88beca69ed490302079
Original scoring revision: 17a3040a23077fdbe374078a22d47d0f41426a94
Source lock SHA256: 475e47870af4132776833e8d3a7e46e157cd8f8da9647f699ee0ead4415b2274
Mapping SHA256: e9e0735430311a6b8ec6aed18ff0b7fbbb32cdf75bd65520095119bb287fc3f3
Upstream jpt-archive: jpt-benchmarks.zip; SHA256 df7dacfabe26e7080da99103c3b0d09b70c08366138c3ae8da41495d836cad2b; source https://drive.google.com/file/d/1D8aKQfvJQ65ELrwW0oITvFogWy96szFW/view
Upstream jpt-scorer: jpt-evaluator-source.txt; SHA256 d75050e5a086ca12301c8ebf000b3bae76fa3a419756d6aba375ceb0277c8794; source https://witness-ai-jpt-ner.hf.space/ static evaluator download, retrieved 2026-11-02
Snapshot date: 2026-11-03
Model checkpoint: 71eec1d5fe57684cd6669ad8a1e1e3e7401a9a55
Artifact integrity: Validated locally against pinned snapshot bytes
Code review: Implementation and recorded evidence independently reviewed 2026-11-07 at dbcba5ebb0f38dfd40f894f956420b47a359c2e1; no independent GPU rerun or general accuracy guarantee.
Benchmark status: Provisional comparison; no independently verified inference badge
Checkpoint: knowledgator/gliner-bi-large-v2.0
Revision: 71eec1d5fe57684cd6669ad8a1e1e3e7401a9a55
Packages: {"gliner": "0.2.30", "torch": "2.6.0+cu124", "transformers": "4.51.3", "huggingface-hub": "0.36.2"}
Threshold: 0.5
Flat Ner: True
Multi Label: False
Device: cuda
Precision: float32
Batch Size: 1
Seed: 0
Schema: original-train-names-including-misc
Decoder: native-flat
Inference Source Head: 05f34bf1870c64751ae813ac614f401a1dd63827
Inference Source Dirty: False
Assets Manifest Sha256: 002daf72a88a466f721a0be2909b811ae0734dc1f6d2f4bb60fd1e181ea7ed5c
Evaluation Date: 2026-11-03
Elapsed Seconds: 116.20300000000134
Peak Gpu Bytes: 2170597888
Timing Scope: Shared RTX3060Ti; loading/preflight included, not isolated throughput.
Training Overlap: Not excluded; no uncontaminated zero-shot claim.
Review Summary: Implementation and recorded evidence independently reviewed 2026-11-07 at dbcba5ebb0f38dfd40f894f956420b47a359c2e1; no independent GPU rerun or general accuracy guarantee.
Source Evidence: {"repository": "amadou-6e/agent-anvil", "revision": "d4a44504e709f3167358b9bf3a463ef456b80534", "path": "tools/llms/entity-extraction/docs/benchmark/evidence/next-local-candidates.json", "sha256": "818f9e53ef4b5392be13f617f74acb6305137018b0897c028b58902af0aa745e"}
Compatibility Note: Fixed0.5threshold, not the card0.3example.

Back to results

NuNER Zero

Generalist entity extraction using the NuNER Zero checkpoint and its label decoding.

NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data (related pretraining paper)
First preprint: 23 Feb 2024

Snapshot evaluated: 3 Nov 2026 | Exact checkpoint release date unverified

Configuration & evidence

Saved predictions remain private. Independent public rescoring of this row is not yet possible.

Full-precision F1: AI 51.89644792293798; Literature 62.94518817883304; Music 74.47751741608613; Politics 75.93058350100603; Science 63.44928064104898
Mean F1: 65.73980353198243
Evidence: Complete retained local GPU predictions, scored with the unchanged released JPT normalized-pair evaluator. No new inference for this publication.
Reproduction coverage: Saved predictions remain private. Independent public rescoring of this row is not yet possible.
Prediction SHA256: eebf079e067c1820237a050d6c8cdf49de3ba51e75da95f74f90c11dcbb0cc7b
Dataset input SHA256: da3f74f360ca5690a8447287a65290f8771bcb0dffe3b88beca69ed490302079
Original scoring revision: 44f08673062afcc91f96a8404589b61b9ac411e5
Source lock SHA256: 475e47870af4132776833e8d3a7e46e157cd8f8da9647f699ee0ead4415b2274
Mapping SHA256: e9e0735430311a6b8ec6aed18ff0b7fbbb32cdf75bd65520095119bb287fc3f3
Upstream jpt-archive: jpt-benchmarks.zip; SHA256 df7dacfabe26e7080da99103c3b0d09b70c08366138c3ae8da41495d836cad2b; source https://drive.google.com/file/d/1D8aKQfvJQ65ELrwW0oITvFogWy96szFW/view
Upstream jpt-scorer: jpt-evaluator-source.txt; SHA256 d75050e5a086ca12301c8ebf000b3bae76fa3a419756d6aba375ceb0277c8794; source https://witness-ai-jpt-ner.hf.space/ static evaluator download, retrieved 2026-11-02
Snapshot date: 2026-11-03
Model checkpoint: c90187673f464518dca09f41689184ed6976242c
Artifact integrity: Validated locally against pinned snapshot bytes
Code review: Implementation and recorded evidence independently reviewed 2026-11-07 at dbcba5ebb0f38dfd40f894f956420b47a359c2e1; no independent GPU rerun or general accuracy guarantee.
Benchmark status: Provisional comparison; no independently verified inference badge
Checkpoint: numind/NuNER_Zero
Revision: c90187673f464518dca09f41689184ed6976242c
Packages: {"gliner": "0.2.30", "torch": "2.6.0+cu124", "transformers": "4.51.3", "huggingface-hub": "0.36.2"}
Threshold: 0.5
Flat Ner: True
Multi Label: False
Device: cuda
Precision: float32
Batch Size: 1
Seed: 0
Schema: original-train-names-including-misc
Decoder: author-adjacent-same-label-token-merge
Inference Source Head: ad4619e193c4aed800cd77a9d0be78c9d588b64d
Inference Source Dirty: False
Assets Manifest Sha256: 5955b58e4040a8c424b77e01d3dfa46f6babce1b6473ee013579e7faaa73ca59
Evaluation Date: 2026-11-03
Elapsed Seconds: 119.98399999999674
Peak Gpu Bytes: 1839356416
Timing Scope: Shared RTX3060Ti; loading/preflight included, not isolated throughput.
Training Overlap: Not excluded; no uncontaminated zero-shot claim.
Review Summary: Implementation and recorded evidence independently reviewed 2026-11-07 at dbcba5ebb0f38dfd40f894f956420b47a359c2e1; no independent GPU rerun or general accuracy guarantee.
Source Evidence: {"repository": "amadou-6e/agent-anvil", "revision": "d4a44504e709f3167358b9bf3a463ef456b80534", "path": "tools/llms/entity-extraction/docs/benchmark/evidence/next-local-candidates.json", "sha256": "818f9e53ef4b5392be13f617f74acb6305137018b0897c028b58902af0aa745e"}
Compatibility Note: GLiNER0.2.30 with documented adjacent-token merge; distinct from card0.1.12 loader.

Back to results

ModernGLiNER Bi

Modern bidirectional label-conditioned entity extractor.

The Million-Label NER: Breaking Scale Barriers with GLiNER bi-encoder (related architecture paper)
First preprint: 11 Feb 2026

Snapshot evaluated: 3 Nov 2026 | Exact checkpoint release date unverified

Configuration & evidence

Saved predictions remain private. Independent public rescoring of this row is not yet possible.

Full-precision F1: AI 55.99999999999999; Literature 59.55087330191296; Music 73.47966894566392; Politics 62.19040801717968; Science 69.19213059857681
Mean F1: 64.08261617266666
Evidence: Complete retained local GPU predictions, scored with the unchanged released JPT normalized-pair evaluator. No new inference for this publication.
Reproduction coverage: Saved predictions remain private. Independent public rescoring of this row is not yet possible.
Prediction SHA256: 7ce6fd4c875d40db62d230299f05de443f014a98cbe3822bcf66e398705f56c2
Dataset input SHA256: da3f74f360ca5690a8447287a65290f8771bcb0dffe3b88beca69ed490302079
Original scoring revision: 1400727ae15ba2967d7db36df31c3da9182f056a
Source lock SHA256: 475e47870af4132776833e8d3a7e46e157cd8f8da9647f699ee0ead4415b2274
Mapping SHA256: e9e0735430311a6b8ec6aed18ff0b7fbbb32cdf75bd65520095119bb287fc3f3
Upstream jpt-archive: jpt-benchmarks.zip; SHA256 df7dacfabe26e7080da99103c3b0d09b70c08366138c3ae8da41495d836cad2b; source https://drive.google.com/file/d/1D8aKQfvJQ65ELrwW0oITvFogWy96szFW/view
Upstream jpt-scorer: jpt-evaluator-source.txt; SHA256 d75050e5a086ca12301c8ebf000b3bae76fa3a419756d6aba375ceb0277c8794; source https://witness-ai-jpt-ner.hf.space/ static evaluator download, retrieved 2026-11-02
Snapshot date: 2026-11-03
Model checkpoint: 2d73e4c02b52f96e2926baffc67b5e78532fc39c
Artifact integrity: Validated locally against pinned snapshot bytes
Code review: Implementation and recorded evidence independently reviewed 2026-11-07 at f0c57ecdcf6cfd617c00cf6785cf75e55b874023; no independent GPU rerun or general accuracy guarantee.
Benchmark status: Provisional comparison; no independently verified inference badge
Checkpoint: knowledgator/modern-gliner-bi-base-v1.0
Revision: 2d73e4c02b52f96e2926baffc67b5e78532fc39c
Packages: {"gliner": "0.2.30", "torch": "2.6.0+cu124", "transformers": "4.51.3", "huggingface-hub": "0.36.2"}
Threshold: 0.5
Flat Ner: True
Multi Label: False
Device: cuda
Precision: float32
Batch Size: 1
Seed: 0
Schema: original-train-names-including-misc
Decoder: native-flat
Inference Source Head: 17a3040a23077fdbe374078a22d47d0f41426a94
Inference Source Dirty: False
Assets Manifest Sha256: 58fd11919cffe927835fd670168297d1abf7d0ef0ab14f02d9b22c02a40fd6dd
Evaluation Date: 2026-11-03
Elapsed Seconds: 99.96800000000076
Peak Gpu Bytes: 901172224
Timing Scope: Shared RTX3060Ti; loading/preflight included, not isolated throughput.
Training Overlap: Not excluded; no uncontaminated zero-shot claim.
Review Summary: Implementation and recorded evidence independently reviewed 2026-11-07 at f0c57ecdcf6cfd617c00cf6785cf75e55b874023; no independent GPU rerun or general accuracy guarantee.
Source Evidence: {"repository": "amadou-6e/agent-anvil", "revision": "d4a44504e709f3167358b9bf3a463ef456b80534", "path": "tools/llms/entity-extraction/docs/benchmark/evidence/local-candidates.json", "sha256": "748158f2191f380039a27fd0102feff31013bee781004d5ed3d7eda0bcc0a3a1"}
Compatibility Note: Checkpoint tokenizer rechecked; predictions identical to original retained run. Configuration and timing describe the tokenizer recheck at inference revision 17a3040; execution is the original scoring receipt at 1400727 for byte-identical predictions (same SHA256).

Back to results

GLiNER2.5

Schema-driven local entity extraction with GLiNER2 AutoExtractor.

GLiNER2: An Efficient Multi-Task Information Extraction System with Schema-Driven Interface (related architecture paper)
First preprint: 24 Jul 2025

Snapshot evaluated: 3 Nov 2026 | Exact checkpoint release date unverified

Configuration & evidence

Saved predictions remain private. Independent public rescoring of this row is not yet possible.

Full-precision F1: AI 52.32300884955753; Literature 57.37781954887219; Music 70.17708823068483; Politics 62.36227935078783; Science 60.504786771105316
Mean F1: 60.54899655020154
Evidence: Complete retained local GPU predictions, scored with the unchanged released JPT normalized-pair evaluator. No new inference for this publication.
Reproduction coverage: Saved predictions remain private. Independent public rescoring of this row is not yet possible.
Prediction SHA256: 3986292fd6471f44fbacf8f18dbbddcf93ee77511b9f581b5e0b3c36dd267eb2
Dataset input SHA256: da3f74f360ca5690a8447287a65290f8771bcb0dffe3b88beca69ed490302079
Original scoring revision: 1400727ae15ba2967d7db36df31c3da9182f056a
Source lock SHA256: 475e47870af4132776833e8d3a7e46e157cd8f8da9647f699ee0ead4415b2274
Mapping SHA256: e9e0735430311a6b8ec6aed18ff0b7fbbb32cdf75bd65520095119bb287fc3f3
Upstream jpt-archive: jpt-benchmarks.zip; SHA256 df7dacfabe26e7080da99103c3b0d09b70c08366138c3ae8da41495d836cad2b; source https://drive.google.com/file/d/1D8aKQfvJQ65ELrwW0oITvFogWy96szFW/view
Upstream jpt-scorer: jpt-evaluator-source.txt; SHA256 d75050e5a086ca12301c8ebf000b3bae76fa3a419756d6aba375ceb0277c8794; source https://witness-ai-jpt-ner.hf.space/ static evaluator download, retrieved 2026-11-02
Snapshot date: 2026-11-03
Model checkpoint: 54785d51df8d86a0d8f2eb212fec15871aad2c4d
Artifact integrity: Validated locally against pinned snapshot bytes
Code review: Implementation and recorded evidence independently reviewed 2026-11-07 at f0c57ecdcf6cfd617c00cf6785cf75e55b874023; no independent GPU rerun or general accuracy guarantee.
Benchmark status: Provisional comparison; no independently verified inference badge
Checkpoint: fastino/gliner2.5-base-v1
Revision: 54785d51df8d86a0d8f2eb212fec15871aad2c4d
Packages: {"gliner2": "2.1.0", "torch": "2.6.0+cu124", "transformers": "5.8.0", "huggingface-hub": "1.33.0"}
Threshold: 0.5
Device: cuda
Precision: float32
Batch Size: 1
Seed: 0
Schema: original-train-names-including-misc
Decoder: native-flat
Inference Source Head: 1400727ae15ba2967d7db36df31c3da9182f056a
Inference Source Dirty: False
Assets Manifest Sha256: 2ec9e75cfaf247b89b41ced66ae1866025fbfb788d1a00102767d5954c679c5f
Evaluation Date: 2026-11-03
Elapsed Seconds: 80.48500000000058
Peak Gpu Bytes: 812482048
Timing Scope: Shared RTX3060Ti; loading/preflight included, not isolated throughput.
Training Overlap: Not excluded; no uncontaminated zero-shot claim.
Review Summary: Implementation and recorded evidence independently reviewed 2026-11-07 at f0c57ecdcf6cfd617c00cf6785cf75e55b874023; no independent GPU rerun or general accuracy guarantee.
Source Evidence: {"repository": "amadou-6e/agent-anvil", "revision": "d4a44504e709f3167358b9bf3a463ef456b80534", "path": "tools/llms/entity-extraction/docs/benchmark/evidence/local-candidates.json", "sha256": "748158f2191f380039a27fd0102feff31013bee781004d5ed3d7eda0bcc0a3a1"}
Compatibility Note: AutoExtractor boundary loader; direct checkpoint tokenizer.
Overlap Policy: flat
Max Len: 4096
Include Spans: True

Back to results

Reproduce & download

Read the reproduction guide. JPT author artifacts can be rescored locally; our saved extractor predictions remain private. Aggregate downloads alone cannot reproduce private extraction runs.

Versioned aggregate downloads for this provisional comparison. No source text, gold annotations, raw predictions, author cache or model weights are bundled.

Authors & rights

Built on CrossNER, JPT, Graphiti and GLiNER. Additional checkpoints: Knowledgator, Fastino and NuMind. These authors do not endorse this comparison.

Original app code: MIT. Original results and documentation: CC BY 4.0. License scope. Upstream materials retain their own terms.

Updates & contributions

Maintainer: Amadou98. Submissions are not enabled for this provisional preview. Public eligibility, correction and dispute policies await the owner's decision; no automatic verification is implied.

Snapshot selected on 2026-11-03 from workspace revision b4584c6166af2c24d2e1a2791580d3e3a6c97876. Scorer, normalization or dataset changes require a distinct version or incompatible track, rather than silently replacing this protocol.