--- license: cc-by-4.0 language: - en pretty_name: CrossNER Extractor Comparison (provisional aggregate results) tags: - named-entity-recognition - evaluation-results - crossner - provisional configs: - config_name: jpt-pairs-v1 data_files: - split: results path: results.csv --- # CrossNER extractor comparison **Published provisional aggregate comparison.** This is a comparison of retained predictions, not the original CrossNER corpus, a training dataset, an official author leaderboard or a newly annotated benchmark. Published as aggregate downloads in the existing Hugging Face Space; this refresh retains provisional scientific status. ## Results and intended use Thirty-five records cover seven systems and five CrossNER domains using the released JPT normalized-pair protocol. Mean domain F1: Graphiti 0.31.2 + Codex gpt-6.1-sol 75.66; JPT-4B author-released predictions 75.26; local GLiNER-large v2.1 66.70; GLiNER Bi-large v2.0 66.16; NuNER Zero 65.74; ModernGLiNER Bi-base v1.0 64.08; GLiNER2.5 Base v1 60.55. Use these for descriptive comparison and aggregate auditing, not a general SOTA claim. Means average unrounded domain scores equally, on a 0 to 100 scale. `results.csv` fields: system, domain, protocol, documents, true_positive, false_positive, false_negative, f1, review_status. `manifest.json` records exact source/prediction pins and full-precision means. No source text, gold or raw predictions are included. The results split is aggregate evidence, not evaluation inputs for running a new extractor. ## Protocol and limits All systems are scored against the released JPT labels, gold and scorer on 2,505 instances: AI 431, literature 416, music 465, politics 650, science 543. Graphiti and local GLiNER predictions came from the original 2,506 CrossNER sentences with name-only schemas, then were aligned using only source inputs to the 2,505 release instances, excluding politics/test/650. JPT predictions came from the author release with richer type definitions. Match normalized (name,type) sets per instance, ignoring offsets. Preserve unsupported Entity false positives; omit only the known extra misc label. Exact normalization and mapping must be reproduced before comparing another row. JPT's row comes from actual author-released inference predictions rescored locally; it is not established as the historical paper run. Its paper-derived five-domain mean is 74.40. JPT uses rich type definitions; our systems used name-only schemas. Graphiti is the configured NER extraction path, not contextual entity resolution. Our extraction schemas included misc before its proposals were dropped; the effect of that extra extraction task is unmeasured. This name/type-pair scorer ignores offsets, so the 0.40-point lead is protocol-specific and does not establish more accurate entity boundaries. No confidence/significance, training contamination or repeated-run stability assessment has been performed. No new model calls produced this release. The published standalone scorer v0.1.0 can rescore separately obtained pinned JPT author artifacts. JPT inference is not rerun. Our extractor predictions remain private, so aggregate downloads alone cannot reproduce those extraction runs. Integrity and arithmetic validation do not establish scientific acceptance. ## Credits CrossNER: Zihan Liu, Yan Xu, Tiezheng Yu, Wenliang Dai, Ziwei Ji, Samuel Cahyawijaya, Andrea Madotto and Pascale Fung, [CrossNER](https://arxiv.org/abs/2012.04373), AAAI 2021, [original data repository](https://github.com/zliucr/CrossNER). JPT evaluator/model/released predictions: Ahmed Ewais, Ahmed Hashish and Amr Ali, [Just Pass Twice](https://aclanthology.org/2026.acl-long.526/), ACL 2026, [author project](https://witness-ai-jpt-ner.hf.space/). GLiNER: Urchade Zaratiana, Nadi Tomeh, Pierre Holat and Thierry Charnois, [GLiNER](https://aclanthology.org/2024.naacl-long.300/), NAACL 2024; [local model](https://huggingface.co/urchade/gliner_large-v2.1). Graphiti: [Zep / Graphiti contributors](https://github.com/getzep/graphiti). The configured chat model is OpenAI Codex gpt-6.1-sol. Agent Anvil contributors provide source alignment, adapters, rescoring and aggregate compilation. Original authors retain ownership of their work; this comparison implies no endorsement. ## License and release status Our original aggregate compilation, provenance and card are licensed under [CC BY 4.0](LICENSE), copyright 2026 amadou-6e. Credit amadou-6e and this comparison release, indicate changes, and preserve the original-author credits above. These terms do not relicense upstream datasets, evaluator code, model weights or the authors' prediction cache, none of which are distributed here. Rights for any additional artifacts must be checked separately. Publication destination: [Amadou98/crossner-extractor-comparison](https://huggingface.co/spaces/Amadou98/crossner-extractor-comparison), an existing public static Space. Only aggregate scores and configuration metadata are included in this update. ## Reviewed local additions (7 November 2026) Four retained runs from 3 November add ModernGLiNER Bi-base v1.0, GLiNER2.5 Base v1, GLiNER Bi-large v2.0 and NuNER Zero. Each uses the same five-domain normalized-pair scorer and alignment. Checkpoint revisions, packages, thresholds and individual execution receipts are recorded in manifest.json. The original three results are unchanged. Scoped independent code/evidence review accepted these additions; reviewers did not rerun GPU inference. The CSV legacy review_status flag describes provisional benchmark admission, not the scoped implementation review. Credits: [Knowledgator bi-encoder work](https://arxiv.org/abs/2602.18487), [GLiNER2 / Fastino](https://arxiv.org/abs/2507.18546), and [NuNER / NuMind](https://arxiv.org/abs/2402.15343). Related papers describe architectures and do not identify these retained runs as paper reproductions. Upstream models retain their own licenses; no weights or predictions are redistributed.