Cyber-JEV

Research question

When a model decides that code deserves review, is it responding to a defect-related change or to the source and style of the code? If a function has been repaired, can its score reflect that change?

Cyber-JEV studies matched contrastive fine-tuning of language models for source-code vulnerability triage. Its intended role is to prioritize functions for deeper investigation with static analysis, rules, or human review.

Approach

1. Reconstruct the context of a fix. Build matched groups containing a vulnerable function, its patched version, and unchanged functions from the same file.
2. Combine classification and within-group ranking. Reference data support classification; matched groups teach the model to rank the vulnerable member above the others. Both tasks update one shared backbone.
3. Score individual functions. Inference takes the function under review. Scores and a review budget determine which functions receive further attention.

The public materials report 2,829 matched groups and a Qwen3.5-4B backbone. Patched and unchanged functions provide weak supervision rather than independently verified benign labels. External matched groups participate in ranking only.

My involvement

I contribute to this research and use this space to explain its questions, methods, and evaluation. The research manuscript provides the full author list and attribution.

Evaluation and limits

The main experiments use historical C/C++ functions in an offline evaluation. The main test set contains 7,270 functions from 57 projects, including 322 labeled vulnerable functions. The reported Cyber-JEV results use a five-model ensemble.

• Overall ranking: Cyber-JEV reaches 0.814 AUROC and Jev 1.13 reaches 0.743. Reference-only fine-tuning reaches 0.804; the aggregate gap does not establish a conclusive AUROC advantage over that baseline.
• Review budget: With 4,031 flagged functions for each system, Cyber-JEV identifies 306 vulnerable functions and Jev 1.13 identifies 283. This observation concerns one offline budget and should be read alongside recall, average precision, and other budgets.
• Patch pairs: Ranking improves over reference-only training on the three publicly reported paired sets. Other training variants still reveal trade-offs between overall ranking and patch sensitivity.

Training and protected evaluation splits are separated by project, CVE, fix commit, and code similarity. Original labels and commit-derived supervision retain their limitations. Live pull-request deployment, new vulnerability discovery, and comparative throughput have not been demonstrated.

Paper and resources

Public materials include the research overview, reported results, and the October 2026 research manuscript. “Presentation source” links to the website code only. Research implementation, model weights, and datasets are not distributed with that page.

Start with the reading guide, or visit the project website and full manuscript.