ABHIJITH SIVAPRASADAN / CONTROL VOLUME
S1 · APPLIED/S3 · CASES/RESEARCH

Literature screening · OpenAlex · human-in-the-loop · evaluation

SynthSift — Evidence synthesis and review workflow

A source-audited research-methods workflow for screening and structured extraction of studies on residential heat-pump adoption, developed to test automation without hiding eligibility errors or human decisions. This project demonstrates how I design auditable data-acquisition and evaluation workflows, distinguish reference labels from model predictions, and make uncertainty in a research pipeline inspectable.

Pythonscikit-learnOpenAlexpytestGitHub Actions
BACK TO CASES ↗
Project illustration; not measured evidencePROJECT REFERENCE / SYNTHSIFT
EVIDENCE / 01

Six frozen OpenAlex query arms yield 538 exact-deduplicated bibliographic candidate records; 330 abstracts and 208 title-only.

EVIDENCE / 02

AI first-pass decisions for all 538 (144 include / 341 exclude / 53 uncertain), 150 unique recheck priorities; original 240 machine decisions preserved.

EVIDENCE / 03

Independent human screening, 96-record blinded second review, adjudication and full-text evidence synthesis remain undone; 16-record pilot is not an external performance estimate.

Source, documentation & reproducibilityRead limitations
Concept illustration for synthsift; not a measured result or map

Work and evidence

  • Corpus: six frozen OpenAlex query arms (600 raw retrievals) produced 538 exact-deduplicated candidates; 330 have abstracts and 208 are title-only records.
  • Reference workflow: a prespecified 240-record screening sample and 96-record blinded second-review subset are frozen for independent human decisions. Their reference screening and adjudication are not yet complete.
  • Full-corpus AI-assisted first pass: all 538 frozen source records now have an explicitly AI-attributed title/available-abstract recommendation (144 include, 341 exclude, 53 uncertain); 150 unique records are flagged for uncertainty, potential version duplication or provisional title-only inclusion. These are not human reference labels.
  • Pilot: a separate provenance-clean 16-record benchmark tests TF-IDF/logistic-regression screening and structured field extraction for nine eligible pilot studies.
  • Reproduction: immutable source snapshot, sampling protocol, resumable screening queue, field-level evaluators, tests and offline CI; manual labels are kept separate from automated predictions.

Method and judgment

What is measured — and what is not

  1. Freeze the acquisition query arms and ranked candidate set rather than silently allowing changing API relevance ranking to alter the experiment.
  2. Preserve DOI/OpenAlex identifier provenance and flag possible study-version duplicates for human review instead of collapsing them automatically.
  3. Evaluate workload reduction and recall only against a labelled reference set, separating unfinished human decisions from frozen labels.
  4. Allow provider-agnostic model prediction files but do not claim an LLM experiment unless a specific executed run and its configuration are committed.

Scope and evidence boundary

Limits that matter.

This is not a completed systematic review or a validated screening model for real reviews. OpenAlex relevance-ranked retrieval is not an exhaustive multi-database search. The 16-record pilot is a pipeline check, not an estimate of external screening performance; large-sample human reference screening, independent adjudication and real LLM evaluation remain unfinished.

Demonstrated judgment: This project demonstrates how I design auditable data-acquisition and evaluation workflows, distinguish reference labels from model predictions, and make uncertainty in a research pipeline inspectable.

Primary evidence

Read the original work.