FusionSieve evaluation on targeted DNA panel sequencing data

Abstract

Summary of the audit

Background

FusionSieve is a research-stage gene-fusion evidence caller for targeted paired-end DNA sequencing. Review of the baseline code found defects in read triage, trimming, duplicate handling, depth units, mate geometry, breakpoint clustering, background statistics, calibration, input validation, and result provenance.

Methods

The baseline and corrected callers were compared retrospectively on the same 32 post-duplicate-removal BAMs. A separate six-sample paired analysis compared processing states before and after Picard duplicate removal. Candidate matching required the same ordered genes, chromosomes, and strands, with both breakpoints within 10 base pairs. No compatible panel of normals, validated model, or truth labels were available.

Results

The baseline produced 2,358 candidate hypotheses and the corrected caller produced 5,994; 2,259 matched, 99 were baseline-only, and 3,735 were corrected-only. The corrected caller assigned 5,952 hypotheses to REJECT and 42 to REVIEW, with no HIGH or PASS calls. Four recurrent split-only clusters accounted for 31 of the 42 reviews. In the paired duplicate-removal analysis, panel-signature support retained 63.6% after removal, compared with 82.8% for the coordinate-family proxy.

Conclusions

The corrections materially changed software behavior and made the workflow safer and more auditable. The duplicate-removal pattern supports concern that panel signatures can split PCR-related families in unremoved BAMs, but neither panel signatures nor coordinate families are a validated molecule count. This study did not measure clinical sensitivity, specificity, predictive value, or limit of detection.

1. Introduction

Why this audit was performed

A fusion caller must distinguish true rearrangement evidence from PCR duplicates, homologous mapping, capture artifacts, read-through, contamination, and low-depth chance events. Small counting or geometry errors can therefore change whether a low-support hypothesis is rejected, reviewed, or promoted.

FusionSieve accepts paired FASTQ or BAM data, aligns reads to a targeted panel reference, extracts split-read and discordant-pair evidence, clusters nearby junctions, calculates artifact and background features, and writes structured and human-readable outputs. The present work was a code and method audit of that workflow. It was not a clinical validation study and did not adjudicate patient diagnoses.

2. Retrospective audit questions

Questions evaluated by the analysis

1. Software behavior: Do the corrections materially change candidate discovery and evidence classification when the input BAMs are held fixed?
2. PCR-family stability: Does panel-signature support fall more after duplicate removal than an exact BAM-coordinate-family proxy, as expected if alignment jitter splits some PCR-related families?
3. Fail-closed reporting: With no validated panel of normals or compatible assay-validated model, does the corrected caller avoid automatic HIGH or PASS output?
4. Technical recurrence: Do near-identical REVIEW hypotheses recur across sample IDs with evidence features that warrant investigation for systematic artifacts, carryover, or contamination?
5. Reproducibility: Are all included inputs, analytical resources, result sets, and software identities recorded and checked?

3. Experimental design and methods

How the comparison was conducted

3.1 Study material and analysis sets

The Dropbox-derived inventory contained 39 BAM rows. Thirty-eight were included: 12 historical post-rmdup libraries, 20 additional post-rmdup libraries used as a stress cohort, and six pre-rmdup processing states paired with six of the historical sample IDs. One undetermined demultiplexing bin was excluded because it was not a biological specimen.

The primary baseline-versus-corrected comparison used 32 distinct post-rmdup sample IDs. The six pre/post pairs were alternate processing states of the same samples; they were not independent replicate libraries. Available metadata did not establish patient or specimen independence. Sample identifiers are intentionally omitted from this public report.

3.2 Analytical conditions

• Assay mode: targeted DNA; platform configuration: Illumina.
• Reference assembly: GRCh37/hg19 targeted panel.
• No read limit; the comparison configuration set minimum mean quality to 0.
• No compatible panel of normals or validated model was used.
• No LLM adjudication was used.
• Automatic HIGH/PASS reporting was disabled.

3.3 Principal corrections evaluated

3.4 Read triage and “adequate query coverage”

Fast colinear triage required at least four uniquely placed sampled seeds, agreement of every unique seed on one alignment diagonal, a first-to-last seed span covering at least 80% of the read, and both ends of that span within 16 bases of the read ends. For a 150-base read, the minimum seed span was therefore 120 bases. The complete proposed same-tile placement also had to meet at least 90% full-read identity. Failure of any check—including zero unique seeds—sent the read to the full chimeric aligner rather than discarding it.

3.5 Candidate matching and endpoints

Old and corrected candidates were eligible to match only when ordered genes, chromosomes, and strands were identical and each breakpoint was within ±10 base pairs. Matching maximized the number of pairs first and then minimized total two-arm breakpoint distance and support difference.

Primary descriptive endpoints were candidate counts, matched and version-specific candidates, and tier distribution. Secondary endpoints included read-processing counters, recurrent REVIEW clusters, and pre/post-rmdup stability of raw, panel-signature, coordinate-family, and conservative support measures. No clinical accuracy statistic was prespecified or calculated because independent truth labels were absent.

4. Results

Observed changes in caller behavior

5. Conclusions

Scientific conclusions supported by the evidence

1. The baseline contained defects capable of changing low-support fusion evidence in either direction. The corrected caller is safer and more auditable, but it is not clinically validated.
2. The corrections materially changed read classification and candidate generation: corrected candidate hypotheses increased from 2,358 to 5,994 on the same 32 post-rmdup IDs.
3. The paired pre/post-rmdup results qualitatively support the concern that panel signatures can overestimate independent support in unremoved BAMs. They do not establish a true molecule count or the clinical magnitude of that bias.
4. Four repeated split-only, low-depth, no-mate-support clusters are a strong technical warning and should be investigated before any biological interpretation.
5. The proper disposition is controlled research and analytical development. No candidate in this exercise should be used for patient reporting without read-level review and orthogonal laboratory confirmation.

6. Clinical implications

How remaining defects could affect a patient-facing result

7. Limitations

What prevents a clinical performance claim

1. No truth-labelled validation cohort or orthogonal confirmation.
2. Patient and specimen independence were not established from available metadata.
3. No compatible, independently constructed panel of normals or assay-validated model was used.
4. PCR-family ambiguity remains because quality-aware UMI correction and family consensus are absent.
5. The support numerator and wild-type denominator can have different effective duplication units.
6. Candidate initiation remains split-led, and the targeted-reference BAM prefilter can miss off-panel evidence.
7. Equal or near-equal chimeric placements are not fully represented or assigned a calibrated placement confidence.
8. BAM mate reconstruction relies too heavily on query names, and incomplete candidate-level coordinate audits can fail open.
9. Some DNA filters are biologically overbroad; coding-frame and transcript consequence remain incomplete.
10. PoN non-calls are left-censored rather than direct locus recounts, and absent-locus depth is approximated.
11. Resolved configuration and all auxiliary analytical resources are not yet bound into one complete compatibility identity.
12. There is no assay-specific LoB, LoD95, precision, robustness, interference, or clinical-validity study.

8. Future directions

Recommended scientific work, in priority order

1. Review the evidence. Inspect all 42 REVIEW hypotheses at read level, prioritizing the four recurrent clusters and the BCR–ABL1 and EML4–ALK hypotheses.
2. Perform orthogonal confirmation. Test priority hypotheses using an independent method appropriate to the specimen, such as targeted PCR plus sequencing, ddPCR, FISH, or an RNA assay.
3. Investigate systematic recurrence. Examine batch, plate, index, run, capture, and sample-processing metadata for the four repeated clusters; assess carryover and contamination controls.
4. Establish molecular counting. Add quality-aware UMI parsing, UMI error correction, family-size QC, and consensus reads, and use the same validated family definition for fusion support and wild-type depth.
5. Build an empirical background. Use independent donors processed with the same assay, recount every candidate locus directly, record arm-specific depth, and hold out normals used for evaluation.
6. Close discovery blind spots. Add discordant and one-end-anchored clustering, local assembly, unmapped/SA evidence, and explicit alternative-placement counts and MAPQ-like confidence.
7. Validate the assay end to end. Use blinded, truth-labelled positives and negatives, dilution series, replicate libraries, multiple batches, operators and sites, prespecified thresholds, and confidence intervals for LoB, LoD95, precision, and specificity.
8. Lock reproducibility. Bind exact BAM/BAI content, resolved configuration, source, panel, PoN, model, knowledge resources, and thresholds to every result and aggregation.

9. Software download and independent validation

Download the research validation kit

How to validate the download

1. Download both files above into the same folder.
2. In Windows PowerShell, calculate the archive checksum:

   Get-FileHash .\fusionsieve-validation-kit.zip -Algorithm SHA256

   Compare the displayed hash with the entry in SHA256SUMS.txt. The values must be identical.
3. Extract the ZIP, open its human-readable validation instructions, and create an isolated Python environment as described there.
4. Run the bundled test suite from the extracted project directory:

   python -m pytest -q -p no:cacheprovider

5. Compare the software version, source hash, test result, and small-example output with the validation manifest included in the kit. The audit reference run produced 143 passed tests, 1 expected skip, 8 parameterized subtests, and 0 failures.

10. Selected scientific references

Methods and validation context

1. Broad Institute. Picard MarkDuplicates documentation.
2. Smith T, Heger A, Sudbery I. UMI-tools: modeling sequencing errors in Unique Molecular Identifiers to improve quantification accuracy. Genome Research. 2017.
3. Bewicke-Copley F, et al. Applications and analysis of targeted genomic sequencing in cancer studies. Computational and Structural Biotechnology Journal. 2019.
4. Uhrig S, et al. Accurate and efficient detection of gene fusions from RNA sequencing data. Genome Research. 2021.
5. Association for Molecular Pathology and College of American Pathologists. Guidelines for validation of next-generation sequencing–based oncology panels.