What I Learned Benchmarking Single-Cell Foundation Models in Liatir
I compared scGPT and Geneformer with PCA, Harmony, and scVI on two single-cell RNA-seq datasets. The results were useful, but not the sweeping foundation-model win one might expect.
A Nature study from Google DeepMind shows that detectable watermarks can be embedded in designed protein sequences and predicted structures while preserving tested function, but robustness and deployment remain open problems.
As generative models become useful for designing proteins rather than only analyzing them, provenance becomes a biological problem. A generated sequence can be synthesized, copied, edited, deposited in a database, or separated from the software that produced it. Metadata can disappear along the way.
A new peer-reviewed Nature paper from Google DeepMind tests a different approach: put the provenance signal into the designed biological object itself. The researchers call the family of methods SynthID Bio. They watermark both protein sequences generated during design and three-dimensional structures predicted by AlphaFold 3, then test whether the signal remains detectable without materially degrading the properties the models are meant to produce.
The result is a useful proof of concept, not a general solution to biological provenance. The sequence experiments include wet-lab validation of designed protein binders, which makes the evidence stronger than a purely computational watermarking study. At the same time, both sequence and structure watermarks have attacks that can remove or weaken them, and the current schemes encode only the presence of a watermark rather than richer provenance information.
Watermarking generated text or images is already a trade-off between detectability, output quality, and robustness to editing. Biology adds a stricter constraint: a sequence watermark is not useful if the altered amino-acid choices destroy the protein's intended function.
For a predicted structure, the analogous constraint is structural accuracy. A watermark that visibly perturbs coordinates could change a researcher's interpretation or contaminate downstream analysis. The authors therefore frame the central question as whether a detectable signal can be introduced while preserving the biological or structural property that made the generated object useful in the first place.
This matters because provenance attached externally to a biological design can be stripped away. The paper discusses possible uses in DNA-synthesis screening and scientific databases, but these are proposed applications rather than demonstrated deployments. SynthID Bio does not establish that a sequence is safe, experimentally valid, or benign. It only attempts to provide evidence that an object was generated through a particular watermarked process.
SynthIDBio-sequence modifies the sampling stage of ProteinMPNN, a model commonly used to choose amino-acid sequences for a specified protein backbone. Instead of adding a marker after generation, the method biases amino-acid selection using a secret key and the preceding sequence context.
The technique adapts tournament sampling from SynthID's text watermarking work. Candidate amino acids receive key-dependent scores, and sampling favors choices that collectively produce a detectable statistical signal. Detection later recomputes those scores from the completed sequence using the secret key.
Protein design makes this harder because ProteinMPNN is normally sampled at low temperature, leaving relatively little uncertainty in which amino acids are plausible. The study therefore evaluates both a non-distortionary method and a stronger distortionary variant. It also adds a watermark-detectability threshold to the normal protein-design filtering pipeline. Designs that do not carry a sufficiently strong signal are discarded, effectively exchanging additional compute and lower pass rates for detectability.
The critical experiment is whether that intervention changes actual protein behavior.
The researchers started from binder backbones previously generated with AlphaProteo and resequenced them with watermarked or standard ProteinMPNN. They tested designs against three targets: VEGF-A, the SARS-CoV-2 receptor-binding domain, and PD-L1. According to the Nature study, the watermarked designs retained binding hit rates and affinity distributions comparable with the non-watermarked controls. The experiments produced low-nanomolar binders to the SARS-CoV-2 target and subnanomolar binders to VEGF-A and PD-L1.
That is stronger evidence than showing only that predicted structures look similar. The watermark survived the transition from digital sequence to synthesized protein while the tested binding function remained measurable in vitro.
It is still a narrow validation. These are designed binders across three targets, not arbitrary enzymes, receptors, complexes, or whole biological systems. Preserving binding in this setting does not establish that the same watermarking procedure will preserve every kind of protein function.
SynthIDBio-structure takes a different route. The researchers fine-tuned the diffusion module of AlphaFold 3 jointly with a detector, adding an objective that encourages predicted coordinates to contain a detectable geometric signature.
The detector uses intra-residue geometric features such as atom-to-atom distances and torsion angles. Because those features do not depend on the absolute orientation of a structure, the signal is inherently insensitive to rigid rotations and translations.
For the authors' recommended watermark strength, the paper reports that standard structural-accuracy metrics, including LDDT and template modelling score, were not lower than the AlphaFold 3 baseline. At a detector false-positive rate of 0.1%, true-positive rates exceeded 99.8% for all three watermark strengths evaluated. For the recommended setting, the reported true-positive rate was 98.99% even when the false-positive rate was reduced to 0.01%.
Those numbers describe performance on the study's evaluation distribution, not a guarantee that an arbitrary structure encountered in the wild can be classified correctly. Detection also becomes harder for very short structures.
The structural watermark tolerates some simple transformations. Small coordinate noise, cropping, and rigid transformations preserve much of the signal. But the paper identifies a more consequential weakness: constrained molecular relaxation with OpenMM and the Amber99sb force field can destroy the watermark. That is important because relaxation is a scientifically ordinary operation rather than an exotic adversarial attack.
Both methods are currently zero-bit watermarks. They answer a narrow question: is the expected watermark present? They do not encode an author, user, generation time, model version, or experiment identifier.
The sequence watermark has another fundamental limitation. Because it is implemented in ProteinMPNN's sampling procedure, a user who controls generation can simply turn it off. An already watermarked sequence can also be resequenced to weaken or remove the signal. The authors report that such removal can reduce estimated binder hit rates, but they explicitly call for further in-vitro work to quantify that trade-off and to test attacks such as partial resequencing.
The structure method embeds watermarking behavior in fine-tuned model weights, making it less trivial to disable during ordinary inference. That does not make it indestructible: molecular relaxation is already a demonstrated removal route in the paper.
There is also an operational problem around keys. Detection depends on secret information, so real deployment requires protocols for sharing detectors or keys among trusted parties. The paper notes that public-key approaches and methods for distinguishing watermarks from multiple providers would be needed for broader ecosystems.
The strongest conclusion is narrower than the idea of a universal provenance layer for synthetic biology. The study demonstrates that, in specific protein-design and structure-prediction workflows, a machine-detectable signature can coexist with the tested useful properties of the output.
For sequences, that conclusion has experimental support because the researchers synthesized and measured binders. For structures, the evidence is computational and evaluates prediction accuracy, detectability, and several transformations. The two methods therefore answer related but different scientific questions.
The work also exposes why provenance cannot substitute for existing biological screening. A watermark says something about origin, not risk. The paper itself argues for layered use alongside conventional sequence screening rather than treating detection as evidence that a synthesis order should automatically be trusted.
The next scientific tests are clear. Sequence watermarking needs broader wet-lab validation across protein functions and more realistic editing or resequencing attacks. Structure watermarking needs robustness to standard relaxation workflows. Both need better attribution than a zero-bit signal, interoperable key management, and evidence from deployment conditions where false positives and false negatives have operational costs.
For now, SynthID Bio is most interesting because it moves biological watermarking beyond an entirely digital exercise. It shows that a provenance signal can survive synthesis in at least one experimentally tested protein-design setting. Whether that can become durable infrastructure for AI-assisted biology remains unresolved.