Case study III — evaluating a protein design (does it get better?)

Case study III — evaluating a protein design (does it get better?)#

Designing a variant is easy; knowing whether it is actually better is the hard part. This case study takes E. coli dihydrofolate reductase (DHFR, the folA gene product), proposes a variant with an ESM-guided design, and then scores it with an in-silico metric panel — comparing structure, stability, function and activity against the wild type.

Honesty up front. These metrics are proxies, not wet-lab numbers. Some are well-calibrated (stability ΔΔG, foldability pLDDT, self-consistency RMSD, EC-retention); catalytic kcat prediction is noisy and should be read as weak signal only. Real proof of improved activity needs experiments — these metrics are for prioritising designs, and for catching bad ones early.

pip install 'omicverse[synbio]'.

import omicverse as ov

ov.plot_set()
print('omicverse', ov.__version__)

🔬 Starting plot initialization...
🧬 Detecting GPU devices…
✅ NVIDIA CUDA GPUs detected: 1
    • [CUDA 0] NVIDIA H100 80GB HBM3
      Memory: 79.1 GB | Compute: 9.0

   ____            _     _    __                  
  / __ \____ ___  (_)___| |  / /__  _____________ 
 / / / / __ `__ \/ / ___/ | / / _ \/ ___/ ___/ _ \ 
/ /_/ / / / / / / / /__ | |/ /  __/ /  (__  )  __/ 
\____/_/ /_/ /_/_/\___/ |___/\___/_/  /____/\___/                                              

🔖 Version: 2.2.1rc1   📚 Tutorials: https://omicverse.readthedocs.io/
✅ plot_set complete.

omicverse 2.2.1rc1

1 — Wild-type DHFR#

The real 159-residue E. coli DHFR sequence. predict_structure folds it with ESMFold; DHFR is a well-behaved fold and scores high pLDDT.

WT = ('MISLIAALAVDRVIGMENAMPWNLPADLAWFKRNTLNKPVIMGRHTWESIGRPLPGRKNIILSSQPGTDDRV'
      'TWVKSVDEAIAACGDVPEIMVIGGGRVYEQFLPKAQKLYLTHIDAEVEGDTHFPDYEPDDWESVFSEFHDADAQNSHSYCFEILERR')
wt = ov.synbio.predict_structure(WT, out_path='dhfr_wt.pdb')
print('WT length', len(WT), '| pLDDT', round(wt.mean_plddt, 1))

[ov.synbio.predict_structure] device=cuda (NVIDIA H100 80GB HBM3, 79 GB) len=159
[ov.synbio.predict_structure] done: mean pLDDT=94.3
WT length 159 | pLDDT 94.3

2 — Design a variant (ESM-guided)#

ml_guided_design scores every single mutation with an ESM saturation scan and greedily combines the most-favoured, non-conflicting ones — a first-round design that avoids obviously deleterious changes (e.g. the catalytic Asp).

designs = ov.synbio.ml_guided_design(WT, n_mutations=3, n_designs=6)
best = designs[0]
VAR = best.sequence
pos = [int(''.join(ch for ch in m if ch.isdigit())) for m in best.mutations]
print('designed mutations:', '+'.join(best.mutations), '| predicted score', round(best.predicted_score, 2))

designed mutations: N34L+F137S+D11N | predicted score 13.17

var = ov.synbio.predict_structure(VAR, out_path='dhfr_var.pdb')
print('variant pLDDT', round(var.mean_plddt, 1))

[ov.synbio.predict_structure] device=cuda (NVIDIA H100 80GB HBM3, 79 GB) len=159
[ov.synbio.predict_structure] done: mean pLDDT=94.3
variant pLDDT 94.3

3 — Structural comparison (WT vs variant)#

view_superposition aligns the two folds and shows them together — wild type in blue, the aligned variant in orange, the mutated residues as sticks. structure_rmsd gives the number behind the picture (the self-consistency RMSD): a small value means the design keeps the fold.

rmsd = ov.synbio.structure_rmsd('dhfr_wt.pdb', 'dhfr_var.pdb')
print(f'Cα RMSD (WT vs variant) = {rmsd:.2f} Å')
ov.synbio.view_superposition('dhfr_wt.pdb', 'dhfr_var.pdb', highlight=pos)

Cα RMSD (WT vs variant) = 0.31 Å

3Dmol.js failed to load for some reason. Please check your browser console for error messages.

<py3Dmol.view at 0x7f92883d2290>

4 — The design scorecard#

evaluate_design runs the whole panel at once and reports each metric with its reliability. It re-uses the folded structure for ΔΔG (ThermoMPNN) and self-consistency, CLEAN for EC-retention, DLKcat for kcat, and an ESM fitness delta over the mutated positions.

dhf = 'Nc1nc2NCC(CNc3ccc(cc3)C(=O)O)Nc2c(=O)[nH]1'   # a dihydrofolate-like substrate
sc = ov.synbio.evaluate_design(VAR, reference=WT, substrate=dhf,
                               pdb='dhfr_wt.pdb', backbone_pdb='dhfr_wt.pdb')
sc.to_frame()

[ov.synbio.predict_structure] device=cuda (NVIDIA H100 80GB HBM3, 79 GB) len=159
[ov.synbio.predict_structure] done: mean pLDDT=94.3
[ov.synbio.variant_effect] model=esm1v scoring=masked-marginals device=cuda (NVIDIA H100 80GB HBM3, 79 GB)
[ov.synbio.stability_ddg] pdb=dhfr_wt.pdb method=thermompnn device=cuda (NVIDIA H100 80GB HBM3, 79 GB)
[evaluate_design] DesignScorecard(pLDDT=94.3, EC_confidence=0.994, kcat=5.71(Δ+3.04), ESM_fitness_delta=13.2, ddG=1.86, scRMSD=0.313)
metric value delta_vs_ref reliability
0 pLDDT 94.2600 NaN high
1 EC_confidence 0.9940 NaN high
2 kcat 5.7054 3.0361 low
3 ESM_fitness_delta 13.1700 NaN medium
4 ddG 1.8650 NaN high
5 scRMSD 0.3130 NaN high

How to read it (honestly):

  • scRMSD (Å) and pLDDT — high reliability. Small RMSD + high pLDDT ⇒ the variant still folds like DHFR.

  • EC_confidence (CLEAN) — high. Stays near 1.0 ⇒ the fold is still recognised as DHFR (EC 1.5.1.3); the design didn’t destroy the enzyme’s identity.

  • ddG (ThermoMPNN, kcal/mol) — high. Negative = stabilising, positive = destabilising. This is the metric to trust for a stability claim.

  • ESM_fitness_delta — medium. Positive = the mutations are evolutionarily plausible; a strongly negative value is a red flag (e.g. hitting a conserved catalytic residue).

  • kcat / Δkcat (DLKcat) — low. Read as weak signal only: DLKcat barely moves for close variants, so a near-zero Δkcat is not evidence of unchanged activity. For a real activity claim you need an assay.

So the scorecard tells you this variant still folds and is still a DHFR, with a modest stability change — and it will honestly refuse to promise higher catalytic activity, which no sequence-only predictor can currently guarantee.

Summary#

metric

what it answers

trust

scRMSD / pLDDT

does it still fold?

high

EC-retention (CLEAN)

is it still this enzyme?

high

ΔΔG (ThermoMPNN)

more/less stable?

high

ESM fitness Δ

evolutionarily plausible?

medium

kcat (DLKcat)

more active?

low — screen experimentally

evaluate_design packages these into one call so every design carries an honest, reproducible scorecard. Combine with the advanced tutorial (which generates the designs) to close the design→evaluate loop.

Contents